Summary

Add a mixin for Pipeline compilers supporting sourcemaps.

Review Request #12760 — Created Jan. 2, 2023 and submitted Jan. 5, 2023, 4:34 p.m.

Information

Owner

chipx86

Repository

Djblets

Branch

release-4.x

Bugs

Depends On

Reviewers

Groups

djblets

People

Description

This introduces a new djblets.pipeline.compilers.mixins module, which
currently contains SourceMapStaleCheckMixin. This new mixin allows
Pipeline compilers to efficiently check whether a file in a bundle needs
to be rebuilt, based on any files that contributed to its sourcemap.

This works by looking up all the source files in the sourcemap and
comparing timestamps. If any timestamps are newer than that of the built
file or sourcemap, or if any files are missing, then the built file is
considered stale and needs to be rebuilt.

The list of files are extracted from the sourcemap and cached for a
period of 1 hour, to avoid having to re-parse the sourcemap every time.

This will be used for the upcoming rollup.js support for bundling
module-based JavaScript files. Unfortunately, it can't currently be used
for LessCSS due to a broken sourcemap implementation that prevents some
files from being included, but that may be tackled separately via a new
plugin or build step.

Testing Done

Tested this thoroughly with the upcoming rollup.js support.

Commits

Summary	ID
Add a mixin for Pipeline compilers supporting sourcemaps. This introduces a new `djblets.pipeline.compilers.mixins` module, which currently contains `SourceMapStaleCheckMixin`. This new mixin allows Pipeline compilers to efficiently check whether a file in a bundle needs to be rebuilt, based on any files that contributed to its sourcemap. This works by looking up all the source files in the sourcemap and comparing timestamps. If any timestamps are newer than that of the built file or sourcemap, or if any files are missing, then the built file is considered stale and needs to be rebuilt. The list of files are extracted from the sourcemap and cached for a period of 1 hour, to avoid having to re-parse the sourcemap every time. This will be used for the upcoming rollup.js support for bundling module-based JavaScript files. Unfortunately, it can't currently be used for LessCSS due to a broken sourcemap implementation that prevents some files from being included, but that may be tackled separately via a new plugin or build step.	954e0e51a9457f9a36a853ae71dbd34e41d2b551

Summary

Add a mixin for Pipeline compilers supporting sourcemaps.

This introduces a new `djblets.pipeline.compilers.mixins` module, which currently contains `SourceMapStaleCheckMixin`. This new mixin allows Pipeline compilers to efficiently check whether a file in a bundle needs to be rebuilt, based on any files that contributed to its sourcemap. This works by looking up all the source files in the sourcemap and comparing timestamps. If any timestamps are newer than that of the built file or sourcemap, or if any files are missing, then the built file is considered stale and needs to be rebuilt. The list of files are extracted from the sourcemap and cached for a period of 1 hour, to avoid having to re-parse the sourcemap every time. This will be used for the upcoming rollup.js support for bundling module-based JavaScript files. Unfortunately, it can't currently be used for LessCSS due to a broken sourcemap implementation that prevents some files from being included, but that may be tackled separately via a new plugin or build step.

954e0e51a9457f9a36a853ae71dbd34e41d2b551

Issues

Description	From	Last Updated
It looks like you updated this with the wrong commit (replaced the sourcemap change with rollup)	david	Jan. 5, 2023, 3:54 p.m.

flake8 passed.

JSHint passed.

Ship it!

```
Ship It!
```

Change Summary:

Updated to intelligently locate the right working directory to run in, to ensure the right rollup.config.js, .babelrc, etc. is used when building another project's files.

Commits:

	Summary	ID
	Add a mixin for Pipeline compilers supporting sourcemaps. This introduces a new `djblets.pipeline.compilers.mixins` module, which currently contains `SourceMapStaleCheckMixin`. This new mixin allows Pipeline compilers to efficiently check whether a file in a bundle needs to be rebuilt, based on any files that contributed to its sourcemap. This works by looking up all the source files in the sourcemap and comparing timestamps. If any timestamps are newer than that of the built file or sourcemap, or if any files are missing, then the built file is considered stale and needs to be rebuilt. The list of files are extracted from the sourcemap and cached for a period of 1 hour, to avoid having to re-parse the sourcemap every time. This will be used for the upcoming rollup.js support for bundling module-based JavaScript files. Unfortunately, it can't currently be used for LessCSS due to a broken sourcemap implementation that prevents some files from being included, but that may be tackled separately via a new plugin or build step.	954e0e51a9457f9a36a853ae71dbd34e41d2b551
	Add support for using rollup.js to package up JavaScript. This introduces a new compiler for Pipeline that interfaces with rollup.js to build JavaScript. Rollup can itself interface with Babel, TypeScript, etc., and can handle resolving ES6 modules. This allows consumers (including us) to begin modernizing JavaScript codebases. The Rollup Pipeline compiler defaults to handling any `index.js`, `index.es6.js`, or `index.ts` files included in the Pipeline bundle. It will then compile the file and include its contents in the resulting Pipeline bundle. This allows projects to iteratively move to Rollup-bundled JavaScript with module support, while still keeping legacy JavaScript in the same bundle. Over time, projects can phase out all but the `index` file, allowing Rollup to handle all the actual JavaScript compilation and allowing Pipeline to handle versioning, resolving, and packaging the resulting file. Upcoming changes will begin to introduce usage of Rollup within the Djblets codebase.	9197641cac8d86314d6f331480c7c913c435d760

Diff:

Revision 2 (+618 -82)

Show changes

	djblets/pipeline/settings.py
	djblets/pipeline/compilers/rollup.py
	docs/djblets/coderef/index.rst

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

The issue has been resolved. Show all issues

It looks like you updated this with the wrong commit (replaced the sourcemap change with rollup)

Change Summary:

Restored the right commit for this change.

Commits:

	Summary	ID
	Add support for using rollup.js to package up JavaScript. This introduces a new compiler for Pipeline that interfaces with rollup.js to build JavaScript. Rollup can itself interface with Babel, TypeScript, etc., and can handle resolving ES6 modules. This allows consumers (including us) to begin modernizing JavaScript codebases. The Rollup Pipeline compiler defaults to handling any `index.js`, `index.es6.js`, or `index.ts` files included in the Pipeline bundle. It will then compile the file and include its contents in the resulting Pipeline bundle. This allows projects to iteratively move to Rollup-bundled JavaScript with module support, while still keeping legacy JavaScript in the same bundle. Over time, projects can phase out all but the `index` file, allowing Rollup to handle all the actual JavaScript compilation and allowing Pipeline to handle versioning, resolving, and packaging the resulting file. Upcoming changes will begin to introduce usage of Rollup within the Djblets codebase.	9197641cac8d86314d6f331480c7c913c435d760
	Add a mixin for Pipeline compilers supporting sourcemaps. This introduces a new `djblets.pipeline.compilers.mixins` module, which currently contains `SourceMapStaleCheckMixin`. This new mixin allows Pipeline compilers to efficiently check whether a file in a bundle needs to be rebuilt, based on any files that contributed to its sourcemap. This works by looking up all the source files in the sourcemap and comparing timestamps. If any timestamps are newer than that of the built file or sourcemap, or if any files are missing, then the built file is considered stale and needs to be rebuilt. The list of files are extracted from the sourcemap and cached for a period of 1 hour, to avoid having to re-parse the sourcemap every time. This will be used for the upcoming rollup.js support for bundling module-based JavaScript files. Unfortunately, it can't currently be used for LessCSS due to a broken sourcemap implementation that prevents some files from being included, but that may be tackled separately via a new plugin or build step.	954e0e51a9457f9a36a853ae71dbd34e41d2b551

Diff:

Revision 3 (+364)

Show changes

djblets/pipeline/compilers/mixins.py

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Ship it!

```
Ship It!
```

Status:: Completed
Change Summary:: Pushed to release-4.x (ad2e2dd)