Merge remote-tracking branch 'upstream/main'

This commit is contained in:
2026-08-26 09:31:34 +08:00
parent 9b8c4bfab6
commit 394eb0285d
34137 changed files with 3589424 additions and 0 deletions
+297
View File
@@ -0,0 +1,297 @@
# Changelog
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
### [2.3.8](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.3.7...v2.3.8) (2020-08-12)
### Chore
* update `serialize-javascript`
### [2.3.7](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.3.6...v2.3.7) (2020-06-03)
### Bug Fixes
* security problem ([#258](https://github.com/webpack-contrib/terser-webpack-plugin/issues/258)) ([9954ad5](https://github.com/webpack-contrib/terser-webpack-plugin/commit/9954ad59895dd18b524253784cd13f1b173f99ed))
### [2.3.6](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.3.5...v2.3.6) (2020-04-25)
### Bug Fixes
* preserve `@license` comments starting with `//` ([d3f0c81](https://github.com/webpack-contrib/terser-webpack-plugin/commit/d3f0c811672920b658edaa99815de9204716fa21))
### [2.3.5](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.3.4...v2.3.5) (2020-02-14)
### Bug Fixes
* do not break code with shebang ([fac58cb](https://github.com/webpack-contrib/terser-webpack-plugin/commit/fac58cb1ee7935710b3b38662c3abbf8dc12a862))
### [2.3.4](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.3.3...v2.3.4) (2020-01-30)
### Bug Fixes
* respect stdout and stderr of workers and do not create warnings ([#215](https://github.com/webpack-contrib/terser-webpack-plugin/issues/215)) ([5708574](https://github.com/webpack-contrib/terser-webpack-plugin/commit/5708574d3337158a02d60a81275467900da5f42d))
* use webpack hash options rather than hard-code MD4 ([#213](https://github.com/webpack-contrib/terser-webpack-plugin/issues/213)) ([330c1f6](https://github.com/webpack-contrib/terser-webpack-plugin/commit/330c1f6cf3468fd6312e86960b272df1591f1a64))
### [2.3.3](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.3.2...v2.3.3) (2020-01-28)
### Bug Fixes
* license files now have .txt suffix by default ([#210](https://github.com/webpack-contrib/terser-webpack-plugin/issues/210)) ([de02f7b](https://github.com/webpack-contrib/terser-webpack-plugin/commit/de02f7b229a6ef91baa353681b1c546784672ab6))
* reduce memory usage ([abfd950](https://github.com/webpack-contrib/terser-webpack-plugin/commit/abfd9506207cf392de63a0629de82145efff2361))
### [2.3.2](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.3.1...v2.3.2) (2020-01-09)
### Bug Fixes
* show error message from jest-worker ([#203](https://github.com/webpack-contrib/terser-webpack-plugin/issues/203)) ([3b28007](https://github.com/webpack-contrib/terser-webpack-plugin/commit/3b280070cde3b233ae703fe2e0ac75b350cb2da7))
### [2.3.1](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.3.0...v2.3.1) (2019-12-17)
### Bug Fixes
* performance ([#200](https://github.com/webpack-contrib/terser-webpack-plugin/issues/200)) ([d2acd75](https://github.com/webpack-contrib/terser-webpack-plugin/commit/d2acd75f4b630af38d2c272f800e755fe395b2dd))
## [2.3.0](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.2.3...v2.3.0) (2019-12-12)
### Features
* support webpack@5 cache ([3649b3d](https://github.com/webpack-contrib/terser-webpack-plugin/commit/3649b3d0bf697288661676b47b33ae88226eb6f5))
### [2.2.3](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.2.2...v2.2.3) (2019-12-11)
### SECURITY
* update `serialize-javascript` to `2.1.2` version.
### [2.2.2](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.2.1...v2.2.2) (2019-12-06)
### SECURITY
* update `serialize-javascript` to `2.1.1` version.
### [2.2.1](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.2.0...v2.2.1) (2019-10-22)
### Bug Fixes
* get rid deprecation warnings for webpack@5 ([#181](https://github.com/webpack-contrib/terser-webpack-plugin/issues/181)) ([0e9b780](https://github.com/webpack-contrib/terser-webpack-plugin/commit/0e9b780390826a9b9f58368f24b39452ca34f00e))
## [2.2.0](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.1.3...v2.2.0) (2019-10-22)
### Features
* map `webpack@5` options to `terser` options ([#177](https://github.com/webpack-contrib/terser-webpack-plugin/issues/177)) ([f4c47aa](https://github.com/webpack-contrib/terser-webpack-plugin/commit/f4c47aa0e3de27bbab3ae0a90a461841bbce5988))
* pass an asset path for the `warningsFilter` option ([#176](https://github.com/webpack-contrib/terser-webpack-plugin/issues/176)) ([9a0a575](https://github.com/webpack-contrib/terser-webpack-plugin/commit/9a0a575585be21118cb4e3611d30e665a6337c3d))
* propagate an error stacktrace from `terser` ([#179](https://github.com/webpack-contrib/terser-webpack-plugin/issues/179)) ([a11e66b](https://github.com/webpack-contrib/terser-webpack-plugin/commit/a11e66b17ae2bb146ded4e64c948d5eece834378))
* enable the `sourceMap` option when the `SourceMapDevToolPlugin` plugin used (non cheap) ([#178](https://github.com/webpack-contrib/terser-webpack-plugin/issues/178)) ([d01c1b5](https://github.com/webpack-contrib/terser-webpack-plugin/commit/d01c1b5fef7f0593c91c1be3d4eb36220c3f465d))
### Bug Fixes
* get rid deprecation warnings for `webpack@5` ([#180](https://github.com/webpack-contrib/terser-webpack-plugin/issues/180)) ([504ea8b](https://github.com/webpack-contrib/terser-webpack-plugin/commit/504ea8b2017e6fa9b8a5f2123025ee5b7b4d80b1))
### [2.1.3](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.1.2...v2.1.3) (2019-10-10)
### Bug Fixes
* invalidate cache when a file name was changed ([#171](https://github.com/webpack-contrib/terser-webpack-plugin/issues/171)) ([7e1d370](https://github.com/webpack-contrib/terser-webpack-plugin/commit/7e1d370ce2520c7c23689c19b22cfbea0265957e))
### [2.1.2](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.1.1...v2.1.2) (2019-09-28)
### Bug Fixes
* regexp for `some` comments ([#168](https://github.com/webpack-contrib/terser-webpack-plugin/issues/168)) ([4c4b1f1](https://github.com/webpack-contrib/terser-webpack-plugin/commit/4c4b1f1))
### [2.1.1](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.1.0...v2.1.1) (2019-09-27)
### Bug Fixes
* logic for extracting and preserving comments ([#166](https://github.com/webpack-contrib/terser-webpack-plugin/issues/166)) ([6bdee64](https://github.com/webpack-contrib/terser-webpack-plugin/commit/6bdee64))
## [2.1.0](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.0.1...v2.1.0) (2019-09-16)
### Bug Fixes
* correct interpretation url for extracted comment file ([#157](https://github.com/webpack-contrib/terser-webpack-plugin/issues/157)) ([aba8ba7](https://github.com/webpack-contrib/terser-webpack-plugin/commit/aba8ba7))
### Features
* emit warning when comment file conlict with an existing asset ([#156](https://github.com/webpack-contrib/terser-webpack-plugin/issues/156)) ([2b4d2a4](https://github.com/webpack-contrib/terser-webpack-plugin/commit/2b4d2a4))
* improve naming of extracted file with comments ([#154](https://github.com/webpack-contrib/terser-webpack-plugin/issues/154)) ([5fe3337](https://github.com/webpack-contrib/terser-webpack-plugin/commit/5fe3337))
### [2.0.1](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v2.0.0...v2.0.1) (2019-09-06)
### Bug Fixes
* reduce memory usage ([#145](https://github.com/webpack-contrib/terser-webpack-plugin/issues/145)) ([815e533](https://github.com/webpack-contrib/terser-webpack-plugin/commit/815e533))
* revert do not run parallel mode when you have only one file ([#146](https://github.com/webpack-contrib/terser-webpack-plugin/issues/146)) ([6613a97](https://github.com/webpack-contrib/terser-webpack-plugin/commit/6613a97))
## [2.0.0](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.4.1...v2.0.0) (2019-09-05)
### ⚠ BREAKING CHANGES
* minimum require Node.js version is `8.9.0`
* the `extractComments` option is `true` by default
* the `cache` option is `true` by default
* the `parallel` option is `true` by default
* using the `extractComments.condition` option with `true` value extract only `some` comments
* the `sourceMap` option now defaults to depending on the `devtool` Webpack option
### Bug Fixes
* do not run parallel mode when you have only one file ([#134](https://github.com/webpack-contrib/terser-webpack-plugin/issues/134)) ([8b88b39](https://github.com/webpack-contrib/terser-webpack-plugin/commit/8b88b39))
* make `extractComments` API more consistent ([#129](https://github.com/webpack-contrib/terser-webpack-plugin/issues/129)) ([37d2df0](https://github.com/webpack-contrib/terser-webpack-plugin/commit/37d2df0))
* parallel on wsl ([#138](https://github.com/webpack-contrib/terser-webpack-plugin/issues/138)) ([0537591](https://github.com/webpack-contrib/terser-webpack-plugin/commit/0537591))
### Features
* enable the cache option by default ([#132](https://github.com/webpack-contrib/terser-webpack-plugin/issues/132)) ([960d249](https://github.com/webpack-contrib/terser-webpack-plugin/commit/960d249))
* enable the extractComments option by default ([ad2471c](https://github.com/webpack-contrib/terser-webpack-plugin/commit/ad2471c))
* enable the parallel option by default ([#131](https://github.com/webpack-contrib/terser-webpack-plugin/issues/131)) ([7b342af](https://github.com/webpack-contrib/terser-webpack-plugin/commit/7b342af))
* respect `devtool` values for source map generation ([#140](https://github.com/webpack-contrib/terser-webpack-plugin/issues/140)) ([dd37ca1](https://github.com/webpack-contrib/terser-webpack-plugin/commit/dd37ca1))
### [1.4.1](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.4.0...v1.4.1) (2019-07-31)
### Bug Fixes
* removed unnecessary dependency ([#111](https://github.com/webpack-contrib/terser-webpack-plugin/issues/111)) ([bc19b72](https://github.com/webpack-contrib/terser-webpack-plugin/commit/bc19b72))
## [1.4.0](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.3.0...v1.4.0) (2019-07-31)
### Features
* generate higher quality SourceMaps ([#109](https://github.com/webpack-contrib/terser-webpack-plugin/issues/109)) ([9d777f0](https://github.com/webpack-contrib/terser-webpack-plugin/commit/9d777f0))
## [1.3.0](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.2.4...v1.3.0) (2019-05-24)
### Features
* update terser to 4 version ([#97](https://github.com/webpack-contrib/terser-webpack-plugin/issues/97)) ([15d1595](https://github.com/webpack-contrib/terser-webpack-plugin/commit/15d1595))
### [1.2.4](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.2.3...v1.2.4) (2019-05-15)
### Bug Fixes
* disable parallel on WSL due bugs ([#90](https://github.com/webpack-contrib/terser-webpack-plugin/issues/90)) ([d9533dd](https://github.com/webpack-contrib/terser-webpack-plugin/commit/d9533dd))
* fallback for cache directory ([#86](https://github.com/webpack-contrib/terser-webpack-plugin/issues/86)) ([3cdd2ed](https://github.com/webpack-contrib/terser-webpack-plugin/commit/3cdd2ed))
<a name="1.2.3"></a>
## [1.2.3](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.2.2...v1.2.3) (2019-02-25)
### Bug Fixes
* invalidate cache after changing node version ([675edfd](https://github.com/webpack-contrib/terser-webpack-plugin/commit/675edfd))
<a name="1.2.2"></a>
## [1.2.2](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.2.1...v1.2.2) (2019-02-04)
### Bug Fixes
* cannot read property 'minify' of undefined ([#69](https://github.com/webpack-contrib/terser-webpack-plugin/issues/69)) ([0593d7c](https://github.com/webpack-contrib/terser-webpack-plugin/commit/0593d7c))
<a name="1.2.1"></a>
## [1.2.1](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.2.0...v1.2.1) (2018-12-27)
### Bug Fixes
* don't crash when no extracted comments ([#49](https://github.com/webpack-contrib/terser-webpack-plugin/issues/49)) ([efad586](https://github.com/webpack-contrib/terser-webpack-plugin/commit/efad586))
<a name="1.2.0"></a>
# [1.2.0](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.1.0...v1.2.0) (2018-12-22)
### Bug Fixes
* `chunks` is a `Set` in webpack@5 ([#19](https://github.com/webpack-contrib/terser-webpack-plugin/issues/19)) ([df8c425](https://github.com/webpack-contrib/terser-webpack-plugin/commit/df8c425))
* catch `work-farm` errors ([#35](https://github.com/webpack-contrib/terser-webpack-plugin/issues/35)) ([2bdcd38](https://github.com/webpack-contrib/terser-webpack-plugin/commit/2bdcd38))
* dedupe extracted comments ([#40](https://github.com/webpack-contrib/terser-webpack-plugin/issues/40)) ([7f4a159](https://github.com/webpack-contrib/terser-webpack-plugin/commit/7f4a159))
* more consistent cache ([#43](https://github.com/webpack-contrib/terser-webpack-plugin/issues/43)) ([36f5f3c](https://github.com/webpack-contrib/terser-webpack-plugin/commit/36f5f3c))
* regenerate `contenthash` when assets was uglified ([#44](https://github.com/webpack-contrib/terser-webpack-plugin/issues/44)) ([7e6f8b1](https://github.com/webpack-contrib/terser-webpack-plugin/commit/7e6f8b1))
### Features
* `chunkFilter` option for filtering chunks ([#38](https://github.com/webpack-contrib/terser-webpack-plugin/issues/38)) ([7ffe57c](https://github.com/webpack-contrib/terser-webpack-plugin/commit/7ffe57c))
* uglify `mjs` by default ([#39](https://github.com/webpack-contrib/terser-webpack-plugin/issues/39)) ([1644620](https://github.com/webpack-contrib/terser-webpack-plugin/commit/1644620))
<a name="1.1.0"></a>
# [1.1.0](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.0.1...v1.1.0) (2018-09-14)
### Bug Fixes
* extract comment conditions is case insensitivity ([19e1e43](https://github.com/webpack-contrib/terser-webpack-plugin/commit/19e1e43))
### Features
* full coverage schema options validation ([#8](https://github.com/webpack-contrib/terser-webpack-plugin/issues/8)) ([68e531e](https://github.com/webpack-contrib/terser-webpack-plugin/commit/68e531e))
<a name="1.0.2"></a>
## [1.0.2](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.0.1...v1.0.2) (2018-08-16)
### Internal
* migrate from `@webpack-contrib/schema-utils` to `schema-utils` ([a5432da](https://github.com/webpack-contrib/terser-webpack-plugin/commit/a5432da))
<a name="1.0.1"></a>
## [1.0.1](https://github.com/webpack-contrib/terser-webpack-plugin/compare/v1.0.0...v1.0.1) (2018-08-15)
### Bug Fixes
* `cpus` length in a chroot environment (`os.cpus()`) ([#4](https://github.com/webpack-contrib/terser-webpack-plugin/issues/4)) ([9375646](https://github.com/webpack-contrib/terser-webpack-plugin/commit/9375646))
<a name="1.0.0"></a>
# 1.0.0 (2018-08-02)
Initial publish release
# Change Log
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
+20
View File
@@ -0,0 +1,20 @@
Copyright JS Foundation and other contributors
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+772
View File
@@ -0,0 +1,772 @@
<div align="center">
<a href="https://github.com/webpack/webpack">
<img width="200" height="200" src="https://webpack.js.org/assets/icon-square-big.svg">
</a>
</div>
[![npm][npm]][npm-url]
[![node][node]][node-url]
[![deps][deps]][deps-url]
[![tests][tests]][tests-url]
[![cover][cover]][cover-url]
[![chat][chat]][chat-url]
[![size][size]][size-url]
# terser-webpack-plugin
This plugin uses [terser](https://github.com/terser-js/terser) to minify your JavaScript.
## Getting Started
To begin, you'll need to install `terser-webpack-plugin`:
```console
$ npm install terser-webpack-plugin --save-dev
```
Then add the plugin to your `webpack` config. For example:
**webpack.config.js**
```js
const TerserPlugin = require('terser-webpack-plugin');
module.exports = {
optimization: {
minimize: true,
minimizer: [new TerserPlugin()],
},
};
```
And run `webpack` via your preferred method.
## Options
### `test`
Type: `String|RegExp|Array<String|RegExp>`
Default: `/\.m?js(\?.*)?$/i`
Test to match files against.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
test: /\.js(\?.*)?$/i,
}),
],
},
};
```
### `include`
Type: `String|RegExp|Array<String|RegExp>`
Default: `undefined`
Files to include.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
include: /\/includes/,
}),
],
},
};
```
### `exclude`
Type: `String|RegExp|Array<String|RegExp>`
Default: `undefined`
Files to exclude.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
exclude: /\/excludes/,
}),
],
},
};
```
### `chunkFilter`
Type: `Function<(chunk) -> boolean>`
Default: `() => true`
Allowing to filter which chunks should be uglified (by default all chunks are uglified).
Return `true` to uglify the chunk, `false` otherwise.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
chunkFilter: (chunk) => {
// Exclude uglification for the `vendor` chunk
if (chunk.name === 'vendor') {
return false;
}
return true;
},
}),
],
},
};
```
### `cache`
> ⚠ Doesn't work with webpack 5!
Type: `Boolean|String`
Default: `true`
Enable file caching.
Default path to cache directory: `node_modules/.cache/terser-webpack-plugin`.
> ℹ️ If you use your own `minify` function please read the `minify` section for cache invalidation correctly.
#### `Boolean`
Enable/disable file caching.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
cache: true,
}),
],
},
};
```
#### `String`
Enable file caching and set path to cache directory.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
cache: 'path/to/cache',
}),
],
},
};
```
### `cacheKeys`
> ⚠ Doesn't work with webpack 5!
Type: `Function<(defaultCacheKeys, file) -> Object>`
Default: `defaultCacheKeys => defaultCacheKeys`
Allows you to override default cache keys.
Default cache keys:
```js
({
terser: require('terser/package.json').version, // terser version
'terser-webpack-plugin': require('../package.json').version, // plugin version
'terser-webpack-plugin-options': this.options, // plugin options
path: compiler.outputPath ? `${compiler.outputPath}/${file}` : file, // asset path
hash: crypto
.createHash('md4')
.update(input)
.digest('hex'), // source file hash
});
```
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
cache: true,
cacheKeys: (defaultCacheKeys, file) => {
defaultCacheKeys.myCacheKey = 'myCacheKeyValue';
return defaultCacheKeys;
},
}),
],
},
};
```
### `parallel`
Type: `Boolean|Number`
Default: `true`
Use multi-process parallel running to improve the build speed.
Default number of concurrent runs: `os.cpus().length - 1`.
> ℹ️ Parallelization can speedup your build significantly and is therefore **highly recommended**.
> ⚠️ If you use **Circle CI** or any other environment that doesn't provide real available count of CPUs then you need to setup explicitly number of CPUs to avoid `Error: Call retries were exceeded` (see [#143](https://github.com/webpack-contrib/terser-webpack-plugin/issues/143), [#202](https://github.com/webpack-contrib/terser-webpack-plugin/issues/202)).
#### `Boolean`
Enable/disable multi-process parallel running.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
parallel: true,
}),
],
},
};
```
#### `Number`
Enable multi-process parallel running and set number of concurrent runs.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
parallel: 4,
}),
],
},
};
```
### `sourceMap`
Type: `Boolean`
Default: `false` (see below for details around `devtool` value and `SourceMapDevToolPlugin` plugin)
**Works only with `source-map`, `inline-source-map`, `hidden-source-map` and `nosources-source-map` values for the [`devtool`](https://webpack.js.org/configuration/devtool/) option.**
Why?
- `eval` wraps modules in `eval("string")` and the minimizer does not handle strings.
- `cheap` has not column information and minimizer generate only a single line, which leave only a single mapping.
The plugin respect the [`devtool`](https://webpack.js.org/configuration/devtool/) and using the `SourceMapDevToolPlugin` plugin.
Using supported `devtool` values enable source map generation.
Using `SourceMapDevToolPlugin` with enabled the `columns` option enables source map generation.
Use source maps to map error message locations to modules (this slows down the compilation).
If you use your own `minify` function please read the `minify` section for handling source maps correctly.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
sourceMap: true,
}),
],
},
};
```
### `minify`
Type: `Function`
Default: `undefined`
Allows you to override default minify function.
By default plugin uses [terser](https://github.com/terser-js/terser) package.
Useful for using and testing unpublished versions or forks.
> ⚠️ **Always use `require` inside `minify` function when `parallel` option enabled**.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
minify: (file, sourceMap) => {
const extractedComments = [];
// Custom logic for extract comments
const { error, map, code, warnings } = require('uglify-module') // Or require('./path/to/uglify-module')
.minify(file, {
/* Your options for minification */
});
return { error, map, code, warnings, extractedComments };
},
}),
],
},
};
```
### `terserOptions`
Type: `Object`
Default: [default](https://github.com/terser-js/terser#minify-options)
Terser minify [options](https://github.com/terser-js/terser#minify-options).
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
terserOptions: {
ecma: undefined,
warnings: false,
parse: {},
compress: {},
mangle: true, // Note `mangle.properties` is `false` by default.
module: false,
output: null,
toplevel: false,
nameCache: null,
ie8: false,
keep_classnames: undefined,
keep_fnames: false,
safari10: false,
},
}),
],
},
};
```
### `extractComments`
Type: `Boolean|String|RegExp|Function<(node, comment) -> Boolean|Object>|Object`
Default: `true`
Whether comments shall be extracted to a separate file, (see [details](https://github.com/webpack/webpack/commit/71933e979e51c533b432658d5e37917f9e71595a)).
By default extract only comments using `/^\**!|@preserve|@license|@cc_on/i` regexp condition and remove remaining comments.
If the original file is named `foo.js`, then the comments will be stored to `foo.js.LICENSE.txt`.
The `terserOptions.output.comments` option specifies whether the comment will be preserved, i.e. it is possible to preserve some comments (e.g. annotations) while extracting others or even preserving comments that have been extracted.
#### `Boolean`
Enable/disable extracting comments.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
extractComments: true,
}),
],
},
};
```
#### `String`
Extract `all` or `some` (use `/^\**!|@preserve|@license|@cc_on/i` RegExp) comments.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
extractComments: 'all',
}),
],
},
};
```
#### `RegExp`
All comments that match the given expression will be extracted to the separate file.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
extractComments: /@extract/i,
}),
],
},
};
```
#### `Function<(node, comment) -> Boolean>`
All comments that match the given expression will be extracted to the separate file.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
extractComments: (astNode, comment) => {
if (/@extract/i.test(comment.value)) {
return true;
}
return false;
},
}),
],
},
};
```
#### `Object`
Allow to customize condition for extract comments, specify extracted file name and banner.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
extractComments: {
condition: /^\**!|@preserve|@license|@cc_on/i,
filename: (file, fileData) => {
// ⚠ webpack 5: there is only fileData parameter
// A file can contain a query string (for example when you have `output.filename: '[name].js?[chunkhash]'`)
// You must consider this
// The "fileData" argument contains object with "filename", "basename", "query"
return file.replace(/\.(\w+)($|\?)/, '.$1.LICENSE.txt$2');
},
banner: (licenseFile) => {
return `License information can be found in ${licenseFile}`;
},
},
}),
],
},
};
```
##### `condition`
Type: `Boolean|String|RegExp|Function<(node, comment) -> Boolean|Object>`
Condition what comments you need extract.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
extractComments: {
condition: 'some',
filename: (file, fileData) => {
// ⚠ webpack 5: there is only fileData parameter
// A file can contain a query string (for example when you have `output.filename: '[name].js?[chunkhash]'`)
// You must consider this
return file.replace(/\.(\w+)($|\?)/, '.$1.LICENSE.txt$2');
},
banner: (licenseFile) => {
return `License information can be found in ${licenseFile}`;
},
},
}),
],
},
};
```
##### `filename`
Type: `String|Function<(string) -> String>`
Default: `[file].LICENSE.txt[query]`
Available placeholders: `[file]`, `[query]` and `[filebase]` (`[base]` for webpack 5).
The file where the extracted comments will be stored.
Default is to append the suffix `.LICENSE.txt` to the original filename.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
extractComments: {
condition: /^\**!|@preserve|@license|@cc_on/i,
filename: 'extracted-comments.js',
banner: (licenseFile) => {
return `License information can be found in ${licenseFile}`;
},
},
}),
],
},
};
```
##### `banner`
Type: `Boolean|String|Function<(string) -> String>`
Default: `/*! For license information please see ${commentsFile} */`
The banner text that points to the extracted file and will be added on top of the original file.
Can be `false` (no banner), a `String`, or a `Function<(string) -> String>` that will be called with the filename where extracted comments have been stored.
Will be wrapped into comment.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
extractComments: {
condition: true,
filename: (file, fileData) => {
// ⚠ webpack 5: there is only fileData parameter
// A file can contain a query string (for example when you have `output.filename: '[name].js?[chunkhash]'`)
// You must consider this
return file.replace(/\.(\w+)($|\?)/, '.$1.LICENSE.txt$2');
},
banner: (commentsFile) => {
return `My custom banner about license information ${commentsFile}`;
},
},
}),
],
},
};
```
### `warningsFilter`
Type: `Function<(warning, source, file) -> Boolean>`
Default: `() => true`
Allow to filter [terser](https://github.com/terser-js/terser) warnings.
Return `true` to keep the warning, a falsy value (`false`/`null`/`undefined`) otherwise.
> ⚠️ The `source` argument will contain `undefined` if you don't use source maps.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
warningsFilter: (warning, source, file) => {
if (/Dropping unreachable code/i.test(warning)) {
return true;
}
if (/source\.js/i.test(source)) {
return true;
}
if (/file\.js/i.test(file)) {
return true;
}
return false;
},
}),
],
},
};
```
## Examples
### Preserve Comments
Extract all legal comments (i.e. `/^\**!|@preserve|@license|@cc_on/i`) and preserve `/@license/i` comments.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
terserOptions: {
output: {
comments: /@license/i,
},
},
extractComments: true,
}),
],
},
};
```
### Remove Comments
If you avoid building with comments, use this config:
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
terserOptions: {
output: {
comments: false,
},
},
extractComments: false,
}),
],
},
};
```
### Custom Minify Function
Override default minify function - use `uglify-js` for minification.
**webpack.config.js**
```js
module.exports = {
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
// Uncomment lines below for cache invalidation correctly
// cache: true,
// cacheKeys: (defaultCacheKeys) => {
// delete defaultCacheKeys.terser;
//
// return Object.assign(
// {},
// defaultCacheKeys,
// { 'uglify-js': require('uglify-js/package.json').version },
// );
// },
minify: (file, sourceMap) => {
// https://github.com/mishoo/UglifyJS2#minify-options
const uglifyJsOptions = {
/* your `uglify-js` package options */
};
if (sourceMap) {
uglifyJsOptions.sourceMap = {
content: sourceMap,
};
}
return require('uglify-js').minify(file, uglifyJsOptions);
},
}),
],
},
};
```
## Contributing
Please take a moment to read our contributing guidelines if you haven't yet done so.
[CONTRIBUTING](./.github/CONTRIBUTING.md)
## License
[MIT](./LICENSE)
[npm]: https://img.shields.io/npm/v/terser-webpack-plugin.svg
[npm-url]: https://npmjs.com/package/terser-webpack-plugin
[node]: https://img.shields.io/node/v/terser-webpack-plugin.svg
[node-url]: https://nodejs.org
[deps]: https://david-dm.org/webpack-contrib/terser-webpack-plugin.svg
[deps-url]: https://david-dm.org/webpack-contrib/terser-webpack-plugin
[tests]: https://github.com/webpack-contrib/terser-webpack-plugin/workflows/terser-webpack-plugin/badge.svg
[tests-url]: https://github.com/webpack-contrib/terser-webpack-plugin/actions
[cover]: https://codecov.io/gh/webpack-contrib/terser-webpack-plugin/branch/master/graph/badge.svg
[cover-url]: https://codecov.io/gh/webpack-contrib/terser-webpack-plugin
[chat]: https://img.shields.io/badge/gitter-webpack%2Fwebpack-brightgreen.svg
[chat-url]: https://gitter.im/webpack/webpack
[size]: https://packagephobia.now.sh/badge?p=terser-webpack-plugin
[size-url]: https://packagephobia.now.sh/result?p=terser-webpack-plugin
+133
View File
@@ -0,0 +1,133 @@
"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
exports.default = void 0;
var _os = _interopRequireDefault(require("os"));
var _pLimit = _interopRequireDefault(require("p-limit"));
var _jestWorker = _interopRequireDefault(require("jest-worker"));
var _serializeJavascript = _interopRequireDefault(require("serialize-javascript"));
var _minify = _interopRequireDefault(require("./minify"));
function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }
const workerPath = require.resolve('./worker');
class TaskRunner {
constructor(options = {}) {
this.taskGenerator = options.taskGenerator;
this.files = options.files;
this.cache = options.cache;
this.availableNumberOfCores = TaskRunner.getAvailableNumberOfCores(options.parallel);
}
static getAvailableNumberOfCores(parallel) {
// In some cases cpus() returns undefined
// https://github.com/nodejs/node/issues/19022
const cpus = _os.default.cpus() || {
length: 1
};
return parallel === true ? cpus.length - 1 : Math.min(Number(parallel) || 0, cpus.length - 1);
}
async runTask(task) {
if (this.worker) {
return this.worker.transform((0, _serializeJavascript.default)(task));
}
return (0, _minify.default)(task);
}
async run() {
const {
availableNumberOfCores,
cache,
files,
taskGenerator
} = this;
let concurrency = Infinity;
if (availableNumberOfCores > 0) {
// Do not create unnecessary workers when the number of files is less than the available cores, it saves memory
const numWorkers = Math.min(files.length, availableNumberOfCores);
concurrency = numWorkers;
this.worker = new _jestWorker.default(workerPath, {
numWorkers
}); // https://github.com/facebook/jest/issues/8872#issuecomment-524822081
const workerStdout = this.worker.getStdout();
if (workerStdout) {
workerStdout.on('data', chunk => {
return process.stdout.write(chunk);
});
}
const workerStderr = this.worker.getStderr();
if (workerStderr) {
workerStderr.on('data', chunk => {
return process.stderr.write(chunk);
});
}
}
const limit = (0, _pLimit.default)(concurrency);
const scheduledTasks = [];
for (const file of files) {
const enqueue = async task => {
let taskResult;
try {
taskResult = await this.runTask(task);
} catch (error) {
taskResult = {
error
};
}
if (cache.isEnabled() && !taskResult.error) {
taskResult = await cache.store(task, taskResult).then(() => taskResult, () => taskResult);
}
task.callback(taskResult);
return taskResult;
};
scheduledTasks.push(limit(() => {
const task = taskGenerator(file).next().value;
if (!task) {
// Something went wrong, for example the `cacheKeys` option throw an error
return Promise.resolve();
}
if (cache.isEnabled()) {
return cache.get(task).then(taskResult => task.callback(taskResult), () => enqueue(task));
}
return enqueue(task);
}));
}
return Promise.all(scheduledTasks);
}
async exit() {
if (!this.worker) {
return Promise.resolve();
}
return this.worker.end();
}
}
exports.default = TaskRunner;
+48
View File
@@ -0,0 +1,48 @@
"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
exports.default = void 0;
var _os = _interopRequireDefault(require("os"));
var _cacache = _interopRequireDefault(require("cacache"));
var _findCacheDir = _interopRequireDefault(require("find-cache-dir"));
var _serializeJavascript = _interopRequireDefault(require("serialize-javascript"));
function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }
class Webpack4Cache {
constructor(compiler, compilation, options) {
this.options = options;
this.cacheDir = options.cache === true ? Webpack4Cache.getCacheDirectory() : options.cache;
}
static getCacheDirectory() {
return (0, _findCacheDir.default)({
name: 'terser-webpack-plugin'
}) || _os.default.tmpdir();
}
isEnabled() {
return !!this.cacheDir;
}
get(task) {
// eslint-disable-next-line no-param-reassign
task.cacheIdent = task.cacheIdent || (0, _serializeJavascript.default)(task.cacheKeys);
return _cacache.default.get(this.cacheDir, task.cacheIdent).then(({
data
}) => JSON.parse(data));
}
store(task, data) {
return _cacache.default.put(this.cacheDir, task.cacheIdent, JSON.stringify(data));
}
}
exports.default = Webpack4Cache;
+82
View File
@@ -0,0 +1,82 @@
"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
exports.default = void 0;
var _getLazyHashedEtag = _interopRequireDefault(require("webpack/lib/cache/getLazyHashedEtag"));
var _serializeJavascript = _interopRequireDefault(require("serialize-javascript"));
var _webpack = require("webpack");
function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }
// eslint-disable-next-line import/extensions,import/no-unresolved
class Cache {
constructor(compiler, compilation, options) {
this.compiler = compiler;
this.compilation = compilation;
this.options = options;
}
isEnabled() {
return !!this.compilation.cache;
}
createCacheIdent(task) {
const {
outputOptions: {
hashSalt,
hashDigest,
hashDigestLength,
hashFunction
}
} = this.compilation;
const hash = _webpack.util.createHash(hashFunction);
if (hashSalt) {
hash.update(hashSalt);
}
hash.update((0, _serializeJavascript.default)(task.cacheKeys));
const digest = hash.digest(hashDigest);
const cacheKeys = digest.substr(0, hashDigestLength);
return `${this.compilation.compilerPath}/TerserWebpackPlugin/${cacheKeys}/${task.file}`;
}
get(task) {
// eslint-disable-next-line no-param-reassign
task.cacheIdent = task.cacheIdent || this.createCacheIdent(task); // eslint-disable-next-line no-param-reassign
task.cacheETag = task.cacheETag || (0, _getLazyHashedEtag.default)(task.asset);
return new Promise((resolve, reject) => {
this.compilation.cache.get(task.cacheIdent, task.cacheETag, (err, result) => {
if (err) {
reject(err);
} else if (result) {
resolve(result);
} else {
reject();
}
});
});
}
store(task, data) {
return new Promise((resolve, reject) => {
this.compilation.cache.store(task.cacheIdent, task.cacheETag, data, err => {
if (err) {
reject(err);
} else {
resolve(data);
}
});
});
}
}
exports.default = Cache;
+5
View File
@@ -0,0 +1,5 @@
"use strict";
const plugin = require('./index');
module.exports = plugin.default;
+455
View File
@@ -0,0 +1,455 @@
"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
exports.default = void 0;
var _path = _interopRequireDefault(require("path"));
var _sourceMap = require("source-map");
var _webpackSources = require("webpack-sources");
var _RequestShortener = _interopRequireDefault(require("webpack/lib/RequestShortener"));
var _webpack = require("webpack");
var _schemaUtils = _interopRequireDefault(require("schema-utils"));
var _serializeJavascript = _interopRequireDefault(require("serialize-javascript"));
var _package = _interopRequireDefault(require("terser/package.json"));
var _options = _interopRequireDefault(require("./options.json"));
var _TaskRunner = _interopRequireDefault(require("./TaskRunner"));
function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }
const warningRegex = /\[.+:([0-9]+),([0-9]+)\]/;
class TerserPlugin {
constructor(options = {}) {
(0, _schemaUtils.default)(_options.default, options, {
name: 'Terser Plugin',
baseDataPath: 'options'
});
const {
minify,
terserOptions = {},
test = /\.m?js(\?.*)?$/i,
chunkFilter = () => true,
warningsFilter = () => true,
extractComments = true,
sourceMap,
cache = true,
cacheKeys = defaultCacheKeys => defaultCacheKeys,
parallel = true,
include,
exclude
} = options;
this.options = {
test,
chunkFilter,
warningsFilter,
extractComments,
sourceMap,
cache,
cacheKeys,
parallel,
include,
exclude,
minify,
terserOptions
};
}
static isSourceMap(input) {
// All required options for `new SourceMapConsumer(...options)`
// https://github.com/mozilla/source-map#new-sourcemapconsumerrawsourcemap
return Boolean(input && input.version && input.sources && Array.isArray(input.sources) && typeof input.mappings === 'string');
}
static buildSourceMap(inputSourceMap) {
if (!inputSourceMap || !TerserPlugin.isSourceMap(inputSourceMap)) {
return null;
}
return new _sourceMap.SourceMapConsumer(inputSourceMap);
}
static buildError(error, file, sourceMap, requestShortener) {
// Handling error which should have line, col, filename and message
if (error.line) {
const original = sourceMap && sourceMap.originalPositionFor({
line: error.line,
column: error.col
});
if (original && original.source && requestShortener) {
return new Error(`${file} from Terser\n${error.message} [${requestShortener.shorten(original.source)}:${original.line},${original.column}][${file}:${error.line},${error.col}]${error.stack ? `\n${error.stack.split('\n').slice(1).join('\n')}` : ''}`);
}
return new Error(`${file} from Terser\n${error.message} [${file}:${error.line},${error.col}]${error.stack ? `\n${error.stack.split('\n').slice(1).join('\n')}` : ''}`);
}
if (error.stack) {
return new Error(`${file} from Terser\n${error.stack}`);
}
return new Error(`${file} from Terser\n${error.message}`);
}
static buildWarning(warning, file, sourceMap, requestShortener, warningsFilter) {
let warningMessage = warning;
let locationMessage = '';
let source = null;
if (sourceMap) {
const match = warningRegex.exec(warning);
if (match) {
const line = +match[1];
const column = +match[2];
const original = sourceMap.originalPositionFor({
line,
column
});
if (original && original.source && original.source !== file && requestShortener) {
({
source
} = original);
warningMessage = `${warningMessage.replace(warningRegex, '')}`;
locationMessage = `[${requestShortener.shorten(original.source)}:${original.line},${original.column}]`;
}
}
} // Todo change order in next major release
if (warningsFilter && !warningsFilter(warning, source, file)) {
return null;
}
return `Terser Plugin: ${warningMessage}${locationMessage}`;
}
static removeQueryString(filename) {
let targetFilename = filename;
const queryStringIdx = targetFilename.indexOf('?');
if (queryStringIdx >= 0) {
targetFilename = targetFilename.substr(0, queryStringIdx);
}
return targetFilename;
}
static hasAsset(commentFilename, assets) {
const assetFilenames = Object.keys(assets).map(assetFilename => TerserPlugin.removeQueryString(assetFilename));
return assetFilenames.includes(TerserPlugin.removeQueryString(commentFilename));
}
static isWebpack4() {
return _webpack.version[0] === '4';
}
*taskGenerator(compiler, compilation, allExtractedComments, file) {
let inputSourceMap;
const asset = compilation.assets[file];
try {
let input;
if (this.options.sourceMap && asset.sourceAndMap) {
const {
source,
map
} = asset.sourceAndMap();
input = source;
if (TerserPlugin.isSourceMap(map)) {
inputSourceMap = map;
} else {
inputSourceMap = map;
compilation.warnings.push(new Error(`${file} contains invalid source map`));
}
} else {
input = asset.source();
inputSourceMap = null;
} // Handling comment extraction
let commentsFilename = false;
if (this.options.extractComments) {
commentsFilename = this.options.extractComments.filename || '[file].LICENSE.txt[query]';
if (TerserPlugin.isWebpack4()) {
// Todo remove this in next major release
if (typeof commentsFilename === 'function') {
commentsFilename = commentsFilename.bind(null, file);
}
}
let query = '';
let filename = file;
const querySplit = filename.indexOf('?');
if (querySplit >= 0) {
query = filename.substr(querySplit);
filename = filename.substr(0, querySplit);
}
const lastSlashIndex = filename.lastIndexOf('/');
const basename = lastSlashIndex === -1 ? filename : filename.substr(lastSlashIndex + 1);
const data = {
filename,
basename,
query
};
commentsFilename = compilation.getPath(commentsFilename, data);
}
if (commentsFilename && TerserPlugin.hasAsset(commentsFilename, compilation.assets)) {
// Todo make error and stop uglifing in next major release
compilation.warnings.push(new Error(`The comment file "${TerserPlugin.removeQueryString(commentsFilename)}" conflicts with an existing asset, this may lead to code corruption, please use a different name`));
}
const callback = taskResult => {
let {
code
} = taskResult;
const {
error,
map,
warnings
} = taskResult;
const {
extractedComments
} = taskResult;
let sourceMap = null;
if (error || warnings && warnings.length > 0) {
sourceMap = TerserPlugin.buildSourceMap(inputSourceMap);
} // Handling results
// Error case: add errors, and go to next file
if (error) {
compilation.errors.push(TerserPlugin.buildError(error, file, sourceMap, new _RequestShortener.default(compiler.context)));
return;
}
const hasExtractedComments = commentsFilename && extractedComments && extractedComments.length > 0;
const hasBannerForExtractedComments = hasExtractedComments && this.options.extractComments.banner !== false;
let outputSource;
let shebang;
if (hasExtractedComments && hasBannerForExtractedComments && code.startsWith('#!')) {
const firstNewlinePosition = code.indexOf('\n');
shebang = code.substring(0, firstNewlinePosition);
code = code.substring(firstNewlinePosition + 1);
}
if (map) {
outputSource = new _webpackSources.SourceMapSource(code, file, map, input, inputSourceMap, true);
} else {
outputSource = new _webpackSources.RawSource(code);
} // Write extracted comments to commentsFilename
if (hasExtractedComments) {
if (!allExtractedComments[commentsFilename]) {
// eslint-disable-next-line no-param-reassign
allExtractedComments[commentsFilename] = [];
} // eslint-disable-next-line no-param-reassign
allExtractedComments[commentsFilename] = allExtractedComments[commentsFilename].concat(extractedComments); // Add a banner to the original file
if (hasBannerForExtractedComments) {
let banner = this.options.extractComments.banner || `For license information please see ${_path.default.relative(_path.default.dirname(file), commentsFilename).replace(/\\/g, '/')}`;
if (typeof banner === 'function') {
banner = banner(commentsFilename);
}
if (banner) {
outputSource = new _webpackSources.ConcatSource(shebang ? `${shebang}\n` : '', `/*! ${banner} */\n`, outputSource);
}
}
} // Updating assets
// eslint-disable-next-line no-param-reassign
compilation.assets[file] = outputSource; // Handling warnings
if (warnings && warnings.length > 0) {
warnings.forEach(warning => {
const builtWarning = TerserPlugin.buildWarning(warning, file, sourceMap, new _RequestShortener.default(compiler.context), this.options.warningsFilter);
if (builtWarning) {
compilation.warnings.push(builtWarning);
}
});
}
};
const task = {
asset,
file,
input,
inputSourceMap,
commentsFilename,
extractComments: this.options.extractComments,
terserOptions: this.options.terserOptions,
minify: this.options.minify,
callback
};
if (TerserPlugin.isWebpack4()) {
const {
outputOptions: {
hashSalt,
hashDigest,
hashDigestLength,
hashFunction
}
} = compilation;
const hash = _webpack.util.createHash(hashFunction);
if (hashSalt) {
hash.update(hashSalt);
}
hash.update(input);
const digest = hash.digest(hashDigest);
if (this.options.cache) {
const defaultCacheKeys = {
terser: _package.default.version,
// eslint-disable-next-line global-require
'terser-webpack-plugin': require('../package.json').version,
'terser-webpack-plugin-options': this.options,
nodeVersion: process.version,
filename: file,
contentHash: digest.substr(0, hashDigestLength)
};
task.cacheKeys = this.options.cacheKeys(defaultCacheKeys, file);
}
} else {
task.cacheKeys = {
terser: _package.default.version,
// eslint-disable-next-line global-require
'terser-webpack-plugin': require('../package.json').version,
'terser-webpack-plugin-options': this.options
};
}
yield task;
} catch (error) {
compilation.errors.push(TerserPlugin.buildError(error, file, TerserPlugin.buildSourceMap(inputSourceMap), new _RequestShortener.default(compiler.context)));
}
}
apply(compiler) {
const {
devtool,
output,
plugins
} = compiler.options;
this.options.sourceMap = typeof this.options.sourceMap === 'undefined' ? devtool && !devtool.includes('eval') && !devtool.includes('cheap') && (devtool.includes('source-map') || // Todo remove when `webpack@5` support will be dropped
devtool.includes('sourcemap')) || plugins && plugins.some(plugin => plugin instanceof _webpack.SourceMapDevToolPlugin && plugin.options && plugin.options.columns) : Boolean(this.options.sourceMap);
if (typeof this.options.terserOptions.module === 'undefined' && typeof output.module !== 'undefined') {
this.options.terserOptions.module = output.module;
}
if (typeof this.options.terserOptions.ecma === 'undefined' && typeof output.ecmaVersion !== 'undefined') {
this.options.terserOptions.ecma = output.ecmaVersion;
}
const optimizeFn = async (compilation, chunks) => {
const matchObject = _webpack.ModuleFilenameHelpers.matchObject.bind( // eslint-disable-next-line no-undefined
undefined, this.options);
const files = [].concat(Array.from(compilation.additionalChunkAssets || [])).concat(Array.from(chunks).filter(chunk => this.options.chunkFilter && this.options.chunkFilter(chunk)).reduce((acc, chunk) => acc.concat(Array.from(chunk.files || [])), [])).filter(file => matchObject(file));
if (files.length === 0) {
return Promise.resolve();
}
const CacheEngine = TerserPlugin.isWebpack4() ? // eslint-disable-next-line global-require
require('./Webpack4Cache').default : // eslint-disable-next-line global-require
require('./Webpack5Cache').default;
const allExtractedComments = {};
const taskGenerator = this.taskGenerator.bind(this, compiler, compilation, allExtractedComments);
const taskRunner = new _TaskRunner.default({
taskGenerator,
files,
cache: new CacheEngine(compiler, compilation, this.options),
parallel: this.options.parallel
});
await taskRunner.run();
await taskRunner.exit();
Object.keys(allExtractedComments).forEach(commentsFilename => {
const extractedComments = new Set([...allExtractedComments[commentsFilename].sort()]); // eslint-disable-next-line no-param-reassign
compilation.assets[commentsFilename] = new _webpackSources.RawSource(`${Array.from(extractedComments).join('\n\n')}\n`);
});
return Promise.resolve();
};
const plugin = {
name: this.constructor.name
};
compiler.hooks.compilation.tap(plugin, compilation => {
if (this.options.sourceMap) {
compilation.hooks.buildModule.tap(plugin, moduleArg => {
// to get detailed location info about errors
// eslint-disable-next-line no-param-reassign
moduleArg.useSourceMap = true;
});
}
if (!TerserPlugin.isWebpack4()) {
const hooks = _webpack.javascript.JavascriptModulesPlugin.getCompilationHooks(compilation);
const data = (0, _serializeJavascript.default)({
terser: _package.default.version,
terserOptions: this.options.terserOptions
});
hooks.chunkHash.tap(plugin, (chunk, hash) => {
hash.update('TerserPlugin');
hash.update(data);
});
} else {
// Todo remove after drop `webpack@4` compatibility
const {
mainTemplate,
chunkTemplate
} = compilation;
const data = (0, _serializeJavascript.default)({
terser: _package.default.version,
terserOptions: this.options.terserOptions
}); // Regenerate `contenthash` for minified assets
for (const template of [mainTemplate, chunkTemplate]) {
template.hooks.hashForChunk.tap(plugin, hash => {
hash.update('TerserPlugin');
hash.update(data);
});
}
}
compilation.hooks.optimizeChunkAssets.tapPromise(plugin, optimizeFn.bind(this, compilation));
});
}
}
var _default = TerserPlugin;
exports.default = _default;
+178
View File
@@ -0,0 +1,178 @@
"use strict";
const {
minify: terserMinify
} = require('terser');
const buildTerserOptions = ({
ecma,
warnings,
parse = {},
compress = {},
mangle,
module,
output,
toplevel,
nameCache,
ie8,
/* eslint-disable camelcase */
keep_classnames,
keep_fnames,
/* eslint-enable camelcase */
safari10
} = {}) => ({
ecma,
warnings,
parse: { ...parse
},
compress: typeof compress === 'boolean' ? compress : { ...compress
},
// eslint-disable-next-line no-nested-ternary
mangle: mangle == null ? true : typeof mangle === 'boolean' ? mangle : { ...mangle
},
output: {
beautify: false,
...output
},
module,
// Ignoring sourceMap from options
sourceMap: null,
toplevel,
nameCache,
ie8,
keep_classnames,
keep_fnames,
safari10
});
function isObject(value) {
const type = typeof value;
return value != null && (type === 'object' || type === 'function');
}
const buildComments = (options, terserOptions, extractedComments) => {
const condition = {};
const commentsOpts = terserOptions.output.comments;
const {
extractComments
} = options;
condition.preserve = typeof commentsOpts !== 'undefined' ? commentsOpts : false;
if (typeof extractComments === 'boolean' && extractComments) {
condition.extract = 'some';
} else if (typeof extractComments === 'string' || extractComments instanceof RegExp) {
condition.extract = extractComments;
} else if (typeof extractComments === 'function') {
condition.extract = extractComments;
} else if (isObject(extractComments)) {
condition.extract = typeof extractComments.condition === 'boolean' && extractComments.condition ? 'some' : typeof extractComments.condition !== 'undefined' ? extractComments.condition : 'some';
} else {
// No extract
// Preserve using "commentsOpts" or "some"
condition.preserve = typeof commentsOpts !== 'undefined' ? commentsOpts : 'some';
condition.extract = false;
} // Ensure that both conditions are functions
['preserve', 'extract'].forEach(key => {
let regexStr;
let regex;
switch (typeof condition[key]) {
case 'boolean':
condition[key] = condition[key] ? () => true : () => false;
break;
case 'function':
break;
case 'string':
if (condition[key] === 'all') {
condition[key] = () => true;
break;
}
if (condition[key] === 'some') {
condition[key] = (astNode, comment) => {
return (comment.type === 'comment2' || comment.type === 'comment1') && /@preserve|@lic|@cc_on|^\**!/i.test(comment.value);
};
break;
}
regexStr = condition[key];
condition[key] = (astNode, comment) => {
return new RegExp(regexStr).test(comment.value);
};
break;
default:
regex = condition[key];
condition[key] = (astNode, comment) => regex.test(comment.value);
}
}); // Redefine the comments function to extract and preserve
// comments according to the two conditions
return (astNode, comment) => {
if (condition.extract(astNode, comment)) {
const commentText = comment.type === 'comment2' ? `/*${comment.value}*/` : `//${comment.value}`; // Don't include duplicate comments
if (!extractedComments.includes(commentText)) {
extractedComments.push(commentText);
}
}
return condition.preserve(astNode, comment);
};
};
const minify = options => {
const {
file,
input,
inputSourceMap,
minify: minifyFn
} = options;
if (minifyFn) {
return minifyFn({
[file]: input
}, inputSourceMap);
} // Copy terser options
const terserOptions = buildTerserOptions(options.terserOptions); // Let terser generate a SourceMap
if (inputSourceMap) {
terserOptions.sourceMap = {
asObject: true
};
}
const extractedComments = [];
terserOptions.output.comments = buildComments(options, terserOptions, extractedComments);
const {
error,
map,
code,
warnings
} = terserMinify({
[file]: input
}, terserOptions);
return {
error,
map,
code,
warnings,
extractedComments
};
};
module.exports = minify;
+169
View File
@@ -0,0 +1,169 @@
{
"additionalProperties": false,
"definitions": {
"file-conditions": {
"anyOf": [
{
"instanceof": "RegExp"
},
{
"type": "string"
}
]
}
},
"properties": {
"test": {
"anyOf": [
{
"$ref": "#/definitions/file-conditions"
},
{
"items": {
"anyOf": [
{
"$ref": "#/definitions/file-conditions"
}
]
},
"type": "array"
}
]
},
"include": {
"anyOf": [
{
"$ref": "#/definitions/file-conditions"
},
{
"items": {
"anyOf": [
{
"$ref": "#/definitions/file-conditions"
}
]
},
"type": "array"
}
]
},
"exclude": {
"anyOf": [
{
"$ref": "#/definitions/file-conditions"
},
{
"items": {
"anyOf": [
{
"$ref": "#/definitions/file-conditions"
}
]
},
"type": "array"
}
]
},
"chunkFilter": {
"instanceof": "Function"
},
"cache": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string"
}
]
},
"cacheKeys": {
"instanceof": "Function"
},
"parallel": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "integer"
}
]
},
"sourceMap": {
"type": "boolean"
},
"minify": {
"instanceof": "Function"
},
"terserOptions": {
"additionalProperties": true,
"type": "object"
},
"extractComments": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string"
},
{
"instanceof": "RegExp"
},
{
"instanceof": "Function"
},
{
"additionalProperties": false,
"properties": {
"condition": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string"
},
{
"instanceof": "RegExp"
},
{
"instanceof": "Function"
}
]
},
"filename": {
"anyOf": [
{
"type": "string"
},
{
"instanceof": "Function"
}
]
},
"banner": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string"
},
{
"instanceof": "Function"
}
]
}
},
"type": "object"
}
]
},
"warningsFilter": {
"instanceof": "Function"
}
},
"type": "object"
}
+19
View File
@@ -0,0 +1,19 @@
"use strict";
const minify = require('./minify');
function transform(options) {
// 'use strict' => this === undefined (Clean Scope)
// Safer for possible security issues, albeit not critical at all here
// eslint-disable-next-line no-new-func, no-param-reassign
options = new Function('exports', 'require', 'module', '__filename', '__dirname', `'use strict'\nreturn ${options}`)(exports, require, module, __filename, __dirname);
const result = minify(options);
if (result.error) {
throw result.error;
} else {
return result;
}
}
module.exports.transform = transform;
+16
View File
@@ -0,0 +1,16 @@
#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
case `uname` in
*CYGWIN*|*MINGW*|*MSYS*)
if command -v cygpath > /dev/null 2>&1; then
basedir=`cygpath -w "$basedir"`
fi
;;
esac
if [ -x "$basedir/node" ]; then
exec "$basedir/node" "$basedir/../rimraf/bin.js" "$@"
else
exec node "$basedir/../rimraf/bin.js" "$@"
fi
@@ -0,0 +1,17 @@
@ECHO off
GOTO start
:find_dp0
SET dp0=%~dp0
EXIT /b
:start
SETLOCAL
CALL :find_dp0
IF EXIST "%dp0%\node.exe" (
SET "_prog=%dp0%\node.exe"
) ELSE (
SET "_prog=node"
SET PATHEXT=%PATHEXT:;.JS;=;%
)
endLocal & goto #_undefined_# 2>NUL || title %COMSPEC% & "%_prog%" "%dp0%\..\rimraf\bin.js" %*
@@ -0,0 +1,28 @@
#!/usr/bin/env pwsh
$basedir=Split-Path $MyInvocation.MyCommand.Definition -Parent
$exe=""
if ($PSVersionTable.PSVersion -lt "6.0" -or $IsWindows) {
# Fix case when both the Windows and Linux builds of Node
# are installed in the same directory
$exe=".exe"
}
$ret=0
if (Test-Path "$basedir/node$exe") {
# Support pipeline input
if ($MyInvocation.ExpectingInput) {
$input | & "$basedir/node$exe" "$basedir/../rimraf/bin.js" $args
} else {
& "$basedir/node$exe" "$basedir/../rimraf/bin.js" $args
}
$ret=$LASTEXITCODE
} else {
# Support pipeline input
if ($MyInvocation.ExpectingInput) {
$input | & "node$exe" "$basedir/../rimraf/bin.js" $args
} else {
& "node$exe" "$basedir/../rimraf/bin.js" $args
}
$ret=$LASTEXITCODE
}
exit $ret
@@ -0,0 +1,747 @@
# Changelog
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
### [13.0.1](https://github.com/npm/cacache/compare/v13.0.0...v13.0.1) (2019-09-30)
### Bug Fixes
* **fix-owner:** chownr.sync quits on non-root uid ([08801be](https://github.com/npm/cacache/commit/08801be))
## [13.0.0](https://github.com/npm/cacache/compare/v12.0.3...v13.0.0) (2019-09-25)
### ⚠ BREAKING CHANGES
* This subtly changes the streaming interface of
everything in cacache that streams, which is, well, everything in
cacache. Most users will probably not notice, but any code that
depended on stream behavior always being deferred until next tick will
need to adjust.
The mississippi methods 'to', 'from', 'through', and so on, have been
replaced with their Minipass counterparts, and streaming interaction
with the file system is done via fs-minipass.
The following modules are of interest here:
- [minipass](http://npm.im/minipass) The core stream library.
- [fs-minipass](http://npm.im/fs-minipass) Note that the 'WriteStream'
class from fs-minipass is _not_ a Minipass stream, but rather a plain
old EventEmitter that duck types as a Writable.
- [minipass-collect](http://npm.im/minipass-collect) Gather up all the
data from a stream. Cacache only uses Collect.PassThrough, which is a
basic Minipass passthrough stream which emits a 'collect' event with
the completed data just before the 'end' event.
- [minipass-pipeline](http://npm.im/minipass-pipeline) Connect one or
more streams into a pipe chain. Errors anywhere in the pipeline are
proxied down the chain and then up to the Pipeline object itself.
Writes go into the head, reads go to the tail. Used in place of
pump() and pumpify().
- [minipass-flush](http://npm.im/minipass-flush) A Minipass passthrough
stream that defers its 'end' event until after a flush() method has
completed (either calling the supplied callback, or returning a
promise.) Use in place of flush-write-stream (aka mississippi.to).
Streams from through2, concat-stream, and the behavior provided by
end-of-stream are all implemented in Minipass itself.
Features of interest to cacache, which make Minipass a particularly good
fit:
- All of the 'endish' events are normalized, so we can just listen on
'end' and know that finish, prefinish, and close will be handled as
well.
- Minipass doesn't waste time [containing
zalgo](https://blog.izs.me/2013/08/designing-apis-for-asynchrony).
- Minipass has built-in support for promises that indicate the end or
error: stream.promise(), stream.collect(), and stream.concat().
- With reliable and consistent timing guarantees, much less
error-checking logic is required. We can be more confident that an
error is being thrown or emitted in the correct place, rather than in
a callback which is deferred, resulting in a hung promise or
uncaughtException.
The biggest downside of Minipass is that it lacks some of the internal
characteristics of node-core streams, which many community modules use
to identify streams. They have no _writableState or _readableState
objects, or _read or _write methods. As a result, the is-stream module
(at least, at the time of this commit) doesn't recognize Minipass
streams as readable or writable streams.
All in all, the changes required of downstream users should be minimal,
but are unlikely to be zero. Hence the semver major change.
### Features
* replace all streams with Minipass streams ([f4c0962](https://github.com/npm/cacache/commit/f4c0962))
* **deps:** Add minipass and minipass-pipeline ([a6545a9](https://github.com/npm/cacache/commit/a6545a9))
* **promise:** converted .resolve to native promise, converted .map and .reduce to native ([220c56d](https://github.com/npm/cacache/commit/220c56d))
* **promise:** individually promisifing functions as needed ([74b939e](https://github.com/npm/cacache/commit/74b939e))
* **promise:** moved .reject from bluebird to native promise ([1d56da1](https://github.com/npm/cacache/commit/1d56da1))
* **promise:** removed .fromNode, removed .join ([9c457a0](https://github.com/npm/cacache/commit/9c457a0))
* **promise:** removed .map, replaced with p-map. removed .try ([cc3ee05](https://github.com/npm/cacache/commit/cc3ee05))
* **promise:** removed .tap ([0260f12](https://github.com/npm/cacache/commit/0260f12))
* **promise:** removed .using/.disposer ([5d832f3](https://github.com/npm/cacache/commit/5d832f3))
* **promise:** removed bluebird ([c21298c](https://github.com/npm/cacache/commit/c21298c))
* **promise:** removed bluebird specific .catch calls ([28aeeac](https://github.com/npm/cacache/commit/28aeeac))
* **promise:** replaced .reduce and .mapSeries ([478f5cb](https://github.com/npm/cacache/commit/478f5cb))
### [12.0.3](https://github.com/npm/cacache/compare/v12.0.2...v12.0.3) (2019-08-19)
### Bug Fixes
* do not chown if not running as root ([2d80af9](https://github.com/npm/cacache/commit/2d80af9))
### [12.0.2](https://github.com/npm/cacache/compare/v12.0.1...v12.0.2) (2019-07-19)
### [12.0.1](https://github.com/npm/cacache/compare/v12.0.0...v12.0.1) (2019-07-19)
* **deps** Abstracted out `lib/util/infer-owner.js` to
[@npmcli/infer-owner](https://www.npmjs.com/package/@npmcli/infer-owner)
so that it could be more easily used in other parts of the npm CLI.
## [12.0.0](https://github.com/npm/cacache/compare/v11.3.3...v12.0.0) (2019-07-15)
### Features
* infer uid/gid instead of accepting as options ([ac84d14](https://github.com/npm/cacache/commit/ac84d14))
* **i18n:** add another error message ([676cb32](https://github.com/npm/cacache/commit/676cb32))
### BREAKING CHANGES
* the uid gid options are no longer respected or
necessary. As of this change, cacache will always match the cache
contents to the ownership of the cache directory (or its parent
directory), regardless of what the caller passes in.
Reasoning:
The number one reason to use a uid or gid option was to keep root-owned
files from causing problems in the cache. In npm's case, this meant
that CLI's ./lib/command.js had to work out the appropriate uid and gid,
then pass it to the libnpmcommand module, which had to in turn pass the
uid and gid to npm-registry-fetch, which then passed it to
make-fetch-happen, which passed it to cacache. (For package fetching,
pacote would be in that mix as well.)
Added to that, `cacache.rm()` will actually _write_ a file into the
cache index, but has no way to accept an option so that its call to
entry-index.js will write the index with the appropriate uid/gid.
Little ownership bugs were all over the place, and tricky to trace
through. (Why should make-fetch-happen even care about accepting or
passing uids and gids? It's an http library.)
This change allows us to keep the cache from having mixed ownership in
any situation.
Of course, this _does_ mean that if you have a root-owned but
user-writable folder (for example, `/tmp`), then the cache will try to
chown everything to root.
The solution is for the user to create a folder, make it user-owned, and
use that, rather than relying on cacache to create the root cache folder.
If we decide to restore the uid/gid opts, and use ownership inference
only when uid/gid are unset, then take care to also make rm take an
option object, and pass it through to entry-index.js.
### [11.3.3](https://github.com/npm/cacache/compare/v11.3.2...v11.3.3) (2019-06-17)
### Bug Fixes
* **audit:** npm audit fix ([200a6d5](https://github.com/npm/cacache/commit/200a6d5))
* **config:** Add ssri config 'error' option ([#146](https://github.com/npm/cacache/issues/146)) ([47de8f5](https://github.com/npm/cacache/commit/47de8f5))
* **deps:** npm audit fix ([481a7dc](https://github.com/npm/cacache/commit/481a7dc))
* **standard:** standard --fix ([7799149](https://github.com/npm/cacache/commit/7799149))
* **write:** avoid another cb never called situation ([5156561](https://github.com/npm/cacache/commit/5156561))
<a name="11.3.2"></a>
## [11.3.2](https://github.com/npm/cacache/compare/v11.3.1...v11.3.2) (2018-12-21)
### Bug Fixes
* **get:** make sure to handle errors in the .then ([b10bcd0](https://github.com/npm/cacache/commit/b10bcd0))
<a name="11.3.1"></a>
## [11.3.1](https://github.com/npm/cacache/compare/v11.3.0...v11.3.1) (2018-11-05)
### Bug Fixes
* **get:** export hasContent.sync properly ([d76c920](https://github.com/npm/cacache/commit/d76c920))
<a name="11.3.0"></a>
# [11.3.0](https://github.com/npm/cacache/compare/v11.2.0...v11.3.0) (2018-11-05)
### Features
* **get:** add sync API for reading ([db1e094](https://github.com/npm/cacache/commit/db1e094))
<a name="11.2.0"></a>
# [11.2.0](https://github.com/npm/cacache/compare/v11.1.0...v11.2.0) (2018-08-08)
### Features
* **read:** add sync support to other internal read.js fns ([fe638b6](https://github.com/npm/cacache/commit/fe638b6))
<a name="11.1.0"></a>
# [11.1.0](https://github.com/npm/cacache/compare/v11.0.3...v11.1.0) (2018-08-01)
### Features
* **read:** add sync support for low-level content read ([b43af83](https://github.com/npm/cacache/commit/b43af83))
<a name="11.0.3"></a>
## [11.0.3](https://github.com/npm/cacache/compare/v11.0.2...v11.0.3) (2018-08-01)
### Bug Fixes
* **config:** add ssri config options ([#136](https://github.com/npm/cacache/issues/136)) ([10d5d9a](https://github.com/npm/cacache/commit/10d5d9a))
* **perf:** refactor content.read to avoid lstats ([c5ac10e](https://github.com/npm/cacache/commit/c5ac10e))
* **test:** oops when removing safe-buffer ([1950490](https://github.com/npm/cacache/commit/1950490))
<a name="11.0.2"></a>
## [11.0.2](https://github.com/npm/cacache/compare/v11.0.1...v11.0.2) (2018-05-07)
### Bug Fixes
* **verify:** size param no longer lost in a verify ([#131](https://github.com/npm/cacache/issues/131)) ([c614a19](https://github.com/npm/cacache/commit/c614a19)), closes [#130](https://github.com/npm/cacache/issues/130)
<a name="11.0.1"></a>
## [11.0.1](https://github.com/npm/cacache/compare/v11.0.0...v11.0.1) (2018-04-10)
<a name="11.0.0"></a>
# [11.0.0](https://github.com/npm/cacache/compare/v10.0.4...v11.0.0) (2018-04-09)
### Features
* **opts:** use figgy-pudding for opts ([#128](https://github.com/npm/cacache/issues/128)) ([33d4eed](https://github.com/npm/cacache/commit/33d4eed))
### meta
* drop support for node@4 ([529f347](https://github.com/npm/cacache/commit/529f347))
### BREAKING CHANGES
* node@4 is no longer supported
<a name="10.0.4"></a>
## [10.0.4](https://github.com/npm/cacache/compare/v10.0.3...v10.0.4) (2018-02-16)
<a name="10.0.3"></a>
## [10.0.3](https://github.com/npm/cacache/compare/v10.0.2...v10.0.3) (2018-02-16)
### Bug Fixes
* **content:** rethrow aggregate errors as ENOENT ([fa918f5](https://github.com/npm/cacache/commit/fa918f5))
<a name="10.0.2"></a>
## [10.0.2](https://github.com/npm/cacache/compare/v10.0.1...v10.0.2) (2018-01-07)
### Bug Fixes
* **ls:** deleted entries could cause a premature stream EOF ([347dc36](https://github.com/npm/cacache/commit/347dc36))
<a name="10.0.1"></a>
## [10.0.1](https://github.com/npm/cacache/compare/v10.0.0...v10.0.1) (2017-11-15)
### Bug Fixes
* **move-file:** actually use the fallback to `move-concurrently` (#110) ([073fbe1](https://github.com/npm/cacache/commit/073fbe1))
<a name="10.0.0"></a>
# [10.0.0](https://github.com/npm/cacache/compare/v9.3.0...v10.0.0) (2017-10-23)
### Features
* **license:** relicense to ISC (#111) ([fdbb4e5](https://github.com/npm/cacache/commit/fdbb4e5))
### Performance Improvements
* more copyFile benchmarks ([63787bb](https://github.com/npm/cacache/commit/63787bb))
### BREAKING CHANGES
* **license:** the license has been changed from CC0-1.0 to ISC.
<a name="9.3.0"></a>
# [9.3.0](https://github.com/npm/cacache/compare/v9.2.9...v9.3.0) (2017-10-07)
### Features
* **copy:** added cacache.get.copy api for fast copies (#107) ([067b5f6](https://github.com/npm/cacache/commit/067b5f6))
<a name="9.2.9"></a>
## [9.2.9](https://github.com/npm/cacache/compare/v9.2.8...v9.2.9) (2017-06-17)
<a name="9.2.8"></a>
## [9.2.8](https://github.com/npm/cacache/compare/v9.2.7...v9.2.8) (2017-06-05)
### Bug Fixes
* **ssri:** bump ssri for bugfix ([c3232ea](https://github.com/npm/cacache/commit/c3232ea))
<a name="9.2.7"></a>
## [9.2.7](https://github.com/npm/cacache/compare/v9.2.6...v9.2.7) (2017-06-05)
### Bug Fixes
* **content:** make verified content completely read-only (#96) ([4131196](https://github.com/npm/cacache/commit/4131196))
<a name="9.2.6"></a>
## [9.2.6](https://github.com/npm/cacache/compare/v9.2.5...v9.2.6) (2017-05-31)
### Bug Fixes
* **node:** update ssri to prevent old node 4 crash ([5209ffe](https://github.com/npm/cacache/commit/5209ffe))
<a name="9.2.5"></a>
## [9.2.5](https://github.com/npm/cacache/compare/v9.2.4...v9.2.5) (2017-05-25)
### Bug Fixes
* **deps:** fix lockfile issues and bump ssri ([84e1d7e](https://github.com/npm/cacache/commit/84e1d7e))
<a name="9.2.4"></a>
## [9.2.4](https://github.com/npm/cacache/compare/v9.2.3...v9.2.4) (2017-05-24)
### Bug Fixes
* **deps:** bumping deps ([bbccb12](https://github.com/npm/cacache/commit/bbccb12))
<a name="9.2.3"></a>
## [9.2.3](https://github.com/npm/cacache/compare/v9.2.2...v9.2.3) (2017-05-24)
### Bug Fixes
* **rm:** stop crashing if content is missing on rm ([ac90bc0](https://github.com/npm/cacache/commit/ac90bc0))
<a name="9.2.2"></a>
## [9.2.2](https://github.com/npm/cacache/compare/v9.2.1...v9.2.2) (2017-05-14)
### Bug Fixes
* **i18n:** lets pretend this didn't happen ([519b4ee](https://github.com/npm/cacache/commit/519b4ee))
<a name="9.2.1"></a>
## [9.2.1](https://github.com/npm/cacache/compare/v9.2.0...v9.2.1) (2017-05-14)
### Bug Fixes
* **docs:** fixing translation messup ([bb9e4f9](https://github.com/npm/cacache/commit/bb9e4f9))
<a name="9.2.0"></a>
# [9.2.0](https://github.com/npm/cacache/compare/v9.1.0...v9.2.0) (2017-05-14)
### Features
* **i18n:** add Spanish translation for API ([531f9a4](https://github.com/npm/cacache/commit/531f9a4))
<a name="9.1.0"></a>
# [9.1.0](https://github.com/npm/cacache/compare/v9.0.0...v9.1.0) (2017-05-14)
### Features
* **i18n:** Add Spanish translation and i18n setup (#91) ([323b90c](https://github.com/npm/cacache/commit/323b90c))
<a name="9.0.0"></a>
# [9.0.0](https://github.com/npm/cacache/compare/v8.0.0...v9.0.0) (2017-04-28)
### Bug Fixes
* **memoization:** actually use the LRU ([0e55dc9](https://github.com/npm/cacache/commit/0e55dc9))
### Features
* **memoization:** memoizers can be injected through opts.memoize (#90) ([e5614c7](https://github.com/npm/cacache/commit/e5614c7))
### BREAKING CHANGES
* **memoization:** If you were passing an object to opts.memoize, it will now be used as an injected memoization object. If you were only passing booleans and other non-objects through that option, no changes are needed.
<a name="8.0.0"></a>
# [8.0.0](https://github.com/npm/cacache/compare/v7.1.0...v8.0.0) (2017-04-22)
### Features
* **read:** change hasContent to return {sri, size} (#88) ([bad6c49](https://github.com/npm/cacache/commit/bad6c49)), closes [#87](https://github.com/npm/cacache/issues/87)
### BREAKING CHANGES
* **read:** hasContent now returns an object with `{sri, size}` instead of `sri`. Use `result.sri` anywhere that needed the old return value.
<a name="7.1.0"></a>
# [7.1.0](https://github.com/npm/cacache/compare/v7.0.5...v7.1.0) (2017-04-20)
### Features
* **size:** handle content size info (#49) ([91230af](https://github.com/npm/cacache/commit/91230af))
<a name="7.0.5"></a>
## [7.0.5](https://github.com/npm/cacache/compare/v7.0.4...v7.0.5) (2017-04-18)
### Bug Fixes
* **integrity:** new ssri with fixed integrity stream ([6d13e8e](https://github.com/npm/cacache/commit/6d13e8e))
* **write:** wrap stuff in promises to improve errors ([3624fc5](https://github.com/npm/cacache/commit/3624fc5))
<a name="7.0.4"></a>
## [7.0.4](https://github.com/npm/cacache/compare/v7.0.3...v7.0.4) (2017-04-15)
### Bug Fixes
* **fix-owner:** throw away ENOENTs on chownr ([d49bbcd](https://github.com/npm/cacache/commit/d49bbcd))
<a name="7.0.3"></a>
## [7.0.3](https://github.com/npm/cacache/compare/v7.0.2...v7.0.3) (2017-04-05)
### Bug Fixes
* **read:** fixing error message for integrity verification failures ([9d4f0a5](https://github.com/npm/cacache/commit/9d4f0a5))
<a name="7.0.2"></a>
## [7.0.2](https://github.com/npm/cacache/compare/v7.0.1...v7.0.2) (2017-04-03)
### Bug Fixes
* **integrity:** use EINTEGRITY error code and update ssri ([8dc2e62](https://github.com/npm/cacache/commit/8dc2e62))
<a name="7.0.1"></a>
## [7.0.1](https://github.com/npm/cacache/compare/v7.0.0...v7.0.1) (2017-04-03)
### Bug Fixes
* **docs:** fix header name conflict in readme ([afcd456](https://github.com/npm/cacache/commit/afcd456))
<a name="7.0.0"></a>
# [7.0.0](https://github.com/npm/cacache/compare/v6.3.0...v7.0.0) (2017-04-03)
### Bug Fixes
* **test:** fix content.write tests when running in docker ([d2e9b6a](https://github.com/npm/cacache/commit/d2e9b6a))
### Features
* **integrity:** subresource integrity support (#78) ([b1e731f](https://github.com/npm/cacache/commit/b1e731f))
### BREAKING CHANGES
* **integrity:** The entire API has been overhauled to use SRI hashes instead of digest/hashAlgorithm pairs. SRI hashes follow the Subresource Integrity standard and support strings and objects compatible with [`ssri`](https://npm.im/ssri).
* This change bumps the index version, which will invalidate all previous index entries. Content entries will remain intact, and existing caches will automatically reuse any content from before this breaking change.
* `cacache.get.info()`, `cacache.ls()`, and `cacache.ls.stream()` will now return objects that looks like this:
```
{
key: String,
integrity: '<algorithm>-<base64hash>',
path: ContentPath,
time: Date<ms>,
metadata: Any
}
```
* `opts.digest` and `opts.hashAlgorithm` are obsolete for any API calls that used them.
* Anywhere `opts.digest` was accepted, `opts.integrity` is now an option. Any valid SRI hash is accepted here -- multiple hash entries will be resolved according to the standard: first, the "strongest" hash algorithm will be picked, and then each of the entries for that algorithm will be matched against the content. Content will be validated if *any* of the entries match (so, a single integrity string can be used for multiple "versions" of the same document/data).
* `put.byDigest()`, `put.stream.byDigest`, `get.byDigest()` and `get.stream.byDigest()` now expect an SRI instead of a `digest` + `opts.hashAlgorithm` pairing.
* `get.hasContent()` now expects an integrity hash instead of a digest. If content exists, it will return the specific single integrity hash that was found in the cache.
* `verify()` has learned to handle integrity-based caches, and forgotten how to handle old-style cache indices due to the format change.
* `cacache.rm.content()` now expects an integrity hash instead of a hex digest.
<a name="6.3.0"></a>
# [6.3.0](https://github.com/npm/cacache/compare/v6.2.0...v6.3.0) (2017-04-01)
### Bug Fixes
* **fixOwner:** ignore EEXIST race condition from mkdirp ([4670e9b](https://github.com/npm/cacache/commit/4670e9b))
* **index:** ignore index removal races when inserting ([b9d2fa2](https://github.com/npm/cacache/commit/b9d2fa2))
* **memo:** use lru-cache for better mem management (#75) ([d8ac5aa](https://github.com/npm/cacache/commit/d8ac5aa))
### Features
* **dependencies:** Switch to move-concurrently (#77) ([dc6482d](https://github.com/npm/cacache/commit/dc6482d))
<a name="6.2.0"></a>
# [6.2.0](https://github.com/npm/cacache/compare/v6.1.2...v6.2.0) (2017-03-15)
### Bug Fixes
* **index:** additional bucket entry verification with checksum (#72) ([f8e0f25](https://github.com/npm/cacache/commit/f8e0f25))
* **verify:** return fixOwner.chownr promise ([6818521](https://github.com/npm/cacache/commit/6818521))
### Features
* **tmp:** safe tmp dir creation/management util (#73) ([c42da71](https://github.com/npm/cacache/commit/c42da71))
<a name="6.1.2"></a>
## [6.1.2](https://github.com/npm/cacache/compare/v6.1.1...v6.1.2) (2017-03-13)
### Bug Fixes
* **index:** set default hashAlgorithm ([d6eb2f0](https://github.com/npm/cacache/commit/d6eb2f0))
<a name="6.1.1"></a>
## [6.1.1](https://github.com/npm/cacache/compare/v6.1.0...v6.1.1) (2017-03-13)
### Bug Fixes
* **coverage:** bumping coverage for verify (#71) ([0b7faf6](https://github.com/npm/cacache/commit/0b7faf6))
* **deps:** glob should have been a regular dep :< ([0640bc4](https://github.com/npm/cacache/commit/0640bc4))
<a name="6.1.0"></a>
# [6.1.0](https://github.com/npm/cacache/compare/v6.0.2...v6.1.0) (2017-03-12)
### Bug Fixes
* **coverage:** more coverage for content reads (#70) ([ef4f70a](https://github.com/npm/cacache/commit/ef4f70a))
* **tests:** use safe-buffer because omfg (#69) ([6ab8132](https://github.com/npm/cacache/commit/6ab8132))
### Features
* **rm:** limited rm.all and fixed bugs (#66) ([d5d25ba](https://github.com/npm/cacache/commit/d5d25ba)), closes [#66](https://github.com/npm/cacache/issues/66)
* **verify:** tested, working cache verifier/gc (#68) ([45ad77a](https://github.com/npm/cacache/commit/45ad77a))
<a name="6.0.2"></a>
## [6.0.2](https://github.com/npm/cacache/compare/v6.0.1...v6.0.2) (2017-03-11)
### Bug Fixes
* **index:** segment cache items with another subbucket (#64) ([c3644e5](https://github.com/npm/cacache/commit/c3644e5))
<a name="6.0.1"></a>
## [6.0.1](https://github.com/npm/cacache/compare/v6.0.0...v6.0.1) (2017-03-05)
### Bug Fixes
* **docs:** Missed spots in README ([8ffb7fa](https://github.com/npm/cacache/commit/8ffb7fa))
<a name="6.0.0"></a>
# [6.0.0](https://github.com/npm/cacache/compare/v5.0.3...v6.0.0) (2017-03-05)
### Bug Fixes
* **api:** keep memo cache mostly-internal ([2f72d0a](https://github.com/npm/cacache/commit/2f72d0a))
* **content:** use the rest of the string, not the whole string ([fa8f3c3](https://github.com/npm/cacache/commit/fa8f3c3))
* **deps:** removed `format-number@2.0.2` ([1187791](https://github.com/npm/cacache/commit/1187791))
* **deps:** removed inflight@1.0.6 ([0d1819c](https://github.com/npm/cacache/commit/0d1819c))
* **deps:** rimraf@2.6.1 ([9efab6b](https://github.com/npm/cacache/commit/9efab6b))
* **deps:** standard@9.0.0 ([4202cba](https://github.com/npm/cacache/commit/4202cba))
* **deps:** tap@10.3.0 ([aa03088](https://github.com/npm/cacache/commit/aa03088))
* **deps:** weallcontribute@1.0.8 ([ad4f4dc](https://github.com/npm/cacache/commit/ad4f4dc))
* **docs:** add security note to hashKey ([03f81ba](https://github.com/npm/cacache/commit/03f81ba))
* **hashes:** change default hashAlgorithm to sha512 ([ea00ba6](https://github.com/npm/cacache/commit/ea00ba6))
* **hashes:** missed a spot for hashAlgorithm defaults ([45997d8](https://github.com/npm/cacache/commit/45997d8))
* **index:** add length header before JSON for verification ([fb8cb4d](https://github.com/npm/cacache/commit/fb8cb4d))
* **index:** change index filenames to sha1s of keys ([bbc5fca](https://github.com/npm/cacache/commit/bbc5fca))
* **index:** who cares about race conditions anyway ([b1d3888](https://github.com/npm/cacache/commit/b1d3888))
* **perf:** bulk-read get+read for massive speed ([d26cdf9](https://github.com/npm/cacache/commit/d26cdf9))
* **perf:** use bulk file reads for index reads ([79a8891](https://github.com/npm/cacache/commit/79a8891))
* **put-stream:** remove tmp file on stream insert error ([65f6632](https://github.com/npm/cacache/commit/65f6632))
* **put-stream:** robustified and predictibilized ([daf9e08](https://github.com/npm/cacache/commit/daf9e08))
* **put-stream:** use new promise API for moves ([1d36013](https://github.com/npm/cacache/commit/1d36013))
* **readme:** updated to reflect new default hashAlgo ([c60a2fa](https://github.com/npm/cacache/commit/c60a2fa))
* **verify:** tiny typo fix ([db22d05](https://github.com/npm/cacache/commit/db22d05))
### Features
* **api:** converted external api ([7bf032f](https://github.com/npm/cacache/commit/7bf032f))
* **cacache:** exported clearMemoized() utility ([8d2c5b6](https://github.com/npm/cacache/commit/8d2c5b6))
* **cache:** add versioning to content and index ([31bc549](https://github.com/npm/cacache/commit/31bc549))
* **content:** collate content files into subdirs ([c094d9f](https://github.com/npm/cacache/commit/c094d9f))
* **deps:** `@npmcorp/move@1.0.0` ([bdd00bf](https://github.com/npm/cacache/commit/bdd00bf))
* **deps:** `bluebird@3.4.7` ([3a17aff](https://github.com/npm/cacache/commit/3a17aff))
* **deps:** `promise-inflight@1.0.1` ([a004fe6](https://github.com/npm/cacache/commit/a004fe6))
* **get:** added memoization support for get ([c77d794](https://github.com/npm/cacache/commit/c77d794))
* **get:** export hasContent ([2956ec3](https://github.com/npm/cacache/commit/2956ec3))
* **index:** add hashAlgorithm and format insert ret val ([b639746](https://github.com/npm/cacache/commit/b639746))
* **index:** collate index files into subdirs ([e8402a5](https://github.com/npm/cacache/commit/e8402a5))
* **index:** promisify entry index ([cda3335](https://github.com/npm/cacache/commit/cda3335))
* **memo:** added memoization lib ([da07b92](https://github.com/npm/cacache/commit/da07b92))
* **memo:** export memoization api ([954b1b3](https://github.com/npm/cacache/commit/954b1b3))
* **move-file:** add move fallback for weird errors ([5cf4616](https://github.com/npm/cacache/commit/5cf4616))
* **perf:** bulk content write api ([51b536e](https://github.com/npm/cacache/commit/51b536e))
* **put:** added memoization support to put ([b613a70](https://github.com/npm/cacache/commit/b613a70))
* **read:** switched to promises ([a869362](https://github.com/npm/cacache/commit/a869362))
* **rm:** added memoization support to rm ([4205cf0](https://github.com/npm/cacache/commit/4205cf0))
* **rm:** switched to promises ([a000d24](https://github.com/npm/cacache/commit/a000d24))
* **util:** promise-inflight ownership fix requests ([9517cd7](https://github.com/npm/cacache/commit/9517cd7))
* **util:** use promises for api ([ae204bb](https://github.com/npm/cacache/commit/ae204bb))
* **verify:** converted to Promises ([f0b3974](https://github.com/npm/cacache/commit/f0b3974))
### BREAKING CHANGES
* cache: index/content directories are now versioned. Previous caches are no longer compatible and cannot be migrated.
* util: fix-owner now uses Promises instead of callbacks
* index: Previously-generated index entries are no longer compatible and the index must be regenerated.
* index: The index format has changed and previous caches are no longer compatible. Existing caches will need to be regenerated.
* hashes: Default hashAlgorithm changed from sha1 to sha512. If you
rely on the prior setting, pass `opts.hashAlgorithm` in explicitly.
* content: Previously-generated content directories are no longer compatible
and must be regenerated.
* verify: API is now promise-based
* read: Switches to a Promise-based API and removes callback stuff
* rm: Switches to a Promise-based API and removes callback stuff
* index: this changes the API to work off promises instead of callbacks
* api: this means we are going all in on promises now
@@ -0,0 +1,16 @@
ISC License
Copyright (c) npm, Inc.
Permission to use, copy, modify, and/or distribute this software for
any purpose with or without fee is hereby granted, provided that the
above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE COPYRIGHT HOLDER DISCLAIMS
ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE
USE OR PERFORMANCE OF THIS SOFTWARE.
@@ -0,0 +1,617 @@
# cacache [![npm version](https://img.shields.io/npm/v/cacache.svg)](https://npm.im/cacache) [![license](https://img.shields.io/npm/l/cacache.svg)](https://npm.im/cacache) [![Travis](https://img.shields.io/travis/npm/cacache.svg)](https://travis-ci.org/npm/cacache) [![AppVeyor](https://ci.appveyor.com/api/projects/status/github/npm/cacache?svg=true)](https://ci.appveyor.com/project/npm/cacache) [![Coverage Status](https://coveralls.io/repos/github/npm/cacache/badge.svg?branch=latest)](https://coveralls.io/github/npm/cacache?branch=latest)
[`cacache`](https://github.com/npm/cacache) is a Node.js library for managing
local key and content address caches. It's really fast, really good at
concurrency, and it will never give you corrupted data, even if cache files
get corrupted or manipulated.
On systems that support user and group settings on files, cacache will
match the `uid` and `gid` values to the folder where the cache lives, even
when running as `root`.
It was written to be used as [npm](https://npm.im)'s local cache, but can
just as easily be used on its own.
_Translations: [español](README.es.md)_
## Install
`$ npm install --save cacache`
## Table of Contents
* [Example](#example)
* [Features](#features)
* [Contributing](#contributing)
* [API](#api)
* [Using localized APIs](#localized-api)
* Reading
* [`ls`](#ls)
* [`ls.stream`](#ls-stream)
* [`get`](#get-data)
* [`get.stream`](#get-stream)
* [`get.info`](#get-info)
* [`get.hasContent`](#get-hasContent)
* Writing
* [`put`](#put-data)
* [`put.stream`](#put-stream)
* [`put*` opts](#put-options)
* [`rm.all`](#rm-all)
* [`rm.entry`](#rm-entry)
* [`rm.content`](#rm-content)
* Utilities
* [`clearMemoized`](#clear-memoized)
* [`tmp.mkdir`](#tmp-mkdir)
* [`tmp.withTmp`](#with-tmp)
* Integrity
* [Subresource Integrity](#integrity)
* [`verify`](#verify)
* [`verify.lastRun`](#verify-last-run)
### Example
```javascript
const cacache = require('cacache')
const fs = require('fs')
const tarball = '/path/to/mytar.tgz'
const cachePath = '/tmp/my-toy-cache'
const key = 'my-unique-key-1234'
// Cache it! Use `cachePath` as the root of the content cache
cacache.put(cachePath, key, '10293801983029384').then(integrity => {
console.log(`Saved content to ${cachePath}.`)
})
const destination = '/tmp/mytar.tgz'
// Copy the contents out of the cache and into their destination!
// But this time, use stream instead!
cacache.get.stream(
cachePath, key
).pipe(
fs.createWriteStream(destination)
).on('finish', () => {
console.log('done extracting!')
})
// The same thing, but skip the key index.
cacache.get.byDigest(cachePath, integrityHash).then(data => {
fs.writeFile(destination, data, err => {
console.log('tarball data fetched based on its sha512sum and written out!')
})
})
```
### Features
* Extraction by key or by content address (shasum, etc)
* [Subresource Integrity](#integrity) web standard support
* Multi-hash support - safely host sha1, sha512, etc, in a single cache
* Automatic content deduplication
* Fault tolerance (immune to corruption, partial writes, process races, etc)
* Consistency guarantees on read and write (full data verification)
* Lockless, high-concurrency cache access
* Streaming support
* Promise support
* Pretty darn fast -- sub-millisecond reads and writes including verification
* Arbitrary metadata storage
* Garbage collection and additional offline verification
* Thorough test coverage
* There's probably a bloom filter in there somewhere. Those are cool, right? 🤔
### Contributing
The cacache team enthusiastically welcomes contributions and project participation! There's a bunch of things you can do if you want to contribute! The [Contributor Guide](CONTRIBUTING.md) has all the information you need for everything from reporting bugs to contributing entire new features. Please don't hesitate to jump in if you'd like to, or even ask us questions if something isn't clear.
All participants and maintainers in this project are expected to follow [Code of Conduct](CODE_OF_CONDUCT.md), and just generally be excellent to each other.
Please refer to the [Changelog](CHANGELOG.md) for project history details, too.
Happy hacking!
### API
#### <a name="ls"></a> `> cacache.ls(cache) -> Promise<Object>`
Lists info for all entries currently in the cache as a single large object. Each
entry in the object will be keyed by the unique index key, with corresponding
[`get.info`](#get-info) objects as the values.
##### Example
```javascript
cacache.ls(cachePath).then(console.log)
// Output
{
'my-thing': {
key: 'my-thing',
integrity: 'sha512-BaSe64/EnCoDED+HAsh=='
path: '.testcache/content/deadbeef', // joined with `cachePath`
time: 12345698490,
size: 4023948,
metadata: {
name: 'blah',
version: '1.2.3',
description: 'this was once a package but now it is my-thing'
}
},
'other-thing': {
key: 'other-thing',
integrity: 'sha1-ANothER+hasH=',
path: '.testcache/content/bada55',
time: 11992309289,
size: 111112
}
}
```
#### <a name="ls-stream"></a> `> cacache.ls.stream(cache) -> Readable`
Lists info for all entries currently in the cache as a single large object.
This works just like [`ls`](#ls), except [`get.info`](#get-info) entries are
returned as `'data'` events on the returned stream.
##### Example
```javascript
cacache.ls.stream(cachePath).on('data', console.log)
// Output
{
key: 'my-thing',
integrity: 'sha512-BaSe64HaSh',
path: '.testcache/content/deadbeef', // joined with `cachePath`
time: 12345698490,
size: 13423,
metadata: {
name: 'blah',
version: '1.2.3',
description: 'this was once a package but now it is my-thing'
}
}
{
key: 'other-thing',
integrity: 'whirlpool-WoWSoMuchSupport',
path: '.testcache/content/bada55',
time: 11992309289,
size: 498023984029
}
{
...
}
```
#### <a name="get-data"></a> `> cacache.get(cache, key, [opts]) -> Promise({data, metadata, integrity})`
Returns an object with the cached data, digest, and metadata identified by
`key`. The `data` property of this object will be a `Buffer` instance that
presumably holds some data that means something to you. I'm sure you know what
to do with it! cacache just won't care.
`integrity` is a [Subresource
Integrity](#integrity)
string. That is, a string that can be used to verify `data`, which looks like
`<hash-algorithm>-<base64-integrity-hash>`.
If there is no content identified by `key`, or if the locally-stored data does
not pass the validity checksum, the promise will be rejected.
A sub-function, `get.byDigest` may be used for identical behavior, except lookup
will happen by integrity hash, bypassing the index entirely. This version of the
function *only* returns `data` itself, without any wrapper.
##### Note
This function loads the entire cache entry into memory before returning it. If
you're dealing with Very Large data, consider using [`get.stream`](#get-stream)
instead.
##### Example
```javascript
// Look up by key
cache.get(cachePath, 'my-thing').then(console.log)
// Output:
{
metadata: {
thingName: 'my'
},
integrity: 'sha512-BaSe64HaSh',
data: Buffer#<deadbeef>,
size: 9320
}
// Look up by digest
cache.get.byDigest(cachePath, 'sha512-BaSe64HaSh').then(console.log)
// Output:
Buffer#<deadbeef>
```
#### <a name="get-stream"></a> `> cacache.get.stream(cache, key, [opts]) -> Readable`
Returns a [Readable Stream](https://nodejs.org/api/stream.html#stream_readable_streams) of the cached data identified by `key`.
If there is no content identified by `key`, or if the locally-stored data does
not pass the validity checksum, an error will be emitted.
`metadata` and `integrity` events will be emitted before the stream closes, if
you need to collect that extra data about the cached entry.
A sub-function, `get.stream.byDigest` may be used for identical behavior,
except lookup will happen by integrity hash, bypassing the index entirely. This
version does not emit the `metadata` and `integrity` events at all.
##### Example
```javascript
// Look up by key
cache.get.stream(
cachePath, 'my-thing'
).on('metadata', metadata => {
console.log('metadata:', metadata)
}).on('integrity', integrity => {
console.log('integrity:', integrity)
}).pipe(
fs.createWriteStream('./x.tgz')
)
// Outputs:
metadata: { ... }
integrity: 'sha512-SoMeDIGest+64=='
// Look up by digest
cache.get.stream.byDigest(
cachePath, 'sha512-SoMeDIGest+64=='
).pipe(
fs.createWriteStream('./x.tgz')
)
```
#### <a name="get-info"></a> `> cacache.get.info(cache, key) -> Promise`
Looks up `key` in the cache index, returning information about the entry if
one exists.
##### Fields
* `key` - Key the entry was looked up under. Matches the `key` argument.
* `integrity` - [Subresource Integrity hash](#integrity) for the content this entry refers to.
* `path` - Filesystem path where content is stored, joined with `cache` argument.
* `time` - Timestamp the entry was first added on.
* `metadata` - User-assigned metadata associated with the entry/content.
##### Example
```javascript
cacache.get.info(cachePath, 'my-thing').then(console.log)
// Output
{
key: 'my-thing',
integrity: 'sha256-MUSTVERIFY+ALL/THINGS=='
path: '.testcache/content/deadbeef',
time: 12345698490,
size: 849234,
metadata: {
name: 'blah',
version: '1.2.3',
description: 'this was once a package but now it is my-thing'
}
}
```
#### <a name="get-hasContent"></a> `> cacache.get.hasContent(cache, integrity) -> Promise`
Looks up a [Subresource Integrity hash](#integrity) in the cache. If content
exists for this `integrity`, it will return an object, with the specific single integrity hash
that was found in `sri` key, and the size of the found content as `size`. If no content exists for this integrity, it will return `false`.
##### Example
```javascript
cacache.get.hasContent(cachePath, 'sha256-MUSTVERIFY+ALL/THINGS==').then(console.log)
// Output
{
sri: {
source: 'sha256-MUSTVERIFY+ALL/THINGS==',
algorithm: 'sha256',
digest: 'MUSTVERIFY+ALL/THINGS==',
options: []
},
size: 9001
}
cacache.get.hasContent(cachePath, 'sha521-NOT+IN/CACHE==').then(console.log)
// Output
false
```
#### <a name="put-data"></a> `> cacache.put(cache, key, data, [opts]) -> Promise`
Inserts data passed to it into the cache. The returned Promise resolves with a
digest (generated according to [`opts.algorithms`](#optsalgorithms)) after the
cache entry has been successfully written.
##### Example
```javascript
fetch(
'https://registry.npmjs.org/cacache/-/cacache-1.0.0.tgz'
).then(data => {
return cacache.put(cachePath, 'registry.npmjs.org|cacache@1.0.0', data)
}).then(integrity => {
console.log('integrity hash is', integrity)
})
```
#### <a name="put-stream"></a> `> cacache.put.stream(cache, key, [opts]) -> Writable`
Returns a [Writable
Stream](https://nodejs.org/api/stream.html#stream_writable_streams) that inserts
data written to it into the cache. Emits an `integrity` event with the digest of
written contents when it succeeds.
##### Example
```javascript
request.get(
'https://registry.npmjs.org/cacache/-/cacache-1.0.0.tgz'
).pipe(
cacache.put.stream(
cachePath, 'registry.npmjs.org|cacache@1.0.0'
).on('integrity', d => console.log(`integrity digest is ${d}`))
)
```
#### <a name="put-options"></a> `> cacache.put options`
`cacache.put` functions have a number of options in common.
##### `opts.metadata`
Arbitrary metadata to be attached to the inserted key.
##### `opts.size`
If provided, the data stream will be verified to check that enough data was
passed through. If there's more or less data than expected, insertion will fail
with an `EBADSIZE` error.
##### `opts.integrity`
If present, the pre-calculated digest for the inserted content. If this option
if provided and does not match the post-insertion digest, insertion will fail
with an `EINTEGRITY` error.
`algorithms` has no effect if this option is present.
##### `opts.algorithms`
Default: ['sha512']
Hashing algorithms to use when calculating the [subresource integrity
digest](#integrity)
for inserted data. Can use any algorithm listed in `crypto.getHashes()` or
`'omakase'`/`'お任せします'` to pick a random hash algorithm on each insertion. You
may also use any anagram of `'modnar'` to use this feature.
Currently only supports one algorithm at a time (i.e., an array length of
exactly `1`). Has no effect if `opts.integrity` is present.
##### `opts.memoize`
Default: null
If provided, cacache will memoize the given cache insertion in memory, bypassing
any filesystem checks for that key or digest in future cache fetches. Nothing
will be written to the in-memory cache unless this option is explicitly truthy.
If `opts.memoize` is an object or a `Map`-like (that is, an object with `get`
and `set` methods), it will be written to instead of the global memoization
cache.
Reading from disk data can be forced by explicitly passing `memoize: false` to
the reader functions, but their default will be to read from memory.
#### <a name="rm-all"></a> `> cacache.rm.all(cache) -> Promise`
Clears the entire cache. Mainly by blowing away the cache directory itself.
##### Example
```javascript
cacache.rm.all(cachePath).then(() => {
console.log('THE APOCALYPSE IS UPON US 😱')
})
```
#### <a name="rm-entry"></a> `> cacache.rm.entry(cache, key) -> Promise`
Alias: `cacache.rm`
Removes the index entry for `key`. Content will still be accessible if
requested directly by content address ([`get.stream.byDigest`](#get-stream)).
To remove the content itself (which might still be used by other entries), use
[`rm.content`](#rm-content). Or, to safely vacuum any unused content, use
[`verify`](#verify).
##### Example
```javascript
cacache.rm.entry(cachePath, 'my-thing').then(() => {
console.log('I did not like it anyway')
})
```
#### <a name="rm-content"></a> `> cacache.rm.content(cache, integrity) -> Promise`
Removes the content identified by `integrity`. Any index entries referring to it
will not be usable again until the content is re-added to the cache with an
identical digest.
##### Example
```javascript
cacache.rm.content(cachePath, 'sha512-SoMeDIGest/IN+BaSE64==').then(() => {
console.log('data for my-thing is gone!')
})
```
#### <a name="clear-memoized"></a> `> cacache.clearMemoized()`
Completely resets the in-memory entry cache.
#### <a name="tmp-mkdir"></a> `> tmp.mkdir(cache, opts) -> Promise<Path>`
Returns a unique temporary directory inside the cache's `tmp` dir. This
directory will use the same safe user assignment that all the other stuff use.
Once the directory is made, it's the user's responsibility that all files
within are given the appropriate `gid`/`uid` ownership settings to match
the rest of the cache. If not, you can ask cacache to do it for you by
calling [`tmp.fix()`](#tmp-fix), which will fix all tmp directory
permissions.
If you want automatic cleanup of this directory, use
[`tmp.withTmp()`](#with-tpm)
##### Example
```javascript
cacache.tmp.mkdir(cache).then(dir => {
fs.writeFile(path.join(dir, 'blablabla'), Buffer#<1234>, ...)
})
```
#### <a name="tmp-fix"></a> `> tmp.fix(cache) -> Promise`
Sets the `uid` and `gid` properties on all files and folders within the tmp
folder to match the rest of the cache.
Use this after manually writing files into [`tmp.mkdir`](#tmp-mkdir) or
[`tmp.withTmp`](#with-tmp).
##### Example
```javascript
cacache.tmp.mkdir(cache).then(dir => {
writeFile(path.join(dir, 'file'), someData).then(() => {
// make sure we didn't just put a root-owned file in the cache
cacache.tmp.fix().then(() => {
// all uids and gids match now
})
})
})
```
#### <a name="with-tmp"></a> `> tmp.withTmp(cache, opts, cb) -> Promise`
Creates a temporary directory with [`tmp.mkdir()`](#tmp-mkdir) and calls `cb`
with it. The created temporary directory will be removed when the return value
of `cb()` resolves, the tmp directory will be automatically deleted once that
promise completes.
The same caveats apply when it comes to managing permissions for the tmp dir's
contents.
##### Example
```javascript
cacache.tmp.withTmp(cache, dir => {
return fs.writeFileAsync(path.join(dir, 'blablabla'), Buffer#<1234>, ...)
}).then(() => {
// `dir` no longer exists
})
```
#### <a name="integrity"></a> Subresource Integrity Digests
For content verification and addressing, cacache uses strings following the
[Subresource
Integrity spec](https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity).
That is, any time cacache expects an `integrity` argument or option, it
should be in the format `<hashAlgorithm>-<base64-hash>`.
One deviation from the current spec is that cacache will support any hash
algorithms supported by the underlying Node.js process. You can use
`crypto.getHashes()` to see which ones you can use.
##### Generating Digests Yourself
If you have an existing content shasum, they are generally formatted as a
hexadecimal string (that is, a sha1 would look like:
`5f5513f8822fdbe5145af33b64d8d970dcf95c6e`). In order to be compatible with
cacache, you'll need to convert this to an equivalent subresource integrity
string. For this example, the corresponding hash would be:
`sha1-X1UT+IIv2+UUWvM7ZNjZcNz5XG4=`.
If you want to generate an integrity string yourself for existing data, you can
use something like this:
```javascript
const crypto = require('crypto')
const hashAlgorithm = 'sha512'
const data = 'foobarbaz'
const integrity = (
hashAlgorithm +
'-' +
crypto.createHash(hashAlgorithm).update(data).digest('base64')
)
```
You can also use [`ssri`](https://npm.im/ssri) to have a richer set of functionality
around SRI strings, including generation, parsing, and translating from existing
hex-formatted strings.
#### <a name="verify"></a> `> cacache.verify(cache, opts) -> Promise`
Checks out and fixes up your cache:
* Cleans up corrupted or invalid index entries.
* Custom entry filtering options.
* Garbage collects any content entries not referenced by the index.
* Checks integrity for all content entries and removes invalid content.
* Fixes cache ownership.
* Removes the `tmp` directory in the cache and all its contents.
When it's done, it'll return an object with various stats about the verification
process, including amount of storage reclaimed, number of valid entries, number
of entries removed, etc.
##### Options
* `opts.filter` - receives a formatted entry. Return false to remove it.
Note: might be called more than once on the same entry.
##### Example
```sh
echo somegarbage >> $CACHEPATH/content/deadbeef
```
```javascript
cacache.verify(cachePath).then(stats => {
// deadbeef collected, because of invalid checksum.
console.log('cache is much nicer now! stats:', stats)
})
```
#### <a name="verify-last-run"></a> `> cacache.verify.lastRun(cache) -> Promise`
Returns a `Date` representing the last time `cacache.verify` was run on `cache`.
##### Example
```javascript
cacache.verify(cachePath).then(() => {
cacache.verify.lastRun(cachePath).then(lastTime => {
console.log('cacache.verify was last called on' + lastTime)
})
})
```
+266
View File
@@ -0,0 +1,266 @@
'use strict'
const util = require('util')
const figgyPudding = require('figgy-pudding')
const fs = require('fs')
const index = require('./lib/entry-index')
const memo = require('./lib/memoization')
const read = require('./lib/content/read')
const Minipass = require('minipass')
const Collect = require('minipass-collect')
const Pipeline = require('minipass-pipeline')
const writeFile = util.promisify(fs.writeFile)
const GetOpts = figgyPudding({
integrity: {},
memoize: {},
size: {}
})
module.exports = function get (cache, key, opts) {
return getData(false, cache, key, opts)
}
module.exports.byDigest = function getByDigest (cache, digest, opts) {
return getData(true, cache, digest, opts)
}
function getData (byDigest, cache, key, opts) {
opts = GetOpts(opts)
const memoized = byDigest
? memo.get.byDigest(cache, key, opts)
: memo.get(cache, key, opts)
if (memoized && opts.memoize !== false) {
return Promise.resolve(
byDigest
? memoized
: {
metadata: memoized.entry.metadata,
data: memoized.data,
integrity: memoized.entry.integrity,
size: memoized.entry.size
}
)
}
return (byDigest ? Promise.resolve(null) : index.find(cache, key, opts)).then(
(entry) => {
if (!entry && !byDigest) {
throw new index.NotFoundError(cache, key)
}
return read(cache, byDigest ? key : entry.integrity, {
integrity: opts.integrity,
size: opts.size
})
.then((data) =>
byDigest
? data
: {
metadata: entry.metadata,
data: data,
size: entry.size,
integrity: entry.integrity
}
)
.then((res) => {
if (opts.memoize && byDigest) {
memo.put.byDigest(cache, key, res, opts)
} else if (opts.memoize) {
memo.put(cache, entry, res.data, opts)
}
return res
})
}
)
}
module.exports.sync = function get (cache, key, opts) {
return getDataSync(false, cache, key, opts)
}
module.exports.sync.byDigest = function getByDigest (cache, digest, opts) {
return getDataSync(true, cache, digest, opts)
}
function getDataSync (byDigest, cache, key, opts) {
opts = GetOpts(opts)
const memoized = byDigest
? memo.get.byDigest(cache, key, opts)
: memo.get(cache, key, opts)
if (memoized && opts.memoize !== false) {
return byDigest
? memoized
: {
metadata: memoized.entry.metadata,
data: memoized.data,
integrity: memoized.entry.integrity,
size: memoized.entry.size
}
}
const entry = !byDigest && index.find.sync(cache, key, opts)
if (!entry && !byDigest) {
throw new index.NotFoundError(cache, key)
}
const data = read.sync(cache, byDigest ? key : entry.integrity, {
integrity: opts.integrity,
size: opts.size
})
const res = byDigest
? data
: {
metadata: entry.metadata,
data: data,
size: entry.size,
integrity: entry.integrity
}
if (opts.memoize && byDigest) {
memo.put.byDigest(cache, key, res, opts)
} else if (opts.memoize) {
memo.put(cache, entry, res.data, opts)
}
return res
}
module.exports.stream = getStream
const getMemoizedStream = (memoized) => {
const stream = new Minipass()
stream.on('newListener', function (ev, cb) {
ev === 'metadata' && cb(memoized.entry.metadata)
ev === 'integrity' && cb(memoized.entry.integrity)
ev === 'size' && cb(memoized.entry.size)
})
stream.end(memoized.data)
return stream
}
function getStream (cache, key, opts) {
opts = GetOpts(opts)
const memoized = memo.get(cache, key, opts)
if (memoized && opts.memoize !== false) {
return getMemoizedStream(memoized)
}
const stream = new Pipeline()
index
.find(cache, key)
.then((entry) => {
if (!entry) {
throw new index.NotFoundError(cache, key)
}
stream.emit('metadata', entry.metadata)
stream.emit('integrity', entry.integrity)
stream.emit('size', entry.size)
stream.on('newListener', function (ev, cb) {
ev === 'metadata' && cb(entry.metadata)
ev === 'integrity' && cb(entry.integrity)
ev === 'size' && cb(entry.size)
})
const src = read.readStream(
cache,
entry.integrity,
opts.concat({
size: opts.size == null ? entry.size : opts.size
})
)
if (opts.memoize) {
const memoStream = new Collect.PassThrough()
memoStream.on('collect', data => memo.put(cache, entry, data, opts))
stream.unshift(memoStream)
}
stream.unshift(src)
})
.catch((err) => stream.emit('error', err))
return stream
}
module.exports.stream.byDigest = getStreamDigest
function getStreamDigest (cache, integrity, opts) {
opts = GetOpts(opts)
const memoized = memo.get.byDigest(cache, integrity, opts)
if (memoized && opts.memoize !== false) {
const stream = new Minipass()
stream.end(memoized)
return stream
} else {
const stream = read.readStream(cache, integrity, opts)
if (!opts.memoize) {
return stream
}
const memoStream = new Collect.PassThrough()
memoStream.on('collect', data => memo.put.byDigest(
cache,
integrity,
data,
opts
))
return new Pipeline(stream, memoStream)
}
}
module.exports.info = info
function info (cache, key, opts) {
opts = GetOpts(opts)
const memoized = memo.get(cache, key, opts)
if (memoized && opts.memoize !== false) {
return Promise.resolve(memoized.entry)
} else {
return index.find(cache, key)
}
}
module.exports.hasContent = read.hasContent
function cp (cache, key, dest, opts) {
return copy(false, cache, key, dest, opts)
}
module.exports.copy = cp
function cpDigest (cache, digest, dest, opts) {
return copy(true, cache, digest, dest, opts)
}
module.exports.copy.byDigest = cpDigest
function copy (byDigest, cache, key, dest, opts) {
opts = GetOpts(opts)
if (read.copy) {
return (byDigest
? Promise.resolve(null)
: index.find(cache, key, opts)
).then((entry) => {
if (!entry && !byDigest) {
throw new index.NotFoundError(cache, key)
}
return read
.copy(cache, byDigest ? key : entry.integrity, dest, opts)
.then(() => {
return byDigest
? key
: {
metadata: entry.metadata,
size: entry.size,
integrity: entry.integrity
}
})
})
}
return getData(byDigest, cache, key, opts).then((res) => {
return writeFile(dest, byDigest ? res : res.data).then(() => {
return byDigest
? key
: {
metadata: res.metadata,
size: res.size,
integrity: res.integrity
}
})
})
}
@@ -0,0 +1,41 @@
'use strict'
const ls = require('./ls.js')
const get = require('./get.js')
const put = require('./put.js')
const rm = require('./rm.js')
const verify = require('./verify.js')
const { clearMemoized } = require('./lib/memoization.js')
const tmp = require('./lib/util/tmp.js')
module.exports.ls = ls
module.exports.ls.stream = ls.stream
module.exports.get = get
module.exports.get.byDigest = get.byDigest
module.exports.get.sync = get.sync
module.exports.get.sync.byDigest = get.sync.byDigest
module.exports.get.stream = get.stream
module.exports.get.stream.byDigest = get.stream.byDigest
module.exports.get.copy = get.copy
module.exports.get.copy.byDigest = get.copy.byDigest
module.exports.get.info = get.info
module.exports.get.hasContent = get.hasContent
module.exports.get.hasContent.sync = get.hasContent.sync
module.exports.put = put
module.exports.put.stream = put.stream
module.exports.rm = rm.entry
module.exports.rm.all = rm.all
module.exports.rm.entry = module.exports.rm
module.exports.rm.content = rm.content
module.exports.clearMemoized = clearMemoized
module.exports.tmp = {}
module.exports.tmp.mkdir = tmp.mkdir
module.exports.tmp.withTmp = tmp.withTmp
module.exports.verify = verify
module.exports.verify.lastRun = verify.lastRun
@@ -0,0 +1,28 @@
'use strict'
const contentVer = require('../../package.json')['cache-version'].content
const hashToSegments = require('../util/hash-to-segments')
const path = require('path')
const ssri = require('ssri')
// Current format of content file path:
//
// sha512-BaSE64Hex= ->
// ~/.my-cache/content-v2/sha512/ba/da/55deadbeefc0ffee
//
module.exports = contentPath
function contentPath (cache, integrity) {
const sri = ssri.parse(integrity, { single: true })
// contentPath is the *strongest* algo given
return path.join.apply(
path,
[contentDir(cache), sri.algorithm].concat(hashToSegments(sri.hexDigest()))
)
}
module.exports.contentDir = contentDir
function contentDir (cache) {
return path.join(cache, `content-v${contentVer}`)
}
@@ -0,0 +1,261 @@
'use strict'
const util = require('util')
const figgyPudding = require('figgy-pudding')
const fs = require('graceful-fs')
const fsm = require('fs-minipass')
const ssri = require('ssri')
const contentPath = require('./path')
const Pipeline = require('minipass-pipeline')
const lstat = util.promisify(fs.lstat)
const readFile = util.promisify(fs.readFile)
const ReadOpts = figgyPudding({
size: {}
})
module.exports = read
const MAX_SINGLE_READ_SIZE = 64 * 1024 * 1024
function read (cache, integrity, opts) {
opts = ReadOpts(opts)
return withContentSri(cache, integrity, (cpath, sri) => {
// get size
return lstat(cpath).then(stat => ({ stat, cpath, sri }))
}).then(({ stat, cpath, sri }) => {
if (typeof opts.size === 'number' && stat.size !== opts.size) {
throw sizeError(opts.size, stat.size)
}
if (stat.size > MAX_SINGLE_READ_SIZE) {
return readPipeline(cpath, stat.size, sri, new Pipeline()).concat()
}
return readFile(cpath, null).then((data) => {
if (!ssri.checkData(data, sri)) {
throw integrityError(sri, cpath)
}
return data
})
})
}
const readPipeline = (cpath, size, sri, stream) => {
stream.push(
new fsm.ReadStream(cpath, {
size,
readSize: MAX_SINGLE_READ_SIZE
}),
ssri.integrityStream({
integrity: sri,
size
})
)
return stream
}
module.exports.sync = readSync
function readSync (cache, integrity, opts) {
opts = ReadOpts(opts)
return withContentSriSync(cache, integrity, (cpath, sri) => {
const data = fs.readFileSync(cpath)
if (typeof opts.size === 'number' && opts.size !== data.length) {
throw sizeError(opts.size, data.length)
}
if (ssri.checkData(data, sri)) {
return data
}
throw integrityError(sri, cpath)
})
}
module.exports.stream = readStream
module.exports.readStream = readStream
function readStream (cache, integrity, opts) {
opts = ReadOpts(opts)
const stream = new Pipeline()
withContentSri(cache, integrity, (cpath, sri) => {
// just lstat to ensure it exists
return lstat(cpath).then((stat) => ({ stat, cpath, sri }))
}).then(({ stat, cpath, sri }) => {
if (typeof opts.size === 'number' && opts.size !== stat.size) {
return stream.emit('error', sizeError(opts.size, stat.size))
}
readPipeline(cpath, stat.size, sri, stream)
}, er => stream.emit('error', er))
return stream
}
let copyFile
if (fs.copyFile) {
module.exports.copy = copy
module.exports.copy.sync = copySync
copyFile = util.promisify(fs.copyFile)
}
function copy (cache, integrity, dest, opts) {
opts = ReadOpts(opts)
return withContentSri(cache, integrity, (cpath, sri) => {
return copyFile(cpath, dest)
})
}
function copySync (cache, integrity, dest, opts) {
opts = ReadOpts(opts)
return withContentSriSync(cache, integrity, (cpath, sri) => {
return fs.copyFileSync(cpath, dest)
})
}
module.exports.hasContent = hasContent
function hasContent (cache, integrity) {
if (!integrity) {
return Promise.resolve(false)
}
return withContentSri(cache, integrity, (cpath, sri) => {
return lstat(cpath).then((stat) => ({ size: stat.size, sri, stat }))
}).catch((err) => {
if (err.code === 'ENOENT') {
return false
}
if (err.code === 'EPERM') {
if (process.platform !== 'win32') {
throw err
} else {
return false
}
}
})
}
module.exports.hasContent.sync = hasContentSync
function hasContentSync (cache, integrity) {
if (!integrity) {
return false
}
return withContentSriSync(cache, integrity, (cpath, sri) => {
try {
const stat = fs.lstatSync(cpath)
return { size: stat.size, sri, stat }
} catch (err) {
if (err.code === 'ENOENT') {
return false
}
if (err.code === 'EPERM') {
if (process.platform !== 'win32') {
throw err
} else {
return false
}
}
}
})
}
function withContentSri (cache, integrity, fn) {
const tryFn = () => {
const sri = ssri.parse(integrity)
// If `integrity` has multiple entries, pick the first digest
// with available local data.
const algo = sri.pickAlgorithm()
const digests = sri[algo]
if (digests.length <= 1) {
const cpath = contentPath(cache, digests[0])
return fn(cpath, digests[0])
} else {
// Can't use race here because a generic error can happen before a ENOENT error, and can happen before a valid result
return Promise
.all(sri[sri.pickAlgorithm()].map((meta) => {
return withContentSri(cache, meta, fn)
.catch((err) => {
if (err.code === 'ENOENT') {
return Object.assign(
new Error('No matching content found for ' + sri.toString()),
{ code: 'ENOENT' }
)
}
return err
})
}))
.then((results) => {
// Return the first non error if it is found
const result = results.find((r) => !(r instanceof Error))
if (result) {
return result
}
// Throw the No matching content found error
const enoentError = results.find((r) => r.code === 'ENOENT')
if (enoentError) {
throw enoentError
}
// Throw generic error
const genericError = results.find((r) => r instanceof Error)
if (genericError) {
throw genericError
}
})
}
}
return new Promise((resolve, reject) => {
try {
tryFn()
.then(resolve)
.catch(reject)
} catch (err) {
reject(err)
}
})
}
function withContentSriSync (cache, integrity, fn) {
const sri = ssri.parse(integrity)
// If `integrity` has multiple entries, pick the first digest
// with available local data.
const algo = sri.pickAlgorithm()
const digests = sri[algo]
if (digests.length <= 1) {
const cpath = contentPath(cache, digests[0])
return fn(cpath, digests[0])
} else {
let lastErr = null
for (const meta of sri[sri.pickAlgorithm()]) {
try {
return withContentSriSync(cache, meta, fn)
} catch (err) {
lastErr = err
}
}
if (lastErr) {
throw lastErr
}
}
}
function sizeError (expected, found) {
const err = new Error(`Bad data size: expected inserted data to be ${expected} bytes, but got ${found} instead`)
err.expected = expected
err.found = found
err.code = 'EBADSIZE'
return err
}
function integrityError (sri, path) {
const err = new Error(`Integrity verification failed for ${sri} (${path})`)
err.code = 'EINTEGRITY'
err.sri = sri
err.path = path
return err
}
@@ -0,0 +1,22 @@
'use strict'
const util = require('util')
const contentPath = require('./path')
const { hasContent } = require('./read')
const rimraf = util.promisify(require('rimraf'))
module.exports = rm
function rm (cache, integrity) {
return hasContent(cache, integrity).then((content) => {
if (content) {
const sri = content.sri
if (sri) {
return rimraf(contentPath(cache, sri)).then(() => true)
}
} else {
return false
}
})
}
@@ -0,0 +1,187 @@
'use strict'
const util = require('util')
const contentPath = require('./path')
const fixOwner = require('../util/fix-owner')
const fs = require('graceful-fs')
const moveFile = require('../util/move-file')
const Minipass = require('minipass')
const Pipeline = require('minipass-pipeline')
const Flush = require('minipass-flush')
const path = require('path')
const rimraf = util.promisify(require('rimraf'))
const ssri = require('ssri')
const uniqueFilename = require('unique-filename')
const { disposer } = require('./../util/disposer')
const fsm = require('fs-minipass')
const writeFile = util.promisify(fs.writeFile)
module.exports = write
function write (cache, data, opts) {
opts = opts || {}
if (opts.algorithms && opts.algorithms.length > 1) {
throw new Error('opts.algorithms only supports a single algorithm for now')
}
if (typeof opts.size === 'number' && data.length !== opts.size) {
return Promise.reject(sizeError(opts.size, data.length))
}
const sri = ssri.fromData(data, {
algorithms: opts.algorithms
})
if (opts.integrity && !ssri.checkData(data, opts.integrity, opts)) {
return Promise.reject(checksumError(opts.integrity, sri))
}
return disposer(makeTmp(cache, opts), makeTmpDisposer,
(tmp) => {
return writeFile(tmp.target, data, { flag: 'wx' })
.then(() => moveToDestination(tmp, cache, sri, opts))
})
.then(() => ({ integrity: sri, size: data.length }))
}
module.exports.stream = writeStream
// writes proxied to the 'inputStream' that is passed to the Promise
// 'end' is deferred until content is handled.
class CacacheWriteStream extends Flush {
constructor (cache, opts) {
super()
this.opts = opts
this.cache = cache
this.inputStream = new Minipass()
this.inputStream.on('error', er => this.emit('error', er))
this.inputStream.on('drain', () => this.emit('drain'))
this.handleContentP = null
}
write (chunk, encoding, cb) {
if (!this.handleContentP) {
this.handleContentP = handleContent(
this.inputStream,
this.cache,
this.opts
)
}
return this.inputStream.write(chunk, encoding, cb)
}
flush (cb) {
this.inputStream.end(() => {
if (!this.handleContentP) {
const e = new Error('Cache input stream was empty')
e.code = 'ENODATA'
// empty streams are probably emitting end right away.
// defer this one tick by rejecting a promise on it.
return Promise.reject(e).catch(cb)
}
this.handleContentP.then(
(res) => {
res.integrity && this.emit('integrity', res.integrity)
res.size !== null && this.emit('size', res.size)
cb()
},
(er) => cb(er)
)
})
}
}
function writeStream (cache, opts) {
opts = opts || {}
return new CacacheWriteStream(cache, opts)
}
function handleContent (inputStream, cache, opts) {
return disposer(makeTmp(cache, opts), makeTmpDisposer, (tmp) => {
return pipeToTmp(inputStream, cache, tmp.target, opts)
.then((res) => {
return moveToDestination(
tmp,
cache,
res.integrity,
opts
).then(() => res)
})
})
}
function pipeToTmp (inputStream, cache, tmpTarget, opts) {
let integrity
let size
const hashStream = ssri.integrityStream({
integrity: opts.integrity,
algorithms: opts.algorithms,
size: opts.size
})
hashStream.on('integrity', i => { integrity = i })
hashStream.on('size', s => { size = s })
const outStream = new fsm.WriteStream(tmpTarget, {
flags: 'wx'
})
// NB: this can throw if the hashStream has a problem with
// it, and the data is fully written. but pipeToTmp is only
// called in promisory contexts where that is handled.
const pipeline = new Pipeline(
inputStream,
hashStream,
outStream
)
return pipeline.promise()
.then(() => ({ integrity, size }))
.catch(er => rimraf(tmpTarget).then(() => { throw er }))
}
function makeTmp (cache, opts) {
const tmpTarget = uniqueFilename(path.join(cache, 'tmp'), opts.tmpPrefix)
return fixOwner.mkdirfix(cache, path.dirname(tmpTarget)).then(() => ({
target: tmpTarget,
moved: false
}))
}
function makeTmpDisposer (tmp) {
if (tmp.moved) {
return Promise.resolve()
}
return rimraf(tmp.target)
}
function moveToDestination (tmp, cache, sri, opts) {
const destination = contentPath(cache, sri)
const destDir = path.dirname(destination)
return fixOwner
.mkdirfix(cache, destDir)
.then(() => {
return moveFile(tmp.target, destination)
})
.then(() => {
tmp.moved = true
return fixOwner.chownr(cache, destination)
})
}
function sizeError (expected, found) {
const err = new Error(`Bad data size: expected inserted data to be ${expected} bytes, but got ${found} instead`)
err.expected = expected
err.found = found
err.code = 'EBADSIZE'
return err
}
function checksumError (expected, found) {
const err = new Error(`Integrity check failed:
Wanted: ${expected}
Found: ${found}`)
err.code = 'EINTEGRITY'
err.expected = expected
err.found = found
return err
}
@@ -0,0 +1,311 @@
'use strict'
const util = require('util')
const crypto = require('crypto')
const figgyPudding = require('figgy-pudding')
const fs = require('graceful-fs')
const Minipass = require('minipass')
const path = require('path')
const ssri = require('ssri')
const contentPath = require('./content/path')
const fixOwner = require('./util/fix-owner')
const hashToSegments = require('./util/hash-to-segments')
const indexV = require('../package.json')['cache-version'].index
const appendFile = util.promisify(fs.appendFile)
const readFile = util.promisify(fs.readFile)
const readdir = util.promisify(fs.readdir)
module.exports.NotFoundError = class NotFoundError extends Error {
constructor (cache, key) {
super(`No cache entry for ${key} found in ${cache}`)
this.code = 'ENOENT'
this.cache = cache
this.key = key
}
}
const IndexOpts = figgyPudding({
metadata: {},
size: {}
})
module.exports.insert = insert
function insert (cache, key, integrity, opts) {
opts = IndexOpts(opts)
const bucket = bucketPath(cache, key)
const entry = {
key,
integrity: integrity && ssri.stringify(integrity),
time: Date.now(),
size: opts.size,
metadata: opts.metadata
}
return fixOwner
.mkdirfix(cache, path.dirname(bucket))
.then(() => {
const stringified = JSON.stringify(entry)
// NOTE - Cleverness ahoy!
//
// This works because it's tremendously unlikely for an entry to corrupt
// another while still preserving the string length of the JSON in
// question. So, we just slap the length in there and verify it on read.
//
// Thanks to @isaacs for the whiteboarding session that ended up with this.
return appendFile(bucket, `\n${hashEntry(stringified)}\t${stringified}`)
})
.then(() => fixOwner.chownr(cache, bucket))
.catch((err) => {
if (err.code === 'ENOENT') {
return undefined
}
throw err
// There's a class of race conditions that happen when things get deleted
// during fixOwner, or between the two mkdirfix/chownr calls.
//
// It's perfectly fine to just not bother in those cases and lie
// that the index entry was written. Because it's a cache.
})
.then(() => {
return formatEntry(cache, entry)
})
}
module.exports.insert.sync = insertSync
function insertSync (cache, key, integrity, opts) {
opts = IndexOpts(opts)
const bucket = bucketPath(cache, key)
const entry = {
key,
integrity: integrity && ssri.stringify(integrity),
time: Date.now(),
size: opts.size,
metadata: opts.metadata
}
fixOwner.mkdirfix.sync(cache, path.dirname(bucket))
const stringified = JSON.stringify(entry)
fs.appendFileSync(bucket, `\n${hashEntry(stringified)}\t${stringified}`)
try {
fixOwner.chownr.sync(cache, bucket)
} catch (err) {
if (err.code !== 'ENOENT') {
throw err
}
}
return formatEntry(cache, entry)
}
module.exports.find = find
function find (cache, key) {
const bucket = bucketPath(cache, key)
return bucketEntries(bucket)
.then((entries) => {
return entries.reduce((latest, next) => {
if (next && next.key === key) {
return formatEntry(cache, next)
} else {
return latest
}
}, null)
})
.catch((err) => {
if (err.code === 'ENOENT') {
return null
} else {
throw err
}
})
}
module.exports.find.sync = findSync
function findSync (cache, key) {
const bucket = bucketPath(cache, key)
try {
return bucketEntriesSync(bucket).reduce((latest, next) => {
if (next && next.key === key) {
return formatEntry(cache, next)
} else {
return latest
}
}, null)
} catch (err) {
if (err.code === 'ENOENT') {
return null
} else {
throw err
}
}
}
module.exports.delete = del
function del (cache, key, opts) {
return insert(cache, key, null, opts)
}
module.exports.delete.sync = delSync
function delSync (cache, key, opts) {
return insertSync(cache, key, null, opts)
}
module.exports.lsStream = lsStream
function lsStream (cache) {
const indexDir = bucketDir(cache)
const stream = new Minipass({ objectMode: true })
readdirOrEmpty(indexDir).then(buckets => Promise.all(
buckets.map(bucket => {
const bucketPath = path.join(indexDir, bucket)
return readdirOrEmpty(bucketPath).then(subbuckets => Promise.all(
subbuckets.map(subbucket => {
const subbucketPath = path.join(bucketPath, subbucket)
// "/cachename/<bucket 0xFF>/<bucket 0xFF>./*"
return readdirOrEmpty(subbucketPath).then(entries => Promise.all(
entries.map(entry => {
const entryPath = path.join(subbucketPath, entry)
return bucketEntries(entryPath).then(entries =>
// using a Map here prevents duplicate keys from
// showing up twice, I guess?
entries.reduce((acc, entry) => {
acc.set(entry.key, entry)
return acc
}, new Map())
).then(reduced => {
// reduced is a map of key => entry
for (const entry of reduced.values()) {
const formatted = formatEntry(cache, entry)
if (formatted) {
stream.write(formatted)
}
}
}).catch(err => {
if (err.code === 'ENOENT') { return undefined }
throw err
})
})
))
})
))
})
))
.then(
() => stream.end(),
err => stream.emit('error', err)
)
return stream
}
module.exports.ls = ls
function ls (cache) {
return lsStream(cache).collect().then(entries =>
entries.reduce((acc, xs) => {
acc[xs.key] = xs
return acc
}, {})
)
}
function bucketEntries (bucket, filter) {
return readFile(bucket, 'utf8').then((data) => _bucketEntries(data, filter))
}
function bucketEntriesSync (bucket, filter) {
const data = fs.readFileSync(bucket, 'utf8')
return _bucketEntries(data, filter)
}
function _bucketEntries (data, filter) {
const entries = []
data.split('\n').forEach((entry) => {
if (!entry) {
return
}
const pieces = entry.split('\t')
if (!pieces[1] || hashEntry(pieces[1]) !== pieces[0]) {
// Hash is no good! Corruption or malice? Doesn't matter!
// EJECT EJECT
return
}
let obj
try {
obj = JSON.parse(pieces[1])
} catch (e) {
// Entry is corrupted!
return
}
if (obj) {
entries.push(obj)
}
})
return entries
}
module.exports.bucketDir = bucketDir
function bucketDir (cache) {
return path.join(cache, `index-v${indexV}`)
}
module.exports.bucketPath = bucketPath
function bucketPath (cache, key) {
const hashed = hashKey(key)
return path.join.apply(
path,
[bucketDir(cache)].concat(hashToSegments(hashed))
)
}
module.exports.hashKey = hashKey
function hashKey (key) {
return hash(key, 'sha256')
}
module.exports.hashEntry = hashEntry
function hashEntry (str) {
return hash(str, 'sha1')
}
function hash (str, digest) {
return crypto
.createHash(digest)
.update(str)
.digest('hex')
}
function formatEntry (cache, entry) {
// Treat null digests as deletions. They'll shadow any previous entries.
if (!entry.integrity) {
return null
}
return {
key: entry.key,
integrity: entry.integrity,
path: contentPath(cache, entry.integrity),
size: entry.size,
time: entry.time,
metadata: entry.metadata
}
}
function readdirOrEmpty (dir) {
return readdir(dir).catch((err) => {
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') {
return []
}
throw err
})
}
@@ -0,0 +1,74 @@
'use strict'
const LRU = require('lru-cache')
const MAX_SIZE = 50 * 1024 * 1024 // 50MB
const MAX_AGE = 3 * 60 * 1000
const MEMOIZED = new LRU({
max: MAX_SIZE,
maxAge: MAX_AGE,
length: (entry, key) => key.startsWith('key:') ? entry.data.length : entry.length
})
module.exports.clearMemoized = clearMemoized
function clearMemoized () {
const old = {}
MEMOIZED.forEach((v, k) => {
old[k] = v
})
MEMOIZED.reset()
return old
}
module.exports.put = put
function put (cache, entry, data, opts) {
pickMem(opts).set(`key:${cache}:${entry.key}`, { entry, data })
putDigest(cache, entry.integrity, data, opts)
}
module.exports.put.byDigest = putDigest
function putDigest (cache, integrity, data, opts) {
pickMem(opts).set(`digest:${cache}:${integrity}`, data)
}
module.exports.get = get
function get (cache, key, opts) {
return pickMem(opts).get(`key:${cache}:${key}`)
}
module.exports.get.byDigest = getDigest
function getDigest (cache, integrity, opts) {
return pickMem(opts).get(`digest:${cache}:${integrity}`)
}
class ObjProxy {
constructor (obj) {
this.obj = obj
}
get (key) {
return this.obj[key]
}
set (key, val) {
this.obj[key] = val
}
}
function pickMem (opts) {
if (!opts || !opts.memoize) {
return MEMOIZED
} else if (opts.memoize.get && opts.memoize.set) {
return opts.memoize
} else if (typeof opts.memoize === 'object') {
return new ObjProxy(opts.memoize)
} else {
return MEMOIZED
}
}
@@ -0,0 +1,30 @@
'use strict'
module.exports.disposer = disposer
function disposer (creatorFn, disposerFn, fn) {
const runDisposer = (resource, result, shouldThrow = false) => {
return disposerFn(resource)
.then(
// disposer resolved, do something with original fn's promise
() => {
if (shouldThrow) {
throw result
}
return result
},
// Disposer fn failed, crash process
(err) => {
throw err
// Or process.exit?
})
}
return creatorFn
.then((resource) => {
// fn(resource) can throw, so wrap in a promise here
return Promise.resolve().then(() => fn(resource))
.then((result) => runDisposer(resource, result))
.catch((err) => runDisposer(resource, err, true))
})
}
@@ -0,0 +1,145 @@
'use strict'
const util = require('util')
const chownr = util.promisify(require('chownr'))
const mkdirp = util.promisify(require('mkdirp'))
const inflight = require('promise-inflight')
const inferOwner = require('infer-owner')
// Memoize getuid()/getgid() calls.
// patch process.setuid/setgid to invalidate cached value on change
const self = { uid: null, gid: null }
const getSelf = () => {
if (typeof self.uid !== 'number') {
self.uid = process.getuid()
const setuid = process.setuid
process.setuid = (uid) => {
self.uid = null
process.setuid = setuid
return process.setuid(uid)
}
}
if (typeof self.gid !== 'number') {
self.gid = process.getgid()
const setgid = process.setgid
process.setgid = (gid) => {
self.gid = null
process.setgid = setgid
return process.setgid(gid)
}
}
}
module.exports.chownr = fixOwner
function fixOwner (cache, filepath) {
if (!process.getuid) {
// This platform doesn't need ownership fixing
return Promise.resolve()
}
getSelf()
if (self.uid !== 0) {
// almost certainly can't chown anyway
return Promise.resolve()
}
return Promise.resolve(inferOwner(cache)).then((owner) => {
const { uid, gid } = owner
// No need to override if it's already what we used.
if (self.uid === uid && self.gid === gid) {
return
}
return inflight('fixOwner: fixing ownership on ' + filepath, () =>
chownr(
filepath,
typeof uid === 'number' ? uid : self.uid,
typeof gid === 'number' ? gid : self.gid
).catch((err) => {
if (err.code === 'ENOENT') {
return null
}
throw err
})
)
})
}
module.exports.chownr.sync = fixOwnerSync
function fixOwnerSync (cache, filepath) {
if (!process.getuid) {
// This platform doesn't need ownership fixing
return
}
const { uid, gid } = inferOwner.sync(cache)
getSelf()
if (self.uid !== 0) {
// almost certainly can't chown anyway
return
}
if (self.uid === uid && self.gid === gid) {
// No need to override if it's already what we used.
return
}
try {
chownr.sync(
filepath,
typeof uid === 'number' ? uid : self.uid,
typeof gid === 'number' ? gid : self.gid
)
} catch (err) {
// only catch ENOENT, any other error is a problem.
if (err.code === 'ENOENT') {
return null
}
throw err
}
}
module.exports.mkdirfix = mkdirfix
function mkdirfix (cache, p, cb) {
// we have to infer the owner _before_ making the directory, even though
// we aren't going to use the results, since the cache itself might not
// exist yet. If we mkdirp it, then our current uid/gid will be assumed
// to be correct if it creates the cache folder in the process.
return Promise.resolve(inferOwner(cache)).then(() => {
return mkdirp(p)
.then((made) => {
if (made) {
return fixOwner(cache, made).then(() => made)
}
})
.catch((err) => {
if (err.code === 'EEXIST') {
return fixOwner(cache, p).then(() => null)
}
throw err
})
})
}
module.exports.mkdirfix.sync = mkdirfixSync
function mkdirfixSync (cache, p) {
try {
inferOwner.sync(cache)
const made = mkdirp.sync(p)
if (made) {
fixOwnerSync(cache, made)
return made
}
} catch (err) {
if (err.code === 'EEXIST') {
fixOwnerSync(cache, p)
return null
} else {
throw err
}
}
}
@@ -0,0 +1,7 @@
'use strict'
module.exports = hashToSegments
function hashToSegments (hash) {
return [hash.slice(0, 2), hash.slice(2, 4), hash.slice(4)]
}
@@ -0,0 +1,56 @@
'use strict'
const fs = require('graceful-fs')
const util = require('util')
const chmod = util.promisify(fs.chmod)
const unlink = util.promisify(fs.unlink)
const stat = util.promisify(fs.stat)
const move = require('move-concurrently')
const pinflight = require('promise-inflight')
module.exports = moveFile
function moveFile (src, dest) {
// This isn't quite an fs.rename -- the assumption is that
// if `dest` already exists, and we get certain errors while
// trying to move it, we should just not bother.
//
// In the case of cache corruption, users will receive an
// EINTEGRITY error elsewhere, and can remove the offending
// content their own way.
//
// Note that, as the name suggests, this strictly only supports file moves.
return new Promise((resolve, reject) => {
fs.link(src, dest, (err) => {
if (err) {
if (err.code === 'EEXIST' || err.code === 'EBUSY') {
// file already exists, so whatever
} else if (err.code === 'EPERM' && process.platform === 'win32') {
// file handle stayed open even past graceful-fs limits
} else {
return reject(err)
}
}
return resolve()
})
})
.then(() => {
// content should never change for any reason, so make it read-only
return Promise.all([
unlink(src),
process.platform !== 'win32' && chmod(dest, '0444')
])
})
.catch(() => {
return pinflight('cacache-move-file:' + dest, () => {
return stat(dest).catch((err) => {
if (err.code !== 'ENOENT') {
// Something else is wrong here. Bail bail bail
throw err
}
// file doesn't already exist! let's try a rename -> copy fallback
return move(src, dest, { Promise, fs })
})
})
})
}
@@ -0,0 +1,42 @@
'use strict'
const util = require('util')
const figgyPudding = require('figgy-pudding')
const fixOwner = require('./fix-owner')
const path = require('path')
const rimraf = util.promisify(require('rimraf'))
const uniqueFilename = require('unique-filename')
const { disposer } = require('./disposer')
const TmpOpts = figgyPudding({
tmpPrefix: {}
})
module.exports.mkdir = mktmpdir
function mktmpdir (cache, opts) {
opts = TmpOpts(opts)
const tmpTarget = uniqueFilename(path.join(cache, 'tmp'), opts.tmpPrefix)
return fixOwner.mkdirfix(cache, tmpTarget).then(() => {
return tmpTarget
})
}
module.exports.withTmp = withTmp
function withTmp (cache, opts, cb) {
if (!cb) {
cb = opts
opts = null
}
opts = TmpOpts(opts)
return disposer(mktmpdir(cache, opts), rimraf, cb)
}
module.exports.fix = fixtmpdir
function fixtmpdir (cache) {
return fixOwner(cache, path.join(cache, 'tmp'))
}
@@ -0,0 +1,291 @@
'use strict'
const util = require('util')
const pMap = require('p-map')
const contentPath = require('./content/path')
const figgyPudding = require('figgy-pudding')
const fixOwner = require('./util/fix-owner')
const fs = require('graceful-fs')
const fsm = require('fs-minipass')
const glob = util.promisify(require('glob'))
const index = require('./entry-index')
const path = require('path')
const rimraf = util.promisify(require('rimraf'))
const ssri = require('ssri')
const hasOwnProperty = (obj, key) =>
Object.prototype.hasOwnProperty.call(obj, key)
const stat = util.promisify(fs.stat)
const truncate = util.promisify(fs.truncate)
const writeFile = util.promisify(fs.writeFile)
const readFile = util.promisify(fs.readFile)
const VerifyOpts = figgyPudding({
concurrency: {
default: 20
},
filter: {},
log: {
default: { silly () {} }
}
})
module.exports = verify
function verify (cache, opts) {
opts = VerifyOpts(opts)
opts.log.silly('verify', 'verifying cache at', cache)
const steps = [
markStartTime,
fixPerms,
garbageCollect,
rebuildIndex,
cleanTmp,
writeVerifile,
markEndTime
]
return steps
.reduce((promise, step, i) => {
const label = step.name || `step #${i}`
const start = new Date()
return promise.then((stats) => {
return step(cache, opts).then((s) => {
s &&
Object.keys(s).forEach((k) => {
stats[k] = s[k]
})
const end = new Date()
if (!stats.runTime) {
stats.runTime = {}
}
stats.runTime[label] = end - start
return Promise.resolve(stats)
})
})
}, Promise.resolve({}))
.then((stats) => {
stats.runTime.total = stats.endTime - stats.startTime
opts.log.silly(
'verify',
'verification finished for',
cache,
'in',
`${stats.runTime.total}ms`
)
return stats
})
}
function markStartTime (cache, opts) {
return Promise.resolve({ startTime: new Date() })
}
function markEndTime (cache, opts) {
return Promise.resolve({ endTime: new Date() })
}
function fixPerms (cache, opts) {
opts.log.silly('verify', 'fixing cache permissions')
return fixOwner
.mkdirfix(cache, cache)
.then(() => {
// TODO - fix file permissions too
return fixOwner.chownr(cache, cache)
})
.then(() => null)
}
// Implements a naive mark-and-sweep tracing garbage collector.
//
// The algorithm is basically as follows:
// 1. Read (and filter) all index entries ("pointers")
// 2. Mark each integrity value as "live"
// 3. Read entire filesystem tree in `content-vX/` dir
// 4. If content is live, verify its checksum and delete it if it fails
// 5. If content is not marked as live, rimraf it.
//
function garbageCollect (cache, opts) {
opts.log.silly('verify', 'garbage collecting content')
const indexStream = index.lsStream(cache)
const liveContent = new Set()
indexStream.on('data', (entry) => {
if (opts.filter && !opts.filter(entry)) {
return
}
liveContent.add(entry.integrity.toString())
})
return new Promise((resolve, reject) => {
indexStream.on('end', resolve).on('error', reject)
}).then(() => {
const contentDir = contentPath.contentDir(cache)
return glob(path.join(contentDir, '**'), {
follow: false,
nodir: true,
nosort: true
}).then((files) => {
return Promise.resolve({
verifiedContent: 0,
reclaimedCount: 0,
reclaimedSize: 0,
badContentCount: 0,
keptSize: 0
}).then((stats) =>
pMap(
files,
(f) => {
const split = f.split(/[/\\]/)
const digest = split.slice(split.length - 3).join('')
const algo = split[split.length - 4]
const integrity = ssri.fromHex(digest, algo)
if (liveContent.has(integrity.toString())) {
return verifyContent(f, integrity).then((info) => {
if (!info.valid) {
stats.reclaimedCount++
stats.badContentCount++
stats.reclaimedSize += info.size
} else {
stats.verifiedContent++
stats.keptSize += info.size
}
return stats
})
} else {
// No entries refer to this content. We can delete.
stats.reclaimedCount++
return stat(f).then((s) => {
return rimraf(f).then(() => {
stats.reclaimedSize += s.size
return stats
})
})
}
},
{ concurrency: opts.concurrency }
).then(() => stats)
)
})
})
}
function verifyContent (filepath, sri) {
return stat(filepath)
.then((s) => {
const contentInfo = {
size: s.size,
valid: true
}
return ssri
.checkStream(new fsm.ReadStream(filepath), sri)
.catch((err) => {
if (err.code !== 'EINTEGRITY') {
throw err
}
return rimraf(filepath).then(() => {
contentInfo.valid = false
})
})
.then(() => contentInfo)
})
.catch((err) => {
if (err.code === 'ENOENT') {
return { size: 0, valid: false }
}
throw err
})
}
function rebuildIndex (cache, opts) {
opts.log.silly('verify', 'rebuilding index')
return index.ls(cache).then((entries) => {
const stats = {
missingContent: 0,
rejectedEntries: 0,
totalEntries: 0
}
const buckets = {}
for (const k in entries) {
if (hasOwnProperty(entries, k)) {
const hashed = index.hashKey(k)
const entry = entries[k]
const excluded = opts.filter && !opts.filter(entry)
excluded && stats.rejectedEntries++
if (buckets[hashed] && !excluded) {
buckets[hashed].push(entry)
} else if (buckets[hashed] && excluded) {
// skip
} else if (excluded) {
buckets[hashed] = []
buckets[hashed]._path = index.bucketPath(cache, k)
} else {
buckets[hashed] = [entry]
buckets[hashed]._path = index.bucketPath(cache, k)
}
}
}
return pMap(
Object.keys(buckets),
(key) => {
return rebuildBucket(cache, buckets[key], stats, opts)
},
{ concurrency: opts.concurrency }
).then(() => stats)
})
}
function rebuildBucket (cache, bucket, stats, opts) {
return truncate(bucket._path).then(() => {
// This needs to be serialized because cacache explicitly
// lets very racy bucket conflicts clobber each other.
return bucket.reduce((promise, entry) => {
return promise.then(() => {
const content = contentPath(cache, entry.integrity)
return stat(content)
.then(() => {
return index
.insert(cache, entry.key, entry.integrity, {
metadata: entry.metadata,
size: entry.size
})
.then(() => {
stats.totalEntries++
})
})
.catch((err) => {
if (err.code === 'ENOENT') {
stats.rejectedEntries++
stats.missingContent++
return
}
throw err
})
})
}, Promise.resolve())
})
}
function cleanTmp (cache, opts) {
opts.log.silly('verify', 'cleaning tmp directory')
return rimraf(path.join(cache, 'tmp'))
}
function writeVerifile (cache, opts) {
const verifile = path.join(cache, '_lastverified')
opts.log.silly('verify', 'writing verifile to ' + verifile)
try {
return writeFile(verifile, '' + +new Date())
} finally {
fixOwner.chownr.sync(cache, verifile)
}
}
module.exports.lastRun = lastRun
function lastRun (cache) {
return readFile(path.join(cache, '_lastverified'), 'utf8').then(
(data) => new Date(+data)
)
}
@@ -0,0 +1,6 @@
'use strict'
const index = require('./lib/entry-index')
module.exports = index.ls
module.exports.stream = index.lsStream
@@ -0,0 +1,95 @@
{
"name": "cacache",
"version": "13.0.1",
"cache-version": {
"content": "2",
"index": "5"
},
"description": "Fast, fault-tolerant, cross-platform, disk-based, data-agnostic, content-addressable cache.",
"main": "index.js",
"files": [
"*.js",
"lib",
"locales"
],
"scripts": {
"benchmarks": "node test/benchmarks",
"prerelease": "npm t",
"postrelease": "npm publish && git push --follow-tags",
"pretest": "standard",
"release": "standard-version -s",
"test": "tap test/*.js",
"test-docker": "docker run -it --rm --name pacotest -v \"$PWD\":/tmp -w /tmp node:latest npm test",
"update-coc": "weallbehave -o . && git add CODE_OF_CONDUCT.md && git commit -m 'docs(coc): updated CODE_OF_CONDUCT.md'",
"update-contrib": "weallcontribute -o . && git add CONTRIBUTING.md && git commit -m 'docs(contributing): updated CONTRIBUTING.md'"
},
"repository": "https://github.com/npm/cacache",
"keywords": [
"cache",
"caching",
"content-addressable",
"sri",
"sri hash",
"subresource integrity",
"cache",
"storage",
"store",
"file store",
"filesystem",
"disk cache",
"disk storage"
],
"author": {
"name": "Kat Marchán",
"email": "kzm@sykosomatic.org",
"twitter": "maybekatz"
},
"contributors": [
{
"name": "Charlotte Spencer",
"email": "charlottelaspencer@gmail.com",
"twitter": "charlotteis"
},
{
"name": "Rebecca Turner",
"email": "me@re-becca.org",
"twitter": "ReBeccaOrg"
}
],
"license": "ISC",
"dependencies": {
"chownr": "^1.1.2",
"figgy-pudding": "^3.5.1",
"fs-minipass": "^2.0.0",
"glob": "^7.1.4",
"graceful-fs": "^4.2.2",
"infer-owner": "^1.0.4",
"lru-cache": "^5.1.1",
"minipass": "^3.0.0",
"minipass-collect": "^1.0.2",
"minipass-flush": "^1.0.5",
"minipass-pipeline": "^1.2.2",
"mkdirp": "^0.5.1",
"move-concurrently": "^1.0.1",
"p-map": "^3.0.0",
"promise-inflight": "^1.0.1",
"rimraf": "^2.7.1",
"ssri": "^7.0.0",
"unique-filename": "^1.1.1"
},
"devDependencies": {
"benchmark": "^2.1.4",
"chalk": "^2.4.2",
"cross-env": "^5.2.1",
"require-inject": "^1.4.4",
"standard": "^14.3.0",
"standard-version": "^7.0.0",
"tacks": "^1.3.0",
"tap": "^14.6.9",
"weallbehave": "^1.2.0",
"weallcontribute": "^1.0.9"
},
"engines": {
"node": ">= 8"
}
}
@@ -0,0 +1,94 @@
'use strict'
const figgyPudding = require('figgy-pudding')
const index = require('./lib/entry-index')
const memo = require('./lib/memoization')
const write = require('./lib/content/write')
const Flush = require('minipass-flush')
const { PassThrough } = require('minipass-collect')
const Pipeline = require('minipass-pipeline')
const PutOpts = figgyPudding({
algorithms: {
default: ['sha512']
},
integrity: {},
memoize: {},
metadata: {},
pickAlgorithm: {},
size: {},
tmpPrefix: {},
single: {},
sep: {},
error: {},
strict: {}
})
module.exports = putData
function putData (cache, key, data, opts) {
opts = PutOpts(opts)
return write(cache, data, opts).then((res) => {
return index
.insert(cache, key, res.integrity, opts.concat({ size: res.size }))
.then((entry) => {
if (opts.memoize) {
memo.put(cache, entry, data, opts)
}
return res.integrity
})
})
}
module.exports.stream = putStream
function putStream (cache, key, opts) {
opts = PutOpts(opts)
let integrity
let size
let memoData
const pipeline = new Pipeline()
// first item in the pipeline is the memoizer, because we need
// that to end first and get the collected data.
if (opts.memoize) {
const memoizer = new PassThrough().on('collect', data => {
memoData = data
})
pipeline.push(memoizer)
}
// contentStream is a write-only, not a passthrough
// no data comes out of it.
const contentStream = write.stream(cache, opts)
.on('integrity', (int) => {
integrity = int
})
.on('size', (s) => {
size = s
})
pipeline.push(contentStream)
// last but not least, we write the index and emit hash and size,
// and memoize if we're doing that
pipeline.push(new Flush({
flush () {
return index
.insert(cache, key, integrity, opts.concat({ size }))
.then((entry) => {
if (opts.memoize && memoData) {
memo.put(cache, entry, memoData, opts)
}
if (integrity) {
pipeline.emit('integrity', integrity)
}
if (size) {
pipeline.emit('size', size)
}
})
}
}))
return pipeline
}
+31
View File
@@ -0,0 +1,31 @@
'use strict'
const util = require('util')
const index = require('./lib/entry-index')
const memo = require('./lib/memoization')
const path = require('path')
const rimraf = util.promisify(require('rimraf'))
const rmContent = require('./lib/content/rm')
module.exports = entry
module.exports.entry = entry
function entry (cache, key) {
memo.clearMemoized()
return index.delete(cache, key)
}
module.exports.content = content
function content (cache, integrity) {
memo.clearMemoized()
return rmContent(cache, integrity)
}
module.exports.all = all
function all (cache) {
memo.clearMemoized()
return rimraf(path.join(cache, '*(content-*|index-*)'))
}
@@ -0,0 +1,3 @@
'use strict'
module.exports = require('./lib/verify')
@@ -0,0 +1,15 @@
The ISC License
Copyright (c) Isaac Z. Schlueter and Contributors
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
@@ -0,0 +1,3 @@
Like `chown -R`.
Takes the same arguments as `fs.chown()`
@@ -0,0 +1,167 @@
'use strict'
const fs = require('fs')
const path = require('path')
/* istanbul ignore next */
const LCHOWN = fs.lchown ? 'lchown' : 'chown'
/* istanbul ignore next */
const LCHOWNSYNC = fs.lchownSync ? 'lchownSync' : 'chownSync'
/* istanbul ignore next */
const needEISDIRHandled = fs.lchown &&
!process.version.match(/v1[1-9]+\./) &&
!process.version.match(/v10\.[6-9]/)
const lchownSync = (path, uid, gid) => {
try {
return fs[LCHOWNSYNC](path, uid, gid)
} catch (er) {
if (er.code !== 'ENOENT')
throw er
}
}
/* istanbul ignore next */
const chownSync = (path, uid, gid) => {
try {
return fs.chownSync(path, uid, gid)
} catch (er) {
if (er.code !== 'ENOENT')
throw er
}
}
/* istanbul ignore next */
const handleEISDIR =
needEISDIRHandled ? (path, uid, gid, cb) => er => {
// Node prior to v10 had a very questionable implementation of
// fs.lchown, which would always try to call fs.open on a directory
// Fall back to fs.chown in those cases.
if (!er || er.code !== 'EISDIR')
cb(er)
else
fs.chown(path, uid, gid, cb)
}
: (_, __, ___, cb) => cb
/* istanbul ignore next */
const handleEISDirSync =
needEISDIRHandled ? (path, uid, gid) => {
try {
return lchownSync(path, uid, gid)
} catch (er) {
if (er.code !== 'EISDIR')
throw er
chownSync(path, uid, gid)
}
}
: (path, uid, gid) => lchownSync(path, uid, gid)
// fs.readdir could only accept an options object as of node v6
const nodeVersion = process.version
let readdir = (path, options, cb) => fs.readdir(path, options, cb)
let readdirSync = (path, options) => fs.readdirSync(path, options)
/* istanbul ignore next */
if (/^v4\./.test(nodeVersion))
readdir = (path, options, cb) => fs.readdir(path, cb)
const chown = (cpath, uid, gid, cb) => {
fs[LCHOWN](cpath, uid, gid, handleEISDIR(cpath, uid, gid, er => {
// Skip ENOENT error
cb(er && er.code !== 'ENOENT' ? er : null)
}))
}
const chownrKid = (p, child, uid, gid, cb) => {
if (typeof child === 'string')
return fs.lstat(path.resolve(p, child), (er, stats) => {
// Skip ENOENT error
if (er)
return cb(er.code !== 'ENOENT' ? er : null)
stats.name = child
chownrKid(p, stats, uid, gid, cb)
})
if (child.isDirectory()) {
chownr(path.resolve(p, child.name), uid, gid, er => {
if (er)
return cb(er)
const cpath = path.resolve(p, child.name)
chown(cpath, uid, gid, cb)
})
} else {
const cpath = path.resolve(p, child.name)
chown(cpath, uid, gid, cb)
}
}
const chownr = (p, uid, gid, cb) => {
readdir(p, { withFileTypes: true }, (er, children) => {
// any error other than ENOTDIR or ENOTSUP means it's not readable,
// or doesn't exist. give up.
if (er) {
if (er.code === 'ENOENT')
return cb()
else if (er.code !== 'ENOTDIR' && er.code !== 'ENOTSUP')
return cb(er)
}
if (er || !children.length)
return chown(p, uid, gid, cb)
let len = children.length
let errState = null
const then = er => {
if (errState)
return
if (er)
return cb(errState = er)
if (-- len === 0)
return chown(p, uid, gid, cb)
}
children.forEach(child => chownrKid(p, child, uid, gid, then))
})
}
const chownrKidSync = (p, child, uid, gid) => {
if (typeof child === 'string') {
try {
const stats = fs.lstatSync(path.resolve(p, child))
stats.name = child
child = stats
} catch (er) {
if (er.code === 'ENOENT')
return
else
throw er
}
}
if (child.isDirectory())
chownrSync(path.resolve(p, child.name), uid, gid)
handleEISDirSync(path.resolve(p, child.name), uid, gid)
}
const chownrSync = (p, uid, gid) => {
let children
try {
children = readdirSync(p, { withFileTypes: true })
} catch (er) {
if (er.code === 'ENOENT')
return
else if (er.code === 'ENOTDIR' || er.code === 'ENOTSUP')
return handleEISDirSync(p, uid, gid)
else
throw er
}
if (children && children.length)
children.forEach(child => chownrKidSync(p, child, uid, gid))
return handleEISDirSync(p, uid, gid)
}
module.exports = chownr
chownr.sync = chownrSync
@@ -0,0 +1,29 @@
{
"author": "Isaac Z. Schlueter <i@izs.me> (http://blog.izs.me/)",
"name": "chownr",
"description": "like `chown -R`",
"version": "1.1.4",
"repository": {
"type": "git",
"url": "git://github.com/isaacs/chownr.git"
},
"main": "chownr.js",
"files": [
"chownr.js"
],
"devDependencies": {
"mkdirp": "0.3",
"rimraf": "^2.7.1",
"tap": "^14.10.6"
},
"tap": {
"check-coverage": true
},
"scripts": {
"test": "tap",
"preversion": "npm test",
"postversion": "npm publish",
"prepublishOnly": "git push origin --follow-tags"
},
"license": "ISC"
}
@@ -0,0 +1,65 @@
declare namespace pMap {
interface Options {
/**
Number of concurrently pending promises returned by `mapper`.
@default Infinity
*/
readonly concurrency?: number;
/**
When set to `false`, instead of stopping when a promise rejects, it will wait for all the promises to settle and then reject with an [aggregated error](https://github.com/sindresorhus/aggregate-error) containing all the errors from the rejected promises.
@default true
*/
readonly stopOnError?: boolean;
}
/**
Function which is called for every item in `input`. Expected to return a `Promise` or value.
@param element - Iterated element.
@param index - Index of the element in the source array.
*/
type Mapper<Element = any, NewElement = any> = (
element: Element,
index: number
) => NewElement | Promise<NewElement>;
}
/**
@param input - Iterated over concurrently in the `mapper` function.
@param mapper - Function which is called for every item in `input`. Expected to return a `Promise` or value.
@returns A `Promise` that is fulfilled when all promises in `input` and ones returned from `mapper` are fulfilled, or rejects if any of the promises reject. The fulfilled value is an `Array` of the fulfilled values returned from `mapper` in `input` order.
@example
```
import pMap = require('p-map');
import got = require('got');
const sites = [
getWebsiteFromUsername('https://sindresorhus'), //=> Promise
'https://ava.li',
'https://github.com'
];
(async () => {
const mapper = async site => {
const {requestUrl} = await got.head(site);
return requestUrl;
};
const result = await pMap(sites, mapper, {concurrency: 2});
console.log(result);
//=> ['https://sindresorhus.com/', 'https://ava.li/', 'https://github.com/']
})();
```
*/
declare function pMap<Element, NewElement>(
input: Iterable<Element>,
mapper: pMap.Mapper<Element, NewElement>,
options?: pMap.Options
): Promise<NewElement[]>;
export = pMap;
@@ -0,0 +1,81 @@
'use strict';
const AggregateError = require('aggregate-error');
module.exports = async (
iterable,
mapper,
{
concurrency = Infinity,
stopOnError = true
} = {}
) => {
return new Promise((resolve, reject) => {
if (typeof mapper !== 'function') {
throw new TypeError('Mapper function is required');
}
if (!(typeof concurrency === 'number' && concurrency >= 1)) {
throw new TypeError(`Expected \`concurrency\` to be a number from 1 and up, got \`${concurrency}\` (${typeof concurrency})`);
}
const ret = [];
const errors = [];
const iterator = iterable[Symbol.iterator]();
let isRejected = false;
let isIterableDone = false;
let resolvingCount = 0;
let currentIndex = 0;
const next = () => {
if (isRejected) {
return;
}
const nextItem = iterator.next();
const i = currentIndex;
currentIndex++;
if (nextItem.done) {
isIterableDone = true;
if (resolvingCount === 0) {
if (!stopOnError && errors.length !== 0) {
reject(new AggregateError(errors));
} else {
resolve(ret);
}
}
return;
}
resolvingCount++;
(async () => {
try {
const element = await nextItem.value;
ret[i] = await mapper(element, i);
resolvingCount--;
next();
} catch (error) {
if (stopOnError) {
isRejected = true;
reject(error);
} else {
errors.push(error);
resolvingCount--;
next();
}
}
})();
};
for (let i = 0; i < concurrency; i++) {
next();
if (isIterableDone) {
break;
}
}
});
};
@@ -0,0 +1,9 @@
MIT License
Copyright (c) Sindre Sorhus <sindresorhus@gmail.com> (sindresorhus.com)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,52 @@
{
"name": "p-map",
"version": "3.0.0",
"description": "Map over promises concurrently",
"license": "MIT",
"repository": "sindresorhus/p-map",
"author": {
"name": "Sindre Sorhus",
"email": "sindresorhus@gmail.com",
"url": "sindresorhus.com"
},
"engines": {
"node": ">=8"
},
"scripts": {
"test": "xo && ava && tsd"
},
"files": [
"index.js",
"index.d.ts"
],
"keywords": [
"promise",
"map",
"resolved",
"wait",
"collection",
"iterable",
"iterator",
"race",
"fulfilled",
"async",
"await",
"promises",
"concurrently",
"concurrency",
"parallel",
"bluebird"
],
"dependencies": {
"aggregate-error": "^3.0.0"
},
"devDependencies": {
"ava": "^2.2.0",
"delay": "^4.1.0",
"in-range": "^2.0.0",
"random-int": "^2.0.0",
"time-span": "^3.1.0",
"tsd": "^0.7.2",
"xo": "^0.24.0"
}
}
@@ -0,0 +1,99 @@
# p-map [![Build Status](https://travis-ci.org/sindresorhus/p-map.svg?branch=master)](https://travis-ci.org/sindresorhus/p-map)
> Map over promises concurrently
Useful when you need to run promise-returning & async functions multiple times with different inputs concurrently.
## Install
```
$ npm install p-map
```
## Usage
```js
const pMap = require('p-map');
const got = require('got');
const sites = [
getWebsiteFromUsername('https://sindresorhus'), //=> Promise
'https://ava.li',
'https://github.com'
];
(async () => {
const mapper = async site => {
const {requestUrl} = await got.head(site);
return requestUrl;
};
const result = await pMap(sites, mapper, {concurrency: 2});
console.log(result);
//=> ['https://sindresorhus.com/', 'https://ava.li/', 'https://github.com/']
})();
```
## API
### pMap(input, mapper, options?)
Returns a `Promise` that is fulfilled when all promises in `input` and ones returned from `mapper` are fulfilled, or rejects if any of the promises reject. The fulfilled value is an `Array` of the fulfilled values returned from `mapper` in `input` order.
#### input
Type: `Iterable<Promise | unknown>`
Iterated over concurrently in the `mapper` function.
#### mapper(element, index)
Type: `Function`
Expected to return a `Promise` or value.
#### options
Type: `object`
##### concurrency
Type: `number`<br>
Default: `Infinity`<br>
Minimum: `1`
Number of concurrently pending promises returned by `mapper`.
##### stopOnError
Type: `boolean`<br>
Default: `true`
When set to `false`, instead of stopping when a promise rejects, it will wait for all the promises to settle and then reject with an [aggregated error](https://github.com/sindresorhus/aggregate-error) containing all the errors from the rejected promises.
## Related
- [p-all](https://github.com/sindresorhus/p-all) - Run promise-returning & async functions concurrently with optional limited concurrency
- [p-filter](https://github.com/sindresorhus/p-filter) - Filter promises concurrently
- [p-times](https://github.com/sindresorhus/p-times) - Run promise-returning & async functions a specific number of times concurrently
- [p-props](https://github.com/sindresorhus/p-props) - Like `Promise.all()` but for `Map` and `Object`
- [p-map-series](https://github.com/sindresorhus/p-map-series) - Map over promises serially
- [p-queue](https://github.com/sindresorhus/p-queue) - Promise queue with concurrency control
- [More…](https://github.com/sindresorhus/promise-fun)
---
<div align="center">
<b>
<a href="https://tidelift.com/subscription/pkg/npm-p-map?utm_source=npm-p-map&utm_medium=referral&utm_campaign=readme">Get professional support for this package with a Tidelift subscription</a>
</b>
<br>
<sub>
Tidelift helps make open source sustainable for maintainers while giving companies<br>assurances about security, maintenance, and licensing for their dependencies.
</sub>
</div>
@@ -0,0 +1,15 @@
The ISC License
Copyright (c) Isaac Z. Schlueter and Contributors
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
@@ -0,0 +1,101 @@
[![Build Status](https://travis-ci.org/isaacs/rimraf.svg?branch=master)](https://travis-ci.org/isaacs/rimraf) [![Dependency Status](https://david-dm.org/isaacs/rimraf.svg)](https://david-dm.org/isaacs/rimraf) [![devDependency Status](https://david-dm.org/isaacs/rimraf/dev-status.svg)](https://david-dm.org/isaacs/rimraf#info=devDependencies)
The [UNIX command](http://en.wikipedia.org/wiki/Rm_(Unix)) `rm -rf` for node.
Install with `npm install rimraf`, or just drop rimraf.js somewhere.
## API
`rimraf(f, [opts], callback)`
The first parameter will be interpreted as a globbing pattern for files. If you
want to disable globbing you can do so with `opts.disableGlob` (defaults to
`false`). This might be handy, for instance, if you have filenames that contain
globbing wildcard characters.
The callback will be called with an error if there is one. Certain
errors are handled for you:
* Windows: `EBUSY` and `ENOTEMPTY` - rimraf will back off a maximum of
`opts.maxBusyTries` times before giving up, adding 100ms of wait
between each attempt. The default `maxBusyTries` is 3.
* `ENOENT` - If the file doesn't exist, rimraf will return
successfully, since your desired outcome is already the case.
* `EMFILE` - Since `readdir` requires opening a file descriptor, it's
possible to hit `EMFILE` if too many file descriptors are in use.
In the sync case, there's nothing to be done for this. But in the
async case, rimraf will gradually back off with timeouts up to
`opts.emfileWait` ms, which defaults to 1000.
## options
* unlink, chmod, stat, lstat, rmdir, readdir,
unlinkSync, chmodSync, statSync, lstatSync, rmdirSync, readdirSync
In order to use a custom file system library, you can override
specific fs functions on the options object.
If any of these functions are present on the options object, then
the supplied function will be used instead of the default fs
method.
Sync methods are only relevant for `rimraf.sync()`, of course.
For example:
```javascript
var myCustomFS = require('some-custom-fs')
rimraf('some-thing', myCustomFS, callback)
```
* maxBusyTries
If an `EBUSY`, `ENOTEMPTY`, or `EPERM` error code is encountered
on Windows systems, then rimraf will retry with a linear backoff
wait of 100ms longer on each try. The default maxBusyTries is 3.
Only relevant for async usage.
* emfileWait
If an `EMFILE` error is encountered, then rimraf will retry
repeatedly with a linear backoff of 1ms longer on each try, until
the timeout counter hits this max. The default limit is 1000.
If you repeatedly encounter `EMFILE` errors, then consider using
[graceful-fs](http://npm.im/graceful-fs) in your program.
Only relevant for async usage.
* glob
Set to `false` to disable [glob](http://npm.im/glob) pattern
matching.
Set to an object to pass options to the glob module. The default
glob options are `{ nosort: true, silent: true }`.
Glob version 6 is used in this module.
Relevant for both sync and async usage.
* disableGlob
Set to any non-falsey value to disable globbing entirely.
(Equivalent to setting `glob: false`.)
## rimraf.sync
It can remove stuff synchronously, too. But that's not so good. Use
the async API. It's better.
## CLI
If installed with `npm install rimraf -g` it can be used as a global
command `rimraf <path> [<path> ...]` which is useful for cross platform support.
## mkdirp
If you need to create a directory recursively, check out
[mkdirp](https://github.com/substack/node-mkdirp).
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env node
var rimraf = require('./')
var help = false
var dashdash = false
var noglob = false
var args = process.argv.slice(2).filter(function(arg) {
if (dashdash)
return !!arg
else if (arg === '--')
dashdash = true
else if (arg === '--no-glob' || arg === '-G')
noglob = true
else if (arg === '--glob' || arg === '-g')
noglob = false
else if (arg.match(/^(-+|\/)(h(elp)?|\?)$/))
help = true
else
return !!arg
})
if (help || args.length === 0) {
// If they didn't ask for help, then this is not a "success"
var log = help ? console.log : console.error
log('Usage: rimraf <path> [<path> ...]')
log('')
log(' Deletes all files and folders at "path" recursively.')
log('')
log('Options:')
log('')
log(' -h, --help Display this usage info')
log(' -G, --no-glob Do not expand glob patterns in arguments')
log(' -g, --glob Expand glob patterns in arguments (default)')
process.exit(help ? 0 : 1)
} else
go(0)
function go (n) {
if (n >= args.length)
return
var options = {}
if (noglob)
options = { glob: false }
rimraf(args[n], options, function (er) {
if (er)
throw er
go(n+1)
})
}
@@ -0,0 +1,29 @@
{
"name": "rimraf",
"version": "2.7.1",
"main": "rimraf.js",
"description": "A deep deletion module for node (like `rm -rf`)",
"author": "Isaac Z. Schlueter <i@izs.me> (http://blog.izs.me/)",
"license": "ISC",
"repository": "git://github.com/isaacs/rimraf.git",
"scripts": {
"preversion": "npm test",
"postversion": "npm publish",
"postpublish": "git push origin --all; git push origin --tags",
"test": "tap test/*.js"
},
"bin": "./bin.js",
"dependencies": {
"glob": "^7.1.3"
},
"files": [
"LICENSE",
"README.md",
"bin.js",
"rimraf.js"
],
"devDependencies": {
"mkdirp": "^0.5.1",
"tap": "^12.1.1"
}
}
@@ -0,0 +1,372 @@
module.exports = rimraf
rimraf.sync = rimrafSync
var assert = require("assert")
var path = require("path")
var fs = require("fs")
var glob = undefined
try {
glob = require("glob")
} catch (_err) {
// treat glob as optional.
}
var _0666 = parseInt('666', 8)
var defaultGlobOpts = {
nosort: true,
silent: true
}
// for EMFILE handling
var timeout = 0
var isWindows = (process.platform === "win32")
function defaults (options) {
var methods = [
'unlink',
'chmod',
'stat',
'lstat',
'rmdir',
'readdir'
]
methods.forEach(function(m) {
options[m] = options[m] || fs[m]
m = m + 'Sync'
options[m] = options[m] || fs[m]
})
options.maxBusyTries = options.maxBusyTries || 3
options.emfileWait = options.emfileWait || 1000
if (options.glob === false) {
options.disableGlob = true
}
if (options.disableGlob !== true && glob === undefined) {
throw Error('glob dependency not found, set `options.disableGlob = true` if intentional')
}
options.disableGlob = options.disableGlob || false
options.glob = options.glob || defaultGlobOpts
}
function rimraf (p, options, cb) {
if (typeof options === 'function') {
cb = options
options = {}
}
assert(p, 'rimraf: missing path')
assert.equal(typeof p, 'string', 'rimraf: path should be a string')
assert.equal(typeof cb, 'function', 'rimraf: callback function required')
assert(options, 'rimraf: invalid options argument provided')
assert.equal(typeof options, 'object', 'rimraf: options should be object')
defaults(options)
var busyTries = 0
var errState = null
var n = 0
if (options.disableGlob || !glob.hasMagic(p))
return afterGlob(null, [p])
options.lstat(p, function (er, stat) {
if (!er)
return afterGlob(null, [p])
glob(p, options.glob, afterGlob)
})
function next (er) {
errState = errState || er
if (--n === 0)
cb(errState)
}
function afterGlob (er, results) {
if (er)
return cb(er)
n = results.length
if (n === 0)
return cb()
results.forEach(function (p) {
rimraf_(p, options, function CB (er) {
if (er) {
if ((er.code === "EBUSY" || er.code === "ENOTEMPTY" || er.code === "EPERM") &&
busyTries < options.maxBusyTries) {
busyTries ++
var time = busyTries * 100
// try again, with the same exact callback as this one.
return setTimeout(function () {
rimraf_(p, options, CB)
}, time)
}
// this one won't happen if graceful-fs is used.
if (er.code === "EMFILE" && timeout < options.emfileWait) {
return setTimeout(function () {
rimraf_(p, options, CB)
}, timeout ++)
}
// already gone
if (er.code === "ENOENT") er = null
}
timeout = 0
next(er)
})
})
}
}
// Two possible strategies.
// 1. Assume it's a file. unlink it, then do the dir stuff on EPERM or EISDIR
// 2. Assume it's a directory. readdir, then do the file stuff on ENOTDIR
//
// Both result in an extra syscall when you guess wrong. However, there
// are likely far more normal files in the world than directories. This
// is based on the assumption that a the average number of files per
// directory is >= 1.
//
// If anyone ever complains about this, then I guess the strategy could
// be made configurable somehow. But until then, YAGNI.
function rimraf_ (p, options, cb) {
assert(p)
assert(options)
assert(typeof cb === 'function')
// sunos lets the root user unlink directories, which is... weird.
// so we have to lstat here and make sure it's not a dir.
options.lstat(p, function (er, st) {
if (er && er.code === "ENOENT")
return cb(null)
// Windows can EPERM on stat. Life is suffering.
if (er && er.code === "EPERM" && isWindows)
fixWinEPERM(p, options, er, cb)
if (st && st.isDirectory())
return rmdir(p, options, er, cb)
options.unlink(p, function (er) {
if (er) {
if (er.code === "ENOENT")
return cb(null)
if (er.code === "EPERM")
return (isWindows)
? fixWinEPERM(p, options, er, cb)
: rmdir(p, options, er, cb)
if (er.code === "EISDIR")
return rmdir(p, options, er, cb)
}
return cb(er)
})
})
}
function fixWinEPERM (p, options, er, cb) {
assert(p)
assert(options)
assert(typeof cb === 'function')
if (er)
assert(er instanceof Error)
options.chmod(p, _0666, function (er2) {
if (er2)
cb(er2.code === "ENOENT" ? null : er)
else
options.stat(p, function(er3, stats) {
if (er3)
cb(er3.code === "ENOENT" ? null : er)
else if (stats.isDirectory())
rmdir(p, options, er, cb)
else
options.unlink(p, cb)
})
})
}
function fixWinEPERMSync (p, options, er) {
assert(p)
assert(options)
if (er)
assert(er instanceof Error)
try {
options.chmodSync(p, _0666)
} catch (er2) {
if (er2.code === "ENOENT")
return
else
throw er
}
try {
var stats = options.statSync(p)
} catch (er3) {
if (er3.code === "ENOENT")
return
else
throw er
}
if (stats.isDirectory())
rmdirSync(p, options, er)
else
options.unlinkSync(p)
}
function rmdir (p, options, originalEr, cb) {
assert(p)
assert(options)
if (originalEr)
assert(originalEr instanceof Error)
assert(typeof cb === 'function')
// try to rmdir first, and only readdir on ENOTEMPTY or EEXIST (SunOS)
// if we guessed wrong, and it's not a directory, then
// raise the original error.
options.rmdir(p, function (er) {
if (er && (er.code === "ENOTEMPTY" || er.code === "EEXIST" || er.code === "EPERM"))
rmkids(p, options, cb)
else if (er && er.code === "ENOTDIR")
cb(originalEr)
else
cb(er)
})
}
function rmkids(p, options, cb) {
assert(p)
assert(options)
assert(typeof cb === 'function')
options.readdir(p, function (er, files) {
if (er)
return cb(er)
var n = files.length
if (n === 0)
return options.rmdir(p, cb)
var errState
files.forEach(function (f) {
rimraf(path.join(p, f), options, function (er) {
if (errState)
return
if (er)
return cb(errState = er)
if (--n === 0)
options.rmdir(p, cb)
})
})
})
}
// this looks simpler, and is strictly *faster*, but will
// tie up the JavaScript thread and fail on excessively
// deep directory trees.
function rimrafSync (p, options) {
options = options || {}
defaults(options)
assert(p, 'rimraf: missing path')
assert.equal(typeof p, 'string', 'rimraf: path should be a string')
assert(options, 'rimraf: missing options')
assert.equal(typeof options, 'object', 'rimraf: options should be object')
var results
if (options.disableGlob || !glob.hasMagic(p)) {
results = [p]
} else {
try {
options.lstatSync(p)
results = [p]
} catch (er) {
results = glob.sync(p, options.glob)
}
}
if (!results.length)
return
for (var i = 0; i < results.length; i++) {
var p = results[i]
try {
var st = options.lstatSync(p)
} catch (er) {
if (er.code === "ENOENT")
return
// Windows can EPERM on stat. Life is suffering.
if (er.code === "EPERM" && isWindows)
fixWinEPERMSync(p, options, er)
}
try {
// sunos lets the root user unlink directories, which is... weird.
if (st && st.isDirectory())
rmdirSync(p, options, null)
else
options.unlinkSync(p)
} catch (er) {
if (er.code === "ENOENT")
return
if (er.code === "EPERM")
return isWindows ? fixWinEPERMSync(p, options, er) : rmdirSync(p, options, er)
if (er.code !== "EISDIR")
throw er
rmdirSync(p, options, er)
}
}
}
function rmdirSync (p, options, originalEr) {
assert(p)
assert(options)
if (originalEr)
assert(originalEr instanceof Error)
try {
options.rmdirSync(p)
} catch (er) {
if (er.code === "ENOENT")
return
if (er.code === "ENOTDIR")
throw originalEr
if (er.code === "ENOTEMPTY" || er.code === "EEXIST" || er.code === "EPERM")
rmkidsSync(p, options)
}
}
function rmkidsSync (p, options) {
assert(p)
assert(options)
options.readdirSync(p).forEach(function (f) {
rimrafSync(path.join(p, f), options)
})
// We only end up here once we got ENOTEMPTY at least once, and
// at this point, we are guaranteed to have removed all the kids.
// So, we know that it won't be ENOENT or ENOTDIR or anything else.
// try really hard to delete stuff on windows, because it has a
// PROFOUNDLY annoying habit of not closing handles promptly when
// files are deleted, resulting in spurious ENOTEMPTY errors.
var retries = isWindows ? 100 : 1
var i = 0
do {
var threw = true
try {
var ret = options.rmdirSync(p, options)
threw = false
return ret
} finally {
if (++i < retries && threw)
continue
}
} while (true)
}
@@ -0,0 +1,11 @@
{
"spellright.language": [
"en"
],
"spellright.documentTypes": [
"markdown",
"latex",
"plaintext",
"javascript"
]
}
@@ -0,0 +1,27 @@
Copyright 2014 Yahoo! Inc.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the Yahoo! Inc. nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL YAHOO! INC. BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,144 @@
Serialize JavaScript
====================
Serialize JavaScript to a _superset_ of JSON that includes regular expressions, dates and functions.
[![npm Version][npm-badge]][npm]
[![Dependency Status][david-badge]][david]
[![Build Status][travis-badge]][travis]
## Overview
The code in this package began its life as an internal module to [express-state][]. To expand its usefulness, it now lives as `serialize-javascript` — an independent package on npm.
You're probably wondering: **What about `JSON.stringify()`!?** We've found that sometimes we need to serialize JavaScript **functions**, **regexps**, **dates**, **sets** or **maps**. A great example is a web app that uses client-side URL routing where the route definitions are regexps that need to be shared from the server to the client. But this module is also great for communicating between node processes.
The string returned from this package's single export function is literal JavaScript which can be saved to a `.js` file, or be embedded into an HTML document by making the content of a `<script>` element.
> **HTML characters and JavaScript line terminators are escaped automatically.**
Please note that serialization for ES6 Sets & Maps requires support for `Array.from` (not available in IE or Node < 0.12), or an `Array.from` polyfill.
## Installation
Install using npm:
```shell
$ npm install serialize-javascript
```
## Usage
```js
var serialize = require('serialize-javascript');
serialize({
str : 'string',
num : 0,
obj : {foo: 'foo'},
arr : [1, 2, 3],
bool : true,
nil : null,
undef: undefined,
inf : Infinity,
date : new Date("Thu, 28 Apr 2016 22:02:17 GMT"),
map : new Map([['hello', 'world']]),
set : new Set([123, 456]),
fn : function echo(arg) { return arg; },
re : /([^\s]+)/g,
big : BigInt(10),
});
```
The above will produce the following string output:
```js
'{"str":"string","num":0,"obj":{"foo":"foo"},"arr":[1,2,3],"bool":true,"nil":null,"undef":undefined,"inf":Infinity,"date":new Date("2016-04-28T22:02:17.000Z"),"map":new Map([["hello","world"]]),"set":new Set([123,456]),"fn":function echo(arg) { return arg; },"re":new RegExp("([^\\\\s]+)", "g"),"big":BigInt("10")}'
```
Note: to produced a beautified string, you can pass an optional second argument to `serialize()` to define the number of spaces to be used for the indentation.
### Automatic Escaping of HTML Characters
A primary feature of this package is to serialize code to a string of literal JavaScript which can be embedded in an HTML document by adding it as the contents of the `<script>` element. In order to make this safe, HTML characters and JavaScript line terminators are escaped automatically.
```js
serialize({
haxorXSS: '</script>'
});
```
The above will produce the following string, HTML-escaped output which is safe to put into an HTML document as it will not cause the inline script element to terminate:
```js
'{"haxorXSS":"\\u003C\\u002Fscript\\u003E"}'
```
> You can pass an optional `unsafe` argument to `serialize()` for straight serialization.
### Options
The `serialize()` function accepts an `options` object as its second argument. All options are being defaulted to `undefined`:
#### `options.space`
This option is the same as the `space` argument that can be passed to [`JSON.stringify`][JSON.stringify]. It can be used to add whitespace and indentation to the serialized output to make it more readable.
```js
serialize(obj, {space: 2});
```
#### `options.isJSON`
This option is a signal to `serialize()` that the object being serialized does not contain any function or regexps values. This enables a hot-path that allows serialization to be over 3x faster. If you're serializing a lot of data, and know its pure JSON, then you can enable this option for a speed-up.
**Note:** That when using this option, the output will still be escaped to protect against XSS.
```js
serialize(obj, {isJSON: true});
```
#### `options.unsafe`
This option is to signal `serialize()` that we want to do a straight conversion, without the XSS protection. This options needs to be explicitly set to `true`. HTML characters and JavaScript line terminators will not be escaped. You will have to roll your own.
```js
serialize(obj, {unsafe: true});
```
#### `options.ignoreFunction`
This option is to signal `serialize()` that we do not want serialize JavaScript function.
Just treat function like `JSON.stringify` do, but other features will work as expected.
```js
serialize(obj, {ignoreFunction: true});
```
## Deserializing
For some use cases you might also need to deserialize the string. This is explicitly not part of this module. However, you can easily write it yourself:
```js
function deserialize(serializedJavascript){
return eval('(' + serializedJavascript + ')');
}
```
**Note:** Don't forget the parentheses around the serialized javascript, as the opening bracket `{` will be considered to be the start of a body.
## License
This software is free to use under the Yahoo! Inc. BSD license.
See the [LICENSE file][LICENSE] for license text and copyright information.
[npm]: https://www.npmjs.org/package/serialize-javascript
[npm-badge]: https://img.shields.io/npm/v/serialize-javascript.svg?style=flat-square
[david]: https://david-dm.org/yahoo/serialize-javascript
[david-badge]: https://img.shields.io/david/yahoo/serialize-javascript.svg?style=flat-square
[travis]: https://travis-ci.org/yahoo/serialize-javascript
[travis-badge]: https://img.shields.io/travis/yahoo/serialize-javascript.svg?style=flat-square
[express-state]: https://github.com/yahoo/express-state
[JSON.stringify]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify
[LICENSE]: https://github.com/yahoo/serialize-javascript/blob/master/LICENSE
@@ -0,0 +1,247 @@
/*
Copyright (c) 2014, Yahoo! Inc. All rights reserved.
Copyrights licensed under the New BSD License.
See the accompanying LICENSE file for terms.
*/
'use strict';
var randomBytes = require('randombytes');
// Generate an internal UID to make the regexp pattern harder to guess.
var UID_LENGTH = 16;
var UID = generateUID();
var PLACE_HOLDER_REGEXP = new RegExp('(\\\\)?"@__(F|R|D|M|S|U|I|B)-' + UID + '-(\\d+)__@"', 'g');
var IS_NATIVE_CODE_REGEXP = /\{\s*\[native code\]\s*\}/g;
var IS_PURE_FUNCTION = /function.*?\(/;
var IS_ARROW_FUNCTION = /.*?=>.*?/;
var UNSAFE_CHARS_REGEXP = /[<>\/\u2028\u2029]/g;
var RESERVED_SYMBOLS = ['*', 'async'];
// Mapping of unsafe HTML and invalid JavaScript line terminator chars to their
// Unicode char counterparts which are safe to use in JavaScript strings.
var ESCAPED_CHARS = {
'<' : '\\u003C',
'>' : '\\u003E',
'/' : '\\u002F',
'\u2028': '\\u2028',
'\u2029': '\\u2029'
};
function escapeUnsafeChars(unsafeChar) {
return ESCAPED_CHARS[unsafeChar];
}
function generateUID() {
var bytes = randomBytes(UID_LENGTH);
var result = '';
for(var i=0; i<UID_LENGTH; ++i) {
result += bytes[i].toString(16);
}
return result;
}
function deleteFunctions(obj){
var functionKeys = [];
for (var key in obj) {
if (typeof obj[key] === "function") {
functionKeys.push(key);
}
}
for (var i = 0; i < functionKeys.length; i++) {
delete obj[functionKeys[i]];
}
}
module.exports = function serialize(obj, options) {
options || (options = {});
// Backwards-compatibility for `space` as the second argument.
if (typeof options === 'number' || typeof options === 'string') {
options = {space: options};
}
var functions = [];
var regexps = [];
var dates = [];
var maps = [];
var sets = [];
var undefs = [];
var infinities= [];
var bigInts = [];
// Returns placeholders for functions and regexps (identified by index)
// which are later replaced by their string representation.
function replacer(key, value) {
// For nested function
if(options.ignoreFunction){
deleteFunctions(value);
}
if (!value && value !== undefined) {
return value;
}
// If the value is an object w/ a toJSON method, toJSON is called before
// the replacer runs, so we use this[key] to get the non-toJSONed value.
var origValue = this[key];
var type = typeof origValue;
if (type === 'object') {
if(origValue instanceof RegExp) {
return '@__R-' + UID + '-' + (regexps.push(origValue) - 1) + '__@';
}
if(origValue instanceof Date) {
return '@__D-' + UID + '-' + (dates.push(origValue) - 1) + '__@';
}
if(origValue instanceof Map) {
return '@__M-' + UID + '-' + (maps.push(origValue) - 1) + '__@';
}
if(origValue instanceof Set) {
return '@__S-' + UID + '-' + (sets.push(origValue) - 1) + '__@';
}
}
if (type === 'function') {
return '@__F-' + UID + '-' + (functions.push(origValue) - 1) + '__@';
}
if (type === 'undefined') {
return '@__U-' + UID + '-' + (undefs.push(origValue) - 1) + '__@';
}
if (type === 'number' && !isNaN(origValue) && !isFinite(origValue)) {
return '@__I-' + UID + '-' + (infinities.push(origValue) - 1) + '__@';
}
if (type === 'bigint') {
return '@__B-' + UID + '-' + (bigInts.push(origValue) - 1) + '__@';
}
return value;
}
function serializeFunc(fn) {
var serializedFn = fn.toString();
if (IS_NATIVE_CODE_REGEXP.test(serializedFn)) {
throw new TypeError('Serializing native function: ' + fn.name);
}
// pure functions, example: {key: function() {}}
if(IS_PURE_FUNCTION.test(serializedFn)) {
return serializedFn;
}
// arrow functions, example: arg1 => arg1+5
if(IS_ARROW_FUNCTION.test(serializedFn)) {
return serializedFn;
}
var argsStartsAt = serializedFn.indexOf('(');
var def = serializedFn.substr(0, argsStartsAt)
.trim()
.split(' ')
.filter(function(val) { return val.length > 0 });
var nonReservedSymbols = def.filter(function(val) {
return RESERVED_SYMBOLS.indexOf(val) === -1
});
// enhanced literal objects, example: {key() {}}
if(nonReservedSymbols.length > 0) {
return (def.indexOf('async') > -1 ? 'async ' : '') + 'function'
+ (def.join('').indexOf('*') > -1 ? '*' : '')
+ serializedFn.substr(argsStartsAt);
}
// arrow functions
return serializedFn;
}
// Check if the parameter is function
if (options.ignoreFunction && typeof obj === "function") {
obj = undefined;
}
// Protects against `JSON.stringify()` returning `undefined`, by serializing
// to the literal string: "undefined".
if (obj === undefined) {
return String(obj);
}
var str;
// Creates a JSON string representation of the value.
// NOTE: Node 0.12 goes into slow mode with extra JSON.stringify() args.
if (options.isJSON && !options.space) {
str = JSON.stringify(obj);
} else {
str = JSON.stringify(obj, options.isJSON ? null : replacer, options.space);
}
// Protects against `JSON.stringify()` returning `undefined`, by serializing
// to the literal string: "undefined".
if (typeof str !== 'string') {
return String(str);
}
// Replace unsafe HTML and invalid JavaScript line terminator chars with
// their safe Unicode char counterpart. This _must_ happen before the
// regexps and functions are serialized and added back to the string.
if (options.unsafe !== true) {
str = str.replace(UNSAFE_CHARS_REGEXP, escapeUnsafeChars);
}
if (functions.length === 0 && regexps.length === 0 && dates.length === 0 && maps.length === 0 && sets.length === 0 && undefs.length === 0 && infinities.length === 0 && bigInts.length === 0) {
return str;
}
// Replaces all occurrences of function, regexp, date, map and set placeholders in the
// JSON string with their string representations. If the original value can
// not be found, then `undefined` is used.
return str.replace(PLACE_HOLDER_REGEXP, function (match, backSlash, type, valueIndex) {
// The placeholder may not be preceded by a backslash. This is to prevent
// replacing things like `"a\"@__R-<UID>-0__@"` and thus outputting
// invalid JS.
if (backSlash) {
return match;
}
if (type === 'D') {
return "new Date(\"" + dates[valueIndex].toISOString() + "\")";
}
if (type === 'R') {
return "new RegExp(" + serialize(regexps[valueIndex].source) + ", \"" + regexps[valueIndex].flags + "\")";
}
if (type === 'M') {
return "new Map(" + serialize(Array.from(maps[valueIndex].entries()), options) + ")";
}
if (type === 'S') {
return "new Set(" + serialize(Array.from(sets[valueIndex].values()), options) + ")";
}
if (type === 'U') {
return 'undefined'
}
if (type === 'I') {
return infinities[valueIndex];
}
if (type === 'B') {
return "BigInt(\"" + bigInts[valueIndex] + "\")";
}
var fn = functions[valueIndex];
return serializeFunc(fn);
});
}
@@ -0,0 +1,36 @@
{
"name": "serialize-javascript",
"version": "4.0.0",
"description": "Serialize JavaScript to a superset of JSON that includes regular expressions and functions.",
"main": "index.js",
"scripts": {
"benchmark": "node -v && node test/benchmark/serialize.js",
"test": "nyc --reporter=lcov mocha test/unit"
},
"repository": {
"type": "git",
"url": "git+https://github.com/yahoo/serialize-javascript.git"
},
"keywords": [
"serialize",
"serialization",
"javascript",
"js",
"json"
],
"author": "Eric Ferraiuolo <edf@ericf.me>",
"license": "BSD-3-Clause",
"bugs": {
"url": "https://github.com/yahoo/serialize-javascript/issues"
},
"homepage": "https://github.com/yahoo/serialize-javascript",
"devDependencies": {
"benchmark": "^2.1.4",
"chai": "^4.1.0",
"mocha": "^7.0.0",
"nyc": "^15.0.0"
},
"dependencies": {
"randombytes": "^2.1.0"
}
}
+96
View File
@@ -0,0 +1,96 @@
{
"name": "terser-webpack-plugin",
"version": "2.3.8",
"description": "Terser plugin for webpack",
"license": "MIT",
"repository": "webpack-contrib/terser-webpack-plugin",
"author": "webpack Contrib Team",
"homepage": "https://github.com/webpack-contrib/terser-webpack-plugin",
"bugs": "https://github.com/webpack-contrib/terser-webpack-plugin/issues",
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/webpack"
},
"main": "dist/cjs.js",
"engines": {
"node": ">= 8.9.0"
},
"scripts": {
"start": "npm run build -- -w",
"clean": "del-cli dist",
"prebuild": "npm run clean",
"build": "cross-env NODE_ENV=production babel src -d dist --copy-files",
"commitlint": "commitlint --from=master",
"security": "npm audit",
"lint:prettier": "prettier \"{**/*,*}.{js,json,md,yml,css,ts}\" --list-different",
"lint:js": "eslint --cache .",
"lint": "npm-run-all -l -p \"lint:**\"",
"test:only": "cross-env NODE_ENV=test jest",
"test:watch": "npm run test:only -- --watch",
"test:coverage": "npm run test:only -- --collectCoverageFrom=\"src/**/*.js\" --coverage",
"pretest": "npm run lint",
"test": "npm run test:coverage",
"prepare": "npm run build",
"release": "standard-version",
"defaults": "webpack-defaults"
},
"files": [
"dist"
],
"peerDependencies": {
"webpack": "^4.0.0 || ^5.0.0"
},
"dependencies": {
"cacache": "^13.0.1",
"find-cache-dir": "^3.3.1",
"jest-worker": "^25.4.0",
"p-limit": "^2.3.0",
"schema-utils": "^2.6.6",
"serialize-javascript": "^4.0.0",
"source-map": "^0.6.1",
"terser": "^4.6.12",
"webpack-sources": "^1.4.3"
},
"devDependencies": {
"@babel/cli": "^7.8.4",
"@babel/core": "^7.9.0",
"@babel/preset-env": "^7.9.5",
"@commitlint/cli": "^8.3.5",
"@commitlint/config-conventional": "^8.3.4",
"@webpack-contrib/defaults": "^6.2.0",
"@webpack-contrib/eslint-config-webpack": "^3.0.0",
"babel-jest": "^25.4.0",
"cross-env": "^6.0.3",
"del": "^5.1.0",
"del-cli": "^3.0.0",
"eslint": "^6.8.0",
"eslint-config-prettier": "^6.11.0",
"eslint-plugin-import": "^2.20.2",
"husky": "^4.2.5",
"jest": "^25.4.0",
"lint-staged": "^10.1.7",
"memfs": "^3.1.2",
"npm-run-all": "^4.1.5",
"prettier": "^1.19.1",
"standard-version": "^7.1.0",
"uglify-js": "^3.9.1",
"webpack": "^4.43.0"
},
"keywords": [
"uglify",
"uglify-js",
"uglify-es",
"terser",
"webpack",
"webpack-plugin",
"minification",
"compress",
"compressor",
"min",
"minification",
"minifier",
"minify",
"optimize",
"optimizer"
]
}