Webpack build hangs with parallel: true

[Pajn]

• Operating System: Windows 10 1810, WSL, Ubuntu 18.04
• Node Version: 8.12.0
• NPM Version: 6.4.1
• webpack Version: 4.19.1
• terser-webpack-plugin Version: 1.1.0

Expected Behavior

That the webpack build completes

Actual Behavior

The webpack build hangs forever (or at least 30 minutes which is the longest I have lasted) after outputting

Compilation  starting…

However changing parallel: true to parallel: false in terser options will cause the build to complete in normal time (around 40s)

Code

new TerserPlugin({
        terserOptions: {
          parse: {
            // we want terser to parse ecma 8 code. However, we don't want it
            // to apply any minfication steps that turns valid ecma 5 code
            // into invalid ecma 5 code. This is why the 'compress' and 'output'
            // sections only apply transformations that are ecma 5 safe
            // https://github.com/facebook/create-react-app/pull/4234
            ecma: 8
          },
          compress: {
            ecma: 5,
            warnings: false,
            // Disabled because of an issue with Uglify breaking seemingly valid code:
            // https://github.com/facebook/create-react-app/issues/2376
            // Pending further investigation:
            // https://github.com/mishoo/UglifyJS2/issues/2011
            comparisons: false,
            // Disabled because of an issue with Terser breaking valid code:
            // https://github.com/facebook/create-react-app/issues/5250
            // Pending futher investigation:
            // https://github.com/terser-js/terser/issues/120
            inline: 2
          },
          mangle: {
            safari10: true
          },
          output: {
            ecma: 5,
            comments: false,
            // Turned on because emoji and regex is not minified properly using default
            // https://github.com/facebook/create-react-app/issues/2488
            ascii_only: true
          }
        },
        // Use multi-process parallel running to improve the build speed
        // Default number of concurrent runs: os.cpus().length - 1
        parallel: true,
        // Enable file caching
        cache: true,
        sourceMap: shouldUseSourceMap
      })

Full webpack config: https://github.com/Pajn/tscomp/blob/cra2/config/webpack.config.prod.js

How Do We Reproduce?

I suspect this can be related to WSL. If you have some debugging tips I can probably run them.

For the exact same state as I have:

1. Clone https://github.com/Pajn/tscomp
2. git checkout terser-bug
3. Clone https://github.com/Pajn/RAXA
4. cd RAXA/packages/web
5. yarn
6. yarn add --dev path/to/cloned/tscomp/repo
7. yarn build, or for more information NODE_ENV=production webpack --config node_modules/tscomp/config/webpack.config.prod.js --verbose --info-verbosity verbose --display verbose --profile

[alexander-akait]

@Pajn Thanks for issue, we have some problem on WSL for uglify plugin webpack-contrib/uglifyjs-webpack-plugin#302, seems problem in worker-farm, thanks for reproducible test repo, i am not familiar with WSL, can you provide instruction how i can setup this env?

[Pajn]

To set up WSL you need Windows 10, then you should be able to follow this guide https://docs.microsoft.com/en-us/windows/wsl/install-win10

After that you can basically treat is as a normal Ubuntu Linux so install node by following this https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions
Git can be installed via sudo apt install git

The only consideration with a case like this is that Linux files are not visible to Windows, so to be able to open files in a graphical editor you need to save them under /mnt/c

[alexander-akait]

@Pajn oh, i don't have windows 10 right now, can debug this only on next week, feel free to debug problem self. Here default arguments to work-farm https://github.com/webpack-contrib/terser-webpack-plugin/blob/master/src/TaskRunner.js#L36, maybe we need improve this check process.platform === 'win32' and pass maxConcurrentCallsPerWorker: 1 on WSL

[Pajn]

You are right that worker-farm is somehow related.
This is what I get from pstree in a stuck build

  │   │   ├─node /usr/share/yarn/bin/yarn.js build
  │   │   │   ├─sh -c tscomp build
  │   │   │   │   └─node /c/Users/Rasmus/Development/RAXA/packages/web/node_modules/.bin/tscomp build
  │   │   │   │       ├─node /c/Users/Rasmus/Development/RAXA/packages/web/node_modules/tscomp/scripts/build.js
  │   │   │   │       │   ├─node /c/Users/Rasmus/Development/RAXA/packages/web/node_modules/tscomp/node_modules/worker-farm/lib/child/index.js /usr/bin/node /c/Users/Rasmus/Development/RAXA/packages/web/node_modules/tscomp/scripts/build.js
  │   │   │   │       │   │   └─5*[{node}]
  │   │   │   │       │   ├─node /c/Users/Rasmus/Development/RAXA/packages/web/node_modules/tscomp/node_modules/worker-farm/lib/child/index.js /usr/bin/node /c/Users/Rasmus/Development/RAXA/packages/web/node_modules/tscomp/scripts/build.js
  │   │   │   │       │   │   └─5*[{node}]
  │   │   │   │       │   └─9*[{node}]
  │   │   │   │       └─5*[{node}]
  │   │   │   └─9*[{node}]

Changing that condition so that maxConcurrentCallsPerWorker: 1 is passed (process.platform is linux) did not help unfortunately.

I tried running the basic example from the worker-farm readme and that worked without hanging so I guess there some problem with file locking.

I'll continue the debugging tomorrow.

[alexander-akait]

@Pajn thanks for helping! How many cpu you are have? Based on output worker-farm is create child processes but don't get response.

[Pajn]

I have four, however the above pstree output is from a run where I limited the number of workers to two, just to see if it did help.

I modified the code so that it would still use worker-farm with only one worker, and then it did finish again. I also tried to attach strace to see if I got some hint of the problem but did not find anything useful in the output. If you are interested I can of course share those outputs.

[alexander-akait]

@Pajn Feel free to publish outputs

[alexander-akait]

Guys i try to reproducible problem but i can't, maybe can create really minimum reproducible test repo with really minimum setup?

[charsleysa]

@evilebottnawi you should be able to recreate this by compiling the an Angular demo app (any should do) with aot and optimization flags turned on.

I get this issue where the TerserPlugin seems to hang with a lot of node processes showing in Task Manager using 0% CPU resource.

[alexander-akait]

@charsleysa can you provide instruction for minimum reproducible test repo?

[charsleysa]

@evilebottnawi it seems to be something to do with having a project sufficiently large enough producing source maps on a production build.

The following instructions assume you have already setup WSL with Node 10.x (node inside WSL) using standard setup instructions. The below instructions are executed inside bash.

1. clone this repo https://github.com/formio/angular-demo
2. cd into the cloned repo directory
3. install the angular cli npm install -g @angular/cli
4. install the repo requirements npm install
5. run a standard build just to check that everything is working ng build
6. run a production build with source maps turned on ng build --configuration=production --project=demo --source-map=true
7. if you watch Task Manager you'll notice that eventually the node processes will just cease utilizing CPU resource with the build reporting 92% chunk asset optimization TerserPlugin (as pictured below)

[alexander-akait]

@charsleysa thanks for instructions, anyway feel free to investigate problem(s), i think problem inside worker-farm, maybe you can find problem faster than i can, because i need time to setup WSL