No error is returned when using a browser that is not installed.

[hmil]

Expected behaviour

Given that the target browser is PhantomJS
and the plugin package for PhantomJS is not installed.
When the following code runs:

  new karma.Server({
    configFile: __dirname + '/karma.conf.js',
    singleRun: true
  }, function(status) {
    console.log(status);
  }).start();

Then the console should print an error and an error status != 0 like this:

12 10 2016 17:25:43.747:ERROR [launcher]: Cannot load browser "PhantomJS": it is not registered! Perhaps you are missing some plugin?
12 10 2016 17:25:43.748:ERROR [karma]: Found 1 load error
{something that is not 0}

Actual behaviour

The console shows a status 0:

12 10 2016 17:25:43.747:ERROR [launcher]: Cannot load browser "PhantomJS": it is not registered! Perhaps you are missing some plugin?
12 10 2016 17:25:43.748:ERROR [karma]: Found 1 load error
0

Environment Details

• Karma version (output of karma --version): 1.3.0
• Relevant part of your karma.config.js file

{
    browsers: ['PhantomJS'],
}

Steps to reproduce the behaviour

See above.

Additional notes

This breaks the recommended usage shown in gulp-karma as the job will exit with status 0 and pass CI checks although no tests actually ran!

[monolithed]

It seems you should install karma-phantomjs-launcher

[hmil]

That's not the point. The point is given that it is not installed, I should obtain an error.

Currently, if you integrate a karma build in some CI environment, if you forget to add the launcher dependency, or if for some reason it doesn't get installed, karma will report this as a success even though none of the tests could actually run!

[monolithed]

I see what you mean. I have got the same error and don't understand why is not documented yet.