Karma - a Javascript Test Runner

Michael G Bielski

Karma is a test runner for JavaScript that runs on Node.js. It is very well suited to testing AngularJS or any other JavaScript projects. Using Karma to run tests using one of many popular JavaScript testing suites (Jasmine, Mocha, QUnit, etc.) and have those tests executed not only in the browsers of your choice, but also on the platform of your choice (desktop, phone, tablet.) Karma is highly configurable, integrates with popular continuous integration packages (Jenkins, Travis, and Semaphore) and has excellent plugin support.

Website: http://karma-runner.github.io/
Version Tested: v0.10 running on NodeJS 0.8.19 running on Windows 7 Professional 64-bit
License and Pricing: Open Source
Support: User mailing list/group on Google Groups ( https://groups.google.com/forum/?fromgroups#!forum/karma-users) and tagged questions on Stack Overflow (http://stackoverflow.com/questions/tagged/karma and http://stackoverflow.com/questions/tagged/karma-runner).

Installation

The recommended installation location of Karma is in your global node_modules directory. Installing Karma is done via the Node Package Manager (NPM). From any command prompt, enter the command: npm install -g karma. Should you desire to install Karma to your local directory you can use: npm install karma -save-dev. This process will get you the default installation of Karma, which includes the karma-chrome-launcher, karma-coverage, and karma-jasmine plugins. There are also several plugins (a list can be found at https://npmjs.org/browse/keyword/karma-plugin) that you should consider installing to make your test management and output easier and more useful. I regularly use the karma-firefox-launcher and karma-ie-launcher plugins. All plugins should be installed before you try to configure or use Karma, but you can always add a new one later with no troubles at all. Installing plugins is just as easy as installing Karma. From any command prompt, enter the command: npm install -g-karma-<plugin name>.

Configuration

Before you run Karma you must configure it. This is the most important step in setting up Karma. The easiest way to get started is to run the init command. In a command window, navigate to your project folder and enter karma init karma.conf.js (this is the default, but you can name the file whatever you want.) The questions are easy to answer and the net result is a properly structured Karma configuration file. You'll need to answer the following questions:

• What testing framework do you want to use?

Jasmine, Mocha, and QUnit are installed by default and can be referenced by name, but if you have installed another one you should name it here. Because each framework is a plugin, you will need to list the plugin file in the plugins section of the configuration file as well (see the sample configuration file below for an example.)

• Do you want to use Require.js?

Require.js is a lazy-loading framework that many developers use to minimize the initial script load times in the browser. If your project uses is, you need to answer YES to this question.

• Do you want to capture a browser automatically?

You should respond with the browser name(s) that you want to use, one per line. Remember that you must have the matching -launcher plugin installed for each browser, and that only the karma-chrome-launcher and karma-chromeCanary-launcher plugin are installed by default. Entering a blank line will move you to the next question.

• What is the location of your source and test files?

This path is relative to the location of Karma. You can enter an absolute path and be assured of directing Karma to the right location, or enter a relative path if your files are located below the Karma folder.

• Should any of the files included by the previous patterns be excluded?

If you use a very broad pattern, you may want to exclude the folder where you store your images, or where there are no .js files to test. Tighter patterns take up more lines in the configuration file, but also eliminate the need for an exclude block.

• Do you want Karma to watch all the files and run tests on change?

Having Karma run continuously is very helpful as you can see your tests and code evolve in a TDD environment. If you don't do TDD, you can opt to only run Karma when you are ready to execute your tests.

For the more adventurous or those that need more than just the basic configuration, you can edit the configuration file and tailor Karma to fit your needs. There are four main sections of the configuration file that you'll want to pay particular attention to: preprocessors, plugins, browsers, and files.

Preprocessors

Preprocessors enable you to do things to your code before it gets sent to the browser to have the tests run on it. Karma comes with the karma-coffee and karma-html2js preprocessors already installed. I regularly use the karma-coverage preprocessor to show me how much of my code I have covered with my tests. For a complete description of what any individual preprocessor does you should read the documentation on its website.

Plugins

While Karma comes with several plugins pre-installed, you need to tell it which ones you want to use for running your tests. There are methods for dynamically determining which plugins get used, or you can just list them out. If your project doesn't change much with respect to how you are testing your code, the static listing is the easiest way to go.

Browsers

Browser support is critical for JavaScript. Karma comes with launchers pre-installed for Chrome, Chrome-Canary, and PhantomJS. Other launchers need to be installed if you want to test your JavaScript in other browsers. I regularly test in Chrome, IE, and Firefox. Karma takes care of capturing and killing the browser processes for you. This enables you to spend more time developing and less time doing the menial processes associated with testing. The default time allowed to capture any one browser is one minute, which should be more than enough time, but you can change that value via the captureTimeout option in the configuration file. You will also need to tell Karma where the executable files are for all browsers except IE. If you want to capture your browser manually (from a tablet, for example) you can point the browser to http://<hostname>:<port> where <hostname> is the hostname or IP address of the computer where Karma is running, and <port> is the port you specified in the configuration file. By default this is port 9876, but if you have changed it in the port section of your configuration file then use that port.

Files

You need to tell Karma which files are required to run your project, which files contain your tests, and which files need to be tested. For convenience, in my configuration file I have broken these files into their respective groups, with required files first, test files second, and files to be tested last. The order in which they appear is the order that they will be included in the browser, so depending upon your project you may need to pay attention to it. Each file or group of files can be flagged for being watched by Karma, included in the browser <script> tag, and served by the Karma server. Files that are watched can trigger Karma to run all of the tests again, so pay particular attention to these. If you simply list all of your files, they will be watched, included, and served, which may result in performance loss during testing on older, slower computers.

Karma runs on NodeJS and the configuration is set up using module.exports so that it gets drawn into and consumed by NodeJS. Older versions of Karma did not have this feature, and they were slightly harder to configure correctly. A good sample of a Karma configuration file looks like this:

module.exports = function(config) {
  config.set({
	// base path, that will be used to resolve files and exclude
	basePath: '../..',

	frameworks: ['jasmine'],

	// list of files / patterns to load in the browser
	files: ['test/client/mocks.js', 'static/karma.src.js', 'test/client/*.spec.js'],

	// list of files to exclude
	exclude: [],

	// use dots reporter, as travis terminal does not support escaping sequences
	// possible values: 'dots', 'progress'
	reporters: ['progress', 'junit'],

// will be resolved to basePath (in the same way as files/exclude patterns)	
junitReporter: {outputFile: 'test-results.xml'},

	// web server port
	port: 9876,

	// enable / disable colors in the output (reporters and logs)
	colors: true,

	// level of logging
	// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN
	|| config.LOG_INFO || config.LOG_DEBUG
	logLevel: config.LOG_INFO,

	// enable / disable watching file and executing tests whenever any file changes
	autoWatch: true,

	// Start these browsers, currently available:
	// - Chrome, ChromeCanary, Firefox, Opera, Safari (only Mac), PhantomJS, IE (only Windows)
	browsers: [process.env.TRAVIS ? 'Firefox' : 'Chrome'],

	// If browser does not capture in given timeout [ms], kill it
	captureTimeout: 20000,

	// Auto run tests on start (when browsers are captured) and exit
	singleRun: false,

	// report which specs are slower than 500ms
	reportSlowerThan: 500,

	// compile coffee scripts
	preprocessors: {'**/*.coffee': 'coffee'},

	plugins: ['karma-jasmine','karma-chrome-launcher',
	'karma-firefox-launcher','karma-junit-reporter']
  });
};

Running Karma

There are two main ways to run Karma: at the command line or via your IDE. I heavily favor running Karma from my IDE (WebStorm) because it keeps me on the same screen more. Regardless of which method you choose, running Karma does the following:

1. Starts a web server in Node.
2. Launches the specified browsers with a default URL that points to that web server.
3. Attempts to capture the browser session in each browser.
4. Upon capture of all browsers, the tests are run and the reports generated.
5. If you have opted to have Karma stay running and monitor files, each time one of those files changes the tests will be re-run and the reports re-generated.
6. If you have opted to only have Karma run once, each browser session is released and the browser is closed.

Test Results

Testing is only useful if you can see the results, and Karma has several options available. The progress reporter is inserted into the configuration file that the init process creates by default. This reporter gives you a complete listing of each test that is run, in the order that they are executed, and whether they pass or fail. The dots reporter is included with Karma for those that run Travis. There are other reporters available as plugins (growl, Junit, Teamcity and Coverage) that can help make your testing process easier and more complete. Each reporter must be listed in the reporters section of the configuration file, and some reporters (such as the coverage reporter) require that they be referenced in the preprocessors section as well. The coverage reporter that is included with Karma is Istanbul and I have found it very well suited for my needs and for use with Karma.

IDE Integration

Part of the beauty of Karma is how easily it can be integrated in to your IDE. Nearly all IDEs have a way to add an external tool that is launched on the command line. Configuring WebStorm 6 and older to run Karma as an external tool is covered in the introductory video that the creator (Vojta Jina) posted via YouTube (http://www.youtube.com/watch?v=MVw8N3hTfCI) and can be found at the 7:39 mark in the video. There is a handy link to this part just below the video. There is also integration built into the upcoming WebStorm 7 (currently accessible via the Early Access Program.) It has been my experience that integrating with WebStorm produces an excellent development environment. Support for configuring external tools isn't limited to JetBrains products. Visual Studio 2008 or newer in any flavor except the Express versions will allow you to configure external tools to launch from the menu (you would need to create a batch file with the requisite Karma commands in it and tell VS to launch that file.) Sublime is also a heavily favored text editor for web development, and it also supports external tools. As long as your favored IDE supports external tools in some way, you should be able to launch Karma and have it run tests.

Documentation

The vast majority of what you need to get up and running with Karma is contained in the Karma website. Most any other question can be answered by searching the Google Group or posting a question on Stack Overflow. Documentation for individual plugins will be found with those plugins. For anything that isn't covered, the Google Group is the best resource to try first, and Stack Overflow is a very close second.

Actual Experience

A large number of developers don't test their JavaScript, and they really should. Too many of them think that testing is hard to set up and hard to do, and Karma is proof of the opposite. My team and I use Karma with Jasmine every day of the week. We found the installation and configuration to be fast and easy to follow. We went from downloading NodeJS to running tests via Karma in under an hour. When you consider that my team and I were totally new to testing, let alone testing JavaScript, this is pretty impressive. We chose Karma because it had support for AngularJS without any configuration changes, and because it would let us test our other scripts as well. In each of these areas it has excelled from day #1. We are now preparing to move our project to Yeoman and have no fear about how our tests will work because Yeoman fully supports Karma. We firmly believe that we could not have chosen a better tool to include in our daily routines.

Summary

Karma is the preferred test runner for projects written with AngularJS and is well on its way to larger acceptance within the greater JavaScript community. Its plugin architecture makes it easily adaptable to other test suites and reporters, all of which add value to the core of Karma. In agile or continuous integration environments, Karma shines as an indispensable tool to development teams, providing an easy and reliable way to modify existing code and craft new code as part of TDD. It is rare that a day goes by without this tool running on my computer.

---

Unit Testing Articles

Writing Testable Code

More Software Testing and JavaScript Knowledge

Unit Testing Tools

Software Testing Magazine

Software Testing Tutorials and Videos

JavaScript Tutorials and Videos

---

Click here to view the complete list of tools reviews

This article was originally published in the Fall 2013 issue of Methods & Tools