Karma - a Javascript Test Runner
Michael G Bielski
Karma is a test runner for JavaScript that runs on Node.js. It is very well suited to testing AngularJS or any other JavaScript projects. Using Karma to run tests using one of many popular JavaScript testing suites (Jasmine, Mocha, QUnit, etc.) and have those tests executed not only in the browsers of your choice, but also on the platform of your choice (desktop, phone, tablet.) Karma is highly configurable, integrates with popular continuous integration packages (Jenkins, Travis, and Semaphore) and has excellent plugin support.
Website: http://karma-runner.github.io/
Version Tested: v0.10 running on NodeJS 0.8.19 running on Windows 7 Professional 64-bit
License and Pricing: Open Source
Support: User mailing list/group on Google Groups ( https://groups.google.com/forum/?fromgroups#!forum/karma-users)
and tagged questions on Stack Overflow (http://stackoverflow.com/questions/tagged/karma and
http://stackoverflow.com/questions/tagged/karma-runner).
Installation
The recommended installation location of Karma is in your global node_modules directory. Installing Karma is done via the Node Package Manager (NPM). From any command prompt, enter the command: npm install -g karma. Should you desire to install Karma to your local directory you can use: npm install karma -save-dev. This process will get you the default installation of Karma, which includes the karma-chrome-launcher, karma-coverage, and karma-jasmine plugins. There are also several plugins (a list can be found at https://npmjs.org/browse/keyword/karma-plugin) that you should consider installing to make your test management and output easier and more useful. I regularly use the karma-firefox-launcher and karma-ie-launcher plugins. All plugins should be installed before you try to configure or use Karma, but you can always add a new one later with no troubles at all. Installing plugins is just as easy as installing Karma. From any command prompt, enter the command: npm install -g-karma-<plugin name>.
Configuration
Before you run Karma you must configure it. This is the most important step in setting up Karma. The easiest way to get started is to run the init command. In a command window, navigate to your project folder and enter karma init karma.conf.js (this is the default, but you can name the file whatever you want.) The questions are easy to answer and the net result is a properly structured Karma configuration file. You'll need to answer the following questions:
Jasmine, Mocha, and QUnit are installed by default and can be referenced by name, but if you have installed another one you should name it here. Because each framework is a plugin, you will need to list the plugin file in the plugins section of the configuration file as well (see the sample configuration file below for an example.)
Require.js is a lazy-loading framework that many developers use to minimize the initial script load times in the browser. If your project uses is, you need to answer YES to this question.
You should respond with the browser name(s) that you want to use, one per line. Remember that you must have the matching -launcher plugin installed for each browser, and that only the karma-chrome-launcher and karma-chromeCanary-launcher plugin are installed by default. Entering a blank line will move you to the next question.
This path is relative to the location of Karma. You can enter an absolute path and be assured of directing Karma to the right location, or enter a relative path if your files are located below the Karma folder.
If you use a very broad pattern, you may want to exclude the folder where you store your images, or where there are no .js files to test. Tighter patterns take up more lines in the configuration file, but also eliminate the need for an exclude block.
Having Karma run continuously is very helpful as you can see your tests and code evolve in a TDD environment. If you don't do TDD, you can opt to only run Karma when you are ready to execute your tests.
For the more adventurous or those that need more than just the basic configuration, you can edit the configuration file and tailor Karma to fit your needs. There are four main sections of the configuration file that you'll want to pay particular attention to: preprocessors, plugins, browsers, and files.
Preprocessors
Preprocessors enable you to do things to your code before it gets sent to the browser to have the tests run on it. Karma comes with the karma-coffee and karma-html2js preprocessors already installed. I regularly use the karma-coverage preprocessor to show me how much of my code I have covered with my tests. For a complete description of what any individual preprocessor does you should read the documentation on its website.
Plugins
While Karma comes with several plugins pre-installed, you need to tell it which ones you want to use for running your tests. There are methods for dynamically determining which plugins get used, or you can just list them out. If your project doesn't change much with respect to how you are testing your code, the static listing is the easiest way to go.
Browsers
Browser support is critical for JavaScript. Karma comes with launchers pre-installed for Chrome, Chrome-Canary, and PhantomJS. Other launchers need to be installed if you want to test your JavaScript in other browsers. I regularly test in Chrome, IE, and Firefox. Karma takes care of capturing and killing the browser processes for you. This enables you to spend more time developing and less time doing the menial processes associated with testing. The default time allowed to capture any one browser is one minute, which should be more than enough time, but you can change that value via the captureTimeout option in the configuration file. You will also need to tell Karma where the executable files are for all browsers except IE. If you want to capture your browser manually (from a tablet, for example) you can point the browser to http://<hostname>:<port> where <hostname> is the hostname or IP address of the computer where Karma is running, and <port> is the port you specified in the configuration file. By default this is port 9876, but if you have changed it in the port section of your configuration file then use that port.
Files
You need to tell Karma which files are required to run your project, which files contain your tests, and which files need to be tested. For convenience, in my configuration file I have broken these files into their respective groups, with required files first, test files second, and files to be tested last. The order in which they appear is the order that they will be included in the browser, so depending upon your project you may need to pay attention to it. Each file or group of files can be flagged for being watched by Karma, included in the browser <script> tag, and served by the Karma server. Files that are watched can trigger Karma to run all of the tests again, so pay particular attention to these. If you simply list all of your files, they will be watched, included, and served, which may result in performance loss during testing on older, slower computers.
Karma runs on NodeJS and the configuration is set up using module.exports so that it gets drawn into and consumed by NodeJS. Older versions of Karma did not have this feature, and they were slightly harder to configure correctly. A good sample of a Karma configuration file looks like this:
module.exports = function(config) {
config.set({
// base path, that will be used to resolve files and exclude
basePath: '../..',
frameworks: ['jasmine'],
// list of files / patterns to load in the browser
files: ['test/client/mocks.js', 'static/karma.src.js', 'test/client/*.spec.js'],
// list of files to exclude
exclude: [],
// use dots reporter, as travis terminal does not support escaping sequences
// possible values: 'dots', 'progress'
reporters: ['progress', 'junit'],
// will be resolved to basePath (in the same way as files/exclude patterns)
junitReporter: {outputFile: 'test-results.xml'},
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN
|| config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: true,
// Start these browsers, currently available:
// - Chrome, ChromeCanary, Firefox, Opera, Safari (only Mac), PhantomJS, IE (only Windows)
browsers: [process.env.TRAVIS ? 'Firefox' : 'Chrome'],
// If browser does not capture in given timeout [ms], kill it
captureTimeout: 20000,
// Auto run tests on start (when browsers are captured) and exit
singleRun: false,
// report which specs are slower than 500ms
reportSlowerThan: 500,
// compile coffee scripts
preprocessors: {'**/*.coffee': 'coffee'},
plugins: ['karma-jasmine','karma-chrome-launcher',
'karma-firefox-launcher','karma-junit-reporter']
});
};
Running Karma
There are two main ways to run Karma: at the command line or via your IDE. I heavily favor running Karma from my IDE (WebStorm) because it keeps me on the same screen more. Regardless of which method you choose, running Karma does the following:
Test Results
Testing is only useful if you can see the results, and Karma has several options available. The progress reporter is inserted into the configuration file that the init process creates by default. This reporter gives you a complete listing of each test that is run, in the order that they are executed, and whether they pass or fail. The dots reporter is included with Karma for those that run Travis. There are other reporters available as plugins (growl, Junit, Teamcity and Coverage) that can help make your testing process easier and more complete. Each reporter must be listed in the reporters section of the configuration file, and some reporters (such as the coverage reporter) require that they be referenced in the preprocessors section as well. The coverage reporter that is included with Karma is Istanbul and I have found it very well suited for my needs and for use with Karma.
IDE Integration
Part of the beauty of Karma is how easily it can be integrated in to your IDE. Nearly all IDEs have a way to add an external tool that is launched on the command line. Configuring WebStorm 6 and older to run Karma as an external tool is covered in the introductory video that the creator (Vojta Jina) posted via YouTube (http://www.youtube.com/watch?v=MVw8N3hTfCI) and can be found at the 7:39 mark in the video. There is a handy link to this part just below the video. There is also integration built into the upcoming WebStorm 7 (currently accessible via the Early Access Program.) It has been my experience that integrating with WebStorm produces an excellent development environment. Support for configuring external tools isn't limited to JetBrains products. Visual Studio 2008 or newer in any flavor except the Express versions will allow you to configure external tools to launch from the menu (you would need to create a batch file with the requisite Karma commands in it and tell VS to launch that file.) Sublime is also a heavily favored text editor for web development, and it also supports external tools. As long as your favored IDE supports external tools in some way, you should be able to launch Karma and have it run tests.
Documentation
The vast majority of what you need to get up and running with Karma is contained in the Karma website. Most any other question can be answered by searching the Google Group or posting a question on Stack Overflow. Documentation for individual plugins will be found with those plugins. For anything that isn't covered, the Google Group is the best resource to try first, and Stack Overflow is a very close second.
Actual Experience
A large number of developers don't test their JavaScript, and they really should. Too many of them think that testing is hard to set up and hard to do, and Karma is proof of the opposite. My team and I use Karma with Jasmine every day of the week. We found the installation and configuration to be fast and easy to follow. We went from downloading NodeJS to running tests via Karma in under an hour. When you consider that my team and I were totally new to testing, let alone testing JavaScript, this is pretty impressive. We chose Karma because it had support for AngularJS without any configuration changes, and because it would let us test our other scripts as well. In each of these areas it has excelled from day #1. We are now preparing to move our project to Yeoman and have no fear about how our tests will work because Yeoman fully supports Karma. We firmly believe that we could not have chosen a better tool to include in our daily routines.
Summary
Karma is the preferred test runner for projects written with AngularJS and is well on its way to larger acceptance within the greater JavaScript community. Its plugin architecture makes it easily adaptable to other test suites and reporters, all of which add value to the core of Karma. In agile or continuous integration environments, Karma shines as an indispensable tool to development teams, providing an easy and reliable way to modify existing code and craft new code as part of TDD. It is rare that a day goes by without this tool running on my computer.
Unit Testing Articles
More Software Testing and JavaScript Knowledge
Software Testing Tutorials and Videos
JavaScript Tutorials and Videos
Click here to view the complete list of tools reviews
This article was originally published in the Fall 2013 issue of Methods & Tools