CatJS - Testing HTML and JavaScript

Software Development Magazine - Project Management, Programming, Software Testing

Scrum Expert - Articles, tools, videos, news and other resources on Agile, Scrum and Kanban

CatJS - Testing HTML and JavaScript

Arik Levin, Ran Snir, Lior Reuven, Eli Mordechai,
@catjsteam, @lastboy13, https://github.com/catjsteam

CatJS is an open source automation framework with a new approach for testing your HTML and JavaScript based web applications on multiple environments such as browsers and mobile devices. With CatJS the developer is not forced to work with a specific method and can integrate his tests on any application creation phase. CatJS only provides the ground and tools for creating and controlling the required tests execution. This results with quality application creation from day one.

Web Site: http://catjsteam.github.io/
Version tested: 0.3.4
System Requirements: Node.js 0.8.0 version or higher
License & Pricing: Apache-2.0
Support: Github Issue tracker: https://github.com/catjsteam/catjs/issues,
GoogleForum: https://groups.google.com/forum/#!forum/catjs

Testing is important

Your application should be tested and not because your manager ask you to, but only because you know that it will help you making less mistakes and better than that, it will help you not to make the same mistakes twice. Embracing that approach will reduce the development / Quality Assurance (QA) lifecycle, meaning quality with faster application creation.

Introduction

Today you can find a few leading technologies that you can use for testing your applications, but with CatJS new methodology and approach you can really get to test your web application and still have a dynamic and rapid development environment, with full End To End (E2E) support. Forget the old style when you have to move between environments and have a cumbersome lifecycle between the development and the QA. With CatJS the developer creates the UI test units, assertions, functional testing side by side with his source code and with that he gets to run the applications on multiple browsers and devices. The applications test themselves, so no need to install special drivers. That makes CatJS framework to be more than just an automation-testing framework.

Developer Zone

One of many things that the developer does before he get to code his new web application, is to choose the right third-party libraries and\or frameworks that will help him to meet his requirements faster and with high quality. He chooses the best IDE he can get so he doesn't waste any time on refactoring the code or looking for the right dependency. But when it comes to test his code, the developers tends to neglect this section, maybe because it’s not a rapid environment or not that handy. Finally, if he decides to create tests or just because his manager forced him to, he then needs to maintain those tests. Because the rapid changes that any web developer experiences, after couple of development lifecycles, usually the developers will comment out the tests code since it’s become obsolete. They have to make it to the production on time, they have no more time to waste on test refactor and that leads to square one. And I don’t even start talking about the QA lifecycle that makes it even more impossible. Think of the amount of man-hours that needs to be invested in such processes. That’s why we have decided to create a new framework that will normalize the burden of those processes and heaviness of creating and running tests scenarios.

With the CatJS technology, the developer can create his tests side by side with his source application code. No one knows the code better than the developer himself, thus he can creates tests fast and easy in the same breath he creates his application’s code and any changes to his code can be tested and verified immediately.

The Technology

How the developer can have such environment that he gets to create his tests side by side with his application's source project? Think of an area in the code that the interpreter will not compile as an actual runnable code. In most of the programming languages comments can be inserted into code and this is a good place for adding annotations. Using the comments section for adding annotations, helped to obtain an environment that the developer can code his application side by side with a test description, without messing his master branch. Another fact, in JavaScript the comments tag section can be removed on the production phase using a minification process. Having that kind of technology combined with the JavaDoc annotations style enabled a way of describing the test within comments. Here is an example of a single test unit that logs out to the console: “This is my first test”

/*
   @[scrap
     @@name test
     @@log “This is my first test”
   @]
*/

You can see that the main annotation block name is “scrap”, let's try to understand, what is a scrap? Scrap is a single test unit that can stand for a UI test operation or an assertion or combining the two together. It's a single test unit that can be later declared as a part of a scenario.

Let's go back to the technologic solution, until this point it's about having scrap annotations within comment. But having just annotations within the code will not achieve anything meaningful. Those annotations need to be interpreted. For that CatJS tool comes to action by creating and maintaining a simple process where it creates a new test deployment that includes the source application code with the additional generated tests functionality and the injected calls to that functionality. All the required libraries and resources are also included in that new deployment.

CatJS - Testing HTML and JavaScript

Figure 1 - CatJS process for creating a new test deployment

Annotations technology and Aspect Oriented Programming (AOP) implementations are available almost for any programming language. Therefore, CatJS technology theoretically can be applied on any development environment. In this release we are focused on client side, web (mobile) application development environments, with pure HTML\JavaScript implementations. Other technologies such as PHP or Java web frameworks that render the HTML\JavaScript code on the server side can use CatJS solution. But there is one obstacle, not like in the client side projects, the HTML\JavaScript code is decentralized and thus will be hard to maintain. Native languages such as Java, C-sharp and others are not supported and are not scheduled for the near future. NodeJS might be another candidate that we might consider to add to our future development list.

Applications that test themselves

With a new test deployment, the application has everything it needs to be able to test itself. All you have to do is call it like you call the original application. Go to the URL in the address bar and this is the interesting part, where the application reads the test configuration file that contains the knowledge about which scenarios should be run and some other test related properties.

Library & Tool

CatJS has two technology parts, the first is the tool that is responsible to work with the project sources, create the new test deployment, run the server, etc. The second part is about the JavaScript library that is responsible for loading the test project, executing the tests, displaying the UI, communicates with CatJS server and more.

E2E solution

With CatJS, an E2E solution can be obtained using the old and available tools. We didn't try to invent the wheel, if something works then why changing it?

Build
CatJS has a built-in task runner that can be used. The Grunt task runner is supported as well.
Runner
With CatJSRunner (https://www.npmjs.org/package/catjsrunner) the tests can be run on multiple devices and browsers.
Reports
CatJS generates a JUnit XML that can be integrated with Jenkins or a report can be generated using XSLT. It will be supported in the next releases.

Installation

CatJS is based on NodeJS technology it means that you have to install NodeJS first. In fact CatJS is a node-packaged module that you can easily install using the Node Packaged Modules (NPM) command that is part of NodeJS technology. From the command line call this command:

npm install -g catjs

-g flag stands for global, it means that you'll get to have catcli command in your system path.

Getting Started

A good place to start will be the CatJS module (https://www.npmjs.org/package/catjs) that has all the information as how to get started including how to install, step by step, how to create a new test and link to a video training (https://t.co/vBgLx9tEFF). CatJS site (http://catjsteam.github.io) includes all of our resources including blogs, examples, docs and other information. Other than that use NodeJS site for any NPM related or other troubleshooting that you might have and you always can contact us for any issue that you encounter.

Features

Annotations
CatJS is a plugin-based framework and can be easily extend to support other technologies. The common annotations give you the tools to create almost any test but a special plugins with syntactic sugar was created that supports: JQM, Sencha Touch and Enyo.
Test Data
Test data support is already included with the testdata.json file that can be used to hold the test application related data and later in the tests can be embedded with a proper API.
Reports
JUnit XML file is supported mainly for Jenkins integration.
Runner
The runner execution has a cross platform support for Chrome, Firefox and Safari browsers. In the devices section, IOS and Android are supported. In additional PhantomJS, a headless Webkit engine, is also supported, but it needs to be installed manually.
Assertions
ChaiJS library is supported for assertions evaluation. Other library can be supported with minimal effort if such requirement is needed.

Future Development

Annotations
AngularJS support
Reports
A web report application support
Developer Tools Tab
New development for creating a developer toolbar for Chrome browser to enable an instant test development environment
Administration Application
An administration view for easy and fast tests maintenance
Parser
The parser will be improved. Currently it's a bit strict and has special syntax support.

Summary

From my vast experience as a web application expert developer for the desktop and mobile on varied environment and methodologies, looking at the big picture, testing your application side by side with the creation process is a time saver by far than the traditional development QA lifecycle. CatJS is a good start to achieve that kind of environment and we can get better with your feedback. Let's make web (mobile) better.

Further Reading

Applications that tests themselves

How to test your application

Advanced testing techniques with CatJS

CatJS Meetup

Resources

http://en.wikipedia.org/wiki/Annotation

http://en.wikipedia.org/wiki/Minification_(programming)

http://en.wikipedia.org/wiki/Javadoc

http://nodejs.org/

https://www.npmjs.org/

More Software Testing Knowledge