REST API Test Automation in Java with Open Source Tools

Vladimir Belorusets, PhD

Introduction

Nowadays, REST API testing has gained much deserved popularity due to the simplicity of verifying backend functionality. Availability of many open source tools and libraries supporting REST API development and testing in Java also added incentives to automate these tests before the GUI development is complete.

REST API regression test automation includes generating code for HTTP calls and comparing the server's actual response with the expected one. In my article "A Unified Framework for All Automation Needs - Part 2" [1], I described how to use the open source Spring Framework to generate REST calls and map JSON and XML responses to Java classes.

In this article, I will describe the automation process in detail. The intended audience is test automation engineers who need practical guidance in navigating through the large number of various tools available on the market. The article does not pretend to be a comprehensive overview of all the tools, but it describes a subset of the tools sufficient for successful and easy regression test automation.

The automation process consists of the following steps:

1. Compose a REST call manually with the help of a REST client tool.

In automating any test case, the first step is to perform that test case manually to see that the application produces the outcomes as expected. Only after that it makes sense to start writing the automation code. REST clients provide a fast and simple way to execute the REST calls manually and observe the responses in a convenient format.

2. Model the endpoint response into an object class using any object-oriented programming language of your choice.

That class will represent the JSON/XML tree structure and provide an easy access to the response values. In the next step, the object of that class will be used to store the response data in a file for future comparisons.

An alternative is to process the REST response as a string without mapping it to an object. Then, you need to apply a parser to that string to extract individual field values. This will make the response validation test much more complex and cumbersome.

3. Generate the same REST call programmatically and serialize the response object into a file as the expected result.

The serialized object represents the reference point. The regression test validates the condition that even after modifying the web service code, the REST call previously implemented continues to return the same data as in the stored object.

4. Create a script that issues the REST API call and compares the response object with the expected object after deserialization.

The comparison between actual and expected results is done via xUnit assertions.

Let's go through the set of tools that supports each step of the process outlined above.

REST Clients

There are plenty of the REST client tools for any taste. They allow generating REST calls without using a programming language and observe the response in JSON or XML format.

We can classify these tools in the following categories:

• Desktop REST client applications
• Web browser based REST clients
• Command line REST clients

Most of the REST clients allow saving REST calls for future reuse either as files or favorite links. They also support common HTTP request methods such as GET, POST, PUT, HEAD, DELETE, OPTIONS, etc.

Desktop Rest Clients

OS	Tool Name	Web Site
Windows	I'm Only Resting	http://www.swensensoftware.com/im-only-resting
Mac OS	CocoaRestClient	http://mmattozzi.github.io/cocoa-rest-client/
Any OS	WizTools.org RESTClient	https://github.com/wiztools/rest-client

Table 1. Desktop REST Clients.

The tools differ mostly in their GUI layout. As an example, 'I'm Only Resting' is presented here.

Figure 1. 'I'm Only Resting' HTTP Client

There are also REST client plugins for popular IDEs that allow you to speed up code development by testing REST API in a separate panel without leaving your IDE.

IDE	Tool Name	Web Site
Eclipse	RESTClient Tool	https://marketplace.eclipse.org/content/rest-client
IntelliJ IDEA (Community Edition)	RESTClient	https://code.google.com/p/restclient-idea-plugin/

Table 2. IDEs REST Plugins

The IntelliJ IDEA plugin above imitates the desktop WizTools.org RESTClient.

Browser REST Clients

Popular browsers have numerous open source REST clients developed as their plug-ins. Some of these are listed below.

Browser	Tool Name	Web Site
Firefox	RESTClient	https://addons.mozilla.org/en-US/firefox/addon/restclient/
Firefox	REST Easy	https://addons.mozilla.org/en-us/firefox/addon/rest-easy/
Chrome	Advanced REST Client	http://chromerestclient.appspot.com/
Chrome	Postman	https://www.getpostman.com/
Chrome	DHC - REST/HTTP API Client	http://restlet.com/products/dhc/

Table 3. Browser REST Clients

Firefox clients are presented as buttons on the browser's toolbar. All Google Chrome clients are accessible via the Chrome App Launcher.

As an example, the Chrome DHC client is presented below.

Figure 2. Chrome DHC Client.

Command Line REST Clients

Tool Name	Web Site
CURL	http://curl.haxx.se/
HTTPie	https://github.com/jkbrzt/httpie

Table 4. CLI REST Clients

The most widely used CLI client for REST calls is cURL which stands for 'Client for URLs'. It allows getting or sending files using URL syntax. In the example below cURL is used for HTTP POST in Windows.

Figure 3. An Example of Using cURL for HTTP POST.

HTTPie client is a user-friendly cURL replacement featuring intuitive commands, JSON support, and syntax highlighting. The same REST call we used with cURL is presented with HTTPie syntax on Figure 4. By default, the Content-Type header set to application/json, and data fields in the message body are presented in JSON format. When a request contains data fields, HTTPie sets the request method to POST.

Figure 4. An Example of Using HTTPie for POST.

Mapping JSON Response

Now it is time to develop the automated tests. Since Java is today the most popular programming language in the world ^[2], the implementation details are given in Java.

The first thing to consider is how to validate the response data with assertions. There are many Java libraries that generate HTTP requests and get the responses as strings: java.net package in Java SE and Apache HttpComponents (http://hc.apache.org/), just to name a few. However, string comparison is not a solution since some data may contain dynamic values, such as ids and dates. Each time you produce the REST calls, these data in the response body will be different and your assertions will fail.

One option is to parse the JSON response string and extract the interesting individual name/value pairs. A popular open source library that provides this capability is JSON.simple (https://code.google.com/p/json-simple/).

In this approach, the test case will consist of multiple assert methods. This is not a good design. It makes the test code longer and harder to perceive. If developers later add new fields, the test code must be updated, which should be avoided since the test logic stays the same and only the data have changed. The pattern to use is a single assertion on an expected object instead of one assertion per each object field ^[3]. It implies mapping the response to a Java class called POJO (Plain Old Java Object), with the class members corresponding to the JSON fields and using this object in assertion, assertEquals(expectedResponseObject, actualObject), given that assertion is customized to compare objects.

To process JSON response as a POJO, I recommend using the open source Spring Framework (http://projects.spring.io/spring-framework/). It comes with the built-in converters that map the web service responses into the relevant Java classes.

MappingJackson2HttpMessageConverter class converts JSON into a POJOs using the Jackson library (http://wiki.fasterxml.com/JacksonDownload), and GsonHttpMessageConverter class does the same with the gson library (https://github.com/google/gson). Both converters are very similar. For the rest of this article, I imply using Jackson. It provides simple data binding by converting JSON to Java data types, such as maps, lists, strings, numbers, booleans and nulls.

After reviewing the JSON response by using one of the REST clients described in the previous section, you need to create a POJO object that Jackson can convert the data to. Jackson requires having getters and setters for each response field.

The Spring Framework contains org.springframework.web.client.RestTemplate class that makes submitting REST calls and converting responses to Jackson POJOs very easy. For each HTTP request method, Spring offers corresponding RestTemplate method that includes Java class for expected JSON response as a parameter: getForObject(), postForObject(), etc.

You can create a POJO manually, which can be time consuming for the complex JSON responses. Open source tools that simplify this procedure are presented below. They generate POJOs online after you paste the JSON string. An example of the jsonschema2pojo web page is presented on Figure 6.

Tool Name	Web Site
Jsonschema2pojo	http://www.jsonschema2pojo.org/
JSON to POJO	http://boldijarpaul.github.io/jsontopojo/
Convert XML or JSON to Java Pojo Classes	http://pojo.sodhanalibrary.com/
Json to Java	http://jsontojava.sinaapp.com/

Table 5. JSON to POJOs Online Converters

The Jackson project also provides classes that can programmatically convert the JSON string into a POJO (https://github.com/FasterXML/jackson-docs/wiki/JacksonSampleSimplePojoMapper).

Figure 6. jsonschema2pojo Online POJO Generator.

Serializing Response Objects

The next step is to perform an end-point call programmatically and serialize the JSON response as the POJOs in files for future references and comparison. However, as I mentioned before, some of the response fields contain data varying from call to call.

To identify those fields, you can issue the same call a few times using one the REST clients. Then, you can compare JSON responses for differences with the help of the online tools listed below.

Tool Name	Web Site
JsonDiffPatch	http://benjamine.github.io/jsondiffpatch/demo/index.html
JSON Diff	http://tlrobinson.net/projects/javascript-fun/jsondiff/
JSON Diff View (Firefox)	https://addons.mozilla.org/en-us/firefox/addon/json-diff-view/
JSON Diff	http://jsondiff.com/

Table 6. Tools for Finding JSON Differences

There is no reason to store those data because the tests will always fail in assertions. A solution is to use the transient keyword for the corresponding POJO variables. The values of the transient fields will not be serialized. But we cannot ignore those fields completely since some of them are mandatory, and an important part of the tests is to guarantee that they are not nulls. Field validation can be performed on the backend as well as on the frontend. To address that issue on the client side, we can use the open source Hibernate Validator library (http://hibernate.org/validator/). It provides annotation-based constraints on class variables and transparent integration with the Spring Framework. The annotations of interest are: @NotNull, @Size(min=.., max=...), @Min(...), and @Max(...). You need to decorate mandatory POJO class variables with these annotations and test the response object using javax.validation.Validator.

Response Verification

The last step in the automated test includes comparison of the deserialized expected object from the file with the actual JSON response mapped to the POJOs. In JUnit, verification is accomplished by using assertions, such as assertEquals(). However, if the fields in the response class contain other objects, out-of-the-box assertions fail even when the objects being compared contain the same values. The reason is that comparison is conducted by using '==' operator applied to the reference variables without recursively checking all object fields. I recommend two libraries to compare object hierarchy field by field in one statement.

Library Name	Web Site
Unitils	http://unitils.org/
Shazamcrest	https://github.com/shazam/shazamcrest

Table 7. Recursive Assertion Libraries

Unitils assertReflectionEquals(expectedObject, actualObject) loops over all fields in both objects and compares their values using reflection. If a field value itself is an object, it will be recursively compared field by field. Reflection assert automatically excludes transient variables from comparison. You can also ignore non-transient values by specifying ReflectionComparatorMode.IGNORE_DEFAULTS mode in the assertReflectionEquals(). This mode excludes all fields that designated as nulls in the expected object.

Shazamcrest is a library that extends the functionality of Hamcrest assertions with a special matcher sameBeanAs(): assertThat(actualObject, sameBeanAs(expectedObject). The comparison is done field by field including fields in object variables. It also provides capability of ignoring fields from matching by specifying which fields to ignore as follows: sameBeanAs(expectedObject).ignore("field1").ignore("field2").

Code Samples

Let's assume as a result of GET request to http://www.myREST.com/person, JSON response will be mapped to the class Person using the Jackson converter. The explanation comments are provided within the code.

 public class Person
{
// using Hibernate Validator annotation
    @NotNull
    private transient String id;
    private String name;
    private Address address;

    public Person(String id, String name, Address address)
    {
        this.id = id;
        this.name = name;
        this.address = address;
    }
// getters and setters must be added
}

public class Address
{
    private String street;
    private String city;
    private int zip;

    public Address(String street, String city, int zip)
    {
        this.street = street;
        this.city = city;
        this.zip = zip;
    }
// getters and setters must be added
}

@Test
public void testREST()
{
// use Spring Framework for generating GET request
  RestTemplate restTemplate = new RestTemplate();
  Person actualPerson = restTemplate.getForObject(
       "http://www.myREST.com/person", Person.class);

// let's assume for illustration that we received the 
// following response object 
  actualPerson = new Person(null, "John Doe", 
	   new Address("1 Main St", "San Mateo", 94404));
// then validation using Hibernate Validator fails        
  assertTrue("Some required fields are nulls",
      RestUtils.validateObject(actualPerson)); 
/*
   the following error message will be displayed:
        java.lang.AssertionError: Some required fields are nulls <2 internal calls>
            at TestPerson.testREST(TestPerson.java:15) <23 internal calls>
        id may not be null
*/
// create your methods serialize to and deserialize from the file to extract the 
// expected POJO        
  Person expectedPerson = Utilities.deserialize("person.ser");
// let's assume that after de-serialization we get the following object        
  expectedPerson = new Person("1234", "John Doe", new Address("1 Main St", 
"San Mateo", 94404));
  
// both assertions pass since they ignore transient id
  assertReflectionEquals(expectedPerson, actualPerson); 
  assertThat(actualPerson, sameBeanAs(expectedPerson)); 

// if the expected object is as follows               
   expectedPerson = new Person("1234", "John Doe", new Address("1 Main St", 
"San Mateo", 94107));
        
   assertReflectionEquals(expectedPerson, actualPerson); 
/*
    the following error message will be displayed applying Unitils assertion
        junit.framework.AssertionFailedError:
        Expected: Person<, name="John Doe", address=Address>
        Actual: Person<, name="John Doe", address=Address>

        --- Found following differences ---
        address.zip: expected: 94107, actual: 94404
*/
   assertThat(actualPerson, sameBeanAs(expectedPerson)); 
/*
    the following message will be displayed applying Shazamcrest
        java.lang.AssertionError:
        Expected: {
            "name": "John Doe",
                    "address": {
                        "street": "1 Main St",
                        "city": "San Mateo",
                        "zip": 94107
            }
        }
        but: address.zip
        Expected: 94107
        got: 94404
       
*/
}

public class RestUtils
{
    public static  boolean validateObject(Type object)
    {
        Validator validator = 	
		Validation.buildDefaultValidatorFactory().getValidator();
        Set> violations = validator.validate(object);

        if (violations.size() > 0)
        {
            for (ConstraintViolation cv : violations)
            {
                System.out.println(cv.getPropertyPath() + " " + cv.getMessage());
            }
            return false;
        }
        return true;
    }
}

Summary

The article describes a set of open source tools that give testers of all levels ability to conduct REST API verification. Manual testers can use REST clients that provide convenient GUI for hitting the end-points while test automation engineers with Java experience can develop comprehensive tests fast. Although the choice of tools is subjective, I hope you enjoyed using them as I did.

References

1. Belorusets, Vladimir. A Unified Framework for All Automation Needs - Part 2. Testing Experience, Issue No 27, pp. 9-13, 2014.

2. TIOBE Index for October 2015 (http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html)

PYPL PopularitY of Programming Language (http://pypl.github.io/PYPL.html)

3. Meszaros, Gerard. xUnit Test Patterns. Addison-Wesley. 2007.

---

More Software Testing and Java Resources

Software Testing Magazine

Software Testing Tutorials and Videos

Java Tutorials and Videos

---

Click here to view the complete list of archived articles

This article was originally published in the Winter 2015 issue of Methods & Tools