Thursday 13 March 2014

Redis Publish Subscribe and Long Polling with Spring's DeferredResult

As well as being key value store, Redis offers a publish subscribe messaging implementation. This post will describe a simple scenario, using Spring Data Redis, of adding a message domain object to a repository via a REST call, publishing that message to a channel, subscribers to that channel receiving that message who as a result set any long polling deferred results with the message.

The two key classes in the Redis publish subscribe mechanism are the RedisTemplate class and the RedisMessageListenerContainer class.

The RedisTemplate contains the JedisConnectionFactory which holds the Redis connection details and as well as the methods to manipulate the key value stores, there’s a publish method called convertAndSend. This method takes two arguments. The first being the channel name of where the messages need to be published to and the second being the object to be sent. 

In this example, the publishing of the message is done after the Message is persisted via an aspect.

@Aspect
@Component
public class MessageAspect extends AbstractRedisAspect {

    private static final Logger LOGGER = LoggerFactory
            .getLogger(MessageAspect.class);

    @Value("${messaging.redis.channel.messages}")
    private String channelName;

    @After("execution(* com.city81.redisPubSub.repository.MessageDao.save(..))")
    public void interceptMessage(JoinPoint joinPoint) {
            
        Message message = (Message) joinPoint.getArgs()[0];
    
        // this publishes the message
        this.redisTemplate.convertAndSend(channelName, message);

    }

}

The RedisMessageListenerContainer, as well as holding the JedisConnectionFactory, holds a map of message listeners where the key is a message listener instance and the value the channel. The message listener instance references a class which implements the onMessage method of the MessageListener interface.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:aop="http://www.springframework.org/schema/aop"
       xmlns:p="http://www.springframework.org/schema/p"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/aop
        http://www.springframework.org/schema/aop/spring-aop-3.2.xsd" >

    <!-- for the redis pub sub aop beans -->
    <aop:aspectj-autoproxy />

    <bean id="jedisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"
          p:host-name="${messaging.redis.hostname}" p:port="${messaging.redis.port}"/>

    <bean id="redisTemplate" class="org.springframework.data.redis.core.RedisTemplate"
          p:connection-factory-ref="jedisConnectionFactory">
    </bean>

    <bean id="messageListener" class="org.springframework.data.redis.listener.adapter.MessageListenerAdapter">
        <constructor-arg>
            <ref bean="messageManager"/>
        </constructor-arg>
    </bean>

    <bean id="redisContainer" class="org.springframework.data.redis.listener.RedisMessageListenerContainer">
        <property name="connectionFactory" ref="jedisConnectionFactory"/>
        <property name="messageListeners">
            <map>
                <entry key-ref="messageListener">
                    <bean class="org.springframework.data.redis.listener.ChannelTopic">
                        <constructor-arg value="${messaging.redis.channel.messages}" />
                    </bean>
                </entry>
            </map>
        </property>
    </bean>

</beans>

When a message is published, those subscribers who are listening to that channel will then receive the published message via the onMessage method. The published message contains the serialised object that was sent in the body of the Redis Message and needs to be deserialised and cast to the original object.

    public void onMessage(
            org.springframework.data.redis.connection.Message redisMessage,
            byte[] pattern) {

        Message message = (Message) SerializationUtils.deserialize(redisMessage.getBody());
        
        // set the deferred results for the user
        for (DeferredResult<Message> deferredResult : this.messageDeferredResultList) {
                deferredResult.setResult(message);
        }

    }

The DeferredResult list is populated by calls to the REST service's getNewMessage method. This will in turn, in the MessageManager, create a DeferredResult object, add it to the list and return the object to the client.


    public DeferredResult<Message> getNewMessage() throws Exception {

        final DeferredResult<Message> deferredResult =
                new DeferredResult<Message>(deferredResultTimeout);

        deferredResult.onCompletion(new Runnable() {
            public void run() {
                messageDeferredResultList.remove(deferredResult);
            }
        });

        deferredResult.onTimeout(new Runnable() {
            public void run() {
                messageDeferredResultList.remove(deferredResult);
            }
        });

        messageDeferredResultList.add(deferredResult);

        return deferredResult;
    }


The GitHub repo for this example contains two simple HTML pages, one which starts a long poll request and another which adds a message. These will call the below REST web service.

@Controller
@RequestMapping("/messages")
public class MessageAPIController {

    @Inject
    private MessageManager messageManager;

    //
    // ADD A MESSAGE
    //
    @RequestMapping(value = "/add", method = RequestMethod.POST,
            produces = "application/json")
    @ResponseBody
    public Message addMessage(
            @RequestParam(required = true) String text) throws Exception {
        return messageManager.addMessage(text);
    }
    
    //
    // LONG POLLING
    //
    @RequestMapping(value = "/watch", method = RequestMethod.GET,
            produces = "application/json")
    @ResponseBody
    public DeferredResult<Message> getNewMessage() throws Exception {
        return messageManager.getNewMessage();
    }
    
    
}


A further enhancement to the above to ensure messages aren't missed in between long polling requests would be to store the messages in Redis in a sorted set with the score being the message's creation timestamp. The Redis publish mechanism could then be used to tell the subscriber that there are new messages in Redis and it could then retrieve them based on the time of the last request, and return a collection of messages back to the client in the DeferredResult object.






Friday 6 December 2013

Spring and Caching JMS Connections

As follow up to previous posts covering JMS, this post will delve into more depth on Spring's CachingConnectionFactory.

Spring provides two implementations of the javax.jms.ConnectionFactory interface, namely, the SingleConnectionFactory and the CachingConnectionFactory. The SingleConnectionFactory returns as you might expect the same single connection upon all calls to the createConnection() method. This is fine for certain scenarios and applications but the CachingConnectionFactory provides a more performant and scalable solution.

By default, a single session is cached so for a multi threaded application you would set the sessionCacheSize to be a more suitable number although this number wouldn't reflect the true number of sessions cached as this figure refers to the size of cache per session acknowledgement type eg AUTO_ACKNOWLEDGE, CLIENT_ACKNOWLEDGE, DUPS_OK_ACKNOWLEDGE and SESSION_TRANSACTED.

By default, the CachingConnectionFactory will cache the Message Producers and Message Consumers for every session. As an aside the Message Consumers are cached using keys which include the JMS selector so the more fine grained the message filter the more Message Consumers there would be, and Message Consumers aren't closed until the session is closed and removed from the pool. An alternative is to use a Listener Container for consuming messages.

Also to be noted is that on creating a CachingConnectionFactory instance, the reconnect on exception flag is set to be true. This should mean that the onException method on the default ExceptionListener class gets called which will reset the connections. You can also override the default exception listener with your own implementation.

The below snippet of XML shows a simple configuration of a CachingConnectionFactory:

<bean id="jmsQueueConnectionFactory"
    class="org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter">
    <property name="targetConnectionFactory" ref="mqConnectionFactory" />
    <property name="username" value="${mq.username}" />
</bean>

<bean id="jmsConnectionFactory" 
    class="org.springframework.jms.connection.CachingConnectionFactory">
    <property name="targetConnectionFactory" ref="jmsQueueConnectionFactory" />
    <property name="sessionCacheSize" value="50" />
    <property name="exceptionListener" ref="jmsExceptionListener" />
</bean>

<bean id="jmsExceptionListener"
    class="com.city81.messaging.MessageMQExceptionListener"">
    <property name="cachingConnectionFactory" ref="jmsConnectionFactory" /">
</bean>

Monday 3 June 2013

Exposing and Managing Links with Spring HATEOAS

As a follow up to the previous Spring HATEOAS post, this post will cover how to use the @ExposesResourceFor annotation as an alternative to using the resource's Controller to obtain links.

It'll also cover how to find links based on a particular relation type in a hypermedia enabled representation.

In addition to the classes in the previous post, the SettlementController class is responsible for settling bets and exposes one method, settleBet, in order to do this. To return a link to the settled bet, the SettlementController uses the linkTo method on the ControllerLinkBuilder class:

@RequestMapping(method = RequestMethod.PUT, value = "/{betId}")
ResponseEntity<SettlementResource> settleBet(@PathVariable Long betId) {

    Settlement settlement = this.settlementService.settleBet(betId);  
    SettlementResource resource = 
        settlementResourceAssembler.toResource(settlement);
    resource.add(
        linkTo(BetController.class).slash(betId).withSelfRel());
    return new ResponseEntity<SettlementResource>(resource, HttpStatus.CREATED);
}  

An alternative approach to this is to enable EntityLinks. By adding the @ExposesResourcesFor annotation to the BetController, the SettlementController need only know about the resource and not it's Controller class.

@Controller
@ExposesResourceFor(Bet.class)
@RequestMapping("/bets")
public class BetController {
...
}

The SettlementController by way of the EntityLinks interface can now obtain the link to a single resource or a link to a resource collection. It can either get the Link or a LinkBuilder. With the latter approach, the relation types can be overwritten:


@Controller
@RequestMapping("/settlements")
public class SettlementController {

    private SettlementService settlementService;
    private SettlementResourceAssembler settlementResourceAssembler;
    private EntityLinks entityLinks;

    @Autowired
    public SettlementController(SettlementService settlementService,
        SettlementResourceAssembler settlementResourceAssembler, EntityLinks entityLinks) {
        this.settlementService = settlementService;
        this.settlementResourceAssembler = settlementResourceAssembler;
        this.entityLinks = entityLinks;
    }

    @RequestMapping(method = RequestMethod.PUT, value = "/{betId}")
    ResponseEntity<SettlementResource> settleBet(@PathVariable Long betId) {

        Settlement settlement = this.settlementService.settleBet(betId);
        SettlementResource resource = 
            settlementResourceAssembler.toResource(settlement);
        resource.add(this.entityLinks.linkToSingleResource(Bet.class, betId));
        return new ResponseEntity<SettlementResource>(
            resource, HttpStatus.CREATED);
    } 
 
}

For a collection link, the call would be:

resource.add(this.entityLinks.linkToCollectionResource(Bet.class)); 

To obtain the builders:

LinkBuilder linkFor = this.entityLinks.linkFor(Bet.class);
LinkBuilder linkForSingleResource = 
    this.entityLinks.linkForSingleResource(Bet.class, betId);

To enable all this functionality, the @EnableEntityLinks annotation must be in your Spring configuration.

Another useful feature that comes with Spring HATEOAS, is the ability to discover links. For a given hypermedia enabled representation, the Link Discoverer API can find a link or links for a particular relation type. For example, from the previous post, to find the cancelBet relation type link of a ResourceSupport object would be:

LinkDiscoverer discoverer = new DefaultLinkDiscoverer();
Link link = discoverer.findLinkWithRel("cancelBet", resource.toString());

The DefaultLinkDiscoverer will expect JSON but since 0.5 of Spring HATEOAS, the HAL variant has been supported via the HalLinkDiscoverer class.

Although still evolving, the Spring HATEOAS project provides a good foundation for implementing a HATEOAS compliant REST service. But if all you require is to expose and manage domain objects via a REST web service then the Spring Data REST project which applies the HATEOAS constraint to persisted entities is worth considering and the Restbucks example an ideal tutorial.


Thursday 9 May 2013

Spring MVC and the HATEOAS constraint

HATEOAS is a REST architecture principle where hypermedia is used to change application state. To change state, the returned resource representation contains links thereby 'constraining' the client on what steps to take next.

The Spring-HATEOAS project aims to assist those writing Spring MVC code in the creation of such links and the assembly of resources returned to the clients.

The example below will cover a simple scenario showing how links are created and returned for the resource, Bet. Each operation on the resource is described below:

  • createBet - this POST operation will create a Bet.
  • updateBet - this PUT operation will update the Bet.
  • getBet - this GET operation will retrieve a Bet.
  • cancelBet - this DELETE operation will cancel the Bet.

@Controller
@RequestMapping("/bets")
public class BetController {

 private BetService betService;
 private BetResourceAssembler betResourceAssembler;

 public BetController(BetService betService,
   BetResourceAssembler betResourceAssembler) {
  this.betService = betService;
  this.betResourceAssembler = betResourceAssembler;
 }

 @RequestMapping(method = RequestMethod.POST)
 ResponseEntity<BetResource> createBet(@RequestBody Bet body) {
  Bet bet = betService.createBet(body.getMarketId(),
    body.getSelectionId(), body.getPrice(), body.getStake(),
    body.getType());
  BetResource resource = betResourceAssembler.toResource(bet);
  return new ResponseEntity<BetResource>(resource, HttpStatus.CREATED);
 }

 @RequestMapping(method = RequestMethod.PUT, value = "/{betId}")
 ResponseEntity<BetResource> updateBet(@PathVariable Long betId,
   @RequestBody Bet body) throws BetNotFoundException, BetNotUnmatchedException {
  Bet bet = betService.updateBet(betId, body);
  BetResource resource = betResourceAssembler.toResource(bet);
  return new ResponseEntity<BetResource>(resource, HttpStatus.OK);
 }

 @RequestMapping(method = RequestMethod.GET, value = "/{betId}")
 ResponseEntity<BetResource> getBet(@PathVariable Long betId) throws BetNotFoundException {
  Bet bet = betService.getBet(betId);
  BetResource resource = betResourceAssembler.toResource(bet);
  if (bet.getStatus() == BetStatus.UNMATCHED) {
   resource.add(linkTo(BetController.class).slash(bet.getId()).withRel("cancel"));
  }
  return new ResponseEntity<BetResource>(resource, HttpStatus.OK);
 }

 @RequestMapping(method = RequestMethod.GET)
 ResponseEntity<List<BetResource>> getBets() {
  List<Bet> betList = betService.getAllBets();
  List<BetResource> resourceList = betResourceAssembler.toResources(betList);
  return new ResponseEntity<List<BetResource>>(resourceList, HttpStatus.OK);
 }

 @RequestMapping(method = RequestMethod.DELETE, value = "/{betId}")
 ResponseEntity<BetResource> cancelBet(@PathVariable Long betId) {
  Bet bet = betService.cancelBet(betId);
  BetResource resource = betResourceAssembler.toResource(bet);
  return new ResponseEntity<BetResource>(resource, HttpStatus.OK);
 }

 @ExceptionHandler
 ResponseEntity handleExceptions(Exception ex) {
  ResponseEntity responseEntity = null;
  if (ex instanceof BetNotFoundException) {
   responseEntity = new ResponseEntity(HttpStatus.NOT_FOUND);
  } else if (ex instanceof BetNotUnmatchedException) {
   responseEntity = new ResponseEntity(HttpStatus.CONFLICT);
  } else {
   responseEntity = new ResponseEntity(HttpStatus.INTERNAL_SERVER_ERROR);
  }
  return responseEntity;
 }
 
}

 
All the operations will create a BetResource for returning to the client. This is done by calling toResource on the BetResourceAssembler class:

public class BetResourceAssembler extends ResourceAssemblerSupport<Bet, BetResource> {

 public BetResourceAssembler() {
  super(BetController.class, BetResource.class);
 }

 public BetResource toResource(Bet bet) {
  BetResource resource = instantiateResource(bet);
  resource.bet = bet;
        resource.add(linkTo(BetController.class).slash(bet.getId()).withSelfRel());
  return resource;
 }

}


This class extends ResourceAssemblerSupport which requires the implementation of a toResource method as it implements the ResourceAssembler interface. This is where the mapping between Bet and BetResource is done. In this case, BetResource is just a wrapper for Bet so it is simply a case of setting the bet attribute. The instantiateResource method will return a BetResource without any links so links can be added at this point if required. In this example a link to self is added. An alternative approach would be to use createResourceWithId which will return a BetResource with the self link. 
 
public class BetResource extends ResourceSupport {
 
 public Bet bet;
 
}

 
Also in this example, links are added to the BetResource within the BetController class to ensure the application of the HATEOAS constraint. If the REST service receives a GET request then a check is made on the status of the Bet. If the Bet is UNMATCHED, then a link to cancel the Bet can be added to the BetResource. This is done in similar fashion to the self link but with the relationship attribute name of cancel.
 
An alternative approach to this is to build a link to a method as opposed to constructing a URI. 
 
resource.add(linkTo(methodOn(BetController.class).cancelBet(betId))
.withRel("cancel")); 
 
The methodOn would create a proxy of the BetController class and as a result the return type of the cancelBet method would have to be capable of proxying. Therefore in this example the return type of cancelBet method would be HttpEntity<Bet> and not ResponseEntity<Bet>. If the latter, then the likely exception from the server would be:
 
[org.springframework.http.ResponseEntitycom.city81.hateoas.rest.BetResource> com.city81.hateoas.controller.BetController.getBet(java.lang.Long) throws com.city81.hateoas.BetNotFoundException]:org.springframework.aop.framework.AopConfigException: Could not generate CGLIB subclass of class [class org.springframework.http.ResponseEntity]: common causes of this problem include using a final class or a non-visible class; nested exception is java.lang.IllegalArgumentException: Superclass has no null constructors but no arguments were given
 
Back to the GET request, and the returned JSON for requesting a Bet resource which has a status of UNMATCHED is shown below:

{
 "links":[
     {"rel":"self","href":http://localhost:8080/hateoas-1-SNAPSHOT/bets/0},
     {"rel":"cancel","href":http://localhost:8080/hateoas-1-SNAPSHOT/bets/0}
],
 "bet":{"id":0,"marketId":1,"selectionId":22,"price":4.0,"stake":2.0,"type":"BACK","status":"UNMATCHED"}
}

The client can therefore use the self link for retrieving and updating the Bet, and also the cancel link to effectively delete it.

This post describes just some of the functionality of the Spring-HATEOAS project which is evolving all the time. For an up to date and more detailed explanation, visit the GitHub pages.
 
 
 
 

Thursday 4 April 2013

Mocking Static Methods in Groovy

Using Groovy to test not only other Groovy classes but also Java classes is an increasing popular approach given frameworks like Spock which facilitate Test Driven and Behaviour Driven Development.

A common problem when testing though is having to deal with legacy code and more often than not having to mock static methods calls. When writing tests in Groovy, the approach to mocking static calls will depend on the type of class you're testing and what type of class has the static method in.

If the Groovy class you're testing makes calls a static method on another Groovy class, then you could use the ExpandoMetaClass which allows you to dynamically add methods, constructors, properties and static methods. The below Account Groovy class has a static method called getType(). The Customer Groovy class you're trying to test calls the getType() static method within it's getAccountType() method:


class Customer {

 String getAccountType() {
  return Account.getType();
 }
 
}

class Account {

 static String getType() {
  return "Personal";
 }

}

class CustomerTest extends GroovyTestCase {
 
 void testGetAccountType() {
  
  Customer cust = new Customer()
  assert cust.getAccountType() == "Personal"

  Account.metaClass.static.getType = {return "Business"}
  
  assert cust.getAccountType() == "Business"
    
 }

}

The CustomerTest Groovy class shows how the getType static method can be overridden using the metaClass property. This isn't strictly mocking, more dynamically changing a class. An alternative approach is to use Spock's GroovyMock to achieve a similar result. (A more detailed description of Spock's mocking and stubbing features can be found here:)


import spock.lang.*

class CustomerTest extends Specification {
 
 def "get account type"() {
   
  setup: 
   GroovyMock(Account, global: true)
   Account.getType() >> "Business"
   Customer cust = new Customer()
  
  when: "obtaining a customer's account type"
   def type = cust.getAccountType()

  then: "the type will be a Business account"
   type == "Business"
    
 }

}

If the Account class is a Java not Groovy class then we can still mock it out using the above methods. The problem with mocking Java static methods though, when using Groovy for testing, is when they are called from Java classes and these calling classes are the ones you are trying to test.

For example, if we now have a Customer Java class and also an Account Java class, and a CustomerTest Groovy class, the ExpandoMetaClass and GroovyMock approaches will not work globally but will work locally. The below CustomerTest will pass.


public class Customer {

 public String getAccountType() {
  return Account.getType();
 }
 
}

public class Account {

 public static String getType() {
  return "Personal";
 }

}
class CustomerTest extends GroovyTestCase {
 
 void testGetAccountType() {
  
  Customer cust = new Customer()
  assert cust.getAccountType() == "Personal"

  Account.metaClass.static.getType = {return "Business"}
  
  assert Account.getType() == "Business"
  assert cust.getAccountType() == "Personal"
    
 }

}

The way around this is to use a mocking framework like Powermock or JMockit. The below amended CustomerTest Groovy class shows how Powermock can be integrated into a GroovyTestCase:


import groovy.mock.interceptor.*
import org.powermock.api.mockito.PowerMockito;
import static org.mockito.Mockito.*;
import org.powermock.core.classloader.annotations.PrepareForTest
import org.junit.runner.RunWith;
import org.powermock.modules.junit4.PowerMockRunner

@RunWith( PowerMockRunner.class )
@PrepareForTest(Account.class)
class CustomerTest extends GroovyTestCase {
 
 public void testGetAccountType() { 
  
  Customer cust = new Customer()
  assert cust.getAccountType() == "Personal"

  PowerMockito.mockStatic(Account.class);
  when(Account.getType()).thenReturn("Business");
    
  assert cust.getAccountType() == "Business"
 
 }

}
Using Mockito's when to mock the return value of the static getType() call enables "Business" to be returned instead of "Personal". This example shows how when writing Groovy tests to test Java classes, using a framework like Powermock can fill the void of mocking Java static methods when called from the Java classes you are trying to test.

Wednesday 27 February 2013

JSON, POJOs and Groovy's HTTPBuilder class


Whilst previous posts have looked at the use of RESTClient, this post will concentrate on its parent class, HTTPBuilder and also touch briefly on the JSONBuilder class. This post will cover the building and parsing of JSON, converting JSON to and from POJOs, adding authentication headers and handling response success and failures.

The HTTPBuilder class provides an API to make HTTP requests and parse the responses, and builds upon the Apache AbstractHttpClient class. The HTTPBuilder class can be instantiated with a URI to be used as default for all request methods. A default content type can also be supplied but for this example it will be the instantiated default of ContentType.ANY.

The below method takes an AccountFilter POJO, creates a JSONBuilder object and instantiates a HTTPBuilder with the default URI. It then makes a request specifying the method (POST) and contentType (JSON). The request body will be converted to a url-encoded form string. The code also shows how headers can be set, in this case, X-Application and X-Authentication using instance variables.


NOTE: The request is retrieving data and not creating or updating data, so would normally be a GET but the criteria can be complex and nested and therefore needs to be sent as JSON in the request body, hence the use of POST. (There is whole discussion on the web about GET requests containing a request body.) 


Set<Customer> listCustomers(AccountFilter accountFilter) {

    def resultList = new ArrayList<Customer>()
    def builder = new JsonBuilder(accountFilter)
    def http = new HTTPBuilder( 'https://beta-api.city81.com/json-rpc' )

 
    // POST request
    http.request (POST, JSON){req->
        body = [
            "jsonrpc" : "2.0",
            "method" : "Bank/v1.0/listCustomers",
            "params" : builder.toString(),
            "id" : 1
        ]

        headers.'X-Application' = this.applicationKey
        headers.'X-Authentication' = this.sessionToken

        // success response handler 
        response.success = { resp, json ->
            json.result.each { 
                resultList.add(new Customer(
                    it.customer.id, it.customer.name))
            }
        }

        // failure response handler 
        response.failure = { resp ->
            println "Unexpected error: ${resp.statusLine.statusCode}"
            println ${resp.statusLine.reasonPhrase}
        }
    }

    return resultList
}

The success response handler will iterate over the JSON result set and create Customer objects adding them to a collection. The failure response handler will print out the status code and reason phrase for the failure of the POST request.

As mentioned above, the code uses JSONBuilder to turn the POJO into JSON. For a Filter POJO with the field accountIds set, the JSON would appear as below:


"params":{"filter":{"accountIds":[1]}}

There is a flaw with this though in that if the filter attribute of MarketFilter is not set then the Filter tag will not be present in the JSON. If the service you are POSTing to requires this as its mandatory (albeit empty) then the req body params line would be:

"params" : [filter:[:]],

which would generate JSON:

"param": {"filter": {} }

This post covers only a small part of HTTPBuilder and one of a number of ways of POSTing a request, generating the JSON and parsing its response, many of which have been simplified with the RESTClient class.

The POJOS used in the above example:

public class Customer {

    private String id;
    private String name;

    public Customer(String id, String name) {
        this.id = id;
        this.name = name;
    }
   
    // ... getters and setters
 }


 public class AccountFilter {

    private Filter filter;
  

    public AccountFilter(Filter filter) {
        this.filter = filter;
    }
   
    // ... getters and setters
 }


public class Filter {
  

    private Set<String> accountIds;

    // ... getters and setters
}

Monday 26 November 2012

Groovy's RESTClient with Spock Extensions

Groovy has an extension to its HTTPBuilder class called RESTClient which makes it fairly easy to test a RESTful web service. This post will describe how to use the RESTClient, how to inject it into a Groovy Spock test class via custom annotations, how to swap the injected client between a real one and a mocked one using Mockito, and how to test it.

The RESTClient class provides methods to perform the HTTP's GET, POST, PUT and DELETE methods. The RESTClient instance can be constructed with a url string and optionally a content type. The HTTP methods can then be called with a named parameter list. For example, the below code will create a RESTClient instance and call the get method passing in the path and a path parameter:

def client = new RESTClient("http://localhost:8080/testWebService/")
def resp = client.get(path : "server/status/ServerOne")

The response returned is an instance of HttpResponseDecorator. This wraps the HttpResponse to allow for easy access to the data which is parsed depending upon the content type. The below class shows it's use within a Spock framework test and how the parsed data can be extracted from the response object. The returned XML is:

<serverstatus>
  <servername>ServerOne</servername>
  <isrunning>true</isrunning>
</serverstatus>

then

<serverstatus>
  <servername>ServerTwo</servername>
  <isrunning>false</isrunning>
</serverstatus>


import groovy.util.slurpersupport.GPathResult
import groovyx.net.http.RESTClient
import spock.lang.*
import groovyx.net.http.ContentType

class InjectedRestServiceTest extends Specification {
 
 @RESTWebClient
 def client
   
 def "Test Server Statuses"() {

  when: "retrieve server status"
  def resp1 = client.get(path : "server/status/ServerOne")
  def resp2 = client.get(path : "server/status/ServerTwo")
  
  then: "test server one response"
  assert resp1.data.serverName.text() == "ServerOne"
  assert resp1.data.isRunning.text() == "true"
  
  then: "test server two response"
  assert resp2.data.serverName.text() == "ServerTwo"
  assert resp2.data.isRunning.text() == "false"
  
 }
 
}

The test asserts are purely there in this example to prove that the response from the real web service are the same as from the mocked web service. In reality, you would be testing some other functionality of which calling a RESTful web service is part of the flow. You wouldn't really assert the response from a mocked RESTClient although you may want to verify that it has been called n times.

The InjectedRestServiceTest class has the RESTClient injected using the custom made annotation @RESTWebClient. There are three classes involved which enable this functionality. First the interface, which is marked as only visible at field level, contains an @ExtensionAnnotation specifying the class which basically tells Spock what to do when it encounters the @RESTClient annotation:

import java.lang.annotation.ElementType
import java.lang.annotation.Retention
import java.lang.annotation.RetentionPolicy
import java.lang.annotation.Target
import org.spockframework.runtime.extension.ExtensionAnnotation

@Retention(RetentionPolicy.RUNTIME)
@Target([ ElementType.FIELD ])
@ExtensionAnnotation(RESTWebClientAnnotationDrivenExtension)

public @interface RESTWebClient{}
The RESTWebClientAnnotationDrivenExtension class in this example only needs to override the visitFieldAnnotation method of the AbstractAnnotationDrivenExtension class because of the annotation's target element type:

import org.spockframework.runtime.extension.AbstractAnnotationDrivenExtension
import org.spockframework.runtime.extension.AbstractMethodInterceptor
import org.spockframework.runtime.extension.IMethodInvocation
import org.spockframework.runtime.model.FieldInfo
import org.spockframework.runtime.model.SpecInfo

class RESTWebClientAnnotationDrivenExtension extends 
 AbstractAnnotationDrivenExtension<RESTWebClient> {

 private static final String url = 
  System.getProperties().getProperty("url")
 private static final boolean useMock = 
  Boolean.parseBoolean(System.getProperties().getProperty("useMock"))
 
 @Override
 void visitFieldAnnotation(RESTWebClient annotation, FieldInfo field) {
  def methodInterceptor = new RESTWebClientMethodInterceptor(field, url, useMock)
  methodInterceptor.install(field.parent)
 }
 
}

The above extension tells Spock that when it encounters @RESTClient, it must create an interceptor with the arguments denoting the field to which the RESTClient instance is to be set, the url of the web service and a boolean to specify if the mock is to be used or the real web service should be used. (The meaning of the two values could be combined to just have a url and if set then the real web service must be used.)

The interceptor will create either a mock RESTClient or a real one. It will do this when the client field needs to be set in the InjectedRestServiceTest class. If the real RESTClient is required then an instance using the url will be created and set using the fieldInfo.writeValue method. If a mock is required then a Mockito mock RESTClient is created and the expected behaviour mocked using instances of the class HttpResponseDecorator (although these could be mocks too.) Again, this is set using the fieldInfo.writeValue method. The mocking code should probably be in its own method or even class but for this simple example its within the setupRESTClient method:

import groovy.mock.interceptor.MockFor
import groovyx.net.http.HttpResponseDecorator
import groovyx.net.http.RESTClient
import org.spockframework.runtime.extension.AbstractMethodInterceptor;
import org.spockframework.runtime.extension.IMethodInvocation;
import org.spockframework.runtime.model.FieldInfo;
import org.spockframework.runtime.model.SpecInfo;

import static org.mockito.Mockito.*
import static org.mockito.Matchers.*

class RESTWebClientMethodInterceptor extends AbstractMethodInterceptor {

 private final FieldInfo fieldInfo
 private final String url
 private final boolean useMock

 RESTWebClientMethodInterceptor(FieldInfo fieldInfo, String url, boolean useMock) {
  this.fieldInfo = fieldInfo
  this.url = url
  this.useMock = useMock
 }

 @Override
 void interceptSetupMethod(IMethodInvocation methodInvocation) {
  setupRESTClient(methodInvocation.target)
  methodInvocation.proceed()
 }

 @Override
 void install(SpecInfo specInfo) {
  specInfo.setupMethod.addInterceptor this
 }

 private void setupRESTClient(target) {

  if (useMock) {

   def xmlServerOne =
     """
      <serverStatus>
        <serverName>ServerOne</serverName>
        <isRunning>true</isRunning>
      </serverStatus>
      """

   def xmlServerTwo =
     """
      <serverStatus>
        <serverName>ServerTwo</serverName>
        <isRunning>false</isRunning>
      </serverStatus>
      """

   def httpResponseDecoratorServerOne = new HttpResponseDecorator(
     null, new XmlParser().parseText(xmlServerOne))
   def httpResponseDecoratorServerTwo = new HttpResponseDecorator(
     null, new XmlParser().parseText(xmlServerTwo))
   def mapServerOne = [path:"server/status/ServerOne"]
   def mapServerTwo = [path:"server/status/ServerTwo"]

   RESTClient mockRESTClient = org.mockito.Mockito.mock(RESTClient)
   when(mockRESTClient.get(mapServerOne)).thenReturn(httpResponseDecoratorServerOne);
   when(mockRESTClient.get(mapServerTwo)).thenReturn(httpResponseDecoratorServerTwo);

   fieldInfo.writeValue(target, mockRESTClient)
  }
  else {
   fieldInfo.writeValue(target, new RESTClient(url))
  }
 }
 
}

Whilst there are different ways to mock and test RESTful web services, the above classes do show how easily RESTful web services can be called and their responses parsed, how annotations can be created and how objects can be mocked. Similar examples could be constructed for different aspects of a Spock test eg features, fixtures etc..

More can be found on extensions at http://code.google.com/p/spock/wiki/SpockBasics#Extensions and on RESTClient at http://groovy.codehaus.org/modules/http-builder/doc/rest.html