Saturday, September 1, 2012

Returning a partial response in a JAX-RS web service - Part 1


This two-part article provides an example of partial response functionality in a Java REST API.

In this example, I am using the Apache CXF framework, Jackson JSON processor, JPA/Hibernate and Spring.

Before I get started, I've got to give a shout out to the guys from apigee.  Their API Best Practices Blog and webinars have been invaluable resources for me.  I have incorporated many of their "Pragmatic REST" recommendations into my API.

What is partial response and why should your API support it?

Partial response is when an API returns only a subset of fields for a requested resource.  This can be done by providing a 'fields' query parameter where the client can specify exactly which fields to return.  Alternatively, the API can default to some subset of fields and only return additional fields when specifically requested.

According to Google,  "when you avoid retrieving and parsing unneeded data, you can significantly improve the efficiency of your client application."  Improved efficiency makes apps faster and reduces the amount of data sent across the wire to your client applications.


Why is it important to support partial response in this technology stack?

As mentioned above, I am using Apache CXF framework, Jackson JSON processor, JPA/Hibernate and Spring.

It is common to work with entities that have complex relationships with other entities.  Entity of type 'A' may have a One To Many relationship with entity of type 'B' which in turn has Many to Many relationship with entity 'C'.  These relationships are defined on my POJOs using JPA annotations and are typically represented as collections.

Anyone familiar with using JPA/Hibernate knows that complex entity relationships can can potentially result in Hibernate executing loads of queries and practically loading the entire database into memory.  Thankfully JPA provides the FetchType.LAZY annotation and Hibernate provides Lazy Loading.  This effectively prevents the loading of unnecessary data until it is actually requested.

Lazy Loading works well on the server side but a REST API needs to serialize data and send it across the wire.  In my case, I'm using Jackson to serialize Java objects into JSON format.  By default, the Jackson ObjectMapper has no awareness of Hibernate and it's Lazy Loading scheme.  During the serialization process, Jackson was touching all the properties on the entity which resulted in Hibernate fetching all the data thereby losing the benefits gained from Lazy Loading.

Thankfully the guys from Jackson were aware of this issue and developed the jackson-module-hibernate.  According to the ReadMe, "This module support JSON serialization and deserialization of Hibernate specific datatypes and properties; especially lazy-loading aspects."  With the addition of this module, Jackson no longer tries to serialize Lazy Loaded collections.

This is great and my hat's off to the guys from Jackson, particularly Tatu who has been very responsive and ready to help with Jackson related questions.

So why is partial response so important here?  Well, the problem arises when the client actually wants one of the Lazy Loaded collections.  We need a way to tell the web service which Lazy Loaded fields we want so that it can initialize them and Jackson can thereby serialize the data.

 An example scenario

Here's an example from a simple Flashcards app I've been working on.  In this app, a Flashcard represents a simple Question and Answer and can be organized by assigning one or more Tags.  Sometimes the client app simply needs to display a list of Flashcard questions.  In this case we need the 'id' and the 'question' associated with each Flashcard.  However, there is another case where the client app needs the same list of Flashcards along with the collection of Tags assigned to each card.

Well, there is of course a lot of ways to skin this cat.  But which solution requires the fewest web services, is the most generic and reusable and has the best performance by limiting the amount of data sent in each response?

Partial responses could solve this problem quite nicely.  Using partial response, we can fulfill the sencario described above while using a single JAX-RS web service.

Here's an example of some URL's that we could use:
  1. GET /api/v1/flashcards/?fields=id,question
  2. GET /api/v1/flashcards/?fields=id,question,tags,name
It should be pretty clear the first URL will provide a list of Flashcards with only the 'id' and 'question' properties.

To understand the second URL, it might help to see  the API doc for a list of properties associated with the Flashcard and Tag entities.  Suffice it to say, the second URL returns the same result as the first URL with the addition of their collection of Tags and the 'name' property of the Tag entity.

In Part 2 of this blog, I'll provide some code examples.

Returning a partial response in a JAX-RS web service - Part 2

See Part 1 of this blog post for background and reasoning for providing partial response in your JAX-RS API.

Here's the basic steps:
  1. A custom Apache CXF RequestHandler filter inspects each Request to look for a 'fields' Query Parameter.
  2. When the 'fields' parameter is found and has a value, we make sure the requested fields are initialized by Hibernate.
  3. Lastly, we use a Jackson ObjectWriter to apply a Jackson filter to the object and then build and return a Response with the resulting JSON.
Here's a link  to the ResponseFilter I'm using in my sample project in Github.

First we inspect the Request and make sure it is a Get, the request succeeded (200 OK), has a JSON response type, and indeed has 'fields' query parameter and corresponding value.

At this point let's make sure any requested lazy loaded fields get initialized.  I'm sure there's a better way of doing the next part.

The code that initializes the fields is beyond scope of this post but here's a link to the class in my Github repo in case you are interested.

Lastly, let's apply the Jackson filter and build the response. Notice we are adding "apiFilter" to the SimpleFilterProvider. Your entity classes will need to be annotated with @JsonFilter("apiFilter") and your CustomObjectMapper should be configured with filterProvider.setFailOnUnknownId(false);

Any reader comments, questions, and suggestions are appreciated. Thanks!