5

I am writing a rest service using spring MVC which produces JSON response. It should allow client to select only the given fields in response, means client can mention the fields he is interested in as url parameter like ?fields=field1,field2.

Using Jackson annotations does not provide what I am looking for as it is not dynamic also the filters in Jackson doesnt seem to be promising enough. So far I am thinking to implement a custom message converter which can take care of this.

Is there any other better way to achieve this? I would like if this logic is not coupled with my services or controllers.

varun
  • 684
  • 1
  • 11
  • 30
  • Have a look at my answer on a similar question leveraging the Jackon's ObjectWriterModifier/ObjectReaderModifier feature to customize filter inside JAX-RS resource method: http://stackoverflow.com/a/23816382/3537858 – Alexey Gavrilov Sep 30 '14 at 07:07

4 Answers4

11

From Spring 4.2, @JsonFilter is supported in MappingJacksonValue

You can directly inject PropertyFilter to MappingJacksonValue in a controller.

@RestController
public class BookController {
    private static final String INCLUSION_FILTER = "inclusion";

    @RequestMapping("/novels")
    public MappingJacksonValue novel(String[] include) {
        @JsonFilter(INCLUSION_FILTER)
        class Novel extends Book {}

        Novel novel = new Novel();
        novel.setId(3);
        novel.setTitle("Last summer");
        novel.setAuthor("M.K");

        MappingJacksonValue res = new MappingJacksonValue(novel);
        PropertyFilter filter = SimpleBeanPropertyFilter.filterOutAllExcept(include);
        FilterProvider provider = new SimpleFilterProvider().addFilter(INCLUSION_FILTER, filter);
        res.setFilters(provider);
        return res;
    }

or you can declare global policy by ResponseBodyAdvice. The following example implements filtering policy by "exclude" parameter.

@ControllerAdvice
public class DynamicJsonResponseAdvice extends AbstractMappingJacksonResponseBodyAdvice {
    public static final String EXCLUDE_FILTER_ID = "dynamicExclude";
    private static final String WEB_PARAM_NAME = "exclude";
    private static final String DELI = ",";
    private static final String[] EMPTY = new String[]{};

    @Override
    protected void beforeBodyWriteInternal(MappingJacksonValue container, MediaType contentType,
            MethodParameter returnType, ServerHttpRequest req, ServerHttpResponse res) {
        if (container.getFilters() != null ) {
            // It will be better to merge FilterProvider
            // If 'SimpleFilterProvider.addAll(FilterProvider)' is provided in Jackson, it will be easier.
            // But it isn't supported yet. 
            return;
        }

        HttpServletRequest baseReq = ((ServletServerHttpRequest) req).getServletRequest();
        String exclusion = baseReq.getParameter(WEB_PARAM_NAME);

        String[] attrs = StringUtils.split(exclusion, DELI);
        container.setFilters(configFilters(attrs));
    }

    private FilterProvider configFilters(String[] attrs) {
        String[] ignored = (attrs == null) ? EMPTY : attrs;
        PropertyFilter filter = SimpleBeanPropertyFilter.serializeAllExcept(ignored);
        return new SimpleFilterProvider().addFilter(EXCLUDE_FILTER_ID, filter);
    }
}
Sanghyuk Jung
  • 111
  • 1
  • 3
5

IMHO, the simplest way to do that would be to use introspection to dynamically generate a hash containing selected fields and then serialize that hash using Json. You simply have to decide what is the list of usable fields (see below).

Here are two example functions able to do that, first gets all public fields and public getters, the second gets all declared fields (including private ones) in current class and all its parent classes :

public Map<String, Object> getPublicMap(Object obj, List<String> names)
        throws IllegalAccessException, IllegalArgumentException, InvocationTargetException  {
    List<String> gettedFields = new ArrayList<String>();
    Map<String, Object> values = new HashMap<String, Object>();
    for (Method getter: obj.getClass().getMethods()) {
        if (getter.getName().startsWith("get") && (getter.getName().length > 3)) {
            String name0 = getter.getName().substring(3);
            String name = name0.substring(0, 1).toLowerCase().concat(name0.substring(1));
            gettedFields.add(name);
            if ((names == null) || names.isEmpty() || names.contains(name)) {
                values.put(name, getter.invoke(obj));
            }
        }
    }
    for (Field field: obj.getClass().getFields()) {
        String name = field.getName();
        if ((! gettedFields.contains(name)) && ((names == null) || names.isEmpty() || names.contains(name))) {
            values.put(name, field.get(obj));
        }
    }
    return values;
}

public Map<String, Object> getFieldMap(Object obj, List<String> names)
        throws IllegalArgumentException, IllegalAccessException  {
    Map<String, Object> values = new HashMap<String, Object>();
    for (Class<?> clazz = obj.getClass(); clazz != Object.class; clazz = clazz.getSuperclass()) {
        for (Field field : clazz.getDeclaredFields()) {
            String name = field.getName();
            if ((names == null) || names.isEmpty() || names.contains(name)) {
                field.setAccessible(true);
                values.put(name, field.get(obj));
            }
        }
    }
    return values;
}

Then you only have to get the result of one of this function (or of one you could adapt to your requirements) and serialize it with Jackson.

If you have custom encoding of you domain objects, you would have to maintain the serialization rules in two different places : hash generation and Jackson serialization. In that case, you could simply generate the full class serialization with Jackson and filter the generated string afterwards. Here is an example of such a filter function :

public String jsonSub(String json, List<String> names) throws IOException {
    if ((names == null) || names.isEmpty()) {
        return json;
    }
    ObjectMapper mapper = new ObjectMapper();
    Map<String, Object> map = mapper.readValue(json, HashMap.class);
    for (String name: map.keySet()) {
        if (! names.contains(name)) {
            map.remove(name);
        }
    }
    return mapper.writeValueAsString(map);
}

Edit : integration in Spring MVC

As you are speaking of a web service and of Jackson, I assume that you use Spring RestController or ResponseBody annotations and (under the hood) a MappingJackson2HttpMessageConverter. If you use Jackson 1 instead, it should be a MappingJacksonHttpMessageConverter.

What I propose is simply to add a new HttpMessageConverter that could make use of one of the above filtering functions, and delegate actual work (and also ancilliary methods) to a true MappingJackson2HttpMessageConverter. In the write method of that new converter, it is possible to have access to the eventual fields request parameter with no need for an explicit ThreadLocal variable thanks to Spring RequestContextHolder. That way :

  • you keep a clear separation of roles with no modification on existing controllers
  • you have no modification in Jackson2 configuration
  • you need no new ThreadLocal variable and simply use a Spring class in a class already tied to Spring since it implements HttpMessageConverter

Here is an example of such a message converter :

public class JsonConverter implements HttpMessageConverter<Object> {

    private static final Logger logger = LoggerFactory.getLogger(JsonConverter.class);
    // a real message converter that will respond to ancilliary methods and do the actual work
    private HttpMessageConverter<Object> delegate =
            new MappingJackson2HttpMessageConverter();

    // allow configuration of the fields name
    private String fieldsParam = "fields";

    public void setFieldsParam(String fieldsParam) {
        this.fieldsParam = fieldsParam;
    }

    @Override
    public boolean canRead(Class<?> clazz, MediaType mediaType) {
        return delegate.canRead(clazz, mediaType);
    }

    @Override
    public boolean canWrite(Class<?> clazz, MediaType mediaType) {
        return delegate.canWrite(clazz, mediaType);
    }

    @Override
    public List<MediaType> getSupportedMediaTypes() {
        return delegate.getSupportedMediaTypes();
    }

    @Override
    public Object read(Class<? extends Object> clazz, HttpInputMessage inputMessage) throws IOException, HttpMessageNotReadableException {
        return delegate.read(clazz, inputMessage);
    }

    @Override
    public void write(Object t, MediaType contentType, HttpOutputMessage outputMessage)
            throws IOException, HttpMessageNotWritableException {
        // is there a fields parameter in request
        String[] fields = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes())
                .getRequest().getParameterValues(fieldsParam);
        if (fields != null && fields.length != 0) {
            // get required field names
            List<String> names = new ArrayList<String>();
            for (String field : fields) {
                String[] f_names = field.split("\\s*,\\s*");
                names.addAll(Arrays.asList(f_names));
            }
            // special management for Map ...
            if (t instanceof Map) {
                Map<?, ?> tmap = (Map<?, ?>) t;
                Map<String, Object> map = new LinkedHashMap<String, Object>();
                for (Entry entry : tmap.entrySet()) {
                    String name = entry.getKey().toString();
                    if (names.contains(name)) {
                        map.put(name, entry.getValue());
                    }
                }
                t = map;
            } else {
                try {
                    Map<String, Object> map = getMap(t, names);
                    t = map;
                } catch (Exception ex) {
                    throw new HttpMessageNotWritableException("Error in field extraction", ex);
                }
            }
        }
        delegate.write(t, contentType, outputMessage);
    }

    /**
     * Create a Map by keeping only some fields of an object
     * @param obj the Object
     * @param names names of the fields to keep in result Map
     * @return a map containing only requires fields and their value
     * @throws IllegalArgumentException
     * @throws IllegalAccessException 
     */
    public static Map<String, Object> getMap(Object obj, List<String> names)
            throws IllegalArgumentException, IllegalAccessException  {
        Map<String, Object> values = new HashMap<String, Object>();
        for (Class<?> clazz = obj.getClass(); clazz != Object.class; clazz = clazz.getSuperclass()) {
            for (Field field : clazz.getDeclaredFields()) {
                String name = field.getName();
                if (names.contains(name)) {
                    field.setAccessible(true);
                    values.put(name, field.get(obj));
                }
            }
        }
        return values;
    }    
}

If you want the converter to be more versatile, you could define an interface

public interface FieldsFilter {
    Map<String, Object> getMap(Object obj, List<String> names)
            throws IllegalAccessException, IllegalArgumentException, InvocationTargetException;
}

and inject it with an implementation of that.

Now you must ask Spring MVC to use that custom message controller.

If you use XML config, you simply declare it in the <mvc:annotation-driven> element :

<mvc:annotation-driven  >
    <mvc:message-converters>
        <bean id="jsonConverter" class="org.example.JsonConverter"/>
    </mvc:message-converters>
</mvc:annotation-driven>

And if you use Java configuration, it is almost as simple :

@EnableWebMvc
@Configuration
public class WebConfig extends WebMvcConfigurerAdapter {

  @Autowired JsonConverter jsonConv;

  @Override
  public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
    converters.add(jsonConv);
    StringHttpMessageConverter stringConverter = new StringHttpMessageConverter();
    stringConverter.setWriteAcceptCharset(false);

    converters.add(new ByteArrayHttpMessageConverter());
    converters.add(stringConverter);
    converters.add(new ResourceHttpMessageConverter());
    converters.add(new SourceHttpMessageConverter<Source>());
    converters.add(new AllEncompassingFormHttpMessageConverter());
    converters.add(new MappingJackson2HttpMessageConverter());
  }
}

but here you have to explicitely add all the default message converters that you need.

Serge Ballesta
  • 143,923
  • 11
  • 122
  • 252
  • But this is not what i want, I dont want to copy the fields into map and than serialize the map, I want to return my domain objects only and have the facility to restrict the fields which needs to be converted to json. – varun Sep 29 '14 at 09:34
  • A rest service does not really return domain objects, but serialized versions of them. And there is no difference between the serialized version of an object, and the one of a map `{ field_name : field_value }`. And Java does not copy the fields but only take a reference to them (except for primitive types) so the overhead should be negligible. And my edited version takes all fields if the list is null or empty, so that you get the full serialization of your domain object if the user does not specify a list of fields. – Serge Ballesta Sep 29 '14 at 09:50
  • I agree with you, but there is a different reason for which I do not want to go with this. What do you suggest how and when should we make a call to this function? I dont think its a good idea to call this function from almost every controller just before it returns and if we use something like http message converters of aop, this all happens automatically which looks much neat. Also with your solution how can we handle more complex objects like lists or maps, than we need to iterate on everything. – varun Sep 29 '14 at 10:00
  • You can use Interceptor and/or custom annotation as in my answer but filter using maps as Serge and Grep suggested. Also, you can use jackson to convert object to a map (and back): `ObjectMapper m = new ObjectMapper(); Map props = m.convertValue(myBean, Map.class); MyBean anotherBean = m.convertValue(props, MyBean.class);` – Krešimir Nesek Sep 29 '14 at 21:25
  • @varun : I have updated my answer with a fully tested configuration using a custom message converter. – Serge Ballesta Sep 30 '14 at 10:08
  • I think this is good way to filter response, but in most situations we need not just filter json response, but do not query database at all with fields that we don't need. Especially if it is some aggregate fields or collections. Filtering on response just saves bandwidth, but doesn't save database from queying unnecessary data (and they can be very heavy queries). I have been studying this issue for a very long time but still not found a good and concise solution with Spring MVC. :-( – Ruslan Stelmachenko Feb 05 '16 at 01:32
3

I've never done this but after looking at this page http://wiki.fasterxml.com/JacksonFeatureJsonFilter it seems that it would be possible to do what you want this way:

1) Create a custom JacksonAnnotationIntrospector implementation (by extending default one) that will use a ThreadLocal variable to choose a filter for current request and also create a custom FilterProvider that would provide that filter.

2) Configure the message converter's ObjectMapper to use the custom introspector and filter provider

3) Create an MVC interceptor for REST service that detects fields request parameter and configures a new filter for current request via your custom filter provider (this should be a thread local filter). ObjectMapper should pick it up through your custom JacksonAnnotationIntrospector.

I'm not 100% certain that this solution would be thread safe (it depends on how ObjectMapper uses annotation introspector and filter provider internally).

- EDIT -

Ok I did a test implementation and found out that step 1) wouldn't work because Jackson caches the result of AnnotationInterceptor per class. I modified idea to apply dynamic filtering only on annotated controller methods and only if the object doesn't have anoter JsonFilter already defined.

Here's the solution (it's quite lengthy):

DynamicRequestJsonFilterSupport class manages the per-request fields to be filtered out:

public class DynamicRequestJsonFilterSupport {

    public static final String DYNAMIC_FILTER_ID = "___DYNAMIC_FILTER";

    private ThreadLocal<Set<String>> filterFields;
    private DynamicIntrospector dynamicIntrospector;
    private DynamicFilterProvider dynamicFilterProvider;

    public DynamicRequestJsonFilterSupport() {
        filterFields = new ThreadLocal<Set<String>>();
        dynamicFilterProvider = new DynamicFilterProvider(filterFields);
        dynamicIntrospector = new DynamicIntrospector();
    }

    public FilterProvider getFilterProvider() {
        return dynamicFilterProvider;
    }

    public AnnotationIntrospector getAnnotationIntrospector() {
        return dynamicIntrospector;
    }

    public void setFilterFields(Set<String> fieldsToFilter) {
        filterFields.set(Collections.unmodifiableSet(new HashSet<String>(fieldsToFilter)));
    }

    public void setFilterFields(String... fieldsToFilter) {
        filterFields.set(Collections.unmodifiableSet(new HashSet<String>(Arrays.asList(fieldsToFilter))));
    }

    public void clear() {
        filterFields.remove();
    }

    public static class DynamicIntrospector extends JacksonAnnotationIntrospector {


        @Override
        public Object findFilterId(Annotated annotated) {
            Object result = super.findFilterId(annotated);
            if (result != null) {
                return result;
            } else {
                return DYNAMIC_FILTER_ID;
            }
        }
    }

    public static class DynamicFilterProvider extends FilterProvider {

        private ThreadLocal<Set<String>> filterFields;

        public DynamicFilterProvider(ThreadLocal<Set<String>> filterFields) {
            this.filterFields = filterFields;
        }

        @Override
        public BeanPropertyFilter findFilter(Object filterId) {
            return null;
        }

        @Override
        public PropertyFilter findPropertyFilter(Object filterId, Object valueToFilter) {
            if (filterId.equals(DYNAMIC_FILTER_ID) && filterFields.get() != null) {
                return SimpleBeanPropertyFilter.filterOutAllExcept(filterFields.get());
            }
            return super.findPropertyFilter(filterId, valueToFilter);
        }
    }
}

JsonFilterInterceptor intercepts controller methods annotated with custom @ResponseFilter annotation.

public class JsonFilterInterceptor implements HandlerInterceptor {

    @Autowired
    private DynamicRequestJsonFilterSupport filterSupport;
    private ThreadLocal<Boolean> requiresReset = new ThreadLocal<Boolean>();

    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
        if (handler instanceof HandlerMethod) {
            HandlerMethod method = (HandlerMethod) handler;
            ResponseFilter filter = method.getMethodAnnotation(ResponseFilter.class);
            String[] value = filter.value();
            String param = filter.param();
            if (value != null && value.length > 0) {
                filterSupport.setFilterFields(value);
                requiresReset.set(true);
            } else if (param != null && param.length() > 0) {
                String filterParamValue = request.getParameter(param);
                if (filterParamValue != null) {
                    filterSupport.setFilterFields(filterParamValue.split(","));
                }
            }
        }
        requiresReset.remove();
        return true;
    }

    @Override
    public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception {
    }

    @Override
    public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {
        Boolean reset = requiresReset.get();
        if (reset != null && reset) {
            filterSupport.clear();
        }
    }
}

Here's the custom @ResponseFilter annotation. You can either define a static filter (via annotation's value property) or a filter based on request param (via annotation's param property):

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface ResponseFilter {
    String[] value() default {};
    String param() default "";
}

You will need to setup the message converter and the interceptor in the config class:

...
@Override
public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
    converters.add(converter());
}

@Bean
JsonFilterInterceptor jsonFilterInterceptor() {
    return new JsonFilterInterceptor();
}

@Override
public void addInterceptors(InterceptorRegistry registry) {
    registry.addInterceptor(jsonFilterInterceptor);
}

@Bean
DynamicRequestJsonFilterSupport filterSupport() {
    return new DynamicRequestJsonFilterSupport();
}

@Bean
MappingJackson2HttpMessageConverter converter() {
    MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
    ObjectMapper mapper = new ObjectMapper();
    mapper.setAnnotationIntrospector(filterSupport.getAnnotationIntrospector());
    mapper.setFilters(filterSupport.getFilterProvider());
    converter.setObjectMapper(mapper);
    return converter;
}
...

And finally, you can use the filter like this:

@RequestMapping("/{id}")
@ResponseFilter(param = "fields")
public Invoice getInvoice(@PathVariable("id") Long id) { ... }

When request is made to /invoices/1?fields=id,number response will be filtered and only id and number properties will be returned.

Please note I haven't tested this thoroughly but it should get you started.

Krešimir Nesek
  • 5,302
  • 4
  • 29
  • 56
  • Your idea is very close to what I was thinking, my idea also involves use of threadlocal. I am sure many people would have done this already And I'm in no hurry so I will wait to accept any answer yet but sure your solution should be thread safe and definitely deserves +1 – varun Sep 24 '14 at 19:33
  • 1
    I edited the answer with a proof of concept implementation of the idea though I agree with you that you should wait to see other answers too. – Krešimir Nesek Sep 24 '14 at 21:00
1

Would populating a HashMap from the object not suite the requirements? You could then just parse the HashMap. I have done something similar with GSON in the past where I had to provide a simple entity and ended up just populating a HashMap and then serializing it, it was far more maintainable than over engineering a whole new system.

Gerrit Brink
  • 977
  • 3
  • 15
  • 26