2

I've been trying to develop a RESTful layer on top of a working Spring MVC 3.1.2 application using Jackson 2.2.2 as a Json (de)serializer. Problem is, it's going way too deep into the references and for a page that used to take at most 1 second to load before now takes 22 seconds server-side only.

The problem is Jackson is going through every single association and takes forever to load everything and to parse it.

I know about @JsonIgnore but well, I'd rather have a depth limitation because if we have let's say:

My amazing explanation

If I put @JsonIgnore on the link between B and C then I'd be good when serializing A but then what if I need to serialize B and want C serialized along? The best way I can think of would be to give the serializer a depth-level limitation. Let's say, depth limit = 1 then it wouldn't serialize C when serializing A but would still serialize it when serializing B. Is there any way to do such thing?

I've seen the @JsonView annotation but it's designed to include properties and not for excluding them. It can be used to exclude some properties but it's only relevant on a one-class level.

Do I need to write my own serializer? Is there a way to implement such a thing if I write my own serializer?

I can't think this isn't addressable but I can't find anything helping my case...

Thanks!

Michael De Keyser
  • 787
  • 1
  • 17
  • 45
  • 1
    Quick note on wording: no, you don't write your own parser, since parser is the thing that reads in JSON. And sounds like you are talking about writing JSON. So it'd be your own JSON Generator or Serializer. Not a big deal as I think question itself is still understandable. – StaxMan Aug 27 '13 at 18:55

3 Answers3

1

If you are using Hibernate, Jackson Hibernate module (https://github.com/FasterXML/jackson-datatype-hibernate) supports disabling loading of "lazy" properties (collections that are lazily fetched). This would allow you to limit how big portion of the object graph is to be accessed and serialized.

Other than that Jackson does not have depth-based limits; and core package knows nothing about specific domain/data modules. Extension modules can be written to change behavior for specific domains; this is how Hibernate module works. And maybe that could be usable for generic JPA-specific functionality.

StaxMan
  • 113,358
  • 34
  • 211
  • 239
1

I've come to a solution a while ago and didn't really have the time and energy to post it, but here we go:

NB: This solution isn't the perfect two-lines solution everyone dreams of but well, it works as I wanted. NB2: This solution requires the XOM library in order to read XML files.

First, how does it work: I create a set of XML files with each file representing ONE entity that is to be serialized by jackson (or needs customized serialization).

Here is an example of such a file - Assignment.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Assignment>
    <endDate/>
    <id/>
    <missionType>
        <id/>
        <name/>
    </missionType>
    <numberOfDaysPerWeek/>
    <project>
        <id/>
        <name/>
    </project>
    <resource>
        <id/>
        <firstName/>
        <lastName/>
        <fte/>
    </resource>
    <role>
        <id/>
        <name/>
    </role>
    <startDate/>
    <workLocation/>
</Assignment>

Here we have the Assignment class represented with each attribute represented as XML elements. Note that any element that isn't represented won't be serialized using the converter I will show later.
Elements with child elements are objects referenced by an Assignment instance. These child elements will be serialized together with the rest.
For example, an Assignment instance has an attribute named "role" that is of the type Role and we want role's id and name serialized.

This is mainly how you choose what and what not to serialize and thus limit the depth of the serialization.

Now to the ObjectConverter class:

import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import org.apache.log4j.Logger;

import nu.xom.Builder;
import nu.xom.Document;
import nu.xom.Element;
import nu.xom.Elements;
import nu.xom.ParsingException;
import nu.xom.ValidityException;

import model.ModelObject;

/**
 * This helper class will convert DOM objects (those implementing ModelObject) into a data structure built on the fly.
 * Typically, a simple object will be converted into a Map<String, Object> where keys will be the object's field names and values be corresponding field values.
 * The convertion uses an XML configuration file that is located in webservices/jackson/converters.
 * 
 * @author mdekeys
 *
 */
public class ObjectConverter {

    private static Logger logger = Logger.getLogger(ObjectConverter.class);
    private static final String CONFIGURATION_DIR = "../standalone/deployments/resources-management.war/WEB-INF/classes/com/steria/rm/webservices/jackson/converters/";

    /**
     * 
     * @param obj The object to convert
     * @param element An XML element (based on XOM library) which represents the object structure.
     * @return Returns the object converted in a corresponding data structure
     */
    @SuppressWarnings("unchecked")
    private static Object serialize(ModelObject obj, Element element) {
        //initialize return value
        Map<String, Object> map = new HashMap<String, Object>();
        //find all child elements
        Elements children = element.getChildElements();
        //loop through children elements
        for (int i = 0; i < children.size(); i++) {
            //get the current child
            Element child = children.get(i);
            //child's qualifiedName shoud be the name of an attribute
            String fieldName = child.getQualifiedName();
            //find get method for this attribute
            Method getMethod = null;
            try {
                getMethod = obj.getConvertedClass().getMethod("get" + firstLetterToUpperCase(fieldName));
            } catch (NoSuchMethodException e) {
                logger.error("Cannot find getter for "+fieldName, e);
                return null;
            } catch (SecurityException e) {
                logger.error("Cannot access getter for "+fieldName, e);
                return null;
            }

            //invoke get method
            Object value = null;
            try {
                value = getMethod.invoke(obj, (Object[]) null);
            } catch (IllegalAccessException e) {
                logger.error("Cannot invoke getter for "+fieldName, e);
                return null;
            } catch (IllegalArgumentException e) {
                logger.error("Bad arguments passed to getter for "+fieldName, e);
                return null;
            } catch (InvocationTargetException e) {
                logger.error("Cannot invoke getter for "+fieldName, e);
                return null;
            }

            //if value is null, return null
            if (value == null || (value instanceof List && ((List<?>) value).size() == 0)) {
                map.put(fieldName, null);
            } else if (value instanceof List<?>) { //if value is a list, recursive call
                map.put(fieldName, serializeList((List<ModelObject>) value, child));
            } else if (value instanceof ModelObject) { //if value is another convertable object, recursive call
                map.put(fieldName, serialize((ModelObject) value, child));
            } else { //simple value, put it in
                map.put(fieldName, value);
            }
        }

        return map;
    }

    /**
     * Intermediary method that is called from outside of this class.
     * @param list List of objects to be converted.
     * @param confFileName Name of the configuration file to be used.
     * @return The list of converted objects
     */
    public static List<Object> serializeList(List<ModelObject> list, String confFileName) {
        return serializeList(list, findRootElement(confFileName));
    }

    /**
     * Method that is called inside this class with an XML element (based on XOM library)
     * @param list List of objects to be converted.
     * @param element XML element (XOM) representing the object's structure
     * @return List of converted objects.
     */
    public static List<Object> serializeList(List<ModelObject> list, Element element) {
        ArrayList<Object> res = new ArrayList<Object>();
        for (ModelObject obj : list) {
            res.add(serialize(obj, element));
        }
        return res;
    }

    /**
     * Method that is called from outside of this class.
     * @param object Object to be converted.
     * @param confFileName Name of the XML file to use for the convertion.
     * @return Converted object.
     */
    public static Object serialize(ModelObject object, String confFileName) {
        return serialize(object, findRootElement(confFileName));
    }

    /**
     * Helper method that is used to set the first letter of a String to upper case.
     * @param str The string to be modified.
     * @return Returns the new String with its first letter in upper case.
     */
    private static String firstLetterToUpperCase(String str) {
        return str.substring(0, 1).toUpperCase() + str.substring(1);
    }

    /**
     * Helper method that is taking an XML configuration file name and returns its the root element (based on XOM library).
     * @param confFileName Name of the XML configuration file
     * @return Returns the root element (XOM based)
     */
    private static Element findRootElement(String confFileName) {
        Builder parser = new Builder();
        Document doc = null;
        String confFile = confFileName + ".xml";
        try {
            doc = parser.build(CONFIGURATION_DIR + confFile);
        } catch (ValidityException e) {
            doc = e.getDocument();
            logger.warn("XML configuration file for "+confFileName+" isn't valid", e);
        } catch (ParsingException e) {
            logger.error("XML configuration file for "+confFileName+" isn't parseable", e);
        } catch (IOException e) {
            logger.error("IOException on XML configuration file for "+confFileName, e);
        }
        return doc.getRootElement();
    }

}

As you can see, the serialize, serializeList and serializeMap methods need a ModelObject argument that is an interface all your serializable objects will have to implement (provided down below).
If you already have an interface that is used to regroup all your domain objects together under one given type then this interface can be used too (you just need to add one method, see below).

Interface ModelObject:

/**
 * Interface that identifies an object as a DOM object and is used by class {@ObjectConverter} to retrieve the class of the object to convert.
 * @author mdekeys
 *
 */
public interface ModelObject {

    /**
     * This method returns the implementer's class
     * @return The implementer Class
     */
    Class<?> getConvertedClass();

}

You use this ObjectConverter as follow:

@Override
@RequestMapping(value = "/populatedLists", method = RequestMethod.GET)
public @ResponseBody Map<String, Object> populateLists() {
    Map<String, Object> map = new HashMap<String, Object>();
    final List<ModelObject> assignments = (List<ModelObject>)(List<?>) this.assignmentService.listAll();
    final List<ModelObject> projects = (List<ModelObject>)(List<?>) this.projectService.listAll();

    map.put("assignments", ObjectConverter.serializeList(assignments, "Assignment"));
    map.put("projects", ObjectConverter.serializeList(projects, "Project"));

    return map;
}

PS: Ignore the weird casting, it's a Java trick for converting a List of XX to a List of YY when you know XX can be casted to YY.

So as you can see, this is used in addition to Jackson: You retrieve list(s) of or a single object from your DB and then convert them using the ObjectConverter's specialized method (serializeList, etc) and providing the key to the XML configuration file (e.g. Assignment.xml). Then you add them to a Map which is serialized by Jackson itself and there you go.

The goal of this ObjectConverter reading XML files is therefore to build a data structure that you can customize using these XML files. This avoids to create a converter class for each object you need to serialize.

The ObjectConverter class will loop through all elements of the XML file and then use the java.lang.reflect package to find these attributes within the object you want to convert.
Note that obviously the spelling is very important in the XML files but the order isn't.

I use that solution myself and using a little app of my own I was able to generate all the XML files and then customize them as I needed. This might seem heavy but this really helped me a lot and I haven't seen any performance hit.

Hope this can help !

Michael De Keyser
  • 787
  • 1
  • 17
  • 45
1

This is a pretty common problem and has already been addressed, check out Jackson's @JsonBackReference annotation. Sample:

@Entity
@Table(name = 'EMPLOYERS')
public class Employer implements Serializable {
    @JsonManagedReference('employer-employee')
    @OneToMany(mappedBy = 'employer', cascade = CascadeType.PERSIST)
    public List getEmployees() {
        return employees;
    }
}

@Entity
@Table(name = 'EMPLOYEES')
public class Employee implements Serializable {
    @JsonManagedReference('employee-benefit')
    @OneToMany(mappedBy = 'employee', cascade = CascadeType.PERSIST)
    public List getBenefits() {
        return benefits;
    }

    @JsonBackReference('employer-employee')
    @ManyToOne(optional = false)
    @JoinColumn(name = 'EMPLOYER_ID')
    public Employer getEmployer() {
        return employer;
    }
}

@Entity
@Table(name = 'BENEFITS')
public class Benefit implements Serializable {
    @JsonBackReference('employee-benefit')
    @ManyToOne(optional = false)
    @JoinColumn(name = 'EMPLOYEE_ID')
    public Employee getEmployee() {
        return employee;
    }
}

Full example.

Filip Spiridonov
  • 34,332
  • 4
  • 27
  • 30
SergeyB
  • 9,478
  • 4
  • 33
  • 47
  • Thank you but this is not what I was asking for. This is about Self-Reference which I addressed in that answer http://stackoverflow.com/a/18120348/1300454 – Michael De Keyser Feb 05 '14 at 08:24