How to load Serialized Objects in Java by parts

Question

I'm trying to serialize objects of big size in Java using Externalizable interface.

Code:

public class ResultsData  implements Externalizable{
    private static final long serialVersionUID = 1L;

    private ArrayList<ALotOfResults1> results1;
    private ArrayList<ALotOfResults2> results2;
    private ArrayList<ALotOfResults3> results3;

    private int selection;

public ResultsData(){

    results1=new ArrayList<ALotOfResults1>();
    results2=new ArrayList<ALotOfResults2>();
    results3=new ArrayList<ALotOfResults3>();

}

//Getter and setter omited   

@Override
public void writeExternal(ObjectOutput out) throws IOException {

    out.writeObject(results1);
    out.writeObject(results2);
    out.writeObject(results3);
}

@Override
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {

    switch(selection) {
        case 0: 
            results1 = (ArrayList)in.readObject();
            break;
        case 1: 
            in.readObject();
            results2 = (ArrayList)in.readObject();
            break;
        case 2:
            in.readObject();
            in.readObject();
            results3 = (ArrayList)in.readObject();
            break;
    }
}

Those three arrayLists filled up during program execution have very big size (14 MB each one).

Code(load/save proccess):

public class ResultsManagement {

public static ResultsData loadResultsData(String path,ResultsData resultsData) {

    try {            

        FileInputStream fisProd = new FileInputStream(path+".res");
        ObjectInputStream oisProd = new ObjectInputStream(fisProd);      
        resultsData.readExternal(oisProd);
        fisProd.close();

    } catch (IOException ioe) {
        System.out.println("Error de IO: " + ioe.getMessage());
    } catch (ClassNotFoundException cnfe) {
        System.out.println("Error de clase no encontrada: " + cnfe.getMessage());
    } catch (Exception e) {
        System.out.println("Error: " + e.getMessage());
    }
    return resultsData;
}

public static void saveResultsData(ResultsData resultsData,String path) {
    try {   
            FileOutputStream fosProd = new FileOutputStream(path+".res");
            ObjectOutputStream oosProd = new ObjectOutputStream(fosProd);
            resultsData.writeExternal(oosProd);
            fosProd.close();

    } catch (IOException ioe) {
        System.out.println("Error de IO: " + ioe.getMessage());
    } catch (Exception e) {
        System.out.println("Error: " + e.getMessage());
    }
}

}

The unbreakable condition is that I want to have only one File, for example, Documents/Project/project1.res

How can I load some parts of the object without loading the other parts? Is it possible? For example, I don't need to load the two first arrayList's (results1 and results2) when I only need to load the third one (results3), but the only way i know to get access to results3 is reading results1 and results2.

Code for the answer of Tinke:

In the ResultsData class:

public static byte[] serialize(Object obj) throws IOException {
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    ObjectOutputStream os = new ObjectOutputStream(out);
    os.writeObject(obj);
    return out.toByteArray();
}

public static Object deserialize(byte[] data) throws IOException, ClassNotFoundException             {
    ByteArrayInputStream in = new ByteArrayInputStream(data);
    ObjectInputStream is = new ObjectInputStream(in);
    return is.readObject();
}

@Override
public void writeExternal(ObjectOutput out) throws IOException {

    byte[] r1 = serialize(results1);
    System.out.println("Bytes in r1: "+r1.length);//42392 Bytes
    out.write(r1);

    byte[] r2 = serialize(results2);
    System.out.println("Bytes in r2: "+r2.length);//19268558 Bytes (a lot of results here)
    out.write(r2);

    out.close();

}

@Override
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {

    switch(selection) {
        case 0: 
            byte[] arrayBytes=new byte[42392];
            in.read(arrayBytes);
            results1 = (ArrayList)deserialize(arrayBytes);
            break;
        case 1: 
            in.skipBytes(42392);
            byte[] arrayBytes2=new byte[19268558];
            in.read(arrayBytes2);
            results2 = (ArrayList)deserialize(arrayBytes2);
            break;
    }
}

I cannot think of any way to do this without splitting the serialized lists into separate files, which breaks your condition stated above. — jgitter, Mar 17 '14 at 16:24

score 0 · Answer 1 · answered Mar 17 '14 at 18:33

0

As a generic solution, while inserting the data, insert the size (in bytes) of each list. then while reading, read the size in and skip those many bytes for each list you wish to skip. The solution could be optimized more depending on your use case and how you insert/access the data in the lists.

answered Mar 17 '14 at 18:33

tinker

1,396
11
20

Thanks for your answer but I've tried it before with no good results because Java serialize objects with headers and other parameters that I don't know. If I try to skip the bytes when i want to load the part interesting for me Java gives me the exception message: "invalid type code: 00". I edit mi post with my code for do what you've said. – JMRA Mar 17 '14 at 19:25
I was asking about the data you load into the lists themselves. Is it arbitrary object data? If there is no restriction and you can change the way you write data, you could think of writing json objects. Since they're written as strings you can figure out how many lines to skip. This is a hack approach but i can't think of anything better right now. – tinker Mar 18 '14 at 11:59
Thanks man, I thought some alternatives but I would like to know if anyone would know how to do it for this specific case – JMRA Mar 18 '14 at 12:23

How to load Serialized Objects in Java by parts

Code:

Code(load/save proccess):

Code for the answer of Tinke:

1 Answers1