5

I've thousands of classes in our java project. Some of them implements serializable interface. Now here's a problem. It's possible someone can go in a class, add new variable that is neither transient nor serializable. Code compiles fine however process would blow up at runtime.

To illustrate this

class Foo implements Serializable {  .... // all good }

class Foo implements Serializable 
{  
    // OOps, executorService is not serializable.  It's not declared as transient either 

    private ExecutorService executorService = ..
}

I'm thinking about writing a unit test that would go thru all classes and ensure "true serializability". I've read some discussions about serialing specific objects. i understand that process but it requires

1) creating an object.
2) serializing and then
3) deserializing.

Is there more efficient and practical approach. perhaps to use reflection. Go thru all classes, if class has serializable then all attributes must be serializable or have transient keyword..

Thoughts ?

Bhesh Gurung
  • 50,430
  • 22
  • 93
  • 142
Ros
  • 401
  • 5
  • 11

4 Answers4

2

1) creating an object. 2) serializing and then 3) deserializing.

This list is not complete; you also need initialization. Consider the example:

class CanBeSerialized implements Serializable {
    private String a; // serializable
    private Thread t; // not serializable
}

class CannotBeSerialized implements Serializable {
    private String a;                // serializable
    private Thread t = new Thread(); // not serializable
}

You can serialize and deserialize the first one, but you'll get NotSerializableException on the second. To complicate the matter further, if interfaces are used, you can never tell if a class will pass serialization, as it's the concrete object of the class behind this interface that will be streamed:

class PerhapsCanBeSerializedButYouNeverKnow implements Serializable {
    private Runnable r; // interface type - who knows?
}

Provided that you could guarantee the following for all your classes and classes used by your classes to be tested:

  • default constructor exists,
  • no interface types in fields,

then you could automatically create and initialize them by reflection, and then test serialization. But that is a really hard condition, isn't it? Otherwise, proper initialization is down to manual work.

You could use reflection in a different way: iterating through a list of Class objects you want to check, getting the Field[] for them, and verifying if they're transient (Field.getModifiers()) or if they implement Serializable directly (Field.getType().getInterfaces()) or indirectly (via a super interface or class). Also, consider how deep you want to check, depending on how deep your serialization mechanism works.

As Ryan pointed out correctly, this static serialization check would fail if the code was evil enough:

class SeeminglySerializable implements Serializable {
    // ...
        private void writeObject/readObject() {
             throw new NotSerializableException();
        }
}

or just if readObject()/writeObject() were badly implemented. To test against this kind of problems, you need to actually test the serialization process, not the code behind it.

MaDa
  • 10,511
  • 9
  • 46
  • 84
  • You still haven't accounted for the magical [readObject and writeObject methods](http://java.sun.com/developer/technicalArticles/ALT/serialization/). If those get used, none of the rules that either of you have listed will apply. – Ryan Stewart Oct 12 '11 at 13:00
  • @RyanStewart I was going to mention that my algorithm is only an approximation, but it slipped me. Thank you! – MaDa Oct 12 '11 at 13:33
1

If serialization is a key part of your app, then include the serialization in your tests. Something like:

@Test
public void aFooSerializesAndDeserializesCorrectly {
    Foo fooBeforeSerialization = new Foo();
    ReflectionUtils.randomlyPopulateFields(foo);
    Foo fooAfterSerialization = Serializer.serializeAndDeserialize(foo);
    assertThat(fooAfterSerialization, hasSameFieldValues(fooBeforeSerialization));
}

Edit: A trivial implementation of randomlyPopulateFields:

public static void randomlyPopulateFields(final Object o) {
    ReflectionUtils.doWithFields(o.getClass(), new ReflectionUtils.FieldCallback() {
        public void doWith(Field field) throws IllegalArgumentException, IllegalAccessException {
            ReflectionUtils.makeAccessible(field);
            ReflectionUtils.setField(field, o, randomValueFor(field.getType()));
        }

        private Random r = new Random();
        private Object randomValueFor(Class<?> type) {
            if (type == String.class) {
                return String.valueOf(r.nextDouble());
            } else if (type == Boolean.class || type == Boolean.TYPE) {
                return r.nextBoolean();
            } else if (type == Byte.class || type == Byte.TYPE) {
                return (byte) r.nextInt();
            } else if (type == Short.class || type == Short.TYPE) {
                return (short) r.nextInt();
            } else if (type == Integer.class || type == Integer.TYPE) {
                return r.nextInt();
            } else if (type == Long.class || type == Long.TYPE) {
                return (long) r.nextInt();
            } else if (Number.class.isAssignableFrom(type) || type.isPrimitive()) {
                return Byte.valueOf("1234");
            } else if (Date.class.isAssignableFrom(type)) {
                return new Date(r.nextLong());
            } else {
                System.out.println("Sorry, I don't know how to generate values of type " + type);
                return null;
            }
        }
    });
}
Ryan Stewart
  • 126,015
  • 21
  • 180
  • 199
  • 1
    What `ReflectionUtils` are you referring to? – MaDa Oct 12 '11 at 07:51
  • None in particular. I think I've seen a utility like that in some library somewhere, but I generally just write my own. Any way of creating an appropriately populated object will work. The point is that if you're going to test serialization, then you need to be serializing objects. – Ryan Stewart Oct 12 '11 at 13:02
  • The problem is, such method cannot be trivially implemented without providing metadata on how to initialize the object. This is why I find your answer misleading, as it tricks into seemingly easy solution to the problem. – MaDa Oct 12 '11 at 13:39
  • Added a simple implementation. The worst part of it is having to deal with all the numeric types individually. It's not very hard. It's also only one possible way to get populated objects for tests. Actual approach would depend on what kind of test infrastructure you've already built up. – Ryan Stewart Oct 13 '11 at 00:48
  • Great. Now you're showing plainly there's no magic behind your method, only hard work. – MaDa Oct 13 '11 at 07:12
0

Put the classes you want to test inside the method initParameters() and run the whole class as JUnit test.

import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.fail;
import static java.lang.reflect.Modifier.FINAL;
import static java.lang.reflect.Modifier.STATIC;
import static java.lang.reflect.Modifier.TRANSIENT;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;

import java.io.Serializable;
import java.lang.reflect.Field;
import java.lang.reflect.GenericArrayType;
import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;
import java.util.Arrays;
import java.util.Collection;
import java.util.Comparator;
import java.util.HashMap;
import java.util.Map;
import java.util.TreeMap;

@SuppressWarnings({"rawtypes"})
@RunWith(Parameterized.class)
public class SerializableTest {
    @Parameterized.Parameter
    public Class clazz;

    @Parameterized.Parameters(name = "{0}")
    public static Collection<Object[]> initParameters() {
        return Arrays.asList(new Object[][]{
                // TODO put your classes here
                {YourClassOne.class},
                {YourClassTwo.class},
        });
    }

    @Test
    @SuppressWarnings("Convert2Diamond")
    public void testSerializableHierarchy() throws ReflectiveOperationException {
        performTestSerializableHierarchy(new TreeMap<Class, Boolean>(new Comparator<Class>() {
            @Override
            public int compare(Class o1, Class o2) {
                return o1.getName().compareTo(o2.getName());
            }
        }), new HashMap<Long, Class>(), clazz);
    }

    @SuppressWarnings("ConstantConditions")
    private void performTestSerializableHierarchy(Map<Class, Boolean> classes, Map<Long, Class> uids, Type type) throws IllegalAccessException {
        if (type instanceof GenericArrayType) {
            performTestSerializableHierarchy(classes, uids, ((GenericArrayType) type).getGenericComponentType());
            return;
        } else if (type instanceof ParameterizedType) {
            performTestSerializableHierarchy(classes, uids, ((ParameterizedType) type).getRawType());

            Type[] types = ((ParameterizedType) type).getActualTypeArguments();
            for (Type parameterType : types) {
                performTestSerializableHierarchy(classes, uids, parameterType);
            }
            return;
        } else if (!(type instanceof Class)) {
            fail("Invalid type: " + type);
            return;
        }

        Class clazz = (Class) type;

        if (clazz.isPrimitive()) {
            return;
        }

        if (clazz.isEnum()) {
            return;
        }

        if (clazz.equals(String.class)) {
            return;
        }

        if (clazz.isArray()) {
            performTestSerializableHierarchy(classes, uids, clazz.getComponentType());
            return;
        }

        if (Collection.class.isAssignableFrom(clazz)) {
            return;
        }

        if (Map.class.isAssignableFrom(clazz)) {
            return;
        }

        if (!Serializable.class.isAssignableFrom(clazz)) {
            fail(clazz + " does not implement " + Serializable.class.getSimpleName());
            return;
        }

        Boolean status = classes.get(clazz);
        if (status == null) {
            classes.put(clazz, false);
        } else if (status) {
            return;
        }

        Field uidField = null;

        try {
            uidField = clazz.getDeclaredField("serialVersionUID");
        } catch (NoSuchFieldException ex) {
            fail(clazz + " does not declare field 'serialVersionUID'");
        }

        assertNotNull(uidField);

        if ((uidField.getModifiers() & (STATIC | FINAL)) != (STATIC | FINAL) || !uidField.getType().equals(long.class)) {
            fail(clazz + " incorrectly declares field 'serialVersionUID'");
        }

        uidField.setAccessible(true);
        long uidValue = (long) uidField.get(null);
        if (uidValue == ((int) uidValue)) {
            fail(uidField + " has invalid value: " + uidValue);
        }

        Class existingClass = uids.get(uidValue);
        if (existingClass != null && !existingClass.equals(clazz)) {
            fail(existingClass + " has assigned 'serialVersionUID' same value as " + clazz);
        }

        for (Field field : clazz.getDeclaredFields()) {
            if ((field.getModifiers() & (STATIC | TRANSIENT)) == 0) {
                performTestSerializableHierarchy(classes, uids, field.getGenericType());
            }
        }

        classes.put(clazz, true);
        uids.put(uidValue, clazz);
    }
}

Oleksii K.
  • 5,359
  • 6
  • 44
  • 72
0

this unit test finds all the classes in packageName that are descendants of Serializable, and checks the declared properties of these classes to see if they are also descendants of Serializable.

library dependencies

dependencies {
    implementation 'io.github.classgraph:classgraph:4.8.151'
}

the unit test

import io.github.classgraph.ClassGraph
import org.junit.Test
import java.io.Serializable
import java.util.LinkedHashSet
import kotlin.reflect.KClass
import kotlin.reflect.full.declaredMemberProperties
import kotlin.reflect.full.isSubclassOf
import kotlin.reflect.jvm.javaField

class SerializableTest
{
    @Test
    fun `all properties of Serializable descendants are also Serializable descendants`()
    {
        // change BuildConfig.APPLICATION_ID to be root package of your project, unless you are also android developer! :D
        val packageName = BuildConfig.APPLICATION_ID

        // this should be okay to keep it as it is as long as you put it in the same package as root of your project
        val classLoader = SerializableTest::class.java.classLoader!!
        val allClasses = tryGetClassesForPackage(packageName,classLoader)
        val nonSerializableMembersOfSerializableSequence = allClasses
            .asSequence()
            .onEach { jClass -> println("found class in package $packageName: $jClass") }
            .filter { jClass -> Serializable::class.java.isAssignableFrom(jClass) }
            .onEach { serializableJClass -> println("found Serializable subclass: $serializableJClass") }
            .filter { serializableJClass -> serializableJClass.kotlin.simpleName != null }
            .onEach { serializableJClass -> println("found non-anonymous Serializable subclass: $serializableJClass") }
            .flatMap()
            { serializableJClass ->
                serializableJClass.kotlin.simpleName
                serializableJClass.kotlin.declaredMemberProperties.asSequence()
                    .onEach { property -> println("found property of Serializable subclass: ${property.name}") }
                    .filter { property -> property.javaField != null }
                    .onEach { property -> println("found java-field-backed property of Serializable subclass: ${property.name}") }
                    .mapNotNull()
                    { kProperty ->
                        val propertyClass = kProperty.returnType.classifier as? KClass<*>
                        val isMemberNonSerializable = propertyClass?.isSubclassOf(Serializable::class) != true
                        if (isMemberNonSerializable) serializableJClass.canonicalName to kProperty.name else null
                    }
            }
        val nonSerializableMembersOfSerializable = nonSerializableMembersOfSerializableSequence.toSet()
        nonSerializableMembersOfSerializable.onEach { serializableJClass ->
            val serializableClassName = serializableJClass.first
            val nonSerializablePropertyName = serializableJClass.second
            System.err.println("found Serializable subclass ($serializableClassName) with non-Serializable member: $nonSerializablePropertyName")
        }
        assert(nonSerializableMembersOfSerializable.isEmpty())
    }

    private fun tryGetClassesForPackage(packageName:String,loader:ClassLoader):Sequence<Class<*>>
    {
        return ClassGraph().acceptPackages(packageName).enableClassInfo().scan().use { scanResult ->
            scanResult.allClasses.names.asSequence().map { className -> loader.loadClass(className) }
        }
    }
}
Eric
  • 16,397
  • 8
  • 68
  • 76