Cannot Instantiate the Type T

Question

I have a class that looks like this...

public class LegionInputFormat
        extends FileInputFormat<NullWritable, LegionRecord> {

    @Override
    public RecordReader<NullWritable, LegionRecord>
            createRecordReader(InputSplit split, TaskAttemptContext context) {

        /* Skipped code for getting recordDelimiterBytes */

        return new LegionRecordReader(recordDelimiterBytes);
    }
}

I'd like to use a generic type so it could return any type of RecordReader specified by the user, like so:

public class LegionInputFormat<T extends RecordReader<NullWritable, LegionRecord>>
        extends FileInputFormat<NullWritable, LegionRecord> {

    @Override
    public RecordReader<NullWritable, LegionRecord>
            createRecordReader(InputSplit split, TaskAttemptContext context) {

        /* Skipped code for getting recordDelimiterBytes */

        return new T(recordDelimiterBytes);
    }
}

As the post title suggests, I'm being told I "cannot instantiate the Type T." From other Stack Exchange posts, I've gathered that this is not possible due to something with how generics work. What I've not been able to gather is an intuitive explanation of why that's the case. I learn best by understanding, so that would be really helpful if somebody can offer it.

I'm also interested in the best practice for accomplishing what I'm looking to do here. Should the constructor for LegionInputFormat accept a RecordReader class, store that, and then reference it later to create a new instance? Or is there a better solution?

(Additional background - context here is Hadoop, but I doubt it matters. I'm a fairly accomplished Data Scientist, but I'm pretty new to Java.)

Answer 1

As the post title suggests, I'm being told I "cannot instantiate the Type T." From other Stack Exchange posts, I've gathered that this is not possible due to something with how generics work.

This is because generics in Java are purely a compile-time feature; the compiler throws away the generics (this is called "type erasure") so that at runtime, there is no such thing as a type variable T, so you cannot do new T(...).

You can do this in Java by passing a Class<T> object to the method that needs to create an instance of T, and then creating an instance through reflection.

Answer 2

In your second code example the compiler has no way of knowing whether T has a constructor that accepts recordDelimiterBytes as argument. This is so because each class is a separate compilation unit, so when LegionInputFormat is compiled, the compiler only knows that T is a RecordReader<NullWritable, LegionRecord>. It doesn’t know which concrete types are used for T, and it has to assume that someone at a later time can stick in any class that extends RecordReader<NullWritable, LegionRecord>. We can tell the compiler something about T using extends, but there is no way in Java that we can specify that T has a constructor T(byte[]) (or whatever the type of recordDelimiterBytes is).

I have used the following solution a couple of times, and even though it requires creating subclasses, I am quite happy with it. The work is still within the generic class. It is now declared abstract:

public abstract class InputFormat<T extends RecordReader<NullWritable, LegionRecord>>
        extends FileInputFormat<NullWritable, LegionRecord> {

    private byte[] recordDelimiterBytes;

    @Override
    public RecordReader<NullWritable, LegionRecord> createRecordReader(InputSplit split, TaskAttemptContext context) {

        /* Skipped code for getting recordDelimiterBytes */

        return constructRecordReader(recordDelimiterBytes);
    }

    // factory method for T objects
    protected abstract RecordReader<NullWritable, LegionRecord> constructRecordReader(byte[] recordDelimiterBytes);
}

For instantiation it requires you write a concrete subclass with just these few lines:

public class LegionInputFormat extends InputFormat<LegionRecordReader> {

    @Override
    protected RecordReader<NullWritable, LegionRecord> constructRecordReader(byte[] recordDelimiterBytes) {
        return new LegionRecordReader(recordDelimiterBytes);
    }

}

In the subclass we know the concrete class of T and therefore the class’ constructor/s, so we can instantiate it. Though not quite as simple as you might have hoped for, I consider the solution nice and clean.

In my own code I have taken the opportunity to declare the factory method as returning type T:

protected abstract T constructRecordReader(byte[] recordDelimiterBytes);

Then you just have to follow up in the implementation:

protected LegionRecordReader constructRecordReader(byte[] recordDelimiterBytes) {

In this case it even gets a few chars shorter. On the other hand, in your case you don’t seem to need it, so you may prefer to stay with the weaker return type of RecordReader<NullWritable, LegionRecord>.