I have a convolutional neural network with a Yolo output-layer and multiple regression output-layers(I just mapped extra output layers to a typical Yolo CNN) using a computation graph. The problem I have is with the dataset, for the Yolo output I have pascal-voc .xml
files and for the regression outputs .csv
files i.e.
OBJECT DETECTION DATA
InputSplit[] data = new FileSplit(image_dir, NativeImageLoader.ALLOWED_FORMATS, random).sample(path_filter, split_weight0,split_weight1);
InputSplit trainData = data[0];
InputSplit testData = data[1];
ObjectDetectionRecordReader recordReaderTrain = new ObjectDetectionRecordReader(height, width, nChannels,gridH, gridW, new VocLabelProvider(DIR));
ObjectDetectionRecordReader recordReaderTest = new ObjectDetectionRecordReader(height, width, nChannels,gridH, gridW, new VocLabelProvider(DIR));
recordReaderTrain.initialize(trainData);
recordReaderTest.initialize(testData);
//commented out since MultiDataSetIterator is meant to be used instead
//RecordReaderDataSetIterator train = new RecordReaderDataSetIterator(recordReaderTrain, batchSize, 1, 1, true);
//RecordReaderDataSetIterator test = new RecordReaderDataSetIterator(recordReaderTest, 1, 1, 1, true);
//train.setPreProcessor(new ImagePreProcessingScaler(0, 1));
//test.setPreProcessor(new ImagePreProcessingScaler(0, 1));
REGRESSION DATA
RecordReader r0= new CSVRecordReader(',');
r0.initialize(new FileSplit(new File( DIR+"/r0.csv")));
RecordReader r1=...
...
I tried to implement the Multi-Task Learning example from RecordReaderMultiDataSetIterator Example 2
MultiDataSetIterator train = new RecordReaderMultiDataSetIterator.Builder(batchSize)
.addReader("rr", recordReaderTrain)
.addReader("r0", r0)
.addInput("???")//recordReaderTrain.getInputImagesData() ?
.addOutput("rr")//recordReaderTrain.getVocLabelData() ?
.addOutput("r0")
.addOutput("r1")
... //addOutput -> r2,r3,...
.build();
How can I correctly configure the input from the record reader that feeds from the VocLabelProvider
with data preprocessing or is it possible to wrap the MultiDataSetIterator
around another dataset iterator i.e. a multi dataset iterator from two dataset iterators from the .xml
object-detection and .csv
regression files