2

I have a convolutional neural network with a Yolo output-layer and multiple regression output-layers(I just mapped extra output layers to a typical Yolo CNN) using a computation graph. The problem I have is with the dataset, for the Yolo output I have pascal-voc .xml files and for the regression outputs .csv files i.e.

OBJECT DETECTION DATA

            InputSplit[] data = new FileSplit(image_dir, NativeImageLoader.ALLOWED_FORMATS, random).sample(path_filter, split_weight0,split_weight1);
            InputSplit trainData = data[0];
            InputSplit testData = data[1];

            ObjectDetectionRecordReader recordReaderTrain = new ObjectDetectionRecordReader(height, width, nChannels,gridH, gridW, new VocLabelProvider(DIR));
            ObjectDetectionRecordReader recordReaderTest = new ObjectDetectionRecordReader(height, width, nChannels,gridH, gridW, new VocLabelProvider(DIR));
            recordReaderTrain.initialize(trainData);
            recordReaderTest.initialize(testData);

            //commented out since MultiDataSetIterator is meant to be used instead  
            //RecordReaderDataSetIterator train = new RecordReaderDataSetIterator(recordReaderTrain, batchSize, 1, 1, true);
            //RecordReaderDataSetIterator test = new RecordReaderDataSetIterator(recordReaderTest, 1, 1, 1, true);
            //train.setPreProcessor(new ImagePreProcessingScaler(0, 1));
            //test.setPreProcessor(new ImagePreProcessingScaler(0, 1)); 

REGRESSION DATA

 RecordReader r0= new CSVRecordReader(',');
 r0.initialize(new FileSplit(new File( DIR+"/r0.csv")));

 RecordReader r1=...
 ...

I tried to implement the Multi-Task Learning example from RecordReaderMultiDataSetIterator Example 2

MultiDataSetIterator train = new RecordReaderMultiDataSetIterator.Builder(batchSize)
                .addReader("rr", recordReaderTrain)
                .addReader("r0", r0)
                .addInput("???")//recordReaderTrain.getInputImagesData() ?
                .addOutput("rr")//recordReaderTrain.getVocLabelData() ?
                .addOutput("r0")
                .addOutput("r1")
                ... //addOutput -> r2,r3,...
                .build();

How can I correctly configure the input from the record reader that feeds from the VocLabelProvider with data preprocessing or is it possible to wrap the MultiDataSetIterator around another dataset iterator i.e. a multi dataset iterator from two dataset iterators from the .xml object-detection and .csv regression files

Ahmad
  • 1,618
  • 5
  • 24
  • 46
linker
  • 821
  • 1
  • 8
  • 20
  • specifically I need help with an example that implements a `MultiDataSetIterator` with `ObjectDetectionRecordReader` and multiple `RecordReader`s – linker Jan 15 '20 at 07:56

0 Answers0