1

I try to write a method taking a BlockingQueue< List < String[]> > and a file path as a String, and adding to the queue each line of the file as an ArrayList containing the line as a String[]. (I will later add more String[] in each ArrayList.)

Exemple:

-queue = {{A1,A2},{B}} (A1,A2, B, are String[] )

-Lines in file = {l1,l2,L3} (l1,l2,l3 are also String[])

Wanted result:

-queue = {{A1,A2},{B},{l1},{l2},{l3}}

My problem is that when I add an element, the queue.add(element) returns true, but no element is added.

My code so far:

public boolean loadOneFile(BlockingQueue<List<String[]>> queue, String path) throws IOException, InterruptedException {

        CSVReader reader = new CSVReader(new FileReader(path), Main.separator.toCharArray()[0], '"', false);

        // Skip header
        int headerSize = reader.readNext().length;
        while (reader.isHasNext()) {
            String[] line = reader.readNext();
            ArrayList<String[]> rowInQueue = new ArrayList<String[]>();
            rowInQueue.add(line); // this line does work
            boolean test = queue.add(rowInQueue);// the variable "test" becomes true...
            int debug = queue.size(); // ...but the variable "debug" stays to 1...
            // ...and queue does not change and only has the same "kinda-empty" element from initialisation.    
            }

        }       
            reader.close();
        return true;
    }

The queue is initialized with a list containing an String[1] with "":

BlockingQueue<List<String[]>> toProcessQueue = new LinkedBlockingQueue<List<String[]>>(50000);
    String[] emptyLine = { "" };

    ArrayList<String[]> emptyCombinaison = new ArrayList<String[]>();
    emptyCombinaison.add(emptyLine);

    BlockingQueue<List<String[]>> emptyQueue = new LinkedBlockingQueue<List<String[]>>();
    boolean test = toProcessQueue.add(emptyCombinaison);

Context: the method is called in a class. The argument is a protected volatile attribute of the class.

Broader context: this is my first step to have a queue with each line of a file matched against each line of another file. Or an arbitrary number of big files. Exemple with 2 files: file1: {A,B} + file 2:{a,b,c} --> queue: {{A,a},{A,b},{A,c},{B,a},{B,b},{B,c}} (order doesn't matter) Bonus: it has to work for big files, by waiting the queue to empty if necessary.

My questions are:

1) Maybe there is a simpler (hence more reliable) method to this first step (1 file in the queue)?

2) Is there a simpler way to do this second step ("cartesian product" of 2 files)?

3) Why does .add(elmt) has no effect in this particular case?

Thanks.

Akita
  • 287
  • 2
  • 8
  • This works fine for me. I do find it a bit weird you keep a `Queue>` instead of just a `Queue` with the lines you read, the list you create only contains a single element anyway. – daniu Jun 18 '18 at 13:58
  • What is the `emptyQueue` used for in your initialization part of the code? I'm assuming you call your method `loadOneFile` with the `toProcessQueue` as parameter? I don't see anything wrong with the `loadOneFile` method itself, though. `queue.size()` should increase if I'm not mistaken. – Kevin Cruijssen Jun 18 '18 at 14:00
  • As for your question on generating the Cartesian product of an arbitrary amount of lists, [the external library Guava has a builtin for it that might be useful](https://stackoverflow.com/a/37490796/1682559). – Kevin Cruijssen Jun 18 '18 at 14:07
  • @daniu I will then add some data in the list (like metadatas for the line). – Akita Jun 18 '18 at 14:32
  • @Kevin Cruijssen Yes, I call loadOneFile with toProcessQueue as parameter. I also think queue.size() should increase (I blocked all the threads that will retrieve elements from the queue). That's why I'm surprised. – Akita Jun 18 '18 at 14:33

0 Answers0