Does anyone know, how to adjust TestNG's DataProvider reading from CSV-file in equal chunks, not sequentially in general? General:
//Read data from CSV-file:
private static Object[][] getDataFromCSV(String fileNameroot) throws IOException {
List<Object[]> records = new ArrayList<>();
String record;
BufferedReader file = new BufferedReader(new InputStreamReader(new FileInputStream(fileNameroot), StandardCharsets.UTF_8));
file.readLine();
while ((record = file.readLine()) != null) {
String[] fields = record.split(", |,");
records.add(fields);
}
file.close();
Object[][] results = new Object[records.size()][];
for (int i = 0; i < records.size(); i++) {
results[i] = records.get(i);
}
return results;
}
Say, if we have 100 rows in file, so, it should (i.e. in 10 threads) for each thread read
firstly 1, 11, 21, ..., 91 rows
then 2, 12, 22, ..., 92
and so on...
till all 100 read
UPD: Currently, only simultaneous 2-way file read (forward and backward (thanks to honorable responders by How to read file from end to start (in reverse order) in Java?)) achieved to meet each other near the middle of the file:
//2-way read data from CSV-file:
private static Object[][] getDataFromCSV(String fileNameroot) throws IOException {
List<Object[]> records = new ArrayList<>();
String record;
String recordReversed;//
long lineCount, lineCountMiddle, curr = 0, currReversed = 0, overlap = 1;
try (Stream<String> stream = Files.lines(Path.of(fileNameroot), StandardCharsets.UTF_8)) {
lineCount = stream.count();
lineCountMiddle = (long) Math.ceil(((double)lineCount)/2d) + overlap;
System.out.println("lineCount = "+ lineCount +",\n lineCountMiddle = "+ lineCountMiddle);
}
BufferedReader file = new BufferedReader(new InputStreamReader(new FileInputStream(fileNameroot), StandardCharsets.UTF_8));
BufferedReader fileReversed = new BufferedReader(new InputStreamReader(new ReverseLineInputStream(new File(fileNameroot)), StandardCharsets.UTF_8));//Important! got from: https://stackoverflow.com/q/8664705/11714800
while (curr < lineCountMiddle && currReversed < lineCountMiddle) {
record = file.readLine();
recordReversed = fileReversed.readLine();
String[] fields = record.split("[,]\\s?|,");
String[] fieldsReversed = recordReversed.split("[,]\\s?|,");
records.add(fields);
records.add(fieldsReversed);
++curr;
++currReversed;
}
file.close();
fileReversed.close();//
Object[][] results = new Object[records.size()][];
for (int i = 0; i < records.size(); i++) {
results[i] = records.get(i);
}
return results;
}