Iterate File Saving Blocks and Skipping Lines

Question

I have data in blocks with non-data lines between the blocks. This code has been working but is not robust. How do I extract blocks and skip non-data blocks without consuming a line in the index test? I'm looking for a straight python solution without loading packages.

I've searched for a relevant example and I'm happy to delete this question if the answer exists.

from __future__ import print_function

BLOCK_DATA_ROWS = 3
SKIP_ROWS = 2
block = 0

with open('array1.dat', 'rb') as f:
    for i in range (2):
        block += 1
        for index, line in enumerate(f):
            if index == BLOCK_DATA_ROWS:
                break
            print(block, 'index', index, 'line', line.rstrip('\r\n'))

        for index, line in enumerate(f):
            if index == SKIP_ROWS:
                break
            print('  skip index', index, 'line', line.rstrip('\r\n'))

Input

1
2
3
4
5
6
7
8
9

Output

1 index 0 line 1
1 index 1 line 2
1 index 2 line 3
  skip index 0 line 5
  skip index 1 line 6
2 index 0 line 8
2 index 1 line 9

Edit

I also want to use a similar iteration approach with an excel sheet:

for row in ws.iter_rows()

Answer 1

In the code posted, the line 4 is read, and the condition index == BLOCK_DATA_ROWS is met, leaving the first loop towards the second one. As f is a generator, when it is called in the second loop, it returns the next element to iterate over, and line 4 has already been returned to loop 1 (it is not printed, but the value is used).

This has to be taken into account in the code. One option is to combine both conditions in the same loop:

from __future__ import print_function

BLOCK_DATA_ROWS = 3
SKIP_ROWS = 2
block = 1

with open('array1.dat', 'r') as f:
    index = 0
    for line in f:
        if index < BLOCK_DATA_ROWS:
            print(block, 'index', index, 'line', line.rstrip('\r\n'))
        elif index < BLOCK_DATA_ROWS+SKIP_ROWS:
            print('  skip index', index, 'line', line.rstrip('\r\n'))
        index += 1
        if index == BLOCK_DATA_ROWS+SKIP_ROWS: # IF!!, not elif
            index = 0
            block += 1

The for i in range(2) has also been removed, and now the code will work for any number of blocks, not just 2.

Which returns:

1 index 0 line 1
1 index 1 line 2
1 index 2 line 3
  skip index 3 line 4
  skip index 4 line 5
2 index 0 line 6
2 index 1 line 7
2 index 2 line 8
  skip index 3 line 9
  skip index 4 line 10