Using Python to Parse a File for Nested Loops

Question

Currently Python reads each line of a file and executes a function doStep(). This function is passed whatever is on the line being read.

data.txt

a
b
c
b

read.py

fin = open('data.txt')
for step in fin:
    doStep(step)

Python will then execute

doStep(a)
doStep(b)
doStep(c)
doStep(b)

Question: If I want to define nested loops in data.txt, how should it be defined and parsed? The nesting and number of loop iterations should be defined in data.txt

For example, by reading data.txt I want to loop this 5 times

doStep('a')
doStep('b')
doStep('c')

and loop this 10 times

doStep('x')
doStep('y')
doStep('z')

and repeat everything 3 times (nesting).

I'm not sure what this has to do with either JSON or pickle? You'll have to define your own file format and detect when a certain format on the line means 'repeat this'. — Martijn Pieters, Mar 11 '14 at 13:27
Are you OK with at least one of the posted answers? Was one of them helpful? — Bach, Jul 16 '14 at 12:52

score 1 · Answer 1 · answered Mar 11 '14 at 13:31

You have a simple grammar in your file, currently (<item>\n)*. To parse this is easy, you just loop line by line.

You now want to increase the complexity of the grammar to include loops. How you do this is up to you. You might want to include a command which says 'jump to line number n' or some kind of recursive grammar which includes repetition.

Start by reading about EBNF. Then read about writing a simple parser. Yours will be much simpler than that example.

Your grammar might look like:

file = file_item*
file_item = single_item | repeated_items
repeated_items = single_item*
single_item = [a-z]*

Alternatively, you can define your own data structures (for example as Python classes) which represent repetitions, and serialise that (e.g. with pickle). You can then deserialise and 'play back' your data structures to retrieve the output.

score 1 · Answer 2 · answered Mar 11 '14 at 13:35

1

This is just an example, but there are many things of doing that.

Data file:

@5
a
b
c
@;
@10
x
y
z
@;

Python code:

times = None
buffer = []
fin = open('data.txt')
for line in fin:
    if line[0] == '@':
        if line[1] == ';':
            for step in buffer * times:
                doStep(step)
            times = None
            buffer = []
            continue
        times = int(line[1:])
        continue
    if times:
        buffer.append(line)
    else:
        doStep(line)

This solution is much worse in general than parsing, but it still a very simple (example) solution for a very simple task (unless you wish to generalize it, it may do).

answered Mar 11 '14 at 13:35

Bach

6,145
7
36
61

Instead of using a bugger, I would keep track of the label position and seek into it. Will save some memory but also requires some changes in your code. – Samy Arous Mar 11 '14 at 13:38
I don't think it will make much difference unless you are dealing with huge files. What about the extra disk activity that would create? What about invalidating the disk cache? – Joe Mar 11 '14 at 13:55

score 1 · Answer 3 · answered Mar 11 '14 at 13:35

The easiest way to implement a loop, would be to do something as easy as a label/loop operation. I would use a syntax which does not conflict with your data.

.. label1
a
b
c
-- label1 5
d
.. label2
x
y
z
-- label2 10

While reading your file I would keep track of the current position in file and keep track of all the labels. A stack can be used to hold the counters for each loop (handling nested loops).

When a loop is found, if the stack is empty put the number of iteration and the loop position on it and seek back to the label position. If the stack is not empty, pop the last value and decrease it. if it's not 0, put it back on the stack.

score 1 · Answer 4 · answered Mar 11 '14 at 13:37

You can place iteration number at first in line and data after it:

data.txt:

5, a, b, c
10, x, y, z

and use split to parse string:

with open('data.txt') as fin:
    for line in fin:
        l = line.split(', ')
        n, steps = l[:1], l[1:]
        for _ in range(n):
            for step in steps:
                doStep(step)

score 1 · Answer 5 · answered Mar 11 '14 at 13:41

1

Instead of an data.txt file you could use a normal data.py file. There you could import the routines from your read.py and executes the funktions/methodes with all the python magic.

answered Mar 11 '14 at 13:41

enthus1ast

2,099
15
22

Using Python to Parse a File for Nested Loops

5 Answers5

Linked