Python: reading N number from file, M at time

Question

My file is this one:

How can I read M number at time? for example 4 at time. Is it necessary to use two for loops?

My goal is to create (N/M)+1 lists with M numbers inside every lists, except the final list (it's the reminder of division N/M)

Have you tried implementing it with two `for` loops? That would be a good place to start, and if you have trouble, you can ask for help. — amiller27, May 15 '16 at 18:25

SparkAndShine · Answer 1 · 2016-05-15T20:01:10.097

Use itertools.islice,

import itertools 
import math

filename = 'test.dat'
N = 9
M = 4

num_rest_lines = N
nrof_lists = int(math.ceil(N*1.0/M))

with open(filename, 'r') as f:
    for i in range(nrof_lists):
        num_lines = min(num_rest_lines, M)
        lines_gen =  itertools.islice(f, num_lines)

        l = [int(line.rstrip()) for line in lines_gen]
        num_rest_lines = num_rest_lines - M

        print(l)    
        # Output
        [14, 3, 21, 37]
        [48, 12, 4, 6]
        [22]

Previous answer: Iterate over a file (N lines) in chunks (every M lines), forming a list of N/M+1 lists.

import itertools 

def grouper(iterable, n, fillvalue=None):
    """iterate in chunks"""
    args = [iter(iterable)] * n
    return itertools.izip_longest(*args, fillvalue=fillvalue)

# Test
filename = 'test.dat'
m = 4
fillvalue = '0'

with open(filename, 'r') as f:
    lists = [[int(item.rstrip()) for item in chuck] for chuck in grouper(f, m, fillvalue=fillvalue)]
    print(lists)
    # Output
    [[14, 3, 21, 37], [48, 12, 4, 6], [22, 4, 0, 0]]

i don't know why izip_longest method doesn't appear when i import itertools. I use Anaconda. i can't call it — Lorenzo Vannucchi, May 17 '16 at 08:27
i type "from itertools import..." but Eclispe don't found izip_method. No error, just this — Lorenzo Vannucchi, May 17 '16 at 08:40
@LorenzoVannucchi, if import a moduly by `from itertools import izip_longest`, this line `itertools.izip_longest` should be replaced with izip_longest`. — SparkAndShine, May 17 '16 at 09:34

Tanu · Accepted Answer · 2016-05-15T18:48:34.857

0

You can use python list slice operator to fetch the number of required elements from a file by reading a file using readlines() where each element of list will be one line of file.

with open("filename") as myfile:
    firstNtoMlines = myfile.readlines()[N:N+M] # the interval you want to read
    print firstNtoMlines

edited May 15 '16 at 18:48

answered May 15 '16 at 18:40

Tanu

1,503
12
21

I've tried with this solution. I've put it into a while cycle – Lorenzo Vannucchi May 17 '16 at 08:22
yes, it should work, my code is in a post above this one – Lorenzo Vannucchi May 17 '16 at 08:33

Lorenzo Vannucchi · Answer 3 · 2016-05-17T08:35:35.093

0

Now my code is this one:

N = 4
M = 0

while (M < 633):
    with open("/Users/Lorenzo/Desktop/X","r") as myFile:
        res = myFile.readlines()[M:N]
        print(res)
    M+=4
    N+=4

so, It should work. My file's got 633 numbers

edited May 17 '16 at 08:35

answered May 17 '16 at 08:22

Lorenzo Vannucchi

359
1
3
11

score -1 · Answer 4 · edited May 23 '17 at 11:44

-1

This has been asked before.

from itertools import izip_longest
izip_longest(*(iter(yourlist),) * yourgroupsize)

For the case of grouping lines in a file into lists of size 4:

with open("file.txt", "r") as f:
    res = izip_longest(*(iter(f)),) * 4)
    print res

Alternative way to split a list into groups of n

edited May 23 '17 at 11:44

Community

1
1

answered May 15 '16 at 18:35

gnicholas

2,041
1
21
32

Python: reading N number from file, M at time

4 Answers4