How do I write in a new text file every x lines? [python]

Question

Let's say I have a list with 10,000 entries. The user inputs a number, say 10. What I need to do is write all of the entries in the list to .txt files, but only 10 to each file. So the program would write the first 10, and then create a new file, and write the next 10... etc.

Thanks

count = 0
ext = 0
path = 'New/' + str(ext) + '.txt'
open(path, 'a').close()
for line in flines:
    f = open(path, 'a')
    if count <= x:
        f.write(line)
        count += 1
    else:
        ext += 1
        path = 'New/' + str(ext) + '.txt'
    count += x
    f.close()

This is what I've tried, amongst some other solutions. I have a feeling that I'm missing something simple.

This is different to the other question pointed out above as that's talking about a text file and this is talking about a list.

You seemed to have explained what you want to achieve... But if you could provide with information such as what you have tried out and where you are stuck, then maybe someone will be willing to help you out... — Alfie, Feb 01 '15 at 10:16
What is `sep` in `count += sep` and `x` in `count <= x` ?And it seems you are not resetting the count variable — Alfie, Feb 01 '15 at 10:23
*I have a feeling that I'm missing something simple* ... Yes, the brackets in the last line — Bhargav Rao, Feb 01 '15 at 10:43
@Alfie Fixed. Sep is the same as x. I just renamed sep to x to make it simple. Bhargav: Yeah, I didn't copy it properly. Fixed. — Awn, Feb 01 '15 at 11:17
Aurora, Both sep and x are undefined, so changing sep to x does not help. And what do you think this does: `open(path, 'a').close()` — 7stud, Feb 01 '15 at 11:19
@7stud That creates the file. Also, they're not undefined, they're defined earlier in the program but that wasn't relevant, so I didn't include it. There's only one, sep. I just wrote it as x above as sep doesn't really mean anything here. — Awn, Feb 01 '15 at 11:25
The *simple* thing would be to use `split`. Like this: `split -l $lines_per_file --numeric-suffixes=1 --suffix-length=5 your_file split_prefix` — Reut Sharabani, Feb 01 '15 at 11:56

Padraic Cunningham · Answer 1 · 2015-02-01T12:26:21.030

2

Very easy do with the itertools.grouper recipe:

from itertools import izip_longest
n = 2

l = ["1", "2", "3", "4", "5", "6"]

def grouper(iterable, n, fillvalue=""):
    args = [iter(iterable)] * n
    return izip_longest(fillvalue=fillvalue, *args)

for ind, ele in enumerate(grouper(l[:-n], n)):
    with open("new_{}.txt".format(ind), "w") as f:
        f.writelines(ele)

If you don't care about losing data just use zip/izip.

To not have any empty lines in the last file and still not lose data, as @Jon Clements suggested we can filter:

it = iter(l)
for num, chunk in enumerate(iter(lambda: list(islice(it, n)), []), 1):
    with open('output{}'.format(num), 'w') as fout:
        fout.write("\n".join(chunk))

edited Feb 01 '15 at 12:26

answered Feb 01 '15 at 11:49

Padraic Cunningham

176,452
29
245
321

The lines will already have an EOL character, so `"\n".join(ele)` is going to add blank lines... Might as well use `f.witelines(ele)` – Jon Clements Feb 01 '15 at 11:52
@JonClements, true I was using a different approach first. Not sure if the OP wants all the entries on a single line or one per line, the data apparently is coming from a list – Padraic Cunningham Feb 01 '15 at 11:55
And I suppose the other thing is that the last file may have up to 9 blank lines... whether that's desirable or not I suppose is down to the OP – Jon Clements Feb 01 '15 at 11:58
@JonClements, the OP can use zip/izip if they don't mind potentially losing data if the len(l) does not divide evenly by n – Padraic Cunningham Feb 01 '15 at 12:04
Or just filter out `''s`... or use a different grouping method, eg: https://gist.github.com/joncle/01e548943b4f53aeccaa – Jon Clements Feb 01 '15 at 12:06
@JonClements, is the original data coming from a file? – Padraic Cunningham Feb 01 '15 at 12:10
I don't believe so - it's just a code snippet I had laying about... easy to adapt for any iterable though... – Jon Clements Feb 01 '15 at 12:11
1

@JonClements, stole it :), just had to use join to separate the data – Padraic Cunningham Feb 01 '15 at 12:24

score 1 · Accepted Answer · answered Feb 01 '15 at 10:58

1

Use a generator to split lines list like this:

>>> def splited_lines_generator(lines, n):
...     for i in xrange(0, len(lines), n):
...             yield lines[i: i + n]
... 
>>> for index, lines in enumerate(splited_lines_generator(['a', 'b', 'c', 'd', 'e'], 2)):
...     with open('new/' + str(index) + '.txt', 'w') as f:
...             f.write('\n'.join(lines))
...

I have used ['a', 'b', 'c', 'd', 'e'] and 2 just as an example. After running this three files will be created, first two will contain two lines and last one will contain a single line (only e). Note that, directory new must be present before running this.

answered Feb 01 '15 at 10:58

taskinoor

45,586
12
116
142

Thank you! Could you comment your code, please? – Awn Feb 01 '15 at 11:20
@Aurora, at which line you are getting confused at? – taskinoor Feb 01 '15 at 11:25
In line two, does len(lines) return the number of entries in the list? What does yield do? What does this mean: 'for index, lines in enumerate' and what's .join()? – Awn Feb 01 '15 at 11:27
@Aurora, `len(lines)` returns the number of element in the list. `yield` creates a generator function. If you are not familiar with this then please Goole `python yield` or `python generator`. You will get tons of tutorials. This is very powerful concept. `enumerate` is simple. It allows you to get rid of manual incrementing a counter. `join` method joins all elements of a list by using a separator. Here `\n` is the separator. – taskinoor Feb 01 '15 at 11:32

How do I write in a new text file every x lines? [python]

2 Answers2