Divide large text file in small text file python

Asked Mar 29 '21 at 20:15

Active Mar 29 '21 at 22:42

Viewed 123 times

I tried this script to divide a large text file (1 GB) into several small text file, but when I execute it, only an empty TXT file is created named "file1.txt"

chunksize = 1000000
fid = 1
with open('texto.txt', encoding="utf-8") as infile:
    print("opened correctly")
    f = open('file%d.txt' % fid, 'w')
    for i, line in enumerate(infile):
        print(i)
        f.write(line)
        print(i, chunksize)
        if not i % chunksize:
            f.close()
            fid += 1
            f = open('file%d.txt' % fid, 'w')
    f.close()

When I try with a small text file the code works fine

edited Mar 29 '21 at 22:42

asked Mar 29 '21 at 20:15

Chariot

2

Just to be sure python is absolute requirement... https://stackoverflow.com/questions/2016894/how-to-split-a-large-text-file-into-smaller-files-with-equal-number-of-lines – yosukesabai Mar 29 '21 at 20:20
@yosukesabai yes – Chariot Mar 29 '21 at 20:21
When I ran this script with `chunksize = 2` it executed correctly and created text files according to chunksize. – Zain Arshad Mar 29 '21 at 20:24
@ZainArshad try it with a 1gb text file – Chariot Mar 29 '21 at 20:25
1

It works with a large file with two caveats: It outputs one line to the first file, and if the file isn't 7-bit clean, it may produce odd results since you don't set the encoding for the output files. – Anon Coward Mar 29 '21 at 20:29
This code works as expected for me (although file1.txt ends up containing only one line, because `enumerate()` starts at zero.) – John Gordon Mar 29 '21 at 20:29
Are you sure you're looking in the right directory for the output files? – John Gordon Mar 29 '21 at 20:30
@JohnGordon Yes, I'm sure – Chariot Mar 29 '21 at 20:33
How many empty output files are you seeing? Just one? – John Gordon Mar 29 '21 at 20:36
The marked duplicate focuses more on chooses the correct line count; you simply replace that with your chunk size. – Prune Mar 29 '21 at 20:37

Divide large text file in small text file python

0 Answers0