Reading data from two txt file and writing related informations to a third txt file

Question

Write a program that reads two country data files, worldpop.txt and worldarea.txt. Both files contain the same countries in the same order. Write a file density.txt that contains country names and population densities (people per square km).

worldpop.txt:

China 1415045928
India 1354051854
U.S. 326766748
Indonesia 266794980
Brazil 210867954
Pakistan 200813818
Nigeria 195875237
Bangladesh 166368149
Russia 143964709
Mexico 130759074
Japan 127185332
Ethiopia 107534882
Philippines 106512074
Egypt 99375741
Viet-Nam 96491146
DR-Congo 84004989
Germany 82293457
Iran 82011735
Turkey 81916871
Thailand 69183173
U.K. 66573504
France 65233271
Italy 59290969

worldarea.txt:

China 9388211
India 2973190
U.S. 9147420
Indonesia 1811570
Brazil 8358140
Pakistan 770880
Nigeria 910770
Bangladesh 130170
Russia 16376870
Mexico 1943950
Japan 364555
Ethiopia 1000000
Philippines 298170
Egypt 995450
Viet-Nam 310070
DR-Congo 2267050
Germany 348560
Iran 1628550
Turkey 769630
Thailand 510890
U.K. 241930
France 547557
Italy 294140

density.txt should be like this:

China 150.7258334947947
India 455.42055973550293
U.S. 35.72228540943785
Indonesia 147.27279652456156
Brazil 25.22905263611282
Pakistan 260.49945257368205
Nigeria 215.06553465748763
Bangladesh 1278.0836521471922
Russia 8.790734065789128
Mexico 67.26462820545795
Japan 348.8783091714556
Ethiopia 107.534882
Philippines 357.2192843009022
Egypt 99.8299673514491
Viet-Nam 311.1914922436869
DR-Congo 37.054757945347475
Germany 236.0955273123709
Iran 50.35874550980934
Turkey 106.43669165703
Thailand 135.41696451290883
U.K. 275.1767205389989
France 119.13512383185677
Italy 201.57397497790168

Program I write:

f=open('worldpop.txt','r')
f2=open('worldarea.txt','r')
out=open('density.txt','w')

for line1 in f: #finding country names
    pos1=line1.find(' ')
    country=line1[0:pos1]+'\n'

for line2 in f: #finding population numbers
    pos2=line2.find(' ')
    population=line2[pos2+1:]

for line3 in f2: #finding area numbers
    pos3=line3.find(' ')
    area=line3[pos3+1:]

for line1 in f: #writing density to a new file
    density=population/area
    out.write(density)
out.close()
f.close()
f2.close()

When I run the program, density.txt is empty. How can I fix this problem? Thanks. Note: I know the alternative solutions of it, but I mainly want to use this method to solve it, so please do not use other methods.

You can only iterate over an opened file object like `f` & `f2` once. After that, if you try to iterate over it again, your loop won't run even once. That's why your output file is empty. — rdas, Apr 10 '20 at 08:51
Does this answer your question? [iterating over file object in Python does not work, but readlines() does but is inefficient](https://stackoverflow.com/questions/13264805/iterating-over-file-object-in-python-does-not-work-but-readlines-does-but-is) — mkrieger1, Apr 10 '20 at 09:00

Patrick Artner · Answer 1 · 2020-04-10T09:04:13.963

Files are stream. They got a "pointer" to where in the stream they are. If you read them until the end, you cannot read again unless you close/reopen the stream or seek to an earlier position => iterating f a second time does not work.

You dont need that, as your countries occure in the same order. You can simple read the population file (one line), strip the newline at the end, then read one line from the area file and extract the area - then you put both together and add a newline and write that into your output.

Example:

with open('worldpop.txt','w') as r:
    r.write("""China 9388211
India 2973190
U.S. 9147420""")

with open('worldarea.txt','w') as f2:
    f2.write("""China 150.7258334947947
India 455.42055973550293
U.S. 35.72228540943785""")

with open('worldpop.txt') as r, open('worldarea.txt') as f2, open('d.txt','w') as out:
    for line1 in r: #finding country names
        l = line1.strip()
        area = next(f2).split()[-1]   # next(.) reads one line for that file

        out.write(f"{l} {area}\n")

with open("d.txt") as d:
    print(d.read())

Output of combined file:

China 9388211 150.7258334947947
India 2973190 455.42055973550293
U.S. 9147420 35.72228540943785

If you want to do calculations, you need to convert the numberstrings into numbers:

with open('worldpop.txt','r') as r, open(
    'worldarea.txt','r') as f2, open('density.txt','w') as out:
    for line1 in r: #finding country names
        l = line1.strip()
        ppl = float(l.strip().split()[-1])  # only use the number, convert to float
        area = float(next(f2).split()[-1])  # only use the number, convert to float

        # how many ppl per 1 area?
        d = ppl / area

    out.write(f"{l} {area} {d:f}\n")

with open("density.txt") as d:
    print(d.read())

To get how many ppl / area live:

China 9388211 150.7258334947947 62286.674967
India 2973190 455.42055973550293 6528.449224
U.S. 9147420 35.72228540943785 256070.402416

score 0 · Answer 2 · answered Apr 10 '20 at 09:01

You code doesn't enter to the last loop, because the f iterator is already exhausted at that point (you can iterate only once on each iterator and you did it already in your first loop).

Try to change your code in the way that you iterate each file only once.

By the way #1 - the same issue happens in your second loop.

By the way #2 - you should convert the extracted values of population and area to numeric types (float in your case) in order to divide them to find the density.

score 0 · Answer 3 · answered Apr 10 '20 at 09:25

Here is your functionnal and tested code :

pop = open('worldpop.txt', 'r') #r is for read
area = open('worldarea.txt', 'r')
out = open('density.txt', 'w')
poplist = pop.readlines() #will read the file and put each line into a list (new element)
arealist = area.readlines()
output = []
for i in range(len(poplist)):
     poplist[i] = poplist[i].split(" ")
     #slices the elements into lists, so we get a list of lists
     poplist[i][1] = poplist[i][1].replace("\n", "")
for i in range(len(arealist)):
     arealist[i] = arealist[i].split(" ")
     arealist[i][1] = arealist[i][1].replace("\n", "")
     #two indexes : we're in a list of lists
for i in range(len(poplist)):
    out.write(poplist[i][0]+" ")
    out.write(str(int(poplist[i][1])/int(arealist[i][1]))+"\n")
out.close()
pop.close()
area.close()

It works ! Instead of your use of loops to read the lines, I just use "readlines" wich makes everything way more simple. Then I just got to clear the lists and get the output.

score 0 · Answer 4 · answered Apr 10 '20 at 09:44

You can try this

Defining files names a variables

WORLD_POP_PATH = 'worldpop.txt'
WORLD_AREA_PATH = 'worldarea.txt'
OUTPUT_PATH = 'density.txt'

Rather than using file position for each country, it is more effective to use a dict with country name as the key.

def generate_dict(file_path):    
    with open(file_path, 'r') as f_world_pop:
        split_line = lambda x: x.split()
        return { 
                line[0]: line[1] 
                for line in 
                    map(split_line, f_world_pop.readlines())
                    }

We generate the world population and world area dict respectively with the content of the archives

world_pop = generate_dict( WORLD_POP_PATH )
world_area = generate_dict( WORLD_AREA_PATH )

And we generate a density list containing the name and pop. density

density = [f'{country} {int(world_pop[country])/int(world_area[country])}' 
               for country in world_pop.keys() ]

Finally, we write the result to output file

with open(OUTPUT_PATH, 'w+') as f_dens:
    f_dens.write('\n'.join(density))

score -1 · Answer 5 · answered Apr 10 '20 at 09:23

Should prefer to use with instead of open About with:What is the python keyword "with" used for?

The next method will iterate over an object instead of loading all file contents into memory

with open("./worldpop.txt") as f, open("./worldarea.txt") as f2, open("./density.txt","w") as out:
    for line in f:
        country, pop = line.split(" ")
        _, area = next(f2).split(" ")
        out.writelines(f"{country} {int(pop)/int(area)}\n")

Reading data from two txt file and writing related informations to a third txt file

5 Answers5