Writing to different columns using python csv

Question

I am making a script that combines a bunch of txt files into one csv file, however because I am using the csv package I can only use writerow and not columns. Right now it just makes a csv file with one column and a bunch of rows.

def l_r_file(dirstart):
print(os.listdir())
with open("config.csv", 'w', newline='')as f:
    writer = csv.writer(f)
    all_dir = os.listdir()
    for x in all_dir[:-1]:
        os.chdir(dirstart + '\\' + x)
        working_file = open('Config.txt', "r")
        commaseperated = working_file.readlines()
        for i in commaseperated:
            writer.writerow([i])
        os.chdir(dirstart)
        working_file.close()

the way the config.txt files are set up is that its just one column of text with many rows. I am trying to set up the csv page so that I can compare the different txt files. So as an example lets say the first row in config.txt says "Emily", and the first row on another config file says bob, I want to be able to compare the two or more side by side in the csv file. But I opened/saved the strings in the txt files into list. This is a problem because usually csv files wants things in a row on the same list, like

rows = [ ['Nikhil', 'COE', '2', '9.0'],  
         ['Sanchit', 'COE', '2', '9.1'],  
         ['Aditya', 'IT', '2', '9.3'],  
         ['Sagar', 'SE', '1', '9.5'],  
         ['Prateek', 'MCE', '3', '7.8'],  
         ['Sahil', 'EP', '2', '9.1']]

where each list would turn into a single row but because of my data I have them in a list like this

List1 = ['emily', 'randy', 'tom']
List2 = ['bob', 'tom', 'astrid']

because I am reading each txt file and storing it into a list. Here's what my txt file might looks like.

emily
randy
tom
eric
lisa

I'm working with a bunch of txt files with a bunch of rows in each of them too. I feel like I am overthinking this. Please let me know if I am not clear thanks!. Also there is probably a easy way to do this on panda but I am trying to do it without panda.

I have tried combining them into a 2d list like this

combined_lst = [['emily', 'randy', 'tom'], ['bob', 'tom', 'astrid']]

and referencing them using a loop. However, since the txt files have different lengths eg. there are more names in one config file than the other it doesn't work.

One thing I have thought about is to reference each item in the list when it comes up, so like the first iteration in for loop for writerow could be like …(list1[I], list2[I]…) however, the list aren't uniform and some may have more than others. — civinkokoko, Oct 28 '20 at 16:58
Instead of writing to the csv as you are reading the files did you try saving each file's contents in a container like a list then add/append each of those to an *overall* container/list? Then you could use `zip` to transpose the data. [Transpose list of lists](https://stackoverflow.com/questions/6473679/transpose-list-of-lists) — wwii, Oct 28 '20 at 17:06
Your [mre] should always include an example of the data - enough to illustrate the problem you are having. Without being able to see the file *structure* it makes it hard. We also need to be able to copy from your question so we can test. — wwii, Oct 28 '20 at 17:08
@wwii Hi, I updated the question to include what it would look like. From my understanding zip would only work if there is a similar amount of items in the list, however, like I said the list aren't uniform so some list may have more items. — civinkokoko, Oct 28 '20 at 17:23
Please don't post images of code, data, or Tracebacks. Copy and paste it as text then format it as code (select it and type `ctrl-k`) ... [Discourage screenshots of code and/or errors](https://meta.stackoverflow.com/questions/303812/discourage-screenshots-of-code-and-or-errors) — wwii, Oct 28 '20 at 17:30
`working_file.read().split()` should give you a list like you want. — wwii, Oct 28 '20 at 18:16

wwii · Accepted Answer · 2020-10-28T18:26:23.093

Read the entire file and split it on whitespace. That should produce a a list with each row in it. Then write the entire row to the csv.

....
    for x in all_dir[:-1]:
        os.chdir(dirstart + '\\' + x)
        with open('Config.txt', "r") as working_file:
            data = working_file.read().split()
            writer.writerow(data)
        os.chdir(dirstart)

In your original, splitlines() gave you a list containing each line.

In [63]: working_file.readlines()
Out[63]: ['emily\n', 'randy\n', 'tom\n', 'eric\n', 'lisa']

So your original could be changed like this.

....
    for x in all_dir[:-1]:
        os.chdir(dirstart + '\\' + x)
        working_file = open('Config.txt', "r")
        commaseperated = working_file.readlines()
        #for i in commaseperated:
        writer.writerow(commaseperated)
        os.chdir(dirstart)
        working_file.close()

Notice that splitlines() retains the newline character at the end of each line. You probably do not want that in your csv file.

This works! I realized some of my data has more than one word so I just did data = working_file.read().split("\n") — civinkokoko, Oct 28 '20 at 18:33

Writing to different columns using python csv

1 Answers1