0

I have a fixed length file like: 0001ABC,DEF1234 The file definition is:

id[1:4]
name[5:11]
phone[12:15]

I need to load this data into a table. I tried to use CSV module and defined the fixed lengths of each field. It is working fine except for the name field. For the NAME field, only the value till ABC is getting loaded. The reason is: As I am using CSV module, it is treating 0001ABC, as a value and only parsing till that.

I tried to use escapechar = ',' while reading the file, but it removes the ',' from the data. I also tried quoting=csv.QUOTE_ALL but that didnt work either.

with open("xyz.csv") as csvfile:
    readCSV = csv.reader(csvfile)
    writeCSV = open("sample_csv", 'w');
    output = csv.writer(writeCSV, dialect='excel', lineterminator="\n")
    for row in readCSV:
         print(row) # to debug #
         data= str(row[0])
         print(data) # to debug #
         id = data[0:4]
         name = data[5:11]
         phone = data[12:15]
         output.writerow([id,name,phone])
    writeCSV.close()

Output of the print commands:

row: ['0001ABC','DEF1234'] data: 0001ABC

Ideally, I expect to see the entire set 0001ABC,DEF1234 in the variable: data. I can then use the parsing as mentioned in the code to break it into different fields.

Can you please let me know where I am going wrong?

Sourav Gupta
  • 227
  • 5
  • 17
  • @ Ken White: Are you suggesting not to use the CSV module. I can use a = open(filename.'r') and then b = a.read() – Sourav Gupta May 02 '19 at 02:08
  • 2
    You don't need the csv.reader() at all. You can just read the line and slice off the substrings you need. See https://stackoverflow.com/q/509211/62576. You don't have a CSV file, because your values are not comma-separated. You're using the wrong tool for the job. – Ken White May 02 '19 at 02:10
  • 1
    @ Ken White : Thanks a lot !!!. It did work. I am using the SYS module instead of csv. I am reading the file in a conventional way: a = open(filename.'r') and then b = a.read(). While writing the data in a csv prior to loading in Redshift, I am using the CSV module. – Sourav Gupta May 02 '19 at 11:11

0 Answers0