Split data in columns in python

Question

I'm trying to write a code that read from an external file a set of values like this:

   4.87320984E-09               1       49.882684417026184            0.18691906898647714            0.22222662961584136
   1.88917193E-09               2       49.882684417026184            0.18691906898647714            0.22222662961584136
   1.30942168E-09               3       49.882684417026184            0.18691906898647714            0.22222662961584136
   7.26580629E-10               4       49.882684417026184            0.18691906898647714            0.22222662961584136
   5.39025047E-10               5       49.882684417026184            0.18691906898647714            0.22222662961584136
   2.04686401E-08               1       68.253977742324736            -5.0977927905402787E-002       0.32783928527485512
   3.08666603E-09               3       68.253977742324736            -5.0977927905402787E-002       0.32783928527485512
   2.88387625E-09               4       68.253977742324736            -5.0977927905402787E-002       0.32783928527485512
   1.15430343E-09               5       68.253977742324736            -5.0977927905402787E-002       0.32783928527485512

I want to store every column in a list and do some maths with them.

import sys
inFile = sys.argv[1]
c1 = []
c2 = []
c3 = []
c4 = []
c5 = []
data = [c1,c2,c3,c4,c5]

with open(inFile,'r') as i:
   lines = i.read().split('\n')
for row in lines:
   #split in columns and do some maths
print data

related (using pandas): https://stackoverflow.com/questions/15026698/how-to-make-separator-in-read-csv-more-flexible-wrt-whitespace — Jean-François Fabre, Feb 21 '18 at 15:44
Possible duplicate of [How to read a dataset from a txt file in Python?](https://stackoverflow.com/questions/25013792/how-to-read-a-dataset-from-a-txt-file-in-python) — Georgy, Feb 21 '18 at 15:46

score 0 · Answer 1 · answered Feb 21 '18 at 15:46

with open("inFile",'r') as dataFile:
   for i in dataFile.readlines():
       print i.split()

Output:

['4.87320984E-09', '1', '49.882684417026184', '0.18691906898647714', '0.22222662961584136']
['1.88917193E-09', '2', '49.882684417026184', '0.18691906898647714', '0.22222662961584136']
['1.30942168E-09', '3', '49.882684417026184', '0.18691906898647714', '0.22222662961584136']
['7.26580629E-10', '4', '49.882684417026184', '0.18691906898647714', '0.22222662961584136']
['5.39025047E-10', '5', '49.882684417026184', '0.18691906898647714', '0.22222662961584136']
['2.04686401E-08', '1', '68.253977742324736', '-5.0977927905402787E-002', '0.32783928527485512']
['3.08666603E-09', '3', '68.253977742324736', '-5.0977927905402787E-002', '0.32783928527485512']
['2.88387625E-09', '4', '68.253977742324736', '-5.0977927905402787E-002', '0.32783928527485512']
['1.15430343E-09', '5', '68.253977742324736', '-5.0977927905402787E-002', '0.32783928527485512']

score 0 · Answer 2 · answered Feb 21 '18 at 15:49

0

import pandas as pd

inFile = YOUR_CSV.csv
df = pd.read_csv(inFile)
print(df.head(3) #prints first 3 rows to check your data

answered Feb 21 '18 at 15:49

mbauer

116
1
4

score 0 · Answer 3 · answered Feb 21 '18 at 15:50

I would definitely recommend Pandas for this task. Something like this should work

import pandas as pd
data = pd.read_csv(inFile, delimiter=r'\s+', header=None)

The delimiter parameter tells it to treat one or more whitespace characters as the separator. For the data shown in your example it should work.

score 0 · Answer 4 · answered Feb 21 '18 at 15:54

A lot of people are recommending pandas, but since this data only contains numbers, you may want to just consider numpy. A good tutorial is here: https://www.dataquest.io/blog/numpy-tutorial-python/.

Here is a sample of how to import data. From there you can use the full numpy library to do math or a multitude of operations on your array. See the tutorial I posted for more info.

import numpy as np

file = 'path_to_your_file.csv'

data_array = np.genfromtxt(file, delimiter=',')

score 0 · Answer 5 · answered Feb 21 '18 at 16:03

Here is the code you are looking for:

import sys
inFile = sys.argv[1]
c1 = []
c2 = []
c3 = []
c4 = []
c5 = []
data = [c1,c2,c3,c4,c5]

with open(inFile,'r') as i:
   for row in line:
       v1, v2, v3, v4, v5 = row.strip().split()
       c1.append(float(v1))
       c2.append(float(v2))
       c3.append(float(v3))
       c4.append(float(v4))
       c5.append(float(v5))
print data

But I would suggest using Pandas read_csv method.

Split data in columns in python

5 Answers5