Read txt file of this type in as a Python numpy.ndarray

Question

I'd like to perform some dimension-reduction (DR) methods such as PCA, ICA and tSNE, maybe LEM on the a data set data.txt to compare the methods.

Therefore, I need to read in the data as a numpy.ndarray. Every line corresponds to a row in the matrix, with delimiter = ' '.

Alternatively, I have the file as a numpy.array now but as a string:

[ '16.72083152\t12.91868366\t14.37818919\n' ... '16.9504402\t7.81951173\t12.81342726']

How can I convert this quickly into a numpy.array of the desired format: n x 3, delimiter for rows = ' ', delimiter between elements in each row = '\t' chopping the '\n' at the end?

A quick answer much appreciated. Other tips as well. Thanks!

I'd suggest that you look into the Pandas library, which has many tools for parsing textual datafiles and data analysis, and also scikit-learn which has more sophisticated dimension-reduction and clustering techniques. — rwp, Mar 25 '18 at 12:31

Dan Steingart · Answer 1 · 2018-03-25T13:42:51.007

2

This should do it

import numpy 
try: from StringIO import StringIO
except ImportError: from io import StringIO

foo = '16.72083152\t12.91868366\t14.37818919\n16.9504402\t7.81951173\t12.81342726\n'
fn = StringIO.StringIO(foo) #make a file object from the string
data = numpy.loadtxt(fn) #use loadtxt with default settings.

edited Mar 25 '18 at 13:42

answered Mar 25 '18 at 12:28

Dan Steingart

765
1
7
15

Thanks! It seems like there is no module "StringIO" for me. I am using version 3.6.3 – Fei Mar 25 '18 at 13:06
use `io` instead of `stringio` per https://stackoverflow.com/questions/11914472/stringio-in-python3 – Dan Steingart Mar 25 '18 at 13:42
but the answer @era_misa obviates that – Dan Steingart Mar 25 '18 at 13:43

score 2 · Answer 2 · answered Mar 25 '18 at 12:30

2

you could just try below code:

import numpy as np
data = np.loadtxt('data.txt',delimiter='\t')

answered Mar 25 '18 at 12:30

era_misa

200
1
9

Read txt file of this type in as a Python numpy.ndarray

2 Answers2