Python reading a text file into a 2D array and accessing the data

Question

I am trying to read data from a text file into a 2D array and then access each element of the data. I have tried a number of different approaches but I am unable to access each element of the data,

Here is an extract of the data,

GRID     16             7.5     5.961539 0.
GRID     17             7.5     11.92308 0.
GRID     18             7.5     17.88461 0.
GRID     19             7.5     23.84615 0.
GRID     20             7.5     29.80769 0.
GRID     21             7.5     35.76923 0.
GRID     22             7.5     41.73077 0.
GRID     23             7.5     47.69231 0.
GRID     24             7.5     53.65384 0.

Using the example here, Import nastran nodes deck in Python using numpy

It imports OK but it as a 1D array and I 'ary[1,1]' for example, I get the following response,

x[1,1]
Traceback (most recent call last):

  File "<ipython-input-85-3e593ebbc211>", line 1, in <module>
    x[1,1]

IndexError: too many indices for array

What I am hoping for is,

I have also tried the following code and again this reads into a 1D array,

ary = []

with open(os.path.join(dir, fn)) as fi:
    for line in fi:
        if line.startswith('GRID'):
            ary.append([line[i:i+8] for i in range(0, len(line), 8)])

and I get the following error,

ary[1,2]
Traceback (most recent call last):

  File "<ipython-input-83-9ac21a0619e9>", line 1, in <module>
    ary[1,2]

TypeError: list indices must be integers or slices, not tuple

I am new to Python but I do have experience with VBA where I have used arrays a lot, but I am struggling to understand how to load an array and how to access the specific data.

Check the `shape` and `dtype`. Don't make assumptions about how the text was loaded. — hpaulj, Dec 19 '18 at 16:36

score 2 · Accepted Answer · answered Dec 19 '18 at 16:22

You can use genfromtxt function.

import numpy as np

ary = np.genfromtxt(file_name, dtype=None)

This will automatically load your file and detect fields type. Now you can access ary by row or by column, for example

In: ary['f1']
Out: array([16, 17, 18, 19, 20, 21, 22, 23, 24])

In: ary[2]
Out: (b'GRID', 18, 7.5, 17.88461, 0.)

or by single element:

In: ary[3]['f1']
Out: 19

In: ary['f1'][3]
Out: 19

Allen · Answer 2 · 2018-12-19T15:59:23.363

You are importing it from a text file? Can you save the text file as a csv? If so, you can easily load the data using pandas.

import pandas as pd

data = pd.read_csv(path_to_file)

Also, it might be that you just need to reshape your numpy array using something like:

x = x.reshape(-1, 4)

EDIT: Since your format is based on fixed width, you would want to use the fixed width in pandas instead of read_csv. Example below uses width of 8.

x = pd.read_fwf(path_to_file, widths=8)

Python reading a text file into a 2D array and accessing the data

2 Answers2