Stack Overflow
Stack Overflow

Questions tagged [genfromtxt]

Numpy function to create arrays from tabular data.

References

287 questions
560
votes
14 answers

How do I read CSV data into a record array in NumPy?

Is there a direct way to import the contents of a CSV file into a record array, just like how R's read.table(), read.delim(), and read.csv() import data into R dataframes? Or should I use csv.reader() and then apply numpy.core.records.fromrecords()?
hatmatrix
  • 42,883
  • 45
  • 137
  • 231
43
votes
1 answer

numpy.genfromtxt produces array of what looks like tuples, not a 2D array—why?

I'm running genfromtxt like below: date_conv = lambda x: str(x).replace(":", "/") time_conv = lambda x: str(x) a = np.genfromtxt(input.txt, delimiter=',', skip_header=4, usecols=[0, 1] + radii_indices, converters={0: date_conv, 1:…
robintw
  • 27,571
  • 51
  • 138
  • 205
40
votes
5 answers

Using numpy.genfromtxt to read a csv file with strings containing commas

I am trying to read in a csv file with numpy.genfromtxt but some of the fields are strings which contain commas. The strings are in quotes, but numpy is not recognizing the quotes as defining a single string. For example, with the data in…
CraigO
  • 565
  • 1
  • 5
  • 6
29
votes
9 answers

Read in all csv files from a directory using Python

I hope this is not trivial but I am wondering the following: If I have a specific folder with n csv files, how could I iteratively read all of them, one at a time, and perform some calculations on their values? For a single file, for example, I do…
FaCoffee
  • 7,609
  • 28
  • 99
  • 174
22
votes
1 answer

Reading data into numpy array from text file

I have a file with some metadata, and then some actual data consisting of 2 columns with headings. Do I need to separate the two types of data before using genfromtxt in numpy? Or can I somehow split the data maybe? What about placing the file…
Nirvan
  • 337
  • 1
  • 3
  • 7
16
votes
8 answers

"Got 1 columns instead of ..." error in numpy

I'm working on the following code for performing Random Forest Classification on train and test sets; from sklearn.ensemble import RandomForestClassifier from numpy import genfromtxt, savetxt def main(): dataset =…
user3466132
  • 269
  • 1
  • 4
  • 11
14
votes
3 answers

NumPy dtype issues in genfromtxt(), reads string in as bytestring

I want to read in a standard-ascii csv file into numpy, which consists of floats and…
user2489252
11
votes
2 answers

numpy genfromtxt/pandas read_csv; ignore commas within quote marks

Consider a file, a.dat, with contents: address 1, address 2, address 3, num1, num2, num3 address 1, address 2, address 3, 1.0, 2.0, 3 address 1, address 2, "address 3, address4", 1.0, 2.0, 3 I am trying to import with numpy.genfromtxt. However the…
Lee
  • 29,398
  • 28
  • 117
  • 170
8
votes
1 answer

NumPy genfromtxt: using filling_missing correctly

I am attempting to process data saved to CSV that may have missing values in an unknown number of columns (up to around 30). I am attempting to set those missing values to '0' using genfromtxt's filling_missing argument. Here is a minimal working…
Thav
  • 387
  • 4
  • 9
7
votes
3 answers

Loading a date in Numpy genfromtxt

I'm trying to import a simple CSV file with Numpy genfromtxt but can't manage to convert the data of first column to dates. Here is my code: import numpy as np from datetime import datetime str2date = lambda x: datetime.strptime(x, '%Y-%m-%d…
Mark Morrisson
  • 2,543
  • 4
  • 19
  • 25
7
votes
3 answers

NumPy: mismatch in size of old and new data-descriptor

I ran into the following problem with NumPy 1.10.2 when reading a CSV file. I cannot figure out how to give explicit datatypes to genfromtxt. Here is the CSV, minimal.csv: x,y 1,hello 2,hello 3,jello 4,jelly 5,belly Here I try to read it with…
Akseli Palén
  • 27,244
  • 10
  • 65
  • 75
7
votes
2 answers

How to read columns of varying length from a text file in NumPy using genfromtxt()?

I have hundreds of text files like these, with each column separated by three spaces. The data is for a year: 12 months and 31 days for each month. Below, I'm only showing below what's relevant to question: 001 DIST - ADILABAD ANDHRA …
user3707588
  • 73
  • 1
  • 6
7
votes
3 answers

unable to read a tab delimited file into a numpy 2-D array

I am quite new to nympy and I am trying to read a tab(\t) delimited text file into an numpy array matrix using the following code: train_data = np.genfromtxt('training.txt', dtype=None, delimiter='\t') File contents: 38 Private 215646 …
Abhi
  • 163
  • 1
  • 1
  • 8
7
votes
1 answer

numpy.genfromtxt: Ambiguous delimiters?

I'm trying to write a generic script, part of which imports files that are either comma-separated or white-space-separated. I'd like the script to recognize either type. Is there a way to specify something like arrayobj = np.genfromtxt(file.txt,…
PikalaxALT
  • 327
  • 3
  • 12
6
votes
3 answers

Filling missing values using numpy.genfromtxt

Despite the advice from the previous questions: -9999 as missing value with numpy.genfromtxt() Using genfromtxt to import csv data with missing values in numpy I still am unable to process a text file that ends with a missing value, a.txt: 1 2 3 4…
Hooked
  • 84,485
  • 43
  • 192
  • 261
1
2 3
…
19 20