Stack Overflow
Stack Overflow

Questions tagged [read-fwf]

14 questions
2
votes
2 answers

`read_fwf` and `vroom_fwf` accidentally skipping first lines?

I'm sure I'm doing something silly, but I can't quite figure it out. Both read_fwf and vroom_fwf are producing files that lack one line (the first line, to be precise) when importing fixed-width files. There are two…
Kim
  • 4,080
  • 2
  • 30
  • 51
2
votes
2 answers

mismatched columnspecs, therefore wrong read values with pd.read_fwf and using colspecs

I am reading a text file using pd.read_fwf as below: import pandas as pd specs_test =[(19, 20),(20, 21),(21, 23),(23,26)] names_test = ["Record_Type","Resident_Status","State_Occurrence_FIPS", "County_Occurrence_FIPS"] test_l =…
BobbyF
  • 431
  • 1
  • 7
  • 19
2
votes
2 answers

Add header to .data file in Pandas

Given a file with the extention of .data, I have read it with pd.read_fwf("./input.data", sep=",", header = None): Out: 0 0 63.0,1.0,1.0,145.0,233.0,1.0,2.0,150.0,0.0,2.3... 1 67.0,1.0,4.0,160.0,286.0,0.0,2.0,108.0,1.0,1.5... 2 …
ah bon
  • 9,293
  • 12
  • 65
  • 148
1
vote
1 answer

Python pandas read_fwf strips white space

I am facing an issue using the read_fwf command from the Python library pandas, same as described in this unresolved question I want to read an ascii file containing results of numeric computations. The file contains repetitive header blocks with a…
TheDronist
  • 68
  • 8
1
vote
0 answers

python pandas read_fwf separate by \t

I have tried these variations, df = pd.read_fwf("test.txt", sep='\t', header = 20, engine='python') df = pd.read_fwf("test.txt", sep=r'\\t', header = 20, engine='python') df = pd.read_fwf("test.txt", sep='\\\\t', header = 20,…
Python account
  • 427
  • 1
  • 5
  • 15
1
vote
1 answer

How to process a file located in Azure blob Storage using python with pandas read_fwf function

I need to open and work on data coming in a text file with python. The file will be stored in the Azure Blob storage or Azure file share. However, my question is can I use the same modules and functions like os.chdir() and read_fwf() I was using in…
MikePy
  • 23
  • 1
  • 6
0
votes
1 answer

R, arrow: How to read .fwf format in R using arrow

I have a data that is stored in several fixed width files (perhaps not fitting memory). It would be great to be able to read that using arrow. p <- 'path_to_my_files' # p contains: a.txt, b.txt, c.txt,.... all fwf dic <- fread('fwf_dictionary.csv')…
LucasMation
  • 2,408
  • 2
  • 22
  • 45
0
votes
0 answers

Stata vs R: loop to load multiple inputs with unique fixed width variables

I am curious if anyone has a good R equivalency for the Stata program command. I have a bunch of fixed width .asc files that I needed to load up individually because fixed width positions for the same data change year to year. Stata program command…
gered
  • 79
  • 5
0
votes
0 answers

Issue when using df_dict2 = pd.read_fwf(io.StringIO(read_data), delim_whitespace=True)

Issue: when loading data frame from string CALLED read_data as follow: import pandas as pd import io read_data = """ operational applied description description …
Khader Mohammad
  • 1
  • 1
0
votes
1 answer

Parse Problematic Fixed width text file to a pandas dataframe

I need to parse a FWF to a df, but the system that exports the file instead of & symbol exports &. This break the fixed-width as the lines with & have now more characters so I cant use read_fwf. I can import it with the below code, but this…
Mike
  • 67
  • 6
0
votes
1 answer

Trying to use pandas read_fwf, but in my case I have start position and size but want to fill values in multiple columns

Trying to use pandas read_fwf, but in my case I have start position named position in the json and column-size having size and I want to use fill values from flatfile in to the multiple columns from the same position. Below is the sample json and…
Arvind
  • 1
0
votes
0 answers

Parsing Part of a file that is fixed width

I've got a file that is fixed width in the first half and table delimited by "," in the second half. When I copy part of the file and save as another file, the python function pandas.read_fwf(file_path) works perfectly as read_fwf can infer column…
Chris Carfango
  • 1
0
votes
1 answer

Truncated Beginning String Characters When Reading txt file to pandas

I am trying to read a txt file to a pandas data using pandas.read_fwf. Here's my line of code: klia_sepang = pd.read_fwf('KLIA_SEPANG.txt', sep='[ ]{1,}') However, I'm finding out that all string of 100th decimal places will be truncated at the…
aigoo
  • 3
  • 2
-1
votes
1 answer

How to fix an error code that removes whitespace separating columns in fwf file python pandas read_fwf

I am reading in an fwf file (using python/pandas' read_fwf) that normally has 2-3 spaces between the columns. However, when a certain error is thrown, the error code produced in the second column takes up extra spaces, and therefore removes the…
Oaklin Keefe
  • 1