Stack Overflow
Stack Overflow

Questions tagged [read.fwf]

25 questions
5
votes
2 answers

Finding bogus data in a pandas dataframe read with read_fwf()

I'm trying to analyse the weather records for New York, using the daily data taken from here: http://cdiac.ornl.gov/epubs/ndp/ushcn/daily_doc.html I'm loading the data with: tf = pandas.read_fwf(io.open('state30_NY.txt'), widths=widths, names=names,…
Marcin
  • 48,559
  • 18
  • 128
  • 201
3
votes
6 answers

R separate lines into columns specified by start and end

I'd like to split a dataset made of character strings into columns specified by start and end. My dataset looks something like this: >head(templines,3) [1] "201801 1 78" [2] "201801 2 67" [3] "201801 1 13" and i'd like to split it by specifying…
chrisjacques
  • 635
  • 1
  • 5
  • 17
2
votes
4 answers

How to tidy a fixed width file with headers every n (varies) rows?

I have time series data in a fixed-width-file where observation rows (n varies depending on the sample size) occur under a "header" row which contains the important meta data (ie. sample number, date, etc.). Both types of rows contain alphanumeric…
Andrew Smith
  • 23
  • 2
2
votes
1 answer

R - How to Read in a .fwf that contains a # sign

I am attempting to read in a large fixed width file into R using read.fwf, but I keep getting the error "Error in scan(file = file, what = what, sep = sep, quote = quote, dec = dec, : line 47 did not have 41 elements". My dataset has 41 columns…
Braden
  • 345
  • 5
  • 11
2
votes
1 answer

read.fwf error "line x did not have 5 elements" - maybe due to special characters

fwf to read fixed width text: lines = NULL lines[1] = ' BUTORPHANOL TARTRATE VIAL 2 MG/ML ' lines[2] = ' B3/AZEL AC/ZINC/B6/COPPER/FA TABLET 600-5-500 ' write(lines,…
YJZ
  • 3,934
  • 11
  • 43
  • 67
1
vote
1 answer

How to properly read fixed-width format files

Relatively new to R and R Studio and I'm trying to reformat a text file to run some analysis on the data within it. I'm currently trying to use read.fwf to tidy the data but seem to be doing something wrong, resulting in various errors. I've…
TheIllusiveNick
  • 49
  • 1
  • 8
1
vote
2 answers

Import .dat file from repository using R

I would like to import this dataset into a df. I'm trying to convert this SAS support file to R code using read.fwf approach Define variables name and lenght as described in the SAS file vars <- c('SEQN', 'HYK1A', 'HYK1B', 'HYK2A', …
Borexino
  • 802
  • 8
  • 26
1
vote
1 answer

Reading fixed width format data into R with entries exceeding column width

I need to use the Annual Building Permits by Metropolitan Area Data distributed by the US Census Bureau, which are downloadable here as fixed width format text files. Here is an excerpt of the file (I've stripped the column names as they aren't in a…
user2424281
  • 23
  • 3
1
vote
1 answer

Is there any way to read data by bytes length in R

Is there any way to read data by bytes length in R like SAS input command? When some multi-bytes characters are in a table as fixed column length, aaa대전11b1 bb 서울21b2 ccc부산갑b3 SAS can read it by bytes length as below. data test; infile…
resol
  • 13
  • 3
1
vote
1 answer

blank.lines.skip = TRUE fails with read.fwf?

There are four blank lines at the end of my file. > data=read.fwf("test2",head=F,widths=c(3,1,-3,4,-1,4),blank.lines.skip = TRUE) > data When I run this code, the blank.lines.skip argument is ignored. I still get blank lines in my output. The…
Peng Peng
  • 1,305
  • 3
  • 16
  • 20
0
votes
1 answer

R, arrow: How to read .fwf format in R using arrow

I have a data that is stored in several fixed width files (perhaps not fitting memory). It would be great to be able to read that using arrow. p <- 'path_to_my_files' # p contains: a.txt, b.txt, c.txt,.... all fwf dic <- fread('fwf_dictionary.csv')…
LucasMation
  • 2,408
  • 2
  • 22
  • 45
0
votes
0 answers

read.fwf in is statement

My data includes different sizes rows for record types. Every format of record types is different. I need to use read.fwf in if statement. I have a basic example below: filexx <- file.choose() conn <- file(filexx,open="r") linn <-readLines(conn) …
Derya Sahin
  • 1
0
votes
2 answers

Use list items as column seperators pd.read_fwf

I have text files containing tables which I want to put into a dataframe. Per file the column headers are the same, but the width is different depending on the content (because they contain names of different lengths for example). So far I managed…
Not_a_Robot
  • 43
  • 8
0
votes
2 answers

R: read.fwf defines integer as numeric

I have a .txt file and am using Rstudio. 200416657210340 1665721 20040608 20090930 20060910 20070910 20080827 20090804 200416657210345 1665721 20040907 20090203 20070331 20080719 200416657210347 1665721 20040914 20091026 20070213…
Thordis
  • 87
  • 1
  • 11
0
votes
2 answers

How do I parse a movie script for lines of dialogue that have consistent spacing with R?

''' A stray SKATEBOARD clips her, causing her to stumble and spill her coffee, as well as the contents of her backpack. The young RIDER dashes over to help, trembling when he sees who his board has…
bob0901
  • 65
  • 5
1
2