0

I want to read a directory of fixed width files into a single dataframe, and have tried using read_fwf from readr to do this using the following code:

library(plyr)
library(tidyverse)

file_list <- dir("./Data", full.names = TRUE)

col_widths <- rep.int(5, 4160)
data_import <- ldply(file_list, function(x) read_fwf(fwf_widths(col_widths), skip = 2))

This returns the error:

 Error: `file` must be a string, raw vector or a connection.

If I do read_fwf(file_list[[1]], fwf_widths(col_widths), skip = 2) then this does successfully import the data from the single file. What is the error that stops read_fwf from being applied to each element of file_list? Am I misunderstanding what ldply does?

Many thanks

CBrowne
  • 3
  • 1

2 Answers2

0

would be a solution with the apply-familiy do the trick?

if yes i suggest the following:

file_list <- <-list.files(pattern = "*./Data")
file_list <-sapply(file_list , read_fwf)
D.J
  • 1,180
  • 1
  • 8
  • 17
  • I get the same error using *apply as with the plyr functions. If I change from read_fwf to fread it reads the files but doesn't correctly populate the data, leading me to think it's something to do with using read_fwf/read.fwf on a list. – CBrowne Aug 26 '20 at 12:25
0

You need to pass the data as first argument in read_fwf. Try :

data_import <- ldply(file_list, function(x) 
                      read_fwf(x, fwf_widths(col_widths), skip = 2))

plyr has been retired so you might be interested in these alternate solutions :

#Option 1
data_import <- purrr::map_df(file_list, function(x) 
                    read_fwf(x, fwf_widths(col_widths), skip = 2))

#Option 2
data_import <- do.call(rbind, lapply(file_list, function(x) 
                   read_fwf(x, fwf_widths(col_widths), skip = 2)))
Ronak Shah
  • 377,200
  • 20
  • 156
  • 213