2

I found the solution for reading a "single" csv file in a Datalab : How can i load my csv from google dataLab to a pandas data frame?

But I wonder how I could read "multiple" csv files in Datalab.

What I tried is like this : 

variable_list = ['IBM', 'SPY']
for variable in variable_list:
  file_path = "gs://chois-trader-bucket/data/" + variable + ".csv"
  %%storage read --object file_path --variable variable

But this one failed because python variable is not compatible with magic command.

How can I deal with multiple csv files effectively?

user3595632
  • 5,380
  • 10
  • 55
  • 111

2 Answers2

1

You can use variables enclosed in braces. 

e.g) %storage read --object {file_path} --variable variable
Jay
  • 397
  • 1
  • 5
  • 19
0

Alternatively, you can do a union of all the csv files in a folder using a one liner in bash as such (assuming no headers in the csv files) :

cat *.csv > unioned_file_name.csv

Then import that one file into pandas.

Jas
  • 794
  • 12
  • 22