1

Is there any way we can load direct excel file into BigQuery, instead of converting to CSV.

I get the files every days in excel format and need to load into BigQuery. Right now converting into CSV manually and loading into BigQuery.

Planning to schedule the job.

If not possible to load the excel files directly into BigQuery then I need to write a process(Python) to convert into CSV before loading into BigQuery.

Please let me know if any better options are there.

Thanks,

KeepLearn
  • 308
  • 2
  • 5
  • 19
  • I had a similar problem last week. I did use powershell to convert an excel file into a beautiful .csv adapted to BigQuery then I used NodeJs in order to use the Google API method "table.import". Since you're using Python, you should take a look at : https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/usage.html#tables Search for "Upload table data from a file:" I personnaly use an automated NodeJs script that upload a 50Mo csv every month. –  Oct 23 '17 at 11:51
  • Did you find an answer? – WJA Feb 18 '19 at 15:11

1 Answers1

1

I think you could achieve above in a few clicks, without any code. You need to use Google Drive and external (federated) tables.

1) You could upload manually you excel files to Google Drive or synchronise them

2) In Google Drive Settings find: 

"**Convert uploads** [x] Convert uploaded files to Google Docs editor format"

and check it.

To access above option go to https://drive.google.com/drive/my-drive, click on the Gear settings icon and then choose Settings.

Now you excel files will be accessible by Big Query

3) Last part: https://cloud.google.com/bigquery/external-data-drive You could access you excel file by URI: https://cloud.google.com/bigquery/external-data-drive#drive-uri and then create table manually using above uri.

You could do last step also by API.

rtbf
  • 1,509
  • 1
  • 16
  • 35