2

I try to export the big query table to Google Cloud storage with format specified as JSON

Here, I noticed like columns with null values are not included in resulting JSON files

So Is there a way to get all the fields of row to be converted into JSON ?

My intention is to export data from table , do some transformations and reload the data back into new table . So , i basically need all fields to be included in generated JSON files .

For example , I tried exporting Bigquery-public-data.samples.wikipedia table

After exporting, JSON rows include only columns with non-null value

{
  "title": "Strait of Messina Bridge",
  "id": "1462053",
  "language": "",
  "wp_namespace": "0",
  "revision_id": "115349459",
  "contributor_ip": "80.129.30.196",
  "timestamp": "1173977859",
  "comment": "/* Controversy and concerns */",
  "num_characters": "20009"
}

Few columns like contrinutor_id , contributor_username , others with null values are not included in generated JSON

dsesto
  • 7,864
  • 2
  • 33
  • 50
Mangai
  • 95
  • 2
  • 6
  • Just a quick question - why the need to export to do your transformations? You could just do them directly in BigQuery. – Graham Polley Dec 18 '17 at 10:01
  • i want to move data from src to dest table , without modifying the src table , but in dest table wanted some transformations – Mangai Dec 18 '17 at 10:16
  • 1
    Yes, so what's the problem? Write a query over the src table (doing your transformations) and write the results to the destination table! This is starting to sound very like your other question here: https://stackoverflow.com/questions/47851957/pricing-of-bq-query-with-destination-table – Graham Polley Dec 18 '17 at 10:23
  • I have a huge amount of data in src table and it is non partitioned. I need to move data into specific partition with less cost. If im going to go for bq query with destination table, im going to be billed so much of querying the src table every time for each day – Mangai Dec 18 '17 at 10:27
  • 1
    There are several workarounds available to working with partitioned tables and avoiding those costs. See here: https://stackoverflow.com/questions/38993877/migrating-from-non-partitioned-to-partitioned-tables – Graham Polley Dec 18 '17 at 10:34
  • Does it have to be JSON format? If you export table to CSV format, then the scheme would be kept. – Xiaoxia Lin Dec 19 '17 at 15:01

0 Answers0