0

Based on many sources I understood that both(BigQuery and Bigtable) are considered as 2 different solutions for data sotrage.

For example here written that we can consider bigQuery as a data storage if we need any statistic(for example sums, averages, counts) about huge amount of data. In contrast, BigTable can be consideres like usual NoSql storage.

From another hand I've read snippet from the book where mentioned that:enter image description here

From this snippet I understood that BigQuery is a tool for querying from anywhere but not for data storage. Could you please clarify ?

gstackoverflow
  • 36,709
  • 117
  • 359
  • 710
  • Does this answer your question? [What's the difference between BigQuery and BigTable?](https://stackoverflow.com/questions/39919815/whats-the-difference-between-bigquery-and-bigtable) – Robert Harvey Mar 03 '20 at 16:14
  • @Robert Harvey sure not. I even mentioned that topic in my topic. – gstackoverflow Mar 03 '20 at 16:18
  • Then explain why it didn't answer your question here. – Robert Harvey Mar 03 '20 at 16:19
  • @Robert Harvey my question was appeared after reading topic you mentioned and I mentioned this topic in my question – gstackoverflow Mar 03 '20 at 16:27
  • 2
    I am voting to close this question. Google has extensive documentation on BigQuery and BigTable. Stack Overflow is for specific well defined programming questions. A simple Google Search shows many articles comparing the two products and use cases. – John Hanley Mar 03 '20 at 16:31
  • @John Hanley I've read most of them. But I still can't find answer to my question. Mybe because of my background or maybe I don't understand smth obvious. – gstackoverflow Mar 03 '20 at 16:33
  • Because of your Stack Overflow ranking, I thought carefully before voting to close your question.. – John Hanley Mar 03 '20 at 16:36
  • It's not at all clear from you question what you want us to "clarify." – Robert Harvey Mar 03 '20 at 16:37
  • @Robert Harvey looks like BigQuery consists of 2 entities: 1- Query engine(mandatory) and 2- datastore(optional). Data store optional because BigQuery can query from external datasources(Bigtable, ClouSQL and so on). Is it correct? If yes could you point me to the place where it is mentioned in the topic you suggested as a duplicate – gstackoverflow Mar 03 '20 at 16:52
  • I never said that specific information was in the duplicate. What makes you think your assertions about how BigQuery works might be incorrect? – Robert Harvey Mar 03 '20 at 16:54
  • 1
    There's a pretty good primer on how BigQuery works [here](https://panoply.io/data-warehouse-guide/bigquery-architecture/). Internally, it uses a data storage mechanism called "Capacitor." You can import your data into BigQuery storage via Batch loads or Streaming. – Robert Harvey Mar 03 '20 at 16:56
  • Storage is not optional with BigQuery. Queries to other sources are federated and the results are stored in BigQuery temporary tables. https://cloud.google.com/bigquery/docs/cloud-sql-federated-queries – John Hanley Mar 03 '20 at 18:21
  • @John Hanley from your link: **BigQuery Cloud SQL federation enables BigQuery to query data residing in Cloud SQL in real-time, without copying or moving data.** – gstackoverflow Mar 03 '20 at 19:28
  • Without copying or moving "source" data. The results are stored in temporary tables in BigQuery. – John Hanley Mar 03 '20 at 19:32
  • @John Hanley ok, agree. But what the reason to do it ? For the cashing reason ? What if someone else will modify underlying datasource ? – gstackoverflow Mar 03 '20 at 19:37
  • There is no link between the datasource and BigQuery after a query completes. – John Hanley Mar 04 '20 at 01:29
  • @JohnHanley what the reason to save query results in the bigQuery storage? – gstackoverflow Mar 04 '20 at 06:30

1 Answers1

0

BigQuery is a Data Warehouse solution in Google Cloud Platform. In BigQuery you can have two kinds of persistent table:

  1. Native/Internal: In this kind of table, you load data from some source (can be some file in GCS, some file that you upload in the load job or you can even create an empty table). After created, this table will store data on a specific BigQuery's storage system.
  2. External: This kind of table is basically a pointer to some external storage like GCS, Google Drive and BigTable

Furthermore, you can also create temporary tables that exists only during the query execution.

Both BigQuery and BigTable can be used as storage for data. The point is that BigQuery is more like an engine to run queries and make aggregations on huge amounts of data that can be on external sources or even inside BigQuery's own storage system. That makes it good for data analysis.

BigTable is more like a NoSQL database that can deal with petabytes of data and gives you mechanisms to perform data analysis as well. As you can see in the image below, BigTable should be chosen intead of BigQuery if you need low-latency.

enter image description here

rmesteves
  • 3,870
  • 7
  • 23
  • Where have you found this picture? I suppose it was taken from GCP official site of previous versions. I am not able to find it on the current site – gstackoverflow Mar 04 '20 at 14:26
  • I think this image is not available anymore on GCP website. However you can still find it in some blogs and certification preparing courses. – rmesteves Mar 04 '20 at 14:34
  • But they removed these pictures and it makes me doubt that it is still relevant – gstackoverflow Mar 04 '20 at 15:22
  • I see. In this link (https://cloud.google.com/products/databases) you can find not this specific image but a described usage of the GCP's Databse products. I hope it helps you – rmesteves Mar 04 '20 at 15:28