I am downloading a dataset from Google BigQuery using pandas like so
df = pd.read_gbq(query, location="US"...)
and it tells me it is downloading. But where is the data saved? I need to know before calling in the heavy loads...
Asked
Active
Viewed 297 times
0
Cherisy Konstanz
- 3
- 1
-
Clarifying question: are you asking where pandas stores the temporary file while it's being downloaded or where the final result is once it's done? – avip Jan 17 '20 at 22:16
-
now all data is in your variable df , print it ...if you want put it into csv try with df.to_csv('myfile.csv') – GiovaniSalazar Jan 17 '20 at 23:50
-
print(df.head(10)) – GiovaniSalazar Jan 17 '20 at 23:51
-
I wonder where pandas stores the temporary file BEFORE I save it to .csv etc. – Cherisy Konstanz Jan 18 '20 at 09:01
1 Answers
0
Python has an object allocator that is responsible for allocating memory within the object memory area. It directly uses the memory of the machine where Jupyter is running to store the DataFrame you obtain from your pd.read_gbq call. Check CPython’s Memory Management for more more insights on this. Here you have different methods to estimate the size in memory that this DataFrame will need.
Javier Bóbeda
- 468
- 2
- 10
-
Thank you very much. Unfortunately, my upvote will not be displayed, since I am new. – Cherisy Konstanz Jan 22 '20 at 20:09