19

Currently, in Databricks if we run the query, it always returns 1000 rows in the first run. If we need all the rows, we need to execute the query again.

In the situations where we know that we need to download full data(1000+ rows), is there a turn around way to execute the query to get all the rows in the first run without re-executing the query?

MrKrizzer
  • 409
  • 2
  • 5
  • 17

1 Answers1

4

There should be a down arrow next to the download button which allows you to Download full results. Are you seeing that in your interface? (Source)

Down arrow next to download button.

Jeremy Caney
  • 7,102
  • 69
  • 48
  • 77
Kaisa Crawford-Taylor
  • 87
  • 1
  • 5
  • 7
    Still doesn't answer the question. It is odd that we can't get the full results without executing the query twice. – Ahmad Abdelghany Nov 05 '20 at 14:02
  • 4
    I'm having this same issue. It's worse than having to run the query twice. If you are using Databricks to generate charts you can't get more than 1000 rows worth of data to show on the chart (this had me scratching my head as to why my sql all of a sudden stopped showing data past 2017). Download full results is very helpful if you are using an external tool to do visualizations etc. – John May 22 '21 at 11:02
  • 3
    im facing the same issue myself. unfortunately - i haven't found a databricks built in solution but a work around if you need all the data to plot it is to use the `toPandas` method to convert the spark dataframe to a pandas data from and use the pandas builtin plotting methods or use matplotlib or seaborn for more sophisticated plotting. – Mr T. Jan 22 '22 at 03:06
  • doesn't answer the question. – Jasmeet Kaur Mar 22 '22 at 20:10