0

I am using the Databricks community edition and working through the ML intro tutorial.

I am able to %fs ls databricks-datasets/COVID/covid-19-data/us-states.csv, but not able to read it through pandas

>>> df = pd.read_csv("/dbfs/databricks-datasets/COVID/covid-19-data/us-states.csv")

FileNotFoundError: [Errno 2] File /dbfs/databricks-datasets/COVID/covid-19-data/us-states.csv does not exist: '/dbfs/databricks-datasets/COVID/covid-19-data/us-states.csv'

Directly open the README.md file in databricks dataset also failed

>>> f = open("/dbfs/databricks-datasets/README.md", "r")

FileNotFoundError: [Errno 2] No such file or directory: '/dbfs/databricks-datasets/README.md'

Any thoughts or suggestions?

enter image description here

Ying Xiong
  • 4,578
  • 8
  • 33
  • 69
  • can you add the output of - `%fs ls databricks-datasets/COVID/covid-19-data/us-states.csv` – Vaebhav Sep 04 '21 at 05:49
  • `/dbfs` isn't supported on community edition. copy file to local system as described in the linked answer – Alex Ott Sep 04 '21 at 09:04
  • Re @Vaebhav: `%fs ls databricks-datasets/COVID/covid-19-data/us-states.csv` returns `dbfs:/databricks-datasets/COVID/covid-19-data/us-states.csv`. Added a screenshot to make it clear. – Ying Xiong Sep 04 '21 at 15:15
  • Re @AlexOtt: Thanks for the answer! Yes, your answer in https://stackoverflow.com/questions/63552922/ worked! – Ying Xiong Sep 04 '21 at 15:16

0 Answers0