4

When i try importing the s3fs library in pyspark using the following code:

import s3fs

I get the following error:

An error was encountered: cannot import name 'maybe_sync' from 'fsspec.asyn' (/usr/local/lib/python3.7/site-packages/fsspec/asyn.py) Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/s3fs/init.py", line 1, in from .core import S3FileSystem, S3File File "/usr/local/lib/python3.7/site-packages/s3fs/core.py", line 12, in from fsspec.asyn import AsyncFileSystem, sync, sync_wrapper, maybe_sync ImportError: cannot import name 'maybe_sync' from 'fsspec.asyn' (/usr/local/lib/python3.7/site-packages/fsspec/asyn.py)

The fsspec package has been installed in my notebook. And I actually had been using it fine for a long time, when this suddenly happened. I tried googling, but could not find this specific error. Has anyone come across this before? And if so, do you know how to solve it?

Andrew Gaul
  • 2,296
  • 1
  • 12
  • 19
thentangler
  • 1,048
  • 2
  • 12
  • 38

2 Answers2

13

Glad to hear this wasn't just me. It looks like if you pip install versions s3fs==0.5.1 and fsspec==0.8.3, that should fix it.

Andrew Gaul
  • 2,296
  • 1
  • 12
  • 19
Paul Fornia
  • 452
  • 3
  • 9
  • 3
    Yeah that worked. Also you have to change your bootstrap settings to install the older version of `fsspec` and `s3fs`. This issue was caused because s3fs released a new version on April 20th and it doesnt use one of the functions that was in the old `fsspec`. They need to allow for backward compatibility :( – thentangler Apr 23 '21 at 15:26
0

be carefull when changing the version of s3fs and fsspec ! in case your code turns even worse than before

pip install --upgrade botocore
pip install --upgrade s3fs
pip install --upgrade sfspec
Tyler2P
  • 2,324
  • 26
  • 22
  • 31
Jugurtha Semai
  • 1
  • 1