ImportError: Missing optional dependency 'S3Fs'. The S3Fs package is required to handle S3 files. Use pip or conda to install S3Fs

Question

I am using AWS Cloud9 as my IDE.

import boto3
import pandas as pd
# import s3fs
# s3_ob=boto3.resource('s3',aws_access_key_id="xxxxxxxxxx",aws_secret_access_key="xxxxxxxxxxxx")
client=boto3.client('s3')
path="s3://xxxxxx/FL_insurance_sample.csv"

df=pd.read_csv(path)
# df.head()
print(df)`

While I am able to get the output CSV file in Pycharm, when I use the same code in Cloud9 IDE on AWS I get the error mentioned in title.

I have installed the S3Fs using the pip install S3Fs and when I do "pip list" it does give me the list installed which contains S3Fs so I am confused, why am I getting this error when the module is already installed and it shows in the pip list, and I also tried uncommenting the import S3Fs and still the same error is there.

Please help me.

Welcome Adithya. Have you tried reinstalling with `pip3 install s3fs --user`. I assume you are using latest version of Python 3, so you should be using `pip3` instead. Furthermore, check that you have activated the conda environment correctly by checking `conda env list` will show you the list of environments you have and the one with * is the currently active one — carkod, Feb 04 '20 at 11:36
it shows pip 3 command not found in the bash,tried both pip3 install s3fs --user and conda env list @carkod — adithya gopikrishnan, Feb 04 '20 at 11:41

Saurabh Jain · Accepted Answer · 2020-06-09T11:41:18.563

18

Installed the s3fs package using pip. It was showing up in pip list. But it did not show up when I did: conda list.

I resolved this problem by doing the following steps:

Installed the s3fs dependecy using conda install -c conda-forge s3fs
Restarted kernel
Imported s3fsand pandas
Read the csv file which was in my s3 bucket.

PS: It was library issue.

edited Jun 09 '20 at 11:41

answered Jun 09 '20 at 11:32

Saurabh Jain

1,600
1
20
30

I'm having the exact same issue except s3fs *does* show up in both pip list and conda list. Any ideas? Thanks – AiDev Feb 09 '21 at 14:12
try this: create a new environment altogether and then switch to it. see if this issue is fixed. most of the times the problem is with the environment. – Saurabh Jain Feb 09 '21 at 14:26
2

Thanks, actually a new env didn't do the trick, but I did need to run "conda install pip" in the env itself first, then use that env's version of pip to install s3fs. – AiDev Feb 09 '21 at 15:30

score 7 · Answer 2 · edited May 02 '21 at 12:52

7

pip3 install s3fs

solved the issue for me.

edited May 02 '21 at 12:52

Red

26,798
7
36
58

answered May 01 '21 at 11:06

Wei Shan Lee

470
2
5
11

score 2 · Answer 3 · edited Nov 13 '20 at 13:52

2

You just have to install the package. Open you Command Prompt and type:
pip install fsspec

edited Nov 13 '20 at 13:52

Roman Patutin

2,171
4
23
27

answered Nov 12 '20 at 16:39

mike black

21
1

score 0 · Answer 4 · answered Jul 04 '23 at 06:32

0

For me, the problem was missing aiobotocore package.

pip install aiobotocore

answered Jul 04 '23 at 06:32

Hezi Zisman

78
2
7

ImportError: Missing optional dependency 'S3Fs'. The S3Fs package is required to handle S3 files. Use pip or conda to install S3Fs

4 Answers4