12

I am trying to use python s3fs to read files in S3 AWS.

I could not find the code to put credential (Access key + Secret) into s3fs code.

Can anyone please help me how to set this info along with s3fs code.

import s3fs
fs = s3fs.S3FileSystem(anon=True)

I am currently on Windows 10.

gerrit
  • 24,025
  • 17
  • 97
  • 170
raju
  • 6,448
  • 24
  • 80
  • 163
  • This is all described pretty directly in the s3fs docs at http://fs-s3fs.readthedocs.io/en/latest/. Correction: I just realized this is about a different library also confusingly named s3fs... – Denise Draper Jun 04 '18 at 07:39

1 Answers1

25

Try the below

import s3fs
fs = s3fs.S3FileSystem(anon=False, key='<Access Key>', secret='<Secret Key>')

Below are the list of parameters for S3FileSystem

Parameters
----------
anon : bool (False)
    Whether to use anonymous connection (public buckets only). If False,
    uses the key/secret given, or boto's credential resolver (environment
    variables, config files, EC2 IAM server, in that order)
key : string (None)
    If not anonymous, use this access key ID, if specified
secret : string (None)
    If not anonymous, use this secret access key, if specified
token : string (None)
    If not anonymous, use this security token, if specified
use_ssl : bool (True)
    Whether to use SSL in connections to S3; may be faster without, but
    insecure
s3_additional_kwargs : dict of parameters that are used when calling s3 api methods.
       Typically used for things like "ServerSideEncryption".
client_kwargs : dict of parameters for the boto3 client
requester_pays : bool (False)
    If RequesterPays buckets are supported.
default_block_size: None, int
    If given, the default block size value used for ``open()``, if no
    specific value is given at all time. The built-in default is 5MB.
default_fill_cache : Bool (True)
    Whether to use cache filling with open by default. Refer to
    ``S3File.open``.
config_kwargs : dict of parameters passed to ``botocore.client.Config``
kwargs : other parameters for boto3 session
omuthu
  • 5,948
  • 1
  • 27
  • 37
  • 1
    Works with other clouds as well if you provide the endpoint with client_kwargs={'endpoint_url': "https://s3.eu-de.cloud-object-storage.appdomain.cloud"}) – Romeo Kienzler Jul 24 '21 at 10:30