28

I am trying to connect to S3 using boto, but it seems to fail. I've tried some workarounds, but they don't seem to work. Can anyone please help me with this. Below is the code. 

import boto

if not boto.config.has_section('Credentials'):
    boto.config.add_section('Credentials')
boto.config.set('Credentials', 'aws_access_key_id', AWS_KEY)
boto.config.set('Credentials', 'aws_secret_access_key', AWS_SECRET_KEY)
if not boto.config.has_section('Boto'):
    boto.config.add_section('Boto')
    boto.config.set('Boto', 'https_validate_certificates', 'False')
    boto.config.add_section('aws info')
    boto.config.set('aws info','aws_validate_certs','False')



s3 = boto.connect_s3(validate_certs=False)
bucket = s3.get_bucket(Bucket_NAME)
John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Siddarth
  • 1,000
  • 1
  • 10
  • 17

7 Answers7

15

Probably your bucket name contains a dot, that's why ssl certificate verification fails. This is quite a frequent problem, see this github issue for example.

Don't use an insecure connection (is_secure=False), instead use OrdinaryCallingFormat:

import boto
conn = boto.s3.connect_to_region('eu-west-1', calling_format=boto.s3.connection.OrdinaryCallingFormat())
bucket = conn.get_bucket(your_bucket)

You probably need to update your AWS Region, e.g. us-east-1

linqu
  • 11,320
  • 8
  • 55
  • 67
12

In boto3, if you are using the s3 client, use verify=False when creating the s3 client. For eg: 

s3 = boto3.client('s3', verify=False)

As mentioned on boto3 documentation, this only turns off validation of SSL certificates. SSL will still be used (unless use_ssl is False), but SSL certificates will not be verified.

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html

peaxol
  • 497
  • 4
  • 13
  • 7
    This solution is insecure and should be used only, if certificate on the other side is invalid and nothing can be done about that. – GwynBleidD Dec 14 '18 at 14:52
  • This is useful for using boto3 with [localstack](https://github.com/localstack/localstack) docker (self-signed cert) – Joe Sadoski May 27 '21 at 18:54
11

I found a way,

used is_secure=False in connect_s3().

Eric O. Lebigot
  • 91,433
  • 48
  • 218
  • 260
Siddarth
  • 1,000
  • 1
  • 10
  • 17
  • This serves as a workaround though the real issue is amazon using invalid certs for subdomains. – WhyNotHugo Mar 07 '15 at 01:43
  • 1
    Thanks @Siddarth: there are many non-working solutions, out there. I would add that `validate_certs=False` does not do anything (its behavior is not documented anyway, like `is_secure`'s behavior: http://boto.readthedocs.org/en/latest/ref/s3.html#boto.s3.connection.S3Connection). – Eric O. Lebigot Apr 25 '15 at 00:12
  • 2
    Yes @EOL I had to go through every non working solution before I ended up with the right one. Was frustrated. – Siddarth Apr 25 '15 at 01:06
6

add verify=False

boto3.resource(
            "s3",
            endpoint_url=<URL>,
            aws_access_key_id=<ID>,
            aws_secret_access_key=<Key>,
            verify=False
        )
saptarshi
  • 151
  • 2
  • 7
3

I encounter this problem, too. My environment is Ubuntu 15.04, Python 2.7.9 and Boto 2.38.0.

Setting the argument validate_certs=False doesn't make it work with the HTTPS connection without valid certificate. After reading the code of boto, I found that it's a behavior of Python's ssl modules. Then I found a solution here: "SSL: CERTIFICATE_VERIFY_FAILED" Error. And the solution does work!!!.

Community
  • 1
  • 1
diabloneo
  • 2,607
  • 2
  • 18
  • 17
  • 1
    Thanks, yes, the monkey patch solution works with python3 and old boto: import ssl ssl._create_default_https_context = ssl._create_unverified_context – jamshid Oct 29 '21 at 03:57
  • 1
    Thanks to both of you. I just wanted to note that after [this](https://github.com/boto/boto/blob/b2a6f08122b2f1b89888d2848e730893595cd001/boto/connection.py#L74) line in boto package, I had to add `ssl._create_default_https_context = ssl._create_unverified_context `. – tash Feb 20 '22 at 00:20
1

macOS users: If you are using the Python 3.6 from the python.org binary installer linked on this page, please carefully read the Important Information displayed during installation; this information is also available after installation by clicking on /Applications/Python 3.6/ReadMe.rtf. There is important information there about changes in the 3.6.0 installer-supplied Python, particularly with regard to SSL certificate validation.

https://www.python.org/downloads/release/python-360/

From ReadMe.rtf at the time of this writing:

Certificate verification and OpenSSL

NEW This variant of Python 3.6 now includes its own private copy of OpenSSL 1.0.2. Unlike previous releases, the deprecated Apple-supplied OpenSSL libraries are no longer used. This also means that the trust certificates in system and user keychains managed by the Keychain Access application and the security command line utility are no longer used as defaults by the Python ssl module. For 3.6.0, a sample command script is included in /Applications/Python 3.6 to install a curated bundle of default root certificates from the third-party certifi package (https://pypi.python.org/pypi/certifi). If you choose to use certifi, you should consider subscribing to the project's email update service to be notified when the certificate bundle is updated.

The bundled pip included with the Python 3.6 installer has its own default certificate store for verifying download connections.

steamer25
  • 9,278
  • 1
  • 33
  • 38
0

Office laptops usually have network monitors installed. Figured out that it was the network monitoring software interfering with python, not letting it verify ssl certs of aws. We had to import its's cert(got from office) onto python's cacert.pem file, then it started working fine.

Vasudev
  • 803
  • 1
  • 7
  • 16