Running AWS CLI through Python returns a "sh: 1: aws: not found" error

Question

I am trying to copy a file into an S3 bucket, using Python, like so:

cmd = 'aws s3 %s %s' % (filename, bucketname)
os.system(cmd)

It gives me a sh: 1: aws: not found error.

However, using s3cmd works just fine.

Why would s3cmd work, but not aws?

Also, I did which aws and it returned: /home/username/anaconda/bin/aws.

which s3cmd returns: /home/username/anaconda/bin/s3cmd.

Why does one work, but not the other, despite having the same root?

@jonnybazookatone: I'll try that. But why would one work, and not the other? My bashrc has the right path too.. — CodingInCircles, Jul 19 '15 at 01:17
I'm not sure, it shouldn't matter. But it'll be interesting if that doesn't work, as it'll suggest some other underlying problem that isn't path related. — jonnybazookatone, Jul 19 '15 at 20:03
@jonnybazookatone: Will do. I'll update this question when I try it out and see what happens. Thanks! — CodingInCircles, Jul 19 '15 at 20:08
@jonnybazookatone: It does work, but I'm still confused as to why it works! — CodingInCircles, Jul 20 '15 at 17:24
@jonnybazookatone: Can you write this as an answer so that I can mark it? Thanks! — CodingInCircles, Aug 19 '15 at 23:23

score 3 · Answer 1 · edited Apr 07 '21 at 19:36

This is because the AWSCLI is not installed in your system. I had also the same problem and I tried installing it with the pip package manager using pip install --user awscli but it didn't work. So I installed it directly in the system as follows:

curl "https://d1vvhvl2y92vvt.cloudfront.net/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

And the error was solved.

score 2 · Accepted Answer · edited May 23 '17 at 11:52

A quick way to troubleshoot the issue is to try the full path on the OS call to see if it is a PATH problem:

cmd = '/path/to/aws s3 %s %s' % (filename, bucketname)
os.system(cmd)

There could be a few reasons why this is a problem, most likely related to the PATH variable (at first guess). However, it might be better to steer away from os.system as noted in the docs (https://docs.python.org/2/library/os.html#os.system) and use some alternative methods.

Using subprocess:

cmd = ['/path/to/aws', 's3', filename, bucketname]
subprocess.Popen(cmd)

Or just use the python AWS client boto3 package. There are many ways, but one quick example from this SO question (How to save S3 object to a file using boto3):

import boto3
s3_client = boto3.client('s3')
s3_client.upload_file(filename, bucketname, filename)

That one is not testable with moto, which can be annoying. Instead if you want to test, you can do something like this:

import boto3
s3_resource = boto3.resource('s3')

with open(filename, 'rb') as f:
    binary = f.read()

s3_resource.Bucket(bucketname).put_object(
    Key=filename,
    Body=binary
)

score 0 · Answer 3 · answered Jun 25 '20 at 03:59

0

Install directly from the zip: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html#cliv2-linux-install

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

answered Jun 25 '20 at 03:59

imriss

1,815
4
31
46

Running AWS CLI through Python returns a "sh: 1: aws: not found" error

3 Answers3