5

Amongst other things I am drawing plots using matplotlib, which I would like to immediately store as S3 objects.

According to the answers provided in this and this other question and the fine manual, I need S3.Object.put() to move my data into AWS and the procedure should be along the lines of

from matplotlib import pyplot as plt
import numpy as np
import boto3
import io

# plot something
fig, ax = plt.subplots()
x = np.linspace(0, 3*np.pi, 500)
a = ax.plot(x, np.sin(x**2))

# get image data, cf. https://stackoverflow.com/a/45099838/1129682
buf = io.BytesIO()
fig.savefig(buf, format="png")
buf.seek(0)
image = buf.read()

# put the image into S3
s3 = boto3.resource('s3', aws_access_key_id=awskey, aws_secret_access_key=awssecret)
s3.Object(mybucket, mykey).put(ACL='public-read', Body=image)

However, I end up with a new S3 object with content-length zero.

The following gives me a new S3 object with content-length 6.

s3.Object(mybucket, mykey).put(ACL='public-read', Body="foobar")

When I put the next line, I end up with content in the S3 object, but its not a usable image:

s3.Object(mybucket, mykey).put(ACL='public-read', Body=str(image))

I can make it work by going through an actual file, like this:

with open("/tmp/iamstupid","wb") as fh:
    fig.savefile(fh, format="png")
s3.Bucket(mybucket).upload_file("/tmp/iamstupid", mykey)

So it seems to work. I am just unable to use the interface correctly. What am I doing wrong? How can I achieve my goal using S3.Object.put()

user1129682
  • 989
  • 8
  • 27

1 Answers1

3

I was able to resolve it. I found the answer in this question. Its a Python3 thing. From what I understand Python3 "usually" works with unicode. If you want single bytes you have to be explicit about it. The correct usage therefore is

s3.Object(mybucket, mykey).put(ACL='public-read', Body=bytes(image))

I find this a bit strange, since buf.read() is already supposed to return an object of type bytes but I did stop wondering, b/c it works now.

user1129682
  • 989
  • 8
  • 27