1

I have a Django model that has a FileField, the file has to be saved in AWS S3.
models.py:

class Search(models.Model):
user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE, related_name='searches')
type = models.CharField(max_length=150)
keyword = models.CharField(max_length=150)
amount = models.IntegerField()
date = models.DateTimeField(auto_now_add=True)
status = models.CharField(max_length=50)
results = models.FileField(blank=True, storage=PrivateMediaStorage())

The upload to AWS S3 works when I create a new object in the admin area and upload a file there. However, I want to create a CSV file in the backend and upload it to S3 from there, not using any form.
Inside the backend function:

    with open(f'results{search_id}.csv', 'w', newline='') as f:
    writer = csv.writer(f)

    header = ['name', 'email', 'follower_count', 'play count', 'share count', 'comment count', 'has email']
    writer.writerow(header)

    for result in results:
        data = [result['author_name'], result['author_email'], result['follower_count'], result['play_count'], result['share_count'], result['comment_count'], result['has_email']]
        writer.writerow(data)

    # Updating database data
    search = Search.objects.get(id=search_id)

    search.status = 'Finished'
    search.results = ContentFile(f)
    search.save()

However, right now this just creates a results.csv file in my Django directory and doesn't upload any file to S3. How can I fix this?

Stevenvdh
  • 21
  • 1

2 Answers2

0

In your current approach, I think you need to do something similar to what's suggested in this post.

Maybe try something like this?

    # replace this line
    # search.results = ContentFile(f)
    search.results.save(new_name, ContentFile(f.read()))
    search.save()
Cory
  • 22,772
  • 19
  • 94
  • 91
0

If you want to upload a file to s3 directly from the script you'd need to use boto3:

  1. Start a boto3 session
  2. Create an object for the s3 object
  3. Access the bucket in the s3 resource
  4. Upload the file

Obviously, that's greatly simplified but all of the documentation can be found here: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-uploading-files.html

Dharman
  • 30,962
  • 25
  • 85
  • 135
James
  • 61
  • 6
  • I have added the following code: import boto3 s3 = boto3.client('s3') s3.upload_fileobj(f, "", "private") However, I now get this error: Input <_io.TextIOWrapper name='results66.csv' mode='w' encoding='UTF-8'> of type: is not supported. is not the real code of course. – Stevenvdh Mar 14 '22 at 16:07
  • Sounds like it's trying to upload a string rather than the actual file, you haven't put an 'str' in there anywhere have you? Also, you'll need access keys for your s3 bucket. – James Mar 15 '22 at 11:30