aws transcoder overwrite files on s3

Question

I'm using the AWS PHP SDK to upload a file to S3 then trancode it with Elastic Transcoder.

First pass everything works fine, the putobject command overwrites the old file (always named the same) on s3:

$s3->putObject([
      'Bucket'     => Config::get('app.aws.S3.bucket'),
      'Key'        => $key,
      'SourceFile' => $path,          
      'Metadata'   => [
        'title'     => Input::get('title')
      ]
    ]);

However when creating a second transcoding job, i get the error:

  The specified object could not be saved in the specified bucket because an object by that name already exists

the transcoder role has full s3 access. Is there a way around this or will i have to delete the files using the sdk everytime before its transcoded?

my create job:

    $result = $transcoder->createJob([
      'PipelineId' => Config::get('app.aws.ElasticTranscoder.PipelineId'),
      'Input' => [
        'Key' => $key
      ],
      'Output' => [
        'Key' => 'videos/'.$user.'/'.$output_key,
        'ThumbnailPattern' => 'videos/'.$user.'/thumb-{count}',
        'Rotate' => '0',
        'PresetId' => Config::get('app.aws.ElasticTranscoder.PresetId')       
      ],
    ]);

score 4 · Answer 1 · answered Apr 01 '14 at 16:50

4

The Amazon Elastic Transcoder service documents that this is the expected behavior here: http://docs.aws.amazon.com/elastictranscoder/latest/developerguide/job-settings.html#job-settings-output-key.

If your workflow requires you to overwrite the same key, then it sounds like you should have the job output somewhere unique and then issue an S3 CopyObject operation to overwrite the older file.

answered Apr 01 '14 at 16:50

Jeremy Lindblom

6,437
27
30

If the bucket is versioned then a file with the same key can be overwritten. This doesn't appear to be documented, but it works. – philwilks Feb 24 '22 at 10:47

score 0 · Answer 2 · answered Feb 24 '22 at 10:47

0

If you enable versioning on the S3 bucket, then Amazon Elastic Transcoder will be happy overwriting the same key with the transcoded version.

answered Feb 24 '22 at 10:47

philwilks

669
5
16

score -1 · Answer 3 · edited Oct 05 '15 at 09:35

I can think of two ways to implement it:

Create two buckets, one for temp file storage (where its uploaded) and another where transcoded file is placed. Post transcoding when new file is created, you can delete temp file.
Use single bucket and upload file with some suffix/prefix. Create transcoded file in same bucket removing prefex/suffix (which you used for temp name).

In both cases for automated deletion of uploaded files you can use Lambda function with S3 notifications.

aws transcoder overwrite files on s3

3 Answers3

Linked