1

my goal is to create a MediaConvert job from a given template by using boto3 with python: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/mediaconvert.html#MediaConvert.Client.create_job

Apparently MediaConvert fails to identify the region of my output s3 bucket. I was under the impression that buckets were global, but even after some tinkering I wasn't able to fix the problem.

Here's the error message from the MediaConvert dashboard:

Could not identify region for bucket s3.Bucket(name='mybucket'): Failed to lookup region of buckets3.Bucket(name='mybucket')

The error code is 1404.

When I click on the Output Group on the dashboard for the job that failed, I get redirected to "https://console.aws.amazon.com/s3/buckets/s3.Bucket(name='mybucket')/?region=us-east-1", which obviously fails to resolve a bucket. The correct path would have been "https://console.aws.amazon.com/s3/buckets/mybucket/?region=us-east-1".

Here is the code that triggers the job:

media_client = boto3.client('mediaconvert', region_name='us-east-1')
endpoints = media_client.describe_endpoints()
customer_media_client = boto3.client('mediaconvert', region_name='us-east-1', endpoint_url=endpoints['Endpoints'][0]['Url'])
customer_media_client.create_job(
                JobTemplate='job-template',
                Role='arn:aws:iam::1234567890:role/MediaConvert',
                Settings=...

In the Settings I use the following OutputGroupSettings:

                        "OutputGroupSettings": {
                            "Type": "FILE_GROUP_SETTINGS",
                            "FileGroupSettings": {
                                "Destination": "s3://%s/" % target_bucket
                            }
                        }

I did verify that the MediaConvert jobs and the S3 buckets are all in the same region (us-east-1).

Any idea what the error is about? If you need more code, please let me know.

I have also asked this question on the aws forums: https://forums.aws.amazon.com/thread.jspa?threadID=304143

michaelbahr
  • 4,837
  • 2
  • 39
  • 75

2 Answers2

2

It looks like an known issue associated with the % string formatting operator used within a Python dictionary.

In commit Issue #14123: Explicitly mention that old style % string formatting has caveats but is not going away any time soon.

The use of a binary operator means that care may be needed in order to format tuples and dictionaries correctly.

Check this answer for more detail.

This explains why by taking the assignment outside of a dict resolves the issue. Consider using the .format() method to replace %.

My Error Message

I got the same error message from MediaConvert, but my issue was missing a "/" after the bucket name in "Destination".

Originally my code was:

FileGroupSettings['Destination'] = 's3://' + bucketName + S3key

By adding a slash, it could locate the right bucket

FileGroupSettings['Destination'] = 's3://' + bucketName + '/' + S3key
curtispy
  • 156
  • 8
0

I resolved the issue by extracting "s3://%s/" % target_bucket into a separate declaration.

s3_target = "s3://%s/" % target_bucket

...

"Destination": s3_target
michaelbahr
  • 4,837
  • 2
  • 39
  • 75