1

I am trying to use the list_objects_v2 function of the Python3 Boto3 S3 API client to list objects from an S3 access point.

Sample Code:

import boto3
import botocore

access_point_arn = "arn:aws:s3:region:account-id:accesspoint/resource"
client = boto3.client('s3')
response = client.list_objects_v2(Bucket=access_point_arn)

Somehow getting the error below:

botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid bucket name "arn:aws:s3:region:account-id:accesspoint/resource": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$"

Based on the documentation: https://docs.aws.amazon.com/AmazonS3/latest/dev/using-access-points.html, i should be able to pass an access point to the list_objects_v2 function as the Bucket name. The odd thing is, this function works locally on my Windows 10 laptop. The same Python3.6 code with the same Boto3 and Botocore package versions throws this error in AWS Glue Python Shell job. I also made sure the Glue role has S3 Full Access and Glue Service policies attached.

I would appreciate if someone can shed some lights on this.

Yun Ling
  • 113
  • 1
  • 8
  • Try without specifying access point in your code when running it on Glue. – Asdfg Aug 28 '20 at 22:29
  • 1
    Access Points are relatively new. I wonder whether Glue is using an older version of boto that doesn't support them? – John Rotenstein Aug 29 '20 at 00:50
  • @JohnRotenstein I already imported boto3 and botocore wheel files as Glue Python class files so make sure they are consistent with my local versions. – Yun Ling Aug 29 '20 at 01:18
  • @Asdfg Not quite following. Do you mean to not specify the Bucket kwarg for the `list_objects_v2` function? – Yun Ling Aug 29 '20 at 01:20
  • Just specify the bucket name without access point. Like Bucket=“MyAwesomeBucket” – Asdfg Aug 29 '20 at 01:23
  • @JohnRotenstein I think you are right. I did a boto version print `boto3.__version__` , the current boto3 version for Glue Python shell job is 1.9.203 which gives me this same error if i try this version on my local machine. The wheel file for boto3 version 1.13.21 i specified did not overwrite the native boto3 version, there might a cache thing going on for that package on the Glue worker. – Yun Ling Aug 29 '20 at 01:34
  • @JohnRotenstein Added another thread regarding the boto3 version: https://stackoverflow.com/questions/63642744/update-aws-glue-python-shell-job-boto3-version – Yun Ling Aug 29 '20 at 02:03

0 Answers0