0

I'm run the below task using Prefect flow task to extract a parquet file from s3bucket and save it locally but I keep getting PermissionError: [WinError 5] Access is denied: error.

@task(log_prints=True, cache_key_fn=task_input_hash, cache_expiration=timedelta(days=1))
def extract_from_s3(color: str, year: int, month: int) -> Path:
    """Download trip data from S3"""
    s3_path = f'data/{color}/{color}_tripdata_{year}-{month:02}.parquet'
    s3_block = S3Bucket.load("prefect-de-zoom-s3")
    local_path = os.path.join("data", color, f"{color}_tripdata_{year}-{month:02}.parquet")
    s3_block.download_object_to_path(from_path=s3_path, to_path=local_path)
    return f'/data/{s3_path}'

I have run my cmd as administrator which is one of the solution I found online but the error still persist.

here is the complete error

 os.remove(new_filename)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\user\\desktop\\flow\\data\\yellow\\yellow_tripdata_2021-01.parquet'
Bello Nasiff
  • 1
  • 1

1 Answers1

0

The double \ looks problematic.

Pathlib' module can be helpful. See this SO answer and other ones on that page.

The .as_posix() method will "Return a string representation of the path with forward slashes (/):"

That might be helpful.

jeffhale
  • 3,759
  • 7
  • 40
  • 56