Terraform error refreshing state access denied

Question

I'm using gitbucket for both my repository and for pipelines. I have a terraform config file with a remote state configured which runs fine on my local machine however it fails when running in gitbucket. I keep getting access denied error. Here's the main.tf:

terraform {
backend "s3" {
    bucket = "zego-terraform-test"
    key    = "test/terraform.tfstate"
    region = "eu-west-1"
  }
}

data "terraform_remote_state" "remote_state" {
  backend = "s3"

  config {
    bucket = "zego-terraform-test"
    key    = "test/terraform.tfstate"
    region = "eu-west-1"
  }
}

variable "region" {}

provider "aws" {
  region     = "${var.region}"
  access_key = {}
  secret_key = {}
  token      = {}
}

module "vpc" {
  source = "./modules/vpc"
}

Here's my gitbucket-pipelines.yml:

image: python:3.5.1
pipelines:
  default:
    - step:
        caches:
          - pip
        script: # Modify the commands below to build your repository.
          - apt-get update
          - apt-get install unzip
          - wget https://releases.hashicorp.com/terraform/0.11.7/terraform_0.11.7_linux_amd64.zip
          - unzip terraform_0.11.7_linux_amd64.zip
          - rm terraform_0.11.7_linux_amd64.zip
          - export PATH="$PATH:${BITBUCKET_CLONE_DIR}"
          - terraform init
            -backend-config "access_key=$AWS_ACCESS_KEY"
            -backend-config "secret_key=$AWS_SECRET_KEY"
            -backend-config "token=$TOKEN"

When I run the .tf file in this pipeline I get this error:

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Error refreshing state: AccessDenied: Access Denied
    status code: 403

When I remove remote state config it runs fine. Why am I getting the access denied error even though I'm using the same creds on my local machine and in gitbucket environment?

Are you sure your credentials are being read in properly? What happens if you have your pipeline run the `aws sts get-caller-identity` command from the AWS CLI? — ydaetskcoR, Apr 14 '18 at 12:43
For future readers, In my case, the region of the bucket was not correct. — Youcef LAIDANI, Feb 12 '23 at 15:42

score 22 · Answer 1 · answered Aug 21 '19 at 21:41

22

Was getting the same error. For our use case, we have to manually remove the terraform.tfstate file under .terraform/ directory and run init again.

answered Aug 21 '19 at 21:41

LeOn - Han Li

9,388
1
65
59

I'm using windows. I accidentally deleted contents inside terraform.tfstate file. After which when i attempt to delete terraform.tfstate file. Its automatically being recreated. i deleted the resource manually in cloud. Now i can't to get rid of this .tfstate file in windows local. – CdVr Oct 18 '20 at 17:56

score 5 · Answer 2 · answered Feb 12 '22 at 23:48

In case a solution has not been found for this issue, you can use either "profile=" or "role_arn=" in the config section of your terraform_remote_state stanza. The same is true for the AWS Provider and the backend configuration.

I chased this issue all day today not realizing that role_arn was available for terraform_backend_state data source.

score 3 · Answer 3 · answered Aug 15 '22 at 09:57

This error might occur when switching between terraform backends. To solve it you can run terraform init -reconfigure to configure the backend to the new one.
It will achieve the same result as removing the terraform.tfstate file under .terraform and run terraform init.

score 3 · Answer 4 · answered Aug 15 '22 at 10:53

Same error happened to me when I was using several aws accounts using profile names.
In my case, I was missing profile property in the backend configuration. Added profile and removed .terraform directory, and ran terraform init - it worked.

terraform {
  required_version = ">= 1.0.0"
  backend "s3" {
    // here!
    profile = "crazyoptimist"
    bucket = "devops-terraform-crazyoptimist"
    key = "terraform.tfstate"
    region = "us-west-2"
    encrypt = true
  }
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

provider "aws" {
  region = "us-west-2"
  profile = "crazyoptimist"
}

score 1 · Answer 5 · answered Apr 13 '18 at 19:03

1

At first glance it seems reasonable. Have you tried having the terraform init and -backend-config's all on one line? I wonder if the - at the beginning is messing with the yml format?

answered Apr 13 '18 at 19:03

Joachim

2,761
1
15
7

score 1 · Answer 6 · answered Jan 06 '23 at 07:25

1

received same error while running terraform init command in codebuild, simply gave s3 bucket access to role created for codebuild and error was resolved

answered Jan 06 '23 at 07:25

omkar shingare

11
1

2

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jan 09 '23 at 13:25

score 0 · Answer 7 · answered Mar 14 '22 at 03:54

In my case the backend file of one of the data blocks of data.tf had permission issues, I just recreated that file and did terraform plan again, the problem sorted. Took ages to figure this out.

data "terraform_remote_state" "gateway" {
  backend = "s3"

  config = {
    bucket = "xxx-terraform-remote"
    key    = "xxx/terraform.tfstate"
    region = "eu-west-1"
  }
}

score 0 · Answer 8 · answered May 04 '22 at 10:45

In my case, there was an issue with the order in which AWS client looks for credentials.

I stored AWS credentials used by terreform in ~/.aws/credentials, but I've also had different AWS credentials set in environment varaibles.

I had to remove AWS credentials from my env variables and it worked.

Alok Aryan · Answer 9 · 2022-09-14T15:33:03.357

0

Terraform will need the following AWS IAM permissions on the target backend bucket:

s3:ListBucket on arn:aws:s3:::mybucket

s3:GetObject on arn:aws:s3:::mybucket/path/to/my/key

s3:PutObject on arn:aws:s3:::mybucket/path/to/my/key

s3:DeleteObject on arn:aws:s3:::mybucket/path/to/my/key

enter image description here

edited Sep 14 '22 at 15:33

answered Sep 14 '22 at 15:31

Alok Aryan

1
1

It would be easier to understand the answer if you could expand on what you mean by path/to/my/key. Instead of inserting a screenshot of a documentation, please add a link to the documentation directly as it might contain other context that could help. Also if you use monospace for actions/ARNs the answer is easier to read, for example s3:ListBucket or arn:aws:s3:::mybucket. You can change a text to monospace by wrapping it in ` (backtick) characters. – akos Sep 21 '22 at 19:41

Terraform error refreshing state access denied

9 Answers9