When refactoring terraform code is there a sane way to update the respective terraform states?

Question

Here is a scenario - you develop terraform modules for use in your R&D organisation. They are in use by one or two micro services already, which translates into a dozen pods. You identify a refactoring opportunity, like pulling out some functionality into its own terraform module. Great, but now you have to update a dozen terraform states, because this is the price of terraform code refactoring.

After one such refactoring I found myself spending 8 hours to update all the terraform states. I do it in an ad hoc fashion - with powershell scripts wrapping the various terraform state commands. One can quickly loose one's sanity doing it too often.

Of course, we could say - do not refactor. But this is not possible, because terraform code is ... code. So, is there a better way to do it? Some kind of a tool that can help, somehow?

Answer 1

I recommend using terraform-state-mover.

It offers an interactive prompt for the terraform state mv command. There is a short demo on github that shows how to use it.

Answer 2

If you maintain a script that imports the complete state, then, at any time, you can discard your local state and do a fresh import.

When using this approach, you won't need to mv state around to support a Terraform code refactor.

---

This kind of bulk import is well suited to a simple scripting language.

Here, I'm using workspaces to store local state...

#! /usr/bin/env bash

set -e ; set -o pipefail

stack=$(terraform workspace list |grep '*' |cut -d' ' -f 2)

# workspace-specific values
if [[ "${stack}" == "dev" ]] ; then
    cdnid=ABCD12345
    origin_access_identity=ABCD23456
elif [[ "${stack}" == "stg" ]] ; then
    cdnid=ABCD34567
    origin_access_identity=ABCD45678
fi

echo "stack: ${stack}"
echo "cdnid: ${cdnid}"
echo "origin_access_identity: ${origin_access_identity}"

## discard local state (with backup)
( cd terraform.tfstate.d/${stack}/ && mv terraform.tfstate terraform.tfstate.$(date +%s).backup ) ||true

terraform import module.bucket.aws_s3_bucket_policy.bucket_policy stuff-${stack}
terraform import module.bucket.aws_s3_bucket_versioning.bucket_versioning stuff-${stack}
terraform import module.bucket.aws_s3_bucket_acl.bucket_acl stuff-${stack}
terraform import module.bucket.aws_s3_bucket_request_payment_configuration.bucket_request_payment_configuration stuff-${stack}
terraform import module.bucket.aws_s3_bucket.bucket stuff-${stack}
terraform import aws_cloudfront_origin_access_identity.origin_access ${origin_access_identity}
terraform import aws_cloudfront_distribution.cdn ${cdnid}

You could run it like this...

$ terraform workspace select dev && bash -x ./import.sh

Then you should get something like this...

$ terraform workspace select dev && terraform plan

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

---

I imagine that this would be a built-in feature of Terraform if it weren't for the complexity of identifying and associating resource IDs.