Remote State & Bootstrapping the AWS Landing Zone

By default, Terraform stores the mapping between your local code files and real-world AWS infrastructure in a local file called terraform.tfstate.

In a team setting or production environment, storing state locally is a recipe for disaster:

Concurrency Conflicts: If two engineers run terraform apply at the same time, they will overwrite each other's changes, corrupting the state.
Plaintext Secrets: Terraform state files store resource parameters in plaintext, including passwords, private keys, and connection strings. Storing these in Git is a major security breach.
Loss of State: If an engineer's laptop crashes, your entire structural mapping is lost.

To solve this, we must configure a Remote Backend using AWS S3 (for durable storage) and DynamoDB (for state locking).

sequenceDiagram
    participant Eng as Engineer/CI
    participant S3 as AWS S3 Bucket (Encrypted)
    participant DB as AWS DynamoDB (Lock Table)
    
    Eng->>DB: 1. Request State Lock (LockID)
    alt Lock Acquired
        DB-->>Eng: Lock granted
        Eng->>S3: 2. Fetch latest state file
        Eng->>Eng: 3. Plan & Apply infrastructure changes
        Eng->>S3: 4. Upload updated state file
        Eng->>DB: 5. Release State Lock
    else Lock Held by Peer
        DB-->>Eng: Lock Denied (Error!)
        Eng->>Eng: Execution Aborted
    end

Bootstrapping the State Infrastructure (The Chicken-and-Egg Problem)

To store our state in S3 and DynamoDB, we need to create the S3 bucket and DynamoDB table with Terraform. But how do we write the backend configuration if the S3 bucket doesn't exist yet?

We solve this using a Two-Stage Bootstrapping Process:

Write the code using a local state backend.
Run terraform apply to provision the S3 bucket and DynamoDB lock table.
Add the backend "s3" configuration block.
Run terraform init to migrate the local state into S3.

Step 1: Write the Bootstrap Code

Create a directory named bootstrap/ and add the following main.tf:

# bootstrap/main.tf

terraform {
  required_version = ">= 1.7.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

# Fetch AWS Account ID dynamically to avoid hardcoding names
data "aws_caller_identity" "current" {}

# 1. Create KMS Key for State Encryption
resource "aws_kms_key" "state_key" {
  description             = "KMS Key for Terraform State S3 Bucket"
  deletion_window_in_days = 10
  enable_key_rotation     = true
}

# 2. Create the S3 Bucket for State Storage
resource "aws_s3_bucket" "state_bucket" {
  bucket        = "codesprintpro-tfstate-${data.aws_caller_identity.current.account_id}"
  force_destroy = false

  lifecycle {
    prevent_destroy = true # Safeguard against accidental destruction
  }
}

resource "aws_s3_bucket_versioning" "state_versioning" {
  bucket = aws_s3_bucket.state_bucket.id
  versioning_configuration {
    status = "Enabled" # Enables history rollbacks of state files
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "state_encryption" {
  bucket = aws_s3_bucket.state_bucket.id

  rule {
    apply_server_side_encryption_by_default {
      kms_master_key_id = aws_kms_key.state_key.arn
      sse_algorithm     = "aws:kms"
    }
  }
}

# Block all public access to state bucket
resource "aws_s3_bucket_public_access_block" "state_public_block" {
  bucket = aws_s3_bucket.state_bucket.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

# 3. Create the DynamoDB Table for Distributed State Locking
resource "aws_dynamodb_table" "state_locks" {
  name         = "terraform-state-locks"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID" # Must match exactly

  attribute {
    name = "LockID"
    type = "S"
  }
}

output "state_bucket_name" {
  value = aws_s3_bucket.state_bucket.id
}

output "dynamodb_table_name" {
  value = aws_dynamodb_table.state_locks.id
}

Run the bootstrap apply:

terraform init
terraform apply -auto-approve

Take note of the S3 bucket name output (e.g. codesprintpro-tfstate-123456789012).

Step 2: Configure and Migrate to the Remote Backend

Now that the backend resources exist in AWS, we can configure our root modules to point to the remote S3 backend. Add the backend configuration to a file named backend.tf:

# backend.tf

terraform {
  backend "s3" {
    bucket         = "codesprintpro-tfstate-123456789012" # Output from Step 1
    key            = "dev/terraform.tfstate"              # Path inside S3 bucket
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-state-locks"
  }
}

Now, initialize the workspace. Terraform will detect the local state file and prompt you to migrate it to S3:

terraform init

Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the 
  newly configured "s3" backend. No existing state was found in the "s3" backend.
  Do you want to copy this state to the new "s3" backend? Enter "yes" to copy and "no"
  to start with an empty state.

  Enter a value: yes

Type yes and hit Enter. Your state file is now fully stored in S3, encrypted, version-controlled, and safeguarded by DynamoDB locks!

Verification

Check your local workspace. The local terraform.tfstate is now inactive and can be safely deleted (or added to .gitignore). To verify that S3 versioning is active, look at your bucket versions inside the AWS S3 Console or run:

aws s3api list-object-versions --bucket codesprintpro-tfstate-123456789012

In the next module, we will explore Module 2: Production AWS Networking & Security, where we will provision our custom VPC, private subnets, and NAT Gateways.

Remote State & Bootstrapping the AWS Landing Zone

Provision, secure, and automate production-grade cloud infrastructure at scale.