Optimizing AWS infrastructure with Terraform: lessons from running 23+ production sites

AWS optimization with Terraform does not start with the discount. It starts with being able to answer two questions without opening the console: how much does each site cost, and how exposed is it.

I manage a fleet of 23+ sites in production. That work taught me something that applies to any cloud: when there are many sites, whatever is not written in code ends up different on every server.

An honest caveat: that fleet runs on AlmaLinux servers with WHM, not on AWS. What follows is the design I propose for bringing that same discipline to AWS with Terraform. The code is illustrative and includes no savings figures, because I'm not going to make them up.

What running 23+ production sites teaches you

Day-to-day work with a fleet like this is unglamorous: tuning Apache and PHP-FPM, recovering disk space, upgrading PHP versions and untangling DNS spread across several providers.

None of those tasks is hard once. The problem is doing them 23 times, at different moments, by different people. That is how drift is born: every server accumulates small decisions nobody documented.

On AWS the same phenomenon has two measurable consequences:

• Cost nobody can attribute. Unused disks, oversized instances and eternal backups show up on the bill with no site name attached.
• Uneven security. One site ends up encrypted and locked down; another, created in a hurry, does not.

Terraform does not fix this just by existing. It fixes it if you use it so that the right way is the only way to create a site.

One module per site, one map for the fleet

The design decision that pays off most is treating the fleet as data. Each site is an entry in a map; the module defines how any of them gets built.

That way, adding site 24 is a four-line change in a pull request, with review and history.

provider "aws" {
  region = var.region
  # The whole fleet inherits these tags: without them, cost can't be attributed.
  default_tags {
    tags = { ManagedBy = "terraform", Fleet = "websites" }
  }
}

variable "sites" {
  type = map(object({
    instance_type = string
    disk_gb       = number
    budget_usd    = number
  }))
}

# A new site is an entry in the map, not an afternoon in the console.
module "site" {
  source   = "./modules/site"
  for_each = var.sites

  name          = each.key
  instance_type = each.value.instance_type
  disk_gb       = each.value.disk_gb
  tags          = { Site = each.key }
}

Two details matter more than they seem to.

The first is for_each over a map rather than count over a list. With count, deleting site 3 shifts the indexes and Terraform wants to recreate everything from 4 onward. With for_each, every site has its own identity.

The second is default_tags. Tags stop depending on the memory of whoever writes the resource. Everything the provider creates comes out tagged.

Automating cost savings with Terraform

Sustained savings do not come from a heroic cleanup every quarter. They come from three automatic controls: visibility, early warning, and a brake before applying.

One budget per site

Because every resource carries the Site tag, you can declare one budget for each entry in the same map. The alert fires on forecasted spend, not on money already spent.

resource "aws_budgets_budget" "site" {
  for_each     = var.sites
  name         = "site-${each.key}"
  budget_type  = "COST"
  limit_amount = tostring(each.value.budget_usd)
  limit_unit   = "USD"
  time_unit    = "MONTHLY"

  # Filter by tag: each site answers for its own spend.
  cost_filter {
    name   = "TagKeyValue"
    values = [format("user:Site$%s", each.key)]
  }

  # Alert on the forecast: you find out before spending, not after.
  notification {
    comparison_operator       = "GREATER_THAN"
    threshold                 = 80
    threshold_type            = "PERCENTAGE"
    notification_type         = "FORECASTED"
    subscriber_sns_topic_arns = [aws_sns_topic.alerts.arn]
  }
}

One requirement that is often forgotten: the tag has to be activated as a cost allocation tag in the billing console. Without that step, the filter matches nothing.

A cost gate in the pipeline

The budget warns you when the spend is already on its way. The gate in CI acts earlier: it reads the plan as JSON and rejects whatever breaks the fleet's rules.

import json
import sys

# Generated with: terraform show -json tfplan > plan.json
plan = json.load(open("plan.json"))
ALLOWED = {"t3.micro", "t3.small", "t3.medium"}
errors = []

for rc in plan.get("resource_changes", []):
    after = rc["change"].get("after") or {}
    # Without a Site tag there is no way to attribute the cost to anyone.
    if "tags_all" in after and "Site" not in (after["tags_all"] or {}):
        errors.append(f"{rc['address']}: missing Site tag")
    # An off-catalog instance gets discussed in the PR, not on the bill.
    if rc["type"] == "aws_instance" and after.get("instance_type") not in ALLOWED:
        errors.append(f"{rc['address']}: instance type outside the catalog")

if errors:
    sys.exit("\n".join(errors))   # non-zero exit: the PR does not merge

It is twenty lines and runs in GitHub Actions after terraform plan. The instance catalog is an example; yours comes from measuring what each site actually consumes.

What cleans itself up

Disk space was a constant lesson with the fleet. On AWS, that problem is prevented with lifecycle rules declared next to the resource:

• S3 backups with an expiry date. A lifecycle rule moves old backups to a cheaper storage class and deletes them when their retention ends.
• Disks that die with their instance. The root volume is flagged for deletion on termination, and extra volumes live in the same module.
• Test environments on a schedule. If an environment is only used during office hours, its shutdown is declared in code too.

Security with Terraform: the baseline in every plan

With many sites, security cannot depend on a per-server checklist. It has to be a property of the module and of the account.

The rule is to declare once, as high up as possible. What you set at account level also protects the site someone creates in a hurry tomorrow.

# Account level: declared once, it protects the whole fleet.
resource "aws_ebs_encryption_by_default" "baseline" {
  enabled = true
}

resource "aws_s3_account_public_access_block" "baseline" {
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

# Inside the site module. Mandatory IMDSv2 cuts off credential theft
# through SSRF, a real risk on sites running third-party plugins.
resource "aws_instance" "web" {
  # ...
  metadata_options {
    http_endpoint = "enabled"
    http_tokens   = "required"
  }
}

The rest of the baseline follows the same logic:

• No SSH open to the internet. Admin access goes through Session Manager or a private network, and the security group exposes only the web ports.
• Least-privilege IAM roles. Each site reads its own bucket and its own secrets, nothing else.
• Secrets out of the code. Passwords live in Secrets Manager or Parameter Store; Terraform only keeps the reference.

What about "real-time" security? Terraform does not monitor, but it does detect. A scheduled plan compares the code against reality and alerts you when someone changed something by hand.

# ⚠️ Production: the state is the source of truth. Back it up before operating.
terraform state pull > "backup-$(date +%F).tfstate"

# Exit code 2 = reality no longer matches the code (drift).
terraform plan -detailed-exitcode -lock-timeout=60s -out=tfplan

# The JSON plan feeds the cost gate and stays as evidence.
terraform show -json tfplan > plan.json

Scheduled nightly in GitHub Actions, that plan turns a manual change into an alert the next morning, instead of a surprise at the next audit.

Best practices for infrastructure management

These are the practices that separate a Terraform repository that scales from one people are afraid to touch:

• Remote, encrypted state with locking. An S3 bucket with versioning and state locking. Never a local file on someone's laptop.
• States split by layer. The account and network baseline in one state; the sites in others. A mistake in one site must not be able to touch everyone's network.
• Plan in the pull request, apply only from CI. The plan is reviewed the way code is reviewed. Nobody applies from their own machine.
• prevent_destroy on anything that holds data. Databases and backup buckets do not get destroyed because of a rename.
• Pinned versions. The provider and modules carry an exact version. Upgrades come in through a pull request, not by accident.
• Import before you recreate. What already exists is brought into the state with an import block. Recreating a production server to "leave it clean" is an unnecessary risk.

Migrating an existing fleet is done in stages. First the account baseline, then a low-risk site, then the rest in groups. Every step with its backup and its reviewed plan.

What NOT to do

• A single state for everything. Slow plans, locks between teams and a blast radius that covers the whole fleet. It is the most common mistake and the most expensive to fix later.
• Mixing console and code. A "quick fix" by hand that never makes it to the repository. The next apply reverts it or, worse, nobody dares to apply.
• Secrets in .tfvars files inside the repository. The state and the Git history keep them forever. Use a secrets manager and pipeline variables.
• Copy and paste instead of modules. Twenty-three almost identical folders are not infrastructure as code. They are twenty-three chances for one to end up different.

The next step

Optimizing AWS with Terraform is, above all, an exercise in order: one module, one map, tags that cannot be skipped, and controls that act before the cost or the risk exists.

If your team is facing infrastructure bottlenecks, complex migrations, or needs to audit the cost and security of its cloud, at Conqr Digital Agency we design architectures that scale. Let's review your case.

If you want to see how I handle cloud and deployment work, take a look at the Cloud and DevOps service and the Linux servers and email service.

FAQ

Should I use a single Terraform state for every site?

No. A single state makes every plan slow and lets one mistake reach the whole fleet. Split by layer: an account and network baseline, and states per group of sites or per environment.

How do I control AWS costs with Terraform?

With three pieces: default tags on the provider, one AWS Budgets budget per site filtered by tag, and a pipeline gate that reads the plan as JSON and rejects untagged resources or anything outside the allowed catalog.

Does Terraform detect changes made by hand in the console?

Yes. A scheduled plan with the -detailed-exitcode flag returns exit code 2 when real infrastructure no longer matches the code. That lets you alert on drift without applying anything.