Skip to content

README.md

Latest commit

 

History

History
107 lines (74 loc) · 5.42 KB

README.md

File metadata and controls

107 lines (74 loc) · 5.42 KB
Raw

Penny Bot — Terraform

Terraform project for Penny bot's infrastructure.

Layout

Two independent configurations, each with its own state:

Path State Key How to Apply Changes Contents
infra/bootstrap/ penny-bot/bootstrap.tfstate Manually The state bucket and all IAM role and policies
infra/ penny-bot/terraform.tfstate CI (deploy-penny.yml) or Manually Everything Penny runs on, excluding IAM
  • All IAM lives in bootstrap/ so that the CI role cannot modify permissions of anyone including its own.

infra/modules/ holds the local modules:

  • constants: the single source of truth for resource names and the ARNs derived from them.
  • bucket: an S3 bucket with encryption, public-access-block, ownership controls, and optional versioning/lifecycle.
  • lambda: a Lambda plus its log group, API Gateway integration, invoke permission and routes.

Usage

Example usage; see Requirements section below. Ensure you're in repository's root directory.

# install tools via mise; requires mise installed
mise install
# login via aws CLI; requires aws CLI installed and configured with a `vapor` profile
aws login --profile vapor
# run lint if needed; requires terraform and tflint installed
./scripts/tf-check.sh

# work with terraform
mise x -- terraform -chdir=infra init
mise x -- terraform -chdir=infra plan
mise x -- terraform -chdir=infra apply

Changing CI permissions

  • All IAM stuff including the role CI assumes (penny-bot-deploy) are defined in infra/bootstrap/iam_deploy.tf.
  • Modifying the roles requires manually applying the changes. The CI intentionally doesn't have permissions to do this.
# 1. grant the permission (you, locally)
mise x -- terraform -chdir=infra/bootstrap apply
# 2. use it
mise x -- terraform -chdir=infra apply

CI can modify Penny's resources but deliberately has no ec2:CreateSecurityGroup/DeleteSecurityGroup and no IAM write access — structural changes of that kind are local applies.

Requirements

  • AWS CLI v2 (install) with a vapor profile resolving to account 177420307256 in eu-west-1.

    • Example ~/.aws/config (see configuring profiles):

      [profile vapor]
account_id = 177420307256
region = eu-west-1
login_session = arn:aws:iam::177420307256:user/mahdi

      Then sign in to the profile (e.g. aws login --profile vapor); see the AWS CLI authentication guide for the available sign-in options.

  • mise (install) — provides the pinned tools declared in .mise.toml. Run mise install, then activate mise in your shell so the pinned tools are on your PATH (or prefix individual commands with mise x --).

  • Terraform 1.15.8 (also enforced by required_version) and TFLint 0.64.0 — pinned in .mise.toml, installed via mise (above).

First run

On a fresh account the bootstrap has a chicken-and-egg problem: infra/bootstrap/backend.tf points at the very S3 bucket that bootstrap creates. So the first apply must use local state, then migrate:

# 1. create the state bucket using local state, ignoring the S3 backend
mise x -- terraform -chdir=infra/bootstrap init -backend=false
mise x -- terraform -chdir=infra/bootstrap apply

# 2. adopt the S3 backend and push the local state into the bucket just created
mise x -- terraform -chdir=infra/bootstrap init -migrate-state

# 3. everything else
mise x -- terraform -chdir=infra init
mise x -- terraform -chdir=infra apply

Also ECR and Lambda deployments might fail since there can be cyclic dependencies between e.g. an ECR image / Lambda executable zip file existing and the ECS/lambda resources being created.

Container image tag

var.penny_image_tag selects the ECS image tag. When unset, it defaults to the tag of the currently deployed task definition (read live via a data source), so local terraform plan stays zero-diff with no hardcoded value. CI passes the freshly built tag explicitly (-var penny_image_tag=<git-sha>). On a fresh account there is no live task definition to read, so you must supply it on the first apply: TF_VAR_penny_image_tag=<tag> terraform apply.

Not in code (needed to replicate elsewhere):

  • Secret values for all 7 prod/penny/penny-bot/* secrets.
  • penny-bot-deployer access-key secret (If needed, a recreation is required.).
  • Shared org resources used as data sources: default VPC, OIDC provider, GithubOIdP-Role (repo:vapor/*).
  • The PENNY_OIDC_ROLE_ARN repository variable, pointing at the penny-bot-deploy role both workflows assume.

Observability

Currently Penny uses some default AWS CloudWatch setup for observability.

Penny is knowingly weak on the observability side. Great observability is simply not needed at Penny's current scale of 1 Discord server / 2 GitHub orgs, so we're cutting on the maintenance and financial costs of having good observability.