A startup's global network on AWS
Transit Gateway planning and the perimeter

Three AWS accounts, five regions, environments on two continents, and a workforce on three. A startup reaches this point faster than anyone plans for — and the network either gets designed once, or gets grown ad-hoc and then excavated forever. These are the field notes from designing it once: the CIDR plan, one Transit Gateway per region peered in a star, route tables as security zones, and a perimeter that consists of exactly three VPN doors.

The problem

The naive answers all stop scaling almost immediately. VPC peering is pairwise: at six VPCs across accounts and regions you're managing O(n²) peerings and their route tables by hand. A VPN per environment gives every engineer four VPN profiles and every office four tunnels. Public endpoints with IP allowlists aren't a network, they're a breach with extra steps. And under all three: if the VPCs were created with overlapping CIDRs — every startup's original sin — nothing can ever be connected to anything without NAT contortions.

Planning: the CIDR map is the architecture

Everything below was only possible because address space was planned globally before the second VPC existed. One supernet mind-map, one /16 per environment, regional blocks for VPN client pools, and no overlaps anywhere on the planet:

Three deliberate moves in that table. Environments live in different accounts — the account boundary is the blast radius, IAM is the first firewall. The entire company fits under one 10.0.0.0/8 supernet with a /16 per network — every route, security group and VPN split-tunnel rule can reason about "the company" as one summarizable block, and a glance at the second octet tells you which environment a packet belongs to. And 10.5.0.0/16 is reserved before it's needed — the vendor lane: any third party that connects by VPC peering gets space from there, so the first vendor with an opinionated CIDR can't collide with the plan.

The solution: a Transit Gateway star

One Transit Gateway per region — and every one of them is deployed in the devops account and shared to the host accounts via AWS RAM. The workload accounts never own network fabric: they receive the share, accept it, and attach their VPC's private subnets. That split is the governance model in one sentence — routing policy lives in an account application teams have no credentials for, while the teams keep full ownership of what runs inside their VPCs. Regions connect by TGW peering — but not as a mesh: the peering matrix is a star with Singapore as the hub, because that's where operations, CI and the primary VPN live. Every remote region peers with the hub and nothing else; traffic between two remote regions has no route, because no such traffic should exist.

# one TGW per region; note both defaults DISABLED — no accidental any-to-any
resource "aws_ec2_transit_gateway" "hub" {          # ap-southeast-1
  amazon_side_asn                 = 64512           # unique ASN per region
  default_route_table_association = "disable"
  default_route_table_propagation = "disable"
  auto_accept_shared_attachments  = "enable"        # RAM-shared spokes attach freely
  vpn_ecmp_support                = "enable"
}

#         region   | hub(SG) | us-east-1 | us-east-2 | eu-central-1 | ap-south-1
#  hub (SG)        |         |     +     |     +     |      +       |     +
#  us-east-1       |    +    |           |           |              |
#  us-east-2       |    +    |           |           |              |
#  eu-central-1    |    +    |           |           |              |
#  ap-south-1      |    +    |           |           |              |
#  — a star, not a mesh: remote↔remote has no route on purpose

resource "aws_ec2_transit_gateway_peering_attachment" "hub_us_east_1" {
  transit_gateway_id      = aws_ec2_transit_gateway.hub.id
  peer_transit_gateway_id = aws_ec2_transit_gateway.us_east_1.id
  peer_region             = "us-east-1"
}
resource "aws_ec2_transit_gateway_peering_attachment_accepter" "us_east_1" {
  transit_gateway_attachment_id = aws_ec2_transit_gateway_peering_attachment.hub_us_east_1.id
  provider                      = aws.us-east-1-devops   # accept on the far side
}

# the network (devops) account owns the TGW and shares it…
resource "aws_ram_resource_share" "tgw_hub" {
  name                      = "tgw-ap-southeast-1"
  allow_external_principals = true
}
resource "aws_ram_resource_association" "tgw" {
  resource_arn       = aws_ec2_transit_gateway.hub.arn
  resource_share_arn = aws_ram_resource_share.tgw_hub.id
}
resource "aws_ram_principal_association" "dev_account" {
  principal          = var.dev_account_id
  resource_share_arn = aws_ram_resource_share.tgw_hub.arn
}

# …the workload account accepts and attaches its private subnets
resource "aws_ec2_transit_gateway_vpc_attachment" "dev_vpc" {
  transit_gateway_id = aws_ec2_transit_gateway.hub.id
  vpc_id             = var.dev_vpc_id
  subnet_ids         = var.dev_private_subnet_ids   # one per AZ
  provider           = aws.dev-account
}

# and the VPC side needs its half of every conversation — return routes:
resource "aws_route" "dev_to_vpn" {
  for_each               = toset(var.dev_private_route_table_ids)
  route_table_id         = each.key
  destination_cidr_block = var.vpn_vpc_cidr          # 10.4.0.0/16
  transit_gateway_id     = aws_ec2_transit_gateway.hub.id
  provider               = aws.dev-account
}

Route tables as security zones

With both TGW defaults disabled, an attachment can reach exactly what the route table it is associated with can reach — nothing else. That turns TGW route tables into zones: which table your attachment consults is the perimeter policy, enforced in routing, below any firewall. One naming rule makes the whole scheme legible: name each route table after the audience that consults it, never after a region or a VPC. A table called "ap-southeast-1 RT" tells an on-call engineer nothing; rt-zone-vpn-clients tells them who is asking and what they're allowed to want.

The same policy as a picture — one row per zone: the attachments associated to it on the left, what its route table knows on the right. The bottom-right box is the entire security argument of this design.

# the dev attachment CONSULTS the dev zone…
resource "aws_ec2_transit_gateway_route_table_association" "dev" {
  transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.zone_dev.id
  transit_gateway_attachment_id  = aws_ec2_transit_gateway_vpc_attachment.dev_vpc.id
}

# …which learns only what dev is allowed to know:
resource "aws_ec2_transit_gateway_route_table_propagation" "dev_sees_vpn" {
  transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.zone_dev.id
  transit_gateway_attachment_id  = aws_ec2_transit_gateway_vpc_attachment.vpn_vpc.id
}
resource "aws_ec2_transit_gateway_route_table_propagation" "dev_sees_devops" {
  transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.zone_dev.id
  transit_gateway_attachment_id  = aws_ec2_transit_gateway_vpc_attachment.devops_vpc.id
}
# note what is ABSENT: no propagation, no static route toward prod.
# dev -> prod is not firewalled; it is unroutable.

The whole schema

Securing the perimeter

The perimeter is small enough to enumerate. Product traffic enters through load balancers in the product VPCs — that's the public, WAF-fronted surface. Human and tooling access has exactly three doors: OpenVPN entry points in Singapore, Frankfurt and Mumbai — one per workforce continent, so nobody hairpins through another hemisphere to reach dev. Each entry point is a small VPN VPC whose client pool has its own CIDR block, which means VPN users are addressable network citizens: security groups and TGW zones apply to them like to any subnet. Everything else — SSH, databases, dashboards, the CI — has no public existence at all; even AWS API calls from the devops VPC ride private VPC endpoints rather than the internet.

What the zoning buys, concretely: a developer connects to the nearest VPN and reaches dev and tooling; an operator's route table additionally knows prod and stage; a dev workload that gets popped can reach the VPN VPC and CI caches — and has no route to production. Not a denied route: an absent one. The deny happens in a routing table nobody can reach from inside a workload VPC, in a devops account the workload has no credentials for.

Caveats

• The star means remote↔remote silence — honor it. The day someone wants Frankfurt VPN users to reach stage directly, the tempting fix is one more peering. That's how stars quietly become meshes. Route them through the hub (it's one static route) or consciously amend the topology — never patch it.
• Every conversation needs both halves. The TGW zone gives the path out; the VPC's own route tables need the path back (per route table, per AZ, on both sides). Most "TGW doesn't work" incidents are a missing return route in a spoke VPC — asymmetric routing fails silently and only in one direction.
• Static routes accumulate; propagations don't. Propagated routes follow attachments automatically; the cross-region static routes (prod, stage, regional VPN pools) are copies in every zone that needs them. Adding a zone or a region means touching each — keep the routes-per-zone matrix in the repo, or drift wins. (Two # need to update comments in this codebase mark exactly where it started.)
• Feature-flagged topology is real, and weird. Stage's whole connectivity sits behind a Terraform flag (count on the peering, its accepter, associations and static routes) — great for cost, but a flag flip is a network change and deserves the same review a peering does.
• TGW is not free. Per-attachment-hour and per-GB pricing means the star is also the cost model: chatty cross-region traffic (CI artifacts to prod, log shipping) deserves an architectural answer — replicate or cache regionally — not a bigger pipe.
• Vendors won't join your TGW. A managed-database vendor peers the old way — plain VPC peering straight into the devops and product VPCs, with subnets carved from the reserved vendor block. Third-party connectivity is a permanent exception lane: keep it inside the CIDR map (10.5.0.0/16 here), or the first vendor with an overlapping range ends the plan.
• The free-tier OpenVPN box is a confession. The main entry point is a single small instance behind an NLB with a secondary ENI. It's the honest startup trade — a few hundred dollars a month saved against a door that can fall over. The NLB health check and a second instance is the upgrade path; know you haven't taken it yet.

Before you build: the decision checklist

• The CIDR plan, before the second VPC. (my choice — everything under one 10/8 supernet, a /16 per environment with the second octet naming it, regional blocks for VPN pools, 10.7 for tooling, the vendor lane reserved in 10.5 before any vendor exists)
• Account topology. Where does the network live? (my choice — a devops account owns every TGW, VPN and the tooling VPC; workload accounts own only their VPC and accept shares via RAM)
• Star or mesh. (my choice — star on the region where operators live; remote↔remote stays unroutable until a workload, not a convenience, demands it)
• Zones. How many route tables, drawn along what line? (my choice — per audience, not per VPC: an operator zone, a VPN-clients zone, a dev zone; prod is only ever a destination, never an audience)
• Propagated vs static. (my choice — propagate within the hub region, static across peerings — cross-region reachability should be a deliberate, greppable line of Terraform)
• Human entry points. How many, where? (my choice — one VPN per workforce continent, each a full network citizen with its own client CIDR; zero SSH bastions, zero public admin surfaces)
• The exception lane. Where do vendors and one-offs connect? (my choice — plain VPC peering, documented in the same CIDR map, explicitly not on the TGW)

The strongest firewall rule is a route that doesn't exist — and the only way to afford that luxury is to have planned the addresses before you needed them.

Related

This is the network under everything else in these notes: the Helm/ArgoCD environments deploy into these VPCs, and the SOC 2 series gets its network-segregation answer from the zone tables above.