Cloud networking architecture — VPC, subnets, routes, peering, endpoints

The substrate is software, and that changes the rules

In the virtual network the enforcement point is not a switch or a firewall appliance. It is the hypervisor, or increasingly a programmable NIC beside it, sitting between the guest and the physical port. When a guest sends a frame, that layer looks up where the destination virtual address currently lives, wraps the packet for transport across the provider's physical network, and unwraps it on the far side. A mapping service, not a broadcast domain, decides where addresses are.

Several practical consequences fall straight out of that design. Broadcast and multicast generally do not work, so anything that depends on gratuitous ARP, a floating virtual address negotiated by VRRP, or a layer-2 heartbeat between cluster members has to be redesigned around a load balancer or a coordination service. Putting an interface into promiscuous mode shows you your own traffic and nothing else, which is precisely why the provider has to offer flow logs as a separate product. And the address the outside world sees is frequently not the address configured on the interface.

What this replaced was slow but legible: a physical topology you could walk, a firewall change measured in working days, and an address range that was fixed when the rack was built. The trade is speed for reversibility. Creating an isolated network is now a thirty-second API call, so teams create dozens of them without a plan, and a single mistaken apply reaches every zone at once. Convergence is also eventual: a rule or route change propagates to many enforcement points, so a policy edit is fast but not instantaneous, and a test run immediately after an apply can honestly report the old behaviour.

The moving parts, and which article owns each

The diagram below is the whole board. The top strip is structure: the virtual network itself, the subnets carved out of it, the route tables that steer, and the gateways that let traffic reach the internet. The middle strip is reachability and policy: connections to other networks, private paths to managed services, name resolution, and the two independent filtering layers. The bottom strip is the part teams skip until an incident: traffic visibility, latency and drop telemetry, and the address-management discipline that keeps all of it possible.

Four of these boxes are covered in depth elsewhere in this collection, and this article deliberately stops short of repeating them. VPC peering covers the two-party connection, why it refuses to forward between peers, and the arithmetic of a full mesh. Managed NAT gateways cover source translation, the port pool, and per-zone placement. Private connectivity covers private endpoints and the service-exposure model. Path MTU discovery covers the discovery protocol itself. What follows is the connective tissue: the decisions that sit above those components and the ways a whole network fails rather than one component.

Address planning is the one choice you cannot take back

Every other setting in a virtual network can be edited. The address range effectively cannot: changing it means renumbering live workloads, and no one does that voluntarily. So the range is chosen on day one, usually by whoever created the first environment, usually by accepting a default, and it constrains the estate for as long as the estate exists.

The arithmetic people get wrong

A /24 looks like 256 usable addresses. It is not. Providers reserve a small block at the bottom and top of every subnet for the implied gateway, the resolver, and future use, typically four or five addresses, so budget around 250. Then count what actually consumes them. Virtual machines are the smallest term. A container platform that gives every pod a routable address will burn addresses an order of magnitude faster than the node count suggests, and nodes often pre-allocate a slab of addresses so they can place pods without a control-plane round trip. Internal load balancers take an address per zone. Each private endpoint takes an address per zone. Managed database instances, cache clusters, and serverless functions attached to the network all draw from the same pool.

When the private ranges run out

RFC1918 offers 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16. The first looks bottomless at roughly sixteen million addresses. It is not, because the unit of consumption is not the address, it is the block. If the platform standard is a /16 per network, then 10.0.0.0/8 holds exactly 256 of them, and an organisation that gives each team a separate network per environment passes 256 within a couple of years. What happens next is the damaging part: someone allocates a range that is already in use, on the reasoning that the new network is isolated and will stay that way. It never stays that way.

The retrofits are all worse than planning. A secondary range bolted onto an existing network buys capacity but does nothing about a collision. Translating an overlapping range behind a proxy range at the boundary does work, and it also destroys address attribution in your logs and breaks any protocol that carries addresses inside the payload. Dual-stack addressing is the only genuine exit, because the global unicast space is large enough that collisions stop being a design constraint. Overlap is also what quietly kills integration after an acquisition: two companies who both standardised on the same friendly range cannot join their networks without one of them renumbering.

Subnets and zones: two jobs in one object

In most providers a subnet is pinned to exactly one availability zone, and that single fact does more architectural work than anything else in the model. (Some providers scope the network globally and the subnet regionally; the binding still exists, it just sits at a different level.) The subnet is simultaneously a failure domain and an allocation pool, and the two roles pull in opposite directions.

As a failure domain it means zone redundancy is expressed by having one subnet per zone per tier. Three zones and three tiers is nine subnets before you have deployed anything, each with its own route-table association and its own filtering, and that multiplication is why network configuration in a mature account is mostly repetition. It also means a resource is only as available as the zones its subnets span, and that a placement decision made by an autoscaler is constrained by which subnets you handed it.

As an allocation pool it means the subnet is where you run out of addresses, and running out is not a networking incident, it is a capacity incident. The scaling event fires, the platform asks for an address in the zone under load, the subnet has none, and the scale-out fails. The application team sees a service that will not grow past two thirds of its intended size and no error in their logs. Most providers will not let you enlarge a subnet in place, so the fix is a new subnet with new associations and new rules, applied under pressure. The alarm worth having is free addresses remaining per subnet, per zone, with a threshold set high enough to leave time for that work. Utilisation as a percentage is the wrong metric, because a small subnet at eighty percent is a different emergency from a large one.

Route tables, the local route, and the silent black hole

Routing inside a virtual network is deliberately simple: a table of destination prefixes with a target for each, evaluated by longest prefix match. Every table is created with one entry you did not write and cannot remove, covering the network's own range and pointing at the internal fabric. That implicit local route is why two instances in different subnets of the same network can talk with no configuration at all, and it is also why inserting an inspection appliance between two subnets is awkward: you are trying to beat a route that is already as specific as the network itself, so you need a more specific prefix to win the match, and providers vary in how much of that they permit.

There is no attribute anywhere that marks a subnet public or private. That property is entirely a consequence of which table the subnet is associated with and what that table's default route points at. A subnet is public because its table sends unmatched traffic to an internet gateway, and private because it sends it somewhere else or nowhere.

destination         target                   why it is there
-----------------   ----------------------   --------------------------------
10.42.0.0/16        local                    implicit, cannot be deleted
10.42.7.0/24        appliance-nic-a          more specific, so it wins
10.90.0.0/16        hub-attachment           reaching another network
storage-prefix      service-endpoint         managed service, stays internal
0.0.0.0/0           nat-zone-a               everything else, outbound only

The failure mode to internalise is what happens when nothing matches. The packet is discarded and nothing is sent back. The client does not receive a refusal, it receives silence, and it will sit there until its own timeout expires. This is worth knowing because the two symptoms separate cleanly: a connection refused means a host received your packet and declined, so routing and filtering are fine and the problem is the process. A hang means either no path forward or no path back, which puts you in the route tables and the stateless filter rather than in the application.

The four ways a packet leaves

Egress is a route-table decision, not a rule decision, and that is a useful property: you can read one table and know which of four paths a workload is on.

Through an internet gateway. Bidirectional and address-transparent, which means the instance needs a public address of its own and is reachable from outside unless a filter says otherwise. Correct for a public load balancer, wrong for almost everything behind one.

Through a managed translation gateway. Outbound only, many private workloads sharing a small number of public addresses. This is the default answer for private compute that still needs to fetch packages and call third-party APIs. The mechanism, its port-pool ceiling and its per-zone placement are covered in the NAT gateway article. The point to carry here is that this path is metered on volume as well as on time, so a route-table entry is also a recurring invoice line, and traffic that could have stayed internal is the usual reason that line is larger than anyone expected.

Through a private endpoint. Traffic to a managed service never leaves the provider's network at all, so it is neither exposed nor billed as internet egress. See private connectivity for how the endpoint and its name resolution work.

Not at all. The most secure configuration, and the most common source of a build that mysteriously hangs. Package installers, operating-system updates, certificate revocation checks, time synchronisation and telemetry agents all want somewhere to go, and each of them fails in its own confusing way when there is no route. Deciding a subnet has no egress is a decision to provide internal mirrors for all of that.

A fifth path exists in regulated environments: an explicit egress proxy or firewall that all outbound traffic is routed through, so policy can be expressed by hostname rather than by address. That matters because the addresses behind a SaaS hostname change without notice, which makes address-based allow-lists for external services a maintenance treadmill.

Security groups versus network ACLs

Two filtering layers exist and they are not variations on a theme. They differ in the property that matters most: whether they remember a connection.

Because a security group tracks state, permitting an outbound connection automatically permits its replies, and the reverse. Because it can name another group as its source, a rule reading "accept from whatever is in the application tier" keeps meaning the right thing through every scale event, replacement and redeploy, with no rule edit. That is the single most valuable feature in the model and the reason security groups, not address lists, should carry your actual intent.

A network ACL evaluates each direction independently and knows nothing about connections. So an outbound rule permitting traffic to a database port is not enough: the reply arrives on a high-numbered ephemeral source port, and an inbound rule has to permit that whole range or the reply is dropped. This produces the layer's signature symptom. The handshake half-completes, or the connection opens and the first substantial response never arrives, and the client hangs. It looks nothing like a filtering problem, and it affects everything in the subnet including services whose owners were not part of the change.

The practical division of labour follows from that asymmetry. Express intent in security groups, where mistakes are loud, local and self-correcting. Use network ACLs sparingly as a coarse subnet-wide backstop, for the small number of statements that are genuinely absolute, such as denying a known-bad range or blocking a port class outright. Keep them few and keep them boring, because the debugging cost of a subtle stateless rule is measured in hours by people who do not know the rule exists. Watch the per-interface rule limits too: they are lower than teams expect, and a group that has grown one address at a time will hit them.

Hybrid connectivity: a tunnel or a circuit

Connecting the virtual network to premises comes down to two options with very different procurement, cost and failure profiles.

An encrypted tunnel over the public internet is a managed gateway on the cloud side, a compatible device on yours, and shared keys. It can be running the same afternoon, it costs little, and it inherits whatever the internet gives you that day for latency, jitter and loss. Throughput per tunnel is capped, and the way past the cap is more tunnels with traffic spread across them, which then requires your routing to actually balance across them.

A dedicated circuit is a physical cross-connect at a facility where the provider has presence, or a port rented from a partner who is already there. You get deterministic bandwidth and stable latency, and you pay for it with a lead time measured in weeks and a contract. Note that a private circuit is not an encrypted one: the packets are off the public internet but they are in clear text on someone's fibre, so link-layer encryption or a tunnel on top is usually a compliance requirement rather than an option.

Both approaches carry their reachability over BGP, and route advertisement is where these projects actually succeed or fail. The recurring failures are worth naming. Advertising a default route from premises into the cloud silently reroutes all your internet-bound traffic through the corporate datacenter, which works and is slow and expensive and nobody notices for a month. Advertising an aggregate from the cloud side that overlaps a range still live on premises sends a fraction of traffic the wrong way, intermittently. Exceeding the advertised prefix limit does not drop the excess prefixes, it drops the session, so an innocuous new range on one side takes down the whole link. And a redundant pair configured without deliberate path preference will happily send traffic out one link and receive replies on the other, which is fine until a stateful device is in the middle.

The last mile is local: a route learned over BGP still has to be propagated into the subnet route tables that need it. A tunnel that is up, with the prefixes learned, and no propagation enabled, produces the most common hybrid ticket there is.

The transit hub, and why the mesh forces it

Direct connections between networks are a two-party primitive. They do not forward, so reaching many networks from many networks means connecting each pair, and the count of connections and route entries grows with the square of the estate. There is a size, usually somewhere around a dozen networks, past which nobody can answer "what can production reach" without auditing every table.

The hub inverts this. Each network attaches once to a routed hub that does forward, so the connection count grows linearly and the hub's own tables become a single authoritative statement of the topology, reviewable in one place. The price is that transitivity is now the default rather than the exception, so isolation has to be recreated deliberately by grouping attachments into separate route domains. The hub is also a shared, metered, quota-limited component on the path of nearly all internal traffic, which makes it both a cost centre and a blast radius. The mesh arithmetic is worked through in the peering article, and one provider's implementation in depth in Transit Gateway.

Name resolution inside the network

Each virtual network is served by a resolver at a predictable address derived from the network's own range, handed to instances by DHCP along with a search domain. Instances get names for free, and that resolver is also the hinge on which private connectivity and hybrid integration turn.

The mechanism that matters is split-horizon resolution: a private zone attached to one or more networks that shadows a name which also exists publicly. The same hostname returns an internal address to a client inside the network and a public address to a client outside it. This is what makes private endpoints transparent, and it is what lets an application configuration be identical in every environment while resolving to something different in each.

The gaps appear at boundaries, in both directions, and neither is created for you. Clients on premises querying a cloud-private name need their resolver to forward to an inbound resolver address inside the network. Cloud instances querying names that only exist on premises need conditional forwarding rules pointing outward. Until something tries to resolve across the boundary, both look fine.

The confusion this produces is distinctive and worth recognising immediately: after a new network connection is established, everything works by address and nothing works by name. Routes were configured, zone associations were not. Nothing is wrong with the network; the resolver in the client's network has simply never heard of the zone.

Two more traps. The same zone name attached to two networks with different records is a split-brain generator that resolves differently depending on where the query originated, which is exactly as fun to debug as it sounds. And caching lives on the client: a runtime that resolves once at startup and holds the result forever will keep dialling an address that failed over an hour ago, so a DNS-based failover strategy is only as good as the least well-behaved client in the fleet.

MTU: unglamorous, and a genuine source of hangs

Inside a virtual network large frames are usually available, commonly around nine thousand bytes, and workloads that move bulk data benefit from them. The moment traffic crosses a gateway, a tunnel or the internet it drops to the ordinary limit or below, and encapsulation and encryption headers take a further bite, so the usable payload over a tunnel can be well under 1400 bytes.

Sizing is supposed to sort itself out. The sender marks packets as not fragmentable, a device in the path that cannot forward one replies with an ICMP message naming the smaller size, and the sender adjusts. The protocol itself is covered in path MTU discovery. What is specific to cloud networks is how easily that feedback message is destroyed.

A stateless subnet filter written as a tidy allow-list of TCP ports discards it. A security group that permits only TCP discards it. In neither case does anyone believe they have done anything to ICMP. The sender therefore never learns, and the resulting symptom is one of the most misleading in the whole stack: the handshake succeeds because handshake packets are tiny, small requests succeed, and anything above a size threshold hangs until it times out. It correlates with payload size rather than with any endpoint, it never appears in application logs, and it is invariably investigated as an application bug first.

Three defences, in order of reliability. Permit the unreachable and fragmentation-needed message types through both filtering layers. Clamp the advertised segment size on tunnel interfaces so endpoints negotiate a safe size instead of discovering it. And do not configure large frames on instances whose traffic leaves through a gateway, because the benefit applies to internal traffic and the cost applies to everything else.

Flow logs, and the question nothing else answers

Sooner or later someone asks what is actually talking to what. In a virtual network there is no span port and no tap, so flow logs are the only answer available. They can be enabled per interface, per subnet or for the whole network, and each record describes an aggregated flow: source and destination address and port, protocol, packet and byte counts, a time window, and a verdict of accepted or rejected.

Be clear about what they are not. They are not packet capture, so there is no payload, no hostname, no process and no user. You get an address, and turning an address into a workload means joining against your own inventory at the time the flow happened. In an environment where addresses are recycled between short-lived workloads within minutes, that mapping is more perishable than the logs themselves, and retaining one without the other leaves you with data you cannot interpret. Aggregation also means records appear minutes after the traffic did, so they are an investigation tool and not the basis of a fast alarm.

The rejected records are the immediately useful half, because they show what your current policy is stopping and therefore where a legitimate caller is being blocked. The accepted records are what you need for the harder job, which is closing something down. That workflow is worth stating: log for a period long enough to include the monthly and quarterly jobs everyone forgets, group accepted flows by peer and destination port, derive a candidate rule set, apply it to a small subset first, and watch for new rejections before rolling it out. There is no shortcut, because the information simply does not exist anywhere else.

Volume is a real consideration on a busy network. Capturing every flow at full fidelity produces a large and continuous stream, so sampling, or capturing only rejections during steady state and switching to full capture during an investigation, is the usual compromise.

Failure modes worth memorising

Asymmetric routing. Traffic leaves by one path and returns by another. Nothing is wrong until a device that keeps connection state sits somewhere in the middle, at which point the return packet arrives at a device with no matching entry and is dropped as unsolicited. The signature is a service that works for short exchanges and fails for long ones, or works within a zone and fails across zones. Every centralised-inspection design has to consider it, and the fix is always to pin both directions of a flow to the same device rather than to add more capacity.

An exhausted subnet capping a service. Covered above, but it belongs on this list because of how it presents: not as a network alert but as a service that will not scale, discovered during the incident it caused. Alarm on remaining addresses per subnet.

The rule nobody can close. Somewhere in every mature estate is a security group permitting a wide port range from everywhere, added during an incident several years ago by someone who has since left, and retained because nobody can prove what depends on it. Ask a team to close it and the honest answer is that they do not know what will break. This is the same problem as the visibility problem, which is why the audit capability and the policy capability are one conversation and not two: without accepted-flow data, tightening a rule is a guess, and a guess that causes an outage teaches the organisation not to try again.

Stale routes after teardown. Removing an attachment or a connection does not always remove the routes that pointed at it, and a route to a target that no longer exists is a black hole rather than an error. Traffic to that prefix disappears silently, which is the worst possible way for a decommissioning task to be incomplete.

Working by accident. A workload functions because of a rule, route or zone association owned by another team in a shared network. Nobody notices until that team cleans up. Shared networks are efficient and they move the boundary of "my configuration" somewhere the owning team cannot see.