service-architecture

Service Architecture

Service architecture is the discipline of decomposing a large system into cooperating services with explicit contracts, operational boundaries, and failure isolation. The goal is not to have as many services as possible — it is to minimize the coupling between parts of the system that change at different rates or have different operational requirements, while keeping the total operational and cognitive cost manageable. This skill catalogs the patterns, trade-offs, and anti-patterns that come up every time a team crosses the boundary from monolith to service-oriented, or from service-oriented to mesh.

Agent affinity: vogels (service-oriented architecture at Amazon scale), dean (internal Google service patterns), hamilton-cloud (economics of operating services at scale)

Concept IDs: cloud-multi-service-coordination, cloud-heat-stack-templates, cloud-runbook-structure

The Core Principle: Coupling vs Cohesion

A well-designed service has high internal cohesion (things inside the service change together) and low external coupling (things outside the service change independently). The unit of service decomposition is not the noun from the domain model — it is the change rate and the failure domain. Two nouns that always change together belong in the same service even if they feel conceptually distinct. Two fields of the same noun may belong in different services if they are owned by different teams and have different availability requirements.

This is why "split every table into its own microservice" is an anti-pattern. Tables that are always read and updated together have high operational coupling even if they are logically separate, and splitting them produces distributed transactions that are much harder to get right than a local one.

The Vogels Principle: You Build It, You Run It

Werner Vogels articulated the operating principle that the team that builds a service also operates it. Pager rotations, on-call burden, postmortem ownership, and feature velocity are all aligned under a single team. This is not primarily a technical pattern — it is an organizational pattern with technical consequences:

• Teams that feel operational pain invest in observability and automation.
• Teams that do not feel operational pain ship features that generate operational pain for others.
• The SLO for a service must be owned by the team that can change the service.

A service boundary is also an accountability boundary. If you cannot say which team owns a service's on-call, the service boundary is wrong.

Service Contract Design

The API of a service is a long-term commitment. Clients depend on it. Breaking changes propagate blast radius across the organization. The design goal is to make it easy to add capability and hard to break existing clients.

Backward Compatibility

• Additive changes are free. New optional fields in requests, new fields in responses, new endpoints, new enum values with a default "unknown" bucket.
• Subtractive and type-change changes are expensive. Removing fields, renaming fields, narrowing types, making optional fields required — all require coordinated client updates.
• Versioning conventions. Path-based (/v1/, /v2/), header-based (Accept: application/vnd.service.v1+json), or capability-based (clients negotiate features). Path-based is simplest and most commonly understood.

The Must-Ignore Rule

Clients should ignore unknown fields in responses and servers should ignore unknown fields in requests. This allows both sides to evolve independently as long as the evolution stays within additive changes.

Idempotency Keys

Network failures mean clients retry. Without idempotency, retries can double-charge, double-enroll, or double-ship. Every mutating endpoint should accept a client-supplied idempotency key (UUID), store it with the result for some retention period, and return the same result for duplicate requests. Stripe popularized this pattern and it is now table stakes for any external API.

Inter-Service Communication Patterns

Synchronous Request/Response

The default. Simple, but creates temporal coupling (the caller cannot proceed until the callee responds) and failure coupling (the callee's unavailability becomes the caller's unavailability). Appropriate when the caller genuinely needs the response to make progress.

Asynchronous Messaging

Producer writes a message to a queue or topic; consumer processes it later. Decouples temporally and tolerates brief failures of either side. Introduces complexity around delivery semantics (at-most-once, at-least-once, exactly-once — the last of which is mostly a lie in practice) and message ordering.

Events and Pub/Sub

A producer emits facts ("order placed") without knowing who consumes them. Consumers subscribe to event types they care about. This is the loosest coupling and the highest flexibility, and also the hardest to debug because the producer has no idea where its events end up.

Request/Reply over Messaging

A hybrid: use messaging for transport but include a reply-to address. Gains retry-friendliness and decoupling at the cost of extra plumbing.

Distributed Transactions: Sagas

Traditional two-phase commit is brittle at cloud scale — any participant failure stalls the entire transaction. The saga pattern decomposes a transaction into a sequence of local transactions, each of which has a compensating action that undoes its effect.

Orchestrated saga. A central coordinator invokes each step and triggers compensations if a step fails. Simpler to reason about, but the coordinator is a bottleneck and single point of failure.

Choreographed saga. Each step emits an event that triggers the next step. No central coordinator, but the control flow is distributed across the services, making it harder to debug.

Both forms give "eventual consistency with compensation" rather than atomic consistency. Applications must be designed to tolerate intermediate states visible to users (e.g., "order confirmed but payment not yet processed" — show "pending" in the UI).

Resilience Patterns

Circuit Breaker

Wraps a downstream call. Tracks failures in a sliding window; if the failure rate crosses a threshold, the circuit "opens" and subsequent calls fail fast without even trying the downstream. After a cooldown, the circuit goes "half-open" and allows a probe request through. This prevents a degraded downstream from consuming all the upstream's thread/connection resources.

Bulkhead

Partition resources so that failure of one partition does not exhaust the whole system. Per-downstream thread pools, per-tenant connection limits, per-endpoint rate limits. Named after ship bulkheads — if one compartment floods, the ship stays afloat.

Timeout and Deadline Propagation

Every RPC needs a timeout. Deadlines should propagate through the call graph: if the original request has 500ms to live, the sub-call should have less than that. Without deadline propagation, slow downstreams can tie up upstream resources for minutes.

Retry with Exponential Backoff and Jitter

Retries help transient failures but can cause thundering herds if all clients retry at the same time. Exponential backoff (wait 1s, 2s, 4s, 8s) plus jitter (add a random factor) spreads the load.

Graceful Degradation

When a downstream is unavailable, serve a reduced experience rather than failing entirely. Cache last-known values, return empty recommendation lists, hide non-essential UI. Users prefer a degraded site to a broken one.

Service Mesh

A service mesh (Istio, Linkerd, Consul Connect) moves cross-cutting concerns — retries, timeouts, circuit breaking, mTLS, observability — out of application code and into a sidecar proxy. Each service pod gets an Envoy (or similar) proxy that handles ingress and egress traffic.

Benefits. Uniform policy, uniform observability, language-agnostic, centralized control plane.

**Cos