In order for a service to be ready to be deployed into production, it must satisfy three critically important quality attributes:
This chapter’s first half, based on chapter 11 of [MP] C. Richardson, Microservices Patterns (Manning, 2019), designs services that satisfy these attributes; the second half, based on chapter 12 of the same book, covers the deployment patterns that get them into production.
Four different aspects of security to consider:
In a traditional monolithic application, when a user logs in with their user ID and password, the client POSTs the credentials to the application, which verifies them and returns a session token; the client includes the token in each subsequent request, and the application itself validates it. Implementing security in a microservice architecture requires deciding who is responsible for authenticating the user — typically the API gateway — and who is responsible for authorization.
Using the API gateway to authenticate a request before forwarding it to the services avoids requiring services to handle a diverse set of authentication mechanisms: only the gateway deals with the various mechanisms and hides this complexity from the services. Clients authenticate with the API gateway (API clients include credentials in each request; login-based clients POST credentials and receive a session token); once the gateway has authenticated a request, it invokes one or more services including the token in each service request, and the service uses the token to validate the request and obtain information about the principal.
Authorization, instead, can be implemented in two places with different trade-offs:
When implementing security in a microservice architecture we need to decide which type of token the API gateway uses to pass user information to the services. Two options:
A JWT has a payload, a JSON object containing information about the user (identity, roles) and metadata such as an expiration date. It is signed with a secret known only to the creator (e.g. the API gateway) and the recipient (the service), so a malicious third party cannot forge or tamper with it. Since a token is self-contained, it is irrevocable: a service will perform the request after verifying the signature and expiration date, and there is no practical way to revoke an individual JWT that has fallen into the hands of a malicious third party — the solution is to issue JWTs with short expiration times (limiting what a malicious party could do), with the application continually reissuing JWTs to keep the session active.
OAuth 2.0 is an authorization protocol originally designed to enable a user of a public cloud service to grant a third-party application access to its information without revealing its password. The key blocks:
In the API-client flow, the gateway authenticates the API client by making a request to the OAuth 2.0 authorization server, which returns an access token; the gateway then makes one or more requests containing the access token to the services. For session-oriented clients, the API client initiates a session by POSTing its credentials to the gateway’s /login endpoint; the gateway returns an access token and a refresh token, and the client supplies both tokens when making requests to the gateway.
Security wrap-up — the three key ideas: the API gateway is responsible for authenticating clients; the gateway and the services use a transparent token (a main example is a JWT) to pass around information about the principal; a service uses the token to obtain the principal’s identity and roles.
A service typically needs various configuration properties to be specified, and their values depend on which environment the service is running in at runtime. A service should be built once by the deployment pipeline and deployed into multiple environments; to this purpose, the appropriate configuration properties must be supplied to the service at runtime using an externalized configuration mechanism. Two main approaches:
Using a configuration server has several benefits: centralized configuration (all properties stored in one place, easier to manage), transparent decryption of sensitive data (encrypting sensitive data such as database credentials is a security best practice), and dynamic reconfiguration (a service could detect updated property values by, for example, polling, and reconfigure itself). Open-source frameworks such as Spring Cloud Config make it easier to run a configuration server.
Many aspects of managing an application in production are outside the scope of the developer (monitoring hardware availability and utilization), but to know what the application is doing, to be alerted if there’s a problem, and to troubleshoot and identify the root cause, there are several patterns that architects and service developers must implement to make the service easier to manage and troubleshoot — the observability patterns. A distinctive feature of most of them is that each pattern has a developer component and an operations component.
/actuator/health by running a set of health checks. The deployment infrastructure must also be configured to invoke the endpoint — e.g. service registries such as Netflix Eureka invoke it to decide whether to route traffic to the instance.jvm_memory_max_bytes or placed_orders), value (numeric), timestamp. Some monitoring systems also support dimensions, arbitrary name–value pairs (machine name, service name, instance identifier) along which samples are aggregated (sums or averages). The service developer is responsible for instrumenting the service and exposing service metrics (plus JVM/framework metrics) to the metrics server — e.g. a Spring Boot service includes the Micrometer Metrics library and a few lines of configuration; the FTGO example increments placedOrders, approvedOrders and rejectedOrders counters. Delivery: push model (the instance invokes the metrics service API, e.g. AWS CloudWatch metrics) or pull model (the metrics service or its local agent invokes a service API, e.g. Prometheus, visualized with Grafana). Alerts on metrics (e.g. the rate of change of placed_orders_total falling below a threshold) enable a quick response to production issues, sometimes before they impact users.The microservice chassis pattern is a framework or set of frameworks responsible for integrating and handling different cross-cutting concerns: externalized configuration, health checks, application metrics, service discovery, circuit breakers, distributed tracing. Its purpose is avoiding setting up these concerns from scratch each time we implement a new service: it significantly reduces the amount of code to write — mostly configuring the chassis to fit our requirements — and enables developers and architects to focus on the service’s business logic. Examples: Spring Boot + Spring Cloud, GoKit or Micro for Go.
The microservice chassis is a good way to implement cross-cutting concerns, but we need one for each programming language we use. An emerging alternative that avoids this problem is to implement some of this functionality outside of the service in a service mesh: a networking infrastructure that mediates the communication between a service and other services and external applications. A tech example is Istio, an open-source service mesh that enables developers to connect, control, monitor and secure microservice architectures — providing observability, robust communication and control even as the number of microservices increases, working with any microservice regardless of its platform, source or vendor, through a unified layer (proxies) between application services and the network.
Deployment is the process — the steps taken by developers and operations — to get software into production; the deployment architecture is the structure of the environment in which that software runs. There have been big changes in the last 30 years. A production environment must implement four key capabilities:
Four approaches/options/patterns to deploy services:
Security — mostly not different from a monolith, but some aspects of application-level security must be implemented differently; configurability — configuration properties (locations and credentials of external services) cannot be hard-wired: an externalized configuration mechanism provides them at runtime, so the service is built once and deployed into multiple environments; observability — understanding the behaviour of the application and troubleshooting problems.
The four aspects: authentication (verifying the identity of the principal), authorization (verifying the principal is allowed to perform the operation), auditing (tracking the operations a user performs), and secure interprocess communication (ideally TLS everywhere). In a monolith the application itself verifies credentials and returns a session token. In a microservice architecture the API gateway is typically responsible for authenticating clients — hiding the diversity of authentication mechanisms from the services — and forwards a token with each request; services validate the token and obtain information about the principal.
In the API gateway: drawbacks are coupling the gateway to the services (lockstep updates) and the practical limitation to role-based access on URL paths — ACLs on individual domain objects require detailed knowledge of the service’s domain logic. In the services: a service can implement role-based authorization for URLs and service methods, and ACLs to manage access to aggregates.
Opaque tokens (typically UUIDs) force the recipient to make a synchronous RPC to a security service to validate the token and retrieve user information, reducing performance and availability and increasing latency. Transparent tokens contain the user information themselves; JWT is the standard: a JSON payload with identity, roles and metadata (e.g. expiration), signed with a secret shared by creator and recipient so it cannot be forged or tampered with. The drawback: a JWT is self-contained and irrevocable — there is no practical way to revoke an individual token that has fallen into the wrong hands — so tokens must have short expiration times and be continually reissued.
Authorization Server: API for authenticating users and obtaining an access token and a refresh token. Access Token: grants access to a Resource Server (format implementation dependent; some implementations use JWTs). Refresh Token: long-lived yet revocable, used to obtain a new access token. Resource Server: a service that uses the access token to authorize access — in a microservice architecture, the services. Client: wants to access a Resource Server — in a microservice architecture, the API Gateway is the OAuth 2.0 client. Session-oriented clients POST credentials to the gateway’s /login endpoint and receive both tokens.
A service should be built once and deployed into multiple environments; the appropriate configuration properties must be supplied at runtime by an externalized configuration mechanism. Push model: the deployment infrastructure passes the properties to the instance when it creates it, e.g. via OS environment variables or a config file. Pull model: the instance reads its properties from a configuration server, which brings centralized configuration, transparent decryption of sensitive data (e.g. database credentials), and dynamic reconfiguration (polling for updated values); e.g. Spring Cloud Config.
A service instance can be running but unable to handle requests (still initializing, e.g. FTGO’s Consumer Service takes ~10 seconds; or failed without terminating, e.g. out of database connections), so it needs to tell the deployment infrastructure whether it can handle requests. Implementation: simply verify access to external infrastructure services (e.g. a test query against the database), or more elaborately execute a synthetic transaction simulating a client invocation; e.g. Spring Boot Actuator’s /actuator/health runs a set of health checks. Invocation: the deployment infrastructure must be configured to call the endpoint periodically and act on the result — e.g. Netflix Eureka invokes it to decide whether to route traffic to the instance.
Log aggregation: service logs are scattered across instances, so a pipeline (e.g. Logstash) sends them to a centralized logging server (e.g. Elasticsearch) with visualization (Kibana) and alerting; services should log to stdout (containers/serverless may have no permanent filesystem). Distributed tracing: each external request gets a unique ID; a trace is the tree of spans (operations with name, start/end) recorded by an instrumentation library (ideally via AOP, e.g. Spring Cloud Sleuth) and stored/visualized by a tracing server (Zipkin, AWS X-Ray); the request ID in log entries ties tracing to log aggregation. Application metrics: samples with name, value and timestamp, plus optional dimensions for aggregation; the developer instruments the service (e.g. Micrometer) and exposes metrics via push (AWS CloudWatch) or pull (Prometheus + Grafana); alerts respond to production issues, sometimes before users are impacted.
Log files are oriented around single-line entries while exceptions span multiple lines; there is no mechanism to track the resolution of exceptions (manual copy/paste into an issue tracker); and duplicate exceptions are not automatically treated as one. An exception tracking service receives reported exceptions (e.g. via REST), de-duplicates them, generates alerts, and manages resolution — with client libraries such as HoneyBadger’s. Audit logging, by contrast, records each user’s actions (identity, action, business objects) via business-logic code, AOP or event sourcing.
A microservice chassis is a framework (or set of frameworks) that integrates cross-cutting concerns — externalized configuration, health checks, application metrics, service discovery, circuit breakers, distributed tracing — so each new service only configures the chassis and developers focus on business logic; examples: Spring Boot + Spring Cloud, GoKit/Micro. The drawback is one chassis per language. A service mesh moves part of this functionality outside the service into a networking infrastructure that mediates service-to-service and external communication (e.g. Istio, through proxies and a control plane), working with any microservice regardless of platform, source or vendor.
Service management interface (create, update and configure services, e.g. a REST API for CLI/GUI tools), runtime service management (keep the desired number of instances running: restart crashed or unhealthy instances, move instances off failed machines), monitoring (insight via logs and metrics, with alerting — observability through monitoring), and request routing (route requests from users to the services).
Language-specific packages (JAR/WAR, source dirs): fast deployment and efficient resource use, but no technology-stack encapsulation, no resource constraints, no isolation, and hard placement decisions — rarely used except when efficiency outweighs everything. VMs: image encapsulates the stack, full isolation, mature cloud infrastructure — but less-efficient resource utilization, slow deployments, and system administration overhead. Containers (Docker, orchestrated by Kubernetes): same encapsulation/isolation benefits as VMs but lightweight — fast to build, move and start (no OS boot) — drawbacks: you administer images and, unless hosted, the container/VM infrastructure. Serverless (AWS Lambda, Azure Functions): no provisioning or system administration, elasticity, usage-based pricing, integration with other services — but long-tail latency and a limited event/request-based programming model (not for long-running services).