Part III — Microservices and Reactive Architectures · Chapter 11

Architectures for Reactive Systems

~45 min read4 interactive widgets5 plates

In this chapter

  1. Reactive systems: transformational versus reactive
  2. Typical characteristics
  3. The Reactive Manifesto
  4. Reactive architecture: the central idea and the rules for modularization
  5. Divide and conquer
  6. Loose coupling and message passing
  7. Location transparency
  8. Vertical and horizontal scalability
  9. Principled failure handling
  10. Delimited consistency
  11. Design criteria for modules — sum-up
  12. Check your understanding

1. Reactive systems: transformational versus reactive

Reactive systems form a broad category of systems designed to respond to external stimuli or events, typically involving requirements about time: they are designed to react promptly to changes in their environment or input. The term “reactive” emphasises the system’s ability to react dynamically to events as they occur.

The classic characterisation, due to Harel and Pnueli [HP89], contrasts them with transformational systems:

TRANSFORMATIONAL  VS  REACTIVE [HP89] TRANSFORMATIONAL accepts input, transforms it, produces outputs batch payroll · compiler · report generator input output one-shot or stepwise transformation REACTIVE repeatedly prompted by the outside world, continuously responds to external inputs thermostat · web server · trading system stimuli responses ongoing relationship with the environment
Plate 11.1 — Transformational systems turn inputs into outputs; reactive systems maintain an ongoing relationship with their environment, repeatedly prompted by it. The distinction is from Harel and Pnueli, 1989.

2. Typical characteristics

Reactive systems typically share five characteristics:

3. The Reactive Manifesto

The Reactive Manifesto (reactivemanifesto.org) states the key properties of a modern distributed system. It must:

THE REACTIVE MANIFESTO Responsive Resilient Elastic Message Driven VALUE: responsive, resilient, elastic FORM: achieved through message-driven MEANS: asynchronous message passing maintainable & extensible systems are the outcome; see also Kuhn, Reactive Design Patterns
Plate 11.2 — The Reactive Manifesto. Responsiveness, resilience and elasticity are the value; their form is a message-driven architecture; the means is asynchronous message passing. The reactive design patterns book by R. Kuhn is the companion reference.

4. Reactive architecture: the central idea and the rules for modularization

The central idea of reactive architecture, taken from the Reactive Manifesto, is a single main decomposition principle:

The main decomposition principle

Decompose the overall business problem in a hierarchical fashion into fully encapsulated modules that communicate only by asynchronous, nonblocking, location-transparent message passing.

This principle directly connects to the concurrency discussion that closed Chapter 9: DDD’s domain model pattern does not capture concurrency, and reactive architecture answers with a metamodel in which the unit of behaviour is a module communicating by messages — the actors of Chapter 12’s territory.

The rules for modularization (the design criteria the module returns to at the end):

  1. A module does one job and does it well.
  2. The responsibility of a module is bounded by the responsibility of its parent.
  3. Module boundaries define the possible granularity of horizontal scaling by replication.
  4. Modules encapsulate failure, and their hierarchy defines supervision.
  5. The lifecycle of a module is bounded by that of its parent.
  6. Module boundaries coincide with transaction boundaries.

5. Divide and conquer

Divide et regna — from Julius Caesar and the Roman Empire: when faced with a number of enemies, create discord and divide them, so you can vanquish them one by one, even though united they would easily have defeated you. Applied to architecture: hierarchical problem decomposition.

Modules can have dependencies on each other: the hierarchy defines dependencies and descendants — each module depends on the modules below it, and the parent’s responsibility bounds its descendants’.

6. Loose coupling and message passing

Hierarchical problem decomposition leads to a set of loosely coupled modules with clearly segregated responsibilities — for example, microservices as modules. Modules interact by means of well-specified interaction protocols based on pure message passing, regardless of whether the collaborating modules execute within the same (virtual) machine or on different network hosts. Asynchronous message passing is the key choice for loose coupling and integration.

Message passing and flow control

Message passing enables flow control: the process of adjusting the transmission rate of a stream of messages to ensure that the receiver is not overwhelmed. Whenever this process informs the sender that it must slow down, the sender is said to experience back pressure. Message passing offers a wider range of options for flow control than synchronous calls, because it includes the notion of queueing.

Message passing and events

Messages naturally represent events, and message passing naturally represents event-driven interactions: an event propagating through a system can also be seen as a message being forwarded along a chain of processing units. Representing events as messages enables the trade-off between latency and throughput to be adjusted case by case, or even dynamically.

7. Location transparency

Location transparency is the property that the source code for sending a message looks the same regardless of where the recipient will process it. Application components interact with each other in a uniform fashion defined by explicit message passing; an object that allows a message to be sent becomes just a handle pointing to its designated recipient. This handle is mobile and can be passed around freely among network nodes.

Careful — not transparent remoting

Transparent remoting (CORBA, Java RMI, Microsoft DCOM — the 1980s and 1990s) tried to unify the programming model for local and remote method invocations, making remote invocation appear the same as local ones. It did not achieve the expected outcome: partial failure cannot be abstracted away, and increased latency brings systemic effects.

Location transparency does not aim to make remote interactions look like local ones. Its goal is to unify the expression of message passing under a common abstraction for both local and remote interaction — for example actors: ActorRefs in Akka, process IDs in Erlang. Local message passing can then be optimised as a special case.

LOCATION TRANSPARENCY Sender business logic handle mobile reference Receiver node A Receiver node B same source code, anywhere on the network ActorRefs in Akka, process IDs in Erlang: the handle is the only thing the sender ever sees
Plate 11.3 — Location transparency: the sender’s code is identical whichever node hosts the receiver. The handle can be passed around freely; the sender never needs to know where the recipient will process the message.

8. Vertical and horizontal scalability

There are two ways to scale:

Vertical scaling (scaling up): adding more power to your current machines — upgrading the CPUs, memory, storage or network speed when a server requires more processing power.

Horizontal scaling (scaling out): adding additional nodes or machines to the infrastructure to cope with new demands — for example adding a new server when the application on the existing one no longer has the capacity to handle the traffic.

Message passing decouples caller and callee, turning them into sender and receiver: this enables vertical scalability, because the receiver is free to use different processing resources than the sender — possible since they do not execute on the same call stack. Location transparency then adds horizontal scalability: the receiver can be placed anywhere on a reachable computer network without the sender needing to know where, so performance can be improved by adding more computers.

9. Principled failure handling

Hierarchical decomposition of the modules yields a supervision structure: resilience requires distributing and compartmentalising systems. In order to restore proper function after a failure, the responsibility of reacting to this event must be delegated to a supervisor, the owner of the module.

“Ownership means commitment”

A supervisor is responsible for monitoring the health of its descendant modules and initiating the start of new ones in case of failure. Only when that does not work does it signal the problem to its own supervisor. Failure is treated as an expected fact of life — it is “not swept under the carpet” — in contrast to the traditional method of throwing exceptions back to the calling module.

Ownership implies lifecycle control: a module will need to create all submodules that it owns, and as supervisor it must be able to re-create them in case of failure — it must literally own the lifecycle of its submodules. The lifecycle of descendant modules is strictly bounded by the supervisor’s lifecycle: the supervisor creates it, and without a supervisor it cannot continue to exist.

Resilience on all levels

Contrast this with failure handling in basic computer programming: a raised exception is propagated up the call stack by the runtime and delivered to the innermost enclosing exception handler that declares itself responsible. The usage hierarchy coincides with supervision collapse: the user of a service gets to handle its failures as well. In reactive design, every module of the hierarchy is a unit of resilience — enabled by the encapsulation of message passing and the flexibility of location transparency. A module can fail and be restored to proper function without its dependents needing to take action: the supervisor, not the users of the module, handles this for everyone else’s benefit. The amount of work to be done during a restart depends on the fraction of the application that has failed — so isolate failure as early as possible: small units mean small, low-cost recovery. This is a kind of architectural pattern, applicable and reusable across different applications and application domains.

SUPERVISION: OWNERSHIP MEANS COMMITMENT Supervisor monitors · restarts · escalates Worker A crashes → restarted Worker B crashes → restarted Worker C crashes → restarted own supervisor escalation only when restart fails
Plate 11.4 — Principled failure handling. The supervisor owns the lifecycle of its workers: it creates them, monitors their health, restarts them on failure, and only escalates to its own supervisor when restart does not work. Failure is compartmentalised at the earliest possible level.

10. Delimited consistency

Strong consistency is not feasible to preserve once a system’s scale grows to a critical size: the cost of coordinating a single global order of all changes within it is high, and adding more (distributed) resources can diminish the system’s capacity instead of increasing it. The solution is to construct systems from small building blocks that are internally consistent but interact in an eventually consistent fashion — preserving atomicity, but giving up complete consistency and isolation at the system level.

The design guidelines:

Distributed entities are characterized by their ability to fail independently. So the main concern of grouping data and behaviour according to transactional boundaries is to ensure that everything that must be consistent is not distributed: a consistent unit must not fail partially — if one part of it fails, then the entire unit must fail.

Exam angle

Unit of failure = unit of consistency. This is the same principle that Chapter 9’s aggregate rules and Chapter 10’s saga pattern express from the DDD side: strong consistency lives inside one unit; between units, consistency is eventual and is coordinated by messages. The reference is Pat Helland’s “Life Beyond Distributed Transactions” (CIDR 2007).

DELIMITED CONSISTENCY Unit A internally consistent transactional boundary unit of failure = unit of consistency if one part fails, the whole unit fails Unit B internally consistent transactional boundary unit of failure = unit of consistency if one part fails, the whole unit fails messages / events between units: eventual consistency, no global order, no distributed transaction everything that must be consistent is not distributed Helland, “Life Beyond Distributed Transactions”, CIDR 2007
Plate 11.5 — Delimited consistency. Units of strong consistency interact through messages and are eventually consistent with one another; a consistent unit must not fail partially.

11. Design criteria for modules — sum-up

The module closes where it opened, with the six design criteria for modules:

  1. A module does one job and does it well.
  2. The scope of a module is bounded by the responsibility of its parent.
  3. Module boundaries define the possible granularity of horizontal scaling by replication.
  4. Modules encapsulate failure, and their hierarchy defines supervision.
  5. The lifecycle of a module is bounded by that of its parent.
  6. Module boundaries coincide with transaction boundaries.
Editor’s note

Read the six criteria against Chapter 9 and you see the same skeleton twice: microservices as encapsulated modules with bounded lifecycles and supervision (deployment, health checks, restarts), and aggregates as transaction boundaries — the unit of consistency inside a service. Reactive architecture generalises the microservice discipline into a full metamodel.

Check your understanding

Define reactive systems and contrast them with transformational systems.

Reactive systems are a broad category of systems designed to respond to external stimuli or events, typically involving requirements about time: they react promptly to changes in their environment. Per Harel and Pnueli [HP89], transformational systems accept input, perform transformations and produce outputs (possibly asking for more input along the way); reactive systems are repeatedly prompted by the outside world and continuously respond, maintaining an ongoing relationship with their environment.

List the five typical characteristics of reactive systems.

Event-driven (responding to events or signals from the environment), concurrency (independent concurrent components handling multiple events), statefulness (internal state evolving with events), asynchrony (handling events independently of the main program flow, aiding responsiveness and scalability), fault tolerance (handling errors and faults gracefully).

State the four properties of the Reactive Manifesto and the value/form/means reading.

The system must react to its users (responsive), react to failure (resilient), react to variable load (elastic), and react to inputs (message-driven). Reading: responsiveness, resilience and elasticity are the value; their form is a message-driven architecture; the means is asynchronous message passing.

State the main decomposition principle of reactive architecture and the six rules for modularization.

Decompose the overall business problem hierarchically into fully encapsulated modules communicating only by asynchronous, nonblocking, location-transparent message passing. Rules: (1) a module does one job and does it well; (2) its responsibility is bounded by its parent’s; (3) module boundaries define the granularity of horizontal scaling by replication; (4) modules encapsulate failure and their hierarchy defines supervision; (5) the lifecycle of a module is bounded by that of its parent; (6) module boundaries coincide with transaction boundaries.

What does hierarchical problem decomposition look like, and what is the role of flow control?

Break the overall task into manageable units and define a hierarchy among them: implementation details at the bottom, components becoming more and more abstract toward the high-level functionality at the top; modules can have dependencies, defining dependencies and descendants. Flow control adjusts the transmission rate of a message stream so the receiver is not overwhelmed; when the sender is told to slow down it experiences back pressure; queueing gives message passing a wider range of flow-control options than synchronous calls.

Define location transparency and contrast it with transparent remoting.

Location transparency: the source code for sending a message looks the same regardless of where the recipient processes it; a sendable object becomes a mobile handle to the recipient, passable among network nodes. Transparent remoting (CORBA, RMI, DCOM) instead tried to make remote method invocation look like local invocation and failed: partial failure cannot be abstracted away, and added latency has systemic effects. Location transparency unifies the expression of message passing for local and remote interaction (actors: ActorRefs in Akka, process IDs in Erlang), optimising local passing as a special case.

How does message passing enable vertical and horizontal scalability?

Message passing decouples caller and callee into sender and receiver, enabling vertical scalability: the receiver is free to use different processing resources than the sender, since they do not share a call stack. Location transparency adds horizontal scalability: the receiver can be placed on any reachable computer without the sender knowing where, so performance improves by adding machines.

Explain “ownership means commitment” and “resilience on all levels.”

A supervisor owns the health of its descendant modules: it monitors them and starts new ones on failure, escalating to its own supervisor only when restart does not work; failure is an expected fact of life, not swept under the carpet. Ownership also implies lifecycle control: the supervisor creates its submodules and they cannot exist without it. Because every module of the hierarchy is a unit of resilience, a module can fail and be restored without its dependents taking action — unlike exception propagation, where the usage hierarchy collapses into supervision and the user of a service handles its failures. Isolate failure as early as possible: small units mean small, low-cost recovery.

What is delimited consistency, and why is strong consistency infeasible at scale?

Strong consistency is not feasible once scale grows to a critical size: coordinating a single global order of all changes is costly, and adding distributed resources can diminish capacity instead of increasing it. The solution: build systems from small building blocks that are internally consistent but interact in an eventually consistent fashion — preserve atomicity, give up complete consistency and isolation. Guidelines: group data and behaviour according to transaction boundaries; encapsulated modules are units of strong consistency and transaction bounds. Since distributed entities fail independently, everything that must be consistent must not be distributed: unit of failure = unit of consistency.

How do the six design criteria connect to microservices and DDD?

Microservices as encapsulated modules: bounded lifecycles, supervision (health checks, restarts), horizontal scaling by replication, failure encapsulated behind boundaries. Aggregates as transaction boundaries: module boundaries coincide with transaction boundaries, and the unit of failure equals the unit of consistency — the same principle the saga pattern applies between services.