Part II — Hands-on networking · Chapter 5

The programmable data plane

~35 min read4 interactive widgets4 plates

In this chapter

  1. What the data plane is
  2. The functions that live in the data plane
  3. Access control lists and TCAMs
  4. Buffering, marking and scheduling
  5. Where OpenFlow stopped being enough
  6. The hardware caught up: PISA switch ASICs
  7. The three goals of a programmable interface
  8. Top-down versus bottom-up design
  9. Check your understanding

1. What the data plane is

The P4 introduction, delivered by Andrea Melis, opens with a definition of the object the whole rest of the course programs. The data plane is about processing packet streams, and every word carries weight: large volume, packets arriving in streams, algorithms that process them. The consequence is a hard constraint:

That is the whole tension of the field in one slide: the data plane must do useful, increasingly sophisticated things, and it must do them in the few nanoseconds a packet spends in the device. Everything in chapters 5 to 8 is about widening what "useful things" can mean without losing the "few nanoseconds".

Key idea

The data plane is defined by a budget, not by a feature list: constant, tiny work per packet. This is why the P4 language later forbids loops, recursion, floating point and dynamic memory (chapter 6) — not out of minimalism, but because anything whose cost is not bounded per byte of header cannot run at line rate.

2. The functions that live in the data plane

The data plane is a bunch of different functionality, and the slides enumerate it. This list is worth memorising because it is the catalogue of everything a programmable data plane might be asked to become:

FunctionWhere you meet it in this course
Packet forwarding (switch)The P4 basic router, chapters 6 and 7
Access control (firewall)ACLs below; iptables/nftables in chapter 14
TunnelingThe P4 myTunnel exercise, chapter 7; GRE/VXLAN, chapter 4
Traffic monitoringP4 counters and INT, chapter 7; monitoring, chapter 14
Buffering and markingECN marking, chapter 7
Shaping and schedulingP4 meters, chapter 7
Deep packet inspection (DPI box)Modbus parsing in P4, chapters 10 and 11

Historically each of these was a separate box — the plethora of middleboxes from chapter 2. The claim of the programmable data plane is that they are all the same kind of thing: parse some headers, match some bitfields, apply some actions. If that is true, then they can all be programs for one kind of device, and the physical boxes collapse into one programmable box.

3. Access control lists and TCAMs

Access control is the archetype, so the slides develop it. Packet filtering is done with Access Control Lists (ACLs) that match on source and destination IP address, source and destination ports, and protocol ID. Beyond stateless filtering there are stateful operations — also for security, for example blocking all TCP SYN packets from outside — which require parsing TCP headers and maintaining per-flow state.

The structure of an ACL is exactly the structure you saw in the OpenFlow flow table of chapter 3, and it is worth naming the three properties precisely:

The interesting engineering detail is how this is done fast. The slides mention multi-dimensional classification algorithms, and then the hardware answer: TCAMs — ternary content addressable memory. A TCAM is what makes wildcard matching at line rate possible; it is the same memory technology that OpenFlow relied on, and the reason both OpenFlow and P4 speak in terms of exact, ternary and longest-prefix matches (chapter 6).

Network Address Translation

NAT is developed as a second stateful example: a mapping between internal and external addresses, at the level of IP addresses (between end-hosts and the NAT) and ports (so each connection is unique). The NAT table entries are created dynamically, which immediately raises the two hard questions the slides pose and leave open: when to remove entries? and what if both ends are behind NAT? These are exactly the questions Netfilter's connection tracking answers in chapter 14.

Try the priority-and-wildcard mechanics directly on an ACL below.

4. Buffering, marking and scheduling

The slides devote real space to queue management, because it is where the data plane stops being about single packets and starts being about the interaction between packets. Four topics appear, and they progress from crude to subtle:

FIFO / Drop Tail: packets served in arrival order; if the queue is full, the arriving packet is dropped. Random Early Detection (RED): drop earlier as a function of buffer size, or mark to signal congestion to end hosts, and handle different traffic classes differently. Active Queue Management (AQM): the queue autotunes to a latency target — CoDel, PIE, FqCoDel, packet-value-based dropping.

Mark a packet to signal something downstream or to the end hosts. ECN — Explicit Congestion Notification reuses IP header fields (the Type of Service bits) to carry buffer state. Two hard questions the slides raise: for end-host marking, how can the network trust the endpoints?; for network marking, how can the network infer application requirements? Both are answered pragmatically by classifying flows on a five-tuple.

Scheduling determines the serving order when there are multiple queues. Strict priority: always serve the highest-priority non-empty queue first. Round robin: cycle through the queues. Weighted fair scheduling: assign weights and serve proportionally. Each is a different fairness policy, and each is a few lines of logic — the kind of logic a programmable data plane can express.

Multiple traffic classes get a separate FIFO queue each — for each flow or class (voice, video) — and a scheduler decides the serving order. Traffic classes are identified using a flow specification based on the five-tuple, with rate limitations distinguishing what conforms to the profile from what is out of profile.

Editor's note

ECN reappears as a concrete P4 exercise in chapter 7: the switch compares the enqueue queue depth against a threshold and, only for packets whose end host opted in, sets the ECN bits to congestion-encountered. What looks like a page of theory here becomes a dozen lines of P4 there — which is exactly the point of a programmable data plane.

5. Where OpenFlow stopped being enough

Chapter 3 left OpenFlow at a cliffhanger: 12 header fields in 2009, 41 by 2013, and still not enough. The P4 introduction picks up exactly there and states what future SDN switches must therefore have:

The slides are honest about why this did not happen sooner. Programmable switches always existed, but they had four fatal problems: hard to program, lack of a standard, lack of a common interface, and — bluntly — definitely not performant and definitely not supported. A configurable switch was a research toy, not a product.

6. The hardware caught up: PISA switch ASICs

Then the hardware changed. A new generation of switch ASICs appeared — the slides name Intel FlexPipe, Cisco Doppler and Tofino — that were finally performant. The abstract machine model behind them is PISA: the Protocol-Independent Switch Architecture, and it is the shape of every P4 target you will meet.

The PISA pipeline: a programmable parser, followed by a programmable match-action pipeline of several stages, followed by a programmable deparser Programmable Parser match +action match +action match +action programmable match-action pipeline (stages in series) Programmable Deparser Programmer declares the headers and their order in the packet defines the tables and the exact processing algorithm declares how the output packet looks on the wire Parse → match/modify/add/remove headers and metadata → deparse (serialise) back onto the wire. Every block marked "programmable" is what a P4 program defines.
Plate 5.1 — PISA in action. A packet is parsed into headers, the headers and intermediate results feed match-action stages that can modify/add/remove them, and the deparser serialises the result. The blue blocks are what you write.

PISA still has a catch, and the slides name it: even with these ASICs you still need a standard interface, because the vendor-specific interfaces are custom and low-level, akin to microcode programming. Fast hardware without a common language is the programmable switch of ten years earlier, just faster. That missing language is P4.

7. The three goals of a programmable interface

The slides state exactly three goals for the programmable interface, and these three goals are the design brief of the P4 language. Learn them as a triple:

GoalWhat it demands
1. Protocol independenceConfigure a packet parser; define a set of typed match+action tables. The switch is not born knowing Ethernet or IP — you tell it.
2. Target independenceProgram without knowledge of switch details; rely on the compiler to configure the target switch. The same program should run on an ASIC, an FPGA, an NPU or a CPU.
3. ReconfigurabilityChange parsing and processing in the field, without swapping hardware.
For the exam

Protocol independence, target independence, reconfigurability. If you can state these three and explain that P4 achieves protocol independence with a configurable parser and typed tables, target independence by leaning on the compiler, and reconfigurability by allowing a new program to be pushed to a running switch — you have the spine of the whole P4 half of the course.

8. Top-down versus bottom-up design

The p4.org slides frame the shift as a reversal of the direction of authority. The status quo was bottom-up design: the ASIC datasheet defines the rules — "this is how I know to process packets" — and the network demands have to fit whatever the fixed-function ASIC already does. OpenFlow was, in this framing, a compromise: any new header-field match required updates to the protocol specification and changes to the runtime at the switch.

P4 proposes top-down design: the user or controller makes the rules — "this is how I want the network to behave and how to switch packets" — and a compiler configures the target below. The datasheet no longer dictates; it is a compilation target.

The benefits the slides list follow directly from that reversal, and the closing slogan is the one to remember: think programming rather than protocols.

BenefitBecause the data plane is now software
New featuresAdd new protocols.
Reduced complexityRemove unused protocols.
Efficient use of resourcesFlexible use of tables.
Greater visibilityNew diagnostic techniques, telemetry.
Software-style developmentRapid design cycle, fast innovation, fix data-plane bugs in the field.
You keep your own ideasThe behaviour is yours, not the vendor's.

The list of what people have actually built with P4 is the proof that this is not theoretical: a Layer 4 load balancer (SilkRoad), low-latency congestion control (NDP), In-band Network Telemetry (INT), in-network caching and coordination (NetCache, NetChain), aggregation for MapReduce — and much more. The programmable devices that run them span the full range: PISA ASICs (Tofino, FlexPipe, Doppler, Cavium/Xpliant), NPUs (EZchip, Netronome), CPUs (Open vSwitch, eBPF, DPDK, VPP) and FPGAs (Xilinx, Altera).

Key idea

"Think programming rather than protocols" is not a slogan about convenience. In the bottom-up world a protocol is a treaty that everyone must ratify before anyone can use it; in the top-down world a protocol is a data structure you declare in your program. The cost of a new protocol drops from years to a recompile — which is the same argument as chapter 3's field-count table, now stated as a principle.

Check your understanding

How does the course define the data plane, and what constraint defines it?

Processing packet streams: large volume, packets in streams, algorithms processing them. The defining constraint is that it must be super fast — a small time to process each packet — matching bitfields and applying simple actions, at end hosts (NIC) and inside the network (router, switch, firewall). The budget is constant, tiny work per packet.

List the functions the slides place in the data plane.

Packet forwarding (switch), access control (firewall), tunneling, traffic monitoring, buffering and marking, shaping and scheduling, and deep packet inspection (DPI box). The claim of the programmable data plane is that these are all the same kind of thing — parse, match, act — and so can be programs for one device.

What are the three defining properties of an ACL, and what hardware makes it fast?

Accept/drop actions in an ordered list; wildcard rules that can overlap, resolved by priority; classification by matching header fields and taking the highest-priority match. The hardware is the TCAM (ternary content addressable memory), which makes wildcard matching possible at line rate.

What two open questions does NAT raise in the slides?

NAT maps internal to external addresses (IP and ports) with dynamically created table entries, which raises when to remove entries? and what if both ends are behind NAT? These are the questions connection tracking answers in chapter 14.

Compare Drop Tail, RED and AQM.

Drop Tail (FIFO): serve in arrival order, drop on a full queue. RED: drop earlier as a function of buffer size, or mark to signal congestion, and treat classes differently. AQM (CoDel, PIE, FqCoDel): the queue autotunes to a latency target. The progression is from crude tail-dropping to latency-aware self-tuning.

What is ECN and which two trust questions does marking raise?

Explicit Congestion Notification reuses IP header fields (Type of Service bits) to carry buffer state, marking packets to signal congestion instead of dropping them. The two questions: for end-host marking, how can the network trust the endpoints? and for network marking, how can the network infer application requirements? Both are addressed by classifying flows on the five-tuple.

What must a future SDN switch have that OpenFlow lacked?

A configurable packet parser (not tied to a specific header format); flexible match+action tables (multiple, in series and/or parallel, matching on all defined fields); and general packet-processing primitives (copy, add, remove, modify header fields and metadata).

Why did programmable switches not succeed earlier, and what changed?

Earlier programmable switches were hard to program, lacked a standard and a common interface, and were not performant or supported. What changed was a new generation of switch ASICs — Intel FlexPipe, Cisco Doppler, Tofino — that were finally fast, built on the PISA architecture. But they still needed a standard interface, because the vendor interfaces were custom and low-level, akin to microcode.

Describe the PISA pipeline.

A programmable parser (the programmer declares the headers and their order), a programmable match-action pipeline of several stages in series (the programmer defines the tables and the exact processing algorithm, which can match, modify, add or remove headers and metadata), and a programmable deparser (the programmer declares how the output packet looks on the wire). Then the packet is serialised back out.

State the three goals of a programmable interface.

Protocol independence (configure a parser, define typed match+action tables), target independence (program without switch details, rely on the compiler), and reconfigurability (change parsing and processing in the field). These three goals are the design brief of P4.

Contrast bottom-up and top-down design.

Bottom-up: the ASIC datasheet defines the rules ("this is how I know to process packets"); network demands must fit fixed-function silicon, and OpenFlow was a compromise where each new match needed a spec update and a runtime change. Top-down: the user/controller defines the behaviour ("this is how I want the network to behave"), and the compiler configures the target. The slogan is think programming rather than protocols.

Name some things built with P4 and the kinds of devices that run it.

Built: a Layer 4 load balancer (SilkRoad), low-latency congestion control (NDP), In-band Network Telemetry (INT), in-network caching/coordination (NetCache, NetChain), MapReduce aggregation. Devices: PISA ASICs (Tofino, FlexPipe, Doppler, Cavium), NPUs (EZchip, Netronome), CPUs (Open vSwitch, eBPF, DPDK, VPP), and FPGAs (Xilinx, Altera).