ONOS and P4Runtime: controlling a P4 data plane

1. The big picture: from a P4 idea to an app

The ONOS+P4 tutorial (materials from the Open Networking Foundation) sets a concrete goal: acquire enough knowledge to build full-stack network applications — to go from a P4 idea to an end-to-end solution. The full stack is four layers, and the whole chapter is about the interfaces between them:

The bottom is familiar: a P4 pipeline is a pipeline of match-action tables (chapter 6), realised on an ASIC, FPGA, NPU or CPU. What is new here is everything above the switch.

2. Why yet another data-plane control API

The tutorial asks the obvious question — can't we reuse OpenFlow or SAI? — and answers it with a table that is the clearest single justification for P4Runtime. Three kinds of independence are wanted, and only P4Runtime has all three:

The three independences read as: the same API works with different switches from different vendors (target); the same API controls any data-plane protocol, standard or custom (protocol); the same API controls many arbitrary pipelines, formally specified (pipeline). The properties of P4Runtime that deliver this: it is designed around the PSA architecture but extensible to others; it is community-developed (initial contribution by Google and Barefoot); it is gRPC/protobuf-based; it is P4 program-independent — the API does not change with the P4 program — and it enables field reconfigurability, the ability to push a new P4 program and re-configure the pipeline without recompiling the switch software stack.

3. protobuf and gRPC in one page

P4Runtime is built on two Google technologies, and the tutorial teaches just enough of each.

syntax = "proto3";
message Person {
  string name = 1;
  int32  id   = 2;
  string email = 3;
}

The P4Runtime service itself, p4runtime.proto, is a gRPC service that lets a local or remote control plane load the pipeline, read and write pipeline state, exchange packets, and arbitrate mastership for distributed control planes:

service P4Runtime {
  rpc Write(WriteRequest) returns (WriteResponse) {}
  rpc Read(ReadRequest) returns (stream ReadResponse) {}
  rpc SetForwardingPipelineConfig(...) returns (...) {}
  rpc GetForwardingPipelineConfig(...) returns (...) {}
  rpc StreamChannel(stream StreamMessageRequest)
      returns (stream StreamMessageResponse) {}
}

4. P4Runtime: the write request and the table entry

Installing a rule means sending a WriteRequest containing one or more Updates, each an INSERT, MODIFY or DELETE of an Entity. The entity can be many things — a table entry, an action-profile member or group, a meter entry, a counter entry, a register entry, a packet-replication-engine entry — which is the API being pipeline-independent in one message definition.

message Update {
  enum Type { UNSPECIFIED = 0; INSERT = 1; MODIFY = 2; DELETE = 3; }
  Type   type = 1;
  Entity entity = 2;
}

To add a table entry the control plane must know three kinds of thing, and this list is the reason P4Info (next section) has to exist:

• The IDs of the P4 entities — tables, field matches, actions, parameters.
• The field matches for the particular table — match type, bit width.
• The parameters for the particular action.

A logical table entry such as hdr.ipv4.dstAddr = 10.0.1.1/32 -> ipv4_forward(00:00:00:00:00:10, 7) becomes a protobuf TableEntry whose fields are all numeric IDs and raw bytes:

table_entry {
  table_id: 33581985
  match {
    field_id: 1
    lpm { value: "\n\000\001\001"  prefix_len: 32 }
  }
  action {
    action_id: 16786453
    params { param_id: 1  value: "\000\000\000\000\000\n" }
    params { param_id: 2  value: "\000\007" }
  }
}

5. P4Info: the contract between the planes

Notice that the TableEntry above contained no names, only numbers: table_id: 33581985, action_id: 16786453. Something has to translate the human names in the P4 program into those IDs, and that something is P4Info.

The P4 compiler generates two outputs, and this pairing is the heart of the whole architecture:

The P4Info entry for the ipv4_forward action and the ipv4_lpm table records exactly the names, IDs, bit widths and match types the control plane needs:

actions {
  id: 16786453
  name: "ipv4_forward"
  params { id: 1  name: "dstAddr"  bitwidth: 48 }
  params { id: 2  name: "port"     bitwidth: 9  }
}
tables {
  id: 33581985
  name: "ipv4_lpm"
  match_fields { id: 1  name: "hdr.ipv4.dstAddr"  bitwidth: 32  match_type: LPM }
  action_ref_id: 16786453
}

6. SetForwardingPipelineConfig

Pushing a whole P4 program to a switch — the field reconfigurability that distinguishes P4Runtime — is done with SetForwardingPipelineConfig. The request bundles the P4Info and the target-specific configuration bits into one ForwardingPipelineConfig, and an Action field controls the commit semantics (VERIFY, VERIFY_AND_SAVE, VERIFY_AND_COMMIT, COMMIT, RECONCILE_AND_COMMIT).

message ForwardingPipelineConfig {
  config.P4Info p4info = 1;
  bytes p4_device_config = 2;   // target-specific P4 configuration
}

The tutorial's punchline is silicon-independent remote control: the same target-independent protobuf table entry, sent over gRPC, reaches a programmable Tofino from vendor A, a fixed-function switch from vendor B and another from vendor C — each with its own P4Runtime server and target driver, each carrying its own P4Info. One controller, many silicons, one message format.

7. ONOS: a network operating system

ONOS — the Open Network Operating System — provides the control plane for a software-defined network: a logically centralised remote controller, with APIs that make it easy to write apps to control a network. It focuses on service-provider access/edge applications, and it runs as a distributed system across many servers for scalability, high availability and performance. It is open source (created by ON.Lab, hosted by the Linux Foundation). P4Runtime support first arrived in the Loon (1.11.0) release, August 2017.

Its architecture is layered, and the layering is what lets one app control OpenFlow and P4Runtime devices alike:

ONOS interacts with devices through behaviors — Java interfaces — bundled into drivers. Named behaviors include DeviceDescriptionDiscovery (read device info and ports), FlowRuleProgrammable (write/read flow rules), PortStatisticsDiscovery (port packet/byte counters) and Pipeliner (FlowObjective-to-FlowRules mapping). A driver is a collection of behavior implementations that use ONOS's protocol libraries to reach the device — and the key takeaway is that apps are independent of the switch control protocol: the same app can work with OpenFlow and P4Runtime devices.

8. FlowRule versus FlowObjective

ONOS offers two network-programming APIs, and choosing between them is choosing between control and portability:

The FlowObjective API is what enables application portability: an app using FlowObjectives can work with switches whose pipelines differ. The mechanism is a three-phase translation, and it is worth knowing as three concrete rewrites:

1. Flow Objective → Flow Rule. One objective maps to many flow rules (the Pipeliner).
2. Flow Rule → Table Entry. Standard headers/actions are mapped to the P4-defined ones — for example ETH_DST → hdr.ethernet.dst_addr (the Pipeline Interpreter).
3. Table Entry → P4Runtime message. P4 names are mapped to P4Info numeric IDs.

9. Pipeconf: bring your own pipeline

How does ONOS learn to control an arbitrary P4 pipeline? Through a pipeconf: a package that bundles everything ONOS needs to understand, control and deploy a pipeline, provided to ONOS as an app (distributable as an .oar binary). A pipeconf has three parts:

The workflow recap ties the whole chapter together: write the P4 program and compile it (obtaining P4Info and target binaries); create the pipeconf (implement the pipeliner and interpreter behaviors in Java); then either use existing pipeline-agnostic apps that program the network with FlowObjectives, or write new pipeline-aware apps that use the same string names of tables, headers and actions as in the P4 program.

A note of realism from the ONOS 1.14 (Owl) support matrix: not everything was supported. Table entries, packet-in/out, action profiles, meters (indirect) and counters mapped to ONOS services, but parser value sets, registers, digests and clone sessions were listed as unsupported, "community help needed" — a useful reminder that the stack is real software with real gaps.

10. A worked controller: DDoS pre-filtering and digests

The CCNC tutorial guide walks through a concrete P4Runtime controller application (a Go P4Runtime client) that combines the data plane and a machine-learning back end. It is the clearest demonstration in the course of why you would push work into the data plane at all.

Exercise 1: DDoS detection with and without data-plane pre-filtering

The setup runs a data plane, a controller, and a CNN (a convolutional neural network detector, "lucid"), then launches an attack from the CIC2017 dataset — which contains a single malicious flow, attacker 172.16.0.1, victim 192.168.10.50. The result is inspected at localhost:3333/ddos.

Exercise 2: digests

The second exercise (simple_digest.p4) sends aggregated data from the data plane to the control plane: the details of 1 out of every 10 packets are sent to the controller — here just the source IP and port and the destination IP and port, though more could be included. The guide names it precisely: this is the building component that makes the interaction between the data plane and the CNN possible. A digest is P4's answer to chapter 6's limitation "pure P4 cannot communicate to the control plane" — an external object, a learning provider, that samples and reports.

Check your understanding