![5
Motivations
u Design an Accelerated Data Plane BMAcc for P4:
u Not a P4 compiler (e.g. PISCES [1], P4ELTE [2]), a P4 target on multicore platforms.
u A substitute of BMv2 but with line rate performance.
u PerformanceAcceleration:
u Leverages DPDK libraries and PMD driver for faster packet I/O.
u Applies multiple optimization techniques: reduced memory copy, multithreading, SSE instr.
u Transparentto P4 Programs:
u Support all P4 programsand DPDK-compatible platforms.
u Not require P4 source code, only JSON config files.
[1] Shahbaz et al. 2016. PISCES: A Programmable, Protocol-Independent Software Switch. In SIGCOMM '16.
[2] Laki et al. 2016. High speed packet forwarding compiled from protocol independent data plane specifications.
In SIGCOMM '16.](https://image.slidesharecdn.com/10-171118035044/85/LF_DPDK17_Accelerating-P4-based-Dataplane-with-DPDK-5-320.jpg)
![7
Evaluation Setup
u 2 Intel Supermicro Servers [1] w/ 2 * 10 GbE NIC cards on each server.
u SM1:
u TX: pktgen - 10 Gbps trafficwith random dst IP
u RX: count the received packets
u SM2:
u BMv2 Simple Router target.
u Lookup table,forward/drop.
DPDK
pktgen
BMv2
Simple
Router
TX
RX
RX
TX Forward
Check rule
10 Gbps
SuperMicro Server 1 SuperMicro Server 2
[1] Intel Supermicro server 1U based on Intel® Xeon® processors D-1540 @ 2.0 GHz, Niantic 82599 10 GbE NIC.](https://image.slidesharecdn.com/10-171118035044/85/LF_DPDK17_Accelerating-P4-based-Dataplane-with-DPDK-7-320.jpg)
LF_DPDK17_Accelerating P4-based Dataplane with DPDK
- 1. DPDK Summit - San Jose – 2017 BMAcc: Accelerating P4- Based Data Plane with DPDK #DPDKSummit PEILONG LI *, XIAOBAN WU *, YAN LUO *, LIANG- MIN WANG +, MARC PEPIN +, ATUL KWATRA+, AND JOHN MORGAN + * UNIVERSITY OF MASSACHUSETTS LOWELL + INTEL CORPORATION
- 2. 2#DPDKSummit Agenda u Background of P4 and BMv2 u Problems and design motivations u Overview of BMAcc & performance optimizations u Performance evaluation u Future directions u Conclusion
- 3. 3 P4 Language and P4 Behavior Model v2 u Programming Protocol-Independent Packet Processors (P4) u Simple semantics, customizable headers & dataplane functions u P4 program à P4 Frontend Compiler à Python IR à Backend Compiler à Target (software switch, NPU, etc.) u P4 Behavior Model version 2 (BMv2) u BMv1 is deprecated: requires re-compilation for every P4 program. u BMv2 is a static executable: software switch consists of building blocks (parser, deparser, match-action tables, etc.) u Configured by JSON: P4 program à p4c-bm à JSON config à BMv2 (static)
- 4. 4 Problems of BMv2 u A great software switch to verify the “behavior” of a P4 program u Poor performance as a software switch u 99.993% packet drop rate with 64-byte packet on 10 Gbps link u Uses libpcap, Linux NIC driver, single-threaded, single RX queue (no RSS), unnecessary memory copy, etc. add port polling and receiving packets init_from_command_line_options push packets into in_buffer receive in_buffer Thread 0 take packets from in_buffer P4 pipeline push packets to out_bufferThread 1 pipeline out_buffer Thread 2 Take the packets from out_buffer and transmit transmit Thread 1 Thread 3 Receiving TransmittingProcessingInitialization
- 5. 5 Motivations u Design an Accelerated Data Plane BMAcc for P4: u Not a P4 compiler (e.g. PISCES [1], P4ELTE [2]), a P4 target on multicore platforms. u A substitute of BMv2 but with line rate performance. u PerformanceAcceleration: u Leverages DPDK libraries and PMD driver for faster packet I/O. u Applies multiple optimization techniques: reduced memory copy, multithreading, SSE instr. u Transparentto P4 Programs: u Support all P4 programsand DPDK-compatible platforms. u Not require P4 source code, only JSON config files. [1] Shahbaz et al. 2016. PISCES: A Programmable, Protocol-Independent Software Switch. In SIGCOMM '16. [2] Laki et al. 2016. High speed packet forwarding compiled from protocol independent data plane specifications. In SIGCOMM '16.
- 6. 6 Design Overview and Three Optimizations u The Design Overview u Three Optimizations u Opt 1: ① PCAP à DPDK; ② Linux driver à PMD; ③ Single thread à RSS multi-queue u Opt 2: ① rm redundantmem copy in Receive; ② rm MUTEX for each parsed header u Opt 3: ① P4 LPM à DPDK LPM; ② SSE instructionsused byDPDK. add port Notify the tasks for slave cores init_from_command_line_options Master: Thread 0 Recv packets from RX queues Pipeline Table Lookup with SSE Send packets to TX ring Slaves: Thread 1, 2, 3, ..., N Multithreaded tasks Detect PMD Devices Slave cores waiting for tasks DPDK EAL RX Queues ...... TX Ring ......
- 7. 7 Evaluation Setup u 2 Intel Supermicro Servers [1] w/ 2 * 10 GbE NIC cards on each server. u SM1: u TX: pktgen - 10 Gbps trafficwith random dst IP u RX: count the received packets u SM2: u BMv2 Simple Router target. u Lookup table,forward/drop. DPDK pktgen BMv2 Simple Router TX RX RX TX Forward Check rule 10 Gbps SuperMicro Server 1 SuperMicro Server 2 [1] Intel Supermicro server 1U based on Intel® Xeon® processors D-1540 @ 2.0 GHz, Niantic 82599 10 GbE NIC.
- 8. 8 Test 1: Hardware Performance Verification u NIC TX Capability: u TX end always sends 64*1024*1024 packets with 256*1024 distinct flows u 99.85% 99.70% 98.66% 98.00% 100.00% 1500 750 64 PacketReceiving Rate Packet Size (Byte) Packet Receiving Performance Test RX Rate u NIC RX Capability: u RX end onlyreceives packets and then drops. u No transmissionto the out-port. Packet Size (byte) Throughput Framing rate Duration 1500 9.8 Gbps 9.9 Gbps 79 Sec 750 9.7 Gbps 9.9 Gbps 41 Sec 64 7.7 Gbps 9.9 Gbps 4.7 Sec
- 9. 9 Test 2: Performance on a Single Core with Different Optimizations u Vanilla BMv2 only supports single thread. u Performance comparison: PCAP (vanilla) à Opt1 à Opt 1,2 à Opt 1,2,3 1. Almost all DPDK versions outperform PCAP 2. More opts à higher performance 3. Opt1 single core version: DPDK has no evident performance gain
- 10. 10 Test 3: Performance on 8 Cores with Different Optimizations u PCAP is not on the chart u Performance comparison: Opt 1 à Opt 1,2 à Opt 1,2,3 1. 3x, 5.5x, 23x increase over single core vanilla BMv2 for large, mid, and small packet sizes 2. Reach line rate for large and mid sized packets with 3 opts 3. How about 64-byte packet?
- 11. 11 Test 4: Find the Performance Killer for Small Packets u Five major stages in P4 Processing: u RX à Parser à LPM à Deparser à TX u Gradually add stages to this pipelineto find the biggest performance drop u In experiment: 4 Cores, 64-byte packet 1. Perf impact breakdown: TX: 20% Parser: 58% Deparser: 5% LPM: 9% 2. TX+RX à Similar to l3fwd (80% PRR as reported) 3. Parser – creates NEW objects for each packet à time consuming Similar to l3fwd 58% Drop
- 12. 12 Test 5: Performance with Various # of Cores u Take the Opt 1,2,3 case (the most optimized) 1. Large packet reaches line rate w/ 4 cores; mid packet w/ 8 cores 2. Performance is almost proportional to # of cores 3. Not shown here, but the results are consistent with Opt 1 and Opt 1-2.
- 13. 13 Test 6: The Performance of LPM Processing u P4 LPM: leverages Judy for creating and accessing dynamic arrays u DPDK LPM: SSE instructionsand cache friendly data structures 1. DPDK-LPM is slightly better for all cases 2. DPDK-LPM performance benefit is more evident when ruleset is smaller and processing cores are fewer because of the overhead of Judy library.
- 14. 14 Conclusion and Future Work u The DPDK-accelerated BMv2 reaches 10 Gbps line rate for mid & large- sized packets, and yields 23x performance boost on the small packets. u To address the Parser impact on 64-byte packet, we need to pre-allocate memory spaces for Packet instances u We proposed multiplepractical optimizationson the BMv2 which are instrumental to all P4-based data plane designs on multicore platforms. u We conductedin-depth performance study on the proposed BMAcc system from architecture and software perspectives.