DPDK layer for porting IPS-IDS
Download as PPTX, PDF•7 likes•2,292 views
The document discusses the porting of Intrusion Detection/Prevention Systems (IDS/IPS) applications to a DPDK (Data Plane Development Kit) platform, addressing packet pipeline bottlenecks, I/O PCIe slot configurations, and the capabilities of Suricata as a network threat detection engine. It highlights improvements in packet processing speeds through user space configurations and custom libraries, while also mentioning the challenges faced, such as virtual device hurdles and timer latency. Finally, the document presents statistical results from test simulations and emphasizes the need for performance optimization in network application processing.
![16
• populateNodeInfo() & displayNodeInfo()
INFO: DPDK Ver: DPDK 16.11.0 rte_eal_process_type: Primary!!
NODE: 0 -- PORT --
^^ 1G ports: 0x0 count: 0 // nodePtr->port1G_map[0]
^^ 10G ports: 0x7 count: 3 // nodePtr->port10G_map[0]
^^ 40G ports: 0x0 count: 0 // nodePtr->port40G_map[0]
NODE: 1 -- PORT --
^^ 1G ports: 0x0 count: 0// nodePtr->port1G_map[1]
^^ 10G ports: 0xf count: 4 // nodePtr->port10G_map[1]
^^ 40G ports: 0x0 count: 0 // nodePtr->port40G_map[1]
• port1G_init, port10G_init, port40G_init
• getCount1gPorts, getCount10gPorts, getCount40gPorts
Numa Wrapper - Coremask & PortMask per NUMA](https://image.slidesharecdn.com/portingapplicatonstodpdk-170410084923/85/DPDK-layer-for-porting-IPS-IDS-16-320.jpg)
DPDK layer for porting IPS-IDS
- 1. Porting IDS/IPS Applications to DPDK Platform
- 2. 2 Agenda • IDS/IPS Application Packet Pipeline • Explore into Bottlenecks Solutions I/O PCIe Slot-NUMA map CPU Custom Libraries Application Packet Filter, Lookup, Distribution and Modeling Ecosystem ViritI/O, Proc-Info, SIMD, Custom Lookup
- 3. Look into Suricata 3 Worker Thread RX NIC Capture Decode Stream Detect Output TX NIC Suricata is a free and open source, mature, fast and robust network threat detection engine. The Suricata engine is capable of real time intrusion detection (IDS), inline intrusion prevention (IPS), network security monitoring (NSM) and offline pcap processing. Suricata's multi-threaded architecture can support high performance multi-core and multiprocesser systems, Jonkman said." -- (Computerworld) Flow identification Stream Identification Stream Capture Buffers & Flows limit Copies Exact match Pattern match
- 4. 4 IDS-IPS in Passive & Active mode Network I/O (Multiple 10Gbit/s Interfaces) Control, Configuration and Stats (CLI and Socket interface) High Speed User Space TCP and SSL stack configured in proxy mode. Clear Text Encrypted Encrypted
- 5. Dive into Bottlenecks Do we need to re-invent the Intrusion Detection, Intrusion Prevention or Network Security Monitoring utility?
- 6. 6 SoC using PCIe virtual dev library Network I/O Config & Mgmt TCP SSL StackDPDK PMD & MBUF Manager User Space • Keep packet in User Space • Reduce latency between NIC to NIC • Smart Filter DPDK PMD library to rescue the I/O bottleneck
- 7. 7 0 100 200 300 400 500 600 700 800 900 1000 64 byte RX 64 byte TX 1500 byte RX 1500 byte TX 480 150 780 220 625 273 944 473 MBITS/SEC PACKET SIZE Packet NIC to NIC PCIe 1 queue SOC allowed up to 32 bi directional PCIe user space queues
- 8. Worker Threads Offloading DPDK worker threads for running RX-TX with Suricata workers (CPU & Application Model) 8
- 9. 9 Suricata using DPDK RX NIC TX NIC Capture Decode Stream Detect Output Worker Threads Capture Decode Stream Detect Output RSS HASH Parse for metadata Match for rule set Buffer & Zero Copy
- 10. 10 Improvement (No Pkt Process) DPDK AF-Workers DPDK AF-Workers Byte 64 Byte 1500 P1 RX 1000 499 1000 826 P1 TX 382 251 1000 416 P2 RX 1000 475 1000 825 P2 TX 382 213 1000 472 1000 499 1000 826 382 251 1000 416 1000 475 1000 825 382 213 1000 472 0 200 400 600 800 1000 1200 MBITS/SEC P1 RX P1 TX P2 RX P2 TX
- 11. Eco-System
- 12. 12 Setup Super micro 4 core Xeon at 2.6Ghz and onboard 2 * 1G i350 (2x PCIe Gen2) DPDK 1 core - 2 worker cores, 1 DPDK RX-TX. AF-Workers - 3 worker cores • Distributed lcore and NIC. ie: single socket interfaces single NIC (4 * 10G). • Single Machine for processing, filter, flow and Suricata. • Reduced packet latency, since there no inter NIC-NIC transmission. • Localized user DPDK and custom Suricata helps in zero copy. Learnings
- 13. Reality Check - All Done? Unexpected Follow on!!!
- 14. 14 Feed back 1. Works Partially 2. Worse Throughput EXPECTATION
- 15. 15 ./testSlot.py +++ DPDK NIC to Physical slot Mapping +++ ++++++++++++++++++++++++++++++++++ Bus: 04:00.2 Slot: 5 Node: 0 Driver: igb_uio Bus: 08:00.3 Slot: 1 Node: 0 Driver: igb_uio Bus: 08:00.0 Slot: 1 Node: 0 Driver: igb_uio Bus: 83:00.1 Slot: 2 Node: 1 Driver: igb_uio Bus: 85:00.1 Slot: 4 Node: 1 Driver: igb_uio Bus: 85:00.2 Slot: 4 Node: 1 Driver: igb_uio Bus: 85:00.3 Slot: 4 Node: 1 Driver: igb_uio ---------------------------------------------------- PCIe address: Numa Slot: Physical slot
- 16. 16 • populateNodeInfo() & displayNodeInfo() INFO: DPDK Ver: DPDK 16.11.0 rte_eal_process_type: Primary!! NODE: 0 -- PORT -- ^^ 1G ports: 0x0 count: 0 // nodePtr->port1G_map[0] ^^ 10G ports: 0x7 count: 3 // nodePtr->port10G_map[0] ^^ 40G ports: 0x0 count: 0 // nodePtr->port40G_map[0] NODE: 1 -- PORT -- ^^ 1G ports: 0x0 count: 0// nodePtr->port1G_map[1] ^^ 10G ports: 0xf count: 4 // nodePtr->port10G_map[1] ^^ 40G ports: 0x0 count: 0 // nodePtr->port40G_map[1] • port1G_init, port10G_init, port40G_init • getCount1gPorts, getCount10gPorts, getCount40gPorts Numa Wrapper - Coremask & PortMask per NUMA
- 17. 17 VirtIO Hurdles 1. Device start & stop not working 2. Link state set up & down fails 3. LSR call back does not work 4. Application proc-info does not shows stats for right primary application. 5. Application proc-info corrupts rte_dev_data when pcap in use
- 18. Timer Hurdles 18 1. Timers used in 1. Reassembly – IP 2. Protocols – TCP, Path Monitoring, ARP 3. Scheduling, Event expiry 4. Stats 2. Latency 1. tick to Software 2. Expiry 3. Application notification 4. Starting 5. Threshold tick values (demo)
- 19. 19 __attribute__((always_inline)) inline int avxChecksumV2(const char * const target, size_t targetLength) { unsigned int checksum = 0; size_t offset = 0; uint16_t *buff = NULL; __m256i vec, lVec, hVec, sum; if(targetLength >= 32) { for(; offset <= targetLength - 32; offset += 32) { vec = _mm256_loadu_si256((__m256i const *)(target + offset)); lVec = _mm256_unpacklo_epi16(vec, (__m256i) {(uint64_t) 0x0, (uint64_t) 0x0, (uint64_t) 0x0, (uint64_t) 0x0}); hVec = _mm256_unpackhi_epi16(vec, (__m256i) {(uint64_t) 0x0, (uint64_t) 0x0, (uint64_t) 0x0, (uint64_t) 0x0}); sum = _mm256_add_epi32(lVec, hVec); sum = _mm256_hadd_epi32(sum, sum); sum = _mm256_hadd_epi32(sum, sum); sum = _mm256_hadd_epi32(sum, sum); checksum += _mm256_extract_epi16(sum, 0) + _mm256_extract_epi16(sum, 15); } } if (targetLength - offset >= 2) { for(;(targetLength -offset) >= 2; offset+=2) checksum += (*(uint16_t *) ((uint8_t *) target + offset) ); offset -= 2; } if (targetLength - offset) checksum += *((uint8_t *) target + offset); checksum = ((checksum & 0xffff0000) >> 16) + (checksum & 0xffff); checksum = ((checksum & 0xffff0000) >> 16) + (checksum & 0xffff); return checksum; } Comparison: 0.468157 task-clock (msec) # 0.461 CPUs utilized ( +- 1.66% ) 0 context-switches # 0.000 K/sec 0 cpu-migrations # 0.000 K/sec 42 page-faults # 0.091 M/sec ( +- 0.58% ) 5,63,229 cycles # 1.203 GHz ( +- 1.82% ) <not supported> stalled-cycles-frontend <not supported> stalled-cycles-backend 4,04,106 instructions # 0.72 insns per cycle ( +- 0.43% ) 78,541 branches # 167.765 M/sec ( +- 0.50% ) 3,538 branch-misses # 4.50% of all branches ( +- 0.30% ) 0.446172 task-clock (msec) # 0.471 CPUs utilized ( +- 4.43% ) 0 context-switches # 0.000 K/sec 0 cpu-migrations # 0.000 K/sec 42 page-faults # 0.095 M/sec ( +- 0.47% ) 5,39,201 cycles # 1.209 GHz ( +- 3.08% ) <not supported> stalled-cycles-frontend <not supported> stalled-cycles-backend 4,03,597 instructions # 0.75 insns per cycle ( +- 0.64% ) 78,162 branches # 175.184 M/sec ( +- 0.53% ) 3,199 branch-misses # 4.09% of all branches ( +- 6.27% ) SIMD Checksum Vectorization isn’t something that will work for all programs, but if yours is data-intensive, perhaps running simulations, processing graphics, or repeated financial calculations, consider vectorization. It might only take a slight rewrite of your program’s data structures and layout to have the compiler auto-vectorize it.
- 21. 21 References • https://blog.selectel.com/introduction-dpdk-architecture-principles/ • https://github.com/vipinpv85?tab=repositories • http://blog.talosintelligence.com/2010/07/innovation-you-keep-using- that- word.html?utm_source=feedburner&utm_medium=feed&utm_campaig n=Feed:+Vrt+(Sourcefire+VRT+- +Vulnerability+Research,+Snort+Rules+and+Explosions • http://blog.talosintelligence.com/2010/06/single-threaded-data- processing.html
- 22. 22
- 23. PROC-INFO Enhancement for fetching Primary Application Port stats 23
- 24. Primary Port Details • Intf: 0 Speed: 10000 Duplex: Full Status: up • - driver:; - if_index: 0 • - driver: Pcap PMD; - if_index: 5 • - driver: net_virtio PMD; - if_index: 7 • -- ADDR - domain:bus:devid:function 0000:0000:06.0; == PCI ID - vendor:device:sub- vendor:sub-device 1af4:1000:1af4:0001 • - driver: net_virtio; - if_index: 0 • -- ADDR - domain:bus:devid:function 0000:0000:07.0; == PCI ID - vendor:device:sub- vendor:sub-device 1af4:1000:1af4:0001 • Intf: 0 Speed: 10000 Duplex: Full Status: up • - driver: net_virtio; - if_index: 0 • - driver: Pcap PM; - if_index: 5 • - driver: Pcap PMD; - if_index: 7 • -- ADDR - domain:bus:devid:function 0000:0000:06.0; == PCI ID - vendor:device:sub- vendor:sub-device 1af4:1000:1af4:0001 • - driver: net_virtio; - if_index: 0 • -- ADDR - domain:bus:devid:function 0000:0000:07.0; == PCI ID - vendor:device:sub- vendor:sub-device 1af4:1000:1af4:0001 24
- 25. 25 New proc-info stats• ######################## NIC statistics for port 0 ######################## • - name: eth_pcap0; - DPDK Port id: 05: veth_fp_adk1@veth_k_adk1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000; link/ether a2:dd:48:c2:65:33 brd ff:ff:ff:ff:ff:ff • ######################## NIC statistics for port 1 ######################## • - name: eth_pcap1; - DPDK Port id: 17: veth_fp_adk2@veth_k_adk2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000; link/ether 0e:42:c4:8b:bf:4c brd ff:ff:ff:ff:ff:ff • ######################## NIC statistics for port 2 ######################## • - name: 0000:00:06.0; - DPDK Port id: 2; - numa node: -1 mtu: 1500 dev_started: 1 promiscuous: 1; - dev_link: speed: 10000 duplex: 1 autoneg: 0 status: 1; - kdrv: 1 • - mac_addrs: 52:54:00:c3:1d:a8; - min_rx_buf_size: 2176; - all_multicast: 0 dev_flags: 1; - nb_rx_queues: 1 nb_tx_queues: 1 • ######################## NIC statistics for port 3 ######################## • - name: 0000:00:07.0; - DPDK Port id: 3; - numa node: -1 mtu: 1500 dev_started: 1 promiscuous: 1; - dev_link: speed: 10000 duplex: 1 autoneg: 0 status: 1; - kdrv: 1 • - mac_addrs: 52:54:00:b5:95:1b; - min_rx_buf_size: 2176; - all_multicast: 0 dev_flags: 1; - nb_rx_queues: 1 nb_tx_queues: 1
- 26. Custom Lookup 16pt Intel Clear Subhead 26
- 27. 27
- 28. 28 1600 950 625 8000 2400 2970 30000 12000 7500 0 5000 10000 15000 20000 25000 30000 35000 1024 2048 4096 CONNECTION/SEC KEY SIZE Linked List Array Hash Array