![TRANSFORMING NETWORKING & STORAGE
2
Technology Disclaimer:
Intel technologies’ features and benefits depend on system configuration and may require enabled
hardware, software or service activation. Performance varies depending on system configuration. No
computer system can be absolutely secure. Check with your system manufacturer or retailer or learn
more at [intel.com].
Performance Disclaimers (include only the relevant ones):
Cost reduction scenarios described are intended as examples of how a given Intel- based product, in the
specified circumstances and configurations, may affect future costs and provide cost savings.
Circumstances will vary. Intel does not guarantee any costs or cost reduction.
Results have been estimated or simulated using internal Intel analysis or architecture simulation or
modeling, and provided to you for informational purposes. Any differences in your system hardware,
software or configuration may affect your actual performance.
General Disclaimer:
© Copyright 2017 Intel Corporation. All rights reserved. Intel, the Intel logo, Intel Inside, the Intel Inside
logo, Intel. Experience What’s Inside are trademarks of Intel. Corporation in the U.S. and/or other
countries. *Other names and brands may be claimed as the property of others.
Legal Disclaimer](https://image.slidesharecdn.com/tldkpresentationmar1st2017m-170307181101/85/DPDK-Layer-4-Packet-Processing-2-320.jpg)
![What does FE config file configure?
Front
End
Core
Config
file
lcore=3,
op=fwd,
laddr=0.0.0.0,
lport=11211,
raddr=0.0.0.0,
rport=0,
fwladdr=::,
fwlport=0,
fwraddr=2001:4860:b002::56,
fwrport=11211
Layer
4
1) Read [Server]
2) Write [Client]
3) Forward [Gateway]](https://image.slidesharecdn.com/tldkpresentationmar1st2017m-170307181101/85/DPDK-Layer-4-Packet-Processing-16-320.jpg)
![FE config record format:
------------------------
lcore=<uint>,op=<"rx|tx|echo|fwd">,
laddr=<ip>,lport=<uint16>,raddr=<ip>,rport=<uint16>,
[txlen=<uint>,fwladdr=<ip>,fwlport=<uint16>,fwraddr=<ip>,fwrport
=<uint16>,
belcore=<uint>]
lcore - EAL lcore to manage that stream(s) in the FE.
op - operation to perform on that stream:
"rx" - do receive only on that stream.
"tx" - do send only on that stream.
"echo" - mimic recvfrom(..., &addr);sendto(..., &addr);
on that stream.
"fwd" - forward packets between streams.
laddr - local address for the stream to open.
lport - local port for the stream to open.
raddr - remote address for the stream to open.
rport - remote port for the stream to open.
txlen - data length to send with each packet ("tx" mode only).
fwladdr - local address for the forwarding stream(s) to open
("fwd mode only).
fwlport - local port for the forwarding stream(s) to open
("fwd mode only).
fwraddr - remote address for the forwarding stream(s) to open
("fwd mode only).
fwrport - remote port for the forwarding stream(s) to open
("fwd mode only).
belcore - EAL lcore to manage that stream(s) in the BE.](https://image.slidesharecdn.com/tldkpresentationmar1st2017m-170307181101/85/DPDK-Layer-4-Packet-Processing-17-320.jpg)
![STOP! OBSERVE the tle_dring structure
WHAT DO YOU SEE?
Producer / Consumer pointers with Head and Tail Pointers.
WHAT VALUE YOU SEE for Head and Tail Pointers?
**********************
continue
continue
****
STOP: Now what value you see for head and tail pointers
******************
continue
# type continue till you see [Inferior 1 (process <> exited normally]
quit
# ********** TEST 2 ****************************************
# Now run the same dring test without gdb
# *********************************************************
./x86_64-native-linuxapp-gcc/app/test_dring
49
#**********************************************************
#************TEST 3 ************************************
# Shall we run the Layer 4 UDP Application? Echo mode?
#*******************************************************
cd $TLDK_ROOT
./02_run_UDP_Echo_Test.sh
# Check that all packet in output pcap file contain valid
udp checksum:](https://image.slidesharecdn.com/tldkpresentationmar1st2017m-170307181101/85/DPDK-Layer-4-Packet-Processing-49-320.jpg)
DPDK & Layer 4 Packet Processing
- 1. DPDK & Layer 4 Packet Processing TLDK – Transport Layer Development Kit M Jay Presentation March 1 2017
- 2. TRANSFORMING NETWORKING & STORAGE 2 Technology Disclaimer: Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at [intel.com]. Performance Disclaimers (include only the relevant ones): Cost reduction scenarios described are intended as examples of how a given Intel- based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction. Results have been estimated or simulated using internal Intel analysis or architecture simulation or modeling, and provided to you for informational purposes. Any differences in your system hardware, software or configuration may affect your actual performance. General Disclaimer: © Copyright 2017 Intel Corporation. All rights reserved. Intel, the Intel logo, Intel Inside, the Intel Inside logo, Intel. Experience What’s Inside are trademarks of Intel. Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. Legal Disclaimer
- 3. What are the problems with tradi1onal TCP/UDP stacks?
- 4. One size “Kernel Stack” does not Fit All App needs “one size does not fit all” Single Kernel Stack App3 needs something else App1 needs High Throughput App2 needs Low Latency
- 5. What if you can have mul1ple “user level” stacks? App1 needs High Throughput App2 needs Low Latency App N needs something else “Context A – a.k.a Stack A – tuned for App 1 group needs” App1 needs High Throughput App2 needs Low Latency “Context B – a.k.a Stack B – tuned for App 2 group needs “Context C – a.k.a Stack C – tuned for App 2 group needs
- 6. Confused Travelers Tradi?onal TCP/UDP Stack Inefficient Cold Cache Cache Unfriendly skbuff One Kernel Stack “does not fit all” app needs Packet I/O in ones & twos? Many more pain points What are the problems with tradi1onal TCP/UDP stacks?
- 8. TLDK – Any Sample Applica1ons?
- 9. You know L2fwd, L3fwd • What about udpfwd?
- 10. The Eternal Loop of /tldk/examples/udpfwd/ main.c
- 12. What Are The Main APIs ?
- 13. 3 Key Modules 1) event.o 2) Udp_ctl.o 3) Udp_rxtx.o
- 14. The Main APIs
- 15. Front End Core + Back End Core Pulling UDP Stack Front End Core Config file Back End Core Config file tle_udp_dev Server Process Client Process Req Response TLE_ UDP_STREAM tle_udp_ctx UDP Local Port UDP Remote Port RX TX Layer 4 Layer 3 Layer 2 “The sockets” “Your” stacks
- 16. What does FE config file configure? Front End Core Config file lcore=3, op=fwd, laddr=0.0.0.0, lport=11211, raddr=0.0.0.0, rport=0, fwladdr=::, fwlport=0, fwraddr=2001:4860:b002::56, fwrport=11211 Layer 4 1) Read [Server] 2) Write [Client] 3) Forward [Gateway]
- 17. FE config record format: ------------------------ lcore=<uint>,op=<"rx|tx|echo|fwd">, laddr=<ip>,lport=<uint16>,raddr=<ip>,rport=<uint16>, [txlen=<uint>,fwladdr=<ip>,fwlport=<uint16>,fwraddr=<ip>,fwrport =<uint16>, belcore=<uint>] lcore - EAL lcore to manage that stream(s) in the FE. op - operation to perform on that stream: "rx" - do receive only on that stream. "tx" - do send only on that stream. "echo" - mimic recvfrom(..., &addr);sendto(..., &addr); on that stream. "fwd" - forward packets between streams. laddr - local address for the stream to open. lport - local port for the stream to open. raddr - remote address for the stream to open. rport - remote port for the stream to open. txlen - data length to send with each packet ("tx" mode only). fwladdr - local address for the forwarding stream(s) to open ("fwd mode only). fwlport - local port for the forwarding stream(s) to open ("fwd mode only). fwraddr - remote address for the forwarding stream(s) to open ("fwd mode only). fwrport - remote port for the forwarding stream(s) to open ("fwd mode only). belcore - EAL lcore to manage that stream(s) in the BE.
- 18. What does BE config file configure? Back End Core Config file # # udpfwd BE config file example # port=0, masklen=16, addr=192.168.0.0, mac=01:de:ad:be:ef:01 port=0, addr=2001:4860:b002::, masklen=64, mac=01:de:ad:be:ef:01 Layer 3 Layer 2
- 19. In addi1on, can I do command line config. as well?
- 20. Yes ! Command line Op1ons configuring As an example: udpfwd --lcores='3,6,8' -w 01:00.0 -- --promisc --rbufs 0x1000 --sbufs 0x1000 --streams 0x100 --fecfg ./fe.cfg --becfg ./be.cfg port=0, lcore=6, lcore=8, rx_offload=0xf, tx_offload=0, ipv4=192.168.1.233, ipv6=2001:4860:b002::28 Will create TLDK UDP context on lcore=6 and lcore=8 (BE lcore) to manage DPDK port 0. Will assign IPv4 address 192.168.1.233 and IPv6 address 2001:4860:b002::28 to that port. The following supported by DPDK RX HW offloads: DEV_RX_OFFLOAD_VLAN_STRIP, DEV_RX_OFFLOAD_IPV4_CKSUM, DEV_RX_OFFLOAD_UDP_CKSUM, DEV_RX_OFFLOAD_ TCP_CKSUM will be enabled on that port. No HW TX offloads will be enabled.
- 21. Have you run udpfwd in echo mode? • Do you have output – we can look at?
- 22. Wireshark Output of the file created by udpfwd – echo mode
- 23. tle_udp_dev What is tle_udp_dev? What it contains?
- 26. Req Response Linked List of UDP_Streams Context RX DNS – 53 DHCP – 67 … TX What is TLE_UDP_STREAM?
- 29. Callback Model for RECEIVE Callback Model for TRANSMIT
- 30. Event Model for RX Event Model for TX
- 31. Can I use Callback for RX & Event no1fica1on for TX? • Vice versa?
- 32. Back Up
- 33. Event/Callback & RX/TX - Combina1ons No?fica?on Choice Event No?fy Set for Send Callback Set for Send No No?fica?on Set for Send Event No?fy set for Rcv For both receive & Send, you get LEVEL TRIGGERED noeficaeon You get LEVEL TRIGGERED For recv & EDGE TRIGGERED noeficaeon for Send Only Receive gets noefied – LEVEL TRIGGERED. No Noeficaeon for Send Callback set for Rcv You get EDGE TRIGGERED For recv & LEVEL TRIGGERED noeficaeon for Send For both receive & Send, you get EDGE TRIGGERED noeficaeon Only Receive gets noefied – EDGE TRIGGERED. No Noeficaeon for Send No No?fica?on set for Rcv Only Send gets noefied – LEVEL TRIGGERED. No Noeficaeon for Recv Only Send gets noefied – EDGE TRIGGERED. No Noeficaeon for Recv No Noeficaeon for both send & Receive
- 34. Key gdb commands ptype x Data Structure Type of stuct x b <funceon name> Set breakpoint clear <funceon name> Clear breakpoint @ funceon info locals gdb - - arg ./<funceon><args> Debugging run conenue
- 35. Key virsh & ssh commands sudo virsh domifaddr <Host Name> Sudo virsh list - - all ssh [email protected].... ip r add default via 192.168… dev ens8 Aper reboot of VMs, to resolve any connecevity issue
- 36. Key git & Misc commands git log Returns 40 bit IDs git checkout <40 bit ID> sed -i 's/r$//' filename Windows control characters issue
- 37. Key Objdump commands Objdump - - syms <file.o> To read the symbols Objdump –S <file.o> Disassembly mixed with source + Assembly Objdump –S <file.o> Objdump –h <file.o> To get details about file Txt, data, bss, stack Objdump -x <file.o> For all headers in detail
- 38. Install puTTY • Click the URL • hwp://the.earth.li/~sgtatham/puwy/0.60/x86/ • Double Click puwy-0.60-installer.exe opeon • You have downloaded on your Download folder • Right Click and Run As Administrator • You have installed PuTTY ! Next Step is To Configure PuTTY
- 39. PuTTY Config: " Session Screen 1 / 2 1) Host name 2) Port 22 3) Connec1on Type SSH 1st select this
- 40. Keep Alive – 30 sec; Enable TCP Keepalives 40
- 41. If you are outside 41
- 42. PuTTY Config: Connec1on -- > Proxy Screen 2 / 2 1) Proxy Type: None 2) Proxy host name: Leave it blank 3) Do DNS name lookup at proxy end: Auto ! Next Step is To Save The Configura?on
- 43. 1) Name The Configura1on. 2) Save It ! Click “Session” on the top led corner. ! It will take you to the 1st screen - shown here . ! In Saved Sessons Box, Enter a name, e.g., dpdk training cluster ! Press Save Bueon. ! From now onwards, you can load the saved session when star?ng PuTTY
- 44. Gejng To The HostVM-<m> through Cluster Jump Server
- 45. Start PuTTY • Find PuTTY that you just installed • Click on icon “puwy.exe”. You will get the PuTTY Config Screen • Select the Saved Session, shown here e.g., dpdk training cluster. • Press Load Buwon. Press Open Buwon. • You will get ssh session shown in 3rd screen asking username • Username: student<m> <m> is given to you. For example it may be student19 or student25 or any other student<m>. • Ask for your specific <m> and use only that. This will avoid overlapping with other teams’ <m>. • Password is same as username. For example, if your username is student19, then password is student19 • Repeat the above steps so that you have many conneceons to the jump server
- 46. How to login and Connect to HostVM-<m> • Login as student<m> password student<m> • The next step is to connect to HostVM-<m> assigned to you. • In case not given, Ask for your own <m> • Note: We are using 1:1 – same student name and same HostVM- name • Type in the following command • ssh HostVM-<m> • Username: password Password: password • sudo su – • cd /home/user/dpdk_hands_on • Vi commands.txt
- 47. Open 2 Windows ssh HostVM-<M> Password is password sudo su – cd / 47 ssh HostVM-<M> Password is password sudo su - cd / vi Cpy_PasteFromMe.txt B) NEXT, in the 2nd Window, enter these commandsA) FIRST, In the 1st Window, enter these commands 1) Only Highlight from the vi in this Window This is your “Highlight to Copy From Window” Don’t Click. i.e., Don’t Select. Only Highlight ./00_installTools.sh 1 2 3Right Click in this Window to Paste
- 48. Cpy_Paste From Me.txt #Cpy_PasteFromMe.txt sudo su - cd / ./00_ReserveHugePage.sh source 01_SetEnvironmentVariables.sh cd $RTE_SDK *************************************************** **** Please be patient *** The following commands will take time to start **************************************************** make config T=$RTE_TARGET make make install T=$RTE_TARGET cd $GTEST_DIR cmake CMakeLists.txt make cd $TLDK_ROOT make clean make all O=$RTE_TARGET 48 # *********** TEST 1 ************************************** # Shall we take a look at key data structures? # For that let us use gdb and run Dynamic Ring Buffer Tests # ********************************************************* # ********************************************************* gdb --arg ./x86_64-native-linuxapp-gcc/app/test_dring # Set BreakPoints b drb_dump b tle_dring_dump run #Examine dynamic ring ptype dr # Examine dynamic buffer ptype db continue **************************
- 49. STOP! OBSERVE the tle_dring structure WHAT DO YOU SEE? Producer / Consumer pointers with Head and Tail Pointers. WHAT VALUE YOU SEE for Head and Tail Pointers? ********************** continue continue **** STOP: Now what value you see for head and tail pointers ****************** continue # type continue till you see [Inferior 1 (process <> exited normally] quit # ********** TEST 2 **************************************** # Now run the same dring test without gdb # ********************************************************* ./x86_64-native-linuxapp-gcc/app/test_dring 49 #********************************************************** #************TEST 3 ************************************ # Shall we run the Layer 4 UDP Application? Echo mode? #******************************************************* cd $TLDK_ROOT ./02_run_UDP_Echo_Test.sh # Check that all packet in output pcap file contain valid udp checksum:
- 50. How do you “repair” the build problem with gtest?
- 51. How To Repair Build error? • unset GMOCK_DIR • unset GTEST_DIR • cd $TLDK_ROOT • cd make clean • Make all O=$RTE_TARGET • cd googletest_feb2017 • cmake CMakeLists.txt • Make • export GTEST_DIR=/googletest_feb2017/googletest • export GMOCK_DIR=/googletest_feb2017/googlemock • cd $TLDK_ROOT • make clean • make all O=$RTE_TARGET