DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
24 likes11,227 views
The document discusses the innovative capabilities of BPF (Berkeley Packet Filter) and XDP (Express Data Path) for enhancing network and application security, particularly in cloud-native environments like microservices. It emphasizes the limitations of traditional L3/L4 networking policies and promotes Cilium as a solution that integrates BPF/XDP with Layer 7 policies for advanced security features. Ultimately, the presentation positions BPF/XDP as a pivotal technology for the future of software-defined networking on Linux.
![BPF: Toolchain – from user to kernel
USER SPACE
SOURCE CODE [C]
</>](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-36-320.jpg)
![BPF: LLVM compiles program code to bytecode
USER SPACE
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-37-320.jpg)
![BPF: Bytecode is loaded and verified into kernel
USER SPACE
KERNELVERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-38-320.jpg)
![BPF: Bytecode runs inside safe kernel sandbox
USER SPACE
KERNELVERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>
SANDBOX
BPF](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-39-320.jpg)
![BPF: Program is attached to event (packet-in)
USER SPACE
KERNELVERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>
SANDBOX
BPF](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-40-320.jpg)
![BPF: Program can redirect to netns & sockets
USER SPACE
KERNELVERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>
SANDBOX
BPF](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-41-320.jpg)
![BPF: Leverage user space tool chain
USER SPACE
KERNEL
connect()
VERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-52-320.jpg)
![BPF: Attach program to connect() syscall (LSM)
USER SPACE
KERNEL
connect()
VERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>
BPF
LSM Hook](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-53-320.jpg)
![BPF: Return EACCESS – No packets created at all
USER SPACE
KERNEL
connect()
EACCESS
VERIFIER +
JIT
SOURCE CODE [C]
</>
BYTE CODE [BPF]
</>
BPF
LSM Hook](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-54-320.jpg)
![XDP [Express Data Path] mode
BPF
Driver
Run BPF Program inside network
driver with access to DMA buffer
Software Stack](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-59-320.jpg)
![XDP [Express Data Path] mode
BPF
Driver
Can drop millions of packets per
Second while under DDoS
Software Stack
drop](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-60-320.jpg)
![XDP [Express Data Path] mode
BPF
Driver
Can pass packets to network stack
Software Stack
drop
Stack](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-61-320.jpg)
![XDP [Express Data Path] mode
BPF
Driver
Can perform loadbalancing and
transmit out the wire again
Software Stack
drop
LB & TX
Stack](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-62-320.jpg)
![75
140
205
240
325
365 370 365
410 412 425
445 450 460 460
490 495 505 515 525
545
565
0
100
200
300
400
500
600
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
BPF redirect() performance
[GBit per core]
Intel Xeon 3.5Ghz Sandy Bridge, 24 Cores,
(1 TCP GSO flow per core, netperf -t TCP_SENDFILE, 10K policies)](https://image.slidesharecdn.com/dockerconcilium-170418181631/85/DockerCon-2017-Cilium-Network-and-Application-Security-with-BPF-and-XDP-81-320.jpg)
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
- 1. CILIUM: NETWORK AND APPLICATION SECURITY WITH BPF AND XDP Thomas Graf Co-founder & CTO Covalent
- 2. Who is this guy?
- 3. Helped build the biggest monolith ever … Who is this guy?
- 4. Helped build the biggest monolith ever … Who is this guy?
- 5. Time to rethink the kernel
- 6. syscalls syscalls Net IOBlock IO Time to rethink the kernel
- 7. Time to rethink the kernel From monolith to “microkernel” with BPF syscalls syscalls BPF BPF BPF BPF Net IOBlock IO
- 8. Time to rethink the kernel From monolith to “microkernel” with BPF syscalls syscalls BPF BPF BPF BPF BPF BPF Security Networking Net IOBlock IO
- 9. BPF is revolutionizing… • Tracing / Profiling
- 10. BPF is revolutionizing… • Tracing / Profiling Container Performance Analysis Brendan Gregg Wed 1:30pm “Black Belt”
- 11. BPF is revolutionizing… • Tracing / Profiling • Networking Container Performance Analysis Brendan Gregg Wed 1:30pm “Black Belt”
- 12. BPF is revolutionizing… • Tracing / Profiling • Networking • Security Container Performance Analysis Brendan Gregg Wed 1:30pm “Black Belt”
- 13. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low Delivery Frequency
- 14. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low 3-Tier App Monthly Moderate Delivery Frequency
- 15. Application Architectures Delivery Frequency Operational Complexity Single Server App Yearly Low Distributed Microservices 10-100 x’s / day Extreme 3-Tier App Monthly Moderate Delivery Frequency
- 16. Network Security has not evolved $ iptables -A INPUT -p tcp -s 15.15.15.3 --dport 80 -m conntrack --ctstate NEW -j ACCEPT The world still runs on iptables matching IPs and ports:
- 17. Your HTTP ports be like …
- 18. Network Security for Microservices Example Gordon is looking for a job…
- 19. Gordon Job Postings Example: Security for Microservices
- 20. GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API Gordon Job Postings Example: Security for Microservices
- 21. GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API GET /jobs/331 Gordon Job Postings Example: Security for Microservices
- 22. L3/L4 GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API iptables -s 10.1.1.1 -p tcp --dport 80 -j ACCEPT GET /jobs/331 Gordon Job Postings Example: Security for Microservices
- 23. L3/L4 GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API exposed exposed exposed GET /jobs/331 Gordon Job Postings Example: Security for Microservices iptables -s 10.1.1.1 -p tcp --dport 80 -j ACCEPT
- 24. Not exactly least privilege Security team is not amused
- 25. GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API GET /jobs/331 Gordon Job Postings Example: Security for Microservices
- 26. L3/L4 GET /healthz GET /jobs/{id} PUT /jobs/{id} POST /jobs API FROM Gordon ALLOW GET /jobs/.* GET /jobs/331 Gordon Job Postings Example: Security for Microservices
- 27. We demand a demo
- 28. BPF - The Superpowers inside Linux
- 29. SANDBOX BPF GET /foo BPF: Transparent redirection into proxy
- 30. SANDBOX BPF Proxy GET /foo redirect rules sk BPF: Transparent redirection into proxy
- 31. SANDBOX BPF Proxy GET /foo redirect rules sk Shared State • Orig Dest IP • Identity BPF: Transparent redirection into proxy
- 32. SANDBOX BPF Proxy GET /foo redirect reinject rules sk sk Shared State • Orig Dest IP • Identity BPF: Transparent redirection into proxy
- 33. SANDBOX BPF Proxy GET /foo rules BPF: Transparent redirection into proxy sk sk 403 Access Denied
- 34. So what is BPF exactly?
- 35. .insns = { BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8), BPF_ST_MEM(BPF_DW, BPF_REG_2, 0, 0), BPF_LD_MAP_FD(BPF_REG_1, 0), BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -152), BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0), BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2), BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_1, 0), BPF_ST_MEM(BPF_DW, BPF_REG_3, 0, 42), BPF_EXIT_INSN(), } What is BPF? Learn more about BPF: docs.cilium.io
- 36. BPF: Toolchain – from user to kernel USER SPACE SOURCE CODE [C] </>
- 37. BPF: LLVM compiles program code to bytecode USER SPACE SOURCE CODE [C] </> BYTE CODE [BPF] </>
- 38. BPF: Bytecode is loaded and verified into kernel USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </>
- 39. BPF: Bytecode runs inside safe kernel sandbox USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> SANDBOX BPF
- 40. BPF: Program is attached to event (packet-in) USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> SANDBOX BPF
- 41. BPF: Program can redirect to netns & sockets USER SPACE KERNELVERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> SANDBOX BPF
- 42. BPF – An opportunity to rethink security policy enforcement
- 43. Status Quo: Policy Enforcement connect()
- 44. Status Quo: Policy Enforcement connect() TCP
- 45. Status Quo: Policy Enforcement connect() TCP Network packets
- 46. Status Quo: Policy Enforcement connect() TCP Network packets veth veth namespace boundary
- 47. Status Quo: Policy Enforcement connect() TCP Network packets iptables veth veth namespace boundary
- 48. Status Quo: Policy Enforcement connect() drop TCP Network packets iptables veth veth namespace boundary
- 49. Status Quo: Policy Enforcement connect() drop TCP Network packets ETIMEDOUT iptables veth veth namespace boundary
- 50. Status Quo: Policy Enforcement connect() drop TCP Network packets ETIMEDOUT/ ECONNREFUSED iptables RST veth veth namespace boundary
- 51. Can we do better? connect()
- 52. BPF: Leverage user space tool chain USER SPACE KERNEL connect() VERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </>
- 53. BPF: Attach program to connect() syscall (LSM) USER SPACE KERNEL connect() VERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> BPF LSM Hook
- 54. BPF: Return EACCESS – No packets created at all USER SPACE KERNEL connect() EACCESS VERIFIER + JIT SOURCE CODE [C] </> BYTE CODE [BPF] </> BPF LSM Hook
- 55. XDP/BPF – The software loadbalancer of the future
- 56. WHAT IF I TOLD YOU XDP allows for 10x IPVS performance
- 57. Source: https://www.netdevconf.org/2.1/slides/apr6/zhou-netdev-xdp-2017.pdf FB moves from IPVS to BPF/XDP for L3/L4 LB XDP throughput IPVS throughput Source:
- 58. Regular BPF mode BPF Driver Software Stack
- 59. XDP [Express Data Path] mode BPF Driver Run BPF Program inside network driver with access to DMA buffer Software Stack
- 60. XDP [Express Data Path] mode BPF Driver Can drop millions of packets per Second while under DDoS Software Stack drop
- 61. XDP [Express Data Path] mode BPF Driver Can pass packets to network stack Software Stack drop Stack
- 62. XDP [Express Data Path] mode BPF Driver Can perform loadbalancing and transmit out the wire again Software Stack drop LB & TX Stack
- 63. How can I use BPF with Docker?
- 72. Cilium Architecture BPF BPF BPF Cilium Agent CLI Monitor Policy Plugins
- 73. Project Status • Initial release two weeks ago • Docker & Kubernetes integration • Looking for feedback and contributions
- 74. Getting Started • Play with our vagrant box: $ git clone https://github.com/cilium/cilium $ cd cilium/examples/getting-started $ vagrant up
- 75. Summary
- 78. Summary • BPF/XDP will drive the future of software based networking on Linux. • Never underestimate the Jedi • Traditional L3/L4 network policies are insufficient for microservices. Least privilege requires HTTP / API / Function awareness.
- 79. Summary • Never underestimate the Jedi • Traditional L3/L4 network policies are insufficient for microservices. Least privilege requires HTTP / API / Function awareness. • BPF/XDP will drive the future of software based networking on Linux. • Cilium brings BPF/XDP and L7 policies to containers and microservices.
- 80. Thank You! github.com/cilium/cilium http://cilium.io/ @ciliumproject Want to chat? DM me! @tgraf__ Don’t forget to vote and grab a shirt on the way out!
- 81. 75 140 205 240 325 365 370 365 410 412 425 445 450 460 460 490 495 505 515 525 545 565 0 100 200 300 400 500 600 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 BPF redirect() performance [GBit per core] Intel Xeon 3.5Ghz Sandy Bridge, 24 Cores, (1 TCP GSO flow per core, netperf -t TCP_SENDFILE, 10K policies)