Building a sdn solution for the deployment of web application stacks in docker
Download as PDF, PPTX4 likes493 views
This document discusses building a SDN solution for deploying web application stacks in Docker containers. It proposes developing a wSDN network plugin driver for Docker's libnetwork that implements the Docker plugin API and network driver protocol. This would allow wSDN to manage container networking and provide features like multi-host networking, IP address management, and tenant isolation across multiple data centers. It also discusses Docker's existing networking limitations and outlines requirements for a SDN solution to address Docker's needs for large web application deployments in a multi-tenant environment.
More Related Content
Similar to Building a sdn solution for the deployment of web application stacks in docker (20)
Building a sdn solution for the deployment of web application stacks in docker
- 1. Building a SDN solution for the deployment of web application stacks in Docker Technical Considerations Jorge Juan Mendoza Pons Cloud Software Engineering & Operations Direction March 2016
- 2. Summary â Significant Assumptions â Docker networking approach, modes and rules â Building a network driver plugin as a remote driver for Dockerâs libnetwork CNM â The Proposed solution: wSDN â Show me the codes: Docker overlay sources (as a sample) â Why Docker needs a better networking/SDN solution ? use cases & requirements â Technical considerations of the proposed solution â How to (simple) test the solutionâs performance â Appendixes
- 3. Significant Assumptions â SDN solution based on a single container format (Docker) and ecosystem â Application-space scope is the deployment of web application 'stacks' in such a system (including their associated delivery pipeline), leaving out of the scope other industrial/IoT/real time video/trading scenarios.
- 4. Docker networking approach and default rules â Whether a container can talk to the world is governed by: â whether the host machine is forwarding its IP packets â whether the hostâs iptables allow this particular connection. â Whether two containers can communicate is governed, at the operating system level, by two factors. â Does the network topology connect the containersâ network interfaces? Docker (default) attaches all containers to a single docker0 bridge, providing a path for packets to travel between them. â Do your iptables allow this particular connection? Docker server will add a default rule to the FORWARD chain with a blanket ACCEPT policy (unless you set --iptables=false when the daemon starts) if you retain the default --icc=true, or else will set the policy to DROP if --icc=false. â The intention of Docker networks is to segregate services such that the only things on a network are things that need to talk to each other: weâll end up having lots of networks with a dynamic amount of containers in them. Networks are all isolated from each other. If two containers are not on the same network, they cannot talk.
- 5. Docker Networking Modes â bridge â creates a Linux bridge device for a network, and gives each container joined to the network an interface on that bridge â host â this mode does not create a separated network stack for a container. The container shares the same network stack with the host which means the container also has full access to the network interfaces of the host â none â a container will have its own isolated network stack but it is left unconfigured and it is all for the user to setup the network stack â overlay â provides native multi-host networking for docker clusters. This driver uses VxLAN to encapsulate traffic, and requires kernels >= 3.16. Docker requires a K/V store so Docker engines can discover each other, this is provided as a daemon flag. Overlay integrates with an in-built IPAM driver which allows to customize how IP addresses are assigned. The docker daemon implements an embedded DNS server which provides built-in service discovery for any container created with a valid name or net-alias or aliased by link.
- 6. Docker networking with libnetwork â libnetwork uses a driver / plugin model to abstract the complexity of the driver implementations by exposing a simple and consistent Network Model to users. â CNM objects and attributes â NetworkController â entry point to bind in-built or remote drivers to networks â Driver â implementation in order to satisfy specific use cases and deployment scenarios â Sandbox â contains the configuration of a container's network stack. This includes management of the container's interfaces, routing table and DNS settings. Local/Container Scope. â Endpoint â represents a Service Endpoint. It provides the connectivity for services exposed by a container in a network. Global Scope within a cluster â Network â provides connectivity between a group of end-points that belong to the same network (single/multiple host) isolated from the rest (the Driver performs the actual work of providing the required connectivity and isolation). Global Scope within a cluster â Cluster â Group of Docker hosts participate it multihost networking. â Attributes: Options & Labels
- 7. Proposed solution: wSDN â Develop a wSDN network plugin by implementing the Docker plugin API and the network plugin protocol (network driver plugins are activated in the same way as other plugins, and use the same kind of protocol) â wSDN Network driver plugin is supported via LibNetwork that includes a proxy which forwards driver operations to a remote process, called a remote driver â Golang interface philosophy â Define one âinterfaceâ per function to encourage composability and compose different solutions for different needs.
- 8. Proposed solution: wSDN 1. wSDN as both a libnetwork remote network driver and IPAM driver implementation 2. wSDN takes care of binding one of a veth pair to a network interface on the host, e. g., Linux bridge, Open vSwitch datapath. 3. libnetwork discovers wSDN via plugin discovery mechanism before the first request is made a. During this process libnetwork makes a HTTP POST call on /Plugin.Active and examines the driver type, which defaults to "NetworkDriver" and "IpamDriver" b. libnetwork also calls the following two API endpoints i. /NetworkDriver.GetCapabilities to obtain the capability of wSDN which defaults to "local" ii. /IpamDriver.GetDefaultAddressSpcaces to get the default address spaces used for the IPAM 4. libnetwork registers wSDN as a remote driver 5. User makes requests against libnetwork with the network driver specifier for wSDN (-d wSDN and --ipam-driver=wSDN from the Docker CLI) 6. libnetwork stores the returned information to its key/value datastore backend (abstracted by libkv)
- 9. Show me the codes â Netlink library â Type Driver interface â Remote driver api â Type Ipam interface â Simple DNS service â Overlay driver
- 10. Whatâs still missing in Docker networking (v1.10.2) â Secure endpoints (otherwise make sure that the channel being overlayed is secured) â Dynamically apply configurations (IPAM, subnet restriction) â Support for larger MTUs (the default libnetwork driver currently doesnât support larger) + dynamic PMTU discovery â Having multiple pluggable drivers participating in handling various network management functionalities (multi-host IPAM, SDN, DNS, etc) â DHCP / dynamic multi-host IPAM driver â Multicast networking support â Tighter integration with microservices main architectural approach: service discovery and load balancing (i.e. UCP clusters) â Critical runtime features for multi-tenants environments (multi-cloud & cross-DC, partition-tolerant & eventually consistent, partially connected network support) (i.e. Docker Cloud & On-premises CaaS)
- 11. The impact of Docker in the SDN field â Ephemeral nature: Docker containers are ephemeral by design, this leads to several potential issues like being able to keep your firewall (or LB) configuration up to date because of difficult IP address management, connect to services that might disappear at any moment, etc â Density Factor: Docker brings densities that will change the way applications behave in a network environment: the subscription ratio will change considerably. There may be 2,000 containers on a physical server in an environment that has 1000+ hosts â Data Gravity: Data should be treated as an object that attracts more objects to it. The more data, the more services and applications it will attract. As more data collects, which is inevitable, compute resources will need to be faster, called to the app in many ways. The compute will swarm or be streamed, which we are seeing already (i.e. AWS Lambda, Google Functions)
- 12. wSDN main Use Case: Delivery pipeline of web app stacks in a multi-tenant clustered environment â One LB IP for each application; One NAT Subnet for the containers of each application; One Application Router for each application subnet â Implement IPAM (both static and DHCP based for Subnet/Bridge/Router/LB Creation cases) across clusters and/or DCs. Autoscaling, Downscaling and destroying events should trigger calls to reconfigure the LB for that app â L3 connectivity between gateways of two clusters and/or routers of the same tenant across DCs â Multi-tenant networking. Tenant Isolation and identification across DCs â Coordinate and synchronize Subnet / LB IP (De)Allocation across DCs â Near flaw-less content delivery (i.e. Netflix intelligent edge networks that cache rich content closer to the user)
- 13. General approach of a SDN solution based on Docker â Network Model â Protocol Support â Container / Group Isolation â Container Discovery / Name Service â Distributed Storage â Encryption Channel â Partially Connected Network Support â Separate vNIC for Container â IP Overlap and Container Subnet Overlap support â Advanced Networking Services (i.e. stateful L4 -L7 services)
- 14. How to simple test (the performance of) our SDN solution â Testing on Amazon Web Services with modern AWS instance types (i.e. m4) that support Jumbo Frames with an MTU of 9001 instead of the old default of 1500. â Using both the iperf3 and qperf test tools to test the performance. Qperf is a very simple tool that does short tests runs, I use iperf3 to do a longer test run, using more parallel connections (higher number of parallel connections causes more overhead) â The default libnetwork driver currently doesnât support larger MTU sizes.
- 15. Appendix I. Docker underlying linux kernel features â namespaces - allow processes to be put into different namespaces, with all processes in each namespace has no clue about the existence of other processes in other namespaces. It then provides a form of lightweight virtualization and resource isolation. There are several types of namespaces used by Dockers to accomplish their goals of building isolated containers: pid, net, ipc, mnt, uts â control groups (cgroups) - it is Linux kernel layer that provides resource management and resource accounting for groups of processes. Docker implement cgroups so that available hardware resources can be shared fairly between containers and also to limit it if necessary. â union file system (UnionFS) - it is a file system that operates by creating layers which is used to provide the building blocks for containers. â container format - it is a wrapper that combines all the previously mentioned technologies. Despite the fact that libcontainer is the default container format, Docker also supports LXC (Linux Container)13
- 16. Appendix II. Container ecosystem map
- 17. Appendix II. Docker ecosystem features
- 18. Appendix III. Docker current limitations Containers promise rapid scalability, flexibility and ease, but they're not right for every workload. 1. Unstable ecosystem (how much of your infra are you going to virtualize ?) 2. Some applications simply need to be monolithic 3. Dependencies (underlaying OS kernel, libraries, binaries) 4. Security & Vulnerability management (compliance) 5. Weaker isolation (dependencies + deep level of authorization) 6. Potential for sprawl 7. Complicated Data Management 8. Repetitive Troubleshooting 9. Limited tools (monitor, manage)
- 19. Appendix IV. SDN common std use cases 1. Automating Network Provisioning for Internal Private Cloud or DC 2. Simplifying Operations by Centralizing Policy and Configuration Management (i.e Support for network "slices") 3. Increasing WAN Utilization Through Traffic Steering 4. Detecting and Mitigating Security Threats 5. Aggregating Data Center Network Tap Traffic 6. Programmability: the ability to use open APIs to link applications to the network. Current networks lack a common set of APIs, which make it very difficult to program applications directly to network resources
- 20. Appendix IV. (cont.) SDN common std use cases 1. Holistic Network Management: SDN allows you to plan your network at the software level and get as complex as you want without worrying about re- cabling at the hardware level. 2. Security: SDN usually provides a more granular security model than traditional hardware/wired security (e.g. NIC validation) 3. Centralized Provisioning: itâs almost trivial to set up completely isolated networks over existing copper. 4. Efficiency: SDN can reduce downtime because everything is done at the software level: you never have to touch the cables. 5. Cost Reduction: SDN network provisioning automation helps lowering operating and expenses (reduces the required human effort).
- 21. References 1. Weaveworks simple distributed SDN for Docker (github) (blog) (docs) 2. Container patterns 3. Battlefield: Calico, Flannel, Weave and Docker Overlay Network (weave) 4. Docker libnetwork (blog) 5. Remote drivers (plugins) 6. Docker changelog (network) (dns) 7. Docker networking: define containersâ comms at the app level 8. Docker multi-host networking (benchmrk) 9. Plumgrid / libnetwork-plugin 10. SDN controllers