fog05: The Fog Computing Infrastructure
1 like•1,505 views
This document introduces fogOS, an open-source distributed computing infrastructure developed by ADLINK's Advanced Technology Office. FogOS aims to provide a decentralized platform for managing heterogeneous compute, storage, networking and I/O resources across cloud and edge devices. It addresses limitations of cloud-centric architectures by bringing more control and analytics closer to data sources. The document outlines FogOS concepts including nodes, entities, networks and plugins. It also demonstrates FogOS through a prototype smart home application deployed across heterogeneous devices in a unified manner.
fog05: The Fog Computing Infrastructure
- 1. Advanced Technology Office Angelo Corsaro, PhD Chief Technology Officer ADLINK Tech. Inc. [email protected] AT() Gabriele Baldoni Technologist ADLINK Tech. Inc. [email protected] FOG COMPUTING THE INFRASTRuCTURE
- 2. ADLINK’s Advanced Technology Office Innovating Together
- 3. Mission Promote technology innovation and accelerate technology adoption across ADLINK BUs. Innovate Promote ADLINK technology vision, excellence and thought leadership. Evangelise Accelerate innovation adoption to out-innovate competitors and win market shares. Win Identify and de-risk technologies that can improve ADLINK competitiveness on the market. Scout
- 4. ADLINK | ATO Areas of Focus Improve the abstraction level for programming (distributed) systems leveraging FPGA/GPU/CPUs. Heterogeneous Computing Infrastructure Real-Time Analytics spanning from data processing to computer vision. Analytics Unified abstraction for compute, storage and networking compatible with the non-functional requirements of IIoT systems (i.e. real-time). Fog Computing Algorithms, architectures and technologies for scalable, fault-tolerant and secure large distributed systems. Distributed Computing Protocol research for low-power network and constrained devices. Low Power Network and Devices
- 6. Our quest is to blend theory and practice to innovate and solve real-world problems efficiently and elegantly. Theory and Practice
- 7. State of Practice Innovating Together
- 8. Cloud Centric Architectures The majority of IoT systems are today cloud- centric These systems are characterised by device- to-cloud communication and in-cloud analytics
- 9. Cloud Computing… What Else? Innovating Together
- 10. Cloud Computing There is sufficient bandwidth to push data to the Cloud. Assumption #1
- 11. This slides have been crafted by Angelo Corsaro Any use of these slides that does include me as Author/Co-Author is plagiary Smart Factory 0.5 TB of data produced per day
- 12. Cloud Computing Connectivity is not an issue. A device will (almost) always be connected to the cloud. Assumption #2
- 13. This slides have been crafted by Angelo Corsaro Any use of these slides that does include me as Author/Co-Author is plagiary Searching Operator…
- 14. Cloud Computing The latency induced by cloud-centralised analytics and control is compatible with the system’s dynamic Assumption #3
- 15. This slides have been crafted by Angelo Corsaro Any use of these slides that does include me as Author/Co-Author is plagiary Autonomous Vehicles coordination of fast moving autonomous vehicles intermittent connectivity dynamic pairing of devices
- 16. Cloud Computing The connectivity cost is negligible Assumption #4
- 17. Cost of connectivity is an issue in Smart Grids as the operator has to pay for the 2G/3G/4G data-link
- 18. Cloud Computing Companies are comfortable with exposing their data to the cloud. Assumption #5
- 19. ARE YOU SURE?
- 20. What’s Really Needed? Cloud Computing gives operationally convenient abstractions and tools to manage and provision data-centre resources Yet, most systems can’t just live with Cloud-Centric Architectures We need an infrastructure that allows us to federate compute, storage, I/O and communication resources regardless of their location
- 21. Horizontal, system-level architecture that distributes computing, storage, control and networking functions closer to the users along a cloud-to-thing continuum Fog Computing | fɒg kəmˈpjuːtɪŋ|
- 23. Vision fogOS aims at providing a decentralised infrastructure for provisioning and managing (1) compute, (2) storage, (3) communication and (3) I/O resources available anywhere across the network. fogOS addresses highly heterogeneous systems even those with extremely resource-constrained nodes.
- 24. — Decentralised Design fogOS can manage and provision any network connected device on which it agent is running Its decentralised architecture allows to manage the system from anywhere and does not need any specific set of nodes running as “servers”
- 25. — The Eclipse Project
- 26. OpenFog and 5GPPP fogOS is one of the infrastructure identified as compliant with the 5G principles and requirements by the EU 5GPPP working group fogOS architecture is compatible with the OpenFog Reference Architecture. Additionally fogOS is used as the reference fog platform in several test-beds
- 27. Users and Press Mm2 Multi-access edge platform manager - NFV (MEPM-V) MEAO fog05 + MEPM-V plugin 5GCity Components ETSI NFV Components ETSI MEC Components Multilayer Orchestrator Operation Support System Os-Ma-nfvo NFVO VNFM (ME app LCM) VNFM (ME platform LCM) Virtualisation Infrastructure Manager Multi-access edge platform (VNF) NFVI Data plane (VFN/PNF) Os-Ma-nfvo Me app (VNF) Service Mm5 Mp1 Mp2 Or-Vnfm Or-Vi Ve-Vnfm-vnf Nf-Vi Nf-Vn Nf-Vn Ve-Vnfm-em Vi-Vnfm = Mm6 Mv3 Mv2 Mv1 ETSI NFV Reference points ETSI MEC Reference points ETSI NFV-MEC Reference points • Mv1 ~ Os-Ma-nfvo • Mv2 ~ Ve-Vnfm-em • Mv3 ~ Ve-Vnfm-vnf Mm3* Mm1 fog05 + MEAO plugin
- 28. — Ecosystem The ATO is focusing on building an ecosystem of technologies that will allow to bring control back to the edge. As such we are working in technology for Content Centric Networking, namely zenoh, and Decentralised Data management, namely YAKS that are core to fogOS and essential for any edge- centric system
- 29. abstractions
- 30. AT() Node A fogOS node is the abstraction used to identify a location at which computation, networking, storage and I/O resources are available. Each node has a running fogOS agent. Cloud IaaS agent agent agent agent agent agent agent
- 31. AT() Entity A fogOS entity is an atomic entity, such as a Virtual Machine, a container, a UniKernel, a binary executable, or a directed acyclic graph (DAG) of entities. Entities may have deployment affinity w.r.t. to each other as well as with respect to compute, storage, I/O and accelerators, e.g., FPGA fogOS uses plug-in for dealing with different kinds of entities VM C UK C BE UK VM: Virtual Machine C: Container UK: Uni Kernel BE: Binary Executable uS: micro Service uS BE UKUK
- 32. AT() Entity and Atomic Entity FSM Entities as well as Atomic Entities have a well defined FSM that represent a set of all possible states of a deployable unit. CONFIGURED RUNNING PAUSED SCALING onPause() onStop() start() stop() onStart() onResume() pause() resume() scale() MIGRATING migrate() TAKING_OFF LANDING
- 33. AT() Networks A fogOS network is a connection between different entities. A network can span across different tiers and is provided as an overlay virtual network using standard technologies, such as VxLAN. These networks can be isolated or connected to the external world. UK C BE UK VM: Virtual Machine C: Container UK: Uni Kernel BE: Binary Executable uS: micro Service uS BE UKUK VM C
- 34. AT() Information Model fogOS’s information model defines the describes associated with nodes, entities and networks. Additionally it provides an abstract way to describe applications and relations between them. It is implemented as a set of JSON Schema.
- 35. AT() Relation with ETSI NFV and MEC IM fogOS information model is a super-set of the ETSI (European Telecommunications Standards Institute) MEC and ETSI NFV Specifically, fogOS supports the declaration of I/O constraints. YANG models have also been defined for fogOS abstractions.
- 36. AT() Architecture fogOS is composed by: NDN. At its lowest level, it leverages a Named Data Network (NDN) infrastructure based on zenoh. DDS can also be used as a transport — not necessarily an NDN YAKS. A distributed key-value store that leverages the NDN for scalability Agent. The core logic of fogOS, it takes care of managing, monitoring and orchestrating entities through plugins Plugins. Plugins provide supports for atomic entities, OS, networks, etc. zenoh YAKS AgentPlugins Network Data Link Physical Transport
- 37. AT() Plugins fog 5 leverages plugins interact and manage: Atomic Entities (Runtimes) Networks OSes Monitoring Resource Orchestration Resource Management For each type of plugin an interface has been defined. For instance, plugins that manage atomic entities have to implement the FSM for the kind of atomic entity they will be managing. zenoh YAKS AgentPlugins Network Data Link Physical Transport
- 38. AT() RO and RM plugins The Resource Orchestrator and Resource Manager plugins are special, because they have to implement the Fog MANO (Management and Orchestration ) stack. They interact with entities (as DAG of atomic entities) in two different ways. • RO. Verification of constraints, initial placement and configuration • RM. Replacement in case of bad performance/constraints, scale, reconfiguration These two plugins also allow fogOS to be mapped in the MEC and NFV MANO stacks. The runtime plugins are the implementation of the VIM (Virtual Infrastructure Manager) in an ETSI point of view fogOS
- 39. AT() Interact with fogø5 To interact with fogOS we provide a set of API for Python3. These API uses interact with fogOS using the distributed data store. The demo that we show uses this API. API Docs: https://atolab.github.io/fog05- doc/fog05.html#module-fog05.api
- 40. demo
- 41. AT() What you will see The demo shows the unification of the storage, networking and computation fabric along a cloud-to-thing continuum
- 42. AT() Heterogeneous Infra The demo will demonstrate the transparent unification of heterogeneous resources, such as: Cloud Server Edge Server Things Innovating Together
- 43. AT() Heterogeneous Applications The heterogeneity will also be demonstrated with respect to the kind of atomic entity deployed. Specifically we will demonstrate the provisioning of an entity composed by: LXD Containers Binary Application
- 44. AT() Atomic Entity composing the demo The demo will show the end-to-end deployment of a ‘SmartHome’ environment leveraging other Eclipse IoT technologies Mosquitto Paho SmartHome (openHab)
- 45. AT() Topology of the demo LXD Container Virtual NIC Physical NIC Physical NIC Connected to overlay Network Native Application wlan1 Physical Network VxLAN Network fos-agent wlan wlan eth0 fos-agent fos-agent Native lan LXD wan LXD wlan1lan fos-agent LXD lan fos-agent LXD lan eth0eth0
- 46. AT() DAG Each Atomic Entity is part of the DAG and may have dependency of one (or more) other atomic entities in the graph Gateway Access Point Sensor App MQTT Broker Smarthome Server LXD Container Native Application
- 47. AT() End-to-End unification fogOS allows the deployment of these atomic entities in the most suitable place with respect its constraints, such as I/O, latency, geofencing, and computing power GW AP S MQTT Broker Smarthome Server LXD Container Native Application
- 48. AT() Run demo on your own This demo can be found on GitHub https://github.com/atolab/ fog05_demo
- 49. Live demo
- 50. Next Steps
- 51. AT() The Path Ahead We are firmly convinced that addressing the non-functional requirements of industrial systems and the preservation of privacy requires to (1) bring control back to the edge and (2) keep the data close to the source and (3) let the data owner decide what and how to share.
- 52. AT() The Path Ahead We will continue to build fogOS and its technology ecosystem to establish this vision. This is an ambitious endeavour but it will introduce a major shift in computing. Let’s build this future together!