Managing and Testing Ensembles of IoT, Network functions, and Clouds

Managing and Testing Ensembles of IoT,
Network functions, and Clouds
Hong-Linh Truong
Distributed Systems Group, TU Wien
truong@dsg.tuwien.ac.at
http://rdsea.github.io
CCI@USC, Los Angeles, 21 Sep 2017 1

Acknowledgements:
Many results are from the joint work with students
and colleagues:
Luca Berardinelli, Duc-Hung Le, Nanjangud C.
Narendra, Christian Proinger, Daniel Moldovan, Ivan
Pavkovic, Georgiana Copil, Stefan Nastic
and from
H2020 U-test (www.u-test.eu)
H2020 Inter-IoT (http://www.inter-iot-project.eu/)
Note: some work are under submission.

Outline
 Application-oriented ensembles of IoT, Network
functions and Cloud resources
 Resource Management
 Monitoring
 Uncertainty and Testing Uncertainties of
Infrastructures
 Conclusions and Future work

Application-oriented Ensembles of IoT,
Network functions, & Clouds: Necessity?

Systems of IoT, Network Functions,
and Clouds
5
CCI@USC, Los Angeles, 21 Sep 2017

Moving to blending IoT, network
functions and cloud resources
 Type 1
 Mainly focus on IoT networks: sensors, IoT gateways, IoT-to-
cloud connectivity (e.g., connect to predix.io, IBM Bluemix,
Azure IoT, Google Cloud, Amazon IoT, etc.)
 Type 2
 Mainly focus on (public/private) services in data centers: e.g.,
IoT data hubs, NoSQL databases, and big data ingest systems
 Type 3
 Equally focus on both IoT and cloud sides and have the need
to control at both sides
 Highly interactions between the two sides, including the network
in the middle

All types of service models
 Cloud resources as services are known
 NFV/5G coming and more service providers at
the edge are available
 Network function virtualization
 IoT infrastructure-as-a-service
 Pay-per-use IoT communication
 IoT data-as-a-service Cloud service models
 Public and private providers

Examples of IoT networks
http://www.sktelecom.com/en/press/detail.do?idx=1172

Some application scenarios
 Emergency responses, on-demand crowd sensing, Geo
Sports monitoring, cyber-physical systems testing, etc.
Geo Sports: Picture courtesy
Future Position X, Sweden

Emergency in a Seaport
 Built a top existing INTER-IoT scenarios
 Coordinating activities when an accident happens
 Example of actions in a resource slice in a seaport:
1) activating monitoring containers with sensitive goods in the port
2) analyzing and controlling robotic cranes and trucks to make sure that
they do not prevent the emergency responses as well as ready to
support the responses
3) sending alarms and controlling vessel arrivals and revising transport
planning,
4) providing information for operational assistance for the emergency
responses
5) activating systems to support the monitoring of people impacted by the
accident using devices and platforms for chronic disease and cognitive
decline prevention.

Examples of IoT data services
IoT data as a service can be offered by different types
of providers
11
Florin-Bogdan Balint, Hong-Linh Truong, On Supporting Contract-aware IoT Dataspace Services, the 5th IEEE International Conference on Mobile
Cloud Computing, Services, and Engineering (MobileCloud 2017), 6-8 April 2017 in San Francisco, USA
.

Ensembles of IoT, Network
functions and clouds

Challenges
 Modeling distributed IoT, network functions and
cloud capabilities in an ensemble
 Slicing end-to-end network of resources
 Composing resources in ensembles of IoT,
network functions and clouds
 (Re-)configuring composed resources
 Testing and monitoring
The SINConcept: http://sincconcept.github.io/
Hong-Linh Truong, Nanjangud Narendra, SINC - An Information-Centric Approach for End-to-End IoT Cloud Resource Provisioning, 2016
International Conference on Cloud Computing Research & Innovation (ICCCRI2016), CloudAsia 2016, May 3-5, 2016, Singapore

Tools treating IoT, network functions,
and cloud services in an isolated
manner are not enough

Service engineering analytics for
IoT Cloud Systems
15CCI@USC, Los Angeles, 21 Sep 2017
http://tuwiendsg.github.io/iCOMOT/
We started in 2011 without network functions in our mind!

Monitoring, Controlling and Testing
IoT Cloud Systems
Check: http://tuwiendsg.github.io/iCOMOT/demo.html

Configuration is not done easy for
the ensembles
 Sensors/gateways and cloud services have different
management systems and interfaces
Deploy different topologies and configure topologies
to work together is hard
IoT sensors Cloud Service

Cross IoT and cloud controls
 IoT gateways control is very different from cloud controls
 Low-level REST API in IoT gateways management and high
level elasticity rules for cloud services
 Different protocols for communicating with resource managers
Control for IoT Gateways Control for Clouds

Too low-level Infrastructure-level
Testing
(From modeling/description)
*testing
strategy=testing plan
Daniel Moldovan,Hong-Linh Truong, A Platform for Run-time Health Verification of Elastic Cyber-physical Systems,
The IEEE International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication
Systems (MASCOTS 2016), September 19-21, Imperial College, London, UK

HARMONIZING RESOURCES
FOR ENSEMBLES?
Duc-Hung Le, Nanjangud C. Narendra, Hong Linh Truong:
HINC - Harmonizing Diverse Resource Information across IoT,
Network Functions, and Clouds. FiCloud 2016: 317-324

Integrating diverse types of resources
 Make a Resource Grid ready for creating ensembes
 Harmonize IoT, network functions and cloud resources
 API Integration and Communication
 Use REST API for obtaining metadata and control of resources
 Sensoring data can be transferred through different middleware
 Work with existing metamodel (IoTivity, OpenHAB, IoTDM, ETSI
MANO, OCCI, CIMI, etc.)
 Rely on scalable cloud communication middleware (e.g., AMQP &
MQTT)
IoT networks Network Function Services Clouds
Resource Grid

Examples of existing
providers/models
Provider Category APIs Information models
FIWare Orion IoT RESTful (NGSI10), one-time
query or subscription
High level attributes on
data and context
FIWare IDAS IoT RESTful for read/write custom
models and assets
Low level resource
model catalogs
IoTivity IoT REST-like OIC protocol, support
C++, Java and JavaScript
Multiple OIC model
OpenHAB IoT RESTful for query and control
IoT resources
Low level resource
model catalogs
OpenDayLight Network Dynamic REST generated from
Yang model (model-driven)
Low level resource
model catalogs
OpenBaton Network RESTful for network service
description
ETSI MANO v1.1.1 data
model
OpenStack Cloud RESTful, multiple language via
SDK, OCCI, CIMI
OpenStack model,
OCCI, CIMI
22

Information model
Physical: Sensor/actualtor/devices in providers’ models
Virtual IoT: SD-Gateway and capabilities.
Network functions: edge-to-edge, edge-to-cloud network.
Clouds: VM, data services, data analytics.
23

Prototype
24
http://sincconcept.github.io/HINC/

Testbed
25

Reducing complexity in accessing
and control resources
26
1. Query data points
2. Control the
resource
3. Query network
functions and clouds

As we able to get resources  deploy and
configuration

MONITORING ENSEMBLES?
CCI@USC, Los Angeles, 21 Sep
2017
28

Key questions in monitoring
 Which metrics make sense? Memory/CPU usage?
 We focus on: response time, delay, cost, availability and
reliability
 Collect monitoring data from multiple resources of
sub-systems
 Gathering and correlating monitoring
 Deal with different integration models: through provider
interfaces versus instrumentation
 Analytics and visualization

Monitoring
components
Hong-Linh Truong, Christian Proinger, SANALYTICS – On Monitoring and Analytics of IoT, Network
Functions and Clouds. 2017. working paper.

Example of an ensemble

Example of low level metrics

UNCERTAINTY AND
APPROACH TO TESTING

Infrastructure uncertainties
 CPS: includes IoT, network functions and clouds
 An ensemble represents a virtual infrastructure
 Cross-system and cross-layer
 Goals
 Characterize and specify possible uncertainties associated
with interactions among elements in CPS infrastructures
 Supporting testing uncertainties and uncertainties
analytics
 Conventional aspects, e.g., infrastructural physical
resources and typical system operations
 Emerging novel aspects: data uncertainties (data/data-
centric CPS), elasticity of CPS resources (w.r.t function
and composition), and Governance (related to
business/trustworthiness)

Uncertainty Model
Locality
0..1
Uncertainty
Lifetime
0..1
Random
0..1
0..*
Effect
0..*
Risk
0..1
0..*
Pattern
0..1
0..*
Cause
0..*
0..*
Measurement
0
..
*
substatement
0..*
Uncertainty
Indeterminacy
Source
1..*
source
0..*
Measure
«enumeration»
IndeterminacyNature
nondeterminism
insufficientResolution
missingInfo
composite
unclassified
type1..*
0..*
Belief
Measurement 0..*
0..* beliefDegree
Belief
Statement
Characterizing Uncertainty
Belief Model
Source: https://www.simula.no/file/d12pdf/download

Data
ElasticityGovernance
Important infrastructure uncertainties
classes
Storage
Uncertainty
Infrastructure
Uncertainty
DataDelivery
Uncertainty
Actuation
Uncertainty
DeploymentTime
Uncertainty
ExecutionEnvironment
Uncertainty
Storage
ComplianceUncertainty
StorageQuality
Uncertainty
StorageDependability
Uncertainty
DataQuality
Uncertainty
DataDelivery
DataDeliveryDependability
Uncertainty
ActuationDependability
Uncertainty
EnvironmentDependability
Uncertainty
Actuation
ApplicationDependability
Uncertainty
Runtime
Uncertainty

Examples – UC1 Geo Sports
Data Delivery uncertainties
Uncertainties affect the infrastructure capabilities
for generating, processing and delivering data
Located in Software in Cyber
Environment, e.g., software components
(execution environments, OS of gateways) in
the infrastructure
At Runtime but Sporadic
Caused by Resource Competition (☒
Technological process)
Indirect impact on Software
Non-functional
aspects
Very Low Low Medium High Extrem
e
Dependability
Legal/complian
ce
Quality
Risk of this type of uncertainties

Key observations:
how to combine topdown approach
and bottom-up approach to leverage
strengths of both models and elastic
systems?
Combining MDE and Elastic Execution

Key issues: Two separate worlds
SUT Infrastructure
Development
Deployment Description
Development
IoT/Cloud Infrastructures
Infrastructure Configurations
Resource
Information
Adaptor/Tool
Deployment Scripts
Preparing CPS under Test
Requirements
Artifact
Repository
Figure source:
Mark Utting and Bruno Legeard. 2006. Practical Model-Based Testing: A Tools
Approach. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
?

Combining MDE with Elastic
Execution
Hong-Linh Truong, Luca Berardinelli, Testing Uncertainty of Cyber-physical Systems in IoT Cloud
Infrastructures – Combining Model-Driven Engineering and Elastic Execution, Workshop on Testing
Embedded and Cyber-Physical Systems, ISSTA 2017, Santa Barbara, 10-14 July, 2017.

Interwoven test execution and
provisioning
IoT Cloud Infrastructures
Modeling
Uncertainties
and SUT
Uncertainty
Profile
System Under
Test (SUT)
Models
Generating
Uncertainty
Test cases
Deploying/
Configuring
SUT
System Under
Test (SUT)
Test cases
Executing
Tests
IoT/Cloud
Resource
Information
Deploying/
Configuring
Testing Utilities
Test
Utilities

Tool pipelines: from MDE to elastic
execution for testing
Key thoughts:
 Different algorithms to create suitable deployment configurations based
on data uncertainties, cost, and time
 Interactions between testing and elasticity control of IoT and Cloud
services

Our current progress
Consider to generate provisioning configurations
from SUT models
Extracting
model
information &
Generating test
configuration
test
strategies
Infrastructural
IoT Cloud
resources
Provisioning &
Configuring
SUT
Executing
Tests
Test plans &
executors
IoT Units
Cloud
T
E
S
T
S
ProviderA
ActuatorsSensors
VMsAnalytics GWs
IoT Units
Cloud
T
E
S
T
S
ProviderB
ActuatorsSensors
VMsAnalytics GWs
Infrastructure
(Class Diagrams)
Behaviour
(State Machine
diagrams)
Uncertainty
Hong-Linh Truong, Luca Berardinelli, Ivan Pavkovic and Georgiana Copil, Modeling and Provisioning IoT Cloud
Systems for Testing Uncertainties, 14th EAI International Conference on Mobile and Ubiquitous Systems:
Computing, Networking and Services (MobiQuitous 2017), Nov 7–10, 2017,Melbourne, Australia

IoT and Cloud Resource Profile

Uncertainty Profile

Generic models for Task Executor
Tasks for testing
also including
deployment and
reconfiguration
of ensembles.

Configuration Generation and
Deployment
 Reuse well-known tools for deployment
 Utilize different information services
 Runtime and (future) uncertainty analytics

Example of BTS monitoring

Example of BTS
monitoring

Examples
"MQTTConfig1": {
"name": "MQTTConfigServer",
"protocolType": "MQTT",
"qosLevel": [],
"type": "CommunicationConfiguration"
}
"MQTTConfig2": {
"name": "MQTTConfigClient",
"protocolType": "MQTT",
"clientID": "",
"serverIP": "35.189.187.208",
"portNumber": 1883,
"topics": ["/gateway/electricity"],
"qosLevel": [2],
"type": "CommunicationConfiguration"
}
services:
ingest:
build: .
volumes:
- ./:/t4u
electricitysensor:
image:
"localhost:5000/t4u/mqttsensor/realsensor:
v01"
iotgateway:
image:
"localhost:5000/t4u/cloudservice/mqttbroke
r:v01"
Enriched model information for
deployment configurations Generated deployment description

Summary
 Ensembles of IoT, network functions and clouds
 Important for various types of applications
 SINC: a conceptual framework
 Elasticity and Uncertainties
 We need to consider more data and network aspects.
 Adaptation and optimization under uncertainties
 Modeling and testing uncertainties are very challenging
 Testing uncertainties require fundamental changes in testing
techniques: deal with elasticity and virtualization
 We need new set of tools and techniques for
managing and testing ensembles.

Future work
 Current research topics
 APIs for programming resource queries and controls
(http://sincconcept.github.io/HINC/)
 Configuration tools (http://tuwiendsg.github.io/SALSA/)
 Uncertainty testing and analytics
(https://github.com/tuwiendsg/COMOT4U/)
 Monitoring and analytics
 Ensemble requirement modeling, composition algorithms and
optimization
 Interoperability issues (http://www.inter-iot-project.eu/)
 Policy execution
Check http://rdsea.github.io and
https://github.com/tuwiendsg/COMOT4U for new update

Thanks for your
attention!
Hong-Linh Truong
Distributed Systems Group
TU Wien
rdsea.github.io

Managing and Testing Ensembles of IoT, Network functions, and Clouds

More Related Content

What's hot (14)

Similar to Managing and Testing Ensembles of IoT, Network functions, and Clouds (20)

More from Hong-Linh Truong (20)

Recently uploaded (20)

Managing and Testing Ensembles of IoT, Network functions, and Clouds