Testing Uncertainty of Cyber-Physical Systems in IoT Cloud Infrastructures: Combining Model-Driven Engineering and Elastic Execution

Testing Uncertainty of Cyber-Physical
Systems in IoT Cloud Infrastructures:
Combining Model-Driven Engineering
and Elastic Execution
Hong-Linh Truong, Luca Berardinelli
Distributed Systems Group, TU Wien
truong@dsg.tuwien.ac.at
http://rdsea.github.io
TECPS'17, 13 July, 2017, Santa Barbara, USA 1

Outline
 Our focused CPS
 IoT Cloud Systems, Elastic Execution, Uncertainty
 Top-down and bottom up approaches
 MDE/MBT versus Elastic Execution
 Combining methods and tool pipelines
 Early results
 Summary

Our view on IoT Cloud Systems
3
TECPS'17, 13 July, 2017, Santa Barbara, USA
Source: Duc-Hung Le, Nanjangud C. Narendra, Hong Linh Truong:
HINC - Harmonizing Diverse Resource Information across IoT, Network Functions, and Clouds. FiCloud 2016: 317-324

Cyber-physical systems in IoT
Cloud Infrastructures
 Heavily based on virtual resources
 Loosely couple and tightly couple interactions
 Different degrees of data and control interactions
(Virtual) Cyber-physical systems

IoT Cloud Systems - Elastic
Execution
5TECPS'17, 13 July, 2017, Santa Barbara, USA
http://tuwiendsg.github.io/iCOMOT/
Examples:
Elastic execution is a fundamental aspect in IoT Cloud Systems,
strongly changing methods for design and execution of CPS/IoT

Testing uncertainty in CPS
 Uncertainty:
 due to lack of knowledge, especially due to the complexity and
diversity of resources and interactions in IoT and Cloud systems
 Supporting testing uncertainties and uncertainties analytics
 Emerging novel aspects: data uncertainties (data/data-centric
CPS), elasticity of CPS resources (w.r.t function and composition),
and Governance (related to business/trustworthiness)
 How to discover them and then deal with them?
 Uncertainty analytics through testing (H2020 U-Test objectives)
 Also adaptation of resources considering uncertainties

U-Test Uncertainty
Model
Locality
0..1
Uncertainty
Lifetime
0..1
Random
0..1
0..*
Effect
0..*
Risk
0..1
0..*
Pattern
0..1
0..*
Cause
0..*
0..*
Measurement
0
..
*
substatement
0..*
Uncertainty
Indeterminacy
Source
1..*
source
0..*
Measure
«enumeration»
IndeterminacyNature
nondeterminism
insufficientResolution
missingInfo
composite
unclassified
type1..*
0..*
Belief
Measurement 0..*
0..* beliefDegree
Belief
Statement
Characterizing Uncertainty
Belief Model
Figure sources and relevant technical reports:
https://www.simula.no/file/d12pdf/download
Core Uncertainty
Conceptual Model
Infrastructure Level
Uncertainty Taxonomy
<<import>>
Application Level
Uncertainty Taxonomy
<<import>>

Data
ElasticityGovernance
Important infrastructure uncertainty
classes
Storage
Uncertainty
Infrastructure
Uncertainty
DataDelivery
Uncertainty
Actuation
Uncertainty
DeploymentTime
Uncertainty
ExecutionEnvironment
Uncertainty
Storage
ComplianceUncertainty
StorageQuality
Uncertainty
StorageDependability
Uncertainty
DataQuality
Uncertainty
DataDelivery
DataDeliveryDependability
Uncertainty
ActuationDependability
Uncertainty
EnvironmentDependability
Uncertainty
Actuation
ApplicationDependability
Uncertainty
Runtime
Uncertainty

Modeling and Provisioning IoT Cloud
Infrastructures for Uncertainty Testing
The Topdown approach (MDE/MBT)

IoT and Cloud Resource Profile

Uncertainty
Profile

Then Model-based Testing (MBT)
Mostly do not
consider how to deal
with the
infrastructures of
SUT
Work with specific
static SUT
deployment
Figure source:
Mark Utting and Bruno Legeard. 2006. Practical Model-Based Testing: A Tools
Approach. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

Elastic Infrastructure Testing
The bottom up approach (from IoT Cloud
systems)
Example from:
Daniel Moldovan,Hong-Linh Truong, A Platform for Run-time Health
Verification of Elastic Cyber-physical Systems, The IEEE International
Symposium on Modelling, Analysis and Simulation of Computer and
Telecommunication Systems (MASCOTS 2016), September 19-21, Imperial
College, London, UK

Infrastructure-level Testing
Approach
(From modeling/description)
*testing strategy=testing plan
Daniel Moldovan,Hong-Linh Truong, A
Platform for Run-time Health
Verification of Elastic Cyber-physical
Systems, The IEEE International
Symposium on Modelling, Analysis and
Simulation of Computer and
Telecommunication Systems
(MASCOTS 2016), September 19-21,
Imperial College, London, UK

Run-Time view on structure of
Elasic IoT Cloud systems/CPS

Infrastructure Testing Platform
Domain-Specific Language for Test Plan
Description
name: "TestName"
description: "human readable description"
timeout: timeInSeconds
Triggers
every: number s|m
event: "Added |"Removed“ |”TestFailed” | “TestPassed”| on ComponentIdentifier
…
Execution
executor: [distinct] ComponentIdentifier for ComponentIdentifier [, ComponentIdentifier]+
where ComponentIdentifier:
• Type. [Service | Process | VirtualContainer | VirtualMachine | PhysicalMachine | PhysicalDevice]
• ID.”ComponentID”
• UUID.”ComponentUUID”
Using this DSL: for the test plan, both deployment/configuration and
testing tasks can be described interwoven

Closing the gap: Combining MDE and
Elastic Execution
Currently we are not able to leverage strength of
topdown approach and bottom-up approach for
testing CPS uncertainty

Public cloud infrastructures
Private cloud infrs.
Base Transceiver Station (BTS)
Case Stduy BTS
 Data uncertainty: influence both monitoring and controlling of
equipment
 Large-scale systems (1000+ BTS)
 Flexible back-end clouds
 Generic enough for other applications (e.g., in smart agriculture)
Sensor
IoT
Gateway
MQTT
Broker
BigQuery
Cassandra
Hadoop FS
Actuator
Optimizer Analytics

Key issues
 Current MDE/MBT tools are not enough for
testing uncertainties
 Models, languages and transformation techniques at
the moment do not support on-demand, pay-per-use,
elasticity of IoT and cloud resources
 Elastic Execution hidden from models for
testing
 IoT and cloud infrastructures are not fixed
 SUT and their configurations are changed

Key issues: Two separate worlds
SUT Infrastructure
Development
Deployment Description
Development
IoT/Cloud Infrastructures
Infrastructure Configurations
Resource
Information
Adaptor/Tool
Deployment Scripts
Preparing CPS under Test
Requirements
Artifact
Repository
Figure source:
Mark Utting and Bruno Legeard. 2006. Practical Model-Based Testing: A Tools
Approach. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
?

Combining MDE with Elastic
Execution

Interwoven test exuection and
provisioning
IoT Cloud Infrastructures
Modeling
Uncertainties
and SUT
Uncertainty
Profile
System Under
Test (SUT)
Models
Generating
Uncertainty
Test cases
Deploying/
Configuring
SUT
System Under
Test (SUT)
Test cases
Executing
Tests
IoT/Cloud
Resource
Information
Deploying/
Configuring
Testing Utilities
Test
Utilities

Tool pipelines: from MDE to elastic
execution for testing
Key thoughts:
 Different algorithms to create suitable deployment configurations based
on data uncertainties, cost, and time
 Interactions between testing and elasticity control of IoT and Cloud
services

Our current progress
Consider to generate provisioning configurations
from SUT models
Extracting
model
information &
Generating test
configuration
test
strategies
Infrastructural
IoT Cloud
resources
Provisioning &
Configuring
SUT
Executing
Tests
Test plans &
executors
IoT Units
Cloud
T
E
S
T
S
ProviderA
ActuatorsSensors
VMsAnalytics GWs
IoT Units
Cloud
T
E
S
T
S
ProviderB
ActuatorsSensors
VMsAnalytics GWs
Infrastructure
(Class Diagrams)
Behaviour
(State Machine
diagrams)
Uncertainty
Hong-Linh Truong, Luca Berardinelli, Ivan Pakovic, Georgiana Copil, Modeling and Provisioning IoT Cloud
Infrastructures for Testing Uncertainties, July, 2017 under submission

Example of BTS monitoring

Elastic Test plan
Generic models for Task Executor
Test
Configuration
Test
Executor
Provisioning
Task
Reconfiguration
Task
Test Case
Execution Task
Uncertainty
Metric

Concrete tool pipeline
HINC
(runtime)Docker
Registry
Artifact
Repository
Extracted
information
(JSON)
Deployment
Description
(YAML/TOSCA)
Generate
Configurations &
Deployment
SALSA, Docker
and gcloud
utilities

Examples
"MQTTConfig1": {
"name": "MQTTConfigServer",
"protocolType": "MQTT",
"qosLevel": [],
"type": "CommunicationConfiguration"
}
"MQTTConfig2": {
"name": "MQTTConfigClient",
"protocolType": "MQTT",
"clientID": "",
"serverIP": "35.189.187.208",
"portNumber": 1883,
"topics": ["/gateway/electricity"],
"qosLevel": [2],
"type": "CommunicationConfiguration"
}
services:
ingest:
build: .
volumes:
- ./:/t4u
electricitysensor:
image:
"localhost:5000/t4u/mqttsensor/realsensor:
v01"
iotgateway:
image:
"localhost:5000/t4u/cloudservice/mqttbroke
r:v01"
Enriched model information for
deployment configurations Generated deployment description

Example: using SALSA for deployment and
configurations
http://tuwiendsg.github.io/SALSA/

Summary
 Testing uncertainties need to deal with elasticity and
virtualization
 Different tools for modeling, provisioning and testing
uncertainties based on MDE/MBT and elastic techniques
 Closing gaps in testing uncertainties by introducing
novel methods and tool pipelines
 Our future work
 Prototype our approach for uncertainty testing
 Machine learning/big data analytics for uncertainty analytics
(www.u-test.eu)
Check https://github.com/tuwiendsg/COMOT4U for new
update

Thanks for your
attention!
Hong-Linh Truong
Distributed Systems Group
TU Wien
rdsea.github.io

Testing Uncertainty of Cyber-Physical Systems in IoT Cloud Infrastructures: Combining Model-Driven Engineering and Elastic Execution

More Related Content

Similar to Testing Uncertainty of Cyber-Physical Systems in IoT Cloud Infrastructures: Combining Model-Driven Engineering and Elastic Execution (20)

More from Hong-Linh Truong (20)

Recently uploaded (20)

Testing Uncertainty of Cyber-Physical Systems in IoT Cloud Infrastructures: Combining Model-Driven Engineering and Elastic Execution