SlideShare a Scribd company logo
10/4/2021 1
Demetris Trihinas
trihinas.d@unic.ac.cy
1
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Composable Energy Modeling for
ML-Driven Drone Applications
The HornEt Framework
Demetris Trihinas, Michalis Agathocleous, Karlen Avogian
Department of Computer Science
ailab @ University of Nicosia
{trihinas.d, agathocleous.m, avogian.k}@unic.ac.cy
10/4/2021 2
Demetris Trihinas
trihinas.d@unic.ac.cy
2
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Drone Technology
• Plethora of Drones coming in different sizes, shape,
capabilities and prices.
• Birth of new and emerging AI-fuelled applications.
• Mckinsey (2021) largest stake of funding for commercial
drone tech (US) is for on-board data mgmt and analysis
empowered by ML and AI in general.
10/4/2021 3
Demetris Trihinas
trihinas.d@unic.ac.cy
3
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Face Detection and Recognition
Image from DroneFaces dataset
Python face_recognition library
10/4/2021 4
Demetris Trihinas
trihinas.d@unic.ac.cy
4
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Crowd Detection
VisDrone 2020 dataset
CNN pre-trained model
10/4/2021 5
Demetris Trihinas
trihinas.d@unic.ac.cy
5
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Power Consumption
• Data processing and dissemination key energy drains in embedded and
mobile devices.
• IoT devices are usually battery-powered which means intense processing
leads to less battery-life…
Low-Cost Adaptive Monitoring Techniques for the Internet of Things. D. Trihinas, G. Pallis and M. D. Dikaiakos, IEEE Transactions on Services Computing, 2018.
10/4/2021 6
Demetris Trihinas
trihinas.d@unic.ac.cy
6
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
ML Comes in Many Forms…
VisDrone 2020 dataset
Algorithm Accuracy (%) Median Inference
Delay (ms)
Flight Time
No ML 31min
CNN 95 1361 24min 48s
FIGs 60 274 27min 22s
DJI Mavic 2 Pro
10/4/2021 7
Demetris Trihinas
trihinas.d@unic.ac.cy
7
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
ML on Drones
• Will it run on my resource-constraint
drone?
• How will it affect energy consumption
and battery autonomy?
• Trade-off between running on-device vs remote inference?
• Could a different (less-intensive) algorithm work by trading
accuracy for flight time?
I have no
idea…
From VisDrone 2020 dataset
10/4/2021 8
Demetris Trihinas
trihinas.d@unic.ac.cy
8
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
HornEt: Energy Modeling and Profiling
1 f r om f l ockai . dr ones. cont r ol l er s i m
por t DJI M
avi c2
2 f r om f l ockai . dr ones. pi l ot s i m
por t
Boundi ngBoxRandom
PosPi l ot
3 f r om f l ockai . dr ones. m
oni t or i ng i m
por t Ener gyPr obe
4
5 def M
yDr oneM
LExper i m
ent ( ) :
6 aut opi l ot = Boundi ngBoxRandom
PosPi l ot ( . . . )
7 m
on = [ Ener gyPr obe( . . ) ]
8 m
ydr one = DJI M
avi c2( pi l ot =aut opi l ot , pr obes=m
on)
9 m
ydr one. act i vat e( ) #st ar t si m
ul at i on
10 m
ydr one. t akeof f ( )
11 whi l e m
ydr one. f l yi ng( ) :
12 . . .
13 m
et r i cs = m
ydr one. get Pr obe( " Ener gy" ) . val ues( )
14 eTot al = m
et r i cs. get ( " t ot al " )
15 eM
ot or s = m
et r i cs. get ( " m
ot or s" )
16 ePr oc = m
et r i cs. get ( " pr ocessi ng" )
17 eCom
m = m
et r i cs. get ( " com
m
uni cat i on" )
Fig. 2: Energy Model Dentrogram for Listing 2
In order to avoid having to manually configure multiple
• Composable modeling for multi-grain
energy reporting.
• Reusable template repository for
specific drones.
Fig. 1: High-Level and Abstract Overview of HornEt
I/O (e.g., writing to flash drive), while Pss denotes the power
driving peripheral sensing devices with essentially only the
image/video unit consuming measurable energy.
III. THE HORNET FRAMEWORK
Finally, when the drone simulation starts, the energy model
can be continuously updated at runtime by “pushing” to
HornEt monitoring data. To achieve this, currently, HornEt
accepts data through the monitoring hooks (denoted as probes)
of FlockAI, albeit, if the integration endpoint is implemented,
any other monitoring solution can be used. In regards to
exporting energy measurements, HornEt can print data on the
console of the Webots (stdout/stderr) during runtime and/or
format and export the entire experiment log in plain text or
formatted with both json and pandas dataframes supported.
B. Implementation
Section II gives a brief, but elaborative, view on the
complexity of energy modeling for drones. As shown, imple-
menting such a model requires the definition of a plethora of
parameters and the measurement of the utilization of several
resources. In order to provide a simple “jump-start” energy
model so that novice users can quickly perform emulated
measurements, but at the same time, provide a comprehen-
10/4/2021 9
Demetris Trihinas
trihinas.d@unic.ac.cy
9
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Drone Energy Modeling
m
por t DJI M
avi c2
m
por t Ener gyPr obe
osPi l ot ( . . . )
pi l ot , pr obes=m
on)
at i on
Ener gy" ) . val ues( )
s" )
ng" )
cat i on" )
eCom
m
)
Fig. 2: Energy Model Dentrogram for Listing 2
In order to avoid having to manually configure multiple
power and/or voltage/current levels just for a ML practitioner
Cant ignore with ML
m
por t DJI M
avi c2
m
por t Ener gyPr obe
osPi l ot ( . . . )
pi l ot , pr obes=m
on)
at i on
Ener gy" ) . val ues( )
s" )
ng" )
cat i on" )
eCom
m
)
Fig. 2: Energy Model Dentrogram for Listing 2
In order to avoid having to manually configure multiple
power and/or voltage/current levels just for a ML practitioner
…
A lot of params to
configure…
m
por t DJI M
avi c2
m
por t Ener gyPr obe
osPi l ot ( . . . )
pi l ot , pr obes=m
on)
at i on
Ener gy" ) . val ues( )
s" )
ng" )
cat i on" )
eCom
m
)
Fig. 2: Energy Model Dentrogram for Listing 2
In order to avoid having to manually configure multiple
power and/or voltage/current levels just for a ML practitioner
What if I want to ignore params of decompose
into deeper levels?
10/4/2021 10
Demetris Trihinas
trihinas.d@unic.ac.cy
10
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Composable Energy Modeling
16 ePr oc = m
et r i cs. get ( " pr ocessi ng" )
17 eCom
m = m
et r i cs. get ( " com
m
uni cat i on" )
18 pr i nt ( eTot al , eM
ot or s, ePr oc, eCom
m
)
19 . . .
20 m
ydr one. t er m
( )
Listing 1: Exemplary FlockAI Blueprint in Python
1 f r om hor net . ener gy i m
por t Ener gyM
odel , Et ot al
2 f r om hor net . ener gy. pr ocessi ng i m
por t Epr oc
3 . . .
4
5 def i ni t Ener gyM
odel ( ) :
6 . . .
7 m
odel = Ener gyM
odel ( )
8 m
odel . addCom
ponent s( [ EM
ot or s( ) , EPr oc( ) , ECom
m
( ) ] )
9
10 cust Pr oc = EPr oc( nam
e=" cust om
- epr oc" )
11 cust Pr oc. r em
oveCom
ponent s( [ " sensor s" , " i o" ] )
12 ef c = cust Pr oc. get Com
ponent ( " f c" )
13 ef c. set desc( " DJI A3 f l i ght cont r ol l er " )
14 ef c. set power ( 8. 3)
15 esoc = cust Pr oc. get Com
ponent ( " soc" )
16 esoc. get Com
ponent ( " cpu- act i ve" ) . set power ( 8)
17 esoc. get Com
ponent ( " cpu- i dl e" ) . set power ( 4)
18
19 m
odel . updat eCom
ponent ( " pr oc" , cust Pr oc)
20 . . .
21 r et ur n m
odel
22 . . .
23 m
ym
odel = i ni t Ener gyM
odel ( )
24 m
ydr one. at t achEner gyM
odel ( m
ym
odel )
Listing 2: Configuring and Attaching HornEt Model to Drone
In order to avoid having to manually configure multiple
power and/or voltage/current levels just for a ML practitioner
to be able to “see” results on his/her emulator, HornEt comes
with a number of pre-filled energy profiles for a number of
popular drones. In the current release these include drones
from the DJI drone manufacturer. Hence, if a user opts to
use a readily available energy profile then the configuration
presented in Listing 2 can be skipped or reduced just to
a couple of lines targeting any specific component the user
wishes to amend.
IV. EVALUATION
This section provides a comprehensive study demonstrating
the main contributions of our work by introducing a realistic
use-case under-going two different deployments so that a ML
practitioner can truly assess the energy consumption KPIs of
his/her ML-driven drone application are met before introduced
into production.
A. Use-Case Scenario
Suppose aML practitioner must test theefficiency of his/her
developed ML solution for a hypothetical -but very realistic-
use-case that employs face recognition for tracking human
targets’ from the air (i.e., social distancing). Towards this, the
drone is given a flight plan including a (bounded) area of
interest and a starting location. The drone will then fly above
Python SDK
Create energy model and
add/remove components
Customize components
(naming, add/remove sub-
components, set values, …)
Reusable templates for DJI
Drones
10/4/2021 11
Demetris Trihinas
trihinas.d@unic.ac.cy
11
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Drone Benchmarking Suite
Python SDK to
design drone
testbeds
Or use pre-
built templates
Deploy ML
models and
configure on-
board/remote
inference
Collect analytic insights
https://unic-ailab.github.io/flockai/
10/4/2021 12
Demetris Trihinas
trihinas.d@unic.ac.cy
12
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Use-Case: Face Detection and Recognition
Image from DroneFaces dataset
Python face_recognition library
DJI Mavic 2 Pro
Running in Webots
with FlockAI and
HornEt introduced
example
10/4/2021 13
Demetris Trihinas
trihinas.d@unic.ac.cy
13
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Experiment Results
Fig. 3: Energy Decomposed into Components (No ML) Fig. 5: Processing Energy Decomposed into its Sub-
Components (On-Board ML)
In these figures, the energy consumption is decomposed by
HornEt into the energy consumed by the motors, processing
and communication components of the drone. From both of
these figures, we immediately observe that Em otor s is the
dominating energy consuming factor. However, despite thesig-
nificance of Em otor s when face recognition is enabled, Epr oc
contributes to approximately 18% of the total energy and this
is far from being considered as negligible. Compared to the
No-ML flight, the processing overhead is increased by 13%.
Note that there is also a 2% increase in the communication
Fig. 3: Energy Decomposed into Components (No ML)
Fig. 4: Energy Decomposed into Components (On-Board ML)
attached to the drone is the one presented in Section III-B
and depicted in Listing 2. As a reminder, the model considers
three main energy components: motors, processing and com-
munication. The only exception from Listing 2 is that we will
not ignore the I/O sub-component for the processing overhead.
Moreover, all experiment scenarios were run 10 times with the
average values presented in the result visualizations.
B. Experiment 1: On-Board Inference
In this experiment, the ML practitioner would like to
evaluate the impact of his/her algorithm in terms of energy
consumption for theutilized drone. Hence, thefacerecognition
Fig. 5: Processing Energ
Components (On-Board ML
In these figures, the energy
HornEt into the energy con
and communication compon
these figures, we immediat
dominating energy consumin
nificance of Em ot or s when f
contributes to approximately
is far from being considere
No-ML flight, the processin
Note that there is also a 2%
overhead due to the dissem
2+ people detected.
In order to further und
algorithm on the energy co
HornEt to further decompo
sub-components. These, are
observe that the majority
consumed by the CPU when
of course, is attributed to th
inference algorithm. Moreo
fraction of the overhead is
flight controller (⇠10%) and
state (⇠9%), while the I/O o
images are not stored.
C. Experiment 2: Remote In
No ML) Fig. 5: Processing Energy Decomposed into its Sub-
Components (On-Board ML)
Fig. 6: Energy Decomposed into Components (Remote ML)
execution of ML algorithms
ful of frameworksarenow ai
consumption data on top of
DronesBench [18] provides
but it is tailored to testing d
take-off and landing in cont
Interestingly, Seewald et
tool for coarse-grained energ
usage as a function of com
interesting tool, presented v
powprofiler implementation
processing component and t
cpu and gpu as reference d
introduce FANETSim [9],
10/4/2021 14
Demetris Trihinas
trihinas.d@unic.ac.cy
14
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Experiment Results
Fig. 6: Energy Decomposed into Components (Remote ML)
Fig. 7: Inference Latency
wireless link is open throughout the flight and a data payload
is exchanged with the ground station. In turn, we observe
something that may not be immediately evident. Despite that
inference is performed remotely, Epr oc has not dropped to the
value it had when the No-ML flight was executed. This is
due to the data preparation tasks of the sensory data and the
ful of frameworks arenow aiming to provide users wit
consumption data on top of an emulation stack. Spe
DronesBench [18] provides detailed power consumpt
but it is tailored to testing drone hardware componen
take-off and landing in controllable (emulated) setting
Interestingly, Seewald et al. introduce powprofiler
tool for coarse-grained energy modeling that describe
usage as a function of component configuration. W
interesting tool, presented via a drone use-case scen
powprofiler implementation is limited to only suppor
processing component and this component only consi
cpu and gpu as reference dimensions. Finally, Trop
introduce FANETSim [9], a java-based simulation t
enables the testing of drone-to-ground station and
to-drone communication overhead. Overhead is exam
terms of message counts and energy consumption. H
the energy model only considers energy relevant to fl
drone (Em ot or s) which is modeled with drone veloci
the only dependent variable.
VI. CONCLUSIONS AND FUTURE WORK
With the remarkable rise of ML empowered dron
detailed modeling for both the mechanical and e
components of a drone are required so that the ener
sumption is estimated with a small degree of inaccura
using an emulator during application design. In th
we have introduced HornEt, a python framework
the production of realistic energy models during the
of ML-driven drone applications. Depending on the
granularity that the ML practitioner requires, the energ
can be customized by adding, amending or dropping
monitorable components of the drone. In turn, HornE
with ready-to-use energy profiles for various popula
so that no additional effort is required for new use
10/4/2021 15
Demetris Trihinas
trihinas.d@unic.ac.cy
15
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Conclusions
• Rise of ML-driven drone applications.
• Difficult to create testing environments for drone applications.
• HornEt is a python framework enabling the production of energy models during the
testing of ML-driven drone applications.
• Depending on the level of required granularity, the energy model can be customized by
adding, amending or dropping different monitorable components of the drone.
• HornEt comes with ready-to-use energy profiles for various popular drones so that no
additional effort is required for new users when getting started.
10/4/2021 16
Demetris Trihinas
trihinas.d@unic.ac.cy
16
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Composable Energy Modeling for
ML-Driven Drone Applications
The HornEt Framework
Thank You!
https://unic-ailab.github.io/flockai/
10/4/2021 17
Demetris Trihinas
trihinas.d@unic.ac.cy
17
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Backup Slides
10/4/2021 18
Demetris Trihinas
trihinas.d@unic.ac.cy
18
The HornEt Framework | TDIS@IC2E 2021
Department of
Computer Science
Related Work
• Several benchmarking suites for testing drone hardware and flight control
(i.e., jmavsim, gazebo, AirSim) but no ML deployment and testing.
• For energy testing, Powprofiler can test processing component, while
FANETSim tests communication component.

More Related Content

What's hot (19)

PDF
A SURVEY - COMPARISON OF MULTIPLIERS USING DIFFERENT LOGIC STYLE
Editor IJMTER
 
PPTX
ANISH_and_DR.DANIEL_augmented_reality_presentation
Anish Patel
 
PDF
Low Energy Task Scheduling based on Work Stealing
LEGATO project
 
PDF
cug2011-praveen
Praveen Narayanan
 
PDF
Software Based calculations of Electrical Machine Design
vivatechijri
 
PDF
Stream Processing
FogGuru MSCA Project
 
PDF
Moldable pipelines for CNNs on heterogeneous edge devices
LEGATO project
 
PDF
Static Energy Prediction in Software: A Worst-Case Scenario Approach
GreenLabAtDI
 
PPTX
Countdown to Zero - Counter Use Cases in Aerospike
Ronen Botzer
 
PDF
Integrated Model Discovery and Self-Adaptation of Robots
Pooyan Jamshidi
 
PDF
Real-Time Pedestrian Detection Using Apache Storm in a Distributed Environment
csandit
 
PPTX
Image Registration Based on CCRE for Remote Sensing Images
Sujoy Sarathi
 
PDF
Mini Project on 4 BIT SERIAL MULTIPLIER
j naga sai
 
PDF
Device Data Directory and Asynchronous execution: A path to heterogeneous com...
LEGATO project
 
PDF
Delivering near real time mobility insights at swisscom
François Garillot
 
PDF
SPEED-UP IMPROVEMENT USING PARALLEL APPROACH IN IMAGE STEGANOGRAPHY
csandit
 
PPTX
Exploring Modeling - Best Practices with Aerospike Data Types
Ronen Botzer
 
PDF
Mobility insights at Swisscom - Understanding collective mobility in Switzerland
François Garillot
 
PDF
A Review: Metaheuristic Technique in Cloud Computing
IRJET Journal
 
A SURVEY - COMPARISON OF MULTIPLIERS USING DIFFERENT LOGIC STYLE
Editor IJMTER
 
ANISH_and_DR.DANIEL_augmented_reality_presentation
Anish Patel
 
Low Energy Task Scheduling based on Work Stealing
LEGATO project
 
cug2011-praveen
Praveen Narayanan
 
Software Based calculations of Electrical Machine Design
vivatechijri
 
Stream Processing
FogGuru MSCA Project
 
Moldable pipelines for CNNs on heterogeneous edge devices
LEGATO project
 
Static Energy Prediction in Software: A Worst-Case Scenario Approach
GreenLabAtDI
 
Countdown to Zero - Counter Use Cases in Aerospike
Ronen Botzer
 
Integrated Model Discovery and Self-Adaptation of Robots
Pooyan Jamshidi
 
Real-Time Pedestrian Detection Using Apache Storm in a Distributed Environment
csandit
 
Image Registration Based on CCRE for Remote Sensing Images
Sujoy Sarathi
 
Mini Project on 4 BIT SERIAL MULTIPLIER
j naga sai
 
Device Data Directory and Asynchronous execution: A path to heterogeneous com...
LEGATO project
 
Delivering near real time mobility insights at swisscom
François Garillot
 
SPEED-UP IMPROVEMENT USING PARALLEL APPROACH IN IMAGE STEGANOGRAPHY
csandit
 
Exploring Modeling - Best Practices with Aerospike Data Types
Ronen Botzer
 
Mobility insights at Swisscom - Understanding collective mobility in Switzerland
François Garillot
 
A Review: Metaheuristic Technique in Cloud Computing
IRJET Journal
 

Similar to Composable Energy Modeling for ML-Driven Drone Applications (20)

PDF
Rapidly Testing ML-Driven Drone Applications - The FlockAI Framework
Demetris Trihinas
 
PPTX
Python - Deep Learning library-PyTorch.pptx
akbarsayeedi32
 
PPTX
Designing a machine learning algorithm for Apache Spark
Marco Gaido
 
PDF
Open Source Predictive Analytics Pipeline with Apache NiFi and MiniFi Princeton
Timothy Spann
 
PDF
Resume - Hsu, Hsiu-Wei
Hsiu Wei Hsu
 
PDF
Artificial Intelligence And Sustainable Energy Systems Volume Ii Luis Hernnde...
dhurwekazaka
 
PPTX
Transfer Leaning Using Pytorch synopsis Minor project pptx
Ankit Gupta
 
PDF
Mission planning of autonomous quadrotors
Ivano Malavolta
 
PPTX
Energy efficiency in wireless sensor network(ce 16 aniket choudhury)
अनिकेत चौधरी
 
PPT
Hairong Qi V Swaminathan
FNian
 
PDF
A Smart Home Testbed for Evaluating XAI with Non-Experts
CHAIproject
 
PPTX
GWU_DRONE AND AI HOW DRONE AND AI RELATED
imman gwu
 
PDF
Deep-Learning-with-PydddddddddddddTorch.pdf
drjigarsoni28
 
PDF
Pydata DC 2018 (Skorch - A Union of Scikit-learn and PyTorch)
Thomas Fan
 
PDF
Enterprise Data Science at Scale
Artem Ervits
 
PDF
ECML PKDD 2021 ML meets IoT Tutorial Part II: Creating ML based Self learning...
Bharath Sudharsan
 
PDF
Tensorflow 2.0 and Coral Edge TPU
Andrés Leonardo Martinez Ortiz
 
PPTX
Machines Can Learn - a Practical Take on Machine Intelligence Using Spring Cl...
Christian Tzolov
 
PDF
Webinar: Machine Learning para Microcontroladores
Embarcados
 
PDF
Neural Networks in the Wild: Handwriting Recognition
John Liu
 
Rapidly Testing ML-Driven Drone Applications - The FlockAI Framework
Demetris Trihinas
 
Python - Deep Learning library-PyTorch.pptx
akbarsayeedi32
 
Designing a machine learning algorithm for Apache Spark
Marco Gaido
 
Open Source Predictive Analytics Pipeline with Apache NiFi and MiniFi Princeton
Timothy Spann
 
Resume - Hsu, Hsiu-Wei
Hsiu Wei Hsu
 
Artificial Intelligence And Sustainable Energy Systems Volume Ii Luis Hernnde...
dhurwekazaka
 
Transfer Leaning Using Pytorch synopsis Minor project pptx
Ankit Gupta
 
Mission planning of autonomous quadrotors
Ivano Malavolta
 
Energy efficiency in wireless sensor network(ce 16 aniket choudhury)
अनिकेत चौधरी
 
Hairong Qi V Swaminathan
FNian
 
A Smart Home Testbed for Evaluating XAI with Non-Experts
CHAIproject
 
GWU_DRONE AND AI HOW DRONE AND AI RELATED
imman gwu
 
Deep-Learning-with-PydddddddddddddTorch.pdf
drjigarsoni28
 
Pydata DC 2018 (Skorch - A Union of Scikit-learn and PyTorch)
Thomas Fan
 
Enterprise Data Science at Scale
Artem Ervits
 
ECML PKDD 2021 ML meets IoT Tutorial Part II: Creating ML based Self learning...
Bharath Sudharsan
 
Tensorflow 2.0 and Coral Edge TPU
Andrés Leonardo Martinez Ortiz
 
Machines Can Learn - a Practical Take on Machine Intelligence Using Spring Cl...
Christian Tzolov
 
Webinar: Machine Learning para Microcontroladores
Embarcados
 
Neural Networks in the Wild: Handwriting Recognition
John Liu
 
Ad

More from Demetris Trihinas (16)

PPTX
Towards Energy and Carbon Footprint and Testing for AI-driven IoT Services
Demetris Trihinas
 
PDF
StreamSight: A Query-Driven Framework Extending Streaming IoT Analytics to th...
Demetris Trihinas
 
PDF
Telling a Story – or Even Propaganda – Through Data Visualization
Demetris Trihinas
 
PDF
Machine Learning Introduction
Demetris Trihinas
 
PPTX
Απεικόνιση και Αλληλεπίδραση Δεδομένων Μεγάλου Όγκου με Διαδραστικούς Χάρτες
Demetris Trihinas
 
PDF
The Data Science Process: From Mining Raw Data to Story Visualization
Demetris Trihinas
 
PDF
From Mining Raw Data to Story Visualization
Demetris Trihinas
 
PDF
Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service ...
Demetris Trihinas
 
PPTX
Low-Cost Approximate and Adaptive Monitoring Techniques for the Internet of T...
Demetris Trihinas
 
PPTX
Adam - Adaptive Monitoring in 5min
Demetris Trihinas
 
PPTX
Low-Cost Adaptive Monitoring Techniques for the Internet of Things
Demetris Trihinas
 
PPTX
AdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT Devices
Demetris Trihinas
 
PPTX
Find A Project
Demetris Trihinas
 
PPTX
Cloud Elasticity and the CELAR Project
Demetris Trihinas
 
PDF
[ccgrid2014] JCatascopia: Monitoring Elastically Adaptive Applications in the...
Demetris Trihinas
 
PPTX
[SummerSoc 2014] Monitoring Elastic Cloud Services
Demetris Trihinas
 
Towards Energy and Carbon Footprint and Testing for AI-driven IoT Services
Demetris Trihinas
 
StreamSight: A Query-Driven Framework Extending Streaming IoT Analytics to th...
Demetris Trihinas
 
Telling a Story – or Even Propaganda – Through Data Visualization
Demetris Trihinas
 
Machine Learning Introduction
Demetris Trihinas
 
Απεικόνιση και Αλληλεπίδραση Δεδομένων Μεγάλου Όγκου με Διαδραστικούς Χάρτες
Demetris Trihinas
 
The Data Science Process: From Mining Raw Data to Story Visualization
Demetris Trihinas
 
From Mining Raw Data to Story Visualization
Demetris Trihinas
 
Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service ...
Demetris Trihinas
 
Low-Cost Approximate and Adaptive Monitoring Techniques for the Internet of T...
Demetris Trihinas
 
Adam - Adaptive Monitoring in 5min
Demetris Trihinas
 
Low-Cost Adaptive Monitoring Techniques for the Internet of Things
Demetris Trihinas
 
AdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT Devices
Demetris Trihinas
 
Find A Project
Demetris Trihinas
 
Cloud Elasticity and the CELAR Project
Demetris Trihinas
 
[ccgrid2014] JCatascopia: Monitoring Elastically Adaptive Applications in the...
Demetris Trihinas
 
[SummerSoc 2014] Monitoring Elastic Cloud Services
Demetris Trihinas
 
Ad

Recently uploaded (20)

PDF
Blitz Campinas - Dia 24 de maio - Piettro.pdf
fabigreek
 
PPTX
Customer Segmentation: Seeing the Trees and the Forest Simultaneously
Sione Palu
 
PDF
Top Civil Engineer Canada Services111111
nengineeringfirms
 
PDF
apidays Munich 2025 - Making Sense of AI-Ready APIs in a Buzzword World, Andr...
apidays
 
PPTX
IP_Journal_Articles_2025IP_Journal_Articles_2025
mishell212144
 
PPTX
MR and reffffffvvvvvvvfversal_083605.pptx
manjeshjain
 
PPTX
White Blue Simple Modern Enhancing Sales Strategy Presentation_20250724_21093...
RamNeymarjr
 
PPTX
Presentation (1) (1).pptx k8hhfftuiiigff
karthikjagath2005
 
PDF
202501214233242351219 QASS Session 2.pdf
lauramejiamillan
 
PPTX
Fluvial_Civilizations_Presentation (1).pptx
alisslovemendoza7
 
PDF
Classifcation using Machine Learning and deep learning
bhaveshagrawal35
 
PPTX
M1-T1.pptxM1-T1.pptxM1-T1.pptxM1-T1.pptx
teodoroferiarevanojr
 
PPTX
UVA-Ortho-PPT-Final-1.pptx Data analytics relevant to the top
chinnusindhu1
 
PPTX
Probability systematic sampling methods.pptx
PrakashRajput19
 
PPTX
Multiscale Segmentation of Survey Respondents: Seeing the Trees and the Fores...
Sione Palu
 
PDF
apidays Munich 2025 - Developer Portals, API Catalogs, and Marketplaces, Miri...
apidays
 
PPTX
Insurance-Analytics-Branch-Dashboard (1).pptx
trivenisapate02
 
PPTX
Data-Users-in-Database-Management-Systems (1).pptx
dharmik832021
 
PPTX
The whitetiger novel review for collegeassignment.pptx
DhruvPatel754154
 
PDF
Blue Futuristic Cyber Security Presentation.pdf
tanvikhunt1003
 
Blitz Campinas - Dia 24 de maio - Piettro.pdf
fabigreek
 
Customer Segmentation: Seeing the Trees and the Forest Simultaneously
Sione Palu
 
Top Civil Engineer Canada Services111111
nengineeringfirms
 
apidays Munich 2025 - Making Sense of AI-Ready APIs in a Buzzword World, Andr...
apidays
 
IP_Journal_Articles_2025IP_Journal_Articles_2025
mishell212144
 
MR and reffffffvvvvvvvfversal_083605.pptx
manjeshjain
 
White Blue Simple Modern Enhancing Sales Strategy Presentation_20250724_21093...
RamNeymarjr
 
Presentation (1) (1).pptx k8hhfftuiiigff
karthikjagath2005
 
202501214233242351219 QASS Session 2.pdf
lauramejiamillan
 
Fluvial_Civilizations_Presentation (1).pptx
alisslovemendoza7
 
Classifcation using Machine Learning and deep learning
bhaveshagrawal35
 
M1-T1.pptxM1-T1.pptxM1-T1.pptxM1-T1.pptx
teodoroferiarevanojr
 
UVA-Ortho-PPT-Final-1.pptx Data analytics relevant to the top
chinnusindhu1
 
Probability systematic sampling methods.pptx
PrakashRajput19
 
Multiscale Segmentation of Survey Respondents: Seeing the Trees and the Fores...
Sione Palu
 
apidays Munich 2025 - Developer Portals, API Catalogs, and Marketplaces, Miri...
apidays
 
Insurance-Analytics-Branch-Dashboard (1).pptx
trivenisapate02
 
Data-Users-in-Database-Management-Systems (1).pptx
dharmik832021
 
The whitetiger novel review for collegeassignment.pptx
DhruvPatel754154
 
Blue Futuristic Cyber Security Presentation.pdf
tanvikhunt1003
 

Composable Energy Modeling for ML-Driven Drone Applications

  • 1. 10/4/2021 1 Demetris Trihinas [email protected] 1 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Composable Energy Modeling for ML-Driven Drone Applications The HornEt Framework Demetris Trihinas, Michalis Agathocleous, Karlen Avogian Department of Computer Science ailab @ University of Nicosia {trihinas.d, agathocleous.m, avogian.k}@unic.ac.cy
  • 2. 10/4/2021 2 Demetris Trihinas [email protected] 2 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Drone Technology • Plethora of Drones coming in different sizes, shape, capabilities and prices. • Birth of new and emerging AI-fuelled applications. • Mckinsey (2021) largest stake of funding for commercial drone tech (US) is for on-board data mgmt and analysis empowered by ML and AI in general.
  • 3. 10/4/2021 3 Demetris Trihinas [email protected] 3 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Face Detection and Recognition Image from DroneFaces dataset Python face_recognition library
  • 4. 10/4/2021 4 Demetris Trihinas [email protected] 4 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Crowd Detection VisDrone 2020 dataset CNN pre-trained model
  • 5. 10/4/2021 5 Demetris Trihinas [email protected] 5 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Power Consumption • Data processing and dissemination key energy drains in embedded and mobile devices. • IoT devices are usually battery-powered which means intense processing leads to less battery-life… Low-Cost Adaptive Monitoring Techniques for the Internet of Things. D. Trihinas, G. Pallis and M. D. Dikaiakos, IEEE Transactions on Services Computing, 2018.
  • 6. 10/4/2021 6 Demetris Trihinas [email protected] 6 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science ML Comes in Many Forms… VisDrone 2020 dataset Algorithm Accuracy (%) Median Inference Delay (ms) Flight Time No ML 31min CNN 95 1361 24min 48s FIGs 60 274 27min 22s DJI Mavic 2 Pro
  • 7. 10/4/2021 7 Demetris Trihinas [email protected] 7 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science ML on Drones • Will it run on my resource-constraint drone? • How will it affect energy consumption and battery autonomy? • Trade-off between running on-device vs remote inference? • Could a different (less-intensive) algorithm work by trading accuracy for flight time? I have no idea… From VisDrone 2020 dataset
  • 8. 10/4/2021 8 Demetris Trihinas [email protected] 8 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science HornEt: Energy Modeling and Profiling 1 f r om f l ockai . dr ones. cont r ol l er s i m por t DJI M avi c2 2 f r om f l ockai . dr ones. pi l ot s i m por t Boundi ngBoxRandom PosPi l ot 3 f r om f l ockai . dr ones. m oni t or i ng i m por t Ener gyPr obe 4 5 def M yDr oneM LExper i m ent ( ) : 6 aut opi l ot = Boundi ngBoxRandom PosPi l ot ( . . . ) 7 m on = [ Ener gyPr obe( . . ) ] 8 m ydr one = DJI M avi c2( pi l ot =aut opi l ot , pr obes=m on) 9 m ydr one. act i vat e( ) #st ar t si m ul at i on 10 m ydr one. t akeof f ( ) 11 whi l e m ydr one. f l yi ng( ) : 12 . . . 13 m et r i cs = m ydr one. get Pr obe( " Ener gy" ) . val ues( ) 14 eTot al = m et r i cs. get ( " t ot al " ) 15 eM ot or s = m et r i cs. get ( " m ot or s" ) 16 ePr oc = m et r i cs. get ( " pr ocessi ng" ) 17 eCom m = m et r i cs. get ( " com m uni cat i on" ) Fig. 2: Energy Model Dentrogram for Listing 2 In order to avoid having to manually configure multiple • Composable modeling for multi-grain energy reporting. • Reusable template repository for specific drones. Fig. 1: High-Level and Abstract Overview of HornEt I/O (e.g., writing to flash drive), while Pss denotes the power driving peripheral sensing devices with essentially only the image/video unit consuming measurable energy. III. THE HORNET FRAMEWORK Finally, when the drone simulation starts, the energy model can be continuously updated at runtime by “pushing” to HornEt monitoring data. To achieve this, currently, HornEt accepts data through the monitoring hooks (denoted as probes) of FlockAI, albeit, if the integration endpoint is implemented, any other monitoring solution can be used. In regards to exporting energy measurements, HornEt can print data on the console of the Webots (stdout/stderr) during runtime and/or format and export the entire experiment log in plain text or formatted with both json and pandas dataframes supported. B. Implementation Section II gives a brief, but elaborative, view on the complexity of energy modeling for drones. As shown, imple- menting such a model requires the definition of a plethora of parameters and the measurement of the utilization of several resources. In order to provide a simple “jump-start” energy model so that novice users can quickly perform emulated measurements, but at the same time, provide a comprehen-
  • 9. 10/4/2021 9 Demetris Trihinas [email protected] 9 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Drone Energy Modeling m por t DJI M avi c2 m por t Ener gyPr obe osPi l ot ( . . . ) pi l ot , pr obes=m on) at i on Ener gy" ) . val ues( ) s" ) ng" ) cat i on" ) eCom m ) Fig. 2: Energy Model Dentrogram for Listing 2 In order to avoid having to manually configure multiple power and/or voltage/current levels just for a ML practitioner Cant ignore with ML m por t DJI M avi c2 m por t Ener gyPr obe osPi l ot ( . . . ) pi l ot , pr obes=m on) at i on Ener gy" ) . val ues( ) s" ) ng" ) cat i on" ) eCom m ) Fig. 2: Energy Model Dentrogram for Listing 2 In order to avoid having to manually configure multiple power and/or voltage/current levels just for a ML practitioner … A lot of params to configure… m por t DJI M avi c2 m por t Ener gyPr obe osPi l ot ( . . . ) pi l ot , pr obes=m on) at i on Ener gy" ) . val ues( ) s" ) ng" ) cat i on" ) eCom m ) Fig. 2: Energy Model Dentrogram for Listing 2 In order to avoid having to manually configure multiple power and/or voltage/current levels just for a ML practitioner What if I want to ignore params of decompose into deeper levels?
  • 10. 10/4/2021 10 Demetris Trihinas [email protected] 10 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Composable Energy Modeling 16 ePr oc = m et r i cs. get ( " pr ocessi ng" ) 17 eCom m = m et r i cs. get ( " com m uni cat i on" ) 18 pr i nt ( eTot al , eM ot or s, ePr oc, eCom m ) 19 . . . 20 m ydr one. t er m ( ) Listing 1: Exemplary FlockAI Blueprint in Python 1 f r om hor net . ener gy i m por t Ener gyM odel , Et ot al 2 f r om hor net . ener gy. pr ocessi ng i m por t Epr oc 3 . . . 4 5 def i ni t Ener gyM odel ( ) : 6 . . . 7 m odel = Ener gyM odel ( ) 8 m odel . addCom ponent s( [ EM ot or s( ) , EPr oc( ) , ECom m ( ) ] ) 9 10 cust Pr oc = EPr oc( nam e=" cust om - epr oc" ) 11 cust Pr oc. r em oveCom ponent s( [ " sensor s" , " i o" ] ) 12 ef c = cust Pr oc. get Com ponent ( " f c" ) 13 ef c. set desc( " DJI A3 f l i ght cont r ol l er " ) 14 ef c. set power ( 8. 3) 15 esoc = cust Pr oc. get Com ponent ( " soc" ) 16 esoc. get Com ponent ( " cpu- act i ve" ) . set power ( 8) 17 esoc. get Com ponent ( " cpu- i dl e" ) . set power ( 4) 18 19 m odel . updat eCom ponent ( " pr oc" , cust Pr oc) 20 . . . 21 r et ur n m odel 22 . . . 23 m ym odel = i ni t Ener gyM odel ( ) 24 m ydr one. at t achEner gyM odel ( m ym odel ) Listing 2: Configuring and Attaching HornEt Model to Drone In order to avoid having to manually configure multiple power and/or voltage/current levels just for a ML practitioner to be able to “see” results on his/her emulator, HornEt comes with a number of pre-filled energy profiles for a number of popular drones. In the current release these include drones from the DJI drone manufacturer. Hence, if a user opts to use a readily available energy profile then the configuration presented in Listing 2 can be skipped or reduced just to a couple of lines targeting any specific component the user wishes to amend. IV. EVALUATION This section provides a comprehensive study demonstrating the main contributions of our work by introducing a realistic use-case under-going two different deployments so that a ML practitioner can truly assess the energy consumption KPIs of his/her ML-driven drone application are met before introduced into production. A. Use-Case Scenario Suppose aML practitioner must test theefficiency of his/her developed ML solution for a hypothetical -but very realistic- use-case that employs face recognition for tracking human targets’ from the air (i.e., social distancing). Towards this, the drone is given a flight plan including a (bounded) area of interest and a starting location. The drone will then fly above Python SDK Create energy model and add/remove components Customize components (naming, add/remove sub- components, set values, …) Reusable templates for DJI Drones
  • 11. 10/4/2021 11 Demetris Trihinas [email protected] 11 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Drone Benchmarking Suite Python SDK to design drone testbeds Or use pre- built templates Deploy ML models and configure on- board/remote inference Collect analytic insights https://unic-ailab.github.io/flockai/
  • 12. 10/4/2021 12 Demetris Trihinas [email protected] 12 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Use-Case: Face Detection and Recognition Image from DroneFaces dataset Python face_recognition library DJI Mavic 2 Pro Running in Webots with FlockAI and HornEt introduced example
  • 13. 10/4/2021 13 Demetris Trihinas [email protected] 13 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Experiment Results Fig. 3: Energy Decomposed into Components (No ML) Fig. 5: Processing Energy Decomposed into its Sub- Components (On-Board ML) In these figures, the energy consumption is decomposed by HornEt into the energy consumed by the motors, processing and communication components of the drone. From both of these figures, we immediately observe that Em otor s is the dominating energy consuming factor. However, despite thesig- nificance of Em otor s when face recognition is enabled, Epr oc contributes to approximately 18% of the total energy and this is far from being considered as negligible. Compared to the No-ML flight, the processing overhead is increased by 13%. Note that there is also a 2% increase in the communication Fig. 3: Energy Decomposed into Components (No ML) Fig. 4: Energy Decomposed into Components (On-Board ML) attached to the drone is the one presented in Section III-B and depicted in Listing 2. As a reminder, the model considers three main energy components: motors, processing and com- munication. The only exception from Listing 2 is that we will not ignore the I/O sub-component for the processing overhead. Moreover, all experiment scenarios were run 10 times with the average values presented in the result visualizations. B. Experiment 1: On-Board Inference In this experiment, the ML practitioner would like to evaluate the impact of his/her algorithm in terms of energy consumption for theutilized drone. Hence, thefacerecognition Fig. 5: Processing Energ Components (On-Board ML In these figures, the energy HornEt into the energy con and communication compon these figures, we immediat dominating energy consumin nificance of Em ot or s when f contributes to approximately is far from being considere No-ML flight, the processin Note that there is also a 2% overhead due to the dissem 2+ people detected. In order to further und algorithm on the energy co HornEt to further decompo sub-components. These, are observe that the majority consumed by the CPU when of course, is attributed to th inference algorithm. Moreo fraction of the overhead is flight controller (⇠10%) and state (⇠9%), while the I/O o images are not stored. C. Experiment 2: Remote In No ML) Fig. 5: Processing Energy Decomposed into its Sub- Components (On-Board ML) Fig. 6: Energy Decomposed into Components (Remote ML) execution of ML algorithms ful of frameworksarenow ai consumption data on top of DronesBench [18] provides but it is tailored to testing d take-off and landing in cont Interestingly, Seewald et tool for coarse-grained energ usage as a function of com interesting tool, presented v powprofiler implementation processing component and t cpu and gpu as reference d introduce FANETSim [9],
  • 14. 10/4/2021 14 Demetris Trihinas [email protected] 14 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Experiment Results Fig. 6: Energy Decomposed into Components (Remote ML) Fig. 7: Inference Latency wireless link is open throughout the flight and a data payload is exchanged with the ground station. In turn, we observe something that may not be immediately evident. Despite that inference is performed remotely, Epr oc has not dropped to the value it had when the No-ML flight was executed. This is due to the data preparation tasks of the sensory data and the ful of frameworks arenow aiming to provide users wit consumption data on top of an emulation stack. Spe DronesBench [18] provides detailed power consumpt but it is tailored to testing drone hardware componen take-off and landing in controllable (emulated) setting Interestingly, Seewald et al. introduce powprofiler tool for coarse-grained energy modeling that describe usage as a function of component configuration. W interesting tool, presented via a drone use-case scen powprofiler implementation is limited to only suppor processing component and this component only consi cpu and gpu as reference dimensions. Finally, Trop introduce FANETSim [9], a java-based simulation t enables the testing of drone-to-ground station and to-drone communication overhead. Overhead is exam terms of message counts and energy consumption. H the energy model only considers energy relevant to fl drone (Em ot or s) which is modeled with drone veloci the only dependent variable. VI. CONCLUSIONS AND FUTURE WORK With the remarkable rise of ML empowered dron detailed modeling for both the mechanical and e components of a drone are required so that the ener sumption is estimated with a small degree of inaccura using an emulator during application design. In th we have introduced HornEt, a python framework the production of realistic energy models during the of ML-driven drone applications. Depending on the granularity that the ML practitioner requires, the energ can be customized by adding, amending or dropping monitorable components of the drone. In turn, HornE with ready-to-use energy profiles for various popula so that no additional effort is required for new use
  • 15. 10/4/2021 15 Demetris Trihinas [email protected] 15 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Conclusions • Rise of ML-driven drone applications. • Difficult to create testing environments for drone applications. • HornEt is a python framework enabling the production of energy models during the testing of ML-driven drone applications. • Depending on the level of required granularity, the energy model can be customized by adding, amending or dropping different monitorable components of the drone. • HornEt comes with ready-to-use energy profiles for various popular drones so that no additional effort is required for new users when getting started.
  • 16. 10/4/2021 16 Demetris Trihinas [email protected] 16 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Composable Energy Modeling for ML-Driven Drone Applications The HornEt Framework Thank You! https://unic-ailab.github.io/flockai/
  • 17. 10/4/2021 17 Demetris Trihinas [email protected] 17 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Backup Slides
  • 18. 10/4/2021 18 Demetris Trihinas [email protected] 18 The HornEt Framework | TDIS@IC2E 2021 Department of Computer Science Related Work • Several benchmarking suites for testing drone hardware and flight control (i.e., jmavsim, gazebo, AirSim) but no ML deployment and testing. • For energy testing, Powprofiler can test processing component, while FANETSim tests communication component.