Is your MQTT broker IoT-ready ?
Is your MQTT broker IoT-ready ?
Andrea Ceiner
WWW.EUROTECH.COM
@ceinerandrea
@EurotechFan
@DevNationConf
Topics
• Challenges of IoT systems
• A quick introduction to MQTT
• A-MQ and MQTT
• Use case applications
• Join the open IoT community
• Q&A
Business Issue
Sometimes M2M Solutions look simple …
Internet /
TCP/IP
Business
Applications
Sensors,
Actuators,
Displays, …
Service
Gateway
Single or no special
application on gateway
A single application
to communicate with
Only one type of
Gateway HW & SW
Only one type of
device / sensor
No connectivity
options required
Business Issue
... but for some of them it gets complex
Internet /
TCP/IP
Ethernet WiFi
3G /
LTE Satellite
WiMAX
Other
Business
Applications
Sensors,
Actuators,
Displays, …
• Multiple services on the
gateway / edge node
• Different connectivity
options
• More than one kind of
vertical market value
proposition
• Customer specific
business logic
2G /
2.5G
xDSL
Cable
Business Issue
... Geographically Dispersed …
Business
Applications
Sensors,
Actuators,
Displays, …
Multi-
Service
Gateway
?
Business Issue
... and with more than one Consumer of the Device Data
@
Business
Applications
Sensors,
Actuators,
Displays, …
?
Where M2M/IoT projects most often fail
M2M
Communication
Infrastructure
Device
Firmware /
Application
Business
Application
Sensors &
Device
Hardware
Business
Application
Integration
1
2 3
4
6
• Selecting and
integrating sensors,
devices, sensors,
human machine
interfaces (HMI),
Meters, legacy field
busses & actuators
• Ensuring long life
support
• Meeting certification
requirements
• Selecting and
integrating
operating system,
device support /
drivers
• Implementing the
business logic
• Optimum
M2M
protocols
• WAN cost
reduction
• Security
• Device data management
• Device life cycle management
• Security
5
• Decoupling of
producers and
consumers of
data
• Write speeds
• Real-time data
streams
• Data storage
• Standard APIs
• Ready to use
adapters for
standard
applications
• CEP / Complex
Event Processing
capabilities
1 2 3 4 6
5
7
• Application development & life cycle management
• Dashboards, user interaction & interfacing
• Integration (Big Data, social networks, enterprise IT)
7
Multi-Tier IoT Architecture
Business
Applications & Intelligence
Sensors,
Actuators,
Displays, …
@
Multi-Service
Gateway
Normalization of operational technologiesNormalizing Operational Technologies M2M/IoT
Integration
Platform
Enterprise
Information
Technologies
Normalizing Information Tech
Operational
Technologies
Everyware Device Cloud
OPERATIONAL TECH intersec INFORMATION TECH
Information
Technologies
Business Applications
BLE? MQTT ? AMQP ? JMS ? HTTP ? REST ?
AMQP
MQTT
MQTT
ZigBee,BLE,WiFi,LoRa,ModBUS,CanBUS,ProfiBUS,…
HTTP
REST API
websockets
JMS
OT VS IT
Everyware Cloud (Red Hat A-MQ 6.* embedded)
OPERATIONAL TECHNOLOGIES
Everyware Cloud
IoT
Analytics
Enterprise
Application
@
Alerts
on
Application
Integration
Complex Event Processing
Platform & Security Management
HealthMonitoring
Big Data Management
Device Management
Device
Connectivity
IoT Analytics
Integration
A-MQ
MQTT Broker
A-MQ key points 3
A-MQ : OSGi Architecture 2
A-MQ : Highly Tunable
• AMQ.XML  we have tuned about 60
parameters
1
Why MQTT
• M2M Messaging Protocol
• Low Bandwidth / Low Power
• 2-way Communication
• Publish and Subscribe
• Hierarchical Topic Namespaces
• Data Payload Agnostic
• Device Initiated Connection
• Firewall-friendly
• SSL and Authenticated
• Large ecosystem
on
Application
Integration
Complex Event Processing
Platform & Security Management
HealthMonitoring
Big Data Management
Device Management
Device
Connectivity
IoT Analytics
Integration
Benefits of MQTT versus HTTP
Source: http://stephendnicholas.com/archives/1217
• Push delivery of messages /
data / events
• MQTT – low latency 2-way
communication
• HTTP – push from client but
poll from server
• Efficient use of network
• Reliable delivery over fragile
network
• Decoupling publishers and
subscribers – one to many
delivery
MQTT is OK …
but NOT ENOUGH
What’s beyond MQTT ?
At the Device Side (client)…
• Transport Service
–Transport Service Abstraction
–Credentials & Authentication management
–Provisioning
6
What’s beyond MQTT ?
At the Device Side (client)…
• Data Services
– Manages remote connectivity and reconnect
policies
– Manages message queuing with priority
– Manages message store while disconnected and
resume publishing on reconnects
5
What’s beyond MQTT ?
At the Device Side (client)…
• Cloud Services
– Data model for telemetry applications
– Topic partitioning across applications
– Compression
– Birth, Death & App certificates
– Request and Response message exchanges
– Simplified code for remote resource management
4
What’s beyond MQTT ?
At the Broker (server)…
• Broker Plugin
– Enforces Authentication
– Enforces Authorization Topic ACLs
– Enforces Account Policies
– Tracks User Logins
3
What’s beyond MQTT ?
At the Broker (server)…
• Device Connectivity
– Tracks Device Connectivity
– Update Device Status
• Data Store
– Store messages on IoT noSQL BigData DB
2
What’s beyond MQTT ?
At the Broker (server)…
• Rules Engine Bundle
– Inspect messages, check rules and trigger
actions
• A-MQ Cluster Support & Protocol translation
• Custom Bundles
1
A-MQ IoT Challenges
• A-MQ instances exposed on the
Internet for device direct connectivity
• Scale on number of connections not just
number of messages
ESB scenario IoT scenario
Recommendations
• Protect the broker instances
– Firewall configuration, secure A-MQ connectors,
SSL enforcements (i.e. <sslContext>),
– Configure broker cluster for reliability for the
creation of a network of brokers
6
Recommendations
• Consider SSL Termination
– Keep it off-broker, as a different SSL-proxy node
decoding SSL-2-noSSL
5
Recommendations
• OSGi is your friend, respect its architecture !
– Deploy Consumers closer to the Broker (no
messages over TCP)
– Use “vm” connector (direct mem links)
4
Recommendations
• Apply General A-MQ Tunings to
configure a Network of Brokers
https://access.redhat.com/documentation/en-
US/Red_Hat_JBoss_A-
MQ/6.0/html/Tuning_Guide/files/PersTuning-
Horizontal.html
3
Recommendations
• Set persistence engine tuning
for high-speed message
storage & retrieval
https://access.redhat.com/documentation/en-
US/Red_Hat_JBoss_A-
MQ/6.0/html/Tuning_Guide/files/PersTuning-
KahaDB.html
2
Recommendations
• MQTT Tuning
– Nio (decoupling java threads from connections)
– Disable JMX due to large number of Topics;
Monitor using advisory messages
(advisorySupport=TRUE in AMQ.XML)
– Increase Durable Topic Prefetch for bulk msg
batch processing (activeMQSubscriptionPrefetch)
1
Recommendations
• From ActiveMQ 5.11, try virtual-topic-subscriptions
– Set a pool of subscribers
– Send a message in round-robin to just one of them rather
then towards them all (as by default publish-subscribe)
0
IoT Use Cases
Public Safety
Energy Management
Bus
Water Management
Public Health
Tram
Train
Metro
Subway
Remote
Monitoring
Fitness
Machines
Logistics
Medical
Transportation
Public Transport
Rail
Metering
Air Conditions
Elderly
Living
Waste Management
Value Transport
Smart City
Environmental
Smart Grid First Responders
Green Houses
Sports Medical Application
Cool Chain Monitoring
Vending
Reverse Vending
Industrial
Ticketing
Smart Buildings
Irrigation
SignageAutomatic Vehicle Location
Remote Monitoring
Retail
Energy Monitoring
Medical
Elderly Living
Smart City
Smart Building
Green Houses
Agriculture
Retail
Application:
Monitor & analyze people flows and correlate with
environmental monitoring data and with Social Media like
Twitter at the RedHat Summit & DevNation 2015 event in Boston.
Key Success Factors:
PCN features & technology (e.g. stereoscopic vision)
Complete set of M2M/ IoT building blocks available
Java centric development of device application
Effective integration with APPS and dashboards via REST API
Complete IoT end-to-end solution with other Red Hat Partners
Short development time
Product:ReliaGATE 10-20 with
PCN-10-01 &
ReliaSENS 18-12
Smart Events: People Flow + Air Qualty
+ Social Media
Application:
Optimize and improve the sanitary services in public buildings.
The system is used to monitor restrooms attendance in order to
activate the staff according to the precise needs (number of
visitors) rather than on a time schedule
Key Success Factors:
PCN features & technology (e.g. stereoscopic vision)
Complete set of building blocks offered
Complete end-to-end solution
Short development time
performance, flexibility and simplicity of data treatment
(storage, download, accessibility, analysis)
Product:
Helios with
PCN-1000
Passenger Counter
The Sanitary Service Optimization
Taxi Queue Optimization
Application:
The goal was to improve the customer service and the
optimization of taxi routing to increase revenues and
reduce waiting times, and, finally, to increase the driver’s
security.
Key Success Factors:
Short time to market due to easy EDC approach
ESF for natively connecting PCN to Cloud
Smart Cameras connected to Cloud
Increased Drivers Security (alarm button with GPS
positioning and Taxi ID)
Increased Customer Satisfaction
Increased Taxi Revenues
Product:
ReliGATE 50-21 with
PCN-1000
Passenger Counter
Application:
The customer required a programmable Intel cellular platform for
monitoring of refrigeration units at grocery stores for energy
and asset management application.
Key Success Factors:
flexibility of ESF
Started with Helios… software portability across HW
Eurotech M2M knowledge and experience
Flexible hardware platform
long term data-metrics storage
ReliaGATE 50-21
Retail Energy & Asset Management
MQTT is IoT-ready !
…
Are you IoT-ready too ?
9+ MILLIONS
JAVA
DEVELOPERS
MOST USED
PROGRAMMING
LANGUAGE FOR
ENTERPRISE
APPS
Why a Java M2M Gateway Stack
Lower time to market and Investment Protection
http://redmonk.com/sogrady/2015/01/14/language-rankings-1-15/
http://java.dzone.com/articles/how-many-java-developers-are
Are you
one of
them ?
Open Source Java M2M Gateway Stack
M2M Gateway Challenges:
• Pressure to add value in shrinking
timeframes
• Velocity of technology changes
outstrips staffing
• Interoperability trumps exclusive
differentiation
• Quest for quality w/o lock-in
Open Source is the
Answer!
• 23 Members
• 15+ new projects
• 1,5M+ lines of source code
• The fastest growing Eclipse
workgroup
http://www.slideshare.net/blackducksoftware/io-t-and-open-source
Founded in
2012 by
Eclipse Kura
Open Java Framework for SW-defined M2M Gateways
https://www.eclipse.org/kura/
https://iot.eclipse.org/java/
Eclipse Kura is attractive
1K downloads/month
You are important !
Kura helps you … Kura needs you
I was lucky to be
involved and get to
contribute to
something that was
important, which is
empowering people
with software.
(Bill Gates)
THANK YOU
Andrea Ceiner
WWW.EUROTECH.COM
@ceinerandrea
@EurotechFan
@DevNationConf

Is your MQTT broker IoT ready?

  • 1.
    Is your MQTTbroker IoT-ready ?
  • 2.
    Is your MQTTbroker IoT-ready ? Andrea Ceiner WWW.EUROTECH.COM @ceinerandrea @EurotechFan @DevNationConf
  • 3.
    Topics • Challenges ofIoT systems • A quick introduction to MQTT • A-MQ and MQTT • Use case applications • Join the open IoT community • Q&A
  • 4.
    Business Issue Sometimes M2MSolutions look simple … Internet / TCP/IP Business Applications Sensors, Actuators, Displays, … Service Gateway Single or no special application on gateway A single application to communicate with Only one type of Gateway HW & SW Only one type of device / sensor No connectivity options required
  • 5.
    Business Issue ... butfor some of them it gets complex Internet / TCP/IP Ethernet WiFi 3G / LTE Satellite WiMAX Other Business Applications Sensors, Actuators, Displays, … • Multiple services on the gateway / edge node • Different connectivity options • More than one kind of vertical market value proposition • Customer specific business logic 2G / 2.5G xDSL Cable
  • 6.
    Business Issue ... GeographicallyDispersed … Business Applications Sensors, Actuators, Displays, … Multi- Service Gateway ?
  • 7.
    Business Issue ... andwith more than one Consumer of the Device Data @ Business Applications Sensors, Actuators, Displays, … ?
  • 8.
    Where M2M/IoT projectsmost often fail M2M Communication Infrastructure Device Firmware / Application Business Application Sensors & Device Hardware Business Application Integration 1 2 3 4 6 • Selecting and integrating sensors, devices, sensors, human machine interfaces (HMI), Meters, legacy field busses & actuators • Ensuring long life support • Meeting certification requirements • Selecting and integrating operating system, device support / drivers • Implementing the business logic • Optimum M2M protocols • WAN cost reduction • Security • Device data management • Device life cycle management • Security 5 • Decoupling of producers and consumers of data • Write speeds • Real-time data streams • Data storage • Standard APIs • Ready to use adapters for standard applications • CEP / Complex Event Processing capabilities 1 2 3 4 6 5 7 • Application development & life cycle management • Dashboards, user interaction & interfacing • Integration (Big Data, social networks, enterprise IT) 7
  • 9.
    Multi-Tier IoT Architecture Business Applications& Intelligence Sensors, Actuators, Displays, … @ Multi-Service Gateway Normalization of operational technologiesNormalizing Operational Technologies M2M/IoT Integration Platform Enterprise Information Technologies Normalizing Information Tech
  • 10.
    Operational Technologies Everyware Device Cloud OPERATIONALTECH intersec INFORMATION TECH Information Technologies Business Applications
  • 11.
    BLE? MQTT ?AMQP ? JMS ? HTTP ? REST ? AMQP MQTT MQTT ZigBee,BLE,WiFi,LoRa,ModBUS,CanBUS,ProfiBUS,… HTTP REST API websockets JMS
  • 12.
  • 13.
    Everyware Cloud (RedHat A-MQ 6.* embedded) OPERATIONAL TECHNOLOGIES
  • 14.
    Everyware Cloud IoT Analytics Enterprise Application @ Alerts on Application Integration Complex EventProcessing Platform & Security Management HealthMonitoring Big Data Management Device Management Device Connectivity IoT Analytics Integration A-MQ MQTT Broker
  • 15.
  • 16.
    A-MQ : OSGiArchitecture 2
  • 17.
    A-MQ : HighlyTunable • AMQ.XML  we have tuned about 60 parameters 1
  • 18.
    Why MQTT • M2MMessaging Protocol • Low Bandwidth / Low Power • 2-way Communication • Publish and Subscribe • Hierarchical Topic Namespaces • Data Payload Agnostic • Device Initiated Connection • Firewall-friendly • SSL and Authenticated • Large ecosystem on Application Integration Complex Event Processing Platform & Security Management HealthMonitoring Big Data Management Device Management Device Connectivity IoT Analytics Integration
  • 19.
    Benefits of MQTTversus HTTP Source: http://stephendnicholas.com/archives/1217 • Push delivery of messages / data / events • MQTT – low latency 2-way communication • HTTP – push from client but poll from server • Efficient use of network • Reliable delivery over fragile network • Decoupling publishers and subscribers – one to many delivery
  • 20.
    MQTT is OK… but NOT ENOUGH
  • 21.
    What’s beyond MQTT? At the Device Side (client)… • Transport Service –Transport Service Abstraction –Credentials & Authentication management –Provisioning 6
  • 22.
    What’s beyond MQTT? At the Device Side (client)… • Data Services – Manages remote connectivity and reconnect policies – Manages message queuing with priority – Manages message store while disconnected and resume publishing on reconnects 5
  • 23.
    What’s beyond MQTT? At the Device Side (client)… • Cloud Services – Data model for telemetry applications – Topic partitioning across applications – Compression – Birth, Death & App certificates – Request and Response message exchanges – Simplified code for remote resource management 4
  • 24.
    What’s beyond MQTT? At the Broker (server)… • Broker Plugin – Enforces Authentication – Enforces Authorization Topic ACLs – Enforces Account Policies – Tracks User Logins 3
  • 25.
    What’s beyond MQTT? At the Broker (server)… • Device Connectivity – Tracks Device Connectivity – Update Device Status • Data Store – Store messages on IoT noSQL BigData DB 2
  • 26.
    What’s beyond MQTT? At the Broker (server)… • Rules Engine Bundle – Inspect messages, check rules and trigger actions • A-MQ Cluster Support & Protocol translation • Custom Bundles 1
  • 27.
    A-MQ IoT Challenges •A-MQ instances exposed on the Internet for device direct connectivity • Scale on number of connections not just number of messages ESB scenario IoT scenario
  • 28.
    Recommendations • Protect thebroker instances – Firewall configuration, secure A-MQ connectors, SSL enforcements (i.e. <sslContext>), – Configure broker cluster for reliability for the creation of a network of brokers 6
  • 29.
    Recommendations • Consider SSLTermination – Keep it off-broker, as a different SSL-proxy node decoding SSL-2-noSSL 5
  • 30.
    Recommendations • OSGi isyour friend, respect its architecture ! – Deploy Consumers closer to the Broker (no messages over TCP) – Use “vm” connector (direct mem links) 4
  • 31.
    Recommendations • Apply GeneralA-MQ Tunings to configure a Network of Brokers https://access.redhat.com/documentation/en- US/Red_Hat_JBoss_A- MQ/6.0/html/Tuning_Guide/files/PersTuning- Horizontal.html 3
  • 32.
    Recommendations • Set persistenceengine tuning for high-speed message storage & retrieval https://access.redhat.com/documentation/en- US/Red_Hat_JBoss_A- MQ/6.0/html/Tuning_Guide/files/PersTuning- KahaDB.html 2
  • 33.
    Recommendations • MQTT Tuning –Nio (decoupling java threads from connections) – Disable JMX due to large number of Topics; Monitor using advisory messages (advisorySupport=TRUE in AMQ.XML) – Increase Durable Topic Prefetch for bulk msg batch processing (activeMQSubscriptionPrefetch) 1
  • 34.
    Recommendations • From ActiveMQ5.11, try virtual-topic-subscriptions – Set a pool of subscribers – Send a message in round-robin to just one of them rather then towards them all (as by default publish-subscribe) 0
  • 35.
    IoT Use Cases PublicSafety Energy Management Bus Water Management Public Health Tram Train Metro Subway Remote Monitoring Fitness Machines Logistics Medical Transportation Public Transport Rail Metering Air Conditions Elderly Living Waste Management Value Transport Smart City Environmental Smart Grid First Responders Green Houses Sports Medical Application Cool Chain Monitoring Vending Reverse Vending Industrial Ticketing Smart Buildings Irrigation SignageAutomatic Vehicle Location Remote Monitoring Retail Energy Monitoring Medical Elderly Living Smart City Smart Building Green Houses Agriculture Retail
  • 36.
    Application: Monitor & analyzepeople flows and correlate with environmental monitoring data and with Social Media like Twitter at the RedHat Summit & DevNation 2015 event in Boston. Key Success Factors: PCN features & technology (e.g. stereoscopic vision) Complete set of M2M/ IoT building blocks available Java centric development of device application Effective integration with APPS and dashboards via REST API Complete IoT end-to-end solution with other Red Hat Partners Short development time Product:ReliaGATE 10-20 with PCN-10-01 & ReliaSENS 18-12 Smart Events: People Flow + Air Qualty + Social Media
  • 37.
    Application: Optimize and improvethe sanitary services in public buildings. The system is used to monitor restrooms attendance in order to activate the staff according to the precise needs (number of visitors) rather than on a time schedule Key Success Factors: PCN features & technology (e.g. stereoscopic vision) Complete set of building blocks offered Complete end-to-end solution Short development time performance, flexibility and simplicity of data treatment (storage, download, accessibility, analysis) Product: Helios with PCN-1000 Passenger Counter The Sanitary Service Optimization
  • 38.
    Taxi Queue Optimization Application: Thegoal was to improve the customer service and the optimization of taxi routing to increase revenues and reduce waiting times, and, finally, to increase the driver’s security. Key Success Factors: Short time to market due to easy EDC approach ESF for natively connecting PCN to Cloud Smart Cameras connected to Cloud Increased Drivers Security (alarm button with GPS positioning and Taxi ID) Increased Customer Satisfaction Increased Taxi Revenues Product: ReliGATE 50-21 with PCN-1000 Passenger Counter
  • 39.
    Application: The customer requireda programmable Intel cellular platform for monitoring of refrigeration units at grocery stores for energy and asset management application. Key Success Factors: flexibility of ESF Started with Helios… software portability across HW Eurotech M2M knowledge and experience Flexible hardware platform long term data-metrics storage ReliaGATE 50-21 Retail Energy & Asset Management
  • 40.
    MQTT is IoT-ready! … Are you IoT-ready too ?
  • 41.
    9+ MILLIONS JAVA DEVELOPERS MOST USED PROGRAMMING LANGUAGEFOR ENTERPRISE APPS Why a Java M2M Gateway Stack Lower time to market and Investment Protection http://redmonk.com/sogrady/2015/01/14/language-rankings-1-15/ http://java.dzone.com/articles/how-many-java-developers-are Are you one of them ?
  • 42.
    Open Source JavaM2M Gateway Stack M2M Gateway Challenges: • Pressure to add value in shrinking timeframes • Velocity of technology changes outstrips staffing • Interoperability trumps exclusive differentiation • Quest for quality w/o lock-in Open Source is the Answer! • 23 Members • 15+ new projects • 1,5M+ lines of source code • The fastest growing Eclipse workgroup http://www.slideshare.net/blackducksoftware/io-t-and-open-source Founded in 2012 by
  • 43.
    Eclipse Kura Open JavaFramework for SW-defined M2M Gateways https://www.eclipse.org/kura/ https://iot.eclipse.org/java/
  • 44.
    Eclipse Kura isattractive 1K downloads/month
  • 45.
    You are important! Kura helps you … Kura needs you I was lucky to be involved and get to contribute to something that was important, which is empowering people with software. (Bill Gates)
  • 46.