Abstract image of a woman sitting on books and studying on a laptop

Adding intelligence to your LoRaWAN devices - The Things Conference on tour

Jan Jongboom, profile picture
Jan Jongboom

The document discusses adding intelligence to LoRaWAN devices by extracting meaningful features from raw sensor data on the device. It notes that currently, 99% of sensor data is discarded due to constraints, but extracting features could enable anomaly detection, classification, and forecasting on the device. It proposes a two-step process: 1) collecting high-resolution raw data from sensors, and 2) extracting features from the raw data on the device to perform intelligent analysis without transmitting all the raw data.

Internet
1 of 36
Downloaded 56 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
Adding intelligence to your LoRaWAN devices Jan Jongboom, Edge Impulse
Jan Jongboom CTO and co-founder, Edge Impulse jan@edgeimpulse.com
3 Typical LoRaWAN sensor in 2019 Vibration sensor (up to 1,000 times per second) Temperature sensor NFC Water & explosion proof Processor capable of running >20 million
 instructions per second
4 But... what does it actually do? Once an hour: • Average motion (RMS) • Peak motion • Current temperature
5 99% of sensor data is discarded due to 
 cost, bandwidth or power constraints. https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/McKinsey%20Digital/Our%20Insights/ The%20Internet%20of%20Things%20The%20value%20of%20digitizing%20the%20physical%20world/The- Internet-of-things-Mapping-the-value-beyond-the-hype.ashx
6 Lots of interesting data gets lost Peak
7 Single numbers can be misleading updown circle avg. RMS 3.3650 3.3515
8 Interesting questions require more data Classification
 What's happening right now? 
 Anomaly detection
 Is this behavior out of the ordinary? 
 Forecasting
 What will happen in the future?
9 Two problems 1. Very hard to answer with rule-based programming if (accX > 9.5) { lorawan.send('someone picked me up'); } 2. Needs to happen on-device, because 
 of this little pesky thing called physics
Copyright © 2019 EdgeImpulse Inc. Let's fix this!
11 Step 1 - Getting raw data High-resolution data straight from devices (100 Hz) Correct labeling Offloading probably not over LoRaWAN (but signaling could) https://pixabay.com/photos/factory-night-view-industrial-pipe-1769429/
12 Raw data (3 samples)
13 Raw data (2,000 samples)
14 Step 2 - Extracting meaningful features Very dependent on your use case Raw data can be notoriously hard to deal with (3s. accelerometer data = 900 data points) Raw data is messy -1.1200, 0.5300, 10.6300, -0.5200, 1.1600, 13.1600, -0.1600, 1.4200, 13.5900, -0.2400, 1.3200, 10.8500, -0.3700, 0.5100, 7.7800, -0.1900, 0.0500, 8.8400, -0.1900, 0.0500, 8.8400, 0.2400, 0.7300, 11.1900, 0.5200, 1.6500, 12.5200, 0.6400, 1.5000, 10.8200, 0.2900, 0.3300, 7.4000, -0.3100, -0.5600, 8.5900, -0.8800, 0.6600, 11.9200, -0.8800, 0.6600, 11.9200, -0.4500, 1.3900, 11.9100, -0.0800, 1.6000, 12.7800, -0.1100, 0.3600, 9.1700, -0.0200, 0.0700, 9.8200, 0.0600, 0.9600, 12.0000, 0.2800, 1.7700, 13.2000, 0.2800, 1.7700, 13.2000, 0.2300, 1.5100, 12.6000, 0.2700, 1.0800, 11.4300, 0.0900, 0.9300, 10.9400, 0.1700, 1.2100, 11.0400, 0.4100, 1.8500, 11.5000, 0.3900, 1.9600, 11.4300, 0.3900, 1.9600, 11.4300, 0.2400, 1.4400, 10.7200, 0.0300, 1.1900, 10.3600, -0.0300, 1.3000, 10.6100, 0.4800, 1.7900, 11.7200, 1.0400, 2.6300, 13.3300, 1.0400, 2.6300, 13.3300, 1.0600, 2.3700, 13.5800, 0.3600, 1.9600, 13.3800, -0.1000, 2.1200, 13.9600, -0.3600, 2.0200, 14.5300, 0.0000, 2.1500, 14.6900, 0.0600, 2.1400, 14.3700, 0.0600, 2.1400, 14.3700, -0.3000, 1.8800, 13.7900, 0.0500, 1.7000, 13.5900, 0.1300, 1.6700, 13.1500, -0.0100, 1.7700, 12.9000, 0.4000, 1.8900, 12.2300, 0.5300, 2.3300, 12.2600, 0.5300, 2.3300, 12.2600, 0.0400, 1.9500, 11.8100, -0.2300, 1.9600, 11.2400, -0.0600, 2.1100, 10.2200, -0.1100, 2.4100, 9.7800, -0.3500, 2.7100, 9.7500, -0.7800, 3.1000, 10.1100, -0.7800, 3.1000, 10.1100, -1.0700, 3.1100, 9.8000, -1.2100, 2.9400, 9.1900, -1.1500, 3.2100, 8.6400, -0.7300, 3.6500, 8.4600, -0.5000, 3.9500, 8.6300, -0.4300, 3.9100, 8.7400, -0.4300, 3.9100, 8.7400, -0.6400, 3.7800, 8.8700, -1.2000, 3.9200, 8.9300, -1.0800, 4.4400, 8.8100, -0.7800, 4.1900, 8.1200, -0.4400, 4.1000, 7.6400, -0.5400, 4.2000, 7.5600, -0.5400, 4.2000, 7.5600, -1.0700, 4.2600, 7.2700, -1.3000, 4.5100, 7.2300, -1.2600, 4.4600, 6.6900, -1.2800, 4.4100, 6.6000, -1.7000, 4.6800, 7.0800, -2.3400, 5.1100, 7.5900, -2.3400, 5.1100, 7.5900, -2.8300, 4.8700, 6.8700, -2.9700, 4.7600, 6.2700, -3.2500, 4.6000, 6.1500, -3.4900, 4.5900, 6.2600, -3.3000, 4.9200, 6.3400, -2.7000, 4.9300, 5.8600, -2.7000, 4.9300, 5.8600, -2.9000, 4.5100, 4.9900, -3.5200, 4.3200, 4.9900, -4.1400, 4.2100, 5.7800, -3.7600, 4.1600, 5.7700, -3.0200, 4.2200, 5.4900, -3.0000, 3.8900, 3.9100, -3.0000, 3.8900, 3.9100, -3.3500, 3.5800, 3.4400, -3.1100, 3.2000, 3.6000, -3.0900, 3.6000, 5.9400, -3.0800, 3.0900, 5.1800, -2.8000, 2.9400, 4.5600, -2.4000, 2.4900, 3.9200, -2.4000, 2.4900, 3.9200, -1.8500, 2.6300, 4.4900, -1.4300, 3.9900, 7.3400, -1.4900, 3.4900, 6.1600, -1.5300, 3.2100, 5.4500, -0.9900, 2.8600, 6.2400, -0.7900, 3.2200, 8.4700, -0.7900, 3.2200, 8.4700, -0.9300, 3.5700, 9.0300, -1.6600, 2.9600, 7.0700, -1.7600, 2.1000, 6.9000, -1.4600, 2.1100, 9.0100, -1.3000, 2.5400, 10.3000, -1.3000, 2.5400, 10.3000, -1.4500, 2.6500, 9.7800, -1.5300, 1.8900, 8.4100, -1.1400, 1.3000, 9.4400, -0.7500, 1.6100, 10.3600, -0.9400, 1.5300, 9.7800, -1.0700, 1.0100, 9.2700, -1.0700, 1.0100, 9.2700, -0.8600, 1.0200, 10.1100, 0.3900, 1.6800, 11.3200, 1.2900, 1.8500, 11.4700, 1.0700, 1.3200, 11.2500, -0.2100, 1.5800, 12.4300, -1.8100, 1.3500, 12.3300, -1.8100, 1.3500, 12.3300, -2.1800, 1.0400, 11.1600, -1.5400, 0.3000, 10.3100, -0.4700, 0.2700, 11.0900, 0.7900, 1.4100, 12.9800, 1.1600, 1.7100, 12.4700, 0.5800, 0.8700, 9.6300, 0.5800, 0.8700, 9.6300, 0.1300, 0.3100, 9.5600, 0.2800, 0.3600, 10.5600, 0.7400, 0.8500, 11.4200, 0.9300, 1.0200, 11.4300, 0.6700, 0.5600, 10.5300, 0.8500, 0.3500, 9.4100, 0.8500, 0.3500, 9.4100, 1.6600, 1.2800, 10.9200, 2.0500, 0.9900, 9.7000, 2.1300, 0.8800, 10.1900, 2.0500, 0.9100, 11.3300, 1.7700, 1.4100, 12.2700, 1.4800, 1.7600, 12.1000, 1.4800, 1.7600, 12.1000, 0.9400, 1.1300, 10.8500, 0.2000, 0.8000, 10.1200, 0.2600, 1.1600, 10.5800, 0.5100, 1.6500, 10.7800, 0.4600, 1.2000, 9.9900, 0.9100, 0.8400, 9.6400, 0.9100, 0.8400, 9.6400, 1.4500, 0.7400, 10.2500, 2.0200, 1.3000, 11.4500, 1.8100, 1.8700, 12.1300, 1.0500, 1.5300, 12.0200, 0.6200, 0.6700, 11.3100, 0.7100, 0.8500, 12.0000, 0.7100, 0.8500, 12.0000, 0.6400, 1.2200, 13.1400, 1.1300, 2.0400, 14.6200, 0.8300, 2.0200, 15.5100, -0.1400, 1.4800, 15.6500, -0.6300, 1.5900, 16.0500, -1.3100, 1.7100, 16.3900, -1.3100, 1.7100, 16.3900, -1.7300, 1.5800, 16.6300, -1.1500, 1.4400, 16.0300, -0.5300, 1.1700, 15.1000, -0.1800, 0.9900, 14.4600, -0.3300, 1.0100, 13.5100, -0.3300, 1.0100, 13.5100, -0.4400, 0.9100, 12.6700, 0.0400, 1.2300, 12.5400, 0.6900, 2.0500, 13.1600, 0.3100, 1.7700, 12.8600, 0.0300, 1.3800, 11.1100, -0.4400, 1.2200, 9.4900, -0.4400, 1.2200, 9.4900, 0.1100, 1.1400, 7.3100, 0.8500, 2.2500, 8.4600, 0.8600, 3.3700, 11.2200, -0.1100, 2.2800, 8.4400, -1.3800, 1.5300, 7.1700, -1.0600, 1.5400, 6.9500, -1.0600, 1.5400, 6.9500, -0.5200, 2.8300, 8.7100, -0.2100, 2.3500, 8.1800, -0.3400, 2.7000, 8.9200, -0.3000, 2.3100, 8.7500, -0.4800, 1.4700, 7.8700, -0.3600, 0.9400, 6.9700, -0.3600, 0.9400, 6.9700, -0.2300, 1.4700, 7.6100, -0.3300, 2.2300, 8.5000, 0.3000, 1.9200, 7.8600, -0.2300, 1.5700, 6.8700, -1.4900, 1.5600, 6.3700, -2.8200, 1.6200, 7.2000, -2.8200, 1.6200, 7.2000, -3.1600, 1.8800, 7.1500, -2.7600, 2.2900, 6.8500, -2.6000, 2.2200, 6.2600, -2.9000, 1.9900, 5.8900, -3.3800, 2.2200, 6.2600, -3.9000, 2.1700, 6.0300, -3.9000, 2.1700, 6.0300, -3.8600, 2.3800, 5.6600, -3.5300, 2.5200, 5.6700, -3.2400, 2.3700, 5.8200, -3.2800, 2.1800, 5.5200, -3.1500, 2.1800, 5.6500, -3.0900, 2.0700, 5.1600, -3.0900, 2.0700, 5.1600, -2.4300, 2.1000, 5.3800, -2.0200, 2.3600, 6.0800, -2.0000, 2.5200, 6.4500, -2.2400, 2.4500, 6.0000, -2.0500, 1.8400, 4.6500, -1.3800, 1.3000, 4.6400, -1.3800, 1.3000, 4.6400, -1.2800, 1.8600, 6.9400, -1.3000, 2.5600, 9.0300, -1.5400, 2.7600, 8.5000, -1.7700, 1.6400, 6.1400, -1.6800, 1.4200, 7.5900, -1.3200, 2.0800, 9.8300, -1.3200, 2.0800, 9.8300, -0.8200, 2.1600, 10.3900, -0.7800, 1.7300, 9.8300, -1.1300, 1.3400, 9.7100, -1.3600, 1.6800, 10.2400, -1.5200, 1.6000, 9.3200, -1.8700, 1.4900, 9.1900, -1.8700, 1.4900, 9.1900, -1.9300, 1.0600, 9.9500, -1.3100, 0.8100, 10.6900, 0.0200, 2.0400, 11.0600, 0.2700, 2.5800, 9.3900, -0.0500, 2.2800, 7.3200, -0.3000, 0.4400, 7.6300, -0.3000, 0.4400, 7.6300, -1.4600, 1.0800, 12.3700, -1.9600, 1.7500, 15.3800, -0.7100, 2.1500, 14.0700, 0.7400, 1.7800, 10.4700, 0.6800, 0.8900, 9.9500, 0.0400, 1.5200, 12.0800, 0.0400, 1.5200, 12.0800, -0.4900, 1.7900, 12.7500
15 Example of a signal processing pipeline 32,000 => 240
16 Example of a signal processing pipeline Raw signal Low-pass filter Frequency domain Spectral power Peaks RMS 900 => 33
17 Raw data (2,000 samples)
18 After signal processing (2,000 samples)
Classification
 What's happening right now? 
 Anomaly detection
 Is this behavior out of the ordinary? 
 Forecasting
 What will happen in the future? 19 Step 3 - Letting the computers figure it out
20 Picking the right algorithm Classification
 Neural network 
 Anomaly detection
 K-means clustering 
 Forecasting
 Regression
21 Neural networks Now possible on device Size of the network still matters (code size + ops) Signal processing is key to smaller networks
22 Neural networks aren't the only game in town Classic ML algorithms are much smaller K-means clustering is super efficient 
 (just compare new data against clusters) Combining classic ML and NN is great
23 Step 4 - Deploying Don't continuously sample. Monitor model performance. Something weird? Send DSP result back to network.
24 In practice
25 Gesture detection Idle Snake Updown Wave Circle
26 Collecting data Collect on same device and same sensor Store raw data in flash Sync via WiFi or serial Labeling directly on device Capture all variations DATA COLLECTED 12m 1s
27
28 Neural network for classification Input layer 
 (33 neurons) Hidden layer
 (20 neurons) Hidden layer
 (10 neurons) Output layer
 (5 neurons) (33 x 20) + (20 x 10) + (10 x 5) = 910 operations
29 Dealing with unknown data Neural network: 85% updown, 15% circle
29 Dealing with unknown data Neural network: 85% updown, 15% circle https://pixabay.com/photos/confused-hands-up-unsure-perplexed-2681507/
30 Clustering Good Bad 32 clusters to compare, very efficient
31 On device Signal processing Neural network converted with uTensor (Arm) Clustering code (On Cortex-M4F, 80 MHz) 16 ms. 1 ms. 40 ms.
32
33 Conclusion Yay, machine learning!
Recap The ML hype is real ML + LoRaWAN = perfect fit Start using the remaining 99% of sensor data edgeimpulse.com
35 Thank you! Slides: janjongboom.com

More Related Content

PDF
Intelligent Edge - Getting started with TinyML for industrial applications
Jan Jongboom
 
PDF
Teaching your sensors new tricks with Machine Learning - Eta Compute webinar
Jan Jongboom
 
PDF
Get started with TinyML - Hackster webinar 9 April 2020
Jan Jongboom
 
PDF
Teaching your sensors new tricks with Machine Learning - CENSIS Tech Summit 2019
Jan Jongboom
 
PDF
Adding intelligence to your LoRaWAN deployment - The Things Virtual Conference
Jan Jongboom
 
PDF
Get started with TinyML - Embedded online conference
Jan Jongboom
 
PDF
TinyML on Arduino - workshop
Jan Jongboom
 
PDF
SPICE MODEL of ZR6_RL=8.2(Ohm) in SPICE PARK
Tsuyoshi Horigome
 
Intelligent Edge - Getting started with TinyML for industrial applications
Jan Jongboom
 
Teaching your sensors new tricks with Machine Learning - Eta Compute webinar
Jan Jongboom
 
Get started with TinyML - Hackster webinar 9 April 2020
Jan Jongboom
 
Teaching your sensors new tricks with Machine Learning - CENSIS Tech Summit 2019
Jan Jongboom
 
Adding intelligence to your LoRaWAN deployment - The Things Virtual Conference
Jan Jongboom
 
Get started with TinyML - Embedded online conference
Jan Jongboom
 
TinyML on Arduino - workshop
Jan Jongboom
 
SPICE MODEL of ZR6_RL=8.2(Ohm) in SPICE PARK
Tsuyoshi Horigome
 

What's hot (20)

PDF
SPICE MODEL of R6NT_RL=22(Ohm) in SPICE PARK
Tsuyoshi Horigome
 
PDF
Future Level Stock Report by Mansukh Investment & Trading Solutions 24/06/2010
Mansukh Investment & Trading Solutions
 
DOC
Semana 6.2
Joellmy
 
PDF
Go Ahead for F&O Report 29 January 2013 Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
PDF
F&O Report 01 March 2012-Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
PDF
Analysis on Future levels of Stock by Mansukh Investment and Trading Solution...
Mansukh Investment & Trading Solutions
 
PDF
Key Supports & Resistances of F&O scrips with change in open interest on dail...
Mansukh Investment & Trading Solutions
 
PDF
Future levels 21.02.13
Mansukh Investment & Trading Solutions
 
PDF
SPICE MODEL of CR123A_RL=2.2(Ohm) in SPICE PARK
Tsuyoshi Horigome
 
PDF
SPICE MODEL of ZR6DC_RL=8.2(Ohm) in SPICE PARK
Tsuyoshi Horigome
 
PDF
20 sep future
Mansukh Investment & Trading Solutions
 
PDF
SPICE MODEL of RN1103FV in SPICE PARK
Tsuyoshi Horigome
 
PDF
SPICE MODEL of DTC123EM in SPICE PARK
Tsuyoshi Horigome
 
PDF
Go Ahead for F&O Report 13 February 2013 Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
XLSX
Trazadores
mariarayo
 
PDF
F&O Report 31 January 2012-Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
PDF
Go Ahead for F&O Report 11 February 2013 Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
PDF
Moneysukh future level report 18/3/2010
Mansukh Investment & Trading Solutions
 
PDF
F&O Report 28 December 2011-Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
PDF
Article on Future Level Stock by Mansukh Investment & Trading Solutions 25/0...
Mansukh Investment & Trading Solutions
 
SPICE MODEL of R6NT_RL=22(Ohm) in SPICE PARK
Tsuyoshi Horigome
 
Future Level Stock Report by Mansukh Investment & Trading Solutions 24/06/2010
Mansukh Investment & Trading Solutions
 
Semana 6.2
Joellmy
 
Go Ahead for F&O Report 29 January 2013 Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
F&O Report 01 March 2012-Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
Analysis on Future levels of Stock by Mansukh Investment and Trading Solution...
Mansukh Investment & Trading Solutions
 
Key Supports & Resistances of F&O scrips with change in open interest on dail...
Mansukh Investment & Trading Solutions
 
Future levels 21.02.13
Mansukh Investment & Trading Solutions
 
SPICE MODEL of CR123A_RL=2.2(Ohm) in SPICE PARK
Tsuyoshi Horigome
 
SPICE MODEL of ZR6DC_RL=8.2(Ohm) in SPICE PARK
Tsuyoshi Horigome
 
20 sep future
Mansukh Investment & Trading Solutions
 
SPICE MODEL of RN1103FV in SPICE PARK
Tsuyoshi Horigome
 
SPICE MODEL of DTC123EM in SPICE PARK
Tsuyoshi Horigome
 
Go Ahead for F&O Report 13 February 2013 Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
Trazadores
mariarayo
 
F&O Report 31 January 2012-Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
Go Ahead for F&O Report 11 February 2013 Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
Moneysukh future level report 18/3/2010
Mansukh Investment & Trading Solutions
 
F&O Report 28 December 2011-Mansukh Investment and Trading Solution
Mansukh Investment & Trading Solutions
 
Article on Future Level Stock by Mansukh Investment & Trading Solutions 25/0...
Mansukh Investment & Trading Solutions
 
Ad

Similar to Adding intelligence to your LoRaWAN devices - The Things Conference on tour (20)

TXT
Port game
Yopi Han
 
PDF
SPICE MODEL of ZR6_RL=4.7(Ohm) in SPICE PARK
Tsuyoshi Horigome
 
PDF
Footing-Biaxial_pdf-a.pdf
ANANDA DS
 
PDF
Tim Rainey - Tameside MBC - channel shift
Socitm
 
XLS
Naca 4 digit-delta
Phan Quoc Thien
 
PDF
Tabla z
Ángel Luis Vicentín
 
PPTX
Balance in SNS
we20
 
PPTX
Balance in sns using energy
Son Aris
 
DOCX
Spec00446
guest2f67152
 
XLS
Epgp09 scm - group submission - group iii - merloni sp a
Rajendra Inani
 
PPTX
2017_10_31_BIF_Spectrum_Presentation_Revised_071117.pptx
Miladrazi1
 
PDF
(597922903) trabajo sobre aguas residuales 1
Juliana Hincapié Rojo
 
PDF
A tabela mostra as dimensões disponíveis em mm para pedido direto incluindo o...
Marcelo Rosa
 
PDF
Malaysia Stock MArket : Trading Stratagy for KLCI 10th April
Alex Gray
 
PDF
Hướng dẫn sử dụng máy đo tốc độ vòng quay Tenmars TM-4100
Tenmars Việt Nam
 
PDF
Zscoretable
Abril Mt
 
PDF
Trading Strategy for 7th oct 2013
Alex Gray
 
PDF
Mesh to micron conversion chart
FernandoGranados28
 
PDF
Standard resistor and_capaciter_values
Vishwanada
 
PDF
RP Consumer Discretionary ex Autos performance 2011
Marco Motta
 
Port game
Yopi Han
 
SPICE MODEL of ZR6_RL=4.7(Ohm) in SPICE PARK
Tsuyoshi Horigome
 
Footing-Biaxial_pdf-a.pdf
ANANDA DS
 
Tim Rainey - Tameside MBC - channel shift
Socitm
 
Naca 4 digit-delta
Phan Quoc Thien
 
Tabla z
Ángel Luis Vicentín
 
Balance in SNS
we20
 
Balance in sns using energy
Son Aris
 
Spec00446
guest2f67152
 
Epgp09 scm - group submission - group iii - merloni sp a
Rajendra Inani
 
2017_10_31_BIF_Spectrum_Presentation_Revised_071117.pptx
Miladrazi1
 
(597922903) trabajo sobre aguas residuales 1
Juliana Hincapié Rojo
 
A tabela mostra as dimensões disponíveis em mm para pedido direto incluindo o...
Marcelo Rosa
 
Malaysia Stock MArket : Trading Stratagy for KLCI 10th April
Alex Gray
 
Hướng dẫn sử dụng máy đo tốc độ vòng quay Tenmars TM-4100
Tenmars Việt Nam
 
Zscoretable
Abril Mt
 
Trading Strategy for 7th oct 2013
Alex Gray
 
Mesh to micron conversion chart
FernandoGranados28
 
Standard resistor and_capaciter_values
Vishwanada
 
RP Consumer Discretionary ex Autos performance 2011
Marco Motta
 
Ad

More from Jan Jongboom (20)

PDF
Tiny intelligent computers and sensors - Open Hardware Event 2020
Jan Jongboom
 
PDF
Machine learning on 1 square centimeter - Emerce Next 2019
Jan Jongboom
 
PDF
Fundamentals of IoT - Data Science Africa 2019
Jan Jongboom
 
PDF
17,000 contributions in 32K RAM - FOSS North 2019
Jan Jongboom
 
PDF
Open Hours: Mbed Simulator
Jan Jongboom
 
PDF
Efficient IoT solutions based on LoRaWAN, The Things Network and Mbed OS
Jan Jongboom
 
PDF
Machine learning on 1 cm2 - Tweakers Dev Summit
Jan Jongboom
 
PDF
Simulating LoRaWAN devices - LoRa Alliance AMM 2019
Jan Jongboom
 
PDF
Develop with Mbed OS - The Things Conference 2019
Jan Jongboom
 
PDF
Firmware Updates over LoRaWAN - The Things Conference 2019
Jan Jongboom
 
PDF
Faster Device Development - GSMA @ CES 2019
Jan Jongboom
 
PDF
Mbed LoRaWAN stack: a case study - LoRa Alliance AMM Tokyo
Jan Jongboom
 
PDF
Simulating microcontrollers - Arm Research Summit 2018
Jan Jongboom
 
PDF
Introduction to Mbed - Etteplan seminar - August 2018
Jan Jongboom
 
PDF
Machine Learning on 1 cm2 - Daho.am 2018
Jan Jongboom
 
PDF
Fundamentals of IoT - Data Science Africa 2018
Jan Jongboom
 
PDF
Intelligent IoT Devices - IoT Bootcamp - Smart City Gothenburg 2018
Jan Jongboom
 
PDF
Embedded Development: meet the web browser - TEQNation 2018
Jan Jongboom
 
PDF
Machine learning on microcontrollers - Tech Power Summit 2018
Jan Jongboom
 
PDF
Deep learning on microcontrollers - IETF 101 - T2TRG
Jan Jongboom
 
Tiny intelligent computers and sensors - Open Hardware Event 2020
Jan Jongboom
 
Machine learning on 1 square centimeter - Emerce Next 2019
Jan Jongboom
 
Fundamentals of IoT - Data Science Africa 2019
Jan Jongboom
 
17,000 contributions in 32K RAM - FOSS North 2019
Jan Jongboom
 
Open Hours: Mbed Simulator
Jan Jongboom
 
Efficient IoT solutions based on LoRaWAN, The Things Network and Mbed OS
Jan Jongboom
 
Machine learning on 1 cm2 - Tweakers Dev Summit
Jan Jongboom
 
Simulating LoRaWAN devices - LoRa Alliance AMM 2019
Jan Jongboom
 
Develop with Mbed OS - The Things Conference 2019
Jan Jongboom
 
Firmware Updates over LoRaWAN - The Things Conference 2019
Jan Jongboom
 
Faster Device Development - GSMA @ CES 2019
Jan Jongboom
 
Mbed LoRaWAN stack: a case study - LoRa Alliance AMM Tokyo
Jan Jongboom
 
Simulating microcontrollers - Arm Research Summit 2018
Jan Jongboom
 
Introduction to Mbed - Etteplan seminar - August 2018
Jan Jongboom
 
Machine Learning on 1 cm2 - Daho.am 2018
Jan Jongboom
 
Fundamentals of IoT - Data Science Africa 2018
Jan Jongboom
 
Intelligent IoT Devices - IoT Bootcamp - Smart City Gothenburg 2018
Jan Jongboom
 
Embedded Development: meet the web browser - TEQNation 2018
Jan Jongboom
 
Machine learning on microcontrollers - Tech Power Summit 2018
Jan Jongboom
 
Deep learning on microcontrollers - IETF 101 - T2TRG
Jan Jongboom
 

Recently uploaded (20)

PPTX
Artificial_Intelligence_Basics use in our daily life
ANUBHABPANIGRAHI3
 
PDF
Top 8 Trusted Sources to Buy Verified Cash App Accounts.pdf
How To Buy Verified Cash App Accounts
 
PPTX
Top Website Bugs That Hurt User Experience – And How Expert Web Design Fixes
e-Definers Technology
 
PPTX
Digital Project Mastery using Autodesk Docs Workshops
waliyuddin3dtech
 
PPTX
Cyber Hygine IN organizations in MSME or
paramkiet
 
PDF
Exploring The Internet Of Things(IOT).ppt
rakeshk19831911
 
PDF
Alethe Consulting Corporate Profile and Solution Aproach
debashisrakshit2025
 
PDF
Course Overview and Agenda cloud security
ElsayedKowaydeh
 
PDF
Buy Cash App Verified Accounts Instantly – Secure Crypto Deal.pdf
pvaonit
 
PPTX
Reading as a good Form of Recreation
Raj Kumble
 
PPT
12 Things That Make People Trust a Website Instantly
Shethil Ahammed
 
PDF
Public for study about wiring to confirm.
wongsa2524
 
PPTX
Basic understanding of cloud computing one need
subhosvims
 
PPTX
10.2981-wlb.2004.021Figurewlb3bf00068fig0001.pptx
katherinecapunays
 
DOCX
Powerful Ways AIRCONNECT INFOSYSTEMS Pvt Ltd Enhances IT Infrastructure in In...
Airconnect pvtltd
 
PDF
healthwealthtech4all-blogspot-com-2025-08-top-5-tech-innovations-that-will-ht...
nareshkumargovindan2
 
PPTX
Partner to Customer - Sales Presentation_V23.01.pptx
ssusera1629e
 
PDF
Alethe Consulting Corporate Profile and Solution Aproach
debashisrakshit2025
 
PDF
KEY COB2 UNIT 1: The Business of businessĐH KInh tế TP.HCM
vannhieu190219
 
PDF
Containerization lab dddddddddddddddmanual.pdf
muler161921
 
Artificial_Intelligence_Basics use in our daily life
ANUBHABPANIGRAHI3
 
Top 8 Trusted Sources to Buy Verified Cash App Accounts.pdf
How To Buy Verified Cash App Accounts
 
Top Website Bugs That Hurt User Experience – And How Expert Web Design Fixes
e-Definers Technology
 
Digital Project Mastery using Autodesk Docs Workshops
waliyuddin3dtech
 
Cyber Hygine IN organizations in MSME or
paramkiet
 
Exploring The Internet Of Things(IOT).ppt
rakeshk19831911
 
Alethe Consulting Corporate Profile and Solution Aproach
debashisrakshit2025
 
Course Overview and Agenda cloud security
ElsayedKowaydeh
 
Buy Cash App Verified Accounts Instantly – Secure Crypto Deal.pdf
pvaonit
 
Reading as a good Form of Recreation
Raj Kumble
 
12 Things That Make People Trust a Website Instantly
Shethil Ahammed
 
Public for study about wiring to confirm.
wongsa2524
 
Basic understanding of cloud computing one need
subhosvims
 
10.2981-wlb.2004.021Figurewlb3bf00068fig0001.pptx
katherinecapunays
 
Powerful Ways AIRCONNECT INFOSYSTEMS Pvt Ltd Enhances IT Infrastructure in In...
Airconnect pvtltd
 
healthwealthtech4all-blogspot-com-2025-08-top-5-tech-innovations-that-will-ht...
nareshkumargovindan2
 
Partner to Customer - Sales Presentation_V23.01.pptx
ssusera1629e
 
Alethe Consulting Corporate Profile and Solution Aproach
debashisrakshit2025
 
KEY COB2 UNIT 1: The Business of businessĐH KInh tế TP.HCM
vannhieu190219
 
Containerization lab dddddddddddddddmanual.pdf
muler161921
 

Adding intelligence to your LoRaWAN devices - The Things Conference on tour

  • 1. Adding intelligence to your LoRaWAN devices Jan Jongboom, Edge Impulse
  • 2. Jan Jongboom CTO and co-founder, Edge Impulse [email protected]
  • 3. 3 Typical LoRaWAN sensor in 2019 Vibration sensor (up to 1,000 times per second) Temperature sensor NFC Water & explosion proof Processor capable of running >20 million
 instructions per second
  • 4. 4 But... what does it actually do? Once an hour: • Average motion (RMS) • Peak motion • Current temperature
  • 5. 5 99% of sensor data is discarded due to 
 cost, bandwidth or power constraints. https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/McKinsey%20Digital/Our%20Insights/ The%20Internet%20of%20Things%20The%20value%20of%20digitizing%20the%20physical%20world/The- Internet-of-things-Mapping-the-value-beyond-the-hype.ashx
  • 6. 6 Lots of interesting data gets lost Peak
  • 7. 7 Single numbers can be misleading updown circle avg. RMS 3.3650 3.3515
  • 8. 8 Interesting questions require more data Classification
 What's happening right now? 
 Anomaly detection
 Is this behavior out of the ordinary? 
 Forecasting
 What will happen in the future?
  • 9. 9 Two problems 1. Very hard to answer with rule-based programming if (accX > 9.5) { lorawan.send('someone picked me up'); } 2. Needs to happen on-device, because 
 of this little pesky thing called physics
  • 10. Copyright © 2019 EdgeImpulse Inc. Let's fix this!
  • 11. 11 Step 1 - Getting raw data High-resolution data straight from devices (100 Hz) Correct labeling Offloading probably not over LoRaWAN (but signaling could) https://pixabay.com/photos/factory-night-view-industrial-pipe-1769429/
  • 12. 12 Raw data (3 samples)
  • 13. 13 Raw data (2,000 samples)
  • 14. 14 Step 2 - Extracting meaningful features Very dependent on your use case Raw data can be notoriously hard to deal with (3s. accelerometer data = 900 data points) Raw data is messy -1.1200, 0.5300, 10.6300, -0.5200, 1.1600, 13.1600, -0.1600, 1.4200, 13.5900, -0.2400, 1.3200, 10.8500, -0.3700, 0.5100, 7.7800, -0.1900, 0.0500, 8.8400, -0.1900, 0.0500, 8.8400, 0.2400, 0.7300, 11.1900, 0.5200, 1.6500, 12.5200, 0.6400, 1.5000, 10.8200, 0.2900, 0.3300, 7.4000, -0.3100, -0.5600, 8.5900, -0.8800, 0.6600, 11.9200, -0.8800, 0.6600, 11.9200, -0.4500, 1.3900, 11.9100, -0.0800, 1.6000, 12.7800, -0.1100, 0.3600, 9.1700, -0.0200, 0.0700, 9.8200, 0.0600, 0.9600, 12.0000, 0.2800, 1.7700, 13.2000, 0.2800, 1.7700, 13.2000, 0.2300, 1.5100, 12.6000, 0.2700, 1.0800, 11.4300, 0.0900, 0.9300, 10.9400, 0.1700, 1.2100, 11.0400, 0.4100, 1.8500, 11.5000, 0.3900, 1.9600, 11.4300, 0.3900, 1.9600, 11.4300, 0.2400, 1.4400, 10.7200, 0.0300, 1.1900, 10.3600, -0.0300, 1.3000, 10.6100, 0.4800, 1.7900, 11.7200, 1.0400, 2.6300, 13.3300, 1.0400, 2.6300, 13.3300, 1.0600, 2.3700, 13.5800, 0.3600, 1.9600, 13.3800, -0.1000, 2.1200, 13.9600, -0.3600, 2.0200, 14.5300, 0.0000, 2.1500, 14.6900, 0.0600, 2.1400, 14.3700, 0.0600, 2.1400, 14.3700, -0.3000, 1.8800, 13.7900, 0.0500, 1.7000, 13.5900, 0.1300, 1.6700, 13.1500, -0.0100, 1.7700, 12.9000, 0.4000, 1.8900, 12.2300, 0.5300, 2.3300, 12.2600, 0.5300, 2.3300, 12.2600, 0.0400, 1.9500, 11.8100, -0.2300, 1.9600, 11.2400, -0.0600, 2.1100, 10.2200, -0.1100, 2.4100, 9.7800, -0.3500, 2.7100, 9.7500, -0.7800, 3.1000, 10.1100, -0.7800, 3.1000, 10.1100, -1.0700, 3.1100, 9.8000, -1.2100, 2.9400, 9.1900, -1.1500, 3.2100, 8.6400, -0.7300, 3.6500, 8.4600, -0.5000, 3.9500, 8.6300, -0.4300, 3.9100, 8.7400, -0.4300, 3.9100, 8.7400, -0.6400, 3.7800, 8.8700, -1.2000, 3.9200, 8.9300, -1.0800, 4.4400, 8.8100, -0.7800, 4.1900, 8.1200, -0.4400, 4.1000, 7.6400, -0.5400, 4.2000, 7.5600, -0.5400, 4.2000, 7.5600, -1.0700, 4.2600, 7.2700, -1.3000, 4.5100, 7.2300, -1.2600, 4.4600, 6.6900, -1.2800, 4.4100, 6.6000, -1.7000, 4.6800, 7.0800, -2.3400, 5.1100, 7.5900, -2.3400, 5.1100, 7.5900, -2.8300, 4.8700, 6.8700, -2.9700, 4.7600, 6.2700, -3.2500, 4.6000, 6.1500, -3.4900, 4.5900, 6.2600, -3.3000, 4.9200, 6.3400, -2.7000, 4.9300, 5.8600, -2.7000, 4.9300, 5.8600, -2.9000, 4.5100, 4.9900, -3.5200, 4.3200, 4.9900, -4.1400, 4.2100, 5.7800, -3.7600, 4.1600, 5.7700, -3.0200, 4.2200, 5.4900, -3.0000, 3.8900, 3.9100, -3.0000, 3.8900, 3.9100, -3.3500, 3.5800, 3.4400, -3.1100, 3.2000, 3.6000, -3.0900, 3.6000, 5.9400, -3.0800, 3.0900, 5.1800, -2.8000, 2.9400, 4.5600, -2.4000, 2.4900, 3.9200, -2.4000, 2.4900, 3.9200, -1.8500, 2.6300, 4.4900, -1.4300, 3.9900, 7.3400, -1.4900, 3.4900, 6.1600, -1.5300, 3.2100, 5.4500, -0.9900, 2.8600, 6.2400, -0.7900, 3.2200, 8.4700, -0.7900, 3.2200, 8.4700, -0.9300, 3.5700, 9.0300, -1.6600, 2.9600, 7.0700, -1.7600, 2.1000, 6.9000, -1.4600, 2.1100, 9.0100, -1.3000, 2.5400, 10.3000, -1.3000, 2.5400, 10.3000, -1.4500, 2.6500, 9.7800, -1.5300, 1.8900, 8.4100, -1.1400, 1.3000, 9.4400, -0.7500, 1.6100, 10.3600, -0.9400, 1.5300, 9.7800, -1.0700, 1.0100, 9.2700, -1.0700, 1.0100, 9.2700, -0.8600, 1.0200, 10.1100, 0.3900, 1.6800, 11.3200, 1.2900, 1.8500, 11.4700, 1.0700, 1.3200, 11.2500, -0.2100, 1.5800, 12.4300, -1.8100, 1.3500, 12.3300, -1.8100, 1.3500, 12.3300, -2.1800, 1.0400, 11.1600, -1.5400, 0.3000, 10.3100, -0.4700, 0.2700, 11.0900, 0.7900, 1.4100, 12.9800, 1.1600, 1.7100, 12.4700, 0.5800, 0.8700, 9.6300, 0.5800, 0.8700, 9.6300, 0.1300, 0.3100, 9.5600, 0.2800, 0.3600, 10.5600, 0.7400, 0.8500, 11.4200, 0.9300, 1.0200, 11.4300, 0.6700, 0.5600, 10.5300, 0.8500, 0.3500, 9.4100, 0.8500, 0.3500, 9.4100, 1.6600, 1.2800, 10.9200, 2.0500, 0.9900, 9.7000, 2.1300, 0.8800, 10.1900, 2.0500, 0.9100, 11.3300, 1.7700, 1.4100, 12.2700, 1.4800, 1.7600, 12.1000, 1.4800, 1.7600, 12.1000, 0.9400, 1.1300, 10.8500, 0.2000, 0.8000, 10.1200, 0.2600, 1.1600, 10.5800, 0.5100, 1.6500, 10.7800, 0.4600, 1.2000, 9.9900, 0.9100, 0.8400, 9.6400, 0.9100, 0.8400, 9.6400, 1.4500, 0.7400, 10.2500, 2.0200, 1.3000, 11.4500, 1.8100, 1.8700, 12.1300, 1.0500, 1.5300, 12.0200, 0.6200, 0.6700, 11.3100, 0.7100, 0.8500, 12.0000, 0.7100, 0.8500, 12.0000, 0.6400, 1.2200, 13.1400, 1.1300, 2.0400, 14.6200, 0.8300, 2.0200, 15.5100, -0.1400, 1.4800, 15.6500, -0.6300, 1.5900, 16.0500, -1.3100, 1.7100, 16.3900, -1.3100, 1.7100, 16.3900, -1.7300, 1.5800, 16.6300, -1.1500, 1.4400, 16.0300, -0.5300, 1.1700, 15.1000, -0.1800, 0.9900, 14.4600, -0.3300, 1.0100, 13.5100, -0.3300, 1.0100, 13.5100, -0.4400, 0.9100, 12.6700, 0.0400, 1.2300, 12.5400, 0.6900, 2.0500, 13.1600, 0.3100, 1.7700, 12.8600, 0.0300, 1.3800, 11.1100, -0.4400, 1.2200, 9.4900, -0.4400, 1.2200, 9.4900, 0.1100, 1.1400, 7.3100, 0.8500, 2.2500, 8.4600, 0.8600, 3.3700, 11.2200, -0.1100, 2.2800, 8.4400, -1.3800, 1.5300, 7.1700, -1.0600, 1.5400, 6.9500, -1.0600, 1.5400, 6.9500, -0.5200, 2.8300, 8.7100, -0.2100, 2.3500, 8.1800, -0.3400, 2.7000, 8.9200, -0.3000, 2.3100, 8.7500, -0.4800, 1.4700, 7.8700, -0.3600, 0.9400, 6.9700, -0.3600, 0.9400, 6.9700, -0.2300, 1.4700, 7.6100, -0.3300, 2.2300, 8.5000, 0.3000, 1.9200, 7.8600, -0.2300, 1.5700, 6.8700, -1.4900, 1.5600, 6.3700, -2.8200, 1.6200, 7.2000, -2.8200, 1.6200, 7.2000, -3.1600, 1.8800, 7.1500, -2.7600, 2.2900, 6.8500, -2.6000, 2.2200, 6.2600, -2.9000, 1.9900, 5.8900, -3.3800, 2.2200, 6.2600, -3.9000, 2.1700, 6.0300, -3.9000, 2.1700, 6.0300, -3.8600, 2.3800, 5.6600, -3.5300, 2.5200, 5.6700, -3.2400, 2.3700, 5.8200, -3.2800, 2.1800, 5.5200, -3.1500, 2.1800, 5.6500, -3.0900, 2.0700, 5.1600, -3.0900, 2.0700, 5.1600, -2.4300, 2.1000, 5.3800, -2.0200, 2.3600, 6.0800, -2.0000, 2.5200, 6.4500, -2.2400, 2.4500, 6.0000, -2.0500, 1.8400, 4.6500, -1.3800, 1.3000, 4.6400, -1.3800, 1.3000, 4.6400, -1.2800, 1.8600, 6.9400, -1.3000, 2.5600, 9.0300, -1.5400, 2.7600, 8.5000, -1.7700, 1.6400, 6.1400, -1.6800, 1.4200, 7.5900, -1.3200, 2.0800, 9.8300, -1.3200, 2.0800, 9.8300, -0.8200, 2.1600, 10.3900, -0.7800, 1.7300, 9.8300, -1.1300, 1.3400, 9.7100, -1.3600, 1.6800, 10.2400, -1.5200, 1.6000, 9.3200, -1.8700, 1.4900, 9.1900, -1.8700, 1.4900, 9.1900, -1.9300, 1.0600, 9.9500, -1.3100, 0.8100, 10.6900, 0.0200, 2.0400, 11.0600, 0.2700, 2.5800, 9.3900, -0.0500, 2.2800, 7.3200, -0.3000, 0.4400, 7.6300, -0.3000, 0.4400, 7.6300, -1.4600, 1.0800, 12.3700, -1.9600, 1.7500, 15.3800, -0.7100, 2.1500, 14.0700, 0.7400, 1.7800, 10.4700, 0.6800, 0.8900, 9.9500, 0.0400, 1.5200, 12.0800, 0.0400, 1.5200, 12.0800, -0.4900, 1.7900, 12.7500
  • 15. 15 Example of a signal processing pipeline 32,000 => 240
  • 16. 16 Example of a signal processing pipeline Raw signal Low-pass filter Frequency domain Spectral power Peaks RMS 900 => 33
  • 17. 17 Raw data (2,000 samples)
  • 18. 18 After signal processing (2,000 samples)
  • 19. Classification
 What's happening right now? 
 Anomaly detection
 Is this behavior out of the ordinary? 
 Forecasting
 What will happen in the future? 19 Step 3 - Letting the computers figure it out
  • 20. 20 Picking the right algorithm Classification
 Neural network 
 Anomaly detection
 K-means clustering 
 Forecasting
 Regression
  • 21. 21 Neural networks Now possible on device Size of the network still matters (code size + ops) Signal processing is key to smaller networks
  • 22. 22 Neural networks aren't the only game in town Classic ML algorithms are much smaller K-means clustering is super efficient 
 (just compare new data against clusters) Combining classic ML and NN is great
  • 23. 23 Step 4 - Deploying Don't continuously sample. Monitor model performance. Something weird? Send DSP result back to network.
  • 25. 25 Gesture detection Idle Snake Updown Wave Circle
  • 26. 26 Collecting data Collect on same device and same sensor Store raw data in flash Sync via WiFi or serial Labeling directly on device Capture all variations DATA COLLECTED 12m 1s
  • 27. 27
  • 28. 28 Neural network for classification Input layer 
 (33 neurons) Hidden layer
 (20 neurons) Hidden layer
 (10 neurons) Output layer
 (5 neurons) (33 x 20) + (20 x 10) + (10 x 5) = 910 operations
  • 29. 29 Dealing with unknown data Neural network: 85% updown, 15% circle
  • 30. 29 Dealing with unknown data Neural network: 85% updown, 15% circle https://pixabay.com/photos/confused-hands-up-unsure-perplexed-2681507/
  • 31. 30 Clustering Good Bad 32 clusters to compare, very efficient
  • 32. 31 On device Signal processing Neural network converted with uTensor (Arm) Clustering code (On Cortex-M4F, 80 MHz) 16 ms. 1 ms. 40 ms.
  • 33. 32
  • 35. Recap The ML hype is real ML + LoRaWAN = perfect fit Start using the remaining 99% of sensor data edgeimpulse.com