SlideShare a Scribd company logo
Intro to TinyML
Technical, implementation with Arduino
Arduino Day 2020 Online Event

Bandung, Indonesia

Mar 21, 2020
Andri Yadi
CEO, DycodeX

Microsoft MVP, Azure
Introduction to AIoT & TinyML - with Arduino
Andri Yadi
Co-founder & CEO of DycodeX
Vice Chairman, Indonesia IoT Association (ASIOTI)
Microsoft MVP, Azure
a (at) dycodex.com | andriyadi.com | github.com/andriyadi
Physicist, Developer, Maker, Community Guy, Entrepreneur
About Me
MicrosoJ Most Valuable Professional (MVP) for 12 years

Code for food & passion for 20 years

Break & make electronic stuOs for 22 years

Trying to change the world through entrepreneurship, 15 years now
I’VE BEEN PROMOTING
AIOT & TINYML
20+ TALKS LAST YEAR
Communication

Networks
IoT High Level Architecture
Gateways /

Base Station
Things
(lots of them)
Internet
Apps
Cloud User
Internet
Ingestion
Infrastructure
Logics
API
GNSS
176
Gyroscope
130
Accelerometer
21
Bluetooth
100
Radio
800
Display
400
in mW
Clearly Radio consumes
almost 50% of power.
65% if no display!
Typical IoT Device’s Energy Consumption
GNSS
176
Gyroscope
130
Accelerometer
21
Bluetooth
100
Radio
800
Display
400
in mW
Clearly Radio consumes
almost 50% of power.
65% if no display!
Typical IoT Device’s Energy Consumption
LPWA for IoT wireless
connectivity is the option
LPWA = Low Power Wide Area Network
You oKen see…
about LPWA connectivity
You oKen see…
10 years baOery life is
GIMMICK!
about LPWA connectivity
You oKen see…
Achievable by 2-3 of these combinations:

Using big capacity ba_ery (e.g. 19Ah)

Transmi_ing data few times a day

Payload size 10s-100s bytes ➙ shob tx
10 years baOery life is
GIMMICK!
Notes:

NB-IoT (or other LPWA connectivities) does consume
much lower power than WiFi or 4G/LTE, due to its low
data rate. Only tens to hundreds mA per-tx
about LPWA connectivity
So, if you have…
Sensors:
GPS / GNSS
Accelerometer, Gyroscope
Body Temperature
Ambient Temperature & Humidity
Barometric Pressure
Ambient Light
MEMS Microphone
Device Removal Detector
SMARTernak electronics board
So, if you have…
Sensors:
GPS / GNSS
Accelerometer, Gyroscope
Body Temperature
Ambient Temperature & Humidity
Barometric Pressure
Ambient Light
MEMS Microphone
Device Removal Detector
You may only transmit:
Current lat, long, speed, direction
Current values or average motion
Current, peak, or lowest body temperature
Current, peak, or avg. temperature & humidity
Current, peak, or avg. Barometric Pressure
Current, peak, or lowest ambient Light
Current, peak, or lowest sound level in db
Detached or not
SMARTernak electronics board
99% of sensor data has to be discarded due to
connectivity’s bandwidth or power constraints
Motor failure detection
Dashboard
For that, usually we collect raw data from vibration sensor and transmit to the Cloud.
In the cloud, raw data are processed to classify whether motor is working normal or fault.
On the other hand, consider a use case…
MicrocontrollerMotor
Motor failure detection
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
Dashboard
For that, usually we collect raw data from vibration sensor and transmit to the Cloud.
In the cloud, raw data are processed to classify whether motor is working normal or fault.
On the other hand, consider a use case…
MicrocontrollerMotor
Motor failure detection
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
Dashboard
For that, usually we collect raw data from vibration sensor and transmit to the Cloud.
In the cloud, raw data are processed to classify whether motor is working normal or fault.
On the other hand, consider a use case…
MicrocontrollerMotor
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
Motor failure detection
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
Fault

detected
Dashboard
For that, usually we collect raw data from vibration sensor and transmit to the Cloud.
In the cloud, raw data are processed to classify whether motor is working normal or fault.
On the other hand, consider a use case…
MicrocontrollerMotor
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
Motor failure detection
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
Fault

detected
Dashboard
If we try to send 1KB of raw data over LPWA (e.g. LoRa),
it may take “forever”, and consume more power
For that, usually we collect raw data from vibration sensor and transmit to the Cloud.
In the cloud, raw data are processed to classify whether motor is working normal or fault.
On the other hand, consider a use case…
MicrocontrollerMotor
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
…it will beat the purpose of low power connectivity
LPWA
Motor failure detection
Dashboard
Due to bandwidth and power constraints, we only send sampled and calculated data
So, what we may do, instead
MicrocontrollerMotor
Peak
Peak value
FFT
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
Sampled/Calculated
LPWA
Motor failure detection
Dashboard
Due to bandwidth and power constraints, we only send sampled and calculated data
So, what we may do, instead
MicrocontrollerMotor
But we may miss lot of interesting events!
Peak
Peak value
FFT
Fault

detected
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
Sampled/Calculated
LPWA
Motor failure detection
Dashboard
There’s on-device “intelligence” to do complete analysis, right on the device.
Then, only transmit the conclusion (just few bytes of data) to the cloud.
Somehow…
MicrocontrollerMotor
On-device
processing
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
LPWA
Motor failure detection
Dashboard
There’s on-device “intelligence” to do complete analysis, right on the device.
Then, only transmit the conclusion (just few bytes of data) to the cloud.
Somehow…
MicrocontrollerMotor
Processed data

0A 0B 01 (➙ Normal)

0A 0B 02 (➙ Fault)
On-device
processing
Fault

detected
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
LPWA
Motor failure detection
Dashboard
There’s on-device “intelligence” to do complete analysis, right on the device.
Then, only transmit the conclusion (just few bytes of data) to the cloud.
Somehow…
MicrocontrollerMotor
That may save power, bandwidth,
and deliver more complete analysis
Processed data

0A 0B 01 (➙ Normal)

0A 0B 02 (➙ Fault)
On-device
processing
Fault

detected
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
LPWA
Motor failure detection
Dashboard
There’s on-device “intelligence” to do complete analysis, right on the device.
Then, only transmit the conclusion (just few bytes of data) to the cloud.
Back to…
MicrocontrollerMotor
Processed data

0A 0B 01 (➙ Normal)

0A 0B 02 (➙ Fault)
On-device
processing
What is this? Machine Learning?
Fault

detected
That may save power, bandwidth,
and deliver more complete analysis
Vibration
raw data

5D 1B 16
56 01 41 …
(1K Bytes)
LPWA
Machine Learning is oKen associated with…
Machine Learning
ClassiYcation 

What’s happening right now?

Anomaly Detection

Is it “normal”?

Forecasting 

What will happen in the future?
Some ML problems
Credit: h_ps://www.slideshare.net/janjongboom/adding-intelligence-to-lorawan-devices-the-things-conference-2020
Machine Learning
ClassiYcation 

What’s happening right now?

Anomaly Detection

Is it “normal”?

Forecasting 

What will happen in the future?
Some ML problems
For those problems,
ML Inference is possible on
ultra-low power, low cost,
tiny Microcontroller
High pelormance: 

Less latency as no raw data need to go to cloud/backend

Still works omine: 

Machine Learning inference happens right on-device. 

No internet connectivity needed to get inference result from cloud.

Encient power and bandwidth consumption: 

Only transmit necessary inference result, which are small in data size. 

Possible to use Low Power Wide Area (LPWA) connectivity, e.g: LoRa or
NB-IoT

Using optimised Edge hardware, even possible to be powered by ba_ery
AI at the Edge
Why the hassle?
Introduction to AIoT & TinyML - with Arduino
So, how can we achieve
TinyML?
ML-accelerated
processor
a class of microprocessor
designed as hardware
acceleration for AI
applications, e.g. for
neural networks, machine
vision and machine
learning

Example:
GPU

FPGA

ASIC

DSP
Development
Board /Accessories
a standalone computing
system with ML-
accelerated processor
(main or co-) as a main or
additional development
unit

Example:
Coral DevBoard or USB
Accelerator

Intel Neural Compute Stick

Sipeed Maix

Arduino Nano 33 BLE Sense

ESPectro32
SoKware Tools
ML compilers

ML Model design tool

ML Model conveber

SDK & libraries

OS

Example:
TensorFlow & TensorFlow
Lite (TFLite)

TFLite for Microcontroller

OpenVino

Kendryte’s nncase

EdgeImpulse
Suppos
Documentation 

Model Zoo

Examples

Datasheets

Hardware SoKware
What do we need?
Optimize &
Conve

(compress, remove,
replace, conve)
Train ML Model

(on your awesome
machine or Cloud)
.h5 

.pb

.cabemodel

ONNX
ML Model
Intermediate
Representation

(IR) File
Transfer IR qle 

to Edge device
Inference

(Edge-optimized
Inference library)
Inference 

Result
AI at the Edge General Pipeline
Optimize

(TOCO)
Train ML Model

(on your awesome
machine or Cloud)
.h5 

.pb
TinyML Pipeline
TFLite
Flat

Buber
Conve

(xxd)
Integrate
into MCU
Firmware
C Byte
Array
Optimize

(TOCO)
Train ML Model

(on your awesome
machine or Cloud)
.h5 

.pb
TinyML Pipeline
TFLite
Flat

Buber
Conve

(xxd)
Integrate
into MCU
Firmware
C Byte
Array
Custom 

Logics
ML Model
Microcontroller (MCU) Firmware
TensorFlow Lite

for Microcontroller
TinyML Pipeline
e.g. leveraging Azure Machine Learning
Azure Machine Learning Pipeline
TensorFlow model Yle
Compress

(toco)TFLite
Flat

Buber
Conve
(xxd)
C Byte
ArrayIntegrate
into MCU
Firmware
TinyML Pipeline
Typical TinyML Model Training
Labelled 

Raw Data
Label1
Label2
Label3
Signal
Processing

(Low/High Pass Filter,

FFT)
Neural 

Network
Output 

Features
Credit: EdgeImpulse.com. Check it out!
Processing Blocks

To extract features
Learning Blocks

To classify new data. 

NN size can be signiqcantly small
due to the extracted features from
previous blocks
AKer Filter
Frequency Domain
using Arduino framework, TFLite, PlahormIO,
ESPectro32 board
Gesture Recognition
DEMO
NB-IoT + GPS/GNSS Kit LoRa/LoRaWAN +
GNSS Kit
Sensors & Actuators Kit
(20+ Peripherals)
3G + GNSS + Sensors
ESPectro32 v2
Optimized for learning STEM,
coding & electronics, also for fast
prototyping.

An AIoT Development Board:

Low-power, dual-core 240 MHz
Microcontroller 

Built-in WiFi & Bluetooth connectivity

7x7 LED matrix for user intelace

Sensors:
Accelerometer & Gyro using LSM6DS3,
Digital temperature sensor, phototransistor

microSD card slot

Extensible via:
40 RPi-compatible pins, Grove connectors,
Micro:bit connector

Extensible with Backpacks: 

NB-IoT/Cellular + GPS/GNSS, LoRa, Sensor
Kits, Motor Driver, and more
ESPectro32
Backpacks
hOps://shop.makestro.com/product/espectro32-v2/
Demo
git clone hOps://github.com/andriyadi/MagicWand-TFLite-ESP32
Train Model
If necessary. Then compress and conveb the model…. Or use pre-trained model, instead
Reference: h_ps://github.com/tensollow/tensollow/tree/master/tensollow/lite/micro/examples/magic_wand/train
Enjoy!
Open Serial Monitor to see the inference result
Recognised gestured also displayed on LED matrix*
* On latest source code, display is already rotated :)
Other Demo
Similar demo, but using Arduino Nano 33 BLE Sense board
hOps://www.youtube.com/watch?v=Lfv3WJnYhX0 hOps://github.com/andriyadi/MagicWand-TFLite-Arduino
TinyML:
TensorFlow Lite for Microcontrollers: h_ps://www.tensollow.org/lite/
microcontrollers

Great book: TinyML, by Daniel Situnayake, Pete Warden

Azure:
Azure Machine Learning: h_ps://azure.microsoJ.com/en-in/services/
machine-learning-service/

DycodeX:
ESPectro32 dev board: h_ps://shop.makestro.com/product/espectro32-
v2/ 

SMARTernak: h_ps://smabernak.com 

Other IoT + AI products & solutions: h_ps://dycodex.com 

Contact me:
andri (at) dycodex.com 

h_p://github.com/andriyadi | hOps://slideshare.net/andri_yadi/
Call to Action
Want to put “AI” in “BrAIns”?
… a.k.a. adopting AI + IoT -> AIoT?
AI + IoT enabler
Keep in touch
hi (at) dycodex.com | https://dycodex.com
Bandung, Indonesia
Ad

Recommended

Implémentation d’une solution de supervision de température et d’humidité pou...
Implémentation d’une solution de supervision de température et d’humidité pou...
Mohammed Lymame
 
Lecture 2: Estrus Detection in Cattle
Lecture 2: Estrus Detection in Cattle
Rabie Fayed
 
The process of translating
The process of translating
University of Panama
 
AI in Higher Education – Challenges & Opportunities #edlw2019
AI in Higher Education – Challenges & Opportunities #edlw2019
EDEN Digital Learning Europe
 
Electric vehicle-new 2018 charging presentation mahesh chandra manav by c&s
Electric vehicle-new 2018 charging presentation mahesh chandra manav by c&s
Mahesh Chandra Manav
 
Brushless DC Motors
Brushless DC Motors
Deepak Kumar Mohapatra
 
Regression analysis
Regression analysis
University of Jaffna
 
Introduction to Artificial Intelligence and Machine Learning
Introduction to Artificial Intelligence and Machine Learning
Emad Nabil
 
Introduction to TinyML - Solomon Muhunyo Githu
Introduction to TinyML - Solomon Muhunyo Githu
Solomon Githu
 
TinyML: Machine Learning for Microcontrollers
TinyML: Machine Learning for Microcontrollers
Robert John
 
Arduino in TinyML with Edge Impulse
Arduino in TinyML with Edge Impulse
Robocraze
 
Tensorflow
Tensorflow
marwa Ayad Mohamed
 
Machine Learning and Internet of Things
Machine Learning and Internet of Things
Sofian Hadiwijaya
 
Introduction to IOT
Introduction to IOT
Zubayer Al Billal Khan
 
TinyML on Arduino - workshop
TinyML on Arduino - workshop
Jan Jongboom
 
IoT Design Principles
IoT Design Principles
ardexateam
 
Intelligent Edge - Getting started with TinyML for industrial applications
Intelligent Edge - Getting started with TinyML for industrial applications
Jan Jongboom
 
Internet of Things
Internet of Things
pkshc01
 
Image classification using CNN
Image classification using CNN
Noura Hussein
 
IoT Networking
IoT Networking
Hitesh Mohapatra
 
Introduction To TensorFlow
Introduction To TensorFlow
Spotle.ai
 
Convolutional Neural Networks
Convolutional Neural Networks
milad abbasi
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
 
IoT implementation and Challenges
IoT implementation and Challenges
Ahmed Banafa
 
IoT Arduino UNO, RaspberryPi with Python, RaspberryPi Programming using Pytho...
IoT Arduino UNO, RaspberryPi with Python, RaspberryPi Programming using Pytho...
Jayanthi Kannan MK
 
AI at the Edge
AI at the Edge
DATAVERSITY
 
Computer science seminar topics
Computer science seminar topics
123seminarsonly
 
OpenCV presentation series- part 1
OpenCV presentation series- part 1
Sairam Adithya
 
Microcontrollers (Rex St. John)
Microcontrollers (Rex St. John)
Future Insights
 
IT Solution through IoT Development
IT Solution through IoT Development
Andri Yadi
 

More Related Content

What's hot (20)

Introduction to TinyML - Solomon Muhunyo Githu
Introduction to TinyML - Solomon Muhunyo Githu
Solomon Githu
 
TinyML: Machine Learning for Microcontrollers
TinyML: Machine Learning for Microcontrollers
Robert John
 
Arduino in TinyML with Edge Impulse
Arduino in TinyML with Edge Impulse
Robocraze
 
Tensorflow
Tensorflow
marwa Ayad Mohamed
 
Machine Learning and Internet of Things
Machine Learning and Internet of Things
Sofian Hadiwijaya
 
Introduction to IOT
Introduction to IOT
Zubayer Al Billal Khan
 
TinyML on Arduino - workshop
TinyML on Arduino - workshop
Jan Jongboom
 
IoT Design Principles
IoT Design Principles
ardexateam
 
Intelligent Edge - Getting started with TinyML for industrial applications
Intelligent Edge - Getting started with TinyML for industrial applications
Jan Jongboom
 
Internet of Things
Internet of Things
pkshc01
 
Image classification using CNN
Image classification using CNN
Noura Hussein
 
IoT Networking
IoT Networking
Hitesh Mohapatra
 
Introduction To TensorFlow
Introduction To TensorFlow
Spotle.ai
 
Convolutional Neural Networks
Convolutional Neural Networks
milad abbasi
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
 
IoT implementation and Challenges
IoT implementation and Challenges
Ahmed Banafa
 
IoT Arduino UNO, RaspberryPi with Python, RaspberryPi Programming using Pytho...
IoT Arduino UNO, RaspberryPi with Python, RaspberryPi Programming using Pytho...
Jayanthi Kannan MK
 
AI at the Edge
AI at the Edge
DATAVERSITY
 
Computer science seminar topics
Computer science seminar topics
123seminarsonly
 
OpenCV presentation series- part 1
OpenCV presentation series- part 1
Sairam Adithya
 
Introduction to TinyML - Solomon Muhunyo Githu
Introduction to TinyML - Solomon Muhunyo Githu
Solomon Githu
 
TinyML: Machine Learning for Microcontrollers
TinyML: Machine Learning for Microcontrollers
Robert John
 
Arduino in TinyML with Edge Impulse
Arduino in TinyML with Edge Impulse
Robocraze
 
Machine Learning and Internet of Things
Machine Learning and Internet of Things
Sofian Hadiwijaya
 
TinyML on Arduino - workshop
TinyML on Arduino - workshop
Jan Jongboom
 
IoT Design Principles
IoT Design Principles
ardexateam
 
Intelligent Edge - Getting started with TinyML for industrial applications
Intelligent Edge - Getting started with TinyML for industrial applications
Jan Jongboom
 
Internet of Things
Internet of Things
pkshc01
 
Image classification using CNN
Image classification using CNN
Noura Hussein
 
Introduction To TensorFlow
Introduction To TensorFlow
Spotle.ai
 
Convolutional Neural Networks
Convolutional Neural Networks
milad abbasi
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
 
IoT implementation and Challenges
IoT implementation and Challenges
Ahmed Banafa
 
IoT Arduino UNO, RaspberryPi with Python, RaspberryPi Programming using Pytho...
IoT Arduino UNO, RaspberryPi with Python, RaspberryPi Programming using Pytho...
Jayanthi Kannan MK
 
Computer science seminar topics
Computer science seminar topics
123seminarsonly
 
OpenCV presentation series- part 1
OpenCV presentation series- part 1
Sairam Adithya
 

Similar to Introduction to AIoT & TinyML - with Arduino (20)

Microcontrollers (Rex St. John)
Microcontrollers (Rex St. John)
Future Insights
 
IT Solution through IoT Development
IT Solution through IoT Development
Andri Yadi
 
AIoT: Intelligence on Microcontroller
AIoT: Intelligence on Microcontroller
Andri Yadi
 
Internet of thing workshop presentation.
Internet of thing workshop presentation.
KamilKlaime1
 
SIGFOX Makers Tour - Porto
SIGFOX Makers Tour - Porto
Nicolas Lesconnec
 
Energy Efficient Computing - 26mar13
Energy Efficient Computing - 26mar13
Ian Phillips
 
NSTA 2013 Denver - ArduBlock and Arduino
NSTA 2013 Denver - ArduBlock and Arduino
Brian Huang
 
Physical Computing and IoT
Physical Computing and IoT
Eduardo Oliveira
 
IOT beginnners
IOT beginnners
udhayakumarc1
 
IOT beginnners
IOT beginnners
udhayakumarc1
 
Embedded systems unit3
Embedded systems unit3
baskaransece
 
embedded_in_Arduino_with_basic_embedded.pptx
embedded_in_Arduino_with_basic_embedded.pptx
acloudinfo2023
 
IoThings you don't even need to hack
IoThings you don't even need to hack
Slawomir Jasek
 
Microprocessor Systems
Microprocessor Systems
Quaid-e-Awam University of Engineering Science and Technology Nawabshah Sindh Pakistan
 
SIGFOX Makers Tour - Dublin
SIGFOX Makers Tour - Dublin
Nicolas Lesconnec
 
Sudhir tms 320 f 2812
Sudhir tms 320 f 2812
vijaydeepakg
 
The (Io)Things you don't even need to hack. Should we worry?
The (Io)Things you don't even need to hack. Should we worry?
SecuRing
 
Iot for smart agriculture
Iot for smart agriculture
Atit Patumvan
 
Design & Implementation Of Fault Identification In Underground Cables Using IOT
Design & Implementation Of Fault Identification In Underground Cables Using IOT
IRJET Journal
 
An_Introduction_to_Microcontrollers.pptx
An_Introduction_to_Microcontrollers.pptx
Stefan Oprea
 
Microcontrollers (Rex St. John)
Microcontrollers (Rex St. John)
Future Insights
 
IT Solution through IoT Development
IT Solution through IoT Development
Andri Yadi
 
AIoT: Intelligence on Microcontroller
AIoT: Intelligence on Microcontroller
Andri Yadi
 
Internet of thing workshop presentation.
Internet of thing workshop presentation.
KamilKlaime1
 
Energy Efficient Computing - 26mar13
Energy Efficient Computing - 26mar13
Ian Phillips
 
NSTA 2013 Denver - ArduBlock and Arduino
NSTA 2013 Denver - ArduBlock and Arduino
Brian Huang
 
Physical Computing and IoT
Physical Computing and IoT
Eduardo Oliveira
 
Embedded systems unit3
Embedded systems unit3
baskaransece
 
embedded_in_Arduino_with_basic_embedded.pptx
embedded_in_Arduino_with_basic_embedded.pptx
acloudinfo2023
 
IoThings you don't even need to hack
IoThings you don't even need to hack
Slawomir Jasek
 
Sudhir tms 320 f 2812
Sudhir tms 320 f 2812
vijaydeepakg
 
The (Io)Things you don't even need to hack. Should we worry?
The (Io)Things you don't even need to hack. Should we worry?
SecuRing
 
Iot for smart agriculture
Iot for smart agriculture
Atit Patumvan
 
Design & Implementation Of Fault Identification In Underground Cables Using IOT
Design & Implementation Of Fault Identification In Underground Cables Using IOT
IRJET Journal
 
An_Introduction_to_Microcontrollers.pptx
An_Introduction_to_Microcontrollers.pptx
Stefan Oprea
 
Ad

More from Andri Yadi (20)

TechInAsia PDC 2019 - Unlocking The Potential of IoT with AI
TechInAsia PDC 2019 - Unlocking The Potential of IoT with AI
Andri Yadi
 
Global Azure Bootcamp 2019 - AIoT powered by Azure
Global Azure Bootcamp 2019 - AIoT powered by Azure
Andri Yadi
 
Opportunities & Challenges in IoT - Future of IoT industry in Indonesia 2019 ...
Opportunities & Challenges in IoT - Future of IoT industry in Indonesia 2019 ...
Andri Yadi
 
Microsoft Azure-powered IoT & AI Solution To Help Farmer
Microsoft Azure-powered IoT & AI Solution To Help Farmer
Andri Yadi
 
IoT Connectivity with LoRa
IoT Connectivity with LoRa
Andri Yadi
 
Ask the Expert: Internet of Things
Ask the Expert: Internet of Things
Andri Yadi
 
Global Azure Bootcamp 2018 - Azure IoT Central
Global Azure Bootcamp 2018 - Azure IoT Central
Andri Yadi
 
Maker Movement toward IoT Ecosystem in Indonesia
Maker Movement toward IoT Ecosystem in Indonesia
Andri Yadi
 
IoT for Agriculture in a Nutshell: Technical Perspective
IoT for Agriculture in a Nutshell: Technical Perspective
Andri Yadi
 
Road to Republic of IoT - IoT Technologies & Machine Learning
Road to Republic of IoT - IoT Technologies & Machine Learning
Andri Yadi
 
IoT Connectivity: The Technical & Potential
IoT Connectivity: The Technical & Potential
Andri Yadi
 
Internet of Things - Technicals
Internet of Things - Technicals
Andri Yadi
 
Global Azure Bootcamp 2017 - Azure IoT Hub with LoRa Connectivity
Global Azure Bootcamp 2017 - Azure IoT Hub with LoRa Connectivity
Andri Yadi
 
The state of NB-IoT in Indonesia
The state of NB-IoT in Indonesia
Andri Yadi
 
Industrial IoT in a Nutshell
Industrial IoT in a Nutshell
Andri Yadi
 
The Rise of Maker Movement in Indonesia
The Rise of Maker Movement in Indonesia
Andri Yadi
 
Road to Republic of IoT - ESP32 Programming and LoRa
Road to Republic of IoT - ESP32 Programming and LoRa
Andri Yadi
 
Talking with Things: What's Hot in Low-Power Long-Range IoT Connectivity
Talking with Things: What's Hot in Low-Power Long-Range IoT Connectivity
Andri Yadi
 
Internet of Things: Intro and Demo
Internet of Things: Intro and Demo
Andri Yadi
 
Entrepreneurship through Making
Entrepreneurship through Making
Andri Yadi
 
TechInAsia PDC 2019 - Unlocking The Potential of IoT with AI
TechInAsia PDC 2019 - Unlocking The Potential of IoT with AI
Andri Yadi
 
Global Azure Bootcamp 2019 - AIoT powered by Azure
Global Azure Bootcamp 2019 - AIoT powered by Azure
Andri Yadi
 
Opportunities & Challenges in IoT - Future of IoT industry in Indonesia 2019 ...
Opportunities & Challenges in IoT - Future of IoT industry in Indonesia 2019 ...
Andri Yadi
 
Microsoft Azure-powered IoT & AI Solution To Help Farmer
Microsoft Azure-powered IoT & AI Solution To Help Farmer
Andri Yadi
 
IoT Connectivity with LoRa
IoT Connectivity with LoRa
Andri Yadi
 
Ask the Expert: Internet of Things
Ask the Expert: Internet of Things
Andri Yadi
 
Global Azure Bootcamp 2018 - Azure IoT Central
Global Azure Bootcamp 2018 - Azure IoT Central
Andri Yadi
 
Maker Movement toward IoT Ecosystem in Indonesia
Maker Movement toward IoT Ecosystem in Indonesia
Andri Yadi
 
IoT for Agriculture in a Nutshell: Technical Perspective
IoT for Agriculture in a Nutshell: Technical Perspective
Andri Yadi
 
Road to Republic of IoT - IoT Technologies & Machine Learning
Road to Republic of IoT - IoT Technologies & Machine Learning
Andri Yadi
 
IoT Connectivity: The Technical & Potential
IoT Connectivity: The Technical & Potential
Andri Yadi
 
Internet of Things - Technicals
Internet of Things - Technicals
Andri Yadi
 
Global Azure Bootcamp 2017 - Azure IoT Hub with LoRa Connectivity
Global Azure Bootcamp 2017 - Azure IoT Hub with LoRa Connectivity
Andri Yadi
 
The state of NB-IoT in Indonesia
The state of NB-IoT in Indonesia
Andri Yadi
 
Industrial IoT in a Nutshell
Industrial IoT in a Nutshell
Andri Yadi
 
The Rise of Maker Movement in Indonesia
The Rise of Maker Movement in Indonesia
Andri Yadi
 
Road to Republic of IoT - ESP32 Programming and LoRa
Road to Republic of IoT - ESP32 Programming and LoRa
Andri Yadi
 
Talking with Things: What's Hot in Low-Power Long-Range IoT Connectivity
Talking with Things: What's Hot in Low-Power Long-Range IoT Connectivity
Andri Yadi
 
Internet of Things: Intro and Demo
Internet of Things: Intro and Demo
Andri Yadi
 
Entrepreneurship through Making
Entrepreneurship through Making
Andri Yadi
 
Ad

Recently uploaded (20)

Crypto Super 500 - 14th Report - June2025.pdf
Crypto Super 500 - 14th Report - June2025.pdf
Stephen Perrenod
 
Mastering AI Workflows with FME - Peak of Data & AI 2025
Mastering AI Workflows with FME - Peak of Data & AI 2025
Safe Software
 
“Why It’s Critical to Have an Integrated Development Methodology for Edge AI,...
“Why It’s Critical to Have an Integrated Development Methodology for Edge AI,...
Edge AI and Vision Alliance
 
Bridging the divide: A conversation on tariffs today in the book industry - T...
Bridging the divide: A conversation on tariffs today in the book industry - T...
BookNet Canada
 
AI VIDEO MAGAZINE - June 2025 - r/aivideo
AI VIDEO MAGAZINE - June 2025 - r/aivideo
1pcity Studios, Inc
 
MuleSoft for AgentForce : Topic Center and API Catalog
MuleSoft for AgentForce : Topic Center and API Catalog
shyamraj55
 
Down the Rabbit Hole – Solving 5 Training Roadblocks
Down the Rabbit Hole – Solving 5 Training Roadblocks
Rustici Software
 
War_And_Cyber_3_Years_Of_Struggle_And_Lessons_For_Global_Security.pdf
War_And_Cyber_3_Years_Of_Struggle_And_Lessons_For_Global_Security.pdf
biswajitbanerjee38
 
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
caoyixuan2019
 
AudGram Review: Build Visually Appealing, AI-Enhanced Audiograms to Engage Yo...
AudGram Review: Build Visually Appealing, AI-Enhanced Audiograms to Engage Yo...
SOFTTECHHUB
 
Can We Use Rust to Develop Extensions for PostgreSQL? (POSETTE: An Event for ...
Can We Use Rust to Develop Extensions for PostgreSQL? (POSETTE: An Event for ...
NTT DATA Technology & Innovation
 
No-Code Workflows for CAD & 3D Data: Scaling AI-Driven Infrastructure
No-Code Workflows for CAD & 3D Data: Scaling AI-Driven Infrastructure
Safe Software
 
Securing Account Lifecycles in the Age of Deepfakes.pptx
Securing Account Lifecycles in the Age of Deepfakes.pptx
FIDO Alliance
 
Murdledescargadarkweb.pdfvolumen1 100 elementary
Murdledescargadarkweb.pdfvolumen1 100 elementary
JorgeSemperteguiMont
 
Creating Inclusive Digital Learning with AI: A Smarter, Fairer Future
Creating Inclusive Digital Learning with AI: A Smarter, Fairer Future
Impelsys Inc.
 
“From Enterprise to Makers: Driving Vision AI Innovation at the Extreme Edge,...
“From Enterprise to Makers: Driving Vision AI Innovation at the Extreme Edge,...
Edge AI and Vision Alliance
 
Edge-banding-machines-edgeteq-s-200-en-.pdf
Edge-banding-machines-edgeteq-s-200-en-.pdf
AmirStern2
 
Enabling BIM / GIS integrations with Other Systems with FME
Enabling BIM / GIS integrations with Other Systems with FME
Safe Software
 
June Patch Tuesday
June Patch Tuesday
Ivanti
 
National Fuels Treatments Initiative: Building a Seamless Map of Hazardous Fu...
National Fuels Treatments Initiative: Building a Seamless Map of Hazardous Fu...
Safe Software
 
Crypto Super 500 - 14th Report - June2025.pdf
Crypto Super 500 - 14th Report - June2025.pdf
Stephen Perrenod
 
Mastering AI Workflows with FME - Peak of Data & AI 2025
Mastering AI Workflows with FME - Peak of Data & AI 2025
Safe Software
 
“Why It’s Critical to Have an Integrated Development Methodology for Edge AI,...
“Why It’s Critical to Have an Integrated Development Methodology for Edge AI,...
Edge AI and Vision Alliance
 
Bridging the divide: A conversation on tariffs today in the book industry - T...
Bridging the divide: A conversation on tariffs today in the book industry - T...
BookNet Canada
 
AI VIDEO MAGAZINE - June 2025 - r/aivideo
AI VIDEO MAGAZINE - June 2025 - r/aivideo
1pcity Studios, Inc
 
MuleSoft for AgentForce : Topic Center and API Catalog
MuleSoft for AgentForce : Topic Center and API Catalog
shyamraj55
 
Down the Rabbit Hole – Solving 5 Training Roadblocks
Down the Rabbit Hole – Solving 5 Training Roadblocks
Rustici Software
 
War_And_Cyber_3_Years_Of_Struggle_And_Lessons_For_Global_Security.pdf
War_And_Cyber_3_Years_Of_Struggle_And_Lessons_For_Global_Security.pdf
biswajitbanerjee38
 
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
caoyixuan2019
 
AudGram Review: Build Visually Appealing, AI-Enhanced Audiograms to Engage Yo...
AudGram Review: Build Visually Appealing, AI-Enhanced Audiograms to Engage Yo...
SOFTTECHHUB
 
Can We Use Rust to Develop Extensions for PostgreSQL? (POSETTE: An Event for ...
Can We Use Rust to Develop Extensions for PostgreSQL? (POSETTE: An Event for ...
NTT DATA Technology & Innovation
 
No-Code Workflows for CAD & 3D Data: Scaling AI-Driven Infrastructure
No-Code Workflows for CAD & 3D Data: Scaling AI-Driven Infrastructure
Safe Software
 
Securing Account Lifecycles in the Age of Deepfakes.pptx
Securing Account Lifecycles in the Age of Deepfakes.pptx
FIDO Alliance
 
Murdledescargadarkweb.pdfvolumen1 100 elementary
Murdledescargadarkweb.pdfvolumen1 100 elementary
JorgeSemperteguiMont
 
Creating Inclusive Digital Learning with AI: A Smarter, Fairer Future
Creating Inclusive Digital Learning with AI: A Smarter, Fairer Future
Impelsys Inc.
 
“From Enterprise to Makers: Driving Vision AI Innovation at the Extreme Edge,...
“From Enterprise to Makers: Driving Vision AI Innovation at the Extreme Edge,...
Edge AI and Vision Alliance
 
Edge-banding-machines-edgeteq-s-200-en-.pdf
Edge-banding-machines-edgeteq-s-200-en-.pdf
AmirStern2
 
Enabling BIM / GIS integrations with Other Systems with FME
Enabling BIM / GIS integrations with Other Systems with FME
Safe Software
 
June Patch Tuesday
June Patch Tuesday
Ivanti
 
National Fuels Treatments Initiative: Building a Seamless Map of Hazardous Fu...
National Fuels Treatments Initiative: Building a Seamless Map of Hazardous Fu...
Safe Software
 

Introduction to AIoT & TinyML - with Arduino

  • 1. Intro to TinyML Technical, implementation with Arduino Arduino Day 2020 Online Event Bandung, Indonesia Mar 21, 2020 Andri Yadi CEO, DycodeX Microsoft MVP, Azure
  • 3. Andri Yadi Co-founder & CEO of DycodeX Vice Chairman, Indonesia IoT Association (ASIOTI) Microsoft MVP, Azure a (at) dycodex.com | andriyadi.com | github.com/andriyadi Physicist, Developer, Maker, Community Guy, Entrepreneur About Me MicrosoJ Most Valuable Professional (MVP) for 12 years Code for food & passion for 20 years Break & make electronic stuOs for 22 years Trying to change the world through entrepreneurship, 15 years now
  • 4. I’VE BEEN PROMOTING AIOT & TINYML 20+ TALKS LAST YEAR
  • 5. Communication Networks IoT High Level Architecture Gateways / Base Station Things (lots of them) Internet Apps Cloud User Internet Ingestion Infrastructure Logics API
  • 6. GNSS 176 Gyroscope 130 Accelerometer 21 Bluetooth 100 Radio 800 Display 400 in mW Clearly Radio consumes almost 50% of power. 65% if no display! Typical IoT Device’s Energy Consumption
  • 7. GNSS 176 Gyroscope 130 Accelerometer 21 Bluetooth 100 Radio 800 Display 400 in mW Clearly Radio consumes almost 50% of power. 65% if no display! Typical IoT Device’s Energy Consumption LPWA for IoT wireless connectivity is the option LPWA = Low Power Wide Area Network
  • 8. You oKen see… about LPWA connectivity
  • 9. You oKen see… 10 years baOery life is GIMMICK! about LPWA connectivity
  • 10. You oKen see… Achievable by 2-3 of these combinations: Using big capacity ba_ery (e.g. 19Ah) Transmi_ing data few times a day Payload size 10s-100s bytes ➙ shob tx 10 years baOery life is GIMMICK! Notes: NB-IoT (or other LPWA connectivities) does consume much lower power than WiFi or 4G/LTE, due to its low data rate. Only tens to hundreds mA per-tx about LPWA connectivity
  • 11. So, if you have… Sensors: GPS / GNSS Accelerometer, Gyroscope Body Temperature Ambient Temperature & Humidity Barometric Pressure Ambient Light MEMS Microphone Device Removal Detector SMARTernak electronics board
  • 12. So, if you have… Sensors: GPS / GNSS Accelerometer, Gyroscope Body Temperature Ambient Temperature & Humidity Barometric Pressure Ambient Light MEMS Microphone Device Removal Detector You may only transmit: Current lat, long, speed, direction Current values or average motion Current, peak, or lowest body temperature Current, peak, or avg. temperature & humidity Current, peak, or avg. Barometric Pressure Current, peak, or lowest ambient Light Current, peak, or lowest sound level in db Detached or not SMARTernak electronics board 99% of sensor data has to be discarded due to connectivity’s bandwidth or power constraints
  • 13. Motor failure detection Dashboard For that, usually we collect raw data from vibration sensor and transmit to the Cloud. In the cloud, raw data are processed to classify whether motor is working normal or fault. On the other hand, consider a use case… MicrocontrollerMotor
  • 14. Motor failure detection Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) Dashboard For that, usually we collect raw data from vibration sensor and transmit to the Cloud. In the cloud, raw data are processed to classify whether motor is working normal or fault. On the other hand, consider a use case… MicrocontrollerMotor
  • 15. Motor failure detection Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) Dashboard For that, usually we collect raw data from vibration sensor and transmit to the Cloud. In the cloud, raw data are processed to classify whether motor is working normal or fault. On the other hand, consider a use case… MicrocontrollerMotor Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes)
  • 16. Motor failure detection Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) Fault detected Dashboard For that, usually we collect raw data from vibration sensor and transmit to the Cloud. In the cloud, raw data are processed to classify whether motor is working normal or fault. On the other hand, consider a use case… MicrocontrollerMotor Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes)
  • 17. Motor failure detection Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) Fault detected Dashboard If we try to send 1KB of raw data over LPWA (e.g. LoRa), it may take “forever”, and consume more power For that, usually we collect raw data from vibration sensor and transmit to the Cloud. In the cloud, raw data are processed to classify whether motor is working normal or fault. On the other hand, consider a use case… MicrocontrollerMotor Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) …it will beat the purpose of low power connectivity LPWA
  • 18. Motor failure detection Dashboard Due to bandwidth and power constraints, we only send sampled and calculated data So, what we may do, instead MicrocontrollerMotor Peak Peak value FFT Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) Sampled/Calculated LPWA
  • 19. Motor failure detection Dashboard Due to bandwidth and power constraints, we only send sampled and calculated data So, what we may do, instead MicrocontrollerMotor But we may miss lot of interesting events! Peak Peak value FFT Fault detected Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) Sampled/Calculated LPWA
  • 20. Motor failure detection Dashboard There’s on-device “intelligence” to do complete analysis, right on the device. Then, only transmit the conclusion (just few bytes of data) to the cloud. Somehow… MicrocontrollerMotor On-device processing Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) LPWA
  • 21. Motor failure detection Dashboard There’s on-device “intelligence” to do complete analysis, right on the device. Then, only transmit the conclusion (just few bytes of data) to the cloud. Somehow… MicrocontrollerMotor Processed data 0A 0B 01 (➙ Normal) 0A 0B 02 (➙ Fault) On-device processing Fault detected Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) LPWA
  • 22. Motor failure detection Dashboard There’s on-device “intelligence” to do complete analysis, right on the device. Then, only transmit the conclusion (just few bytes of data) to the cloud. Somehow… MicrocontrollerMotor That may save power, bandwidth, and deliver more complete analysis Processed data 0A 0B 01 (➙ Normal) 0A 0B 02 (➙ Fault) On-device processing Fault detected Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) LPWA
  • 23. Motor failure detection Dashboard There’s on-device “intelligence” to do complete analysis, right on the device. Then, only transmit the conclusion (just few bytes of data) to the cloud. Back to… MicrocontrollerMotor Processed data 0A 0B 01 (➙ Normal) 0A 0B 02 (➙ Fault) On-device processing What is this? Machine Learning? Fault detected That may save power, bandwidth, and deliver more complete analysis Vibration raw data 5D 1B 16 56 01 41 … (1K Bytes) LPWA
  • 24. Machine Learning is oKen associated with…
  • 25. Machine Learning ClassiYcation What’s happening right now? Anomaly Detection Is it “normal”? Forecasting What will happen in the future? Some ML problems Credit: h_ps://www.slideshare.net/janjongboom/adding-intelligence-to-lorawan-devices-the-things-conference-2020
  • 26. Machine Learning ClassiYcation What’s happening right now? Anomaly Detection Is it “normal”? Forecasting What will happen in the future? Some ML problems For those problems, ML Inference is possible on ultra-low power, low cost, tiny Microcontroller
  • 27. High pelormance: Less latency as no raw data need to go to cloud/backend Still works omine: Machine Learning inference happens right on-device. No internet connectivity needed to get inference result from cloud. Encient power and bandwidth consumption: Only transmit necessary inference result, which are small in data size. Possible to use Low Power Wide Area (LPWA) connectivity, e.g: LoRa or NB-IoT Using optimised Edge hardware, even possible to be powered by ba_ery AI at the Edge Why the hassle?
  • 29. So, how can we achieve TinyML?
  • 30. ML-accelerated processor a class of microprocessor designed as hardware acceleration for AI applications, e.g. for neural networks, machine vision and machine learning Example: GPU FPGA ASIC DSP Development Board /Accessories a standalone computing system with ML- accelerated processor (main or co-) as a main or additional development unit Example: Coral DevBoard or USB Accelerator Intel Neural Compute Stick Sipeed Maix Arduino Nano 33 BLE Sense ESPectro32 SoKware Tools ML compilers ML Model design tool ML Model conveber SDK & libraries OS Example: TensorFlow & TensorFlow Lite (TFLite) TFLite for Microcontroller OpenVino Kendryte’s nncase EdgeImpulse Suppos Documentation Model Zoo Examples Datasheets Hardware SoKware What do we need?
  • 31. Optimize & Conve (compress, remove, replace, conve) Train ML Model (on your awesome machine or Cloud) .h5 .pb .cabemodel ONNX ML Model Intermediate Representation (IR) File Transfer IR qle to Edge device Inference (Edge-optimized Inference library) Inference Result AI at the Edge General Pipeline
  • 32. Optimize (TOCO) Train ML Model (on your awesome machine or Cloud) .h5 .pb TinyML Pipeline TFLite Flat Buber Conve (xxd) Integrate into MCU Firmware C Byte Array
  • 33. Optimize (TOCO) Train ML Model (on your awesome machine or Cloud) .h5 .pb TinyML Pipeline TFLite Flat Buber Conve (xxd) Integrate into MCU Firmware C Byte Array Custom Logics ML Model Microcontroller (MCU) Firmware TensorFlow Lite for Microcontroller
  • 34. TinyML Pipeline e.g. leveraging Azure Machine Learning Azure Machine Learning Pipeline TensorFlow model Yle Compress (toco)TFLite Flat Buber Conve (xxd) C Byte ArrayIntegrate into MCU Firmware TinyML Pipeline
  • 35. Typical TinyML Model Training Labelled Raw Data Label1 Label2 Label3 Signal Processing (Low/High Pass Filter, FFT) Neural Network Output Features Credit: EdgeImpulse.com. Check it out! Processing Blocks To extract features Learning Blocks To classify new data. NN size can be signiqcantly small due to the extracted features from previous blocks AKer Filter Frequency Domain
  • 36. using Arduino framework, TFLite, PlahormIO, ESPectro32 board Gesture Recognition DEMO
  • 37. NB-IoT + GPS/GNSS Kit LoRa/LoRaWAN + GNSS Kit Sensors & Actuators Kit (20+ Peripherals) 3G + GNSS + Sensors ESPectro32 v2 Optimized for learning STEM, coding & electronics, also for fast prototyping. An AIoT Development Board: Low-power, dual-core 240 MHz Microcontroller Built-in WiFi & Bluetooth connectivity 7x7 LED matrix for user intelace Sensors: Accelerometer & Gyro using LSM6DS3, Digital temperature sensor, phototransistor microSD card slot Extensible via: 40 RPi-compatible pins, Grove connectors, Micro:bit connector Extensible with Backpacks: NB-IoT/Cellular + GPS/GNSS, LoRa, Sensor Kits, Motor Driver, and more ESPectro32 Backpacks hOps://shop.makestro.com/product/espectro32-v2/
  • 38. Demo
  • 40. Train Model If necessary. Then compress and conveb the model…. Or use pre-trained model, instead Reference: h_ps://github.com/tensollow/tensollow/tree/master/tensollow/lite/micro/examples/magic_wand/train
  • 41. Enjoy! Open Serial Monitor to see the inference result Recognised gestured also displayed on LED matrix* * On latest source code, display is already rotated :)
  • 42. Other Demo Similar demo, but using Arduino Nano 33 BLE Sense board hOps://www.youtube.com/watch?v=Lfv3WJnYhX0 hOps://github.com/andriyadi/MagicWand-TFLite-Arduino
  • 43. TinyML: TensorFlow Lite for Microcontrollers: h_ps://www.tensollow.org/lite/ microcontrollers Great book: TinyML, by Daniel Situnayake, Pete Warden Azure: Azure Machine Learning: h_ps://azure.microsoJ.com/en-in/services/ machine-learning-service/ DycodeX: ESPectro32 dev board: h_ps://shop.makestro.com/product/espectro32- v2/ SMARTernak: h_ps://smabernak.com Other IoT + AI products & solutions: h_ps://dycodex.com Contact me: andri (at) dycodex.com h_p://github.com/andriyadi | hOps://slideshare.net/andri_yadi/ Call to Action
  • 44. Want to put “AI” in “BrAIns”? … a.k.a. adopting AI + IoT -> AIoT?
  • 45. AI + IoT enabler Keep in touch hi (at) dycodex.com | https://dycodex.com Bandung, Indonesia