SlideShare a Scribd company logo

TinyML on Arduino - workshop

Jan Jongboom
Jan Jongboom

This document provides an introduction to TinyML and machine learning for embedded devices. It discusses extracting meaningful features from sensor data, developing machine learning models, and deploying models on microcontrollers with limited resources. The goal is to enable on-device intelligence using TinyML to analyze sensor data without sending raw data to the cloud.

1 of 42
Downloaded 28 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
Jan Jongboom 22 September 2020 Get started with TinyML
2 Hello! Jan Jongboom jan@edgeimpulse.com Hello!
3 Requirements Edge Impulse CLI Arduino CLI Make sure your board is visible via: arduino-cli board list docs.edgeimpulse.com -> Arduino Nano 33 BLE Sense Support: ask your local instructors, or post on forum.edgeimpulse.com (this is being monitored right now)
4 Some things are easy... https://www.flickr.com/photos/free-stock/8425195959 -10°C
5 Some things were hard, but are now easy https://www.pickpik.com/smartphone-location-lg-navigation-maps-media-photo-138415
6 Some things are still hard... https://www.pxfuel.com/en/free-photo-ovxox Do I hear glass breaking?
7 Some things are still hard... Is this machine vibrating differently? https://www.flickr.com/photos/uwnews/26143149238
8 But, no longer impossible... https://xkcd.com/1425/
Machine learning is great at finding patterns in messy data (anything you can't reason about in Excel)
10 Machine learning?
11 Machine learning?
https://www.flickr.com/photos/oceanyamaha/7091324605 12 What is it good for? Recognizing sounds Detecting abnormal vibration https://pixabay.com/photos/washing-machine-wash-cat-4120449/ Biosignal analysis https://www.flickr.com/photos/sheishine/16696564563 Anything with messy, high-resolu3on sensor data
13 Enabling new use cases Sensor fusion http://www.gierad.com/projects/supersensor/
14 Machine learning?
15 Downsides I'm not rich enough to develop 5,000 custom processors Centralized Lots of power required to send data back and forth Ethical aspect Not me
16 Typical sensor in 2020 Vibration sensor (up to 1,000 times per second) Temperature sensor Water & explosion proof Can send data >10km using 25 mW power Processor capable of running >20 million instructions per second
17 But what do they actually do? Once an hour: • Average motion (RMS) • Peak motion • Current temperature
18 99% of sensor data is discarded due to cost, bandwidth or power constraints. https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/McKinsey%20Digital/Our%20Insights/ The%20Internet%20of%20Things%20The%20value%20of%20digitizing%20the%20physical%20world/The- Internet-of-things-Mapping-the-value-beyond-the-hype.ashx
19 On-device intelligence is the only solution Vibra&on pa+ern heard that lead to fault state in a weekTemperature varies in a way that I've never seen before Machine oscillates different than all other machines in the factory
20 On-device intelligence is the only solution Temperature varies in a way that I've never seen before (0x1)
21 TinyML Inspired by "OK Google" Focus on inferencing, not training Machine learning model is just a mathematical function with lots of parameters Accuracy vs. speed, reducing parameters, hardware- optimized paths Targeting battery-powered microcontrollers Pete Warden Neil Tan
22 Pretty wide ecosystem CMSIS-NN
23 v From 0 to model https://www.flickr.com/photos/120586634@N05/14491303478
24 1. Everything starts with raw data Get data at the highest resolu3on possible - e.g. using serial or directly over WiFi
25 2. Extracting meaningful features Very dependent on your use case Raw data can be notoriously hard to deal with (3s. accelerometer data = 900 data points, 1s. audio data = 16,000 data points) Raw data is messy -1.1200, 0.5300, 10.6300, -0.5200, 1.1600, 13.1600, -0.1600, 1.4200, 13.5900, -0.2400, 1.3200, 10.8500, -0.3700, 0.5100, 7.7800, -0.1900, 0.0500, 8.8400, -0.1900, 0.0500, 8.8400, 0.2400, 0.7300, 11.1900, 0.5200, 1.6500, 12.5200, 0.6400, 1.5000, 10.8200, 0.2900, 0.3300, 7.4000, -0.3100, -0.5600, 8.5900, -0.8800, 0.6600, 11.9200, -0.8800, 0.6600, 11.9200, -0.4500, 1.3900, 11.9100, -0.0800, 1.6000, 12.7800, -0.1100, 0.3600, 9.1700, -0.0200, 0.0700, 9.8200, 0.0600, 0.9600, 12.0000, 0.2800, 1.7700, 13.2000, 0.2800, 1.7700, 13.2000, 0.2300, 1.5100, 12.6000, 0.2700, 1.0800, 11.4300, 0.0900, 0.9300, 10.9400, 0.1700, 1.2100, 11.0400, 0.4100, 1.8500, 11.5000, 0.3900, 1.9600, 11.4300, 0.3900, 1.9600, 11.4300, 0.2400, 1.4400, 10.7200, 0.0300, 1.1900, 10.3600, -0.0300, 1.3000, 10.6100, 0.4800, 1.7900, 11.7200, 1.0400, 2.6300, 13.3300, 1.0400, 2.6300, 13.3300, 1.0600, 2.3700, 13.5800, 0.3600, 1.9600, 13.3800, -0.1000, 2.1200, 13.9600, -0.3600, 2.0200, 14.5300, 0.0000, 2.1500, 14.6900, 0.0600, 2.1400, 14.3700, 0.0600, 2.1400, 14.3700, -0.3000, 1.8800, 13.7900, 0.0500, 1.7000, 13.5900, 0.1300, 1.6700, 13.1500, -0.0100, 1.7700, 12.9000, 0.4000, 1.8900, 12.2300, 0.5300, 2.3300, 12.2600, 0.5300, 2.3300, 12.2600, 0.0400, 1.9500, 11.8100, -0.2300, 1.9600, 11.2400, -0.0600, 2.1100, 10.2200, -0.1100, 2.4100, 9.7800, -0.3500, 2.7100, 9.7500, -0.7800, 3.1000, 10.1100, -0.7800, 3.1000, 10.1100, -1.0700, 3.1100, 9.8000, -1.2100, 2.9400, 9.1900, -1.1500, 3.2100, 8.6400, -0.7300, 3.6500, 8.4600, -0.5000, 3.9500, 8.6300, -0.4300, 3.9100, 8.7400, -0.4300, 3.9100, 8.7400, -0.6400, 3.7800, 8.8700, -1.2000, 3.9200, 8.9300, -1.0800, 4.4400, 8.8100, -0.7800, 4.1900, 8.1200, -0.4400, 4.1000, 7.6400, -0.5400, 4.2000, 7.5600, -0.5400, 4.2000, 7.5600, -1.0700, 4.2600, 7.2700, -1.3000, 4.5100, 7.2300, -1.2600, 4.4600, 6.6900, -1.2800, 4.4100, 6.6000, -1.7000, 4.6800, 7.0800, -2.3400, 5.1100, 7.5900, -2.3400, 5.1100, 7.5900, -2.8300, 4.8700, 6.8700, -2.9700, 4.7600, 6.2700, -3.2500, 4.6000, 6.1500, -3.4900, 4.5900, 6.2600, -3.3000, 4.9200, 6.3400, -2.7000, 4.9300, 5.8600, -2.7000, 4.9300, 5.8600, -2.9000, 4.5100, 4.9900, -3.5200, 4.3200, 4.9900, -4.1400, 4.2100, 5.7800, -3.7600, 4.1600, 5.7700, -3.0200, 4.2200, 5.4900, -3.0000, 3.8900, 3.9100, -3.0000, 3.8900, 3.9100, -3.3500, 3.5800, 3.4400, -3.1100, 3.2000, 3.6000, -3.0900, 3.6000, 5.9400, -3.0800, 3.0900, 5.1800, -2.8000, 2.9400, 4.5600, -2.4000, 2.4900, 3.9200, -2.4000, 2.4900, 3.9200, -1.8500, 2.6300, 4.4900, -1.4300, 3.9900, 7.3400, -1.4900, 3.4900, 6.1600, -1.5300, 3.2100, 5.4500, -0.9900, 2.8600, 6.2400, -0.7900, 3.2200, 8.4700, -0.7900, 3.2200, 8.4700, -0.9300, 3.5700, 9.0300, -1.6600, 2.9600, 7.0700, -1.7600, 2.1000, 6.9000, -1.4600, 2.1100, 9.0100, -1.3000, 2.5400, 10.3000, -1.3000, 2.5400, 10.3000, -1.4500, 2.6500, 9.7800, -1.5300, 1.8900, 8.4100, -1.1400, 1.3000, 9.4400, -0.7500, 1.6100, 10.3600, -0.9400, 1.5300, 9.7800, -1.0700, 1.0100, 9.2700, -1.0700, 1.0100, 9.2700, -0.8600, 1.0200, 10.1100, 0.3900, 1.6800, 11.3200, 1.2900, 1.8500, 11.4700, 1.0700, 1.3200, 11.2500, -0.2100, 1.5800, 12.4300, -1.8100, 1.3500, 12.3300, -1.8100, 1.3500, 12.3300, -2.1800, 1.0400, 11.1600, -1.5400, 0.3000, 10.3100, -0.4700, 0.2700, 11.0900, 0.7900, 1.4100, 12.9800, 1.1600, 1.7100, 12.4700, 0.5800, 0.8700, 9.6300, 0.5800, 0.8700, 9.6300, 0.1300, 0.3100, 9.5600, 0.2800, 0.3600, 10.5600, 0.7400, 0.8500, 11.4200, 0.9300, 1.0200, 11.4300, 0.6700, 0.5600, 10.5300, 0.8500, 0.3500, 9.4100, 0.8500, 0.3500, 9.4100, 1.6600, 1.2800, 10.9200, 2.0500, 0.9900, 9.7000, 2.1300, 0.8800, 10.1900, 2.0500, 0.9100, 11.3300, 1.7700, 1.4100, 12.2700, 1.4800, 1.7600, 12.1000, 1.4800, 1.7600, 12.1000, 0.9400, 1.1300, 10.8500, 0.2000, 0.8000, 10.1200, 0.2600, 1.1600, 10.5800, 0.5100, 1.6500, 10.7800, 0.4600, 1.2000, 9.9900, 0.9100, 0.8400, 9.6400, 0.9100, 0.8400, 9.6400, 1.4500, 0.7400, 10.2500, 2.0200, 1.3000, 11.4500, 1.8100, 1.8700, 12.1300, 1.0500, 1.5300, 12.0200, 0.6200, 0.6700, 11.3100, 0.7100, 0.8500, 12.0000, 0.7100, 0.8500, 12.0000, 0.6400, 1.2200, 13.1400, 1.1300, 2.0400, 14.6200, 0.8300, 2.0200, 15.5100, -0.1400, 1.4800, 15.6500, -0.6300, 1.5900, 16.0500, -1.3100, 1.7100, 16.3900, -1.3100, 1.7100, 16.3900, -1.7300, 1.5800, 16.6300, -1.1500, 1.4400, 16.0300, -0.5300, 1.1700, 15.1000, -0.1800, 0.9900, 14.4600, -0.3300, 1.0100, 13.5100, -0.3300, 1.0100, 13.5100, -0.4400, 0.9100, 12.6700, 0.0400, 1.2300, 12.5400, 0.6900, 2.0500, 13.1600, 0.3100, 1.7700, 12.8600, 0.0300, 1.3800, 11.1100, -0.4400, 1.2200, 9.4900, -0.4400, 1.2200, 9.4900, 0.1100, 1.1400, 7.3100, 0.8500, 2.2500, 8.4600, 0.8600, 3.3700, 11.2200, -0.1100, 2.2800, 8.4400, -1.3800, 1.5300, 7.1700, -1.0600, 1.5400, 6.9500, -1.0600, 1.5400, 6.9500, -0.5200, 2.8300, 8.7100, -0.2100, 2.3500, 8.1800, -0.3400, 2.7000, 8.9200, -0.3000, 2.3100, 8.7500, -0.4800, 1.4700, 7.8700, -0.3600, 0.9400, 6.9700, -0.3600, 0.9400, 6.9700, -0.2300, 1.4700, 7.6100, -0.3300, 2.2300, 8.5000, 0.3000, 1.9200, 7.8600, -0.2300, 1.5700, 6.8700, -1.4900, 1.5600, 6.3700, -2.8200, 1.6200, 7.2000, -2.8200, 1.6200, 7.2000, -3.1600, 1.8800, 7.1500, -2.7600, 2.2900, 6.8500, -2.6000, 2.2200, 6.2600, -2.9000, 1.9900, 5.8900, -3.3800, 2.2200, 6.2600, -3.9000, 2.1700, 6.0300, -3.9000, 2.1700, 6.0300, -3.8600, 2.3800, 5.6600, -3.5300, 2.5200, 5.6700, -3.2400, 2.3700, 5.8200, -3.2800, 2.1800, 5.5200, -3.1500, 2.1800, 5.6500, -3.0900, 2.0700, 5.1600, -3.0900, 2.0700, 5.1600, -2.4300, 2.1000, 5.3800, -2.0200, 2.3600, 6.0800, -2.0000, 2.5200, 6.4500, -2.2400, 2.4500, 6.0000, -2.0500, 1.8400, 4.6500, -1.3800, 1.3000, 4.6400, -1.3800, 1.3000, 4.6400, -1.2800, 1.8600, 6.9400, -1.3000, 2.5600, 9.0300, -1.5400, 2.7600, 8.5000, -1.7700, 1.6400, 6.1400, -1.6800, 1.4200, 7.5900, -1.3200, 2.0800, 9.8300, -1.3200, 2.0800, 9.8300, -0.8200, 2.1600, 10.3900, -0.7800, 1.7300, 9.8300, -1.1300, 1.3400, 9.7100, -1.3600, 1.6800, 10.2400, -1.5200, 1.6000, 9.3200, -1.8700, 1.4900, 9.1900, -1.8700, 1.4900, 9.1900, -1.9300, 1.0600, 9.9500, -1.3100, 0.8100, 10.6900, 0.0200, 2.0400, 11.0600, 0.2700, 2.5800, 9.3900, -0.0500, 2.2800, 7.3200, -0.3000, 0.4400, 7.6300, -0.3000, 0.4400, 7.6300, -1.4600, 1.0800, 12.3700, -1.9600, 1.7500, 15.3800, -0.7100, 2.1500, 14.0700, 0.7400, 1.7800, 10.4700, 0.6800, 0.8900, 9.9500, 0.0400, 1.5200, 12.0800, 0.0400, 1.5200, 12.0800, -0.4900, 1.7900, 12.7500
26 Example of a signal processing pipeline 32,000 => 240
27 Before and after feature extraction
Classification What's happening right now? Anomaly detection Is this behavior out of the ordinary? Forecasting What will happen in the future? 28 3. Letting the computers figure it out
29 Picking the right algorithm Classification Neural network Anomaly detection K-means clustering Forecasting Regression
30 4. Deploying Signal processing, neural network and anomaly detec&on
Let's build something 🚀
32 Arduino Nano 33 BLE Sense Cortex-M4F 64MHz 256K RAM Accelerometer + microphone
33 Edge Impulse - TinyML as a service Embedded or edge compute deployment options Test Edge Device Impulse Dataset Acquire valuable training data securely Test impulse with real-time device data flows Enrich data and generate ML process Real sensors in real time Open source SDK Free for developers: edgeimpulse.com
34 Let's build a gesture detection model 1. Collect some data, e.g. 2 minutes per class ('idle', 'wave', 'updown'), in continuous motion (use three dots -> 'Crop sample' if you've collected noise). 2. Create a DSP + ML pipeline to learn from this data. 3. Working? Deploy back to the Nano 33 BLE Sense.
35 Before you flash to the Nano Stop the serial daemon
36 Workbook docs.edgeimpulse.com -> Continuous motion recognition
Let's build something else 🚀
38 Connect device to new project 1. Create a new project 2. Run: edge-impulse-daemon --clean
39 Scene classification 1. Am in a forest or a conference room? Is the sink on or off? Once again continuous data helps. 2. Define two scenes ('noise' and a scene): collect 5 minutes of data per class. You can do this both on the Nano (13 seconds per time) or on your phone (Devices > Add new device). E.g. music playing vs. no music playing. 3. Again DSP + ML pipeline. 4. Deploy back to device. 5. Feeling adventurous? Export to Arduino library.
40 Scene classification caveat
41 You can do much more with Edge Impulse Full freedom in signal processing blocks (see 'Building custom processing blocks') Also tutorials on computer vision, discrete events (like keywords) Full freedom in neural network blocks We have a nice community, come and build with us! forum.edgeimpulse.com
42 Thank you! Slides: janjongboom.com
Ad

Recommended

Arduino TinyML & IoT
Arduino TinyML & IoT
Robocraze
 
Introduction to AIoT & TinyML - with Arduino
Introduction to AIoT & TinyML - with Arduino
Andri Yadi
 
Screenless display
Screenless display
chnaveed
 
Wireless robot ppt
Wireless robot ppt
Varun B P
 
Human Activity Recognition using Smartphone's sensor
Human Activity Recognition using Smartphone's sensor
Pankaj Mishra
 
Silent sound-technology ppt final
Silent sound-technology ppt final
Lohit Dalal
 
Tweet sentiment analysis
Tweet sentiment analysis
Anil Shrestha
 
Introduction to TinyML - Solomon Muhunyo Githu
Introduction to TinyML - Solomon Muhunyo Githu
Solomon Githu
 
Screenless Display PPT
Screenless Display PPT
Vikas Kumar
 
Currency recognition system using image processing
Currency recognition system using image processing
Fatima Akhtar
 
imageprocessing-abstract
imageprocessing-abstract
Jagadeesh Kumar
 
Computer vision introduction
Computer vision introduction
Wael Badawy
 
Introductory Virtual Reality in Unity3d
Introductory Virtual Reality in Unity3d
Bond University
 
virtual reality ppt
virtual reality ppt
nirav radadiya
 
COMPUTER GRAPHICS PROJECT REPORT
COMPUTER GRAPHICS PROJECT REPORT
vineet raj
 
Object detection.pptx
Object detection.pptx
shradhaketkale2
 
Blue eyes technology
Blue eyes technology
Priyanka Datta
 
Arduino in TinyML with Edge Impulse
Arduino in TinyML with Edge Impulse
Robocraze
 
Object detection presentation
Object detection presentation
AshwinBicholiya
 
Internship report on AI , ML & IIOT and project responses full docs
Internship report on AI , ML & IIOT and project responses full docs
Rakesh Arigela
 
Driver DrowsiNess System
Driver DrowsiNess System
Gurunadh Guru
 
Augmented reality (Access virtual world)
Augmented reality (Access virtual world)
chirag thakkar
 
Android controlled robot
Android controlled robot
Satyendra Gupta
 
Computer Vision - Real Time Face Recognition using Open CV and Python
Computer Vision - Real Time Face Recognition using Open CV and Python
Akash Satamkar
 
SCREENLESS DISPLAY
SCREENLESS DISPLAY
Mahad Mumtaz
 
Machine learning in image processing
Machine learning in image processing
Data Science Thailand
 
Automatic number plate recognition (anpr)
Automatic number plate recognition (anpr)
AbhishekChoudhary464889
 
silent sound technology final report(17321A0432) (1).pdf
silent sound technology final report(17321A0432) (1).pdf
ssuser476810
 
Get started with TinyML - Hackster webinar 9 April 2020
Get started with TinyML - Hackster webinar 9 April 2020
Jan Jongboom
 
Teaching your sensors new tricks with Machine Learning - Eta Compute webinar
Teaching your sensors new tricks with Machine Learning - Eta Compute webinar
Jan Jongboom
 

More Related Content

What's hot (20)

Screenless Display PPT
Screenless Display PPT
Vikas Kumar
 
Currency recognition system using image processing
Currency recognition system using image processing
Fatima Akhtar
 
imageprocessing-abstract
imageprocessing-abstract
Jagadeesh Kumar
 
Computer vision introduction
Computer vision introduction
Wael Badawy
 
Introductory Virtual Reality in Unity3d
Introductory Virtual Reality in Unity3d
Bond University
 
virtual reality ppt
virtual reality ppt
nirav radadiya
 
COMPUTER GRAPHICS PROJECT REPORT
COMPUTER GRAPHICS PROJECT REPORT
vineet raj
 
Object detection.pptx
Object detection.pptx
shradhaketkale2
 
Blue eyes technology
Blue eyes technology
Priyanka Datta
 
Arduino in TinyML with Edge Impulse
Arduino in TinyML with Edge Impulse
Robocraze
 
Object detection presentation
Object detection presentation
AshwinBicholiya
 
Internship report on AI , ML & IIOT and project responses full docs
Internship report on AI , ML & IIOT and project responses full docs
Rakesh Arigela
 
Driver DrowsiNess System
Driver DrowsiNess System
Gurunadh Guru
 
Augmented reality (Access virtual world)
Augmented reality (Access virtual world)
chirag thakkar
 
Android controlled robot
Android controlled robot
Satyendra Gupta
 
Computer Vision - Real Time Face Recognition using Open CV and Python
Computer Vision - Real Time Face Recognition using Open CV and Python
Akash Satamkar
 
SCREENLESS DISPLAY
SCREENLESS DISPLAY
Mahad Mumtaz
 
Machine learning in image processing
Machine learning in image processing
Data Science Thailand
 
Automatic number plate recognition (anpr)
Automatic number plate recognition (anpr)
AbhishekChoudhary464889
 
silent sound technology final report(17321A0432) (1).pdf
silent sound technology final report(17321A0432) (1).pdf
ssuser476810
 
Screenless Display PPT
Screenless Display PPT
Vikas Kumar
 
Currency recognition system using image processing
Currency recognition system using image processing
Fatima Akhtar
 
imageprocessing-abstract
imageprocessing-abstract
Jagadeesh Kumar
 
Computer vision introduction
Computer vision introduction
Wael Badawy
 
Introductory Virtual Reality in Unity3d
Introductory Virtual Reality in Unity3d
Bond University
 
virtual reality ppt
virtual reality ppt
nirav radadiya
 
COMPUTER GRAPHICS PROJECT REPORT
COMPUTER GRAPHICS PROJECT REPORT
vineet raj
 
Object detection.pptx
Object detection.pptx
shradhaketkale2
 
Blue eyes technology
Blue eyes technology
Priyanka Datta
 
Arduino in TinyML with Edge Impulse
Arduino in TinyML with Edge Impulse
Robocraze
 
Object detection presentation
Object detection presentation
AshwinBicholiya
 
Internship report on AI , ML & IIOT and project responses full docs
Internship report on AI , ML & IIOT and project responses full docs
Rakesh Arigela
 
Driver DrowsiNess System
Driver DrowsiNess System
Gurunadh Guru
 
Augmented reality (Access virtual world)
Augmented reality (Access virtual world)
chirag thakkar
 
Android controlled robot
Android controlled robot
Satyendra Gupta
 
Computer Vision - Real Time Face Recognition using Open CV and Python
Computer Vision - Real Time Face Recognition using Open CV and Python
Akash Satamkar
 
SCREENLESS DISPLAY
SCREENLESS DISPLAY
Mahad Mumtaz
 
Machine learning in image processing
Machine learning in image processing
Data Science Thailand
 
Automatic number plate recognition (anpr)
Automatic number plate recognition (anpr)
AbhishekChoudhary464889
 
silent sound technology final report(17321A0432) (1).pdf
silent sound technology final report(17321A0432) (1).pdf
ssuser476810
 

Similar to TinyML on Arduino - workshop (20)

Get started with TinyML - Hackster webinar 9 April 2020
Get started with TinyML - Hackster webinar 9 April 2020
Jan Jongboom
 
Teaching your sensors new tricks with Machine Learning - Eta Compute webinar
Teaching your sensors new tricks with Machine Learning - Eta Compute webinar
Jan Jongboom
 
Get started with TinyML - Embedded online conference
Get started with TinyML - Embedded online conference
Jan Jongboom
 
Adding intelligence to your LoRaWAN devices - The Things Conference on tour
Adding intelligence to your LoRaWAN devices - The Things Conference on tour
Jan Jongboom
 
Adding intelligence to your LoRaWAN deployment - The Things Virtual Conference
Adding intelligence to your LoRaWAN deployment - The Things Virtual Conference
Jan Jongboom
 
Teaching your sensors new tricks with Machine Learning - CENSIS Tech Summit 2019
Teaching your sensors new tricks with Machine Learning - CENSIS Tech Summit 2019
Jan Jongboom
 
Intelligent Edge - Getting started with TinyML for industrial applications
Intelligent Edge - Getting started with TinyML for industrial applications
Jan Jongboom
 
Wrangle 2016: (Lightning Talk) FizzBuzz in TensorFlow
Wrangle 2016: (Lightning Talk) FizzBuzz in TensorFlow
WrangleConf
 
Neural Network Back Propagation Algorithm
Neural Network Back Propagation Algorithm
Martin Opdam
 
Introduction to Machine Learning
Introduction to Machine Learning
Big_Data_Ukraine
 
Machine Learning Essentials Demystified part2 | Big Data Demystified
Machine Learning Essentials Demystified part2 | Big Data Demystified
Omid Vahdaty
 
Ai_Project_report
Ai_Project_report
Ravi Gupta
 
Convolution as matrix multiplication
Convolution as matrix multiplication
Edwin Efraín Jiménez Lepe
 
Estado del Arte de la IA
Estado del Arte de la IA
Plain Concepts
 
Neural network and mlp
Neural network and mlp
partha pratim deb
 
Intro matlab-nn
Intro matlab-nn
soldier78
 
numpyxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
numpyxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
sin3divcx
 
Intro to Machine Learning with TF- workshop
Intro to Machine Learning with TF- workshop
Prottay Karim
 
Machine learning and its parameter is discussed here
Machine learning and its parameter is discussed here
RevathiSundar4
 
Deep Learning con CNTK by Pablo Doval
Deep Learning con CNTK by Pablo Doval
Plain Concepts
 
Get started with TinyML - Hackster webinar 9 April 2020
Get started with TinyML - Hackster webinar 9 April 2020
Jan Jongboom
 
Teaching your sensors new tricks with Machine Learning - Eta Compute webinar
Teaching your sensors new tricks with Machine Learning - Eta Compute webinar
Jan Jongboom
 
Get started with TinyML - Embedded online conference
Get started with TinyML - Embedded online conference
Jan Jongboom
 
Adding intelligence to your LoRaWAN devices - The Things Conference on tour
Adding intelligence to your LoRaWAN devices - The Things Conference on tour
Jan Jongboom
 
Adding intelligence to your LoRaWAN deployment - The Things Virtual Conference
Adding intelligence to your LoRaWAN deployment - The Things Virtual Conference
Jan Jongboom
 
Teaching your sensors new tricks with Machine Learning - CENSIS Tech Summit 2019
Teaching your sensors new tricks with Machine Learning - CENSIS Tech Summit 2019
Jan Jongboom
 
Intelligent Edge - Getting started with TinyML for industrial applications
Intelligent Edge - Getting started with TinyML for industrial applications
Jan Jongboom
 
Wrangle 2016: (Lightning Talk) FizzBuzz in TensorFlow
Wrangle 2016: (Lightning Talk) FizzBuzz in TensorFlow
WrangleConf
 
Neural Network Back Propagation Algorithm
Neural Network Back Propagation Algorithm
Martin Opdam
 
Introduction to Machine Learning
Introduction to Machine Learning
Big_Data_Ukraine
 
Machine Learning Essentials Demystified part2 | Big Data Demystified
Machine Learning Essentials Demystified part2 | Big Data Demystified
Omid Vahdaty
 
Ai_Project_report
Ai_Project_report
Ravi Gupta
 
Convolution as matrix multiplication
Convolution as matrix multiplication
Edwin Efraín Jiménez Lepe
 
Estado del Arte de la IA
Estado del Arte de la IA
Plain Concepts
 
Neural network and mlp
Neural network and mlp
partha pratim deb
 
Intro matlab-nn
Intro matlab-nn
soldier78
 
numpyxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
numpyxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
sin3divcx
 
Intro to Machine Learning with TF- workshop
Intro to Machine Learning with TF- workshop
Prottay Karim
 
Machine learning and its parameter is discussed here
Machine learning and its parameter is discussed here
RevathiSundar4
 
Deep Learning con CNTK by Pablo Doval
Deep Learning con CNTK by Pablo Doval
Plain Concepts
 
Ad

More from Jan Jongboom (20)

Tiny intelligent computers and sensors - Open Hardware Event 2020
Tiny intelligent computers and sensors - Open Hardware Event 2020
Jan Jongboom
 
Machine learning on 1 square centimeter - Emerce Next 2019
Machine learning on 1 square centimeter - Emerce Next 2019
Jan Jongboom
 
Fundamentals of IoT - Data Science Africa 2019
Fundamentals of IoT - Data Science Africa 2019
Jan Jongboom
 
17,000 contributions in 32K RAM - FOSS North 2019
17,000 contributions in 32K RAM - FOSS North 2019
Jan Jongboom
 
Open Hours: Mbed Simulator
Open Hours: Mbed Simulator
Jan Jongboom
 
Efficient IoT solutions based on LoRaWAN, The Things Network and Mbed OS
Efficient IoT solutions based on LoRaWAN, The Things Network and Mbed OS
Jan Jongboom
 
Machine learning on 1 cm2 - Tweakers Dev Summit
Machine learning on 1 cm2 - Tweakers Dev Summit
Jan Jongboom
 
Simulating LoRaWAN devices - LoRa Alliance AMM 2019
Simulating LoRaWAN devices - LoRa Alliance AMM 2019
Jan Jongboom
 
Develop with Mbed OS - The Things Conference 2019
Develop with Mbed OS - The Things Conference 2019
Jan Jongboom
 
Firmware Updates over LoRaWAN - The Things Conference 2019
Firmware Updates over LoRaWAN - The Things Conference 2019
Jan Jongboom
 
Faster Device Development - GSMA @ CES 2019
Faster Device Development - GSMA @ CES 2019
Jan Jongboom
 
Mbed LoRaWAN stack: a case study - LoRa Alliance AMM Tokyo
Mbed LoRaWAN stack: a case study - LoRa Alliance AMM Tokyo
Jan Jongboom
 
Simulating microcontrollers - Arm Research Summit 2018
Simulating microcontrollers - Arm Research Summit 2018
Jan Jongboom
 
Introduction to Mbed - Etteplan seminar - August 2018
Introduction to Mbed - Etteplan seminar - August 2018
Jan Jongboom
 
Machine Learning on 1 cm2 - Daho.am 2018
Machine Learning on 1 cm2 - Daho.am 2018
Jan Jongboom
 
Fundamentals of IoT - Data Science Africa 2018
Fundamentals of IoT - Data Science Africa 2018
Jan Jongboom
 
Intelligent IoT Devices - IoT Bootcamp - Smart City Gothenburg 2018
Intelligent IoT Devices - IoT Bootcamp - Smart City Gothenburg 2018
Jan Jongboom
 
Embedded Development: meet the web browser - TEQNation 2018
Embedded Development: meet the web browser - TEQNation 2018
Jan Jongboom
 
Machine learning on microcontrollers - Tech Power Summit 2018
Machine learning on microcontrollers - Tech Power Summit 2018
Jan Jongboom
 
Deep learning on microcontrollers - IETF 101 - T2TRG
Deep learning on microcontrollers - IETF 101 - T2TRG
Jan Jongboom
 
Tiny intelligent computers and sensors - Open Hardware Event 2020
Tiny intelligent computers and sensors - Open Hardware Event 2020
Jan Jongboom
 
Machine learning on 1 square centimeter - Emerce Next 2019
Machine learning on 1 square centimeter - Emerce Next 2019
Jan Jongboom
 
Fundamentals of IoT - Data Science Africa 2019
Fundamentals of IoT - Data Science Africa 2019
Jan Jongboom
 
17,000 contributions in 32K RAM - FOSS North 2019
17,000 contributions in 32K RAM - FOSS North 2019
Jan Jongboom
 
Open Hours: Mbed Simulator
Open Hours: Mbed Simulator
Jan Jongboom
 
Efficient IoT solutions based on LoRaWAN, The Things Network and Mbed OS
Efficient IoT solutions based on LoRaWAN, The Things Network and Mbed OS
Jan Jongboom
 
Machine learning on 1 cm2 - Tweakers Dev Summit
Machine learning on 1 cm2 - Tweakers Dev Summit
Jan Jongboom
 
Simulating LoRaWAN devices - LoRa Alliance AMM 2019
Simulating LoRaWAN devices - LoRa Alliance AMM 2019
Jan Jongboom
 
Develop with Mbed OS - The Things Conference 2019
Develop with Mbed OS - The Things Conference 2019
Jan Jongboom
 
Firmware Updates over LoRaWAN - The Things Conference 2019
Firmware Updates over LoRaWAN - The Things Conference 2019
Jan Jongboom
 
Faster Device Development - GSMA @ CES 2019
Faster Device Development - GSMA @ CES 2019
Jan Jongboom
 
Mbed LoRaWAN stack: a case study - LoRa Alliance AMM Tokyo
Mbed LoRaWAN stack: a case study - LoRa Alliance AMM Tokyo
Jan Jongboom
 
Simulating microcontrollers - Arm Research Summit 2018
Simulating microcontrollers - Arm Research Summit 2018
Jan Jongboom
 
Introduction to Mbed - Etteplan seminar - August 2018
Introduction to Mbed - Etteplan seminar - August 2018
Jan Jongboom
 
Machine Learning on 1 cm2 - Daho.am 2018
Machine Learning on 1 cm2 - Daho.am 2018
Jan Jongboom
 
Fundamentals of IoT - Data Science Africa 2018
Fundamentals of IoT - Data Science Africa 2018
Jan Jongboom
 
Intelligent IoT Devices - IoT Bootcamp - Smart City Gothenburg 2018
Intelligent IoT Devices - IoT Bootcamp - Smart City Gothenburg 2018
Jan Jongboom
 
Embedded Development: meet the web browser - TEQNation 2018
Embedded Development: meet the web browser - TEQNation 2018
Jan Jongboom
 
Machine learning on microcontrollers - Tech Power Summit 2018
Machine learning on microcontrollers - Tech Power Summit 2018
Jan Jongboom
 
Deep learning on microcontrollers - IETF 101 - T2TRG
Deep learning on microcontrollers - IETF 101 - T2TRG
Jan Jongboom
 
Ad

Recently uploaded (20)

NOC Services for maintaining network as MSA.ppt
NOC Services for maintaining network as MSA.ppt
ankurnagar22
 
AI theory work for students to understand the logic
AI theory work for students to understand the logic
areeba15775n
 
Topic 1 Foundational IT Infrastructure_.pptx
Topic 1 Foundational IT Infrastructure_.pptx
oneillp100
 
Unlocking Business Growth Through Targeted Social Engagement
Unlocking Business Growth Through Targeted Social Engagement
Digital Guider
 
Topic 2 - Cloud Computing Basics,,,.pptx
Topic 2 - Cloud Computing Basics,,,.pptx
oneillp100
 
COMPUTER ETHICS AND CRIME.......................................................
COMPUTER ETHICS AND CRIME.......................................................
FOOLKUMARI
 
05-introduction-to-operating-systems.pptx
05-introduction-to-operating-systems.pptx
pepecompany1
 
IAREUOUSTPIDWHY$)CHARACTERARERWUEEJJSKWNSND
IAREUOUSTPIDWHY$)CHARACTERARERWUEEJJSKWNSND
notgachabite123
 
SAP_S4HANA_eCommerce_Integration_Presentation.pptx
SAP_S4HANA_eCommerce_Integration_Presentation.pptx
vemulavenu484
 
The it.com Domains Brand Book for registrars, domain resellers and hosting co...
The it.com Domains Brand Book for registrars, domain resellers and hosting co...
it.com Domains
 
Internet & Protocols : A Blueprint of the Internet System
Internet & Protocols : A Blueprint of the Internet System
cpnabil59
 
3 years of Quarkus in production, what have we learned - Devoxx Polen
3 years of Quarkus in production, what have we learned - Devoxx Polen
Jago de Vreede
 
SAP_S4HANA_ChatGPT_Integration_Presentation.pptx
SAP_S4HANA_ChatGPT_Integration_Presentation.pptx
vemulavenu484
 
最新版美国威斯康星大学绿湾分校毕业证（UWGB毕业证书）原版定制
最新版美国威斯康星大学绿湾分校毕业证（UWGB毕业证书）原版定制
Taqyea
 
DDos Mitigation Strategie, presented at bdNOG 19
DDos Mitigation Strategie, presented at bdNOG 19
APNIC
 
Top Mobile App Development Trends Shaping the Future
Top Mobile App Development Trends Shaping the Future
ChicMic Studios
 
Internet Download Manager (IDM) 6.42.40 Crack Download
Internet Download Manager (IDM) 6.42.40 Crack Download
Puppy jhon
 
In order to install and use the device software, your computer must meet the ...
In order to install and use the device software, your computer must meet the ...
raguclc
 
AMOR PROHIBIDO MURMURAN POR LAS CALLES PORQUE
AMOR PROHIBIDO MURMURAN POR LAS CALLES PORQUE
GFGLaboratorios
 
LpQuantueer rtwrt 1e erere errerqer m.ppt
LpQuantueer rtwrt 1e erere errerqer m.ppt
cyberesearchprof
 
NOC Services for maintaining network as MSA.ppt
NOC Services for maintaining network as MSA.ppt
ankurnagar22
 
AI theory work for students to understand the logic
AI theory work for students to understand the logic
areeba15775n
 
Topic 1 Foundational IT Infrastructure_.pptx
Topic 1 Foundational IT Infrastructure_.pptx
oneillp100
 
Unlocking Business Growth Through Targeted Social Engagement
Unlocking Business Growth Through Targeted Social Engagement
Digital Guider
 
Topic 2 - Cloud Computing Basics,,,.pptx
Topic 2 - Cloud Computing Basics,,,.pptx
oneillp100
 
COMPUTER ETHICS AND CRIME.......................................................
COMPUTER ETHICS AND CRIME.......................................................
FOOLKUMARI
 
05-introduction-to-operating-systems.pptx
05-introduction-to-operating-systems.pptx
pepecompany1
 
IAREUOUSTPIDWHY$)CHARACTERARERWUEEJJSKWNSND
IAREUOUSTPIDWHY$)CHARACTERARERWUEEJJSKWNSND
notgachabite123
 
SAP_S4HANA_eCommerce_Integration_Presentation.pptx
SAP_S4HANA_eCommerce_Integration_Presentation.pptx
vemulavenu484
 
The it.com Domains Brand Book for registrars, domain resellers and hosting co...
The it.com Domains Brand Book for registrars, domain resellers and hosting co...
it.com Domains
 
Internet & Protocols : A Blueprint of the Internet System
Internet & Protocols : A Blueprint of the Internet System
cpnabil59
 
3 years of Quarkus in production, what have we learned - Devoxx Polen
3 years of Quarkus in production, what have we learned - Devoxx Polen
Jago de Vreede
 
SAP_S4HANA_ChatGPT_Integration_Presentation.pptx
SAP_S4HANA_ChatGPT_Integration_Presentation.pptx
vemulavenu484
 
最新版美国威斯康星大学绿湾分校毕业证（UWGB毕业证书）原版定制
最新版美国威斯康星大学绿湾分校毕业证（UWGB毕业证书）原版定制
Taqyea
 
DDos Mitigation Strategie, presented at bdNOG 19
DDos Mitigation Strategie, presented at bdNOG 19
APNIC
 
Top Mobile App Development Trends Shaping the Future
Top Mobile App Development Trends Shaping the Future
ChicMic Studios
 
Internet Download Manager (IDM) 6.42.40 Crack Download
Internet Download Manager (IDM) 6.42.40 Crack Download
Puppy jhon
 
In order to install and use the device software, your computer must meet the ...
In order to install and use the device software, your computer must meet the ...
raguclc
 
AMOR PROHIBIDO MURMURAN POR LAS CALLES PORQUE
AMOR PROHIBIDO MURMURAN POR LAS CALLES PORQUE
GFGLaboratorios
 
LpQuantueer rtwrt 1e erere errerqer m.ppt
LpQuantueer rtwrt 1e erere errerqer m.ppt
cyberesearchprof
 

TinyML on Arduino - workshop

  • 1. Jan Jongboom 22 September 2020 Get started with TinyML
  • 3. 3 Requirements Edge Impulse CLI Arduino CLI Make sure your board is visible via: arduino-cli board list docs.edgeimpulse.com -> Arduino Nano 33 BLE Sense Support: ask your local instructors, or post on forum.edgeimpulse.com (this is being monitored right now)
  • 4. 4 Some things are easy... https://www.flickr.com/photos/free-stock/8425195959 -10°C
  • 5. 5 Some things were hard, but are now easy https://www.pickpik.com/smartphone-location-lg-navigation-maps-media-photo-138415
  • 6. 6 Some things are still hard... https://www.pxfuel.com/en/free-photo-ovxox Do I hear glass breaking?
  • 7. 7 Some things are still hard... Is this machine vibrating differently? https://www.flickr.com/photos/uwnews/26143149238
  • 8. 8 But, no longer impossible... https://xkcd.com/1425/
  • 9. Machine learning is great at finding patterns in messy data (anything you can't reason about in Excel)
  • 12. https://www.flickr.com/photos/oceanyamaha/7091324605 12 What is it good for? Recognizing sounds Detecting abnormal vibration https://pixabay.com/photos/washing-machine-wash-cat-4120449/ Biosignal analysis https://www.flickr.com/photos/sheishine/16696564563 Anything with messy, high-resolu3on sensor data
  • 13. 13 Enabling new use cases Sensor fusion http://www.gierad.com/projects/supersensor/
  • 15. 15 Downsides I'm not rich enough to develop 5,000 custom processors Centralized Lots of power required to send data back and forth Ethical aspect Not me
  • 16. 16 Typical sensor in 2020 Vibration sensor (up to 1,000 times per second) Temperature sensor Water & explosion proof Can send data >10km using 25 mW power Processor capable of running >20 million instructions per second
  • 17. 17 But what do they actually do? Once an hour: • Average motion (RMS) • Peak motion • Current temperature
  • 18. 18 99% of sensor data is discarded due to cost, bandwidth or power constraints. https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/McKinsey%20Digital/Our%20Insights/ The%20Internet%20of%20Things%20The%20value%20of%20digitizing%20the%20physical%20world/The- Internet-of-things-Mapping-the-value-beyond-the-hype.ashx
  • 19. 19 On-device intelligence is the only solution Vibra&on pa+ern heard that lead to fault state in a weekTemperature varies in a way that I've never seen before Machine oscillates different than all other machines in the factory
  • 20. 20 On-device intelligence is the only solution Temperature varies in a way that I've never seen before (0x1)
  • 21. 21 TinyML Inspired by "OK Google" Focus on inferencing, not training Machine learning model is just a mathematical function with lots of parameters Accuracy vs. speed, reducing parameters, hardware- optimized paths Targeting battery-powered microcontrollers Pete Warden Neil Tan
  • 24. 24 1. Everything starts with raw data Get data at the highest resolu3on possible - e.g. using serial or directly over WiFi
  • 25. 25 2. Extracting meaningful features Very dependent on your use case Raw data can be notoriously hard to deal with (3s. accelerometer data = 900 data points, 1s. audio data = 16,000 data points) Raw data is messy -1.1200, 0.5300, 10.6300, -0.5200, 1.1600, 13.1600, -0.1600, 1.4200, 13.5900, -0.2400, 1.3200, 10.8500, -0.3700, 0.5100, 7.7800, -0.1900, 0.0500, 8.8400, -0.1900, 0.0500, 8.8400, 0.2400, 0.7300, 11.1900, 0.5200, 1.6500, 12.5200, 0.6400, 1.5000, 10.8200, 0.2900, 0.3300, 7.4000, -0.3100, -0.5600, 8.5900, -0.8800, 0.6600, 11.9200, -0.8800, 0.6600, 11.9200, -0.4500, 1.3900, 11.9100, -0.0800, 1.6000, 12.7800, -0.1100, 0.3600, 9.1700, -0.0200, 0.0700, 9.8200, 0.0600, 0.9600, 12.0000, 0.2800, 1.7700, 13.2000, 0.2800, 1.7700, 13.2000, 0.2300, 1.5100, 12.6000, 0.2700, 1.0800, 11.4300, 0.0900, 0.9300, 10.9400, 0.1700, 1.2100, 11.0400, 0.4100, 1.8500, 11.5000, 0.3900, 1.9600, 11.4300, 0.3900, 1.9600, 11.4300, 0.2400, 1.4400, 10.7200, 0.0300, 1.1900, 10.3600, -0.0300, 1.3000, 10.6100, 0.4800, 1.7900, 11.7200, 1.0400, 2.6300, 13.3300, 1.0400, 2.6300, 13.3300, 1.0600, 2.3700, 13.5800, 0.3600, 1.9600, 13.3800, -0.1000, 2.1200, 13.9600, -0.3600, 2.0200, 14.5300, 0.0000, 2.1500, 14.6900, 0.0600, 2.1400, 14.3700, 0.0600, 2.1400, 14.3700, -0.3000, 1.8800, 13.7900, 0.0500, 1.7000, 13.5900, 0.1300, 1.6700, 13.1500, -0.0100, 1.7700, 12.9000, 0.4000, 1.8900, 12.2300, 0.5300, 2.3300, 12.2600, 0.5300, 2.3300, 12.2600, 0.0400, 1.9500, 11.8100, -0.2300, 1.9600, 11.2400, -0.0600, 2.1100, 10.2200, -0.1100, 2.4100, 9.7800, -0.3500, 2.7100, 9.7500, -0.7800, 3.1000, 10.1100, -0.7800, 3.1000, 10.1100, -1.0700, 3.1100, 9.8000, -1.2100, 2.9400, 9.1900, -1.1500, 3.2100, 8.6400, -0.7300, 3.6500, 8.4600, -0.5000, 3.9500, 8.6300, -0.4300, 3.9100, 8.7400, -0.4300, 3.9100, 8.7400, -0.6400, 3.7800, 8.8700, -1.2000, 3.9200, 8.9300, -1.0800, 4.4400, 8.8100, -0.7800, 4.1900, 8.1200, -0.4400, 4.1000, 7.6400, -0.5400, 4.2000, 7.5600, -0.5400, 4.2000, 7.5600, -1.0700, 4.2600, 7.2700, -1.3000, 4.5100, 7.2300, -1.2600, 4.4600, 6.6900, -1.2800, 4.4100, 6.6000, -1.7000, 4.6800, 7.0800, -2.3400, 5.1100, 7.5900, -2.3400, 5.1100, 7.5900, -2.8300, 4.8700, 6.8700, -2.9700, 4.7600, 6.2700, -3.2500, 4.6000, 6.1500, -3.4900, 4.5900, 6.2600, -3.3000, 4.9200, 6.3400, -2.7000, 4.9300, 5.8600, -2.7000, 4.9300, 5.8600, -2.9000, 4.5100, 4.9900, -3.5200, 4.3200, 4.9900, -4.1400, 4.2100, 5.7800, -3.7600, 4.1600, 5.7700, -3.0200, 4.2200, 5.4900, -3.0000, 3.8900, 3.9100, -3.0000, 3.8900, 3.9100, -3.3500, 3.5800, 3.4400, -3.1100, 3.2000, 3.6000, -3.0900, 3.6000, 5.9400, -3.0800, 3.0900, 5.1800, -2.8000, 2.9400, 4.5600, -2.4000, 2.4900, 3.9200, -2.4000, 2.4900, 3.9200, -1.8500, 2.6300, 4.4900, -1.4300, 3.9900, 7.3400, -1.4900, 3.4900, 6.1600, -1.5300, 3.2100, 5.4500, -0.9900, 2.8600, 6.2400, -0.7900, 3.2200, 8.4700, -0.7900, 3.2200, 8.4700, -0.9300, 3.5700, 9.0300, -1.6600, 2.9600, 7.0700, -1.7600, 2.1000, 6.9000, -1.4600, 2.1100, 9.0100, -1.3000, 2.5400, 10.3000, -1.3000, 2.5400, 10.3000, -1.4500, 2.6500, 9.7800, -1.5300, 1.8900, 8.4100, -1.1400, 1.3000, 9.4400, -0.7500, 1.6100, 10.3600, -0.9400, 1.5300, 9.7800, -1.0700, 1.0100, 9.2700, -1.0700, 1.0100, 9.2700, -0.8600, 1.0200, 10.1100, 0.3900, 1.6800, 11.3200, 1.2900, 1.8500, 11.4700, 1.0700, 1.3200, 11.2500, -0.2100, 1.5800, 12.4300, -1.8100, 1.3500, 12.3300, -1.8100, 1.3500, 12.3300, -2.1800, 1.0400, 11.1600, -1.5400, 0.3000, 10.3100, -0.4700, 0.2700, 11.0900, 0.7900, 1.4100, 12.9800, 1.1600, 1.7100, 12.4700, 0.5800, 0.8700, 9.6300, 0.5800, 0.8700, 9.6300, 0.1300, 0.3100, 9.5600, 0.2800, 0.3600, 10.5600, 0.7400, 0.8500, 11.4200, 0.9300, 1.0200, 11.4300, 0.6700, 0.5600, 10.5300, 0.8500, 0.3500, 9.4100, 0.8500, 0.3500, 9.4100, 1.6600, 1.2800, 10.9200, 2.0500, 0.9900, 9.7000, 2.1300, 0.8800, 10.1900, 2.0500, 0.9100, 11.3300, 1.7700, 1.4100, 12.2700, 1.4800, 1.7600, 12.1000, 1.4800, 1.7600, 12.1000, 0.9400, 1.1300, 10.8500, 0.2000, 0.8000, 10.1200, 0.2600, 1.1600, 10.5800, 0.5100, 1.6500, 10.7800, 0.4600, 1.2000, 9.9900, 0.9100, 0.8400, 9.6400, 0.9100, 0.8400, 9.6400, 1.4500, 0.7400, 10.2500, 2.0200, 1.3000, 11.4500, 1.8100, 1.8700, 12.1300, 1.0500, 1.5300, 12.0200, 0.6200, 0.6700, 11.3100, 0.7100, 0.8500, 12.0000, 0.7100, 0.8500, 12.0000, 0.6400, 1.2200, 13.1400, 1.1300, 2.0400, 14.6200, 0.8300, 2.0200, 15.5100, -0.1400, 1.4800, 15.6500, -0.6300, 1.5900, 16.0500, -1.3100, 1.7100, 16.3900, -1.3100, 1.7100, 16.3900, -1.7300, 1.5800, 16.6300, -1.1500, 1.4400, 16.0300, -0.5300, 1.1700, 15.1000, -0.1800, 0.9900, 14.4600, -0.3300, 1.0100, 13.5100, -0.3300, 1.0100, 13.5100, -0.4400, 0.9100, 12.6700, 0.0400, 1.2300, 12.5400, 0.6900, 2.0500, 13.1600, 0.3100, 1.7700, 12.8600, 0.0300, 1.3800, 11.1100, -0.4400, 1.2200, 9.4900, -0.4400, 1.2200, 9.4900, 0.1100, 1.1400, 7.3100, 0.8500, 2.2500, 8.4600, 0.8600, 3.3700, 11.2200, -0.1100, 2.2800, 8.4400, -1.3800, 1.5300, 7.1700, -1.0600, 1.5400, 6.9500, -1.0600, 1.5400, 6.9500, -0.5200, 2.8300, 8.7100, -0.2100, 2.3500, 8.1800, -0.3400, 2.7000, 8.9200, -0.3000, 2.3100, 8.7500, -0.4800, 1.4700, 7.8700, -0.3600, 0.9400, 6.9700, -0.3600, 0.9400, 6.9700, -0.2300, 1.4700, 7.6100, -0.3300, 2.2300, 8.5000, 0.3000, 1.9200, 7.8600, -0.2300, 1.5700, 6.8700, -1.4900, 1.5600, 6.3700, -2.8200, 1.6200, 7.2000, -2.8200, 1.6200, 7.2000, -3.1600, 1.8800, 7.1500, -2.7600, 2.2900, 6.8500, -2.6000, 2.2200, 6.2600, -2.9000, 1.9900, 5.8900, -3.3800, 2.2200, 6.2600, -3.9000, 2.1700, 6.0300, -3.9000, 2.1700, 6.0300, -3.8600, 2.3800, 5.6600, -3.5300, 2.5200, 5.6700, -3.2400, 2.3700, 5.8200, -3.2800, 2.1800, 5.5200, -3.1500, 2.1800, 5.6500, -3.0900, 2.0700, 5.1600, -3.0900, 2.0700, 5.1600, -2.4300, 2.1000, 5.3800, -2.0200, 2.3600, 6.0800, -2.0000, 2.5200, 6.4500, -2.2400, 2.4500, 6.0000, -2.0500, 1.8400, 4.6500, -1.3800, 1.3000, 4.6400, -1.3800, 1.3000, 4.6400, -1.2800, 1.8600, 6.9400, -1.3000, 2.5600, 9.0300, -1.5400, 2.7600, 8.5000, -1.7700, 1.6400, 6.1400, -1.6800, 1.4200, 7.5900, -1.3200, 2.0800, 9.8300, -1.3200, 2.0800, 9.8300, -0.8200, 2.1600, 10.3900, -0.7800, 1.7300, 9.8300, -1.1300, 1.3400, 9.7100, -1.3600, 1.6800, 10.2400, -1.5200, 1.6000, 9.3200, -1.8700, 1.4900, 9.1900, -1.8700, 1.4900, 9.1900, -1.9300, 1.0600, 9.9500, -1.3100, 0.8100, 10.6900, 0.0200, 2.0400, 11.0600, 0.2700, 2.5800, 9.3900, -0.0500, 2.2800, 7.3200, -0.3000, 0.4400, 7.6300, -0.3000, 0.4400, 7.6300, -1.4600, 1.0800, 12.3700, -1.9600, 1.7500, 15.3800, -0.7100, 2.1500, 14.0700, 0.7400, 1.7800, 10.4700, 0.6800, 0.8900, 9.9500, 0.0400, 1.5200, 12.0800, 0.0400, 1.5200, 12.0800, -0.4900, 1.7900, 12.7500
  • 26. 26 Example of a signal processing pipeline 32,000 => 240
  • 27. 27 Before and after feature extraction
  • 28. Classification What's happening right now? Anomaly detection Is this behavior out of the ordinary? Forecasting What will happen in the future? 28 3. Letting the computers figure it out
  • 29. 29 Picking the right algorithm Classification Neural network Anomaly detection K-means clustering Forecasting Regression
  • 30. 30 4. Deploying Signal processing, neural network and anomaly detec&on
  • 32. 32 Arduino Nano 33 BLE Sense Cortex-M4F 64MHz 256K RAM Accelerometer + microphone
  • 33. 33 Edge Impulse - TinyML as a service Embedded or edge compute deployment options Test Edge Device Impulse Dataset Acquire valuable training data securely Test impulse with real-time device data flows Enrich data and generate ML process Real sensors in real time Open source SDK Free for developers: edgeimpulse.com
  • 34. 34 Let's build a gesture detection model 1. Collect some data, e.g. 2 minutes per class ('idle', 'wave', 'updown'), in continuous motion (use three dots -> 'Crop sample' if you've collected noise). 2. Create a DSP + ML pipeline to learn from this data. 3. Working? Deploy back to the Nano 33 BLE Sense.
  • 35. 35 Before you flash to the Nano Stop the serial daemon
  • 38. 38 Connect device to new project 1. Create a new project 2. Run: edge-impulse-daemon --clean
  • 39. 39 Scene classification 1. Am in a forest or a conference room? Is the sink on or off? Once again continuous data helps. 2. Define two scenes ('noise' and a scene): collect 5 minutes of data per class. You can do this both on the Nano (13 seconds per time) or on your phone (Devices > Add new device). E.g. music playing vs. no music playing. 3. Again DSP + ML pipeline. 4. Deploy back to device. 5. Feeling adventurous? Export to Arduino library.
  • 41. 41 You can do much more with Edge Impulse Full freedom in signal processing blocks (see 'Building custom processing blocks') Also tutorials on computer vision, discrete events (like keywords) Full freedom in neural network blocks We have a nice community, come and build with us! forum.edgeimpulse.com