TensorFlow Lite for mobile & IoT
0 likes244 views
TensorFlow Lite is a machine learning framework designed for deploying models on mobile and IoT devices, enabling edge computing with lower latency and minimal resource consumption. It supports various applications such as image recognition and voice synthesis, with over 2 billion devices utilizing it globally. The document outlines the process for getting started with TensorFlow Lite, including using pretrained models, developing custom models, and optimizing performance.
TensorFlow Lite for mobile & IoT
- 1. AI for Mobile and IoT Devices: TensorFlow Lite Bringing the world to life with TensorFlow Lite PyLadies Tensorflow All Around - Mia Chang
- 2. What is TensorFlow? End-to-end open source ML platform TensorFlow is a machine learning platform that enables you to develop, train and deploy machine learning and deep learning models anywhere
- 3. TENSORFLOW TENSORFLOW TENSORFLOW.JS TENSORFLOW.JS TFX TFX TENSORBOARD TFX TENSORFLOW LITE TENSORFLOW SWIFT FOR TF SWIFT FOR TFSWIFT FOR TF
- 5. TensorFlow Lite is a framework for deploying ML on mobile devices and embedded systems
- 7. Edge ML Explosion ● Lower latency ● Network connectivity
- 8. Edge ML explosion ● Lower latency ● Network connectivity ● Privacy preserving
- 9. On device ML allows for a new generation of products
- 10. New opportunities ● Uses little compute power
- 11. New opportunities ● Uses little compute power ● Works on limited memory platforms
- 12. New opportunities ● Uses little compute power ● Works on limited memory platforms ● Consumes less battery
- 13. Convert once, deploy anywhere We’re simplifying ML on-device
- 14. Many different things A huge array of machine learning use cases What can I do with it?
- 16. Dance Like Built on TensorFlow Lite using the latest in segmentation, pose and GPU techniques all on-device.
- 17. 5 on-device tasks ○ Running 2 body-part segmentation models ○ Matching the segmentation models ○ Playing a video ○ Encoding a video ○ And running dynamic time warping
- 19. What are other people building with it? Major use cases
- 20. Classification Prediction Audio Translation Voice synthesis Image Object detection Object location OCR Gesture recognition Facial modelling Segmentation Clustering Compression Super resolution Speech Recognition Text to Speech Speech to Text Text Content Video generation Text generation Audio generation
- 21. Have now deployed TensorFlow Lite in production. More than 2B devices globally. Source: https://medium.com/tensorflow/recap-of-the-2019-tensorflow-dev-summit-1b5ede42da8d
- 22. Photos GBoard YouTube NetEase iQiyi Assistant ML KitUber Airbnb Cloud
- 23. TensorFlow Lite powers ML Kit ● ML Kit is out-of-the-box proprietary models that you can run on device
- 24. How do I get started? A basic walkthrough
- 25. Easy to get started 1 Jump start Use our pretrained models or retrain 2 Custom model Develop & deploy your custom model 3 Performance Benchmark, validate & accelerate your models. 4 Optimize Try our Model Optimization Toolkit
- 26. Easy to get started 1 Jump start Use our pretrained models or retrain 2 Custom model Develop & deploy your custom model 3 Performance Benchmark, validate & accelerate your models. 4 Optimize Try our Model Optimization Toolkit
- 27. Download some pretrained models ● Grab a model from tensorflow.org/lite/models ● Bring your own TensorFlow Lite converted model Jump start
- 28. Retrain a model ● Transfer learning reuses a model developed for a different, related task ● Speeds up and simplifies the training process. ● Try the “TensorFlow for poets” codelab Jump start
- 29. Get up and running Jump Start Load your model Transforming data Use the resulting output Run inference
- 30. Easy to get started 1 Jump start Use our pretrained models or retrain 2 Custom model Deploy your custom model 3 Performance Benchmark, validate & accelerate your models. 4 Optimize Try our Model Optimization Toolkit
- 31. Converting your model Custom Model TensorFlow (estimator or keras) Saved Model TF Lite Model TF Lite Converter
- 32. Easy to get started 1 Jump start Use our pretrained models or retrain 2 Custom model Develop & deploy your custom model 3 Performance Benchmark, validate & accelerate your models. 4 Optimize Try our Model Optimization Toolkit
- 33. Benchmark, validate & accelerate your models. ● Utilize the TensorFlow Lite benchmark tooling ● Validate that the model gets the right accuracy, size & performance ● Utilize GPU acceleration via the Delegation API Performance
- 34. Goal: As fast as possible on all hardware (CPU, GPU, DSP, NPU) Get your models running as fast as possible Performance
- 35. Performance MEMORY / LTE MODEM CPU GPU DSP Phone CPU, GPU & DSP - what are they? ● CPU - Central Processing Unit ● GPU - Graphics Processing Unit ● DSP - Digital Signal Processor
- 36. CPU 83 ms CPU 1.8x 47 ms GPU 5.5x 15 ms EdgeTPU 42x 2 ms Incredible inference performance CPU w/ Quantization GPU OpenGL Float16 Quantized Fixed-point CPU on MobileNet V1 Pixel 3 - Single Threaded CPU MobileNet V1
- 37. Easy to Get Started 1 Jump start Use our pretrained or retrained Models 2 Custom model Develop & deploy your custom model 3 Performance Benchmark, validate & accelerate your models. 4 Optimize Try our Model Optimization Toolkit
- 38. Try our Model Optimization Toolkit ● Smaller, faster graphs utilizing the latest in quantization. ● Post-training quantization ● During-training quantization Optimize
- 39. Is TensorFlow Lite only for mobile?
- 40. TensorFlow Lite for microcontrollers MCU More than 150B microcontrollers exist globally today
- 41. What are they? ● No operating system ● Tens of KB of RAM & Flash ● Only CPU, memory & I/O peripherals Small computer on a single circuit IO RAM CPU ROM MCU
- 42. Try it out Q&A Contact twitter @bymiachang https://colab.research.google.com/drive/1ILXQNdf_CX6--_hgQa8S1uIP3MjXarTz