“Deploying Deep Learning Applications on FPGAs with MATLAB,” a Presentation from MathWorks
0 likes148 views
The document discusses the deployment of deep learning applications on FPGAs using MATLAB, highlighting essential challenges, system requirements, and a collaborative workflow for AI model design and training. It also details methods for optimization and integration with various hardware platforms, and provides guidance on prototyping and running deep learning algorithms efficiently. Key features include support for different precision formats, an overview of FPGA deployment processes, and practical examples for application logic.
![© 2020 MathWorks
Application
logic
Deep Learning HDL Toolbox
19
Layer
control
instructions
Weights &
Activations
Re-train
Modify network
% Create target object
hTarget = dlhdl.Target(…)
% Create workflow object, using the target
hW = dlhdl.Workflow(…);
% Compile the network
hW.compile;
% Program the bitstream and deploy the compiled network and weights
hW.deploy;
% Run prediction
[score, speed] = hW.predict(img, ‘Profile’, ‘on’);>> deepNetworkDesigner
Parameters Speed
140 MB 18 fps
84 MB 45 fps
68 MB 139 fps
Quantize
>> deepNetworkQuantizer
Generate
HDL
Iterate and Converge on Deep Learning FPGA Deployment from MATLAB](https://image.slidesharecdn.com/ericksonmathworks2020embeddedvisionsummitfinal-201130142731/85/Deploying-Deep-Learning-Applications-on-FPGAs-with-MATLAB-a-Presentation-from-MathWorks-19-320.jpg)
“Deploying Deep Learning Applications on FPGAs with MATLAB,” a Presentation from MathWorks
- 1. © 2020 MathWorks Deploying Deep Learning Application on FPGAs with MATLAB Jack Erickson Technical Marketing September 2020
- 2. © 2020 MathWorks Airborne Image Analysis Deep Learning Deployment on Embedded Devices 2 Autonomous Driving Industrial Inspection Medical Image Analysis Wireless Modulation Classification Radar Signature Classification
- 3. © 2020 MathWorks System Requirements Drive Network Design 3 Industrial Inspection Hardware/Software Engineers Systems Engineer Deep Learning Practitioner Camera specs Accuracy Latency Cost Power
- 4. © 2020 MathWorks Challenges of Deploying Deep Learning to FPGA Hardware: Convolution Each stride is an 11x11x3 matrix multiply-accumulate →105M floating-point multiply operations! 55 55 96 filters stride=4 224 224 11x11 96 filters of 11x11x3 of 32-bit parameters →140k bytes 11x11 →1.16M bytes of activations
- 5. © 2020 MathWorks Challenges of Deploying Deep Learning to FPGA Hardware conv 1 conv 2 conv 3 conv 4 conv 5 fc6 fc7 fc8input Total 140K 1.2M 3.5M 5.2M 1.8M 148M 64M 16M Parameters (Bytes) n/a 230 M 1.1M 728K 252K 252K 168K 16K 16K 4K Activations (Bytes) 588K 3.1 M 105M 223M 149M 112M 74M 37M 16M 4MFLOPs n/a 720 M Off-chip RAM Block RAM DSP Slices
- 6. © 2020 MathWorks Deploying Deep Learning to FPGA Hardware Requires Collaboration 140K 1.2M 3.5M 5.2M 1.8M 148M 64M 16M Parameters (Bytes) conv 1 conv 2 conv 3 conv 4 conv 5 fc6 fc7 fc8 n/a input Total 230 M 1.1M 728K 252K 252K 168K 16K 16K 4K Activations (Bytes) 588K 3.1 M 105M 223M 149M 112M 74M 37M 16M 4MFLOPs n/a 720 M Resize Acquire data Output / display Mem i/f Optimize • Network /layers • Fixed-point quantization • Processor micro-architecture
- 7. © 2020 MathWorks A Collaborative AI Workflow Model design and tuning Hardware accelerated training Interoperability AI Modeling Integration with complex systems System verification and validation System simulation System Design Data cleansing and preparation Simulation- generated data Human insight Data Preparation 7 Enterprise systems Embedded devices Edge, cloud, desktop Deployment I Iteration and Refinement
- 8. © 2020 MathWorks Design and Analyze Your Networks in MATLAB Classification Learner app to try different classifiers and find the best fit for your data set Deep Network Designer app to build, visualize, and edit deep learning networks Model design and tuning Hardware accelerated training Interoperability AI Modeling 8
- 9. © 2020 MathWorks 9 MATLAB Interoperates with Other AI Frameworks Caffe importer Keras importer Model design and tuning Hardware accelerated training Interoperability AI Modeling
- 10. © 2020 MathWorks Application logic Deploy from MATLAB to a Variety of Hardware Platforms 10 FPGA CPU GPU Enterprise systems Embedded devices Edge, cloud, desktop Deployment
- 11. © 2020 MathWorks FPGA Deployment from MATLAB
- 12. © 2020 MathWorks Application logic Get Started Prototyping on FPGA with Deep Learning HDL ToolboxTM Hardware support package Deep learning processor with I/O and external memory interfaces • Int8 or single precision • Supported boards: • Xilinx: ZCU102 or ZC706 • Intel: Arria10 SoC • http://mathworks.com/hardware-support.html FPGA Bitstream Layer control instructions Weights & Activations Fully Connected Module Convolution Module Processor Control Memory Access Activations Activations Activations
- 13. © 2020 MathWorks Defect Detection Example Application logic Pre-processing: Extract regions and resize Post-processing: Annotate and label Inference: Predict using trained network FPGA
- 14. © 2020 MathWorks Run Deep Learning on FPGA from MATLAB 14
- 15. © 2020 MathWorks Application logic Prototyping: Design Exploration and Customization 15 Layer control instructions Weights & Activations Re-train
- 16. © 2020 MathWorks Design Exploration and Customization 16
- 17. © 2020 MathWorks Optimizing Deep Learning Applications Requires Collaboration Systems Engineer Deep Learning Practitioner Hardware/Software Engineers Fully Connected Module Convolution Module Processor Control Memory Access Activations Activations Activations x +/- Σ 32 / x +/- Σ 8 / x +/- Σ 8 / x +/- Σ 8 / x +/- Σ 8 /
- 18. © 2020 MathWorks INT8 Quantization 18
- 19. © 2020 MathWorks Application logic Deep Learning HDL Toolbox 19 Layer control instructions Weights & Activations Re-train Modify network % Create target object hTarget = dlhdl.Target(…) % Create workflow object, using the target hW = dlhdl.Workflow(…); % Compile the network hW.compile; % Program the bitstream and deploy the compiled network and weights hW.deploy; % Run prediction [score, speed] = hW.predict(img, ‘Profile’, ‘on’);>> deepNetworkDesigner Parameters Speed 140 MB 18 fps 84 MB 45 fps 68 MB 139 fps Quantize >> deepNetworkQuantizer Generate HDL Iterate and Converge on Deep Learning FPGA Deployment from MATLAB
- 20. © 2020 MathWorks Application logic Deploy from MATLAB to a Variety of Hardware Platforms 20 FPGA CPU GPU Tune for system requirements Prototype from MATLAB Configure and generate RTL Deep Learning HDL Toolbox
- 21. © 2020 MathWorks Resource Slide Deep Learning Solutions in MATLAB https://www.mathworks.com/solutions/deep-learning.html Deep Learning HDL Toolbox https://www.mathworks.com/products/deep-learning-hdl.html Onramp: Deep Learning in MATLAB https://www.mathworks.com/learn/tutorials/deep-learning-onramp.html MathWorks FPGA Solutions Page https://www.mathworks.com/solutions/fpga-asic-soc-development.html 21