PyTorch Python Tutorial | Deep Learning Using PyTorch | Image Classifier Using PyTorch | Edureka
17 likes2,900 views
The document provides an overview of deep learning, focusing on the PyTorch library and how it compares to TensorFlow. It discusses key concepts such as neural networks, the creation and training procedures for models, and the use of data for tasks like image classification. Additionally, it includes practical steps for implementing a convolutional neural network using the CIFAR-10 dataset.
PyTorch Python Tutorial | Deep Learning Using PyTorch | Image Classifier Using PyTorch | Edureka
- 2. Agenda What is Deep Learning? What is PyTorch? Creating a Neural Network PyTorch v/s TensorFlow Use-Case of PyTorch Summary
- 3. Copyright © 2017, edureka and/or its affiliates. All rights reserved. What is Deep Learning? We will begin by looking at what led to formation of Deep Learning.
- 4. AI, Machine Learning and Deep Learning
- 5. What is Deep Learning? Input Layer Hidden Layer 1 Hidden Layer 2 Output Layer A collection of statistical machine learning techniques used to learn feature hierarchies often based on artificial neural networks
- 6. AI & Deep Learning Training www.edureka.co Programming languages for Neural Networks Multiple languages allow support for working with Neural Networks! Copyright © 2018, edureka and/or its affiliates. All rights reserved.
- 7. AI & Deep Learning Training www.edureka.co Maximised support! LisP
- 8. AI & Deep Learning Training www.edureka.co Python for Deep Learning Let’s focus on the libraries Python has to offer! Copyright © 2018, edureka and/or its affiliates. All rights reserved.
- 9. AI & Deep Learning Training www.edureka.co Python Libraries 4 Python Deep Learning libraries we’ve found to be the most useful and popular!
- 10. AI & Deep Learning Training www.edureka.co Python Libraries 4 Python Deep Learning libraries we’ve found to be the most useful and popular!
- 11. AI & Deep Learning Training www.edureka.co What is PyTorch? The new kid in the block! Copyright © 2018, edureka and/or its affiliates. All rights reserved.
- 12. AI & Deep Learning Training www.edureka.co PyTorch – An Introduction Dynamic Computation Graphs Python Support Easy to Use API Actively used Fast and Feels Native Support for CUDA PyTorch is a Python based scientific computing package
- 13. AI & Deep Learning Training www.edureka.co PyTorch - Origin • PyTorch is a cousin of lua-based Torch framework. • PyTorch is not a simple set of wrappers to support popular language.
- 14. AI & Deep Learning Training www.edureka.co PyTorch Installation Let’s up open up PyCharm and see how to install PyTorch! Copyright © 2018, edureka and/or its affiliates. All rights reserved.
- 15. AI & Deep Learning Training www.edureka.co Major PyTorch Concepts An insight into what PyTorch has to offer to the developers! Copyright © 2018, edureka and/or its affiliates. All rights reserved.
- 16. AI & Deep Learning Training www.edureka.co PyTorch - Concepts Neural Network Layer - Stores State or Learnable Weights Terminologies Tensor Imperative N-Dimensional Array Running on GPU Variable Node in Computational Graph - To store Data and Gradient Module
- 17. AI & Deep Learning Training www.edureka.co The NumPy Bridge! Breakthrough performance!
- 18. AI & Deep Learning Training www.edureka.co The NumPy Bridge! • Converting Torch Tensor to NumPy array and vice versa. • Torch Tensor and NumPy array will share underlying memory locations. Torch Tensor NumPy array NumPy PyTorch
- 19. AI & Deep Learning Training www.edureka.co Autograd Module Saves time by calculating differentiation of parameters at forward pass Automatic Differentiation Replay Backwards Compute Gradients Negative of the slope Calculate the minima of the trace function Records Operations Training Dataset
- 20. AI & Deep Learning Training www.edureka.co Autograd Module torch.Tensor Central Class of Package .requires_grad as True Track all Operations Attribute After computation Call .backward() Gradients Computed Stored in .grad Attribute Stop Tracking History Call .detach() torch.no_grad()
- 21. AI & Deep Learning Training www.edureka.co Creating a Neural Network An insight into how a Neural Network can be created and used Copyright © 2018, edureka and/or its affiliates. All rights reserved.
- 22. AI & Deep Learning Training www.edureka.co Creating a Neural Network Simple Feed-Forward Network Neural Networks torch.nn package using Autograd Define models and differentiate them Layers Methods Forward(input) Returns output nn.Module Start Construct
- 23. AI & Deep Learning Training www.edureka.co Training Algorithm Inputs Calculate Loss Back Propagation Update Parameters train_batch labels_batch output_batch PyTorch Variables Derivatives calculated automatically
- 24. AI & Deep Learning Training www.edureka.co Creating a Neural Network Training procedure for a Neural Network is as follows: Define the Neural Network Iterate over Dataset Process the Input Compute the Loss Propagate Gradients Back Update the weights weight = weight - learning_rate * gradient
- 25. AI & Deep Learning Training www.edureka.co Creating a Neural Network Torch.Tensor nn.Module nn.Parameter Autograd.Function Multi-dimensional array with support for autograd operations like backward() Neural Network module for encapsulating parameters to move them to GPU Tensor which is registered as parameter when assigned as attribute to Module Implements forward and backward definitions of an autograd operation
- 26. AI & Deep Learning Training www.edureka.co Creating a Neural Network At this point, we covered: • Defining a Neural Network • Processing Inputs and Calling Backwards • Computing the Loss • Updating the Weights of the Network
- 27. AI & Deep Learning Training www.edureka.co PyTorch v/s TensorFlow An insight comparing both of the frameworks Copyright © 2018, edureka and/or its affiliates. All rights reserved.
- 28. AI & Deep Learning Training www.edureka.co Comparison – An overview Dynamic Computational Graphs Static Computational Graphs Can make use of Standard Python Flow Control Cannot make use of Standard Python Flow Control Support for Python Debuggers Cannot use native Python Debuggers Dynamic Inspection of Variables and Gradients Cannot inspect Variables and Gradients Dynamically Research Production
- 29. AI & Deep Learning Training www.edureka.co PyTorch – Use Case Let’s look at the working and generation of an image classifier. Copyright © 2018, edureka and/or its affiliates. All rights reserved.
- 30. AI & Deep Learning Training www.edureka.co Problem Statement Generating an Image Classifier which predicts data based on an Image-Set by constructing a Neural Network which is used by companies like Google to create Image-to-Text Applications such as Translation etc. Using this we optimise accuracy of data obtained!
- 31. AI & Deep Learning Training www.edureka.co Dataset What about data? Standard Python Packages can be used to load data into numpy array. Then can be converted into a torch.*Tensor. torchvision package helps to avoid writing boilerplate code Image • Pillow • OpenCV Audio • Scipy • Librosa Text • SpaCy • Cython
- 32. AI & Deep Learning Training www.edureka.co CIFAR10 • Let’s use the CIFAR10 dataset. • The images in CIFAR-10 are of size 3x32x32, i.e. 3-channel colour images of 32x32 pixels in size.
- 33. AI & Deep Learning Training www.edureka.co Flow Diagram Start Load the Dataset Read the Dataset Normalize test Dataset using torchvision Define Convolution Neural Network (CNN) Define Loss Function Train the Network Test the Network Based on Trained Data End Repeat the process to Decrease the Loss Pre-processing of dataset Make Prediction on the Test Data
- 34. AI & Deep Learning Training www.edureka.co Training an Image Classifier We will do the following steps in order: Load and Normalize CIFAR10 using torchvision Define Convolution Neural Network Define Loss Function Train the Network on Training Data Test the Network on Test Data Update the weights weight = weight - learning_rate * gradient
- 35. AI & Deep Learning Training www.edureka.co Step 1: Loading and Normalizing CIFAR10 Using torchvision, it’s extremely easy to load CIFAR10! Load and Normalize CIFAR10 Define CNN Define Loss Function Train the Network Test the Network Update the weights Load Data Normalize Data Convert into Tables Inputs
- 36. AI & Deep Learning Training www.edureka.co Step 2: Define a Convolution Neural Network 3-Channel Images Red Green Blue Load and Normalize CIFAR10 Define CNN Define Loss Function Train the Network Test the Network Update the weights
- 37. AI & Deep Learning Training www.edureka.co Step 3: Define a Loss Function and Optimizer Classification Cross-Entropy Loss Load and Normalize CIFAR10 Define CNN Define Loss Function Train the Network Test the Network Update the weights
- 38. AI & Deep Learning Training www.edureka.co Step 4: Train the Network Loop over Data Iterator Feed the Inputs Optimize Training Load and Normalize CIFAR10 Define CNN Define Loss Function Train the Network Test the Network Update the weights
- 39. AI & Deep Learning Training www.edureka.co Step 5: Test the Network on Test Data Check for Changes in Model Predict Output Class Label Check against ground-truth Add sample to Correct Predictions Correct Incorrect Load and Normalize CIFAR10 Define CNN Define Loss Function Train the Network Test the Network Update the weights
- 40. AI & Deep Learning Training www.edureka.co Results • The results seem pretty good. Network did learn something! • Let us look at how the network performs on the whole dataset.
- 41. AI & Deep Learning Training www.edureka.co Session In A Minute What is Deep Learning? Programming Languages Deep Learning Libraries PyTorch PyTorch v/s TensorFlow Use-Case Implementation
- 42. Copyright © 2018, edureka and/or its affiliates. All rights reserved.