GPU Computing for Data Science
434 likes33,727 views
The document discusses the advantages and applications of GPU computing in data science, highlighting its parallel processing capabilities that significantly enhance tasks like Monte Carlo simulations and machine learning. It outlines the differences between CPU and GPU architecture, the need for specific hardware, and programming frameworks like CUDA and OpenCL for effective GPU utilization. Additionally, it provides resources, examples, and case studies demonstrating GPU's effectiveness in accelerating data processing tasks.
GPU Computing for Data Science
- 1. GPU Computing for Data Science John Joo [email protected] Data Science Evangelist @ Domino Data Lab
- 2. Outline • Why use GPUs? • Example applications in data science • Programming your GPU
- 3. Case Study: Monte Carlo Simulations • Simulate behavior when randomness is a key component • Average the results of many simulations • Make predictions
- 4. Little Information in One “Noisy Simulation” Price(t+1) = Price(t) e InterestRate•dt + noise
- 5. Many “Noisy Simulations” ➡ Actionable Information Price(t+1) = Price(t) e InterestRate•dt + noise
- 6. Monte Carlo Simulations Are Often Slow • Lots of simulation data is required to create valid models • Generating lots of data takes time • CPU works sequentially
- 7. CPUs designed for sequential, complex tasks Source: Mythbusters https://youtu.be/-P28LKWTzrI
- 8. GPUs designed for parallel, low level tasks Source: Mythbusters https://youtu.be/-P28LKWTzrI
- 9. GPUs designed for parallel, low level tasks Source: Mythbusters https://youtu.be/-P28LKWTzrI
- 10. Applications of GPU Computing in Data Science • Matrix Manipulation • Numerical Analysis • Sorting • FFT • String matching • Monte Carlo simulations • Machine learning • Search Algorithms for GPU Acceleration • Inherently parallel • Matrix operations • High FLoat-point Operations Per Sec (FLOPS)
- 11. GPUs Make Deep Learning Accessible Google Datacenter Stanford AI Lab # of machines 1,000 3 # of CPUs or GPUs 2,000 CPUs 12 GPUs Cores 16,000 18,432 Power used 600 kW 4 kW Cost $5,000,000 $33,000 Adam Coates, Brody Huval,Tao Wang, David Wu, Bryan Catanzaro, Ng Andrew ; JMLR W&CP 28 (3) : 1337–1345, 2013
- 12. CPU vs GPU Architecture: Structured for Different Purposes CPU 4-8 High Performance Cores GPU 100s-1000s of bare bones cores
- 13. Both CPU and GPU are required CPU GPU Compute intensive functions Everything else General Purpose GPU Computing (GPGPU) Heterogeneous Computing
- 14. Getting Started: Hardware • Need a computer with GPU • GPU should not be operating your display Spin up a GPU/CPU computer with 1 click. 8 CPU cores, 15 GB RAM 1,536 GPU cores, 4GB RAM
- 16. Programming CPU • Sequential • Write code top to bottom • Can do complex tasks • Independent Programming GPU • Parallel • Multi-threaded - race conditions • Low level tasks • Dependent on CPU Getting Started: Software
- 17. Talking to your GPU CUDA and OpenCL are GPU computing frameworks
- 18. Choosing How to Interface with GPU: Simplicity vs Flexibility Application specific libraries General purpose GPU libraries Custom CUDA/ OpenCL code Flexibility Simplicity Low Low High High
- 19. Application Specific Libraries Python • Theano - Symbolic math • TensorFlow - ML • Lasagne - NN • Pylearn2 - ML • mxnet - NN • ABSsysbio - Systems Bio R • cudaBayesreg - fMRI • mxnet - NN • rpud -SVM • rgpu - bioinformatics Tutorial on using Theano, Lasagne, and no-learn: http://blog.dominodatalab.com/gpu-computing-and-deep-learning/
- 20. General Purpose GPU Libraries • Python and R wrappers for basic matrix and linear algebra operations • scikit-cuda • cudamat • gputools • HiPLARM • Drop-in library
- 21. Drop-in Library Credit: NVIDIA Also works for Python! http://scelementary.com/2015/04/09/nvidia-nvblas-in-numpy.html
- 22. Custom CUDA/OpenCL Code 1. Allocate memory on the GPU 2. Transfer data from CPU to GPU 3. Launch the kernel to operate on the CPU cores 4. Transfer results back to CPU
- 23. Example of using Python and CUDA: Monte Carlo Simulations • Using PyCuda to interface Python and CUDA • Simulating 3 million paths, 100 time steps each
- 24. Python Code for CPU Python/PyCUDA Code for GPU 8 more lines of code
- 25. Python Code for CPU Python/PyCUDA Code for CPU 1. Allocate memory on the GPU
- 26. Python Code for CPU Python/PyCUDA Code for CPU 2. Transfer data from CPU to GPU
- 27. Python Code for CPU Python/PyCUDA Code for CPU 3. Launch the kernel to operate on the CPU cores
- 28. Python Code for CPU Python/PyCUDA Code for CPU 4. Transfer results back to CPU
- 29. Python Code for CPU 26 sec Python/PyCUDA Code for CPU 8 more lines of code 1.5 sec 17x speed up
- 30. Some sample Jupyter notebooks • https://app.dominodatalab.com/johnjoo/gpu_examples • Monte Carlo example using PyCUDA • PyCUDA example compiling CUDA C for kernel instructions • Scikit-cuda example of matrix multiplication • Calculating a distance matrix using rpud
- 31. More resources • NVIDIA • https://developer.nvidia.com/how-to-cuda-python • Berkeley GPU workshop • http://www.stat.berkeley.edu/scf/paciorek- gpuWorkshop.html • Duke Statistics on GPU (Python) • http://people.duke.edu/~ccc14/sta-663/ CUDAPython.html • Andreas Klockner’s webpage (Python) • http://mathema.tician.de/ • Summary of GPU libraries • http://fastml.com/running-things-on-a-gpu/
- 32. More resources • Walk through of CUDA programming in R • http://blog.revolutionanalytics.com/2015/01/parallel- programming-with-gpus-and-r.html • List of libraries for GPU computing in R • https://cran.r-project.org/web/views/ HighPerformanceComputing.html • Matrix computations in Machine Learning • http://numml.kyb.tuebingen.mpg.de/numl09/ talk_dhillon.pdf