Generative and Meta-Programming - Modern C++ Design for Parallel Computing
1 like2,063 views
The document discusses generative and meta-programming in the context of modern C++ design for parallel computing. It presents the NT2 library, aimed at providing a simple interface for scientific computing and high-performance computing, alongside the Quaff project designed to create C++ skeleton libraries for efficient parallel programming. The presentation emphasizes the importance of user-friendly tools that address the needs of developers in scientific applications.
![Introduction Generative Programming NT2 Quaff Conclusion
Parallel Skeletons in a nutshell
Basic Principles [COLE 89]
There are patterns in parallel applications
Those patterns can be generalized in Skeletons
Applications are assembled as combination of such patterns
LSU Seminar
15 / 23](https://image.slidesharecdn.com/main-130222140101-phpapp02/85/Generative-and-Meta-Programming-Modern-C-Design-for-Parallel-Computing-27-320.jpg)
![Introduction Generative Programming NT2 Quaff Conclusion
Parallel Skeletons in a nutshell
Basic Principles [COLE 89]
There are patterns in parallel applications
Those patterns can be generalized in Skeletons
Applications are assembled as combination of such patterns
Functionnal point of view
Skeletons are Higher-Order Functions
Skeletons support a compositionnal semantic
Applications become composition of state-less functions
LSU Seminar
15 / 23](https://image.slidesharecdn.com/main-130222140101-phpapp02/85/Generative-and-Meta-Programming-Modern-C-Design-for-Parallel-Computing-28-320.jpg)
Generative and Meta-Programming - Modern C++ Design for Parallel Computing
- 1. Introduction Generative Programming NT2 Quaff Conclusion Generative and Meta-Programming Modern C++ Design for Parallel Computing Joel Falcou 05/02/2011 LRI, University Paris Sud XI LSU Seminar 1 / 23
- 2. Introduction Generative Programming NT2 Quaff Conclusion Context In Scientific Computing ... there is Scientific LSU Seminar 2 / 23
- 3. Introduction Generative Programming NT2 Quaff Conclusion Context In Scientific Computing ... there is Scientific Applications are domain driven Users = Developers Users are reluctant to changes LSU Seminar 2 / 23
- 4. Introduction Generative Programming NT2 Quaff Conclusion Context In Scientific Computing ... there is Scientific Applications are domain driven Users = Developers Users are reluctant to changes there is Computing LSU Seminar 2 / 23
- 5. Introduction Generative Programming NT2 Quaff Conclusion Context In Scientific Computing ... there is Scientific Applications are domain driven Users = Developers Users are reluctant to changes there is Computing Computing requires performance ... ... which implies architectures specific tuning ... which requires expertise ... which may or may not be available LSU Seminar 2 / 23
- 6. Introduction Generative Programming NT2 Quaff Conclusion Context In Scientific Computing ... there is Scientific Applications are domain driven Users = Developers Users are reluctant to changes there is Computing Computing requires performance ... ... which implies architectures specific tuning ... which requires expertise ... which may or may not be available The Problem People using computers to do science want to do science first. LSU Seminar 2 / 23
- 7. Introduction Generative Programming NT2 Quaff Conclusion The Problem – and how we want to solve it The Facts The ”Library to bind them all” doesn’t exist (or we should have it already ) All those users want to take advantage of new architectures Few of them want to actually handle all the dirty work The Ends Provide a ”familiar” interface that let users benefit from parallelism Helps compilers to generate better parallel code Increase sustainability by decreasing amount of code to write The Means Generative Programming Template Meta-Programming Embedded Domain Specific Languages LSU Seminar 3 / 23
- 8. Introduction Generative Programming NT2 Quaff Conclusion Talk Layout 1 Introduction 2 Generative Programming 3 NT2 4 Quaff 5 Conclusion LSU Seminar 4 / 23
- 9. Introduction Generative Programming NT2 Quaff Conclusion Generative Programming Domain Specific Generative Component Concrete Application Application Description Translator Parametric Sub-components LSU Seminar 5 / 23
- 10. Introduction Generative Programming NT2 Quaff Conclusion Generative Programming as a Tool Available techniques Dedicated compilers External pre-processing tools Languages supporting meta-programming LSU Seminar 6 / 23
- 11. Introduction Generative Programming NT2 Quaff Conclusion Generative Programming as a Tool Available techniques Dedicated compilers External pre-processing tools Languages supporting meta-programming LSU Seminar 6 / 23
- 12. Introduction Generative Programming NT2 Quaff Conclusion Generative Programming as a Tool Available techniques Dedicated compilers External pre-processing tools Languages supporting meta-programming Definition of Meta-programming Meta-programming is the writing of computer programs that analyse, transform and generate other programs (or them- selves) as their data. LSU Seminar 6 / 23
- 13. Introduction Generative Programming NT2 Quaff Conclusion From Generative to Meta-programming Meta-programmable languages template H ASKELL metaOcaml C++ LSU Seminar 7 / 23
- 14. Introduction Generative Programming NT2 Quaff Conclusion From Generative to Meta-programming Meta-programmable languages template H ASKELL metaOcaml C++ LSU Seminar 7 / 23
- 15. Introduction Generative Programming NT2 Quaff Conclusion From Generative to Meta-programming Meta-programmable languages template H ASKELL metaOcaml C++ C++ meta-programming Relies on the C++ template sub-language Handles types and integral constants at compile-time Proved to be Turing-complete LSU Seminar 7 / 23
- 16. Introduction Generative Programming NT2 Quaff Conclusion Embedded Domain Specific Languages What’s an EDSL ? DSL = Domain Specific Language Declarative language, easy-to-use, fitting the domain EDSL = DSL within a general purpose language EDSL in C++ Relies on operator overload abuse (Expression Templates) Carry semantic information around code fragment Generic implementation become self-aware of optimizations Exploiting static AST At the expression level: code generation At the function level: inter-procedural optimization LSU Seminar 8 / 23
- 17. Introduction Generative Programming NT2 Quaff Conclusion Expression Templates matrix x(h,w),a(h,w),b(h,w); = x = cos(a) + (b*a); x + expr<assign ,expr<matrix&> ,expr<plus cos * , expr<cos ,expr<matrix&> > a b a , expr<multiplies ,expr<matrix&> ,expr<matrix&> > #pragma omp parallel for >(x,a,b); for(int j=0;j<h;++j) { for(int i=0;i<w;++i) { Arbitrary Transforms applied x(j,i) = cos(a(j,i)) on the meta-AST + ( b(j,i) * a(j,i) ); } } LSU Seminar 9 / 23
- 18. Introduction Generative Programming NT2 Quaff Conclusion What is NT 2 ? A Scientific Computing Library Provide a simple, M ATLAB-like interface for users Provide high-performance computing entities and primitives Easily extendable LSU Seminar 10 / 23
- 19. Introduction Generative Programming NT2 Quaff Conclusion What is NT 2 ? A Scientific Computing Library Provide a simple, M ATLAB-like interface for users Provide high-performance computing entities and primitives Easily extendable Principles Take a .m file, copy to a .cpp file LSU Seminar 10 / 23
- 20. Introduction Generative Programming NT2 Quaff Conclusion What is NT 2 ? A Scientific Computing Library Provide a simple, M ATLAB-like interface for users Provide high-performance computing entities and primitives Easily extendable Principles Take a .m file, copy to a .cpp file Add #include <nt2/nt2.hpp> and do cosmetic changes LSU Seminar 10 / 23
- 21. Introduction Generative Programming NT2 Quaff Conclusion What is NT 2 ? A Scientific Computing Library Provide a simple, M ATLAB-like interface for users Provide high-performance computing entities and primitives Easily extendable Principles Take a .m file, copy to a .cpp file Add #include <nt2/nt2.hpp> and do cosmetic changes Compile the file and link with libnt2.a LSU Seminar 10 / 23
- 22. Introduction Generative Programming NT2 Quaff Conclusion M ATLAB you said ? R = I(:,:,1); G = I(:,:,2); B = I(:,:,3); Y = min(abs(0.299.*R+0.587.*G+0.114.*B),235); U = min(abs(-0.169.*R-0.331.*G+0.5.*B),240); V = min(abs(0.5.*R-0.419.*G-0.081.*B),240); LSU Seminar 11 / 23
- 23. Introduction Generative Programming NT2 Quaff Conclusion Now with NT 2 table<double> R = I(_,_,1); table<double> G = I(_,_,2); table<double> B = I(_,_,3); table<double> Y, U, V; Y = min(abs(0.299*R+0.587*G+0.114*B),235); U = min(abs(-0.169*R-0.331*G+0.5*B),240); V = min(abs(0.5*R-0.419*G-0.081*B),240); LSU Seminar 12 / 23
- 24. Introduction Generative Programming NT2 Quaff Conclusion Now with NT 2 table<float,settings(shallow,of_size_<640,480>)> R = I(_,_,1); table<float,settings(shallow,of_size_<640,480>)> G = I(_,_,2); table<float,settings(shallow,of_size_<640,480>)> B = I(_,_,3); table<float,settings(of_size_<640,480>)> Y, U, V; Y = min(abs(0.299*R+0.587*G+0.114*B),235), U = min(abs(-0.169*R-0.331*G+0.5*B),240), V = min(abs(0.5*R-0.419*G-0.081*B),240); LSU Seminar 12 / 23
- 25. Introduction Generative Programming NT2 Quaff Conclusion Some Performances RGB2YUV timing (in cycles/pixels) Size 128x128 256x256 512x512 1024x1024 M ATLAB 2010a (2 cores) 85 89 97 102 C (1 core) 23.6 23.8 23.9 24.0 NT2 (1 core) 22.3 22.4 22.2 24.1 NT2 OpenMP (2 cores) 11.4 11.4 11.5 12.2 NT2 SIMD 5.5 5.6 5.6 5.9 NT2 SIMD+OpenMP 2.9 2.9 2.9 3.0 NT2 SIMD speed-up 3.9 3.9 3.9 4.0 NT2 OpenMP speed-up 1.92 1.96 1.98 1.95 NT2 vs M ATLAB speed-up 29.3 30.7 33.5 34 LSU Seminar 13 / 23
- 26. Introduction Generative Programming NT2 Quaff Conclusion Some real life applications LSU Seminar 14 / 23
- 27. Introduction Generative Programming NT2 Quaff Conclusion Parallel Skeletons in a nutshell Basic Principles [COLE 89] There are patterns in parallel applications Those patterns can be generalized in Skeletons Applications are assembled as combination of such patterns LSU Seminar 15 / 23
- 28. Introduction Generative Programming NT2 Quaff Conclusion Parallel Skeletons in a nutshell Basic Principles [COLE 89] There are patterns in parallel applications Those patterns can be generalized in Skeletons Applications are assembled as combination of such patterns Functionnal point of view Skeletons are Higher-Order Functions Skeletons support a compositionnal semantic Applications become composition of state-less functions LSU Seminar 15 / 23
- 29. Introduction Generative Programming NT2 Quaff Conclusion Spotting skeletons when you see one Pipeline Farm Processus 3 F2 Processus 1 Processus 2 Processus 4 Processus 6 Processus 7 F1 Distrib. F2 Collect. F3 Processus 5 F2 LSU Seminar 16 / 23
- 30. Introduction Generative Programming NT2 Quaff Conclusion Objectives Proposing a C++ skeletons library Limit runtime overhead Use the formal model skeleton as guidelines LSU Seminar 17 / 23
- 31. Introduction Generative Programming NT2 Quaff Conclusion Objectives Proposing a C++ skeletons library Limit runtime overhead Use the formal model skeleton as guidelines What we want to write run( pipeline( seq(F1), seq(F2), seq(F3) ) ) LSU Seminar 17 / 23
- 32. Introduction Generative Programming NT2 Quaff Conclusion Objectives Proposing a C++ skeletons library Limit runtime overhead Use the formal model skeleton as guidelines What we want to run if( rank == 0 ) do { out = F1(); MPI_Send(&out,1,MPI_INT,1,0,MPI_COMM_WORLD); } while( isValid(out) ); if( rank == 1 ) do { MPI_Recv(&in,1,MPI_INT,0,0,MPI_COMM_WORLD,&s); out = F2(in); MPI_Send(&in,1,MPI_INT,2,0,MPI_COMM_WORLD); } while ( isValid(out) ) if( rank == 2 ) do { MPI_Recv(&in,1,MPI_INT,1,0,MPI_COMM_WORLD,&s); F2(in); } while ( isValid(in) ) LSU Seminar 17 / 23
- 33. Introduction Generative Programming NT2 Quaff Conclusion Design Rationale Code generation process Generating the skeleton tree at compile-time Turning this tree into a process network using MPL Producing the C + MPI target code using Fusion PIPE Génération Production φ 2 Génération AST du RPSC φ 2 φ 2 de code C+MPI C C++ φ1 FARM3 φ3 MPI φ 1 f φ 3 φ2 LSU Seminar 18 / 23
- 34. Introduction Generative Programming NT2 Quaff Conclusion Quaff over the CELL The OCELLE project IEF/CEA/TRT Tools for heterog. proc. MPI and skeleton for the Cell Developing for the CELL How to find a good mapping for an application ? LSU Seminar 19 / 23
- 35. Introduction Generative Programming NT2 Quaff Conclusion Harris Corner Detector Mul Ixx=Ix*Ix Gauss Sxx Grad X Ix coarsity I Mul Ixy=Ix*Iy Gauss Sxy Sxx*Syy-Sxy² K Grad Y Iy Mul Iyy=Iy*Iy Gauss Syy LSU Seminar 20 / 23
- 36. Introduction Generative Programming NT2 Quaff Conclusion Harris Corner Detector Mul Ixx=Ix*Ix Gauss Sxx Grad X Ix coarsity I Mul Ixy=Ix*Iy Gauss Sxy Sxx*Syy-Sxy² K Grad Y Iy Mul Iyy=Iy*Iy Gauss Syy Skeleton Code and Benchmarks void full_chain_harris(tile const&,tile&) { run( pardo<8>((seq(grad),seq(mul),seq(gauss),seq(coarsity))); } LSU Seminar 20 / 23
- 37. Introduction Generative Programming NT2 Quaff Conclusion Harris Corner Detector Mul Ixx=Ix*Ix Gauss Sxx Grad X Ix coarsity I Mul Ixy=Ix*Iy Gauss Sxy Sxx*Syy-Sxy² K Grad Y Iy Mul Iyy=Iy*Iy Gauss Syy Skeleton Code and Benchmarks void half_chain_harris(tile const&,tile&) { run( pardo<4>((seq(grad),seq(mul)) | (seq(gauss),seq(coarsity))); } LSU Seminar 20 / 23
- 38. Introduction Generative Programming NT2 Quaff Conclusion Harris Corner Detector Mul Ixx=Ix*Ix Gauss Sxx Grad X Ix coarsity I Mul Ixy=Ix*Iy Gauss Sxy Sxx*Syy-Sxy² K Grad Y Iy Mul Iyy=Iy*Iy Gauss Syy Skeleton Code and Benchmarks void no_chain_harris(tile const&,tile&) { run( pardo<2>( seq(grad) | seq(mul) | seq(gauss) | seq(coarsity)); } LSU Seminar 20 / 23
- 39. Introduction Generative Programming NT2 Quaff Conclusion Harris Corner Detector Mul Ixx=Ix*Ix Gauss Sxx Grad X Ix coarsity I Mul Ixy=Ix*Iy Gauss Sxy Sxx*Syy-Sxy² K Grad Y Iy Mul Iyy=Iy*Iy Gauss Syy Results (PACT 09) Version Full-chain Half-chain No-chain Manual 11.26 8.36 9.97 Skell 11.86 8.64 10.43 Overhead 5.33% 3.35% 4.61% LSU Seminar 20 / 23
- 40. Introduction Generative Programming NT2 Quaff Conclusion Let’s round this up! Parallel Computing for Scientist Parallel programmign tools are required BUT: ... They need to cater to users’ needs ! ... Users should feel «at home» LSU Seminar 21 / 23
- 41. Introduction Generative Programming NT2 Quaff Conclusion Let’s round this up! Parallel Computing for Scientist Parallel programmign tools are required BUT: ... They need to cater to users’ needs ! ... Users should feel «at home» Our claims Software Libraries built as Generic and Generative components can solve a large chunk of parallelism related problems while being easy to use. EDSL are an efficient way to design such libraries C++ provides a wide selection of idioms to achieve this goal. LSU Seminar 21 / 23
- 42. Introduction Generative Programming NT2 Quaff Conclusion Prospectives What we’re cooking at the moment Get a public release of NT 2 and Quaff NT 2 : Automatic matlab conversion with source to source compiler Quaff: Simplify new skeletons design by providing a Semantic Rules EDSL What we’ll be cooking in the future Multi-stage programming for skeleton over the Grid/Cloud NT 2 supports for embedded architectures LSU Seminar 22 / 23
- 43. Thanks for your attention