Parallel and Distributed Computing Chapter 5
- 3. Introduction to OpenMP •OpenMP Is open Multi-Processing •Open is free i.e. freeware •this is not hardware or not a software even not a tool but its is an API, i.e. it is an library that is used with some programming languages •OpenMP is available for C, C++ and FORTRAN Languages •OpenMP is for shared memory model, create multiple threads to share the memory of main process.
- 4. Introduction to OpenMP •It is managed by consortium OpenMP Architecture Review Board (OpenMP ARB) Which is formed by several companies like AMD, Intel, IBM, HP, Nvidia, Oracle etc.
- 5. OpenMP •Shared memory, thread-based parallelism • Shared memory process consists of multiple threads •Explicit Programming • Programmer has full control over parallelization. OpenMP is not an automatic parallel programming model. •Compiler Directive based • Parallelism is specified through the use of compiler directives which are embedded in the source code.
- 6. OpenMP •Each process starts with one main thread this thread is called as master thread in OpenMP. • For particular block of code, we create multiple threads along with this master thread, theses extra threads other than master are called slave Thread.
- 7. OpenMP It is also called fork-join Model, because all slave threads after execution get joined to the master thread i.e. Process starts with single master threads and ends with single master thread.
- 8. Parallel Memory Architecture •Shared Memory •Distributed Memory •Hybrid
- 9. Header file use in C,C++ Language For C, C++ we need to include “omp.h” as header file.
- 10. OpenMP has directives that allow programmer to: •specify the parallel region •specify whether the variables in the parallel section are private or shared •specify how/if the threads are synchronized •specify how to parallelize loops •specify how the works is divided between threads (scheduling)
- 11. Simple Example to create Threads #include<omp.h> int main(){ #pragma omp parallel { int id = omp_get_thread_num(); printf("hello %dn",id); } }
- 12. • By default the number of threads created is equal to the numbers of processors cores
- 13. Creating required number of threads #include<omp.h> int main(){ #pragma omp parallel num_threads(7) { int id = omp_get_thread_num(); printf("hello %dn",id); } }
- 14. Creating required number of threads • If you want to create specific number of threads, use num_threads() and a number indicating number of threads to be created should be passed as argument to num_threads(). • In previous example, seven threads will be created each one will be responsible for printing the required task.
- 15. Creating multiple threads using for loop int main(){ #pragma omp parallel { int id = omp_get_thread_num(); for(int i=0;i<3;i++) { printf(“Hellow Wordn");} } }
- 16. Creating multiple threads using for loop In the previous snippet, since we are not mentioning the number of threads, number of threads will be equl to the number of cores. This for loop will have iterations which are done by these many number of threads in PC
- 17. Allocating different work to different threads In OpenMP, we can allocate different work to different thread by using SECTIONS e.g.
- 18. Allocating different work to different threads #include<omp.h> int main(){ #pragma omp parallel sections num_threads(3) { #pragma omp section { printf("hello word onen"); } #pragma omp section { printf("hello word twon"); } #pragma omp section { printf("hello word threen"); } } In this Example; we have create three threads by mentioning num_threads(3) and each thread is printing different line.
- 19. How do threads interact • OpenMP is a multithreading, shared address model and threads communicate by sharing variables • Un intended sharing of data may cause race conditions • So to control race conditions we use synchronization to protect data conflicts • However synchronization is expensive so we many need to change how data is accessed to minimized the need for synchornization
- 20. Synchronization • Critical: enclosed code block will be executed by only one thread at a time, not simultaneously. Often use to protect shared data from race condition. • Atomic: memory update (write, read) in next instruction will be performed atomically. Doesn’t make entre statement atomic but only memory update is atomic. Better performance than critical • Ordered: structured block is executed in order in which iterations would be executed in sequential loop
- 21. Synchronization • Barrier: each thread waits until all of other threads of a team have reached this point. A work-sharing construct has an implicit barrier synchronization at the end. • nowail: specifies that threads completing assigned work can proceed without waiting for all threads in the team to finish. In the absence of this clause, threads encounter a barrier synchronization at the end of the work sharing construct.
- 22. Synchronization • Synchronization is used to impose order constraints and to protect access to shared data • #pragma omp critical: this directive identifies section of code that must be executed by single thread at a time • At the start of critical region it is identified by a given name until no other thread in the program is executing a critical region with the same name.
- 23. Synchronize Threads in OpenMP In OpenMP we can avoid race condition among threads by using preprocessor directive “pragma omp critical” int main() { #pragma omp parallel { #pragma omp critical { int x=x+1; }}} We have created 5 threads, here only one thread will increment value of x at one time.
- 24. List of some functions of OpenMP Library • Omp_set_num_threads(4): request certain number of threads • Omp_get_thread_num(): returning a thread ID • #pragma omp parallel num threads(3): clause to run parallel with specific number of threads