Whether I can designate the device number to allocate an array. For example, if I have two GPUs what should I do to use the second GPU’s
memory.
The other question: If I can look two GPUs as one?
If I can look two GPUs as one?
No. While you can use multiple devices in the same program, they are separate devices and need to be managed separately.
The easiest and recommended method for multi-GPU programming is to use MPI+OpenACC where each MPI rank uses it’s own GPU. There’s several training videos and blog post on how to do this.
See Class #2: https://developer.nvidia.com/openacc-advanced-course
Also this guide: https://www.pgroup.com/doc/openaccmpi17tut.pdf
Otherwise, you can either use the API call “acc_set_device_num” or the directive “!$acc set device_num()” to set the device number. You can then manage each device’s data and compute.
-Mat
I wanted to reassure that what you mean is that one function can’t execute on two GPUs?
For example, I allocate an integer A in GPU1 and allocate an array B in GPU2. Can I execute a function that includes the expression
B(1) = B(2) + A
?
If I just use CUDA Fortran, can I realize the above-mentioned code?
Hi NVD,
The device memories on the devices are separate so device A can not access device B’s memory. This is true for OpenACC, CUDA Fortran, and CUDA C.
The closest you can come to this is if you use CUDA unified memory where the runtime manages the device memory for you. Then both GPUs will “see” the same memory.
Though compute would still not be implicitly split between the two devices. Maybe in the future this will be possible, but not currently.
-Mat