Reply To: Problem with multi GPU
OpenLB – Open Source Lattice Boltzmann Code › Forums › on OpenLB › General Topics › Problem with multi GPU › Reply To: Problem with multi GPU
Yes, it’s laminar/cylinder3d case. This also happen when I remove and restart. I haven’t changed anything in the code.
I see this problem when I run with MPI by this command “”mpirun -np 4 bash -c ‘export CUDA_VISIBLE_DEVICES=${OMPI_COMM_WORLD_LOCAL_RANK}; ./cylinder3d'””
If I use ./cylinder3d, I can open the file normally.
Here is my workstation when I run with MPI:
| NVIDIA-SMI 470.239.06 Driver Version: 470.239.06 CUDA Version: 11.4 |
|——————————-+———————-+———————-+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 00000000:05:00.0 Off | 0 |
| N/A 44C P0 57W / 149W | 369MiB / 11441MiB | 30% Default |
| | | N/A |
+——————————-+———————-+———————-+
| 1 Tesla K80 Off | 00000000:06:00.0 Off | 0 |
| N/A 36C P0 70W / 149W | 359MiB / 11441MiB | 27% Default |
| | | N/A |
+——————————-+———————-+———————-+
| 2 Tesla K80 Off | 00000000:09:00.0 Off | 0 |
| N/A 43C P0 59W / 149W | 369MiB / 11441MiB | 28% Default |
| | | N/A |
+——————————-+———————-+———————-+
| 3 Tesla K80 Off | 00000000:0A:00.0 Off | 0 |
| N/A 36C P0 71W / 149W | 359MiB / 11441MiB | 29% Default |
| | | N/A |
+——————————-+———————-+———————-+
+—————————————————————————–+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 16351 C ./cylinder3d 366MiB |
| 1 N/A N/A 16352 C ./cylinder3d 356MiB |
| 2 N/A N/A 16353 C ./cylinder3d 366MiB |
| 3 N/A N/A 16355 C ./cylinder3d 356MiB
Thanks