Skip to content

Reply To: Multi GPUs Calculation


Thank you. Sorry I noticed the linker of -L/home/ubuntu2204/opt/openmpi/lib was successful at cavity3dBenchmark when built.

I tried an execution at cavity3dBenchmark hoever I got an error (as same as previous one)
:$ mpirun -np 2 bash -c ‘export CUDA_VISIBLE_DEVICES=${OMPI_COMM_WORLD_LOCAL_RANK}; ./cavity3d’
The call to cuIpcOpenMemHandle failed. This is an unrecoverable error
and will cause the program to abort.
Hostname: YujiShimojima
cuIpcOpenMemHandle return value: 217
address: 0x1310200000
Check the cuda.h file for what the return value means. A possible cause
for this is not enough free device memory. Try to reduce the device
memory footprint of your application.
[user:01607] Failed to register remote memory, rc=-1
[user:01608] Failed to register remote memory, rc=-1
corrupted size vs. prev_size while consolidating
[user:01607] *** Process received signal ***
[user:01607] Signal: Aborted (6)
[user:01607] Signal code: (-6)
[user:01603] 1 more process has sent help message help-mpi-common-cuda.txt / cuIpcOpenMemHandle failed
[user:01603] Set MCA parameter “orte_base_help_aggregate” to 0 to see all help / error messages