Skip to content

Multiple GPU compiler error: cannot find -lmpi_cxx: No such file or director

OpenLB – Open Source Lattice Boltzmann Code Forums on OpenLB General Topics Multiple GPU compiler error: cannot find -lmpi_cxx: No such file or director

Viewing 3 posts - 1 through 3 (of 3 total)
  • Author
    Posts
  • #8756
    aseidler
    Participant

    Hi,
    i am trying to implement a multi-gpu simulation on the HPC in dresden. I know that i some how have to modify my flags but i have no clue how to do this. I implemented the gpu_mixed_only like this:

    ######################### GPU Only Calculation #######################
    CXX := nvcc
    CC := nvcc

    CXXFLAGS := -O3
    CXXFLAGS += -std=c++17# –forward-unknown-to-host-compiler
    CXXFLAGS += -Xcompiler -I/software/rome/r23.10/OpenMPI/4.1.4-GCC-11.3.0/include

    #Single GPU
    #PARALLEL_MODE := NONE

    #Parallel GPU
    PARALLEL_MODE := MPI

    MPIFLAGS := -lmpi_cxx -lmpi

    PLATFORMS := CPU_SISD GPU_CUDA

    # for e.g. RTX 30* (Ampere), see table in rules.mk for other options
    CUDA_ARCH :=80

    FLOATING_POINT_TYPE := float

    USE_EMBEDDED_DEPENDENCIES := ON

    –alex

    #8757
    Adrian
    Keymaster

    If you want to use nvcc directly for the entire compilation you quite likely will need to manually prescribe all MPI flags. Those are usually set by the mpicxx wrapper, you can get them for your environment using mpicxx --showme:compile resp. --showme:link. This is what is suggested in the config/gpu_openmpi.mk config (which I assume you started with?)

    I would recommend to use the config/gpu_openmpi_mixed.mk as a starting point. You can compare config/gpu_horeka_nvidiahpc_mixed.mk to see how this looks like for a real world cluster.

    I assume you already have selected the CUDA and MPI modules (with CUDA-awareness compiled in) for your cluster? If so I should be able to guide you to a working config given the output of mpicxx --showme and (optionally) the path to the CUDA module.

    You may also be lucky and be able to use config/gpu_openmpi_mixed.mk basically unmodified if the cluster modules are set up well. This recently happened to me on the Karolina cluster at IT4I (unfortunately this is also the only cluster I encountered where all required modules were set up fully, removing the need for any manual flag twiddling).

    #8758
    aseidler
    Participant

    Hi Adiran,

    I guess i managed to solve the problem on my own.
    I had some problems with “nvcc fatal : Unknown option ‘-Wl,-rpath'” but just deleting the path und changing the MPIFLAGS to

    MPIFLAGS := -L/software/rome/r23.10/OpenMPI/4.1.4-GCC-11.3.0/lib -L/software/rome/r23.10/hwloc/2.7.1-GCCcore-11.3.0/lib -L/software/rome/r23.10/libevent/2.1.12-GCCcore-11.3.0/lib -lmpi

    solved the issue for now.

    Thanks a lot

    -Alex

Viewing 3 posts - 1 through 3 (of 3 total)
  • You must be logged in to reply to this topic.