Skip to content

Mixed Mode Compilation issue on HPC

Due to recent bot attacks we have changed the sign-up process. If you want to participate in our forum please send a message via our contact form.

› Forums › OpenLB › General Topics › Mixed Mode Compilation issue on HPC

Viewing 8 posts - 1 through 8 (of 8 total)
  • Author
    Posts
  • November 2, 2025 at 12:46 pm #10923
    leonc
    Participant

    Hello there,

    I am a new openlb (version 1.8.1) user and I am trying to compile it on the Gadi HPC, Australia (https://nci.org.au/our-systems/hpc-systems) utilising the V100 GPUs. I load the following modules

    module load gcc/12.2.0
    module load openmpi/4.1.7
    module load cuda/12.3.2

    and in the config.mk file

    CXX := mpic++
    CC := gcc

    CXXFLAGS := -O3 -Wall -march=x86-64-v3
    CXXFLAGS += -std=c++20

    PARALLEL_MODE := MPI

    PLATFORMS := CPU_SISD GPU_CUDA

    CUDA_CXX := nvcc
    CUDA_CXXFLAGS := -O3 -std=c++20

    CUDA_ARCH := 70

    FLOATING_POINT_TYPE := float

    USE_EMBEDDED_DEPENDENCIES := ON

    OLB_MIXED_MODE_INCLUDE_CPP := ON

    The compilation works (and runs) on cavity example but I have compilations issues when trying to run the turbulent channel3d case. Initially I had issues with the default.mixed.mk file with an unescaped “#'” in so edit that with “#” in default.mixed.mk in the indented line:

    OLB_MIXED_MODE_INCLUDE := \#include <olb.h>\n
    ifdef OLB_MIXED_MODE_INCLUDE_CPP
    OLB_MIXED_MODE_INCLUDE := $(CPP_FILES:%=\n\#include “../%”\n)
    endif

    However, I still faced other issues with the default.mixed.mk file. I get an error that it is missing some symbol:

    channel3d.cpp:(.text._ZNSt17_Function_handlerIFPN3olb18AbstractCollisionOIdNS0_11descriptors5D3Q19IJNS2_5FORCEENS2_6TENSORENS2_8VELOCITYENS2_8POROSITYENS2_9VELOCITY2ENS2_16AVERAGE_VELOCITYEEEEEENS0_8PlatformEEZNS0_15DynamicsPromiseIdSA_EC4INS0_8dynamics5TupleIdSA_NS0_7momenta5TupleINSK_11BulkDensityENSK_31MovingPorousMomentumCombinationINSK_12BulkMomentumEEENSK_10BulkStressENSK_11DefineToNEqEEENS0_10equilibria10ThirdOrderENS0_9collision17ParameterFromCellINSV_6HYBRIDENSW_INSV_10HYBRID_RHOENSV_39LocalVanDriestSmagorinskyEffectiveOmegaINSV_14ExternalRhoHRREEEEEEENSI_18DefaultCombinationEEEEENS0_4meta2idIT_EEEUlSD_E0_E9_M_invokeERKSt9_Any_dataOSD_[_ZNSt17_Function_handlerIFPN3olb18AbstractCollisionOIdNS0_11descriptors5D3Q19IJNS2_5FORCEENS2_6TENSORENS2_8VELOCITYENS2_8POROSITYENS2_9VELOCITY2ENS2_16AVERAGE_VELOCITYEEEEEENS0_8PlatformEEZNS0_15DynamicsPromiseIdSA_EC4INS0_8dynamics5TupleIdSA_NS0_7momenta5TupleINSK_11BulkDensityENSK_31MovingPorousMomentumCombinationINSK_12BulkMomentumEEENSK_10BulkStressENSK_11DefineToNEqEEENS0_10equilibria10ThirdOrderENS0_9collision17ParameterFromCellINSV_6HYBRIDENSW_INSV_10HYBRID_RHOENSV_39LocalVanDriestSmagorinskyEffectiveOmegaINSV_14ExternalRhoHRREEEEEEENSI_18DefaultCombinationEEEEENS0_4meta2idIT_EEEUlSD_E0_E9_M_invokeERKSt9_Any_dataOSD_]+0x85): undefined reference to `_ZN3olb23ConcreteBlockCollisionOIdNS_11descriptors5D3Q19IJNS1_5FORCEENS1_6TENSORENS1_8VELOCITYENS1_8POROSITYENS1_9VELOCITY2ENS1_16AVERAGE_VELOCITYEEEELNS_8PlatformE2ENS_8dynamics5TupleIdS9_NS_7momenta5TupleINSD_11BulkDensityENSD_31MovingPorousMomentumCombinationINSD_12BulkMomentumEEENSD_10BulkStressENSD_11DefineToNEqEEENS_10equilibria10ThirdOrderENS_9collision17ParameterFromCellINSO_6HYBRIDENSP_INSO_10HYBRID_RHOENSO_39LocalVanDriestSmagorinskyEffectiveOmegaINSO_14ExternalRhoHRREEEEEEENSB_18DefaultCombinationEEEEC1Ev’

    The HPC staff is not too sure what the issue is and suggested I get help. Thank you in advance!

    Leon

    November 2, 2025 at 6:57 pm #10924
    Adrian
    Keymaster

    This is unusual, can you share which version of GNU Make you are using? The change to the escape in the mixed config should not be necessary (I also just re-tested the build of channel3d in mixed mode to be sure).

    In any case: Did you do a full rebuild of the example after you tweaked the config? (make clean; make) Also, just to be safe: Did you change anything else beside the config? Is there a specific reason why you added the OLB_MIXED_MODE_INCLUDE_CPP := ON to the config.mk? This flag is already set by default in the example’s Makefile.

    November 3, 2025 at 1:01 am #10925
    leonc
    Participant

    Hello Adrian,

    Thank you for your response. I am using GNU Make 4.2.1.

    Yes, I did make clean; make when recompiling.

    No, I did not change anything else other than the config.mk and the default.mixed.mk files.

    OLB_MIXED_MODE_INCLUDE_CPP := ON. The HPC staff advised us to include it when they were running the cavity testcase. Yes, I agree that it would be redundant as it is already in the Makefile. I did switch it to OFF but still had a compilation error.

    Thank you!

    Leon

    November 7, 2025 at 10:37 pm #10936
    Adrian
    Keymaster

    Sorry for the delay, I can reproduce the issue in GNU Make 4.2.1, it works for 4.4. I’ll look into it and provide a fix for you.

    November 8, 2025 at 9:47 am #10938
    Adrian
    Keymaster

    So I think I was able to reproduce your exact issue by:

    1. fixing the escape issue for GNU Make 4.2.1
    2. compiling without GPU support (e.g. by using the default config.mk)
    3. Switching on mixed compilation mode GPU support
    4. Recompiling just the case, not the core library
    5. => linker errors for the same symbols that you got

    So your setup should be fixable by executing make clean-core additionally and then rebuilding (forgot to mention this before)

    I am still looking into a general fix for the escape issue as escaping the include fixes 4.2.1 but breaks 4.4.1. Will keep you updated.

    November 11, 2025 at 1:55 am #10945
    leonc
    Participant

    Hi Adrian,
    Thank you for your response and guidance. I ran the following commands (on Make 4.2.1)
    1. make clean; make clean-core and recompiled the base folder make
    2. compiled the test case make clean; make

    but still received the compilation error. I tried loading another version of GNU Make 4.4.1, used the original default.mixed.mk, repeated steps 1 and 2 and got the following error when compiling the channel3d case

    mpic++ channel3d.o -o channel3d -L../../../external/lib  -lpthread -lz -ltinyxml2 -L../../../build/lib -L . -lolbcuda -lolbcore -lcuda -lcudadevrt -lcudart
channel3d.o: In function <code>_ZNSt17_Function_handlerIFPN3olb18AbstractCollisionOIfNS0_11descriptors5D3Q19IJNS2_5FORCEENS2_6TENSORENS2_8VELOCITYENS2_</code>

    ….

    collect2: error: ld returned 1 exit status
make: *** [../../../default.mixed.mk:142: channel3d] Error 1

    Thank you again for your kind help.
    Leon

    • This reply was modified 4 months ago by leonc.
    • This reply was modified 4 months ago by leonc.
    November 20, 2025 at 2:10 pm #10983
    Adrian
    Keymaster

    Sorry, I can not reproduce this. Did you try retracing the working steps in a clean checkout? This would ensure that the trial and error did not leave any broken objects lying around for the linker to pick up.

    February 25, 2026 at 4:18 am #11068
    leonc
    Participant

    Hi Adrian,

    Managed to get OpenLB installed! To those interested, steps are as below.
    Downloaded version 1.9
    loaded the following modules

    module load pbs gcc/14.2.0 openmpi/5.0.8 cuda/12.9.0

    in config.mk settings

    OLB_MIXED_MODE_INCLUDE_CPP := ON
CXX             := mpic++
CC              := mpicc
CXXFLAGS        := -O3 -march=x86-64-v3
CXXFLAGS        += -std=c++20
PARALLEL_MODE   := MPI
PLATFORMS       := CPU_SISD GPU_CUDA
CUDA_CXX        := nvcc
CUDA_CXXFLAGS   := -std=c++20 -allow-unsupported-compiler
CUDA_LDFLAGS    := -L/apps/cuda/12.9.0/lib64
CUDA_ARCH       := 70
FLOATING_POINT_TYPE := float
USE_EMBEDDED_DEPENDENCIES := ON

    and editing the escaping “#” in default.mixed.mk in the indented line:

    OLB_MIXED_MODE_INCLUDE := \#include <olb.h>\n
ifdef OLB_MIXED_MODE_INCLUDE_CPP
OLB_MIXED_MODE_INCLUDE := $(CPP_FILES:%=\n\#include “../%”\n)
endif

    Kind regards,
    Leon

  • Author
    Posts
Viewing 8 posts - 1 through 8 (of 8 total)
  • You must be logged in to reply to this topic.