Skip to content

cavity3d , run-time error ‘invalid device symbol’, CUDA 11.7

OpenLB – Open Source Lattice Boltzmann Code Forums on OpenLB General Topics cavity3d , run-time error ‘invalid device symbol’, CUDA 11.7

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #6598
    hungryfish
    Participant

    The default example ‘olb-1.5r0/examples/laminar/cavity3d’ brings this runtime error under CUDA 11.7 on a V100, do you have any advice for me? (downgrading complex)

    [prepareGeometry] Prepare Geometry … OK
    terminate called after throwing an instance of ‘std::runtime_error’
    what(): invalid device symbol
    Aborted

    Environment:
    used ‘gpu_only.mk’

    mpic++ –version
    g++ (GCC) 10.2.1 20210130 (Red Hat 10.2.1-11)
    Copyright (C) 2020 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions. There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

    nvcc –version
    nvcc: NVIDIA (R) Cuda compiler driver
    Copyright (c) 2005-2021 NVIDIA Corporation
    Built on Mon_Oct_11_21:27:02_PDT_2021
    Cuda compilation tools, release 11.4, V11.4.152
    Build cuda_11.4.r11.4/compiler.30521435_0

    gcc –version
    gcc (GCC) 10.2.1 20210130 (Red Hat 10.2.1-11)
    Copyright (C) 2020 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions. There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

    cavity3d]$ nvidia-smi
    Tue May 31 14:18:28 2022
    +—————————————————————————–+
    | NVIDIA-SMI 515.43.04 Driver Version: 515.43.04 CUDA Version: 11.7 |
    |——————————-+———————-+———————-+
    | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
    | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
    | | | MIG M. |
    |===============================+======================+======================|
    | 0 Tesla V100-PCIE… Off | 00000000:21:01.0 Off | 0 |
    | N/A 32C P0 34W / 250W | 0MiB / 32768MiB | 1% Default |
    | | | N/A |
    +——————————-+———————-+———————-+

    +—————————————————————————–+
    | Processes: |
    | GPU GI CI PID Type Process name GPU Memory |
    | ID ID Usage |
    |=============================================================================|
    | No running processes found |
    +—————————————————————————–+

    cavity3d]$ make clean
    rm -f tmp/*.* tmp/vtkData/*.* tmp/vtkData/data/*.* tmp/imageData/*.* tmp/imageData/data/*.* tmp/gnuplotData/*.* tmp/gnuplotData/data/*.*
    rm -f cavity3d.o cavity3d.d cavity3d
    [demo1706@nodus-p2s-2xlarge-8-3307-0 cavity3d]$ make
    make -C ../../../external
    make[1]: Entering directory `/mnt/nfs/olb-1.5r0/external’
    make -C zlib
    make[2]: Entering directory `/mnt/nfs/olb-1.5r0/external/zlib’
    make[2]: Nothing to be done for `all’.
    make[2]: Leaving directory `/mnt/nfs/olb-1.5r0/external/zlib’
    cp zlib/build/libz.a lib/
    make -C tinyxml
    make[2]: Entering directory `/mnt/nfs/olb-1.5r0/external/tinyxml’
    make[2]: Nothing to be done for `all’.
    make[2]: Leaving directory `/mnt/nfs/olb-1.5r0/external/tinyxml’
    cp tinyxml/build/libtinyxml.a lib/
    make[1]: Leaving directory `/mnt/nfs/olb-1.5r0/external’
    nvcc -O3 -std=c++17 –generate-code=arch=compute_86,code=[compute_86,sm_86] –extended-lambda –expt-relaxed-constexpr -x cu -Xcudafe “–diag_suppress=implicit_return_from_non_void_function –display_error_number –diag_suppress=20014 –diag_suppress=20011” -DPLATFORM_CPU_SISD -DPLATFORM_GPU_CUDA -I../../../src -I../../../external/zlib -I../../../external/tinyxml -c -o cavity3d.o cavity3d.cpp
    nvcc cavity3d.o -o cavity3d -L../../../external/lib -lz -ltinyxml -lcuda -lcudadevrt -lcudart
    [demo1706@nodus-p2s-2xlarge-8-3307-0 cavity3d]$ ls -ltr
    total 53744
    -rw-r–r– 1 demo1706 demo1706 71 Apr 14 12:00 Makefile
    -rw-r–r– 1 demo1706 demo1706 303836 Apr 14 12:00 cavity3d.pvsm
    -rw-r–r– 1 demo1706 demo1706 9416 Apr 14 12:00 cavity3d.cpp
    -rw-rw-r– 1 demo1706 demo1706 102325 May 31 14:00 cavity3d.d
    -rw-rw-r– 1 demo1706 demo1706 29892392 May 31 14:07 cavity3d.o
    -rwxrwxr-x 1 demo1706 demo1706 24713096 May 31 14:07 cavity3d
    [demo1706@nodus-p2s-2xlarge-8-3307-0 cavity3d]$ ./cavity3d
    [Directories] Directory ./tmp/ created.
    [Directories] Directory ./tmp/imageData/ created.
    [Directories] Directory ./tmp/imageData/data/ created.
    [Directories] Directory ./tmp/vtkData/ created.
    [Directories] Directory ./tmp/vtkData/data/ created.
    [Directories] Directory ./tmp/gnuplotData/ created.
    [Directories] Directory ./tmp/gnuplotData/data/ created.
    [UnitConverter] —————– UnitConverter information —————–
    [UnitConverter] — Parameters:
    [UnitConverter] Resolution: N= 30
    [UnitConverter] Lattice velocity: latticeU= 0.1
    [UnitConverter] Lattice relaxation frequency: omega= 1.96464
    [UnitConverter] Lattice relaxation time: tau= 0.509
    [UnitConverter] Characteristical length(m): charL= 1
    [UnitConverter] Characteristical speed(m/s): charU= 1
    [UnitConverter] Phys. kinematic viscosity(m^2/s): charNu= 0.001
    [UnitConverter] Phys. density(kg/m^d): charRho= 1
    [UnitConverter] Characteristical pressure(N/m^2): charPressure= 0
    [UnitConverter] Mach number: machNumber= 0.173205
    [UnitConverter] Reynolds number: reynoldsNumber= 1000
    [UnitConverter] Knudsen number: knudsenNumber= 0.000173205
    [UnitConverter]
    [UnitConverter] — Conversion factors:
    [UnitConverter] Voxel length(m): physDeltaX= 0.0333333
    [UnitConverter] Time step(s): physDeltaT= 0.00333333
    [UnitConverter] Velocity factor(m/s): physVelocity= 10
    [UnitConverter] Density factor(kg/m^3): physDensity= 1
    [UnitConverter] Mass factor(kg): physMass= 3.7037e-05
    [UnitConverter] Viscosity factor(m^2/s): physViscosity= 0.333333
    [UnitConverter] Force factor(N): physForce= 0.111111
    [UnitConverter] Pressure factor(N/m^2): physPressure= 100
    [UnitConverter] ————————————————————-
    [prepareGeometry] Prepare Geometry …
    [SuperGeometry3D] cleaned 0 outer boundary voxel(s)
    [SuperGeometry3D] cleaned 0 inner boundary voxel(s)
    [SuperGeometry3D] the model is correct!
    [CuboidGeometry3D] —Cuboid Stucture Statistics—
    [CuboidGeometry3D] Number of Cuboids: 1
    [CuboidGeometry3D] Delta (min): 0.0333333
    [CuboidGeometry3D] (max): 0.0333333
    [CuboidGeometry3D] Ratio (min): 1
    [CuboidGeometry3D] (max): 1
    [CuboidGeometry3D] Nodes (min): 29791
    [CuboidGeometry3D] (max): 29791
    [CuboidGeometry3D] Weight (min): 29791
    [CuboidGeometry3D] (max): 29791
    [CuboidGeometry3D] ——————————–
    [SuperGeometryStatistics3D] materialNumber=1; count=24389; minPhysR=(0.0333333,0.0333333,0.0333333); maxPhysR=(0.966667,0.966667,0.966667)
    [SuperGeometryStatistics3D] materialNumber=2; count=4561; minPhysR=(0,0,0); maxPhysR=(1,1,1)
    [SuperGeometryStatistics3D] materialNumber=3; count=841; minPhysR=(0.0333333,1,0.0333333); maxPhysR=(0.966667,1,0.966667)
    [prepareGeometry] Prepare Geometry … OK
    terminate called after throwing an instance of ‘std::runtime_error’
    what(): invalid device symbol
    Aborted

    ***********
    The same runtime error occurs in ‘cavity3d’

    #6599
    Adrian
    Keymaster

    The CUDA_ARCH value needs to match the targeted GPU. i.e. for a V100 (Volta) 70 or 72 should be good (lower is also possible but will lead to additional startup time for bytecode translation).

    See also the reference table in rules.mk mentioned in the comment for CUDA_ARCH in config/gpu_only.mk.

    Note that further adaptation (e.g. following config/gpu_openmpi.mk) will be needed for MPI-based multi-GPU execution (regarding the inclusion of mpic++ output).

    Especially for the laminar/cavity3d example you will want to significantly increase the spatial resolution from the default value to take advantage of the GPU.

    See also e.g. the forum/gpu-examples thread.

    • This reply was modified 2 months, 1 week ago by Adrian.
    • This reply was modified 2 months, 1 week ago by Adrian.
Viewing 2 posts - 1 through 2 (of 2 total)
  • You must be logged in to reply to this topic.