We have just released a new video on our OpenLB YouTube Channel.
In order to showcase the usability and performance of OpenLB’s GPU support, a turbulent nozzle flow was simulated on the HoreKa (KIT, Germany) supercomputer. This case was adapted for higher resolution from the turbulence/nozzle3d example included in OpenLB 1.5. It utilizes a Smagorinsky LES model for the bulk flow and non-local interpolated boundaries for the in- and outflow conditions on top of a single-precision D3Q19 lattice. The simulation’s 2.5 billion cells were computed on 120 Nvidia A100 GPUs divided into 30 nodes. This resulted in a performance of ~250 billion cell updates per second. ParaView was utilized to generate the visualization.
The executive committee announces the closing of the fifth LBM Spring School with OpenLB Software Lab. We were happy to host 51 participants from 8 countries, including 4 invited speakers in Kraków, Poland. This year’s poster award goes to Pavel Eichler (Czech Technical University in Prague).
Next year, the 6th spring school is planned to take place at the University of Greenwich in England/UK from 2023 June 5th to 9th.
On behalf of the spring school executive committee, Nicolas Hafen, Mathias J. Krause, Jan E. Marquardt, Paweł Madejski, Tomasz Kuś, Navaneethan Subramanian, Maciej Bujalski, Karolina Chmiel.
Early bird registration is open until the 10th of May 2022 for the Fifth Spring School on Lattice Boltzmann Methods with OpenLB Software Lab. It is held in Kraków, Poland, from 6th to 10th of June 2022. The school offers a special lecture on LBM on high performance computers and, for the first time, using GPUs with OpenLB (v. 1.5).
The developer team is very happy to announce the release of the next version of OpenLB. The updated open-source Lattice Boltzmann (LB) code is now available for download.
Major new features include support for GPUs using CUDA, vectorized collision steps on SIMD CPUs, a new implementation of our resolved particle system as well as the possibility of simulating free surface flows and reactions.
Release notes
Core changes and features:
Support for GPUs using CUDA
Support for SIMD collision steps (AVX2 / AVX-512)
New Dynamics concept (including Momenta)
New PostProcessor concept
New resolved particle system implementation
New automatic code generation of CSE-optimized functions
New physical models:
Reactions
Free surface flows
Other changes:
Solver class for structuring simulations
Forward Algorithmic Differentiation
Finite difference methods (FDM) and LBM for advection diffusion (reaction) equations (ADRE)
New examples:
Free surface flows:
fallingDrop(2,3)d
breakingDam2d
deepFallingDrop2d
rayleighInstability3d
New free energy examples:
binaryShearFlow2d
fourRollMill2d
New ADRE examples based on FDM and LBM
advectionDiffusion3d
advectionDiffusionPipe3d
advectionDiffusionReaction2d
reactionFiniteDifferences2d
Compatibility tested on:
Systems
Various Linux distributions
NixOS 21.11
Ubuntu 20.04.4 LTS
Red Hat Enterprise Linux 8.2
Windows WSL 1 and 2
Mac OS 11.6
Compilers
GCC 9, 10, 11
Clang 13
Intel C++ 19, 2021.4
Nvidia CUDA 11.4
Nvidia HPC SDK 21.3
MPI
OpenMPI 3.1, 4.1
Intel MPI 2021.3.0
GPU support
Early benchmarks confirm good GPU utilization for the established lid driven cavity benchmark with local velocity boundaries including non local edge and corner treatment. For example a single precision 1000^3 cavity is simulated at a cell throughput of 42.2 GLUPs on two HoreKa GPU nodes featuring four Nvidia A100 accelerators each, compared to 24.8 GLUPs on a single node. This yields a strong parallel efficiency of 85%.
The same benchmark on two CPU-only nodes utilizing AVX-512 and hybrid parallelization yields a performance of 2.7 GLUPs, leading to a speedup of 15.6 for the GPU code.
Other cases such as a turbulent nozzle flow with non-local interpolated boundaries and Smagorinsky BGK LES also perform well. E.g. for a nozzle flow resolved by 360 million cells and distributed to two GPU nodes we obtain ~36 GLUPs.
As this is the first public release of GPU support in OpenLB, not all features are currently supported. However, due to extensive refactoring of our Dynamics and Post Processor concepts the vast majority of local collision steps and a core set of non local boundaries work transparently on GPUs. This includes support for large scale simulations on multi GPU clusters.
Existing legacy post processors are straight forward to adapt to the new approach. The core idea of which is to implement both local- and non-local cell operations as abstract templates accepting the concept of a cell instead of a specific implementation thereof. For further details see the userguide, specifically the sections on Dynamics and Post Processors.
Examples that work on GPUs without changing a single line of code include:
laminar/cavity(2,3)d
laminar/cavity3dBenchmark
laminar/cylinder(2,3)d
laminar/poiseuille(2,3)d
laminar/bstep(2,3)d
laminar/powerLaw2d
turbulence/nozzle3d
turbulence/venturi3d
turbulence/tgv3d
advectionDiffusionReaction/advectionDiffusion3d
freeSurface/(deep)fallingDrop2d
freeSurface/breakingDam2d
On CPUs, all existing post processors, dynamics and all other features continue to be supported.
We are hard at work expanding the list of GPU-aware features and examples, specifically full boundary condition coverage as well as particle and multi-lattice coupling. Feel free in contacting us if you are interested in joining our efforts on further developing this or any other aspect of the open source framework OpenLB.
Citation
A. Kummerländer, S. Avis, H. Kusumaatmaja, F. Bukreev, D. Dapelo, S. Großmann, N. Hafen, C. Holeksa, A. Husfeldt, J. Jeßberger, L. Kronberg, J. Marquardt, J. Mödl, J. Nguyen, T. Pertzel, S. Simonis, L. Springmann, N. Suntoyo, D. Teutscher, M. Zhong and M.J. Krause.
OpenLB Release 1.5: Open Source Lattice Boltzmann Code. Version 1.5. Apr. 2022.
PS: Please consider joining the developer team by contributing your code. Together we can strengthen the LB community by sharing our research in an open and reproducible way! Feel free to contact us here.
We are happy to announce that the first version of our Android App OpenLBar is ready for release. OpenLBar allows you to view the results of OpenLB simulations in augmented reality. Below you will find various example screenshots.
Feel free to download the new App and have fun with our example simulations in augmented reality.
Our Spring School event in 2022 was originally scheduled for the 21th to the 25th of March 2022. We have now decided to postpone the event, due to obvious reasons. The Spring School will now take place from the 6th to the 10th of June 2022. All attendees will receive an additional notification via email. We are optimistic that we can meet in person on this date.
In addition we have updated all webpages regarding the Spring School with the new dates. All details regarding the payment of your attendance fee are now available. The registration is still open for anyone who is interested.
Registration is now open for the Fifth Spring School on Lattice Boltzmann Methods with OpenLB Software Lab that will be held in Kraków, Poland from 21st to 25th of March 2022. The spring school introduces scientists and applicants from industry to the theory of LBM and trains them on practical problems. The first half of the week is dedicated to the theoretical fundamentals of LBM up to ongoing research on selected topics. Followed by mentored training on case studies using OpenLB in the second half, where the participants gain deep insights into LBM and its applications. This educational concept offers a comprehensive and personal guided approach to LBM. Participants also benefit from the knowledge exchange during the poster session, coffee breaks, and the excursion. We look forward to your participation.
Keep in mind that the number of participants is limited and that the registration follows a first come first serve principle.
On behalf of the spring school executive committee, Nicolas Hafen, Mathias J. Krause, Jan E. Marquardt, Paweł Madejski, Tomasz Kuś, Navaneethan Subramanian, Maciej Bujalski and Karolina Chmiel
(Covid) Virus Risk Simulation of a Person Breathing Using an Air Ventilation Systems with LBM – Simulations in Process Engineering
Are ventilation systems effective against aerosol emission from breathing?
A typical way of transmitting viruses like SARS-CoV-2 is through saliva aerosol particles which are sprayed in the air by coughing or sneezing but also through breathing. If these droplets are inhaled by another person there is a high infection risk. To minimize the potential risk of transmitting the virus through the air ventilation systems should keep aerosol particles concentrations low.
The simulation is able to show the turbulent flight path of the aerosol stream transmitted by a sitting person. This demonstrates the importance and difficulties of a ventilation systems in order to decrease infection risk by reducing aerosol particle concentrations.
These videos present the aerosol distributions generated by a breathing human. In every breath, 10,000 particles with diameter of 1 μm are omitted through the mouth. The mesh contains 65.5 millions cells (δx=4.8mm). 10 seconds were simulated using a cluster with 400 cores (20 nodes). The source code is available for download on the OpenLB website.
The fluid flow in the artificial set-up was validated by means of experments and other simulations. These benchmark was published in “Numerical evaluation of thermal comfort using a large eddy lattice Boltzmann method”, M Siodlaczek, M Gaedtke, S Simonis, M Schweiker, N Homma, MJ Krause, Building and Environment, 107618 .
We have just released a new video on our OpenLB YouTube Channel.
The OpenLBee showcase illustrates a bee at a realistic Reynolds number in its landing approach. The bee was placed in a rectangular flow domain. The flow is driven by a velocity inlet and a pressure outlet boundary condition, while for the other plane of the box a symmetry conditions is set. An LES-LBM turbulence scheme is used and a wall model is applied to deal with the boundary layer separation of the thin wings in an efficient way. For the visualization Blender was used.
For further information please view the dedicated Show Case: OpenLBee
We have just released a new video on our OpenLB YouTube Channel.
These videos present the aerosol distributions generated by a breathing human. In every breath, 10,000 particles with diameter of 1 μm are omitted through the mouth. The mesh contains 65.5 million cells (δx=4.8mm). 10 seconds were simulated using a cluster with 400 cores (20 nodes). The results show the very different aerosol distributions (yellow) and the deposit particles (blue) in two different scenarios: without background air flow (left) and with an air flow of 10 m/s by a filter vent (right).
