Validating Lid-driven cavity with High RE number

[thanhphatvt]

Hi all,
I’m trying to compare the result of open LB with DNS result of lid-driven cavity problem with Re number = 18000. The LES was applied to open LB code and the resolution is 100x100x100. I found the result has some discrepancy with DNS results. I’m trying to fix it by changing the resolution, time-relaxation, u_max in the code but it can no be fixed. Do you have any suggestion for me to fixing this problem? Thank you so much and have a nice day!
https://drive.google.com/drive/folders/1vbL-bn5Z3uZ2nScnCMS3pZPKCeDKEukQ?usp=drive_link

[Adrian]

Did you try higher resolutions and/or other collision models (which LES)?

What is “some discrepancy”, what is your error expectation?

(Your folder is not shared publicly)

[thanhphatvt]

Hi Adrian,
I tried with higher resolution, but the result is the same. As you can see in my result, I think at the peak, the error is quite high, I want to reduce the discrepancy at that peak.
I’ll try other collision model.
Thank you!

[thanhphatvt]

I’m sorry, I just fixed the link

[mathias]

For 18,000, I guess you need much higher resulutions to map the boundray layer.

[thanhphatvt]

Dear Mathias,
I tried with resolution 200x200x200 but the result is quite the same. Should I increase the resolution?
Thank you so much!

[mathias]

Yes, 200 means 8 million cells, thats nothing and can be done on my laptop in minutes! You need to do a convergence study.. and plot all the resultions, you should see convergence

[thanhphatvt]

Dear Mathias,
Got it! I’ll do the mesh independence test. It runs quite slow on my desktop. I use K80 card.
Thanks

[Adrian]

The performance of the default laminar/cavity3d example case can be significantly improved for your usecase, e.g. you should reduce the VTK output to only what you need, remove the (slow) image output using the heatmap writer and so on. You can compare the optimized laminar/cavity3dBenchmark to get an upper bound on what you can expect using OpenLB on this GPU.

What execution mode do you use? (just to make sure that you are not e.g. using MPI+CUDA on a single GPU)

[thanhphatvt]

Dear Adrian,
I used gpu_openmpi.mk config. I will reduce the VTK output.
I’m running the case. I will update it after finishing.
Thanks for your help!
Have a nice day!

[Adrian]

Ok, compiling with MPI support is fine as long as you don’t create multiple MPI processes sharing the same GPU (this will still work but degrade de performance due to unnecessary communication and smaller work units on the GPU). If you use a single K80 you can start the case without MPI just using ./cavity3d (you of course may already do this exactly this way, just making sure)

[thanhphatvt]

Dear Adrian,
Yes, I’m using ./cavity3d for run this case.
Thanks

[thanhphatvt]

Dear Mathias and Adrian,
I already reduced the VTK export. I’m running the lid-driven cavity with Smagorinski model at Re = 18000 with resolution 300 in 1000 second. I think my simulation is too slow. It’s about 1 days for 20% of sim.
Do you have any suggestions for me to improve the computation speed?
Thank you so much!

[Adrian]

What is your throughput in MLUPs? Which GPU? Did you disable the gnuplot output? Did you relate the throughput to the theoretical maximum GPU bandwidth? If so, what is the gap?

[thanhphatvt]

Dear Adrian,
I used K80, the MLUPs is 250. I disabled the GNUplot. I don’t know how to check the GPU bandwidth.
Thank you!

[Author]

Posts