The parallel in-house research CFD&CAA code NOISEtte
|
- Aerodynamics and aeroacoustics simulations
- High-accuracy algorithms for mixed-element hybrid meshes
- Eddy-resolving modeling of compressible viscous flows
- Multilevel MPI+OpenMP+OpenCL heterogeneous parallelization
- Language: C++11, fully-portable (Linux, Windows, whatever)
|
Mathematical basis
- Compressible Navier – Stokes Equations
- Linearized Euler and Navier – Stokes equations
- Reynolds averaged Navier – Stokes (RANS) equations:
SA, K-epsilon, K-omega, SST
- Large Eddy Simulation (LES):
Smagorinsky, S3PQ, S3QR, S3PR, WALE, Sigma, Vreman, Verstappen
- Detached Eddy Simulation (DES) and modifications
- Immersed boundary condition (IBC)
- FW/H far field acoustics
Documentation on math models and numerical methods |
Numerical Techniques Implemented
Space Approximation • Mixed-element unstructured meshes (elements up to 6 faces) • Edge-based reconstruction schemes (EBR)
• WENO and MUSCL-TVD extensions for discontinuous solutions • Riemann solvers: Roe, Rusanov, HLLE, HLLC, Godunov, ... • Low-Mach: Turkel, Rieper, Thornber • Boundary conditions: non-reflecting, solid walls, periodic Time Integration • Explicit Runge – Kutta method (up to 4th order) • Implicit method (up to 2th order) based on Newton linearization • Preconditioned BiCGSTAB for block sparse matrices |
Current applications
- Complex flows around obstacles considering acoustic effects
- Modeling of subsonic and supersonic jets
- Flows around wings and rotor blades
- Flows around cavities and BFS configurations
- Helicopter main rotor and tail rotor simulations
- Numerical experiments on acoustic liners (fan noise reduction devices)
- Modeling of impedance tubes with resonator chambers
|
Current development directions
- Dynamically adapting meshes with constant topology
- Immersed boundary conditions
- New subgrid scales for unstructured meshes
- High-Mach flows
- Sliding meshes, high-accuracy sliding interfaces
- Improvement of eddy-resolving modeling technology
- New high-accuracy numerical schemes
|
Parallel performance
Parallel algorithm is based on a hybrid MPI+OpenMP+OpenCL parallelization for modern hybrid supercomputer architectures.
Computing domain is decomposed between cluster nodes, then between MPI processes inside nodes, then among OpenMP threads of MPI processes
Parallel performance in real applications with EBR5 scheme, implicit time integration: HPC4 of KIAE, flow around a rotor blade, IDDES, 22M nodes (left); OpenMP performance on a 24-core CPU (Intel Xeon 8160), a round jet, IDDES, 1.6M nodes (center); Lomonosov, a 3D cavity, DES, 160M nodes (right)
Parallel performance on hybrid systems in real applications with EBR5 scheme, implicit time integration, IDDES turbulence modeling approach: K60-GPU, nodes with 2 16-core CPU Intel Xeon Gold 6142 and 4 GPU NVIDIA V100, mesh 80M nodes, flow around a turbine blade (left); Lomonosov 2, nodes with 1 14-core CPU Intel Xeon E5-2697v3 and 1 GPU NVIDIA K40, mesh 12.5M nodes, flow around a cylinder (right).
This heterogeoenous MPI+OpenMP+OpenCL parallel implementation was developed within the framework of the Russian Science Foundation project 19-11-00299. It is highly portable and works fine on multicore CPUs, including Elbrus architecture; manycore accelerators, such as Intel Xeon Phi; GPUs from various verndors, including NVIDIA, AMD, Intel; indegrated CPU+GPU devices.
|
Publications:
- A. Gorobets, P. Bakhvalov. Heterogeneous CPU+GPU parallelization for high-accuracy scale-resolving simulations of compressible turbulent flows on hybrid supercomputers Computer Physics Communications 2021. 108231. https://doi.org/10.1016/j.cpc.2021.108231
- Bakhvalov P.A., Surnachev M.D., Method of averaged element splittings for diffusion terms discretization in vertex-centered framework // Journal of Computational Physics, Vol. 450, 2022, 110819. http://doi.org/10.1016/j.jcp.2021.110819
- A. Gorobets, P. Bakhvalov, A. Duben, P. Rodionov. Acceleration of NOISEtte Code for Scale-resolving Supercomputer Simulations of Turbulent Flows. Lobachevskii Journal of Mathematics. Vol 41, No 8, pp. 1463–1474, 2020. https://doi.org/10.1134/S1995080220080077
- A. Gorobets, P. Bakhvalov. Improving Reliability of Supercomputer CFD Codes on Unstructured Meshes. Supercomputing Frontiers and Innovations. 2019. Vol. 6, No. 4, pp. 44-56. http://dx.doi.org/10.14529/jsfi190403
- A. V. Gorobets, M. I. Neiman-Zade, S. K. Okunev, A. A. Kalyakin, S. A. Soukov. Performance of Elbrus-8C Processor in Supercomputer CFD Simulations. Mathematical Models and Computer Simulations. 2019. vol. 11. pp. 914–923. https://doi.org/10.1134/S2070048219060073
- A.Gorobets. Parallel Algorithm of the NOISEtte Code for CFD and CAA Simulations. Lobachevskii Journal of Mathematics. 2018, Vol. 39, No. 4, pp. 524–532. https://doi.org/10.1134/S1995080218040078
- Bakhvalov Pavel, Kozubskaya Tatiana. EBR-WENO scheme for solving gas dynamics problems with discontinuities on unstructured meshes. Computers and Fluids. 2017. Vol. 157, p. 312-324. https://doi.org/10.1016/j.compfluid.2017.09.004
- Bakhvalov Pavel, Abalakin Ilya, Kozubskaya Tatiana. Edge-based reconstruction schemes for unstructured tetrahedral meshes. International Journal for Numerical Methods in Fluids. 2016. Vol.81(6). P. 331–356. https://doi.org/10.1002/fld.4187
- Gorobets Andrey. Parallel technology for numerical modeling of fluid dynamics problems by high-accuracy algorithms. Computational Mathematics and Mathematical Physics. 2015. Vol. 55(4). P. 638-649. https://doi.org/10.1134/S0965542515040065
- Абалакин И.В., Бахвалов П.А., Горобец А.В., Дубень А.П., Козубская Т.К., Параллельный программный комплекс NOISETTE для крупномасштабных расчетов задач аэродинамики и аэроакустики, Вычислительные методы и программирование. т.13 (2012), стр. 110-125. (PDF)
Recent applications:
- S.M. Bosniakov, A.V. Wolkov, A.P. Duben, V.I. Zapryagarev, T.K. Kozubskaya, S.V. Mikhaylov, A.I. Troshin, V.O. Tsvetkova. Comparison of two higher accuracy unstructured scale-resolving approaches applied to dual-stream nozzle jet simulation. Math. Mod. and Comp. Simul.(2020) 12, 368–377. http://doi.org/10.1134/S2070048220030102
- I.V. Abalakin , V.G. Bobkov, T.K. Kozubskaya, V.A. Vershkov, B.S. Kritsky, R.M. Mirgazov, Numerical Simulation of Flow around Rigid Rotor in Forward Flight. Fluid Dynamics, 2020, Vol. 55, No. 4, pp. 534–544. http://doi.org/10.1134/s0015462820040011
- S.M. Bosnyakov, A.P. Duben, A.A. Zheltovodov, T.K. Kozubskaya, S.V. Matyash, S.V. Mikhailov. Numerical simulation of supersonic separated flow over inclined backward-facing step using RANS and LES methods. Math. Mod. and Comp. Simul. 2020. 12. 453-463. https://doi.org/10.1134/S2070048220040043
- Дубень А.П., Жданова Н.С., Козубская Т.К. Численное исследование влияния дефлектора на аэродинамические и акустические характеристики турбулентного течения в каверне. Известия Российской академии наук. Механика жидкости и газа. 2017, № 4, C. 1–12. DOI:10.7868/S0568528117040107
- Duben Alexey, Kozubskaya Tatiana. Jet Noise Simulation Using Quasi-1D Schemes on Unstructured Meshes. AIAA AVIATION Forum 5-9 June 2017, Denver, Colorado 23rd AIAA/CEAS Aeroacoustics Conference. DOI:10.2514/6.2017-3856
- Абалакин И.В., Аникин В.А., Бахвалов П.А., Бобков В.Г., Козубская Т.К. Численное исследование аэродинамических и акустических свойств винта в кольце. Известия Российской академии наук. Механика жидкости и газа. 2016. №3. С. 130-145.
Abalakin Ilya, Anikin Viktor, Bakhvalov Pavel, Bobkov Vladimir, Kozubskaya Tatiana. Numerical Investigation of the Aerodynamic and Acoustical Properties of a Shrouded Rotor. Fluid Dynamics. 2016. Vol. 51(3). P. 419-433. DOI:10.1134/S0015462816030145
- Даньков Б.Н., Дубень А.П., Козубская Т.К. Численное моделирование возникновения автоколебательного процесса возле трехмерного обратного уступа при трансзвуковом режиме обтекания. Известия Российской академии наук. Механика жидкости и газа. 2016. N 4. С. 108-119.
Dankov Boris, Duben Alexey, Kozubskaya Tatiana. Numerical modeling of the self-oscillation onset near a three-dimensional backward-facing step in a transonic flow. Fluid Dynamics. July 2016, Vol. 51(4), pp 534–543. DOI: 10.1134/S001546281604013X
- Dankov Boris, Duben Alexey, Kozubskaya Tatiana. Numerical simulation of the transonic turbulent flow around a wedge-shaped body with a backward-facing step. Mathematical Models and Computer Simulations. 2016 Vol.8(3), pp. 274–284. doi:10.1134/S2070048216030054.
|
|