CAA lab :: research code
Computational Aerodynamics & Aeroacoustics Laboratory
Keldysh Institute of Applied Mathematics of RAS
 
4, Miusskaya Sq., Moscow, 125047, Russia, phone: (7 499) 2207218
Main page


The parallel in-house research CFD&CAA code NOISEtte

  • Aerodynamics and aeroacoustics simulations
  • High-accuracy algorithms for mixed-element hybrid meshes
  • Eddy-resolving modeling of compressible viscous flows
  • Multilevel MPI+OpenMP+OpenCL heterogeneous parallelization 
  • Language: C++11, fully-portable (Linux, Windows, whatever)

 

Mathematical basis

  • Compressible Navier – Stokes Equations 
  • Linearized Euler and Navier – Stokes equations
  • Reynolds averaged Navier – Stokes (RANS) equations:
    SA, K-epsilon, K-omega, SST
  • Large Eddy Simulation (LES):
    Smagorinsky, S3PQ, S3QR, S3PR, WALE, Sigma, Vreman, Verstappen
  • Detached Eddy Simulation (DES) and modifications
  • Immersed boundary condition (IBC)
  • FW/H far field acoustics

     

Documentation on math models and numerical methods

Numerical Techniques Implemented

Space Approximation
   • Mixed-element unstructured meshes (elements up to 6 faces)
   • Edge-based reconstruction schemes (EBR) 

        

   • WENO and MUSCL-TVD extensions for discontinuous solutions  
   • Riemann solvers: Roe, Rusanov, HLLE, HLLC, Godunov, ...
   • Low-Mach: Turkel, Rieper, Thornber  
   • Boundary conditions: non-reflecting, solid walls, periodic
     
Time Integration
   • Explicit Runge – Kutta method (up to 4th order)
   • Implicit method (up to 2th order) based on Newton linearization 
   • Preconditioned BiCGSTAB for block sparse matrices

 

Current applications

  • Complex flows around obstacles considering acoustic effects
  • Modeling of subsonic and supersonic jets
  • Flows around wings and rotor blades
  • Flows around cavities and BFS configurations
  • Helicopter main rotor and tail rotor simulations
  • Numerical experiments on acoustic liners (fan noise reduction devices)
  • Modeling of impedance tubes with resonator chambers

Current development directions

  • Dynamically adapting meshes with constant topology
  • Immersed boundary conditions
  • New subgrid scales for unstructured meshes
  • High-Mach flows
  • Sliding meshes, high-accuracy sliding interfaces
  • Improvement of eddy-resolving modeling technology
  • New high-accuracy numerical schemes

 

Parallel performance

Parallel algorithm is based on a hybrid MPI+OpenMP+OpenCL parallelization for modern hybrid supercomputer architectures.

Computing domain is decomposed between cluster nodes, then between MPI processes inside nodes, then among OpenMP threads of MPI processes

 Multilevel decomposition MPI+OpenMP

Parallel performance in real applications with EBR5 scheme, implicit time integration: HPC4 of KIAE, flow around a rotor blade, IDDES, 22M nodes (left); OpenMP performance on a 24-core CPU (Intel Xeon 8160), a round jet, IDDES, 1.6M nodes (center); Lomonosov, a 3D cavity, DES, 160M nodes (right)

Parallel efficiency of Noisette in real applications

Parallel performance on hybrid systems in real applications with EBR5 scheme, implicit time integration, IDDES turbulence modeling approach:
K60-GPU, nodes with 2 16-core CPU Intel Xeon Gold 6142 and 4 GPU NVIDIA V100, mesh 80M nodes, flow around a turbine blade (left); 
Lomonosov 2, nodes with 1 14-core CPU Intel Xeon E5-2697v3 and 1 GPU NVIDIA K40, mesh 12.5M nodes, flow around a cylinder (right).

This heterogeoenous MPI+OpenMP+OpenCL parallel implementation was developed within the framework of the Russian Science Foundation project 19-11-00299. It is highly portable and works fine on multicore CPUs, including Elbrus architecture; manycore accelerators, such as Intel Xeon Phi; GPUs from various verndors, including NVIDIA, AMD, Intel; indegrated CPU+GPU devices.  

 Publications:

      Parallel implementation:
  • A. Gorobets, P. Bakhvalov. Heterogeneous CPU+GPU parallelization for high-accuracy scale-resolving simulations of compressible turbulent flows on hybrid supercomputers // Computer Physics Communications 2021. 108231. https://doi.org/10.1016/j.cpc.2021.10823 
      Discretization:
  • P. Bakhvalov, T. Kozubskaya, P. Rodionov. EBR schemes with curvilinear reconstructions for hybrid meshes // Computers and Fluids, 2022, 239, 105352. https://doi.org/10.1016/j.compfluid.2022.105352 
  • P. Bakhvalov, M.Surnachev, Method of averaged element splittings for diffusion terms discretization in vertex-centered framework // Journal of Computational Physics, Vol. 450, 2022, 110819. http://doi.org/10.1016/j.jcp.2021.110819  

      Other references:

  • I. Abalakin, P. Bakhvalov, V. Bobkov, A. Duben, A. Gorobets, T. Kozubskaya, P. Rodionov, N. Zhdanova. NOISEtte CFD&CAA Supercomputer Code for Research and Applications // Supercomputing Frontiers and Innovations, 2024, 11(2), 78–101. http://doi.org/10.14529/jsfi240206 
  • A. Gorobets, P. Bakhvalov, A. Duben, P. Rodionov. Acceleration of NOISEtte Code for Scale-resolving Supercomputer Simulations of Turbulent Flows. Lobachevskii Journal of Mathematics. Vol 41, No 8, pp. 1463–1474, 2020.  https://doi.org/10.1134/S1995080220080077
  • A. Gorobets, P. Bakhvalov. Improving Reliability of Supercomputer CFD Codes on Unstructured Meshes. Supercomputing Frontiers and Innovations. 2019. Vol. 6, No. 4, pp. 44-56. http://dx.doi.org/10.14529/jsfi190403 
  • A. V. Gorobets, M. I. Neiman-Zade, S. K. Okunev, A. A. Kalyakin, S. A. Soukov. Performance of Elbrus-8C Processor in Supercomputer CFD Simulations. Mathematical Models and Computer Simulations. 2019. vol. 11. pp. 914–923. https://doi.org/10.1134/S2070048219060073  
  • A.Gorobets. Parallel Algorithm of the NOISEtte Code for CFD and CAA Simulations. Lobachevskii Journal of Mathematics. 2018, Vol. 39, No. 4, pp. 524–532.  https://doi.org/10.1134/S1995080218040078
  • Bakhvalov Pavel, Kozubskaya Tatiana. EBR-WENO scheme for solving gas dynamics problems with discontinuities on unstructured meshes. Computers and Fluids. 2017. Vol. 157, p. 312-324. https://doi.org/10.1016/j.compfluid.2017.09.004
  • Bakhvalov Pavel, Abalakin Ilya, Kozubskaya Tatiana. Edge-based reconstruction schemes for unstructured tetrahedral meshes. International Journal for Numerical Methods in Fluids. 2016. Vol.81(6). P. 331–356. https://doi.org/10.1002/fld.4187
  • Gorobets Andrey. Parallel technology for numerical modeling of fluid dynamics problems by high-accuracy algorithms. Computational Mathematics and Mathematical Physics. 2015. Vol. 55(4). P. 638-649. https://doi.org/10.1134/S0965542515040065
  • Абалакин И.В., Бахвалов П.А., Горобец А.В., Дубень А.П., Козубская Т.К., Параллельный программный комплекс NOISETTE для крупномасштабных расчетов задач аэродинамики и аэроакустики, Вычислительные методы и программирование. т.13 (2012), стр. 110-125. (PDF)
     

 Recent applications:

  • S.M. Bosniakov, A.V. Wolkov, A.P. Duben, V.I. Zapryagarev, T.K. Kozubskaya, S.V. Mikhaylov, A.I. Troshin, V.O. Tsvetkova. Comparison of two higher accuracy unstructured scale-resolving approaches applied to dual-stream nozzle jet simulation. Math. Mod. and Comp. Simul.(2020) 12, 368–377.  http://doi.org/10.1134/S2070048220030102 
  • I.V. Abalakin , V.G. Bobkov, T.K. Kozubskaya, V.A. Vershkov, B.S. Kritsky, R.M. Mirgazov, Numerical Simulation of Flow around Rigid Rotor in Forward Flight. Fluid Dynamics, 2020, Vol. 55, No. 4, pp. 534–544. http://doi.org/10.1134/s0015462820040011 
  • S.M. Bosnyakov, A.P. Duben, A.A. Zheltovodov, T.K. Kozubskaya, S.V. Matyash, S.V. Mikhailov. Numerical simulation of supersonic separated flow over inclined backward-facing step using RANS and LES methods. Math. Mod. and Comp. Simul. 2020. 12. 453-463. https://doi.org/10.1134/S2070048220040043  
  • Дубень А.П., Жданова Н.С., Козубская Т.К. Численное исследование влияния дефлектора на аэродинамические и акустические характеристики турбулентного течения в каверне. Известия Российской академии наук. Механика жидкости и газа. 2017, № 4, C. 1–12. DOI:10.7868/S0568528117040107
  • Duben Alexey, Kozubskaya Tatiana. Jet Noise Simulation Using Quasi-1D Schemes on Unstructured Meshes. AIAA AVIATION Forum 5-9 June 2017, Denver, Colorado 23rd AIAA/CEAS Aeroacoustics Conference. DOI:10.2514/6.2017-3856
  • Абалакин И.В., Аникин В.А., Бахвалов П.А., Бобков В.Г., Козубская Т.К. Численное исследование аэродинамических и акустических свойств винта в кольце. Известия Российской академии наук. Механика жидкости и газа. 2016. №3. С. 130-145.
    Abalakin Ilya, Anikin Viktor, Bakhvalov Pavel, Bobkov Vladimir, Kozubskaya Tatiana. Numerical Investigation of the Aerodynamic and Acoustical Properties of a Shrouded Rotor. Fluid Dynamics. 2016. Vol. 51(3). P. 419-433. DOI:10.1134/S0015462816030145
  • Даньков Б.Н., Дубень А.П., Козубская Т.К. Численное моделирование возникновения автоколебательного процесса возле трехмерного обратного уступа при трансзвуковом режиме обтекания. Известия Российской академии наук. Механика жидкости и газа. 2016.  N 4. С. 108-119.
    Dankov Boris, Duben Alexey, Kozubskaya Tatiana. Numerical modeling of the self-oscillation onset near a three-dimensional backward-facing step in a transonic flow. Fluid Dynamics. July 2016, Vol. 51(4),  pp 534–543. DOI: 10.1134/S001546281604013X
  • Dankov Boris, Duben Alexey, Kozubskaya Tatiana. Numerical simulation of the transonic turbulent flow around a wedge-shaped body with a backward-facing step. Mathematical Models and Computer Simulations. 2016 Vol.8(3), pp. 274–284. doi:10.1134/S2070048216030054.
 

 

Main page  |   Contacts
Web-design: Cherepock, dualks