We present a numerical scheme geared for high performance computation of wall-bounded turbulent flows. The number of all-to-all communications is decreased to only six instances by using a two-dimensional (pencil) domain decomposition and utilizing the favourable scaling of the CFL time-step constraint as compared to the diffusive time-step constraint. As the CFL condition is more restrictive at high driving, implicit time integration of the viscous terms in the wall-parallel directions is no longer required. This avoids the communication of non-local information to a process for the computation of implicit derivatives in these directions. We explain in detail the numerical scheme used for the integration of the equations, and the underlying parallelization. The code is shown to have very good strong and weak scaling to at least 64 K cores.
van der Poel, E., Ostilla Monico, R., Donners, J., & Verzicco, R. (2015). A pencil distributed finite difference code for strongly turbulent wall-bounded flows. Computers and fluids, 116, 10-16. https://doi.org/10.1016/j.compfluid.2015.04.007