Making waves - Open-Source Release of the TPLS Flow Solver

Ultra-high Resolution Two-Phase Level Set (TPLS) code was released as Open Source on June 10, 2013 – during the launch event held in the School of Engineering.

The TPLS Project is an international collaboration between three universities: Prash Valluri (School of Engineering, University of Edinburgh), Lennon Ó Náraigh (University College Dublin), Iain Bethune and David Scott (both from Edinburgh Parallel Computing Centre at the University of Edinburgh) and Prof. Peter Spelt (Université de Lyon and and in the Laboratoire de Mécanique des Fluides et d’Acoustique (LMFA), CNRS, Ecole Centrale de Lyon, France).

TPLS (Two-Phase Level Set) is a 3D two-phase flow solver developed by Dr Prash Valluri of the University of Edinburgh's School of Engineering and Dr Lennon Ó Náraigh of University College Dublin. It simulates the interface between two fluid phases, a phenomenon with a number of industrial applications including oil and gas transport, cleaning processes, distillation/absorption, and evaporative cooling of microelectronics.

To do this, an ultra-high resolution 3D Direct Numerical Simulation approach is used combined with the Level-Set method for tracking the developing interface between the phases. Simulations of this sort are highly computationally intensive, so HPC resources such as HECToR are required. TPLS employs a 2D MPI process decomposition coupled with a hybrid OpenMP parallelisation scheme to allow scaling to 1000s of CPU cores. The image on the right shows the detailed interface structure from a high flow-rate simulation showing both linear and non-linear features including 'pinch-off' of droplets.

While Prash, Lennon and Peter have been collaborating since 2009, Prash’s collaboration with Iain Bethune (EPCC) began after the success at the Heriot-Watt Crucible III research proposal competition, and since May 2012 have successfully applied for funding from the HECToR dCSE scheme and EPSRC to improve the performance and scalability of the code. The development work has been carried out since July 2012 by Dr David Scott of EPCC, and EPCC have been able to achieve speedups of over 50% in the pressure solver (the most expensive part of the code) by replacing the bespoke Jacobi/SOR solver with Krylov Subspace methods from the PETSc library, and by optimising the existing use of MPI and OpenMP in the code. As an example, a two-phase flow system in a channel with 28 Million elements can be solved across 1028 cores within 24 h.

To allow other research groups to benefit from our research and code development, we have just released the TPLS code on Sourceforge under the Open-Source BSD License. We are continuing to add new functionality into the code - the Diffuse Interface Method (DIM) to simulate phase change and contact line motion, immersed boundary methods to simulate solid/liquid slurry flows - and a parallel I/O scheme to overcome the bottleneck caused by the existing serial I/O approach when scaling up to larger simulations. These new features will also be added to the public release, and we hope that others will begin to make use of the code and ultimately contribute to future developments of TPLS.

Wave generation and pinch-off events at high flowrates – simulated using TPLS
Wave generation and pinch-off events at high flowrates – simulated using TPLS