Exciting news! We're transitioning to the Statewide California Earthquake Center. Our new website is under construction, but we'll continue using this website for SCEC business in the meantime. We're also archiving the Southern Center site to preserve its rich history. A new and improved platform is coming soon!

Enabling performance portability in wave propagation simulation on future supercomputers via Kokkos

Rui Wu, Yongfei Wang, & Christine A. Goulet

Submitted September 11, 2022, SCEC Contribution #12486, 2022 SCEC Annual Meeting Poster #118 (PDF)

Poster Image: 
Numerical simulation of seismic wave propagation is a fundamental tool for seismological studies such as for the inversion of heterogeneous velocity structures, the study of seismic source processes, and for hazard assessment. The advancement of high-performance computing (HPC) has enabled seismic wave simulations to run at an unprecedented scale that captures an increasing level of resolution on supercomputers. However, as HPC technology continues to advance, software porting challenges arise. For example, computational application kernels that were originally developed with some native programming model for a particular kind of hardware (e.g., CUDA on NVIDIA GPUs) may not be supported by next-generation HPCs and a different programming model must be learned for porting (e.g., HIP on AMD GPU and DPC++ on Intel GPUs).
We seek to address this issue by using a unified system to port our existing code to diverse platforms (e.g., multi-CPU or hybrid CPU-GPU) without the need to develop recoding-intensive, system-specific software. Kokkos is a suitable programming model for this purpose. It is based on C++ and provides abstractions for both parallel execution of code and data management. More importantly, it supports a majority of current HPC platforms and planned next-generation platforms (e.g., Frontier system at the Oak Ridge National Laboratory).
In this project, we explored Kokkos’ capabilities by performing multiple numerical experiments. We first implemented and visualized the Mandelbrot set and then compared the performances of different configurations of target numerical stencils, and analyzed the results of implementing different methods. We then implemented Kokkos in an acoustic wave propagation simulation in a 2D heterogeneous media. We implemented multiple internal designs of Kokkos based on the acoustic wave equation, including 2D range with/without vector and 2D flat vector with/without view. By comparing the performance of different methods, we learned an optimal solution to implement large-scale seismic wave propagation. In the future, we plan to expand to 2D and 3D seismic wave simulation for more realistic scenarios. We are also considering implementing MPI with Kokkos to enable multi-node and multi-GPU systems for solving extreme-scale problems. These technical preparations will eventually allow us to bring the Kokkos performance portability into an existing highly scalable seismic wave propagation (AWP or CyberShake) and rupture dynamics (SORD) codes.

Key Words
wave propagation, acoustic wave, Kokkos, high-performance computing, HPC, wave simulator

Citation
Wu, R., Wang, Y., & Goulet, C. A. (2022, 09). Enabling performance portability in wave propagation simulation on future supercomputers via Kokkos. Poster Presentation at 2022 SCEC Annual Meeting.


Related Projects & Working Groups
Computational Science (CS)