
The Center for Computational Geosciences and Optimization addresses modeling of the solid and fluid earth systems, with emphasis on large scale simulation and inversion on supercomputers. Problems of interest include forward and inverse modeling of regional and global seismic wave propagation, mantle convection, atmospheric and subsurface contaminant transport, ocean dynamics, and flow in porous media. Research in the CCGO is conducted jointly with collaborators from the Jackson School of Geosciences, other ICES centers, the College of Engineering, the Department of Computer Sciences, other universities including Carnegie Mellon, Penn, MIT, Columbia, and Emory, and Sandia National Labs. Related inverse and optimization problems in the mechanical and biomedical engineering sciences are also being pursued.
Omar Ghattas – John A. and Katherine G. Jackson Chair in Computational Geosciences
Tan Bui – Postdoc
Carsten Burstedde – Postdoc
Georg Stadler – Postdoc
Lucas Wilcox – ICES Postdoc
Pearl Flath – Graduate Student
James Martin – Graduate Student
Jennifer Worthen – Graduate Student
Shan Yang – Graduate Student
Robert Hoelscher – Assistant to the Director & Faculty
ALPS stands for Adaptive Large-scale Parallel Simulations. The main focus is to develop a software library that handles arbitrary refinement and coarsening of (hexahedral) meshes. The mesh needs to be distributed between processors with a minimal amount of globally shared information. Load-balancing of the mesh also needs to be performed on-line and in-core. A typical mesh arising in mantle convection simulations is shown below.

An adapted mesh from a mantle convection simulation
Handling adaptive meshes in parallel is generally considered difficult due to the communication costs associated with the irregular shapes arising in mesh partitioning. Finding a partition with equally-distributed workload is itself an NP hard problem which we solve heuristically by space-filling curves. The results below demonstrate at the example of the solution of an advection-diffusion equation that this approach is scalable from 1 to 32,768 cores.

This plot shows the parallel speedup of an explicit PDE solver for the advection-diffusion equation in three dimensions for three different problem sizes. The speedup is almost linear from 1 to 32768 cores.
Rhea - Adaptive mantle convection simulations
Mantle convection is the principal control on the thermal and geological evolution of the Earth. Mantle convection modeling involves solution of the mass, momentum, and energy equations for a viscous, creeping, incompressible non-Newtonian fluid at high Rayleigh and Peclet numbers. Our goal is to conduct global mantle convection simulations that can resolve faulted plate boundaries, down to 1 km scales. Uniform resolution leads to trillion element meshes, which are intractable even on petascale supercomputers. Thus parallel mesh adaptivity is essential.
In this project we develop Rhea, a new generation mantle convection code designed to scale to hundreds of thousands of cores. Rhea is built on ALPS, our parallel octree-based adaptive finite element library that supports new distributed data structures and parallel algorithms for dynamic coarsening, refinement, rebalancing, and repartitioning of the mesh.

Thermal isosurfaces and adaptively refined mesh in mantle convection simulation.
Contact person: Omar Ghattas
Email: omar@ices.utexas.edu
Phone: 512-232-4304