Quantifying Uncertainty in the Energy Exascale Earth System Model

A unique strategy sharpens understanding of the behavior and physics within the model’s atmospheric component.

The Science

The atmospheric component of the U.S. Department of Energy’s recently released Energy Exascale Earth System Model version 1 (EAMv1) includes many new features to improve modeling of water cycle processes. Nonlinear interactions among the new features create a significant challenge for understanding EAMv1’s behavior and tuning the various parameters in the physics parameterizations. Researchers at DOE’s Pacific Northwest National Laboratory led an effort to understand and quantify structural errors and identify the most influential parameters within EAMv1. Scientists quantified the simulation sensitivity to those parameters by designing and conducting short ensemble simulations, which provided an opportunity to evaluate and optimize model fidelity in a systematic and computationally efficient manner.

The Impact

Modeling water cycle processes such as clouds and precipitation is a significant challenge in Earth system modeling, but water availability and extreme storms have important implications for energy production and use. This study provides a comprehensive picture of EAMv1’s behavior and improves understanding of model sensitivity to parameters and their interactions in the model. The key findings will help guide next-generation development to reduce model uncertainty in projecting future water cycle change. The short ensemble simulation strategy also provides insights for optimizing use of DOE’s leadership computing facilities for exascale Earth system modeling.


Improving a model’s predictive skill requires tuning to optimize the model representations of physical processes relative to those observed in the real world. Models are commonly tuned one parameter at a time, which can lead to improvements in one aspect at the expense of degradation in another. To address the confounding effects of process interactions, researchers identified 18 parameters that could play a significant role in the representation of cloud microphysics, turbulence, and convection in EAMv1. These processes collectively represent major uncertainty in modeling the Earth’s water cycle. The team conducted more than 6,000 five-day simulations that perturbed the parameters simultaneously using the Latin hypercube sampling method. From the perturbed parameter ensemble (PPE) simulations and the use of different skill score functions, researchers identified the most sensitive parameters, quantified how the model responded to changes of the parameters for both global mean and spatial distribution, and estimated the maximum likelihood of model parameter space for a number of important fidelity metrics. Comparison of the parametric sensitivity using simulations of two different simulation lengths suggested that PPE using short simulations had some bearing on understanding parametric sensitivity of longer simulations. Results from this analysis provided a more comprehensive picture of EAMv1’s behavior. The difficulty in reducing biases—offsets from observations—in multiple variables simultaneously highlights the need to characterize model structural uncertainty (so-called embedded errors) to inform future development efforts.

Principal Investigator(s)

L. Ruby Leung
Pacific Northwest National Laboratory

Yun Qian
Pacific Northwest National Laboratory


This research was supported as part of the Energy Exascale Earth System Model (E3SM) project, funded by the U.S. Department of Energy (DOE) Office of Science, Office of Biological and Environmental Research Earth System Modeling program. B.Y. at Nanjing University was supported by the National Natural Science Foundation of China (41675101). V.L. at UWM was supported by grant DE-SC0016287 through a Climate Model Development and Validation (CMDV) project funded by the Office of Biological and Environmental Research in the DOE Office of Science. This research used high-performance computing resources from the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory, supported by the DOE Office of Science under contract DE-AC05-00OR22725, PNNL Institutional Computing, and the National Energy Research Scientific Computing Center (NERSC), a DOE Office of Science user facility supported under contract DEAC02-05CH11231.


Qian, Y., H. Wan, B. Yang, J.-C. Golaz, B. Harrop, Z. Hou, V.E. Larson, L.R. Leung, G. Lin, W. Lin, P.-L. Ma, H.-Y. Ma, P. Rasch, B. Singh, H. Wang, S. Xie, and K. Zhang. “Parametric Sensitivity and Uncertainty Quantification in the Version 1 of E3SM Atmosphere Model Based on Short Perturbed Parameter Ensemble Simulations.” Journal of Geophysical Research: Atmospheres 123(23), 13,046-13,073 (2018). [DOI:10.1029/2018JD028927]