Past Event: Oden Institute Seminar

Data-Driven Bayesian Model Selection: Parameter Space Dimension Reduction using Automatic Relevance Determination Priors

Mohammad Khalil, Sandia National Laboratories, Livermore, California

Abstract

Bayes’ theorem provides parameter estimates that blend prior knowledge of the system parameters with indirect observational data. Bayesian model selection utilizes such estimates in comparing the suitability of many plausible models using the so-called model evidence the probability that randomly selected parameters from the prior would generate the observed data. There are various approaches to prescribe the prior distribution depending on the level of knowledge of the modeler. Popular priors include diffuse priors Jeffrey’s priors conjugate priors and informative priors. The choice of prior distribution and associated parameters that parametrize such priors (called hyper-parameters) has a major impact on any Bayesian estimation procedure and subsequent model selection analysis. In the context of feature selection automatic relevance determination (ARD), aka sparse Bayesian learning, is an effective tool for pruning large numbers of irrelevant features leading to a sparse explanatory subset. It does so by regularizing the Bayesian inference solution space using a parameterized data-dependent prior distribution that effectively prunes away redundant or superfluous features. The hyper-parameter of each ARD prior explicitly represents the relevance of the corresponding model parameter. The hyper-parameters are estimated using the observational data by performing evidence maximization or type-II maximum likelihood. In the context of model selection ARD priors aid in finding the best model nested under the envisioned model. ARD provides a flexible Bayesian platform to find the optimal nested model by eliminating the need to propose candidate nested models and associated prior pdfs. Thus ARD priors effectively reduce the parameter space dimension of the inference procedure based on available observations. This talk will motivate the use of ARD priors in the context of physics-based Bayesian model selection. Results will be presented for an application to model selection of complex aeroelastic systems modeled by coupled nonlinear stochastic ordinary differential equations using noisy wind-tunnel experimental observations. The experiments consist of a NACA0012 airfoil undergoing limit cycle oscillation in the transitional Reynolds number regime. The parameter likelihood is computed by marginalizing over the posterior pdfs of the uncertain time-varying state vector which is obtained using non-linear filtering (extended Kalman filter). A brief intro into state estimation via filtering (Kalman-based and particle filters) will be presented, time-permitting. Bio: Mohammad Khalil is a Senior Member of the Technical Staff at Sandia National Laboratories, California in the Quantitative Modeling and Analysis department. He holds a B.Sc. in microbiology and Immunology and a B.Eng. in Computer and Electrical Engineering from McGill University, Canada, and M.Sc. and Ph.D. degrees in Civil and Environmental Engineering from Carleton University, Canada. He has more than 10 years of experience developing Bayesian inference algorithms for statistical model calibration, parameter estimation, and data assimilation, with applications in fluid-structure interaction, combustion modeling, nonlinear structural dynamics, wildfire forecasting, and time-series analysis.