Bayesian Model Averaging Home Page

Bayesian Model Averaging is a technique designed to help account for the uncertainty inherent in the model selection process, something which traditional statistical analysis often neglects. By averaging over many different competing models, BMA incorporates model uncertainty into conclusions about parameters and prediction. BMA has been applied successfully to many statistical model classes including linear regression, generalized linear models, Cox regression models, and discrete graphical models, in all cases improving predictive performance. Details on these applications can be found in the papers below.

Please contact Chris Volinsky if you are have any relevant research you would like to see on this page.

Bayesian Model Averaging Software

R Packages

Original S-PLUS Software designed by Adrian E. Raftery and/or Chris Volinsky:

Bayesian Model Averaging for Generalized Linear Models: bic.glm.
Bayesian Model Averaging for Proportional hazard models: bic.surv
Bayesian Model Averaging for linear regression: bicreg
Bayesian Model Averaging for logistic regression: bic.logit
Bayesian generalized linear modeling and model comparison: glib
(NOTE: if using glib with Splus 6.1 or greater, you will need to install the deprecated function glim first.)

S-PLUS software designed by Jennifer Hoeting:

BMA.shar is a collection of programs that perform Bayesian simultaneous variable selection and outlier identification (SVO) via Markov chain Monte Carlo model composition (MC^3). For more info read the README file.

bma.glm: code for implementing BMA as found in "The Use of Bayesian Model Averaging to better represent uncertainty in ecological models." by Wintle, McCarthy, Volinsky, and Kavanagh, to appear in Conservation Biology. Here is some documentation:

Fortran (f77) software designed by Fernández/Ley/Steel:

Crime dataset and f77 code implementing Markov chain Monte Carlo model composition (MC^3) in "Benchmark Priors for Bayesian Model Averaging".
 
 

Bayesian Model Averaging Papers

"Using Bayesian Model Averaging to Calibrate Forecast Ensembles."
Adrian E. Raftery , Fadoua Balabdaoui , Tilmann Gneiting and Michael Polakowski (2003).
Technical Report no. 440, Department of Statistics, University of Washington.

Website which shows the implementation of the above paper to weather forecasts, by Patrick Tewson

"Long-Run Performance of Bayesian Model Averaging"
Adrian E. Raftery , and Yingye Zheng (2003)
Technical Report no. 433, Department of Statistics, University of Washington.
Uncertainty of the Liberal Peace
Cullen F. Goenner
A Method for Simultaneous Variable and Transformation Selection in Linear Regression
Jennifer Hoeting, Adrian E. Raftery and David Madigan (2002).
Journal of Computational and Graphical Statistics 11 (485-507)
Variable selection and Bayesian model averaging in epidemiological case-control studies.
Viallefont, V., Raftery, A.E. and Richardson, S.(2001)
Statistics in Medicine, 20, 3215-3230.
Model Uncertainty in Cross-Country Growth Regressions
Carmen Fernandez, Eduardo Ley, and Mark Steel (2001)
Journal of Applied Econometrics 16 (563-576)
All papers by Fernandez/Ley/Steel can also be found here
Benchmark Priors for Bayesian Model Averaging
Carmen Fernandez, Eduardo Ley, and Mark Steel (2001)
Journal of Econometrics 100 (381-427)
Bayesian information criterion for censored survival models
Chris Volinsky and Adrian E. Raftery (2000)
Biometrics 56 (256-262)
The choice of variables in multivariate regression: a non-conjugate Bayesian decision theory approach
P.J. Brown, T. Fearn, and M. Vanucci. (1999)
Biometrika, 86
Bayesian Model Averaging (review paper) PDF Version
Jennifer Hoeting, David Madigan, Adrian Raftery and Chris Volinsky (1999)
Statistical Science 14, 382-401.  (NOTE: The printed version in Statistical Science has many typesetting errors, and the journal refused to re-publish it correctly. Please access the paper from the link above, and not from the journal itself)
Bayesian model order determination and basis selection for seemingly unrelated regressions
C.C. Holmes, B.K. Mallick and D.G.T. Denison (1999)
Technical Report, Statistics Section, Department of Maths, Imperial College
Bayesian Wavelength Selection in Multicomponent Analysis
P.J. Brown, M. Vannucci, and T.Fearn (1998)
Journal of Chemometrics, 12, 173-182.
Multivariate Bayesian Variable Selection and Prediction
P.J. Brown, M. Vannucci, and T.Fearn (1998)
Journal of the Royal Statistical Society, Ser. B, 60, 627-642.
Perfect simulation for orthogonal model mixing
C.C. Holmes and B.K. Mallick (1998)
Technical Report, Statistics Section, Department of Maths, Imperial College
Multiple Shrinkage and Subset Selection in Wavelets
Merlise Clyde, Giovanni Parmigiani, and Brani Vidakovic (1998)
Biometrika, 85, 391-402.
Benchmark Priors For Bayesian Model Averaging
Carmen Fernandez, Eduardo Ley, Mark F.J. Steel (1998).
Documento de Trabajo 98-06, Fedea, Madrid, Spain 1998
Bayesian Model Averaging in Proportional Hazard Models: Predicting the Risk of a Stroke
Chris Volinsky, David Madigan, Adrian E. Raftery, and Richard A. Kronmal (1997).
Applied Statistics, 46, 443-448.
Bayesian Model Averaging for Censored Survival Models
Chris Volinsky (1997).
University of Washington Statistics Department Ph.D. Dissertation.
Accounting for Model Uncertainty in Poisson Regression Models: Does Particulate Matter Particularly Matter?
Merlise Clyde and Heather DeSimone-Sasinowska (1997).
ISDS Discussion Paper 97-06
Bayesian Model Averaging for Linear Regression Models
Adrian E. Raftery, David Madigan, and Jennifer A. Hoeting (1997).
Journal of the American Statistical Association, 92, 179-191.
Prediction Via Orthoganalized Model Mixing
Merlise Clyde, Heather DeSimone, and Giovanni Parmigiani (1996)
Journal of the American Statistical Association, 91, 1197-1208.
A Method for Simultaneous Variable Selection and Outlier Identification in Linear Regression
Jennifer Hoeting, Adrian E. Raftery and David Madigan (1996).
Computational Statistics and Data Analysis, 22, 251-270
Bayesian Model Averaging
David Madigan, Adrian E. Raftery, Chris Volinsky, Jennifer Hoeting (1996).
AAAI Workshop on Integrating Multiple Learned Models, 1996, 77-83
Bayesian Model Averaging and Model Selection for Markov Equivalence Classes of Acyclic Digraphs.
David Madigan, Steen A. Andersson, Michael D. Perlman, and Chris T. Volinsky (1996).
Commumications in Statistics: Theory and Methods, 25, 2493-2520.
Statistical Modeling of Fishing Activities in the North Atlantic
Carmen Fernández, Mark F. J. Steel, and Eduardo Ley (1996)
CentER for Economic Research : Discussion Paper 97111.
Accounting for Model Uncertainty in Survival Analysis Improves Predictive Performance.
Adrian Raftery, David Madigan, and Chris T. Volinsky (1995).
Bayesian Statistics 5, Oxford University Press, 323-349.
Assessment and Propagation of Model Uncertainty
David Draper (1995)
Journal of the Royal Statistical Society B, 57, 45--97.

Econometrics Literature

Model combination has been discussed extensively in the econometric literature, usually in the context of combining several experts' forecasts. Bates and Granger (1969) is the forerunner, inspiring a flurry of activity in the field in the early 1970's.
"The combination of forecasts".
J.M. Bates and C.W.J. Granger (1969).
Operations Research Aquarterly, 20, 451-468.
[A seminal paper on combining forecasts, inspiring a flurry of activity in the field]
"Bayesian and non-Bayesian Methods for Combining Models and Forecasts with Applications to Forecasting International Growth Rates".
C. Min and A. Zellner (1990).
J. of Econometrics, 56,(1993) 89-118
"To Combine or not to Combine? Issues of Combining Forecasts".
F.C. Palm and A. Zellner (1992)
J. of Forecasting, 11, 687-701
"Experience with forecasting univariate time series and the combination of forecasts (with discussion)".
P. Newbold and C.W.J. Granger (1974)
Journal of the Royal Statistical Society A, 137, 131-149.
[Reading in front of RSS. Comments from statisticians are interesting, and mostly negative toward the idea of combining models.]

Tutorials

Tutorial on Learning With Bayesian Networks
David Heckerman (1995).
Microsoft Research Technical Report 95-06.

Also see the BMA web page by Mark Steel, featuring his work with Fernandez and Ley.

Bayesian Model Averaging People

Researchers interested in Bayesian Model Averaging, model mixing, and other similar pursuits:

Merlise Clyde, Duke University
David Draper,
Carmen Fernández, University of Saint Andrews
Ed George, University of Texas
David Heckerman, Microsoft Research
Jennifer Hoeting, Colorado State University
Chris Holmes, Imperial College
Eduardo Ley, Research Department, IMF
David Madigan, University of Washington
Giovanni Parmigiani, Duke University
Adrian Raftery, University of Washington
Heather DeSimone Sasinowska, Clemson University
Marina Vannucci, Texas A&M University
Chris Volinsky, AT&T Research
Arnold Zellner, University of Chicago Graduate School of Business

Go back to Chris Volinsky's Home page