Astrophysics (Index)About

Markov chain Monte Carlo

(model technique to approximate multi-dimensional integrals)

Markov chain Monte Carlo (MCMC) is a technique in calculating an approximation of a multi-dimensional definite integral. It is useful in determining approximations of probabilities that are impractical to calculate directly. The development of the technique (beginning in the 1950s) and its use with computers is one of the keys to the increased popularity of Bayesian statistics.

A Markov chain is a process that goes from state to state according to given probabilities that do not depend upon history, i.e., the probabilities regarding which state will follow the current state are fixed, and, for example, if it happens to return to its current the same state at some time in the future, the probabilities would be the same as at present. (Note that some processes that seemingly don't fit this pattern can be modeled as a Markov chain through inclusion of more states.)

In computational mathematics/science, a Monte Carlo method is the determination of the probability of various outcomes by trying many random examples, and counting and/or recording the various outcomes. It depends upon the randomization of input data, depending on probability to produce good results, and was given the name Monte Carlo because of the place's association with gambling, thus chance.

MCMC incorporates both these methods in an effective manner, making it practical for approximating the solutions of multi-dimensional integrals, effectively solving sets of differential equations too complex for more straight-forward numerical integration techniques. For statistics, the general idea is that a Markov chain can be devised that generates simulated samples of a probability distribution, generating these sample instances with this distribution's frequency. An approximation of the distribution is gathered by counting the samples generated, e.g., binning them to make a histogram. The advantage is that such a Markov chain can at times be devised in cases where determining the distribution through solving equations analytically or through (other) numerical methods is impossible or impractical. In determining such a distribution, it is effectively evaluating a complicated integral, thus can be adapted to other applications requiring similar integral evaluations.

Further reading:

Referenced by pages:
Bayesian statistics
forward model
Monte Carlo method
numerical analysis
spectral energy distribution (SED)