Maximum a posteriori estimation

Maximum a Posteriori Estimation

File:Bimodal density.svg

A bimodal density function, which can be used to illustrate the concept of maximum a posteriori estimation.

Maximum a posteriori estimation (MAP) is a method of statistical inference used to estimate an unknown quantity, which is a parameter in a statistical model. It is a type of point estimation that incorporates prior knowledge about the parameter through the use of a prior distribution.

MAP estimation is closely related to Bayesian inference, where the goal is to update the probability distribution for a parameter based on observed data. The MAP estimate is the mode of the posterior distribution, which is the distribution of the parameter after taking into account both the prior distribution and the likelihood of the observed data.

Mathematical Formulation

In the context of MAP estimation, we are interested in estimating a parameter \( \theta \) given observed data \( X \). The MAP estimate \( \hat{\theta}_{\text{MAP}} \) is defined as:

\[ \hat{\theta}_{\text{MAP}} = \underset{\theta}{\operatorname{argmax}} \ p(\theta | X) \]

where \( p(\theta | X) \) is the posterior probability of \( \theta \) given \( X \). By Bayes' theorem, the posterior probability is proportional to the product of the likelihood function \( p(X | \theta) \) and the prior probability \( p(\theta) \):

\[ p(\theta | X) \propto p(X | \theta) \cdot p(\theta) \]

Thus, the MAP estimate can also be expressed as:

\[ \hat{\theta}_{\text{MAP}} = \underset{\theta}{\operatorname{argmax}} \ [\log p(X | \theta) + \log p(\theta)] \]

Comparison with Maximum Likelihood Estimation

Maximum likelihood estimation (MLE) is another method of point estimation that does not incorporate prior information. The MLE is the value of \( \theta \) that maximizes the likelihood function \( p(X | \theta) \). In contrast, MAP estimation incorporates both the likelihood and the prior distribution.

In cases where the prior distribution is uniform, the MAP estimate coincides with the MLE. However, when prior information is available, MAP estimation can provide more accurate estimates by incorporating this additional information.

Applications

MAP estimation is widely used in various fields such as machine learning, signal processing, and image processing. It is particularly useful in situations where prior knowledge about the parameter is available and can be quantified in the form of a prior distribution.

In machine learning, MAP estimation is used in Bayesian networks and Gaussian processes. In image processing, it is used for tasks such as image denoising and image reconstruction.

Related Pages

• Bayesian inference
• Maximum likelihood estimation
• Point estimation
• Bayes' theorem