Multivariate normal distribution

Multivariate Gaussian

GaussianScatterPCA

Multivariate normal probability in different domains

Probabilities of functions of normal vectors

Classification of several multivariate normals

Multivariate normal distribution is a fundamental concept in statistics, representing the extension of the normal distribution to multiple variables. It plays a crucial role in various fields, including economics, engineering, biology, and psychology, due to its ability to model the relationships between several normally distributed variables.

Definition

The multivariate normal distribution of an n-dimensional random vector X = [X₁, X₂, ..., Xₙ] is defined by its mean vector μ and covariance matrix Σ. The mean vector μ contains the means of the individual variables, and the covariance matrix Σ contains the covariances between all pairs of variables. The distribution is denoted as X ~ N(μ, Σ).

Properties

• Symmetry: Each variable in a multivariate normal distribution follows a normal distribution.
• Linear Combinations: Any linear combination of the variables in a multivariate normal distribution is also normally distributed.
• Marginal Distributions: The marginal distribution of any subset of the variables is also normally distributed.
• Conditional Distributions: The conditional distribution of a subset of variables, given the others, is normally distributed.

Characterization

A random vector X is said to have a multivariate normal distribution if any linear combination of its components has a univariate normal distribution. This can also be characterized using the moment-generating function or the characteristic function of X.

Applications

The multivariate normal distribution is widely used in multivariate analysis, machine learning, and statistical inference, among other areas. It is fundamental in the General Linear Model (GLM), Principal Component Analysis (PCA), and Factor Analysis. It also underpins many methods in spatial statistics and time series analysis.

Estimation

The parameters of a multivariate normal distribution, the mean vector, and the covariance matrix can be estimated using the sample mean and the sample covariance matrix from observed data. These estimations are crucial for further statistical analysis and inference.

Challenges

While the multivariate normal distribution is a powerful tool, its application requires the assumption that the data are normally distributed in multiple dimensions, which may not always be the case. Additionally, estimating the covariance matrix can be challenging, especially when the dimensionality of the data is high compared to the number of observations.