Prab: CSV import

2025-02-18T05:04:25Z

CSV import

← Older revision		Revision as of 05:04, 18 February 2025
Line 58:		Line 58:
	* Fisher, R. A. (1936). "The Use of Multiple Measurements in Taxonomic Problems". Annals of Eugenics. 7 (2): 179–188.		* Fisher, R. A. (1936). "The Use of Multiple Measurements in Taxonomic Problems". Annals of Eugenics. 7 (2): 179–188.
	{{dictionary-stub1}}		{{dictionary-stub1}}
			<gallery>
			File:Linear_discriminant_analysis_plot.png\|Linear Discriminant Analysis Plot
			File:Fisher2classes.png\|Fisher's Linear Discriminant for Two Classes
			File:4class3ddiscriminant.png\|3D Discriminant for Four Classes
			File:3dProjections.png\|3D Projections in Linear Discriminant Analysis
			</gallery>

Prab: CSV import

2024-02-24T13:21:54Z

CSV import

New page

== Linear Discriminant Analysis ==

Linear Discriminant Analysis (LDA) is a statistical technique used in machine learning and pattern recognition to find a linear combination of features that characterizes or separates two or more classes of objects or events. It is a supervised learning algorithm that has been widely used in various fields, including computer vision, bioinformatics, and finance.

=== Overview ===

LDA aims to find a linear projection of the data that maximizes the separation between different classes while minimizing the variance within each class. The goal is to reduce the dimensionality of the data while preserving the discriminatory information. By projecting the data onto a lower-dimensional space, LDA can effectively classify new instances based on their feature values.

=== Mathematical Formulation ===

Let's consider a dataset with *n* samples and *d* features, where each sample belongs to one of *k* classes. The goal of LDA is to find a transformation matrix **W** that maps the original *d*-dimensional feature space to a lower-dimensional space, such that the between-class scatter is maximized and the within-class scatter is minimized.

The between-class scatter matrix **SB** is defined as the sum of the outer products of the difference between the class means and the overall mean:

**SB** = Σ(*mi* - *m*)(*mi* - *m*)^T,

where *mi* is the mean vector of class *i* and *m* is the overall mean vector.

The within-class scatter matrix **SW** is defined as the sum of the covariance matrices of each class:

**SW** = Σ**Si**,

where **Si** is the covariance matrix of class *i*.

The objective of LDA is to find the transformation matrix **W** that maximizes the ratio of the determinant of **SB** to the determinant of **SW**. This can be achieved by solving the generalized eigenvalue problem:

**SB** **W** = λ**SW** **W**,

where λ is the eigenvalue and **W** is the eigenvector matrix.

=== Steps of Linear Discriminant Analysis ===

1. Compute the mean vectors and covariance matrices for each class.
2. Compute the between-class scatter matrix **SB** and the within-class scatter matrix **SW**.
3. Solve the generalized eigenvalue problem **SB** **W** = λ**SW** **W** to obtain the eigenvectors and eigenvalues.
4. Sort the eigenvectors in descending order based on their corresponding eigenvalues.
5. Select the top *k* eigenvectors to form the transformation matrix **W**.
6. Project the data onto the lower-dimensional space using **W**.

=== Applications ===

Linear Discriminant Analysis has various applications in different domains. Some of the notable applications include:

- Face Recognition: LDA has been widely used in face recognition systems to extract discriminative features from facial images and improve classification accuracy.
- Bioinformatics: LDA has been applied to analyze gene expression data and identify genes that are differentially expressed between different biological conditions.
- Finance: LDA has been used in financial markets to classify and predict stock price movements based on various financial indicators.

=== Conclusion ===

Linear Discriminant Analysis is a powerful technique for dimensionality reduction and classification. By finding a linear projection that maximizes the separation between classes, LDA can effectively classify new instances based on their feature values. Its applications span across various fields, making it a valuable tool in machine learning and pattern recognition.

== See Also ==
* [[Principal Component Analysis]]
* [[Support Vector Machines]]
* [[Naive Bayes Classifier]]

== References ==
* Fisher, R. A. (1936). "The Use of Multiple Measurements in Taxonomic Problems". Annals of Eugenics. 7 (2): 179–188.
{{dictionary-stub1}}

Linear discriminant analysis - Revision history

Prab: CSV import

Prab: CSV import