The Central Limit Theorem (CLT) is a fundamental concept in probability theory that describes the behavior of the sampling distribution of the mean of a large number of independent and identically distributed random variables. It states that, under certain conditions, the sampling distribution of the mean tends towards a normal distribution even if the original variables themselves are not normally distributed. This theorem is significant because it implies that probabilistic and statistical methods that work for normal distributions can be applicable to many problems involving other types of distributions.
The theorem has several variants, but its most common form requires that the random variables be independent and identically distributed (i.i.d.). The theorem has seen many changes during the formal development of probability theory. Previous versions of the theorem date back to 1811, but in its modern general form, this fundamental result in probability theory was precisely stated as late as 1920.
An elementary form of the theorem states that if we have a random sample of independent observations from a population with overall expected value (average) and finite variance, then the limit as n approaches infinity of the distribution of the standardized sample mean tends towards a standard normal distribution. In other words, if we obtain a large sample of observations, each observation being randomly produced in a way that does not depend on the values of the other observations, and compute the average (arithmetic mean) of the observed values, then if this procedure is performed many times, resulting in a collection of observed averages, the central limit theorem says that if the sample size was large enough, the probability distribution of these averages will closely approximate a normal distribution.