Background
Many statistical methods rely on asymptotic results such as the Law of Large Numbers (LLN) and the Central Limit Theorem (CLT). These results provide the theoretical basis for sample averages, Normal approximations, confidence intervals, and many other forms of statistical inference.
However, their reliability depends on assumptions about the underlying data-generating distribution.
This project used simulation to investigate what happens when those assumptions are satisfied, weakened, or violated. I compared a well-behaved Poisson setting with heavy-tailed Pareto and Cauchy distributions, focusing on whether increasing sample size was sufficient to produce stable averages and approximately Normal sampling distributions.
Research Goals
- Empirically demonstrate LLN convergence under a well-behaved probability distribution.
- Examine what happens to sample averages when the underlying distribution lacks the moments required by classical asymptotic theory.
- Test the CLT approximation under Pareto distributions with different tail behaviour.
- Use simulation, distributional comparisons, and QQ analysis to identify where standard statistical assumptions become unreliable.
- Distinguish between cases where larger samples improve inference and cases where increasing sample size alone does not resolve the underlying problem.
Methodology
- Poisson Simulation: Generated repeated Poisson samples across increasing sample sizes and measured how frequently the sample mean fell within a fixed interval around the population mean.
- Cauchy Simulation: Repeated the sample-mean experiment using Cauchy-distributed observations to examine whether averages stabilised as sample size increased.
- Pareto Simulation: Generated repeated samples from Pareto distributions and studied the empirical distribution of their sample means.
- CLT Comparison: Compared simulated sample-mean distributions with the Normal approximation predicted by the classical CLT.
- QQ Analysis: Used quantile-quantile plots to identify deviations from Normal behaviour, particularly in the tails.
- Sensitivity to Tail Behaviour: Compared Pareto distributions with finite variance against a boundary case where variance is undefined.
- Tools: Python, NumPy, SciPy, matplotlib
Results
The simulations showed that sample size alone does not determine whether classical statistical approximations are reliable. The properties of the underlying distribution are equally important.
- Poisson: Sample means increasingly concentrated around the population mean as sample size grew, providing a clear empirical demonstration of the LLN.
- Cauchy: Sample means did not stabilise in the same way. Because the Cauchy distribution does not have a finite population mean or variance, averaging additional observations did not produce the concentration observed in the Poisson case.
- Pareto with finite variance: As sample size increased, the distribution of sample means became progressively closer to the Normal approximation predicted by the CLT. For the simulated Pareto setting with shape parameter 5, the central part of the approximation was already substantially improved by a sample size of approximately 30.
- Tail behaviour: QQ plots showed that convergence in the centre of the distribution could occur before the tails were well approximated, revealing discrepancies that would be easy to miss from a histogram or aggregate statistic alone.
- Pareto with shape parameter 2: The mean exists but the variance is undefined. Under this setting, the assumptions required for the classical finite-variance CLT are not satisfied, and the simulated sample means showed substantial departures from Normal behaviour.
The central result was therefore not simply that larger samples improve estimation, but that asymptotic guarantees depend critically on the distributional conditions under which they are invoked.
Contributions
- Built a simulation framework for comparing convergence behaviour across light- and heavy-tailed distributions.
- Empirically demonstrated the contrast between stable LLN behaviour for Poisson data and unstable sample averages under Cauchy data.
- Examined how Pareto tail behaviour affects the speed and validity of Normal approximations for sample means.
- Combined histograms, theoretical density comparisons, and QQ plots to distinguish central convergence from persistent tail discrepancies.
- Identified concrete cases in which commonly used statistical assumptions fail because required moments do not exist.
Impact
This project strengthened my interest in the reliability of quantitative models under uncertainty.
Statistical and machine-learning systems often produce estimates, forecasts, or uncertainty measures that are subsequently used to support decisions. Those outputs can appear precise while depending on assumptions that may not hold under unusual, heavy-tailed, or changing data conditions.
The simulations demonstrated a broader principle that became important in my later work: before relying on a model's output, it is necessary to understand the conditions under which its statistical guarantees remain valid and the conditions under which they can fail.