The law of large numbers: averages converge, individual outcomes do not

State the theorem with its assumptions

A familiar version considers independent, identically distributed variables with a finite expectation. Their sample average converges to that expectation in the appropriate law-of-large-numbers sense. For the elementary finite-variance argument used here, assume a common finite variance as well. The MIT lecture introduction explains convergence in probability: for any fixed positive tolerance, the chance that the average differs from the expectation by more than that tolerance tends to zero. This is an asymptotic statement, not a specified deadline at which every possible sample must become accurate.

Define a binary example without hidden machinery

Let each hypothetical trial return 1 with probability one half and 0 with probability one half, independently of the others. A single trial has mean one half and variance one quarter. The average of n trials is the fraction of ones, with variance 1/(4n). Its standard deviation is 1/(2√n). The next trial remains either zero or one; it does not become half of a success as n grows. Naming the average explicitly prevents a theorem about collective behaviour from being misread as a change in the individual outcome distribution.

Compare the scale of average fluctuations

Quadrupling the sample size halves this standard deviation. These values are not guaranteed error limits; they describe dispersion under the model. A particular average can move away from one half when one more observation is added. Convergence does not mean that every step improves, and it does not identify which finite sample path will look smooth.

Use a bound without pretending it is exact

The Carnegie Mellon lecture on deviation bounds derives a finite-variance bound through Chebyshev’s inequality. For this binary model, P(|average−0.5|≥ε) is at most 1/(4nε²), capped at one when the expression exceeds one. With n=10,000 and ε=0.05, the bound is 0.01. It is an upper bound, not the exact probability. The calculation illustrates how assumptions and a chosen tolerance produce a quantitative statement. A tighter distribution-specific calculation may give a smaller value, but it must be justified separately.

Explain why imbalance need not be repaid

Suppose the first ten trials happen to contain eight ones. Under independence, the next trial still has probability one half of being one. Future observations can dilute the early proportion without being biased toward zeros. If the next ninety trials contain exactly forty-five ones, the cumulative proportion becomes 53/100, closer to one half despite no corrective mechanism. That continuation is an illustrative possible record, not a promised outcome. The absolute excess count can remain nonzero, or grow, while its size relative to the growing sample becomes smaller.

Check whether the model remains applicable

Dependence, changing probabilities and selection effects can undermine a simple independent-sample argument. Repeating the same observation in a file does not add new independent information. Collecting only successful trials changes the sample being averaged. Penn State’s discussion of sample diversity explicitly warns against treating an independent process as self-correcting. In real research, record the collection mechanism and any changes over time. A large biased dataset can estimate the wrong target very precisely, so size alone is not a substitute for a suitable design.

Separate mathematical sample size from independent information

Copying one recorded outcome into a thousand spreadsheet rows produces a thousand entries without producing a thousand independent observations. Their average is still the original value, so the usual independent-sample variance reduction does not apply. The law-of-large-numbers argument depends on how observations are generated, not merely on the length of a file. Record the experimental or sampling unit and identify repeated measurements or duplicates. A growing dataset is informative only in the sense supported by its collection mechanism and the theorem’s assumptions.

Large-number interpretation checklist

• Identify the average and its target expectation.
• State independence and distribution assumptions.
• Distinguish individual outcomes from cumulative counts.
• Choose a tolerance before interpreting convergence.
• Separate bounds from exact probabilities.
• Do not assume step-by-step improvement or compensation.
• Check selection and changes in the process.

When demonstrating the idea with a simulation, display several sample paths rather than only a particularly smooth one. Keep the simulation separate from proof: it illustrates possible behaviour under the chosen model, while the theorem establishes a mathematical statement under its assumptions. Reporting both the average and the sample size helps readers understand why the same numerical deviation has different significance in a tiny and a very large sample.

Sources and verification

• Weak law of large numbers lecture introduction — MIT OpenCourseWare — 2026-10-23
• Deviation bounds and the law of large numbers — Carnegie Mellon University — 2026-10-23
• The diversity of samples, STAT 100 — Penn State — 2026-10-23