Kolmogorov-Smirnov Goodness-of-Fit Test

NIST/SEMATECH Section 1.3.5.16 Kolmogorov-Smirnov Goodness-of-Fit Test

What It Is

The Kolmogorov-Smirnov (K-S) test compares the empirical cumulative distribution function (ECDF) of a sample with a theoretical CDF. The test statistic is the maximum absolute difference between the two CDFs, making it a distribution-free test that does not require binning.

When to Use It

Use the Kolmogorov-Smirnov test for goodness-of-fit testing when you want to avoid the binning required by the chi-square test. It is applicable to any continuous distribution and is particularly useful when the sample size is small to moderate. The two-sample variant can also compare two empirical distributions directly. The K-S test is widely used as a quick screening tool for distributional fit.

How to Interpret

Compare the D_n statistic to the critical value from the Kolmogorov-Smirnov table at the chosen significance level. If D_n exceeds the critical value, reject the null hypothesis that the data follow the specified distribution. The K-S test has less power than the Anderson-Darling test because it weights all parts of the distribution equally, whereas the Anderson-Darling test gives more weight to the tails. The K-S test is most sensitive to differences near the center of the distribution. For composite hypotheses (parameters estimated from data), the standard critical values are conservative -- use the Lilliefors correction for normality testing.

Assumptions and Limitations

The K-S test requires a fully specified continuous null distribution. When parameters are estimated from the data, the standard critical values are no longer valid and modified tables (Lilliefors) must be used. The test is designed for continuous distributions; extensions to discrete and censored data exist but require modified critical values.

Reference: NIST/SEMATECH e-Handbook, Section 1.3.5.16

Formulas

K-S Statistic

D = \max_{1 \le i \le n} \left( F_0(x_{(i)}) - \frac{i-1}{n},\; \frac{i}{n} - F_0(x_{(i)}) \right)

The maximum distance between the empirical and hypothesized CDFs, computed at each ordered data point as the larger of the two one-sided distances. This accounts for the step-function nature of the empirical CDF.

Empirical CDF

F_n(x) = \frac{1}{n}\sum_{i=1}^{n} \mathbf{1}(x_i \leq x)

The empirical CDF is a step function that increases by 1/n at each data point, representing the proportion of observations less than or equal to x.

Python Example

import numpy as np
from scipy import stats

# Sample data
data = np.array([
    12.1, 11.5, 13.2, 12.8, 11.9, 12.4, 13.0, 12.6,
    11.7, 12.3, 13.5, 12.0, 11.8, 12.9, 12.2, 13.1
])

# K-S test against a normal distribution (parameters estimated)
mu, sigma = np.mean(data), np.std(data, ddof=1)
d_stat, p_value = stats.kstest(data, 'norm', args=(mu, sigma))

print(f"D-statistic: {d_stat:.4f}")
print(f"p-value:     {p_value:.6f}")
print(f"Normal at alpha=0.05: {p_value > 0.05}")

# Two-sample K-S test
sample_b = np.array([14.2, 13.8, 15.1, 14.5, 13.9, 14.7, 15.0, 14.3])
d2, p2 = stats.ks_2samp(data, sample_b)
print(f"\nTwo-sample D: {d2:.4f}, p={p2:.6f}")