Why is mathematics important when two datasets have similar averages but different shapes? Because a distribution contains much more than its mean. The Epps–Singleton two-sample test compares information from empirical characteristic functions: averages of sine and cosine values calculated from observations. It offers a striking connection between statistics, trigonometry and the idea of a frequency signature.
Think of two collections of waiting times. One is concentrated in a narrow band. Another mixes very short and very long waits. Their averages could match while the experience is quite different. Asking whether the underlying distributions are the same is broader than asking whether their means are equal, their medians match or one observation tends to exceed another.
This article develops that broader question using original numerical examples. It explains what the Epps–Singleton test computes, why covariance matters, how discrete observations fit the method, and why a finite signature cannot prove two populations identical. The examples are educational scenarios, not claims about actual services, schools or products.
Choose your reading route
Start with the question, follow the mechanism, or jump straight to the checked calculation. All teaching sections remain open below.
What distribution equality actually asks · From a value to a point on the unit circle · A discrete worked dataset · Small samples and singularity · Practice and checking answers
Full section index · eduKateSG Mathematics
Full section index
Sections 1–6
Sections 7–12
Sections 13–18
Did You Know? A characteristic function exists for every real-valued probability distribution, including distributions whose mean is not finite. That does not mean every practical test built from a few characteristic-function values is automatically reliable. Existence of a mathematical object and calibration of a finite-sample procedure are separate achievements.
What distribution equality actually asks
For two independent populations with distribution functions F and G, the null is F=G. That means the probabilities assigned to all relevant ranges agree, not merely that a few summary measures match. The alternative is that the populations differ somewhere in their distribution.
Under this null, a different centre, spread, skewness, tail pattern or pattern of discrete probabilities can count as a departure. The test does not automatically identify which feature changed. That diagnosis requires plots, summaries, domain knowledge and sometimes a more targeted follow-up.
Do not confuse this question with “are the samples identical?” Two independent samples from the same population usually contain different values. A statistical test asks whether the differences between their summaries are unusually large relative to sampling variation under the null.
Nor does failure to reject establish equality. A modest sample may miss an important difference; a particular test may have little sensitivity to a chosen alternative. If practical similarity is the objective, specify a tolerance and an equivalence framework instead of turning a large p-value into a certificate of sameness.
From a value to a point on the unit circle
Choose a frequency t. Transform an observation x into exp(itx)=cos(tx)+i sin(tx). In geometric terms, tx is an angle and the transformed observation is a point on the unit circle. The characteristic function φ(t) is the population average of these complex points.
For observed values x₁,…,x_m, the empirical version is φ̂_X(t)=(1/m)Σ exp(itx_j). Its real component is the mean cosine and its imaginary component is the mean sine. Each individual component is bounded between −1 and 1, even if the original measurements have extreme values.
This is not a time-series Fourier transform of an ordered recording. The input order is irrelevant: shuffling observations leaves the average unchanged. The transformation summarises the empirical distribution, not the sequence in which measurements were taken.
The frequency controls how quickly the circle wraps as x changes. Small t values move nearby observations through small angles; larger t values rotate more rapidly. Different frequencies therefore highlight different features, much as different questions can reveal different aspects of the same dataset.
Back to contents · Previous · Next
A hand calculation before the full test
Take X=[0,0] and Y=[−1,1], and evaluate at t=π/2. Every X observation maps to 1+0i, so φ̂_X(t)=1. The Y observations map to −i and +i, whose average is zero. The empirical signatures differ even though both sample means are zero.
At t=2π, however, all four values map to 1. The signatures agree at this frequency. That does not make the distributions equal; it shows that one frequency can miss a difference completely because of the periodic transformation.
The example is intentionally tiny and is not a valid request to run SciPy’s Epps–Singleton interface, which requires at least five observations in each sample. It is a picture of the mechanism. Keeping conceptual demonstrations separate from inferentially supported sample sizes prevents a common teaching error.
A learner can draw the unit-circle points and their vector average. When points cluster in one direction, the average has a large magnitude; when they cancel, the magnitude is smaller. Both direction and magnitude can differ between distributions, so retaining sine as well as cosine information matters.
Back to contents · Previous · Next
Why more than one frequency helps
The full characteristic function uniquely identifies a probability distribution. But a computer cannot inspect every real frequency. The practical Epps–Singleton construction selects a finite set and compares the resulting real and imaginary components.
With two frequencies t₁ and t₂, each sample produces a four-component vector: mean cos(t₁x), mean sin(t₁x), mean cos(t₂x), mean sin(t₂x). The difference between the two vectors is the evidence vector d. Under distribution equality, d should be close to zero in a sampling-variability sense.
SciPy’s maintained implementation uses default positive frequency parameters (0.4,0.8), scaled through the pooled semi-interquartile range. These defaults are an implementation choice informed by the method’s research history, not a theorem that those two numbers are optimal for every population.
More frequencies are not automatically better. Each adds components, covariance parameters and potential redundancy. A small dataset may not support a stable high-dimensional comparison. A well-motivated finite signature is preferable to an unexamined collection of frequencies chosen after looking at the smallest p-value.
Back to contents · Previous · Next
Covariance gives the signature a fair scale
The components of d are not independent. Sine and cosine values come from the same observations, and two frequencies can respond to the same pattern. Their sampling variations can also have different magnitudes. Simply adding squared raw component differences ignores both facts.
A covariance matrix records those relationships. If g_t(x) is the vector of selected sine and cosine features, the covariance of the difference of sample means is estimated from the two groups’ feature covariances and sample sizes. Schematically, V̂=S_X/m+S_Y/n.
The central quadratic form is dᵀV̂⁺d, where V̂⁺ denotes an appropriate generalised inverse if needed. It is a multivariate version of dividing a difference by its uncertainty. A direction in feature space that naturally varies greatly should not count as strongly as an equally sized difference in a very stable direction.
The maintained implementation includes details of scaling, covariance estimation, effective rank and small-sample correction. Those details matter when reproducing the exact software output. The schematic formula teaches the logic; the tested implementation and its documented version define the numerical procedure being reported.
Back to contents · Previous · Next
A discrete worked dataset
Let X contain the integers 0,1,2,3,4,5 repeated five times, giving m=30. Let Y contain 0,1,2,3,4,5,6,7,8 repeated four times, giving n=36. Repeated values are genuine discrete observations here, not a reason to insert artificial decimals.
For an independent illustration of the empirical characteristic function at the unscaled frequency t=0.5, X has mean cosine about 0.21188860 and mean sine about 0.63769351. Y has mean cosine about −0.14541791 and mean sine about 0.31774393. These four numbers can be reproduced by a spreadsheet with cosine and sine columns.
Those t=0.5 values are not the complete test input. They illustrate one chosen frequency, whereas the test below uses SciPy’s default two-frequency parameters and its pooled-scale adjustment. Reporting this distinction avoids pretending that an intermediate diagram numerically explains every part of a multivariate statistic.
Running the standard Epps–Singleton procedure on the two full samples gives statistic approximately 14.98487072 and p≈0.00473270 in SciPy 1.17.0. In this example the effective reference dimension is four. At a predeclared 5% threshold the result rejects common distribution membership under the method’s calibration.
The outcome is unsurprising: Y assigns probability to 6,7,8 while X never does in these samples, and their centres and spreads also differ. The p-value does not say which of those features is responsible, nor establish that a particular real-world cause created the difference.
Back to contents · Previous · Next
Why discrete data are a useful application
Many observations are counts: errors per item, arrivals per interval, number of attempts or scores on a short assessment. A continuous-distribution assumption can be awkward when ties are part of the actual population. Epps–Singleton does not require the underlying distributions to be continuous.
That is a meaningful distinction from the familiar continuous-null calibration of a Kolmogorov–Smirnov comparison. It is not a licence to call any implementation of any test invalid on discrete data. Alternative calibrations and discrete-specific methods exist; the scientific question and actual software procedure must be stated.
Discrete support can create another difficulty: feature vectors may be linearly dependent. If there are very few distinct outcomes, the covariance matrix can be rank deficient. That affects the effective comparison dimension and the trustworthiness of numerical inversion.
Always report the number of distinct values and inspect warnings. A function returning a number is not proof that the data support every component of a default frequency signature. With a tiny number of categories, a directly modelled contingency-table comparison may be easier to interpret.
Back to contents · Previous · Next
Scaling is part of the mathematical model
A frequency has reciprocal units to the measurement so that tx is dimensionless. If x is measured in seconds, t is interpreted in inverse seconds. Converting seconds to milliseconds without adjusting frequency changes the angles and therefore the signature.
The pooled semi-interquartile scaling in SciPy’s implementation helps make its default frequency parameters meaningful across numerical scales. The semi-interquartile range is half the difference between the upper and lower quartiles. It is a robust scale summary, not a guarantee that every dataset has a usable nonzero value.
If the pooled interquartile range is zero, blindly dividing by it is not meaningful. This can happen with many tied observations even when some other values differ. A specialist alternative, a justified frequency scale or a different test may be necessary; do not suppress the warning and publish the result.
A student exercise is to multiply every observation by ten and compare both the raw-frequency signature and the scale-adjusted test. Explain which calculation should change and why. This connects dimensional reasoning from school mathematics with a sophisticated statistical implementation.
Back to contents · Previous · Next
Translation and rotation of the signature
Adding the same constant c to every observation multiplies each characteristic-function value by exp(itc). At one frequency, that rotates the average point without changing its magnitude. The relation follows directly from exp(it(x+c))=exp(itc)exp(itx).
This is a useful algebra check, but avoid concluding from it that every simplified ad hoc statistic is translation invariant. A properly transformed covariance-weighted comparison has to track the corresponding rotation of all components as well.
If only one group is shifted, the relative signatures generally change. That is appropriate because distribution equality includes location equality. Subtracting each group’s own mean before testing would remove some location information and would therefore change the hypothesis being assessed.
Preprocessing is not neutral housekeeping. Centring, scaling, trimming and rounding can each alter which distribution is being compared. List the steps before analysis and explain whether they are part of the scientific target or merely an implementation convention.
Back to contents · Previous · Next
What the p-value does and does not mean
The reported p-value comes from an asymptotic chi-square reference, with implementation-specific adjustments. It describes how unusual the statistic would be under the null calibration. It is not the probability that the distributions are equal or the percentage of observations that came from the same source.
A small p-value can occur with a tiny but systematic difference in a large sample. Conversely, a scientifically important difference may be missed by a small sample. Always accompany a test with interpretable summaries: quantiles, support, tail frequencies, spread and plots suited to the domain.
If p is large, do not write that the datasets are interchangeable. Interchangeability may require tolerances on errors, safety, fairness or performance that this omnibus test never specifies. A formal equivalence analysis begins with those tolerances, not with a convenient non-significant result.
The term “omnibus” describes the broad distributional question. It does not mean a finite-frequency procedure detects every possible difference at every sample size. A selected signature can be weak against some alternatives, especially if their differences occur in directions the chosen frequencies barely register.
Back to contents · Previous · Next
Small samples and singularity
SciPy requires at least five observations per sample, but a minimum accepted input is not a recommendation that five observations provide reliable inference in every situation. Current documentation recommends greater caution for small continuous samples and describes a correction when both sample sizes are below 25.
An asymptotic reference is a large-sample approximation. Its quality depends on more than the total number of rows: support size, feature covariance and imbalance all matter. A severe warning should lead to diagnosis, not an automatic switch to whichever setting returns the desired conclusion.
Singularity means that some feature directions are redundant. A generalised inverse can handle mathematical redundancy, but numerical near-singularity can make a result sensitive to tiny changes. Record the rank, warnings and sensitivity to defensible settings when those issues arise.
For a classroom project, compare the default method with a transparent empirical distribution plot and a simulation under a known null. If the p-values are badly calibrated in that simulation, investigate before treating the procedure as evidence. Simulation is an audit of the chosen scenario, not a universal validation certificate.
Back to contents · Previous · Next
An investigation about equal means
Construct population A with equal probabilities at −1 and +1. Construct population B with equal probabilities at −2 and +2. Both have mean zero, but B has larger spread. At t=π/2 the cosine average for A is zero and for B is −1, while the sine averages cancel in both.
Now choose t=π. A’s cosine average is −1 and B’s is +1. The frequency changes the contrast, but both reveal a difference that a mean-only comparison misses. Draw the four support points on the unit circle to make the cancellation visible.
Generate independent samples from these populations rather than copying deterministic repetitions and pretending they are a random experiment. The deterministic support calculation describes population features; a sampling simulation studies the distribution of estimated features. Both are valuable, but they answer different questions.
Ask students to design a frequency at which the two signatures coincide. Finding one is a productive failure case: it demonstrates why matching one trigonometric summary does not establish equality. The activity develops algebraic reasoning as well as statistical caution.
Back to contents · Previous · Next
An investigation about data drift
Suppose a hypothetical service records requests per minute in two periods. A distribution comparison might reveal a change in the count pattern. Before applying an independent-samples method, however, examine temporal dependence: neighbouring minutes may be correlated or share a daily cycle.
Pooling all morning and evening observations without accounting for time composition can produce a difference even if the service behaves identically at each corresponding time. Stratification, matched periods or a time-series model may be needed to answer the operational question.
If the test flags drift, use descriptive follow-up to identify whether it concerns centre, variability, zero frequency or extreme counts. The test is an alarm about a distributional contrast, not a diagnosis of the system component responsible.
A responsible monitoring plan also addresses repeated testing. Running a fixed 5% test every minute does not preserve a 5% chance of ever raising a false alarm. Sequential procedures, control charts or an explicitly managed alert policy are separate mathematical work.
Back to contents · Previous · Next
An investigation about educational data
Two independent sets of rubric scores might have similar averages but different proportions at the lowest and highest bands. An empirical characteristic-function comparison can illustrate why distribution shape matters. The practical response should still focus on learning needs rather than declaring one class superior.
Check that the same rubric and marking rules apply. If one group receives a harder task or a different scoring scale, the observed distributions are not directly comparable as measures of the same thing. More sophisticated inference cannot repair a measurement mismatch.
Protect privacy and avoid publishing small-cell distributions that identify pupils. Synthetic examples are usually sufficient for learning the mechanism. Real analysis needs appropriate consent, access controls and a clear purpose beyond ranking children.
The importance of mathematics in education includes this restraint. Numeracy is not merely knowing how to calculate a result; it is recognising when a comparison is meaningful, when the data are insufficient and when a conclusion would exceed the evidence.
Back to contents · Previous · Next
A reproducible computation plan
Keep the original observations and define the unit of analysis. Record sample sizes, missing values, distinct-value counts and any exclusions. Separate raw data from transformed feature columns so every step can be inspected.
For a hand-check frequency, calculate tx, cos(tx) and sin(tx), explicitly using radians. Spreadsheet trigonometric functions generally expect radians. Accidentally feeding degrees can produce plausible-looking but entirely different signatures.
Next reproduce the maintained test using its documented defaults and version. Do not replace the covariance-weighted statistic with a sum of squared component differences and continue calling it Epps–Singleton. If writing an educational implementation, compare its output against trusted software across several manufactured cases.
Finally write a result paragraph that names the distribution-equality question, the independent-sample design, statistic, calibration and limitations. Save code, input and parameters together. A future reader should not need to guess whether a scale adjustment or small-sample correction was active.
Back to contents · Previous · Next
A learning route for students
Start with averages of vectors. Draw points on a circle and calculate their mean coordinates. Then connect coordinates to sine and cosine. This makes the complex notation a compact representation of familiar geometry rather than an intimidating new object.
Move to empirical distributions: list how often each observed value occurs and show that the empirical characteristic function is a weighted average of unit-circle points. Reordering rows does not change it, which helps distinguish a distributional summary from sequence analysis.
Study covariance next. Two features can move together, so four summaries do not necessarily contain four independent pieces of information. A small scatterplot of feature values can build intuition before matrices and generalised inverses appear.
Only after these foundations introduce the quadratic test statistic and chi-square approximation. Students can understand why the mechanism matters without memorising every implementation detail. The aim is transferable problem-solving: represent, calculate, validate and state the boundary.
Back to contents · Previous · Next
Guidance for parents and educators
Do not begin by asking a learner to memorise “empirical characteristic function”. Ask whether two datasets with the same average must tell the same story. Once they produce a counterexample, the need for a richer representation becomes natural.
Encourage an explainable diagram beside every formula. A unit circle, a count table and a small covariance sketch each answer a different part of the question. A student who can move between those representations is building the connections that advanced mathematics depends on.
Avoid using specialist enrichment as a label of ability or a promise of future employment. Statistics connects to science, computing, quality analysis and research, but those fields also require communication, domain expertise and ethical judgement. Keep pathways open.
Praise a discovered limitation as well as a correct calculation. Finding a frequency that misses a difference, identifying zero scale or spotting dependent observations is evidence of understanding. Good mathematics includes knowing when a method should not be used.
Back to contents · Previous · Next
Practice and checking answers
Check 1: At t=0, every exp(itx)=1. Therefore every empirical characteristic function equals one there. That frequency supplies no distributional contrast, which explains why useful test frequencies are positive and nonzero.
Check 2: If all observations are zero, the empirical signature is 1+0i at every frequency. A default scale-normalised procedure may be undefined because its pooled scale is zero. The descriptive function exists even when the test calibration fails.
Check 3: Does matching the mean imply matching the characteristic function? No. The symmetric ±1 and ±2 populations have the same mean but different trigonometric averages at many frequencies.
Check 4: Does matching two chosen frequencies prove equality? No. A finite set of summaries can miss other differences. The uniqueness theorem concerns the entire characteristic function, not two isolated evaluations.
Check 5: Should observations be sorted before calculating the empirical characteristic function? Sorting is unnecessary because addition is order-independent. If sorting changes your result beyond rounding effects, the implementation likely contains an error.
Back to contents · Previous · Next
Three failure cases that make the method clearer
Equal frequencies are redundant, not extra evidence
Suppose a learner evaluates t=0.4 twice and treats the resulting components as two separate views of the data. The extra columns duplicate information. Their covariance matrix is singular because one feature is an exact copy of another, and no genuine inferential dimension has been added.
This explains the documented requirement for distinct positive frequency parameters. Repeating a measurement-derived summary is not equivalent to collecting an independent observation. The same lesson appears elsewhere in statistics: adding a duplicated spreadsheet column can make a table larger without making its evidence richer. Test the calculator with a deliberately repeated frequency and verify that the redundancy is detected rather than rewarded.
A phase plot can conceal magnitude
Two empirical characteristic functions may point in similar directions while having different lengths. A plot showing only their angles can hide that difference. Conversely, equal lengths can conceal different directions. The real and imaginary components together carry the information used by the feature comparison.
Ask students to draw vectors (1,0), (0,1) and (0.5,0). The first two have equal length but different direction; the first and third share direction but differ in length. Neither a magnitude-only nor an angle-only summary fully describes these vectors. This simple geometry helps explain why a complex-valued representation is more than a decorative notation for a single number.
A large sample can expose an irrelevant difference
Imagine a synthetic manufacturing process whose output distribution changes by an amount far smaller than the relevant engineering tolerance. With enough independent observations, an omnibus equality test may detect that difference. The result can be statistically convincing and operationally unimportant at the same time.
Before data collection, define what would matter: a tolerance violation probability, a quantile shift or another domain-specific quantity. Use that target alongside the distribution test. Do not choose a practical threshold after seeing the p-value simply to dismiss an inconvenient finding. A good decision combines evidence about difference with a declared standard of consequence.
Back to contents · Previous · Next
A project connecting trigonometry and statistics
Create two synthetic discrete distributions on a short integer support. Draw their probability bars, then calculate the population cosine and sine averages at five chosen frequencies. Label these as population calculations because the probabilities are known by construction.
Next generate independent samples and calculate empirical signatures. On one diagram, mark the known population vector and several sampled estimates at the same frequency. The cloud of estimates illustrates sampling uncertainty without requiring a complicated test formula first.
Repeat with larger sample sizes. The empirical vectors should generally cluster more tightly, though individual runs need not improve monotonically. Record the random seed and generation rule so a classmate can reproduce the demonstration. Do not select only the prettiest run as though it represented every possible sample.
Finally compare two frequency choices. One may reveal the designed difference more strongly than another. Explain this as sensitivity to a particular alternative, not a universal ranking of frequencies. The project succeeds when a student can connect the bar chart, unit-circle geometry and covariance calculation as views of one probabilistic object.
Back to contents · Previous · Next
Questions to ask before a distribution comparison
- Are the observations independent within and between the two samples?
- Do both datasets measure the same quantity in the same units?
- Was the population composition comparable, or did sampling mix change?
- Is the pooled scale positive and meaningful for the selected implementation?
- How many distinct outcomes exist, and do the features have enough rank?
- Were frequency choices and preprocessing rules fixed before examining significance?
- What practical summaries would explain a detected difference?
- Is the intended conclusion equality, equivalence, drift detection or causal change?
These questions prevent a broad test from becoming a substitute for a clear study. A distribution can differ because of selection, measurement, time composition or a genuine process change. The statistic responds to the observed contrast but does not identify which explanation is correct.
For parents and students reading a research headline, this is a useful habit even without performing the calculation. Ask what population was sampled, what feature the test compared and whether the conclusion says more than that feature supports. Mathematical literacy includes evaluating the route from data to claim.
Back to contents · Previous · Next
Frequently asked questions
Is this a normality test?
Not in its two-sample form. It asks whether two underlying distributions agree, without specifying that either is normal. Other procedures use characteristic functions for normality, but sharing a representation does not make the hypotheses identical.
Can it compare count data?
Yes, continuity of the population distribution is not required. Still inspect support size, ties, scale and covariance rank. A documented implementation’s acceptance of the input does not guarantee good calibration for every sparse count pattern.
Why not just compare histograms?
Histograms are useful diagnostics but depend on bin boundaries and do not automatically supply an inferential calibration. The test provides a formal comparison of selected features; the plot helps explain what the difference may mean. They complement one another.
Does a smaller p-value mean a larger difference?
Not necessarily. Sample size and variability affect p-values. Report interpretable distributional contrasts as well, such as quantile differences or changes in meaningful tail probabilities, rather than treating the p-value as an effect size.
Can I tune the frequencies until the result is significant?
No. Searching settings and reporting only the most favourable result changes the error rate. Predefine the rule or treat the search as exploratory and validate it independently with a suitable inferential plan.
What is the wider mathematical lesson?
A distribution can be represented through transformed averages, and covariance turns several related contrasts into a coherent comparison. The method connects trigonometry, linear algebra and probability while reminding us that every finite representation has blind spots.
Back to contents · Previous · Next
Keep the original measurements in view
A frequency signature is a representation of data, not a replacement for the underlying record. Preserve the observations, units, collection dates where relevant and measurement definitions so any surprising result can be traced back to its source.
If a signature comparison changes after a software update, inspect documented implementation changes and recompute a small manufactured case before attributing the change to the real world. A computational difference and a population difference are separate explanations.
The same discipline applies when comparing exports from two systems. A changed rounding rule, missing-value code or unit conversion can alter the empirical distribution without any substantive process change. Checking the data contract is often the most valuable first mathematical action, even when a sophisticated test is available.
Back to contents · Previous · Next
Useful next reading and primary references
Epps and Singleton introduced the method in their 1986 paper, An omnibus test for the two-sample problem using the empirical characteristic function. For current parameters, scaling, sample-size cautions and implementation behaviour, consult SciPy’s Epps–Singleton documentation.
For a contrast based on an empirical distribution function rather than a frequency signature, see Anderson–Darling tail weighting and goodness-of-fit. For a narrower spread question, read Fligner–Killeen robust variance equality.
The eduKateSG Mathematics hub provides the wider route. Why mathematics? Because it lets us choose what a comparison preserves, understand what its representation misses, and turn a surprising output into an explanation we can check.