A standard error estimates the standard deviation of an estimator’s sampling distribution. It describes how much an estimate—such as a mean, proportion, difference, regression coefficient or treatment effect—would typically vary across repeated samples or assignments generated under the same design and model. Smaller standard errors mean greater sampling precision; larger standard errors mean greater sampling instability. Standard errors are therefore not measures of how spread out individual observations are, and they do not automatically include bias from bad measurement, confounding, selection or model misspecification.
A dataset gives us an estimate.
The standard error asks how shaky that estimate is.
Not emotionally shaky.
Sampling shaky.
The governing question: if the same estimator were applied to another sample produced by the same information-generating process, how far would the estimate normally move?
NIST’s statistical glossary defines standard error as the standard deviation of a statistic’s sampling distribution. That definition is short enough to memorise and deep enough to organise most of what follows.
Quick Read
ESTIMAND → ESTIMATOR → SAMPLING / ASSIGNMENT DESIGN → SAMPLING DISTRIBUTION → VARIANCE OF ESTIMATOR → STANDARD ERROR → t / z / WALD STATISTIC → CONFIDENCE INTERVAL / TEST → DESIGN / DEPENDENCE / MODEL CHECK → DECISION
1. Standard Error Belongs to an Estimator
The mean has a standard error.
A proportion has a standard error.
A regression coefficient has a standard error.
A median can have a standard error.
A treatment-effect estimator has a standard error.
The formula depends on the estimator and design.
2. Standard Error Is Not Standard Deviation
Standard deviation describes spread among observations.
Standard error describes spread among hypothetical repeated estimates.
If student scores have SD = 15, that tells us how dispersed students are.
If the class mean has SE = 2, that tells us how much the estimated mean would vary across repeated comparable samples.
3. The Standard Error of an IID Sample Mean Is σ/√n
For independent observations with population standard deviation σ, the sample mean has standard error:
SE(ȳ) = σ/√n
In practice σ is usually unknown and replaced by the sample standard deviation s, producing an estimated standard error s/√n.
4. The Square-Root Law Makes Precision Expensive
Double sample size and standard error shrinks by only √2.
Quadruple sample size and standard error halves.
To reduce standard error by a factor of ten, roughly one hundred times the independent information is needed.
5. Standard Error Is a Measure of Precision, Not Accuracy
A scale can be miscalibrated by 2 kg and repeat almost perfectly.
The mean weight estimate can have a tiny standard error.
It can still be biased by 2 kg.
Precision means repeatability of the estimator under its assumed design, not closeness to the scientific truth.
6. Standard Error Is an Estimated Quantity
The true sampling variance of an estimator is usually unknown.
We estimate it from observed data, a model, the randomisation design or resampling.
The reported standard error therefore has uncertainty of its own.
Small samples can make standard-error estimates especially unstable.
7. Student’s t Distribution Accounts for Estimating the Standard Error
For a normal population with unknown variance, replacing σ with s adds uncertainty.
The standardised sample mean follows a t distribution with n−1 degrees of freedom rather than an exact normal distribution.
Small-sample confidence intervals are wider because the standard error itself had to be estimated.
8. Standard Error Connects Directly to Test Statistics
A common test statistic has the form:
(estimate − null value) / standard error
This asks how many estimated sampling-standard-deviation units the observed estimate lies from the null target.
The reference distribution determines how unusual that distance is.
9. Standard Error Connects Directly to Confidence Intervals
In many regular problems, an approximate interval is:
estimate ± critical value × SE
The critical value comes from a normal, t or other reference distribution.
See How Confidence Intervals Work.
10. A Smaller Standard Error Narrows Many Intervals
If the critical value stays similar, halving the standard error roughly halves the width of a Wald-style confidence interval.
That is why sample size, design efficiency and measurement precision matter directly to inferential resolution.
11. Statistical Significance Often Changes When Standard Error Changes
Hold the effect estimate fixed.
Make the standard error smaller.
The test statistic grows in magnitude and the p-value often falls.
Significance therefore depends on effect magnitude relative to uncertainty, not effect magnitude alone.
12. Large Samples Can Make Tiny Effects Highly Significant
If n is enormous, standard errors can become very small.
A scientifically trivial effect can then sit many standard errors from zero.
This is why standard error must be interpreted beside effect size and practical thresholds.
13. Small Standard Error Does Not Mean the Model Is Right
A misspecified regression can estimate the wrong linear summary with extraordinary precision.
Its standard error can be tiny because the sample is huge.
The model specification can still be scientifically wrong.
Precision operates inside the model’s representation.
14. Standard Errors for Proportions Depend on p
For an independent Bernoulli sample, the ordinary plug-in standard error of a proportion is:
√(p̂(1−p̂)/n)
It is largest near p = 0.5 and smaller near 0 or 1.
Near boundaries and with rare events, normal approximations can still be poor even when this estimated SE is small.
15. Differences of Independent Estimates Add Variances, Not Standard Errors
If two independent estimates have standard errors SE₁ and SE₂, the variance of their difference is the sum of variances:
SE(diff) = √(SE₁² + SE₂²)
Simply adding standard errors is incorrect.
16. Covariance Matters When Estimates Are Not Independent
For two dependent estimators A and B:
Var(A−B) = Var(A) + Var(B) − 2Cov(A,B)
Pairing, repeated measures and shared controls create covariance that can materially change precision.
17. Paired Designs Can Reduce Standard Error Dramatically
Measure the same student before and after an intervention.
If students who start high also finish high, the two measurements are positively correlated.
Analysing within-student changes removes much stable person-to-person variation.
The standard error of the average change can be much smaller than an unpaired comparison.
18. Regression Standard Errors Come From Residual Variation and Predictor Geometry
In ordinary least squares, coefficient uncertainty depends on residual variance and the inverse of X′X.
More residual noise widens standard errors.
More informative predictor spread narrows them.
Collinearity widens them.
19. Multicollinearity Can Make Individual Coefficients Imprecise While Predictions Stay Good
Two predictors carry nearly the same information.
The model can predict Y from their combination.
But many different coefficient pairs produce similar predictions.
Individual coefficient standard errors become large because the model cannot cleanly allocate the shared signal.
20. A High Variance Inflation Factor Is a Precision Warning
Variance inflation factors quantify how much linear predictability from other covariates inflates a coefficient’s variance relative to an orthogonal design.
There is no universal magic VIF cutoff that determines whether the model is scientifically valid.
The consequence depends on whether separate coefficient interpretation matters.
21. Homoskedasticity Is a Variance Assumption Behind Classical OLS Standard Errors
Classical linear-model standard errors assume residual variance is constant conditional on predictors.
If residual spread grows with fitted values, the ordinary coefficient estimates can remain unbiased under exogeneity while the usual variance formula becomes wrong.
22. Heteroskedasticity-Robust Standard Errors Change the Variance Formula
Huber–White or sandwich estimators allow conditional error variance to vary across observations under suitable independence and large-sample conditions.
The coefficient estimates remain the same as OLS.
The estimated covariance matrix changes.
“Robust” here means robust to heteroskedastic variance misspecification—not robust to everything.
23. HC0, HC1, HC2 and HC3 Are Different Robust-SE Corrections
HC0 is the basic sandwich estimator.
HC1 applies a degrees-of-freedom scaling.
HC2 and HC3 adjust more strongly for leverage and can perform better in smaller samples.
Software defaults differ, so “robust SE” should be specified precisely.
24. Cluster-Robust Standard Errors Address Within-Cluster Dependence
Students within one school share conditions.
Repeated observations within one patient share biology.
Cluster-robust covariance estimators allow arbitrary dependence within clusters under stated asymptotic conditions while treating clusters as the independent units.
The effective information is driven more by cluster count than row count.
25. Five Thousand Students in Ten Schools Are Not Ten Thousand Independent Facts
A large number of rows can conceal a small number of independent clusters.
Cluster-robust asymptotics can perform poorly with very few clusters.
Small-cluster corrections, randomisation inference, multilevel models or cluster-level analyses may be needed.
26. Two-Way Clustering Handles More Than One Dependence Dimension
Financial returns can be correlated within firms over time and across firms within the same date.
Two-way cluster-robust estimators account for dependence along both dimensions under appropriate asymptotics.
Dependence structure belongs in the standard-error design.
27. Serial Correlation Makes Ordinary Time-Series Standard Errors Too Small
If today’s residual resembles yesterday’s residual, observations contain less independent information than IID formulas assume.
Newey–West and other heteroskedasticity-and-autocorrelation-consistent estimators adjust covariance for specified lag dependence.
Lag choice becomes another modelling decision.
28. Survey Standard Errors Must Respect Weights, Strata and Clusters
Complex probability surveys are not IID row samples.
Variance estimators use Taylor linearisation, replicate weights, jackknife, bootstrap or balanced repeated replication to reproduce the survey design.
Running an ordinary unweighted SE formula on survey rows can produce false precision.
29. Finite-Population Corrections Reduce Standard Error for Large Sampling Fractions
If a simple random sample without replacement includes a large fraction of a finite population, the remaining uncertainty is smaller than with sampling from an effectively infinite population.
The finite-population correction reduces the variance accordingly.
30. Repeated Measures Need Subject-Level Dependence
Ten measurements from one student do not supply the same information as measurements from ten independent students.
Mixed models, generalised estimating equations, subject-level clustering or appropriate repeated-measures covariance models account for within-person dependence.
31. Multilevel Models Estimate Dependence Rather Than Only Correcting the SE
Cluster-robust SEs modify the uncertainty around a marginal or conditional coefficient while leaving much of the mean model unchanged.
Hierarchical models explicitly model group-level variation and dependence through varying effects.
The next canonical owner is How Hierarchical Models Work.
32. Generalised Linear Models Use Model-Based Information for Standard Errors
Logistic, Poisson and other likelihood-based regressions estimate covariance from the curvature of the log-likelihood or related information matrices.
Under correct specification and regularity, inverse Fisher information approximates estimator variance.
Misspecification motivates sandwich alternatives.
33. Fisher Information Is Inverse Precision Geometry
A sharply curved likelihood changes quickly when the parameter moves.
The data strongly distinguish nearby parameter values.
Information is high and the standard error tends to be small.
A flat likelihood means weak distinction and larger standard error.
See How Likelihood Works.
34. Wald Standard Errors Are Local Approximations
Near the maximum-likelihood estimate, approximate the log-likelihood by a quadratic.
The inverse curvature produces a variance estimate.
This works well in regular large samples and can work badly near boundaries, with weak identification or severe asymmetry.
35. Profile-Likelihood Intervals Can Reveal What One SE Hides
A symmetric estimate ± 1.96SE interval assumes locally symmetric uncertainty on the chosen scale.
If the likelihood is strongly skewed, profile-likelihood intervals can be asymmetric and more faithful to the support geometry.
A standard error is a useful compression, not the entire uncertainty surface.
36. The Delta Method Propagates Standard Errors Through Smooth Transformations
If θ̂ is approximately normal and g is smooth, then:
Var(g(θ̂)) ≈ [g′(θ)]² Var(θ̂)
This supports approximate standard errors for odds ratios, marginal effects and nonlinear combinations of estimates.
37. Covariance Matrices Let Standard Errors Travel Through Multivariable Functions
When a quantity depends on several correlated estimated parameters, variances alone are insufficient.
The full covariance matrix tells us how uncertainties move together.
Gradient-based delta methods then propagate that joint uncertainty into derived estimates.
38. Bootstrap Standard Errors Replace an Analytic Formula With Resampling
Generate many bootstrap samples.
Recompute the estimator each time.
Take the standard deviation of the bootstrap estimates.
That value estimates the estimator’s standard error under the bootstrap approximation.
See How Bootstrap Works.
39. Bootstrap Standard Errors Still Need the Right Resampling Unit
Resampling individual rows from clustered data produces the wrong standard error.
Time series may need blocks.
Matched designs may need pair resampling.
Survey data may need replicate weights.
Resampling must imitate the actual information structure.
40. Jackknife Standard Errors Use Leave-One-Out Perturbations
Delete one observational unit at a time.
Recompute the estimator.
The variability among those leave-one-out estimates approximates sampling variance for many smooth estimators.
Jackknife also reveals influential observations.
41. Randomisation-Based Standard Errors Come From Assignment Mechanisms
In designed experiments, uncertainty can be defined over possible treatment assignments rather than population sampling.
Neyman-style randomisation inference produces variance estimators for differences in treatment-group means under the known assignment design.
The source of randomness should be stated explicitly.
42. Model-Based and Design-Based Standard Errors Can Answer Different Questions
A model-based SE conditions on a probability model for outcomes.
A design-based SE may derive uncertainty from random sampling or random assignment while treating observed values as fixed finite-population quantities.
The numerical answers can be similar while the inferential interpretation differs.
43. Missing Data Can Make the Reported Standard Error Too Small
Single imputation fills missing values and then often treats them as observed facts.
That can underestimate uncertainty because imputation uncertainty is ignored.
Multiple imputation combines within-imputation and between-imputation variance through Rubin-style rules under assumptions.
44. Multiple Imputation Adds Missing-Value Uncertainty to Sampling Uncertainty
Each imputed dataset produces an estimate and variance.
Variation inside datasets reflects ordinary estimation uncertainty.
Variation across imputations reflects uncertainty about missing values.
The combined standard error increases when missingness makes inference less certain.
45. Estimated Propensity Scores Add Nuisance-Estimation Uncertainty
Inverse-probability weights are often estimated rather than known.
Naive variance formulas that pretend estimated weights are fixed can be wrong.
Influence-function, sandwich or bootstrap methods can propagate nuisance-estimation uncertainty under suitable conditions.
46. Doubly Robust Estimators Need Their Own Standard-Error Theory
Augmented inverse weighting, targeted estimation and orthogonal-score methods combine nuisance models with a target estimating equation.
Influence functions often supply asymptotic standard errors.
Cross-fitting can make those approximations valid with flexible machine-learning nuisance models under conditions.
47. Regularisation Makes Naive Standard Errors Difficult
Ridge deliberately biases coefficients.
Lasso both shrinks and selects.
Ordinary OLS variance formulas do not describe these altered estimators.
Post-selection inference, debiased estimators, bootstrap procedures or Bayesian posterior uncertainty may be needed depending on the goal.
48. Model Selection Makes Naive Standard Errors Optimistic
Search many models.
Keep the one with the strongest coefficient.
Then report the coefficient’s usual fixed-model SE.
The uncertainty from searching among models has vanished from the formula.
See How Statistical Model Selection Works.
49. Standard Errors Are Not Posterior Standard Deviations
A frequentist standard error estimates repeated-sampling variability of an estimator.
A Bayesian posterior standard deviation describes uncertainty about a parameter conditional on the prior, likelihood and observed data.
They can be close numerically in regular large samples and carry different meanings.
50. Standard Errors Can Be Tiny Around a Confounded Effect
A million-person observational study estimates an association with SE = 0.001.
An unmeasured confounder produces most of the association.
The standard error says the association parameter is estimated precisely under the model.
It does not say the causal effect is precisely known.
51. Standard Errors Can Be Tiny Around the Wrong Construct
An app measures daily logins for ten million students.
The average login count is estimated with microscopic SE.
The educational question is durable transfer.
Precision about logins is not precision about learning.
52. A Standard Error Is Conditional on the Analysis Procedure Actually Used
If outliers were removed after inspection, the uncertainty procedure should account for that decision if it materially affected selection.
If a transformation was chosen after trying several, that search is part of the procedure.
Standard errors calculated for a simplified fictional analysis can be smaller than uncertainty in the real adaptive workflow.
53. Sensitivity Analysis Complements Standard Errors
Standard error often quantifies random variation under one model.
Sensitivity analysis asks how conclusions move when uncertain assumptions change.
If assumption sensitivity dwarfs sampling SE, the inferential bottleneck is not sample size.
See How Sensitivity Analysis Works.
54. Power Calculations Are Standard-Error Engineering
For a fixed meaningful effect, power increases when the estimator’s standard error decreases.
Researchers can reduce SE by increasing sample size, reducing noise, improving design balance, using paired designs or measuring predictive covariates appropriately.
Power is not only a sample-size problem.
55. Better Covariates Can Reduce Standard Error in Randomised Trials
Baseline scores predict final scores strongly.
Adjusting for baseline can reduce unexplained outcome variance while preserving randomised treatment identification under a pre-specified valid model.
That can make the treatment estimate more precise without enrolling more participants.
56. The Hostile Test: SD Reported as SE
A chart shows mean test score = 70 with error bars of ±15.
The caption calls them standard errors.
But 15 was the individual-score standard deviation.
The graph makes the mean look ten times less precise than it is for n = 100 under IID assumptions.
57. The Second Hostile Test: Robust SE Used to Claim Robust Causality
An observational regression uses HC3 standard errors.
The paper calls the causal conclusion “robust”.
HC3 addresses one variance misspecification.
It does not remove unmeasured confounding, reverse causation or measurement bias.
58. The Third Hostile Test: 20,000 Rows, Eight Clusters
A school study has 20,000 student records from eight schools.
Cluster-robust SEs are reported using asymptotics that rely on many clusters.
The huge row count hides the small independent cluster count.
Alternative small-cluster methods are needed.
59. The Fourth Hostile Test: One Imputation, Ordinary SE
Thirty percent of outcomes are missing.
They are filled once with predicted values.
The completed dataset is analysed as though every value were observed.
The standard error ignores uncertainty in the missing outcomes.
60. The Fifth Hostile Test: Selected Model, Fixed-Model SE
Five hundred models are searched.
The model with the largest treatment coefficient is chosen.
The reported SE assumes that exact model was fixed before data collection.
Selection uncertainty has disappeared from the analysis.
61. Primary School: Standard Error Begins as “How Much Does the Average Move?”
Draw ten counters from a large bag and calculate their average value.
Repeat many times.
The individual counter values may vary widely.
The averages vary less.
The standard error is about how much our answer moves, not how much the things we measured differ from one another.
62. Secondary School: Keep SD and SE on Different Lines
Students should routinely write:
- SD: spread of observations;
- SE: spread of repeated estimates.
This one distinction prevents a remarkable amount of confusion later.
63. JC and University: Standard Error Becomes a Design-Specific Variance Estimate
At higher levels, learners should reconstruct:
- estimand;
- estimator;
- sampling distribution;
- variance estimator;
- standard error;
- t and z standardisation;
- design matrix;
- heteroskedasticity;
- clustering;
- serial correlation;
- survey design;
- bootstrap and jackknife alternatives;
- post-selection limits.
64. Where Standard Errors Fit in the eduKateSG “How Works” Landscape
- How Sampling Distributions Work — owns the distribution whose standard deviation the SE estimates.
- How Statistical Estimation Works — owns estimators and their bias–variance properties.
- How Confidence Intervals Work — uses standard errors to build many intervals.
- How P-Values Work — uses standardised statistics built from SEs.
- How Regression Works — owns coefficient models and residual structure.
- How Bootstrap Works — supplies computational SE estimates when analytic variance is difficult.
- How Sensitivity Analysis Works — tests assumption uncertainty beyond one SE formula.
Standard Errors own one precise canonical job: estimate the scale of repeated-sample variability of an estimator under a specified design and model, providing the local precision unit used by many intervals, test statistics and power calculations.
65. What This Article Does Not Claim
- Standard error is not standard deviation.
- A small SE does not imply an unbiased or causally valid estimate.
- A robust SE is robust only to the variance/dependence misspecification it targets.
- Large row counts do not guarantee small valid SEs when effective independent units are few.
- Ordinary IID formulas are wrong for many clustered, repeated, temporal and survey designs.
- One imputation followed by ordinary SEs can understate missing-data uncertainty.
- Post-selection and regularised estimators require uncertainty methods that reflect the procedure.
- Wald SEs can be poor near parameter boundaries or under weak identification.
- Posterior standard deviations and frequentist standard errors have different interpretations.
- Standard errors quantify one class of uncertainty and do not replace sensitivity analysis for untestable assumptions.
66. A Compact Standard-Error Audit
- What estimator does this standard error belong to?
- What repeated-sampling or assignment process defines its variance?
- What is the independent information unit?
- Is the reported quantity SD or SE?
- What analytic or resampling variance estimator is used?
- Are observations independent?
- Is heteroskedasticity present?
- Should HC, cluster or HAC covariance be used?
- How many independent clusters exist?
- Does survey design require weights, strata or replicate variance?
- Does finite-population sampling require a correction?
- Are repeated measurements modelled appropriately?
- Is the likelihood regular enough for Wald curvature approximations?
- Would profile likelihood or bootstrap intervals reveal asymmetry?
- Are nonlinear transformations propagating covariance correctly?
- Does missing-data handling include imputation uncertainty?
- Are nuisance models estimated?
- Was model selection performed?
- Was regularisation used?
- Could systematic bias dominate the SE?
- Does the practical decision require more than sampling precision?
67. Frequently Asked Questions
What is a standard error?
A standard error is an estimate of the standard deviation of a statistic’s sampling distribution. It quantifies how much the estimator would vary across repeated samples or assignments under a specified design and model.
What is the difference between standard deviation and standard error?
Standard deviation describes variation among observations. Standard error describes variation among hypothetical repeated estimates.
Why does standard error get smaller with sample size?
For many estimators, independent information accumulates with sample size. For the IID sample mean, SE = σ/√n, so averaging more independent observations reduces repeated-sample variability.
What are robust standard errors?
Robust or sandwich standard errors modify the estimated covariance matrix so inference can remain valid under selected departures such as heteroskedasticity or clustering. The exact robustness depends on the estimator used.
68. Authoritative Research Corridor
- NIST/SEMATECH Statistical Glossary — Standard Error
- NIST/SEMATECH — Central Limit Theorem and SE of the Mean
- NIST/SEMATECH — Regression Residual and Assumption Checking
- eduKateSG — How Sampling Distributions Work
- eduKateSG — How Bootstrap Works
- eduKateSG — How Regression Works
Final Thought: The Standard Error Is the Scale of How Much an Answer Could Have Moved
An estimate looks fixed once printed.
But it is one realised outcome of an information-generating process.
Another sample could have moved it.
Another classroom.
Another patient cohort.
Another randomisation.
Another day of measurement.
The standard error gives that unseen movement a scale.
A standard error does not tell us whether the answer is right. It tells us how much the answer would normally wobble if the same inferential machine were run again.