Chapter map

1. 1. Design
2. 2. Response
3. 3. Full estimate
4. 4. Full residuals
5. 5. Full SSE
6. 6. Good sketch
7. 7. Poor sketch
8. 8. Full check
9. 9. Ratio
10. 10. Embedding
11. 11. Leverage
12. 12. Projection
13. 13. Conditioning
14. 14. Repetition
15. 15. Conclusion

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

---

1. Design

This is where modelling choices begin to affect evidence. Set X=(1,1,1,1) transpose.

A concrete situation

There is one coefficient.

What the mathematics does not license

Record dimensions. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Write X. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

2. Response

The central mathematical idea is concrete. Set y=(1,2,3,4) transpose.

A concrete situation

There are four observations.

What the mathematics does not license

Keep order. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Write y. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

3. Full estimate

Here the geometry becomes operational. Average y.

A concrete situation

Beta full is 2.5.

What the mathematics does not license

Solve normal equation. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Record beta. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

4. Full residuals

This step deserves a careful pause. Subtract 2.5.

A concrete situation

Residuals are -1.5,-.5,.5,1.5.

What the mathematics does not license

Keep signs. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Record vector. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

5. Full SSE

The useful question is not only how to calculate, but what the calculation assumes. Square and sum.

A concrete situation

The optimum SSE is 5.

What the mathematics does not license

Use all rows. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Record 5. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

6. Good sketch

A small example makes the mechanism visible. Select rows one and four.

A concrete situation

Sketched beta is 2.5.

What the mathematics does not license

This is one realised sample. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Record estimate. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

7. Poor sketch

This is where modelling choices begin to affect evidence. Select rows one and two.

A concrete situation

Sketched beta is 1.5.

What the mathematics does not license

Same sample size. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Record estimate. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

8. Full check

The central mathematical idea is concrete. Evaluate beta 1.5 on all rows.

A concrete situation

The SSE is 9.

What the mathematics does not license

Do not judge on sketch alone. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Record 9. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

9. Ratio

Here the geometry becomes operational. Divide 9 by 5.

A concrete situation

The objective ratio is 1.8.

What the mathematics does not license

Use same objective. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Record ratio. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

10. Embedding

This step deserves a careful pause. Test norms on the relevant subspace.

A concrete situation

One vector is insufficient.

What the mathematics does not license

State distortion. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Record result. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

11. Leverage

The useful question is not only how to calculate, but what the calculation assumes. Compute design leverage.

A concrete situation

It is uniform in this example.

What the mathematics does not license

Do not overgeneralize. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Record scores. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

12. Projection

A small example makes the mechanism visible. Try a random-sign transform.

A concrete situation

Rows are mixed.

What the mathematics does not license

State normalization. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Record sketch. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

13. Conditioning

This is where modelling choices begin to affect evidence. Inspect singular values of X.

A concrete situation

General designs may be ill-conditioned.

What the mathematics does not license

Separate predictions and coefficients. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Record condition. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

14. Repetition

The central mathematical idea is concrete. Repeat independent sketches.

A concrete situation

Variability becomes visible.

What the mathematics does not license

Report seed policy. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Record distribution. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

15. Conclusion

Here the geometry becomes operational. State S,sketch size,rescaling and full residual.

A concrete situation

Worked SSE values are 5 and 9.

What the mathematics does not license

Keep algorithmic and statistical claims separate. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Write conclusion. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

A seven-day practice lab

1. Day 1: define the decision, observation unit, X variables and Y or reference condition.
2. Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
3. Day 3: build a transparent baseline and write what it assumes.
4. Day 4: fit the main method inside a training-only pipeline; record every parameter.
5. Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
6. Day 6: compare validation metrics with the practical workload created by errors.
7. Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

• Which part of the model is a mathematical definition, and which part is a choice?
• What small calculation would catch a sign, scale or dimension error?
• What evidence comes from training, validation and the untouched test?
• What benign condition could look like a fault or prediction failure?
• How would the answer change for another response, asset, school project or time period?
• What should a human review before acting on the score?

A method-specific audit before deployment

Randomized least squares replaces a large regression problem min_beta ||X beta-y||2 with a smaller sketched problem min_beta ||S X beta-S y||2. The sketching matrix S may sample rows, combine them with random signs or project them into fewer dimensions.

Consider X=(1,1,1,1) transpose and y=(1,2,3,4) transpose. Full least squares estimates beta as the mean, 2.5. The full residual sum of squares is (1-2.5)^2+(2-2.5)^2+(3-2.5)^2+(4-2.5)^2=5.

If a two-row sketch keeps the first and fourth rows with equal weight, the sketched estimate is also 2.5. If it keeps the first two rows, the sketched estimate is 1.5. Evaluated on all four observations, beta=1.5 has residual sum of squares 9.

The contrast shows why one successful sketch is not a proof. A subspace embedding seeks to preserve the lengths of every vector in the relevant column space, within stated distortion. Preserving one observed residual alone is weaker.

Algorithmic guarantees usually treat X and y as fixed and compare the sketched objective with the full optimum. Statistical guarantees add a data-generating model and may study prediction or estimation efficiency. These are different questions and can require different sketch sizes.

Leverage-score sampling adapts to the geometry of X. Random projections mix rows. Uniform sampling can be adequate when leverage is well spread. The appropriate method depends on coherence, sparsity, pass constraints and the theorem being used.

Rescaling is essential for row sampling. A selected row is commonly divided by the square root of its sampling probability and the number of draws. The exact convention depends on whether sampling is with replacement and how S is constructed.

Did You Know? Sketching can reduce the number of equations while leaving the number of unknown coefficients unchanged. The goal is to preserve the geometry needed to solve the original problem, not merely to delete data.

A preconditioned iterative solver can use a sketch to accelerate convergence without accepting the sketched solution as the final estimate. This separates sketch construction from the final accuracy target and is useful when high precision is required.

Students should evaluate a sketched coefficient on the full objective whenever possible. Compare ||X betahat-y|| with the optimum, inspect coefficient error under stated rank conditions and repeat random sketches to observe variability.

Rank deficiency and ill-conditioning complicate the story. If X has nearly dependent columns, small geometric distortions can cause large coefficient changes even when predictions remain similar. Report conditioning and distinguish coefficient from prediction accuracy.

For streaming data, S may be applied in one pass. For sparse data, an input-sparsity transform can be attractive. Runtime claims need the data access model, sparsity and target accuracy; a sketch is not automatically faster on every device or matrix.

In statistics and AI, an approximate linear solve does not validate the linear model, response labels or deployment decision. Residual diagnostics, out-of-sample tests and fairness review remain outside the algebraic speedup.

For parents and students, the transferable lesson is that a shorter practice set can represent a full topic only when it preserves the structure that matters. Variety and coverage matter more than simply reducing the number of questions.

A responsible report gives X and y dimensions, target rank, sketch family, sketch size, sampling probabilities, rescaling, seed policy, conditioning, sketched and full residuals, coefficient comparison and the exact probability statement.

The normal equations amplify conditioning because they involve X transpose X. Stable solvers commonly use QR or SVD, and sketched systems deserve the same care. A fast sketch followed by an unstable solve can lose the accuracy that the embedding was designed to preserve.

Prediction error and coefficient error can separate when columns are correlated. Two coefficient vectors may produce similar X beta values even when the coefficients differ greatly. Therefore a claim about preserved residuals should not be rewritten as a claim that every coefficient is close without additional singular-value assumptions.

If y contains sensitive outcomes, publishing the sketch matrix or sampled indices may reveal records. Reproducibility can use controlled seeds, code, distributions and aggregate diagnostics while protecting data. Privacy mechanisms introduce their own noise and must not be confused with ordinary randomized sketching.

An intercept should be handled deliberately. Centering before sketching, including an explicit constant column, and applying a transform that preserves that column are not identical operations. The worked one-column example is an intercept-only model, which makes this choice visible rather than hidden.

Weights, missingness and constraints create further variants. Weighted least squares changes the geometry before S is applied; imputation changes X and y; non-negativity changes the feasible set. A guarantee for ordinary unconstrained least squares should not be transferred to these versions without checking the relevant result.

---

Frequently asked questions

What is randomized least squares?

Solve a smaller problem using SX and Sy instead of all rows of X and y.

What is a subspace embedding?

A map that approximately preserves norms throughout the relevant subspace.

Can two sketches of the same size differ?

Yes. The worked estimates have full SSE 5 and 9.

Why check the full objective?

The sketch can look good on sampled equations while fitting the full data poorly.

Are algorithmic and statistical guarantees identical?

No. They answer different questions and use different assumptions.

What must be reported?

Dimensions, sketch family and size, probabilities, rescaling, conditioning, seed, coefficients and full residuals.

---

Primary source and fact-check note

Garvesh Raskutti and Michael W. Mahoney, “A Statistical Perspective on Randomized Sketching for Ordinary Least-Squares,” JMLR 17, 2016. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

• Leverage-score source
• Randomized SVD
• Schatten norms
• How Mathematics Works
• Study and learning methods

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Matrix audit: geometry, sampling and approximation logic

Objective audit

Evaluate the sketch solution on the full problem.

Embedding audit

State subspace distortion and probability.

Condition audit

Check rank and singular values of the design.

Inference audit

Separate computational and statistical guarantees.

Mechanism workshop: calculate, challenge and transfer

Manual reconstruction

Build the smallest exact example for regression sketches. Label shapes, ranks, norms and assumptions.

Assumption map

Place PSD, rank, sampling, rescaling and probability conditions beside the step they permit.

Counterexample

Choose a poor row, column, landmark or sketch and identify the first failed conclusion.

Sensitivity

Change one sample, probability, kernel parameter or sketch size while holding the rest fixed.

Independent verification

Recompute products, scores, eigenvalues and residuals through a second route.

Simulation

Repeat the randomized procedure and plot realised error without turning frequency into certainty.

Engineering transfer

Map the mechanism to compression, kernels, regression or scientific computing.

AI transfer

Explain a learning use while separating speed, accuracy, privacy and fairness.

Parent explanation

Describe coverage and representative examples in clear language without guaranteed results.

Limit statement

Write one supported claim, one overclaim and one repaired conclusion.

Evidence notebook: reconstruct every step

Evidence notebook 1: Design

Step 1 isolates design. Set X=(1,1,1,1) transpose. The checked result is: There is one coefficient. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Record dimensions. Write X. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 2: Response

Step 2 isolates response. Set y=(1,2,3,4) transpose. The checked result is: There are four observations. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Keep order. Write y. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 3: Full estimate

Step 3 isolates full estimate. Average y. The checked result is: Beta full is 2.5. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Solve normal equation. Record beta. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 4: Full residuals

Step 4 isolates full residuals. Subtract 2.5. The checked result is: Residuals are -1.5,-.5,.5,1.5. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Keep signs. Record vector. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 5: Full SSE

Step 5 isolates full sse. Square and sum. The checked result is: The optimum SSE is 5. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Use all rows. Record 5. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 6: Good sketch

Step 6 isolates good sketch. Select rows one and four. The checked result is: Sketched beta is 2.5. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: This is one realised sample. Record estimate. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 7: Poor sketch

Step 7 isolates poor sketch. Select rows one and two. The checked result is: Sketched beta is 1.5. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Same sample size. Record estimate. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 8: Full check

Step 8 isolates full check. Evaluate beta 1.5 on all rows. The checked result is: The SSE is 9. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Do not judge on sketch alone. Record 9. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 9: Ratio

Step 9 isolates ratio. Divide 9 by 5. The checked result is: The objective ratio is 1.8. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Use same objective. Record ratio. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 10: Embedding

Step 10 isolates embedding. Test norms on the relevant subspace. The checked result is: One vector is insufficient. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: State distortion. Record result. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 11: Leverage

Step 11 isolates leverage. Compute design leverage. The checked result is: It is uniform in this example. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Do not overgeneralize. Record scores. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 12: Projection

Step 12 isolates projection. Try a random-sign transform. The checked result is: Rows are mixed. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: State normalization. Record sketch. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 13: Conditioning

Step 13 isolates conditioning. Inspect singular values of X. The checked result is: General designs may be ill-conditioned. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Separate predictions and coefficients. Record condition. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 14: Repetition

Step 14 isolates repetition. Repeat independent sketches. The checked result is: Variability becomes visible. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Report seed policy. Record distribution. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 15: Conclusion

Step 15 isolates conclusion. State S,sketch size,rescaling and full residual. The checked result is: Worked SSE values are 5 and 9. Preserve shapes, rank, sampling rule, probabilities, kernel parameters, sketch definition and units beside every number. The safeguard is: Keep algorithmic and statistical claims separate. Write conclusion. Change one input while holding the others fixed, recalculate the product, score, approximation or full residual, and explain whether the change comes from subspace coverage, conditioning, landmark choice, rescaling, randomness or rounding. Ask a second reader to reproduce the arithmetic. Finish with the narrow conclusion and the evidence still missing.

Failure cases and responsible recovery

Sketch-only evaluation

Always check the full objective when feasible. Build a small counterexample, select an eligible theorem and state the missing evidence before calculating again.

One-draw proof

One successful sketch is not a guarantee. Build a small counterexample, select an eligible theorem and state the missing evidence before calculating again.

No rescaling

Row samples need the stated weight convention. Build a small counterexample, select an eligible theorem and state the missing evidence before calculating again.

Rank neglect

Ill-conditioning can destabilize coefficients. Build a small counterexample, select an eligible theorem and state the missing evidence before calculating again.

Guarantee mixing

Algorithmic and statistical claims differ. Build a small counterexample, select an eligible theorem and state the missing evidence before calculating again.

Speed promise

Hardware and sparsity affect runtime. Build a small counterexample, select an eligible theorem and state the missing evidence before calculating again.

Model validation shortcut

A fast solve does not validate the model. Build a small counterexample, select an eligible theorem and state the missing evidence before calculating again.

Seed omission

Randomized results need reproducible parameters. Build a small counterexample, select an eligible theorem and state the missing evidence before calculating again.