Chapter map

1. 1. Matrix
2. 2. SVD
3. 3. Rank one
4. 4. Rank-one residual
5. 5. Spectral error
6. 6. Frobenius square
7. 7. Frobenius error
8. 8. Rank two
9. 9. Rank-two residual
10. 10. Rank-two errors
11. 11. Optimality
12. 12. Uniqueness
13. 13. Compression
14. 14. Preprocessing
15. 15. Scope
16. 16. Conclusion

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

---

1. Matrix

The central mathematical idea is concrete. Set A=diag(5,3,1).

A concrete situation

Its rank is 3.

What the mathematics does not license

Record dimensions. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Write A. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

2. SVD

Here the geometry becomes operational. Read singular values.

A concrete situation

They are 5,3,1.

What the mathematics does not license

Order descending. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Record spectrum. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

3. Rank one

This step deserves a careful pause. Keep only 5.

A concrete situation

A1=diag(5,0,0).

What the mathematics does not license

Use truncated SVD. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Record A1. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

4. Rank-one residual

The useful question is not only how to calculate, but what the calculation assumes. Compute A-A1.

A concrete situation

It is diag(0,3,1).

What the mathematics does not license

Track discarded tail. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Record residual. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

5. Spectral error

A small example makes the mechanism visible. Take the largest discarded singular value.

A concrete situation

The error is 3.

What the mathematics does not license

Name the norm. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Record result. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

6. Frobenius square

This is where modelling choices begin to affect evidence. Sum 3^2+1^2.

A concrete situation

The value is 10.

What the mathematics does not license

Use all discarded values. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Record 10. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

7. Frobenius error

The central mathematical idea is concrete. Take sqrt(10).

A concrete situation

Approximately 3.162278.

What the mathematics does not license

Round last. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Record result. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

8. Rank two

Here the geometry becomes operational. Keep 5 and 3.

A concrete situation

A2=diag(5,3,0).

What the mathematics does not license

Use same SVD. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Record A2. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

9. Rank-two residual

This step deserves a careful pause. Compute A-A2.

A concrete situation

It is diag(0,0,1).

What the mathematics does not license

Track tail. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Record residual. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

10. Rank-two errors

The useful question is not only how to calculate, but what the calculation assumes. Both spectral and Frobenius errors are 1.

A concrete situation

Only one value is discarded.

What the mathematics does not license

Name both norms. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Record results. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

11. Optimality

A small example makes the mechanism visible. Compare with any rank-k candidate.

A concrete situation

No eligible candidate has lower error.

What the mathematics does not license

This is theorem content. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Record claim. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

12. Uniqueness

This is where modelling choices begin to affect evidence. Check sigma_k>sigma_k+1.

A concrete situation

A gap supports unique subspace.

What the mathematics does not license

Handle ties separately. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Record gap. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

13. Compression

The central mathematical idea is concrete. Count stored factors separately.

A concrete situation

Error is not storage.

What the mathematics does not license

Do not conflate metrics. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Record costs. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

14. Preprocessing

Here the geometry becomes operational. State centring and scaling.

A concrete situation

They change the matrix.

What the mathematics does not license

Keep pipeline visible. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Record choices. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

15. Scope

This step deserves a careful pause. Separate approximation from interpretation.

A concrete situation

Retained factors need not be causes.

What the mathematics does not license

Do not overclaim. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Record limit. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

16. Conclusion

The useful question is not only how to calculate, but what the calculation assumes. State rank, norm and discarded spectrum.

A concrete situation

Rank-one errors are 3 and sqrt10.

What the mathematics does not license

Keep objective precise. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Write conclusion. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

A seven-day practice lab

1. Day 1: define the decision, observation unit, X variables and Y or reference condition.
2. Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
3. Day 3: build a transparent baseline and write what it assumes.
4. Day 4: fit the main method inside a training-only pipeline; record every parameter.
5. Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
6. Day 6: compare validation metrics with the practical workload created by errors.
7. Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

• Which part of the model is a mathematical definition, and which part is a choice?
• What small calculation would catch a sign, scale or dimension error?
• What evidence comes from training, validation and the untouched test?
• What benign condition could look like a fault or prediction failure?
• How would the answer change for another response, asset, school project or time period?
• What should a human review before acting on the score?

A method-specific audit before deployment

The Eckart–Young–Mirsky theorem identifies the best rank-k approximation to a matrix under the spectral norm and, in Mirsky’s extension, under every unitarily invariant norm. Truncating the singular value decomposition keeps the k largest singular components and discards the rest.

Let A=diag(5,3,1). Its rank-one truncated SVD is A1=diag(5,0,0). The spectral-norm error ||A-A1||2 is 3, the first discarded singular value. The Frobenius error is sqrt(3^2+1^2)=sqrt(10), approximately 3.162278.

The rank-two approximation A2=diag(5,3,0) has spectral and Frobenius error 1. These numbers show two different summaries of the same discarded tail. The operator norm reports the worst remaining stretch; the Frobenius norm combines all discarded energy quadratically.

The theorem is an optimization result, not merely a construction. It says no other matrix of rank at most k can achieve a smaller eligible error. A candidate factorization with an attractive interpretation may be useful, but it cannot beat the truncated SVD on the stated unitarily invariant norm objective.

Uniqueness needs care. If sigma_k is strictly larger than sigma_{k+1}, the associated best subspace is separated. At a tie, multiple optimal rank-k approximations can exist because different bases inside the tied singular subspace produce the same error.

Low-rank approximation appears in compression, denoising, recommendation systems, latent semantic analysis and reduced-order models. The theorem does not say that discarded components are only noise or that retained factors are meaningful causes. It optimizes a matrix norm under a rank constraint.

Data preprocessing matters. Centreing, scaling, missing-value treatment and weighting can change the matrix and therefore the SVD. The theorem guarantees optimality for the matrix and norm actually supplied, not for an unstated scientific target.

For students, this is a vivid example of abstraction becoming practical. Rank, orthogonality, singular values and norms lead to a concrete compression recipe. Yet responsible use still requires checking which norm matches the cost of the application.

Did You Know? The best spectral-norm error for rank k is sigma_{k+1}, while the best Frobenius error is the square root of the sum of squares of all discarded singular values. One SVD supports both formulas, but the meanings of the errors are different.

A compression report should state the original dimensions, chosen rank, retained singular values, discarded singular values, norm and achieved error. If storage or runtime is discussed, calculate those quantities separately rather than treating mathematical approximation error as a direct measure of speed.

Parents and students can view truncated SVD as a disciplined summary: keep the strongest independent patterns, measure what was left out and state the criterion. The same habit works in everyday reasoning—summaries are useful when their omissions and error measure remain visible.

The theorem can also be verified through a variational lower bound. Any rank-k matrix B has a nontrivial null direction inside a suitable (k+1)-dimensional right-singular subspace of A. Along that direction, the residual A-B must retain at least the (k+1)st singular value in operator norm. The truncated SVD attains this lower bound, proving optimality. This argument explains why no clever reweighting of k components can reduce the worst remaining stretch below sigma_{k+1}.

For Frobenius error, orthogonality makes the discarded singular components add through the Pythagorean rule. Their squared magnitudes sum to the residual squared norm. The calculation resembles energy accounting, but an application should reserve physical or statistical “energy” language for contexts where that interpretation is justified. In a generic matrix, it is safest to describe squared singular magnitudes and the chosen norm directly.

A responsible model-selection workflow repeats the approximation for several k values and plots the achieved error against storage, computation and task performance. The theorem supplies the optimal matrix approximation at each fixed k; it does not decide where the curve becomes acceptable. Parents and students can transfer this lesson to revision: a shorter summary may be optimal under a page limit, but the appropriate limit and acceptable omission still depend on the purpose.

The rank constraint should be checked after reconstruction. Numerical rounding can produce tiny nonzero singular values in a matrix intended to have rank k. Report a tolerance separately from the mathematical rank, and verify the factor dimensions. This prevents a nominal rank-k approximation from quietly storing or using more degrees of freedom than the comparison allows.

---

Frequently asked questions

What is the best rank-k approximation?

The SVD truncated after the k largest singular values.

What is the spectral error?

The first discarded singular value, sigma_{k+1}.

What is the Frobenius error?

The square root of the sum of squares of all discarded singular values.

Is the optimum always unique?

Not when the cutoff crosses a tie.

Does optimal compression find causal factors?

No. It optimizes a matrix norm.

What must be reported?

Preprocessing, rank, norm, retained and discarded spectra, error and uniqueness conditions.

---

Primary source and fact-check note

Carl Eckart and Gale Young, “The Approximation of One Matrix by Another of Lower Rank,” Psychometrika 1, 1936; extended by Leon Mirsky, 1960. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

• Mirsky’s unitarily invariant norms
• Singular-value perturbation bounds
• Ky Fan primary paper
• How Mathematics Works
• Study and learning methods

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Spectral audit: assumptions, norms and optimization logic

SVD audit

Order singular values and vectors consistently.

Rank audit

Keep exactly k components.

Norm audit

Match the error formula to the chosen norm.

Meaning audit

Do not call optimal factors causal.

Mechanism workshop: calculate, challenge and transfer

Manual reconstruction

Build the smallest matrix example for optimal low-rank approximation. Label spectra, dimensions, norms and assumptions.

Assumption map

Place normality, rank, orthogonality, separation and norm conditions beside the step they permit.

Counterexample

Remove one hypothesis and identify the first invalid conclusion.

Sensitivity

Change one singular value, pairing, gap or cutoff at a time.

Independent verification

Recompute spectra, norms and residuals through a second route.

Simulation

Generate bounded perturbations while keeping theorem and observation separate.

Engineering transfer

Map the mechanism to compression, signal analysis, numerical solvers or spectral data.

AI transfer

Explain a learning use without promising intelligence, accuracy or employment.

Parent explanation

Describe the reasoning habit in plain language without guaranteed results.

Limit statement

Write one supported claim, one overclaim and one repaired conclusion.

Evidence notebook: reconstruct every spectral step

Evidence notebook 1: Matrix

Step 1 isolates matrix. Set A=diag(5,3,1). The checked result is: Its rank is 3. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Record dimensions. Write A. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 2: SVD

Step 2 isolates svd. Read singular values. The checked result is: They are 5,3,1. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Order descending. Record spectrum. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 3: Rank one

Step 3 isolates rank one. Keep only 5. The checked result is: A1=diag(5,0,0). Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Use truncated SVD. Record A1. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 4: Rank-one residual

Step 4 isolates rank-one residual. Compute A-A1. The checked result is: It is diag(0,3,1). Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Track discarded tail. Record residual. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 5: Spectral error

Step 5 isolates spectral error. Take the largest discarded singular value. The checked result is: The error is 3. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Name the norm. Record result. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 6: Frobenius square

Step 6 isolates frobenius square. Sum 3^2+1^2. The checked result is: The value is 10. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Use all discarded values. Record 10. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 7: Frobenius error

Step 7 isolates frobenius error. Take sqrt(10). The checked result is: Approximately 3.162278. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Round last. Record result. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 8: Rank two

Step 8 isolates rank two. Keep 5 and 3. The checked result is: A2=diag(5,3,0). Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Use same SVD. Record A2. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 9: Rank-two residual

Step 9 isolates rank-two residual. Compute A-A2. The checked result is: It is diag(0,0,1). Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Track tail. Record residual. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 10: Rank-two errors

Step 10 isolates rank-two errors. Both spectral and Frobenius errors are 1. The checked result is: Only one value is discarded. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Name both norms. Record results. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 11: Optimality

Step 11 isolates optimality. Compare with any rank-k candidate. The checked result is: No eligible candidate has lower error. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: This is theorem content. Record claim. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 12: Uniqueness

Step 12 isolates uniqueness. Check sigma_k>sigma_k+1. The checked result is: A gap supports unique subspace. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Handle ties separately. Record gap. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 13: Compression

Step 13 isolates compression. Count stored factors separately. The checked result is: Error is not storage. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Do not conflate metrics. Record costs. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 14: Preprocessing

Step 14 isolates preprocessing. State centring and scaling. The checked result is: They change the matrix. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Keep pipeline visible. Record choices. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 15: Scope

Step 15 isolates scope. Separate approximation from interpretation. The checked result is: Retained factors need not be causes. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Do not overclaim. Record limit. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Evidence notebook 16: Conclusion

Step 16 isolates conclusion. State rank, norm and discarded spectrum. The checked result is: Rank-one errors are 3 and sqrt10. Preserve dimensions, ordering, norm, permutation, rank or gap beside every number. The safeguard is: Keep objective precise. Write conclusion. Change one input while holding the others fixed, recalculate the full spectrum or singular spectrum, and explain whether the change comes from matching, discarded components, separation, basis geometry or rounding. Ask a second reader to reproduce the calculation. Finish with the narrow conclusion and the evidence still missing.

Failure cases and responsible recovery

Unsorted truncation

Keep the largest singular values. Build a small counterexample, choose an eligible theorem and state the missing evidence before calculating again.

Norm mixing

Spectral and Frobenius errors have different formulas. Build a small counterexample, choose an eligible theorem and state the missing evidence before calculating again.

Rank equality

The feasible set is rank at most k. Build a small counterexample, choose an eligible theorem and state the missing evidence before calculating again.

Uniqueness assumption

Ties can create multiple optima. Build a small counterexample, choose an eligible theorem and state the missing evidence before calculating again.

Preprocessing omission

Centring and scaling change the matrix. Build a small counterexample, choose an eligible theorem and state the missing evidence before calculating again.

Noise claim

Discarded components are not automatically noise. Build a small counterexample, choose an eligible theorem and state the missing evidence before calculating again.

Storage conflation

Approximation error is not runtime or file size. Build a small counterexample, choose an eligible theorem and state the missing evidence before calculating again.

Prediction guarantee

Best matrix fit does not guarantee downstream accuracy. Build a small counterexample, choose an eligible theorem and state the missing evidence before calculating again.