The singular value decomposition reveals the dominant directions, rank and conditioning of a matrix, but forming a complete dense decomposition may be unnecessary or impossible for a huge sparse problem. Golub–Kahan bidiagonalisation repeatedly multiplies by A and A transpose to build two orthonormal bases and a much smaller bidiagonal matrix. Mathematics matters because orthogonality, Krylov subspaces, three-term recurrences and singular values explain how large least-squares and low-rank questions can be compressed without explicitly forming A transpose A—and why loss of orthogonality, breakdown and stopping rules must still be managed.
The practical search question is not simply “what is this named method?” but “which mathematical structure makes the method work, what does it guarantee, and what evidence would show that it is a poor choice?” This article keeps those questions together so students can connect mathematics education with computing, engineering, data reasoning and careful everyday decisions without turning one technique into a career promise.
Quick navigation: Why it matters · Mechanism · Worked example · Deeper ideas · Investigations · Limits · Learning route · Parent guidance · FAQs · Sources
Why this mathematics matters
Mathematics is important here because it turns an apparently visual or computational trick into a chain of reasons. The representation tells us what information is kept. The equation or invariant explains each legal step. The complexity analysis describes the work. The boundary conditions say when a conclusion should not be trusted. That combination is a practical form of problem-solving literacy.
A strong learner should be able to move in both directions: from a real task to a mathematical model, and from a formal guarantee back to its real meaning. Naming an algorithm is not enough. The useful achievement is to explain why the next operation is valid, check a small instance independently, and notice which assumptions the implementation has added.
Source: Golub and Kahan’s 1965 SIAM paper on singular values and the pseudoinverse.
Primary or implementation reference: The official DOI record for the original Golub–Kahan paper.
How the mechanism works
Choose a unit right starting vector v_1 and set beta_0=0. For j=1,2,… compute p_j=A v_j−beta_(j−1)u_(j−1), set alpha_j=||p_j|| and u_j=p_j/alpha_j. Then compute r_j=A^T u_j−alpha_j v_j, set beta_j=||r_j|| and, if nonzero, v_(j+1)=r_j/beta_j. The coefficients alpha_j and beta_j form a small upper bidiagonal matrix B_k, while U_k and V_k collect the left and right basis vectors. Singular values of B_k approximate selected singular values of A. Practical implementations add reorthogonalisation, convergence tests, restarts and safeguards for zero or tiny coefficients.
In exact arithmetic the recurrences produce orthonormal columns and satisfy A V_k=U_k B_k plus a boundary term, while A^T U_k is represented by the transpose bidiagonal relation. The right basis spans a Krylov subspace generated by A^T A and the start vector; the left basis is linked to A A^T. Working through A and A^T avoids explicitly squaring the condition number by forming normal equations. Ritz-like singular values from the projected bidiagonal problem can converge to extreme singular values, supporting algorithms such as LSQR and partial SVD. Finite precision can create duplicate-looking converged values unless orthogonality is monitored.
Keep a four-column notebook labelled object, rule, guarantee and caveat. In the first column name the inputs and stored state. In the second write one update precisely. In the third state what remains true after the update. In the fourth record a tie, approximation, domain restriction or hardware assumption. This routine prevents symbols from drifting away from their meaning.
Worked example and core calculations
Let A=diag(3,4) and start v_1=(1,1)/sqrt(2). Then p_1=A v_1=(3,4)/sqrt(2), so alpha_1=||p_1||=5/sqrt(2) and u_1=(3/5,4/5). Next r_1=A^T u_1−alpha_1 v_1=(9/5,16/5)−(5/2,5/2)=(−7/10,7/10). Thus beta_1=7sqrt(2)/10 and v_2=(−1,1)/sqrt(2). Check that v_1 dot v_2=0 and u_1 has unit norm. A second recurrence step completes a 2 by 2 bidiagonal representation whose singular values are exactly 3 and 4. The first step already shows how A and A transpose generate paired directions.
How to audit the calculation
- Recompute every intermediate value directly from the definition.
- Keep labels, indices, signs, units and normalisation visible.
- Test a case with a known exact answer.
- State tie rules, random seeds, tolerances and boundary conventions.
- Compare with a slower transparent baseline or trusted implementation.
- Separate agreement of numbers from suitability of the real-world model.
Small examples are not childish; they are diagnostic instruments. A five-line trace can reveal a reversed inequality, a missing scale factor or an inconsistent label convention before a large dataset hides the same error behind plausible output.
Four deeper ideas
Bidiagonal is a useful middle form
A rectangular matrix is reduced to a narrow band whose singular values are much cheaper to study.
To transfer this idea, ask what would remain true if labels, scale or input order changed. Then identify the step whose justification would fail if the stated assumption were removed. This turns the paragraph into a reusable reasoning pattern rather than a fact to memorise.
Two bases grow together
Multiplication by A creates left directions; multiplication by A transpose returns information to the right space.
To transfer this idea, ask what would remain true if labels, scale or input order changed. Then identify the step whose justification would fail if the stated assumption were removed. This turns the paragraph into a reusable reasoning pattern rather than a fact to memorise.
Did you know? The method avoids explicitly forming A transpose A
That preserves sparsity and avoids an unnecessary numerical squaring of condition number in many computations.
To transfer this idea, ask what would remain true if labels, scale or input order changed. Then identify the step whose justification would fail if the stated assumption were removed. This turns the paragraph into a reusable reasoning pattern rather than a fact to memorise.
Small coefficients have two meanings
An exact zero can signal invariant-subspace completion, while a tiny finite-precision value may require a numerical breakdown decision.
To transfer this idea, ask what would remain true if labels, scale or input order changed. Then identify the step whose justification would fail if the stated assumption were removed. This turns the paragraph into a reusable reasoning pattern rather than a fact to memorise.
Twelve practical investigations
1. Two-by-two diagonal
Reproduce the worked first step. Check all norms and dot products.
What the mathematics reveals: The recurrence is transparent. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Keep radicals exact. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
2. Rank-one matrix
Start outside the nullspace. Observe early termination.
What the mathematics reveals: Happy breakdown can be informative. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Check residuals. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
3. Start in nullspace
Choose A v_1=0. Detect alpha_1=0.
What the mathematics reveals: The start carries no visible direction. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Restart deliberately. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
4. Rectangular matrix
Use a 3 by 2 example. Build left and right bases.
What the mathematics reveals: The method is naturally rectangular. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Track dimensions. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
5. Sparse matrix
Store only nonzeros. Count A and A transpose products.
What the mathematics reveals: Matrix-free work can be efficient. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Verify adjoint code. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
6. Clustered singular values
Choose two nearby values. Compare iteration counts.
What the mathematics reveals: Spectral separation affects convergence. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Use careful stopping. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
7. Loss of orthogonality
Run many steps in low precision. Measure Gram matrices.
What the mathematics reveals: Round-off changes the recurrence. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Reorthogonalise as needed. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
8. Partial SVD
Keep only the top two modes. Compare with dense SVD.
What the mathematics reveals: Projected problems approximate extremes. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Report residuals. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
9. Least squares
Connect the bidiagonal model to LSQR. Compare solution histories.
What the mathematics reveals: The recurrence powers iterative solvers. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Do not conflate algorithms. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
10. Scaled columns
Multiply one column by a large factor. Inspect singular values and conditioning.
What the mathematics reveals: Units affect geometry. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Scale with domain meaning. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
11. Restarted run
Limit subspace size. Compare memory and convergence.
What the mathematics reveals: Restarts trade storage for information. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Retain useful directions. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
12. Independent check
Use a trusted dense SVD on a toy. Compare singular triplets.
What the mathematics reveals: Reference computations verify the implementation. Evidence to collect: write the input, predict the next state, carry out the calculation, and compare the result with an independently produced reference. Explain the exact invariant, equation, inequality or counting rule that supports the conclusion. Then vary one assumption and record whether the result changes smoothly, discontinuously or not at all.
Limit to keep visible: Align sign conventions. A useful report places this boundary beside the result rather than hiding it in a final disclaimer.
Limits and misconceptions
Golub–Kahan needs reliable products with A and A transpose. If only one is available or they are inconsistent adjoints, the recurrence may not represent the intended problem. Finite precision erodes orthogonality and can produce spurious duplicate approximations, motivating selective or full reorthogonalisation. A poor starting vector can have negligible component in a desired singular direction. Very clustered singular values may converge slowly. Partial decompositions do not reveal every direction, and a small projected residual must be interpreted with scaling. Memory, restarts and stopping criteria matter for large problems. Singular values describe the supplied matrix, not whether its data or model is scientifically valid.
Five misconceptions to challenge
- Bidiagonalisation is already a full SVD: It produces a reduced problem from which singular information is extracted.
- A transpose A should always be formed first: Explicit normal equations can lose sparsity and worsen conditioning effects.
- Exact-arithmetic orthogonality persists automatically: Round-off can require reorthogonalisation.
- A zero recurrence coefficient is always a failure: It can be a happy breakdown showing an invariant subspace is complete.
- Largest singular values explain every important feature: Application relevance may depend on smaller directions, scaling or uncertainty.
Another broad misconception is that advanced mathematics automatically creates intelligence, admission, income or employment. It does not. Deliberate study can strengthen modelling, abstraction, calculation and explanation, while real pathways also depend on interests, communication, domain knowledge, opportunities and circumstances outside one topic.
A learning route for students
Move from a visible trace to a symbolic rule, then to code and critique. For every step below, prepare three pieces of evidence: a correct hand example, a boundary or failure case, and a short explanation using the relevant invariant. Do not advance merely because software returned a number.
Step 1: Multiply by a matrix and its transpose
Begin with the smallest non-trivial instance that lets you multiply by a matrix and its transpose. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 2: Normalise vectors
Begin with the smallest non-trivial instance that lets you normalise vectors. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 3: Build orthonormal bases
Begin with the smallest non-trivial instance that lets you build orthonormal bases. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 4: Trace a three-term recurrence
Begin with the smallest non-trivial instance that lets you trace a three-term recurrence. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 5: Assemble a bidiagonal matrix
Begin with the smallest non-trivial instance that lets you assemble a bidiagonal matrix. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 6: Interpret krylov subspaces
Begin with the smallest non-trivial instance that lets you interpret Krylov subspaces. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 7: Estimate singular values
Begin with the smallest non-trivial instance that lets you estimate singular values. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 8: Detect happy breakdown
Begin with the smallest non-trivial instance that lets you detect happy breakdown. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 9: Measure loss of orthogonality
Begin with the smallest non-trivial instance that lets you measure loss of orthogonality. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
Step 10: Validate projected residuals
Begin with the smallest non-trivial instance that lets you validate projected residuals. Predict the result before calculating. Write each state change clearly enough that another student can reproduce it. Compare with a trusted implementation only after the hand trace is complete.
Next, change one feature—size, ordering, sign, tolerance, labelling or data distribution—and explain which part of the reasoning changes. Finish by teaching the step without code and by naming one input for which the method is unhelpful. This predict–calculate–vary–explain cycle is a strong test of transferable understanding.
A four-week practice plan
- Week 1 — vocabulary and diagrams: define each object, redraw the mechanism and reproduce the worked example.
- Week 2 — equations and manufactured tests: derive the key relation and test inputs with known answers.
- Week 3 — implementation and evidence: use a trusted library, record parameters and compare with a transparent baseline.
- Week 4 — transfer and critique: apply the method to a new context, report limits and identify when a different method is better.
Guidance for parents and educators
Ask “what stays true?” and “what could make this conclusion fail?” before asking for speed. Invite the student to draw the input and output, estimate the result and locate the first step that disagrees with a reference. These prompts reveal conceptual understanding more clearly than memorised terminology.
Advanced enrichment should sit beside secure school foundations in algebra, functions, geometry, statistics and computational thinking. It need not accelerate a student into a fixed career identity. A healthy goal is curiosity with discipline: the learner becomes comfortable reading unfamiliar notation, checking an example, revising a model and communicating uncertainty.
Evidence prompts for a useful conversation
- Which quantity is observed and which is inferred?
- Which step saves work, and why is it allowed?
- Is the claim exact, approximate, probabilistic or empirical?
- What units and conventions are in use?
- Which input makes the method struggle?
- What simple baseline could check the answer?
- What would count as evidence that the parameters are poor?
Mini-project assessment
A strong mini-project contains the original question, a small reproducible dataset, a hand-worked example, code or a spreadsheet trace, at least one boundary test, a comparison with a baseline, and a paragraph on limits. Assess the reasoning trail as well as the final output. Repairing a model after a surprising result is valuable mathematical progress.
Six evidence checks for a finished project
Check 1: Multiply by a matrix and its transpose
Ask the student to demonstrate how to multiply by a matrix and its transpose without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Check 2: Normalise vectors
Ask the student to demonstrate how to normalise vectors without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Check 3: Build orthonormal bases
Ask the student to demonstrate how to build orthonormal bases without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Check 4: Trace a three-term recurrence
Ask the student to demonstrate how to trace a three-term recurrence without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Check 5: Assemble a bidiagonal matrix
Ask the student to demonstrate how to assemble a bidiagonal matrix without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Check 6: Interpret krylov subspaces
Ask the student to demonstrate how to interpret Krylov subspaces without relying on a hidden software step. The evidence should include a labelled input, the exact rule used, one intermediate state and a result that can be checked independently. If the answer changes after labels or order change, the student should say whether that reflects the mathematics or only a convention.
Then request one deliberately awkward example. The learner should predict the difficulty before running the method, explain what the output does and does not establish, and name a reasonable alternative or safeguard. This check joins correctness with judgement: a technically valid calculation is useful only when its assumptions and interpretation remain visible.
Frequently asked questions
What does Golub–Kahan bidiagonalisation do?
It reduces a large matrix to a small bidiagonal form using products with A and A transpose.
Why is bidiagonal form useful?
Its singular values and vectors are cheaper to compute and can approximate those of the original matrix.
Does it form A transpose A?
Not explicitly; the Krylov relationship appears through alternating products.
What are alpha and beta?
Norms that become diagonal and off-diagonal entries of the reduced matrix.
What is a Krylov subspace?
A span generated by repeated action of an operator on a starting vector.
What is happy breakdown?
An exact zero coefficient indicating no new basis direction is needed.
Why reorthogonalise?
Finite precision can make supposedly orthogonal basis vectors drift together.
Where is it used?
Partial SVD, least squares, regularisation and large sparse matrix analysis.
How is convergence checked?
Use projected residual estimates and the stability of requested singular values.
What should students know first?
Matrices, transpose, norms, orthogonality, singular values and iteration.
Useful next reading and sources
Source: Golub and Kahan’s 1965 SIAM paper on singular values and the pseudoinverse.
Source: The official DOI record for the original Golub–Kahan paper.
Related eduKateSG reading: Cholesky factorisation and linear solves.
Related eduKateSG reading: Lawson–Hanson nonnegative least squares.
Related eduKateSG reading: Broyden rank-one Jacobian updates.
Read sources with a question in mind: which definition, theorem, default or limitation is being supported? An authoritative link is useful evidence, but it does not replace a worked calculation or a clear statement of how the source applies to the example.
A cheerful final thought
Mathematics becomes friendlier when every symbol has a job and every claim has a boundary. Start small, make the state visible, test the awkward case, and celebrate the moment when a pattern becomes explainable. That is the practical benefit of learning mathematics: not certainty about every future, but a growing ability to ask better questions and build trustworthy answers.