How Mathematics Works | Linear Algebra

Linear algebra is the mathematics of vectors, linear transformations and systems whose structure can be preserved under addition and scaling. It is one of the most reusable languages in modern mathematics because many problems become easier when they can be expressed as movement inside a vector space.

Linear algebra sits behind computer graphics, engineering, optimisation, statistics, quantum mechanics, control systems, machine learning and scientific computing. It is not merely “matrices.” Matrices are one representation of a deeper structure: how quantities combine, transform and interact across many dimensions at once.

Series route: Mathematics Learning Hub → How Mathematics Works → Linear Algebra.

---

1. What linear algebra is

Linear algebra studies vector spaces, linear combinations, linear transformations and the systems generated by them. The defining feature is linearity: adding inputs adds outputs, and scaling an input scales the output by the same factor.

For a linear transformation T, the rules T(u+v)=T(u)+T(v) and T(cu)=cT(u) hold. These two properties allow complicated systems to be decomposed and rebuilt from simpler parts.

2. Vectors encode quantities with components

A vector can represent position, velocity, force, features, probabilities, coefficients or any object that behaves according to vector-space rules.

In two or three dimensions, vectors can be visualised as arrows. In higher dimensions, the geometric picture becomes harder to draw, but the algebraic structure remains.

3. Vector addition combines components

Adding vectors combines corresponding components. Geometrically, this can represent successive displacements or combined forces. Algebraically, it preserves the vector-space structure.

This simple rule is important because it allows complex states to be built from simpler directions.

4. Scalar multiplication changes magnitude without changing direction class

Multiplying a vector by a scalar stretches, shrinks or reverses it. This operation gives meaning to linear combinations such as a₁v₁ + a₂v₂ + … + aₙvₙ.

Linear combinations are the central construction of linear algebra. They ask what can be built from a given collection of vectors.

5. Span describes reach

The span of a set of vectors is the collection of all their linear combinations. It describes the region of the vector space those vectors can generate.

If two nonparallel vectors live in the plane, their span is the entire plane. If they are parallel, their span is only a line.

6. Linear independence measures redundancy

Vectors are linearly independent when none can be constructed from the others. Dependence means at least one vector is redundant relative to the rest.

This matters because mathematics often seeks the smallest nonredundant set that still generates the whole space.

7. A basis is a coordinate machine

A basis is a linearly independent spanning set. Once a basis is chosen, every vector in the space has a unique coordinate representation relative to that basis.

This means coordinates are not the vector itself. They are the vector’s description relative to a chosen reference frame.

8. Dimension counts independent directions

The dimension of a vector space is the number of vectors in any basis. It measures how many independent directions are required to describe the space.

A line has dimension one, a plane two, ordinary three-dimensional space three, but datasets and models may live in hundreds or millions of dimensions.

9. Matrices represent linear transformations

Once bases are chosen, a linear transformation can be represented by a matrix. Matrix-vector multiplication computes how coordinates change under the transformation.

The matrix is therefore not merely a rectangular table of numbers. It is a compressed instruction set for a linear map.

10. Matrix multiplication composes transformations

If matrix A represents one transformation and B another, then BA represents performing A first and then B, under compatible dimensions.

This is why matrix multiplication is not generally commutative. Order matters because transformation composition usually depends on order.

11. Systems of linear equations become one matrix problem

A system such as 2x+y=5 and x−y=1 can be written compactly as Ax=b. This packaging allows general algorithms to solve many equations simultaneously.

Gaussian elimination transforms the augmented matrix while preserving the solution set, eventually revealing whether the system has one solution, infinitely many or none.

12. Row operations preserve solution structure

Swapping equations, scaling an equation by a nonzero number and adding a multiple of one equation to another preserve the solution set.

This is linear algebra’s version of equivalent transformation: change the representation while protecting the invariant.

13. Rank measures effective independent information

The rank of a matrix measures the dimension of its column space and row space. Informally, it tells us how many independent directions or constraints the matrix actually carries.

A large matrix can have low rank when much of its information is redundant.

14. The null space captures what disappears

The null space of A contains all vectors x satisfying Ax=0. These are directions that the transformation collapses to zero.

Null spaces are crucial for understanding underdetermined systems, constraints and loss of information under transformations.

15. Rank-nullity balances visible and invisible dimensions

For a linear map from a finite-dimensional domain, the rank-nullity theorem says that the dimension of the domain equals rank plus nullity.

This is a conservation law for dimensions: independent input directions either survive into the output or disappear into the null space.

16. Determinants measure oriented volume scaling

The determinant of a square matrix measures how the transformation scales oriented volume. A determinant of zero means the transformation collapses dimension and is not invertible.

The determinant is therefore not just a formula. It is a geometric diagnostic of invertibility and volume change.

17. Invertibility means the transformation can be reversed

An invertible matrix represents a one-to-one linear transformation with a linear inverse. No information is lost.

Many apparently different conditions become equivalent for square matrices: nonzero determinant, full rank, trivial null space and existence of an inverse.

18. Dot products measure alignment

The dot product combines two vectors into a scalar. Geometrically it measures alignment through lengths and angle.

Orthogonal vectors have zero dot product. This concept supports projections, least squares, Fourier methods and geometry in higher dimensions.

19. Orthogonality creates clean decompositions

Orthogonal directions do not interfere with one another under the inner-product geometry. Orthogonal bases make coordinates easier to interpret and calculations more stable.

Normalising orthogonal vectors creates orthonormal bases, which preserve lengths and simplify projections.

20. Projection finds the closest component

Projecting a vector onto a subspace finds the part of the vector lying inside that subspace and leaves an orthogonal residual.

This geometric operation becomes one of the foundations of least-squares fitting.

21. Least squares solves inconsistent systems approximately

Real datasets often produce more equations than unknowns, and no exact solution satisfies them all. Least squares chooses the solution whose residual has minimum squared length.

Geometrically, it projects the observed vector onto the column space of the model matrix. Statistical regression therefore has a linear-algebraic core.

22. Eigenvectors reveal preferred directions

An eigenvector of a transformation is a nonzero direction whose direction is preserved, though its magnitude may scale. The scaling factor is the eigenvalue.

Eigenvectors reveal internal axes along which a transformation acts especially simply.

23. Eigenvalues expose stability and long-run behaviour

Repeated application of a matrix amplifies directions associated with large eigenvalue magnitudes and suppresses others. This helps explain long-run behaviour in dynamical systems, Markov chains and iterative algorithms.

24. Diagonalisation simplifies repeated transformations

If a matrix has enough independent eigenvectors, it can be represented in a basis where the transformation is diagonal. Powers of the matrix then become much easier to compute.

This is another representation strategy: choose coordinates that make the mechanism simple.

25. Singular value decomposition is a universal structural lens

The singular value decomposition factors a matrix into rotations or reflections and axis-aligned scaling. It applies even when the matrix is not square.

SVD reveals effective rank, dominant directions and low-dimensional structure. It is fundamental in compression, noise reduction, recommendation systems and numerical analysis.

26. A worked mechanism: solving a two-equation system

Consider 2x+y=5 and x−y=1.

1. Write the augmented matrix [[2,1|5],[1,−1|1]].
2. Replace the first row by first row minus twice the second: [0,3|3].
3. So y=1.
4. Substitute into x−y=1 to get x=2.

Elimination is not just arithmetic. It is a sequence of equivalent transformations preserving the common solution.

27. Common linear-algebra failure modes

• Matrix-as-table thinking: manipulating entries without seeing the transformation.
• Dimension mismatch: attempting operations whose shapes are incompatible.
• Basis confusion: treating coordinates as absolute rather than basis-dependent.
• Rank blindness: missing redundancy or hidden constraints.
• Eigenvalue ritual: calculating eigenpairs without interpreting what they mean.
• Exact-solution obsession: failing to recognise when least squares is the appropriate problem.

28. Linear algebra and geometry

Rotations, reflections, projections and scaling are geometric transformations with matrix representations. Coordinate geometry becomes dramatically more powerful when multiple dimensions are handled uniformly.

29. Linear algebra and differential equations

Systems of linear differential equations can be written in matrix form. Eigenvalues and eigenvectors often reveal whether solutions grow, decay, oscillate or combine modes.

30. Linear algebra and statistics

Regression, covariance matrices, principal-component analysis and multivariate models are deeply linear-algebraic. Data points become vectors; variables become axes; model fitting becomes projection.

31. Linear algebra and machine learning

Modern machine learning represents data, parameters and activations as vectors and tensors. Training repeatedly applies linear transformations combined with nonlinear operations.

Even when models are highly nonlinear overall, linear algebra remains the computational substrate.

32. Linear algebra as a machine

A useful machine model is:

State Vector → Basis/Coordinates → Linear Transformation → Rank/Null Structure → Decomposition → Solution, Projection or Dynamics.

The machine fails when dimensions are incompatible, basis choices are ignored, rank deficiency is hidden or matrix operations are performed without structural interpretation.

33. What mastery looks like

• see vectors as objects, not just coordinate lists;
• distinguish span, independence, basis and dimension;
• interpret matrices as transformations;
• solve systems through structure-preserving operations;
• read rank and null spaces as information architecture;
• use projections and least squares geometrically;
• interpret eigenvalues, eigenvectors and singular values;
• move comfortably between algebraic, geometric and computational views.

34. Conclusion

Linear algebra works by organising many quantities into vector spaces and describing structure-preserving transformations between them. Bases give coordinates. Matrices encode transformations. Rank measures independent information. Null spaces reveal lost directions. Eigenvectors expose preferred modes. Projections and decompositions turn difficult systems into simpler components.

Algebra describes relationships. Linear algebra describes whole systems of relationships at once. That is why it has become one of the core mathematical languages of the modern world.

---

How Mathematics Works | Batch 03

• How Mathematics Works | Discrete Mathematics
• Linear Algebra — this article
• How Mathematics Works | Differential Equations
• How Mathematics Works | Real Analysis

Return to the Mathematics Learning Hub or the How Mathematics Works root.