Choose your reading route
eduKateSG · Why Mathematics? · Kernel PCA for Nonlinear Process Monitoring: Kernel Matrices, Feature Spaces, Centering, T-Squared and SPE
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 6–10
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 11–15
Check assumptions
See where design and calibration matter.Route 4 · Chapters 16–20
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 21–24
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why mathematics is important becomes practical when healthy process data follow a curved relationship that a flat PCA subspace cannot represent well. Kernel PCA (KPCA) defines inner products through a positive-semidefinite kernel, centres the resulting Gram matrix, and solves an eigenvalue problem corresponding to PCA in an implicit feature space. Monitoring research builds feature-space T-squared and squared prediction error statistics from that representation.
For training points 0 and 2, use radial-basis kernel k(x,z)=exp(−0.5(x−z)²). The off-diagonal similarity is e⁻²≈0.1353, while a new point 1 has equal similarity e⁻⁰·⁵≈0.6065 to both. After centering the two-point Gram matrix, its nonzero eigenvalue is 1−e⁻²≈0.8647. The squared feature-space distance between the training points is 2−2e⁻²≈1.7293. The arithmetic shows how a kernel changes geometry without explicitly writing feature coordinates.
Did You Know? The kernel trick computes feature-space inner products directly, so an algorithm can use a very high-dimensional representation without constructing every feature.
This article answers one intent: The mechanism replaces ordinary dot products with a centred kernel matrix, then performs an eigenvalue decomposition whose scores and residuals describe nonlinear feature-space structure.
Kernel PCA performs linear PCA in an implicit feature space defined by a positive-semidefinite kernel. The worked radial-basis example exposes similarities, Gram-matrix centering, an eigenvalue and feature-space distance without constructing feature coordinates. Later chapters add kernel and bandwidth choice, T-squared and SPE, pre-images, scaling and computational limits. Students transfer functions, matrices and geometry; practitioners transfer a nonlinear monitoring tool whose sensitivity depends on transparent kernel design.
Full chapter index
Chapters 1–5 · Understand the question
Chapters 6–10 · Reproduce the example
Chapters 11–15 · Check assumptions
Chapters 16–20 · Explore applications
Chapters 21–24 · Study and verify
Begin with the object being measured, because a detector name is less useful than the statistic it changes. A curved healthy manifold can create large linear-PCA residuals even when operation is normal. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
A reaction path bends with temperature and concentration. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Nonlinearity can also reflect mixed regimes. Check the cause first. Transfer the idea by changing one input at a time and explaining the direction before computing.
A useful audit begins by writing the claim in plain language, then identifying the observation, reference model and comparison rule. In this chapter, the central claim is: A curved healthy manifold can create large linear-PCA residuals even when operation is normal. Test it first on a short sequence where every intermediate value can be inspected. Then change one condition and explain why the conclusion does or does not survive.
The mathematical value appears when a pattern is connected to one reproducible calculation. A kernel returns an inner product in an implicit feature space when validity conditions hold. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
Nearby RBF points have larger similarity. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
An arbitrary similarity need not define a valid kernel. Check positive semidefiniteness. Keep raw evidence beside the transformed statistic so diagnosis remains possible.
For transfer, sketch a before-and-after case and label what is measured, what is predicted and what is inferred. The concrete case here is: Nearby RBF points have larger similarity. That example is illustrative rather than evidence about a particular school, learner or machine. A sound explanation keeps calculation, interpretation and action in separate sentences.
Separate the mechanism from the interpretation so a number does not become unsupported certainty. Use exp(−γ‖x−z‖²) with γ controlling locality. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
Here γ=.5. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Very large or small γ destroys useful geometry. Tune with validation. Compare with a simple baseline and an alternative cause before deciding.
Reliability work needs a counterexample as well as a successful example. Here the important boundary is: Very large or small γ destroys useful geometry. Ask what other mechanism could create the same observed pattern, what additional measurement would separate the explanations, and how uncertainty should be recorded. This turns a formula into a defensible decision process.
Keep timing, units and assumptions visible before applying a threshold. Evaluate every training-pair kernel to form K. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
K=[[1,e⁻²],[e⁻²,1]]. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Memory grows quadratically with sample count. Plan computation. Explain the key equation aloud and point to the term carrying each meaning.
A student can practise this idea without specialised equipment. Start with a tiny table, reproduce the stated operation, and predict how the output changes when one entry is doubled, delayed or removed. The practical next step is: Plan computation. Explain the result aloud, including the assumption that would most change the answer.
Make the example auditable by predicting direction, calculating and checking a nearby case. Double-centre K so mapped training features have zero mean. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
The centred matrix has opposite diagonal and off-diagonal signs. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Incorrect new-point centering causes deployment errors. Store training means. Report observation, assumption, calculation and authorised action as separate layers.
A useful audit begins by writing the claim in plain language, then identifying the observation, reference model and comparison rule. In this chapter, the central claim is: Double-centre K so mapped training features have zero mean. Test it first on a short sequence where every intermediate value can be inspected. Then change one condition and explain why the conclusion does or does not survive.
Begin with the object being measured, because a detector name is less useful than the statistic it changes. Compute e⁻²≈.1353 and e⁻⁰·⁵≈.6065. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
The midpoint is equally similar to both training points. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Similarity is not probability. Label it geometrically. Transfer the idea by changing one input at a time and explaining the direction before computing.
For transfer, sketch a before-and-after case and label what is measured, what is predicted and what is inferred. The concrete case here is: The midpoint is equally similar to both training points. That example is illustrative rather than evidence about a particular school, learner or machine. A sound explanation keeps calculation, interpretation and action in separate sentences.
| Quantity | Value |
|---|---|
| Training points | 0 and 2 |
| Kernel | exp(−0.5(x−z)²) |
| Off-diagonal k(0,2) | e⁻²≈0.1353 |
| New-point similarities k(0,1),k(2,1) | e⁻⁰·⁵≈0.6065 each |
| Centred Gram eigenvalue | 1−e⁻²≈0.8647 |
| Feature-space distance² between 0 and 2 | 2−2e⁻²≈1.7293 |
The mathematical value appears when a pattern is connected to one reproducible calculation. Diagonalise the centred Gram matrix to obtain feature-space principal directions. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
The nonzero eigenvalue is 1−e⁻²≈.8647. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Eigenvalue normalisation conventions vary. Document the formula. Keep raw evidence beside the transformed statistic so diagnosis remains possible.
Reliability work needs a counterexample as well as a successful example. Here the important boundary is: Eigenvalue normalisation conventions vary. Ask what other mechanism could create the same observed pattern, what additional measurement would separate the explanations, and how uncertainty should be recorded. This turns a formula into a defensible decision process.
Separate the mechanism from the interpretation so a number does not become unsupported certainty. Project a new point using its centred kernel vector and normalised eigenvectors. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
The symmetric midpoint has zero contrast score for the two-point direction. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Scores depend on the training set and centering. Freeze the baseline. Compare with a simple baseline and an alternative cause before deciding.
A student can practise this idea without specialised equipment. Start with a tiny table, reproduce the stated operation, and predict how the output changes when one entry is doubled, delayed or removed. The practical next step is: Freeze the baseline. Explain the result aloud, including the assumption that would most change the answer.
Keep timing, units and assumptions visible before applying a threshold. Weight retained kernel-score magnitudes by retained eigenvalues. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
A large movement along the learned nonlinear direction raises T². Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Distributional limits are rarely automatic. Calibrate empirically. Explain the key equation aloud and point to the term carrying each meaning.
A useful audit begins by writing the claim in plain language, then identifying the observation, reference model and comparison rule. In this chapter, the central claim is: Weight retained kernel-score magnitudes by retained eigenvalues. Test it first on a short sequence where every intermediate value can be inspected. Then change one condition and explain why the conclusion does or does not survive.
Make the example auditable by predicting direction, calculating and checking a nearby case. Measure mapped-feature energy not explained by retained kernel components. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
A point dissimilar to the learned manifold raises SPE. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
SPE is not a distance in original units. Use contributions cautiously. Report observation, assumption, calculation and authorised action as separate layers.
For transfer, sketch a before-and-after case and label what is measured, what is predicted and what is inferred. The concrete case here is: A point dissimilar to the learned manifold raises SPE. That example is illustrative rather than evidence about a particular school, learner or machine. A sound explanation keeps calculation, interpretation and action in separate sentences.
Begin with the object being measured, because a detector name is less useful than the statistic it changes. Compute k(x,x)+k(z,z)−2k(x,z). The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
For 0 and 2 the result is about 1.7293. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Kernel distance sensitivity changes with γ. Report bandwidth. Transfer the idea by changing one input at a time and explaining the direction before computing.
Reliability work needs a counterexample as well as a successful example. Here the important boundary is: Kernel distance sensitivity changes with γ. Ask what other mechanism could create the same observed pattern, what additional measurement would separate the explanations, and how uncertainty should be recorded. This turns a formula into a defensible decision process.
The mathematical value appears when a pattern is connected to one reproducible calculation. Retain stable feature-space directions without fitting kernel noise. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
The two-point example has one nonzero centred component. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Large kernels can yield many tiny eigenvalues. Use held-out validation. Keep raw evidence beside the transformed statistic so diagnosis remains possible.
A student can practise this idea without specialised equipment. Start with a tiny table, reproduce the stated operation, and predict how the output changes when one entry is doubled, delayed or removed. The practical next step is: Use held-out validation. Explain the result aloud, including the assumption that would most change the answer.
Separate the mechanism from the interpretation so a number does not become unsupported certainty. Choose kernel family and parameters for the monitoring goal. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
RBF, polynomial and linear kernels encode different geometry. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Searching many settings inflates optimism. Use nested validation. Compare with a simple baseline and an alternative cause before deciding.
A useful audit begins by writing the claim in plain language, then identifying the observation, reference model and comparison rule. In this chapter, the central claim is: Choose kernel family and parameters for the monitoring goal. Test it first on a short sequence where every intermediate value can be inspected. Then change one condition and explain why the conclusion does or does not survive.
Keep timing, units and assumptions visible before applying a threshold. Estimate T² and SPE limits using representative baseline scores and residuals. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
Empirical quantiles avoid unsupported Gaussian claims. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Tail estimates need enough independent data. Respect autocorrelation. Explain the key equation aloud and point to the term carrying each meaning.
For transfer, sketch a before-and-after case and label what is measured, what is predicted and what is inferred. The concrete case here is: Empirical quantiles avoid unsupported Gaussian claims. That example is illustrative rather than evidence about a particular school, learner or machine. A sound explanation keeps calculation, interpretation and action in separate sentences.
Make the example auditable by predicting direction, calculating and checking a nearby case. Use sensitivity analysis, reconstruction ideas or variable perturbations to localise a signal. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
Perturb one sensor and recompute SPE. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Feature-space contributions are less direct than linear PCA contributions. Confirm in raw units. Report observation, assumption, calculation and authorised action as separate layers.
Reliability work needs a counterexample as well as a successful example. Here the important boundary is: Feature-space contributions are less direct than linear PCA contributions. Ask what other mechanism could create the same observed pattern, what additional measurement would separate the explanations, and how uncertainty should be recorded. This turns a formula into a defensible decision process.
Begin with the object being measured, because a detector name is less useful than the statistic it changes. Mapping back from feature space to an original-space reconstruction may lack an exact solution. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
A feature-space point has no simple input vector. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Approximate pre-images add optimisation error. Separate detection from reconstruction. Transfer the idea by changing one input at a time and explaining the direction before computing.
A student can practise this idea without specialised equipment. Start with a tiny table, reproduce the stated operation, and predict how the output changes when one entry is doubled, delayed or removed. The practical next step is: Separate detection from reconstruction. Explain the result aloud, including the assumption that would most change the answer.
The mathematical value appears when a pattern is connected to one reproducible calculation. Standardise original variables before computing distances unless the kernel is designed otherwise. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
One large-unit sensor can dominate RBF distance. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Scale choices are embedded in every kernel value. Archive them. Keep raw evidence beside the transformed statistic so diagnosis remains possible.
A useful audit begins by writing the claim in plain language, then identifying the observation, reference model and comparison rule. In this chapter, the central claim is: Standardise original variables before computing distances unless the kernel is designed otherwise. Test it first on a short sequence where every intermediate value can be inspected. Then change one condition and explain why the conclusion does or does not survive.
Separate the mechanism from the interpretation so a number does not become unsupported certainty. The full Gram matrix costs quadratic storage and cubic-style eigensolver work in naive implementations. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
Ten thousand samples create one hundred million entries. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Approximations change statistics. Validate Nyström or random features. Compare with a simple baseline and an alternative cause before deciding.
For transfer, sketch a before-and-after case and label what is measured, what is predicted and what is inferred. The concrete case here is: Ten thousand samples create one hundred million entries. That example is illustrative rather than evidence about a particular school, learner or machine. A sound explanation keeps calculation, interpretation and action in separate sentences.
Keep timing, units and assumptions visible before applying a threshold. A KPCA signal can trigger nonlinear-consistency checks and targeted inspection. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
SPE rises beyond an empirical limit. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
It does not prove a particular sensor failed. Seek independent evidence. Explain the key equation aloud and point to the term carrying each meaning.
Reliability work needs a counterexample as well as a successful example. Here the important boundary is: It does not prove a particular sensor failed. Ask what other mechanism could create the same observed pattern, what additional measurement would separate the explanations, and how uncertainty should be recorded. This turns a formula into a defensible decision process.
Make the example auditable by predicting direction, calculating and checking a nearby case. The topic connects functions, distance, matrices, centering and eigenvalues. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
A learner can calculate the three RBF similarities. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
The kernel trick changes representation, not arithmetic truth. Draw the similarity geometry. Report observation, assumption, calculation and authorised action as separate layers.
A student can practise this idea without specialised equipment. Start with a tiny table, reproduce the stated operation, and predict how the output changes when one entry is doubled, delayed or removed. The practical next step is: Draw the similarity geometry. Explain the result aloud, including the assumption that would most change the answer.
Begin with the object being measured, because a detector name is less useful than the statistic it changes. Specify variables, scaling, kernel, parameters, centering, components, limits, approximations and actions. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
Tests include curvature, regime changes, distant points and sensor bias. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Missing centering constants break online scores. Version the full transform. Transfer the idea by changing one input at a time and explaining the direction before computing.
A useful audit begins by writing the claim in plain language, then identifying the observation, reference model and comparison rule. In this chapter, the central claim is: Specify variables, scaling, kernel, parameters, centering, components, limits, approximations and actions. Test it first on a short sequence where every intermediate value can be inspected. Then change one condition and explain why the conclusion does or does not survive.
The mathematical value appears when a pattern is connected to one reproducible calculation. Linear PCA uses ordinary dot products; KPCA substitutes kernel inner products to represent nonlinear structure. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
A curved manifold may become flatter in feature space. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
KPCA costs more and is harder to interpret. Compare against linear PCA. Keep raw evidence beside the transformed statistic so diagnosis remains possible.
For transfer, sketch a before-and-after case and label what is measured, what is predicted and what is inferred. The concrete case here is: A curved manifold may become flatter in feature space. That example is illustrative rather than evidence about a particular school, learner or machine. A sound explanation keeps calculation, interpretation and action in separate sentences.
Separate the mechanism from the interpretation so a number does not become unsupported certainty. KPCA does not automatically discover the correct nonlinearity. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
Change γ from .5 to .05 and recompute similarities. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
A good training fit can still generalise poorly. Use held-out faults. Compare with a simple baseline and an alternative cause before deciding.
Reliability work needs a counterexample as well as a successful example. Here the important boundary is: A good training fit can still generalise poorly. Ask what other mechanism could create the same observed pattern, what additional measurement would separate the explanations, and how uncertainty should be recorded. This turns a formula into a defensible decision process.
Keep timing, units and assumptions visible before applying a threshold. Kernel eigenvalue research defines KPCA; process-monitoring studies develop feature-space T², SPE and diagnosis. The importance of mathematics here is that the statement becomes a quantity or logical test that another reader can reproduce.
Recent eduKate articles cover PCA, DPCA and CVA. Follow the order of operations, keep the time index and units, and check whether the direction makes sense before trusting the number.
Benchmark improvements depend on kernel design. Validate local geometry. Explain the key equation aloud and point to the term carrying each meaning.
A student can practise this idea without specialised equipment. Start with a tiny table, reproduce the stated operation, and predict how the output changes when one entry is doubled, delayed or removed. The practical next step is: Validate local geometry. Explain the result aloud, including the assumption that would most change the answer.
The nearest published companions are PCA T-squared and SPE monitoring, dynamic PCA, canonical variate analysis and set-membership fault diagnosis. The eduKateSG Mathematics Learning Hub, eduKateSingapore Mathematics World, eduKate Sengkang Mathematics Hub, Bukit Timah Tutor Mathematics Hub and eduKate Punggol Mathematics Pathway provide wider learning routes. Primary sources: https://doi.org/10.1162/089976698300017467, https://doi.org/10.1016/j.ces.2003.09.012, https://doi.org/10.1016/j.chemolab.2004.05.001.
Frequently asked questions
What is a kernel in KPCA?
It is a valid similarity function that acts as an inner product in an implicit feature space.
Why must the Gram matrix be centred?
PCA assumes centred features, so kernel values must be centred consistently for training and new points.
How is KPCA different from PCA?
KPCA performs PCA in a kernel-defined feature space and can represent nonlinear input geometry.
What does feature-space SPE mean?
It is mapped-feature energy outside the retained kernel principal components, not an original-unit error.
How should the RBF bandwidth be selected?
Choose it with held-out validation for the monitoring goal and compare against simpler kernels.
Extended practice laboratory
Practice case 1: Off-diagonal kernel
Setup: Use points 0 and 2 with γ=.5.
Checking target: Confirm e⁻².
Practice case 2: Midpoint similarities
Setup: Use new point 1.
Checking target: Confirm equal e⁻⁰·⁵ values.
Practice case 3: Build Gram matrix
Setup: Fill the two-by-two K.
Checking target: Check symmetry.
Practice case 4: Centre K
Setup: Subtract row and column means.
Checking target: Check zero row sums.
Practice case 5: Eigenvalue
Setup: Diagonalise centred K.
Checking target: Confirm 1−e⁻².
Practice case 6: Feature distance
Setup: Use 2−2e⁻².
Checking target: Confirm about 1.7293.
Practice case 7: Change gamma
Setup: Use γ=.05.
Checking target: Compare similarities.
Practice case 8: Distant point
Setup: Use x=10.
Checking target: Inspect kernel vector.
Practice case 9: Scale variable
Setup: Multiply coordinates by 10.
Checking target: Retune γ.
Practice case 10: Decision memo
Setup: Report high feature-space SPE.
Checking target: Avoid causal certainty.