Did you know? A shop can discover that customers often buy apples and milk together. But a second glance might reveal that bread is even more common, whether or not apples were purchased. A pattern that sounds impressive can become ordinary once we compare it with the background frequency.
Association rule learning searches records for item combinations that occur together more often than a chosen baseline. The classic Apriori algorithm is one method for finding frequently co-occurring sets before rules are generated from them.
Quick answer: define a transaction → count individual items → keep itemsets above a minimum support → join promising itemsets → prune impossible candidates → form association rules → calculate support, confidence and lift → validate whether the pattern matters. It is pattern discovery, not proof that buying one item causes someone to buy another.
These ideas help explain recommendation logic, simple market-basket analysis and why the denominator matters. A six-receipt example makes the whole process visible without software.
Association rules in 30 seconds
- Transaction: one recorded collection of items, such as products purchased together in one receipt.
- Itemset: an unordered set of items, such as {Apples, Milk}.
- Support: the proportion of all transactions containing an itemset.
- Minimum support: the analyst’s threshold for deciding which itemsets are frequent enough to examine.
- Antecedent: the ‘if’ part of a rule, such as Apples.
- Consequent: the ‘then’ part, such as Milk; the two itemsets do not overlap in the standard association-rule definition.
- Confidence: the proportion of antecedent transactions that also contain the consequent.
- Lift: the rule’s confidence divided by the consequent’s overall support; it compares the conditional frequency with a baseline.
Our six fictional shopping baskets
Imagine six invented receipts from a classroom practice worksheet. They are not actual eduKateSG customer data and contain no personal information.
Receipt 1: Apples, Milk, Bread. Receipt 2: Apples, Milk. Receipt 3: Apples, Bread.
Receipt 4: Milk, Bread. Receipt 5: Apples, Milk, Bread. Receipt 6: Bread.
Count the baskets, not the number of items inside each basket. A receipt containing three apples still counts once as an Apples-containing transaction in this simple binary model.
Step 1: Make ‘a transaction’ unambiguous
A transaction might be a receipt, a single visit or a deliberately defined event. Those are not interchangeable. One shopper could have several receipts, and a product may be returned after purchase.
Set the unit of analysis first. If the question is about co-purchase within a receipt, keep one row per receipt. Mixing household, customer and receipt-level counts will distort support calculations.
Also document which products are excluded, whether refunds are represented and how missing purchase records are handled.
Step 2: Encode each basket as yes or no
Association algorithms often use a binary transaction matrix. Put one receipt on each row and one product on each column. Mark True when the receipt contains that product and False otherwise.
For our three-item example, Receipt 1 has True for Apples, Milk and Bread, while Receipt 6 has False, False and True. This is a representation of presence, not purchase quantity or spending amount.
The mlxtend Apriori guide demonstrates one-hot transaction encoding and the algorithm’s frequent-itemset output.
Step 3: Calculate single-item support
Apples appears on Receipts 1, 2, 3 and 5: 4/6 = 66.7%. Milk appears on Receipts 1, 2, 4 and 5: 4/6 = 66.7%.
Bread appears on Receipts 1, 3, 4, 5 and 6: 5/6 = 83.3%.
These are overall frequencies. Notice that bread already occurs in most receipts even before we ask whether it appears with apples. This baseline will become important when evaluating association rules.
Step 4: Count two-item combinations
The pair Apples and Milk appears on Receipts 1, 2 and 5: 3/6 = 50% support.
Apples and Bread appears on Receipts 1, 3 and 5: 3/6 = 50%. Milk and Bread also appears on Receipts 1, 4 and 5: 3/6 = 50%.
The three-itemset Apples, Milk and Bread appears only on Receipts 1 and 5: 2/6 = 33.3% support.
Record the actual numerator and denominator, not only a rounded percentage. Tiny datasets can produce very unstable estimates.
Step 5: Set a minimum support level
Suppose our teaching threshold is 50%. An itemset is frequent if it occurs in at least 3 of our 6 receipts.
Each individual item is frequent, and each two-item pair is frequent. The three-item combination appears only twice, so it is not frequent under this rule.
A lower threshold would retain more candidates but require more work. A higher threshold could discard meaningful niche patterns. The threshold is a research choice, not an objective definition of interesting behaviour.
Step 6: Understand the Apriori property
Apriori uses a simple logical shortcut: if an itemset is frequent, every subset of it must also be frequent. Equivalently, if a subset is too rare, any larger set containing it cannot reach a higher support.
Why? Every basket containing Apples, Milk and Bread must also contain Apples and Milk. Adding requirements cannot create new transactions that did not meet the smaller requirement.
In our example, the triple is not frequent at 50%, so we do not proceed to examine larger sets containing that triple. This downward-closure principle lets Apriori prune many unnecessary candidates.
The algorithm typically grows candidate itemsets from size one to size two, three and beyond, while retaining those meeting the support threshold.
Step 7: Turn frequent pairs into rules
From frequent {Apples, Milk} we may consider Apples → Milk and Milk → Apples. The arrow means ‘when the antecedent occurs in a transaction, how often does the consequent occur as well?’
It is not a physical or causal arrow. It does not claim buying apples makes a person crave milk. It is a statement about co-occurrence rates in a specified sample.
For larger itemsets, there are several possible ways to divide items into antecedent and consequent, so analysts usually apply rule-quality thresholds rather than presenting every possible rule.
Step 8: Calculate confidence for Apples → Milk
The support of Apples and Milk together is 3/6. The support of Apples by itself is 4/6. Their ratio is (3/6) ÷ (4/6) = 3/4 = 75%.
Therefore the sample confidence of Apples → Milk is 75%. Three of the four receipts containing apples also contain milk.
This is a conditional frequency, not a claim about future shoppers. It becomes more informative when considered alongside how common milk is overall.
Step 9: Calculate lift and check the baseline
Milk occurs in 4 of all 6 baskets, so the overall probability of a random basket containing milk in this worksheet is 4/6 ≈ 0.667.
The lift for Apples → Milk is confidence divided by support of Milk: 0.75 ÷ (4/6) = 1.125.
A lift above 1 indicates the two items occurred together more often than the product of their marginal frequencies would suggest under an independence baseline in this sample. Here the difference is modest: 1.125, not some dramatic proof of a real behavioural relationship.
To make a general population claim, a real analyst would need more observations, a reliable sampling process and appropriate validation.
Step 10: High confidence can be misleading
Now check Apples → Bread. Apples and Bread appeared together on 3 receipts; Apples appeared on 4. Confidence is again 3/4 = 75%.
But Bread appears on 5 of the 6 receipts overall. Its baseline support is 5/6 ≈ 83.3%.
So lift is 0.75 ÷ (5/6) = 0.9, below 1. Despite a seemingly high 75% confidence, bread was less common among Apples receipts than in the full six-receipt sample.
This is the central lesson: always compare a conditional rate with its baseline. Otherwise a very common consequent can make an ordinary rule look impressive.
Step 11: Confidence is directional
Support of a pair is symmetric: support(Apples, Milk) equals support(Milk, Apples). But confidence is conditional on which side of the arrow is the antecedent.
Our small dataset happens to contain the same number of Apples and Milk receipts, so both confidence values are 75%. Do not generalise from that coincidence.
If Apples appeared in many more receipts than Milk, the two conditional confidences could differ substantially. The direction is part of the rule definition even when the co-occurrence set is the same.
Step 12: Understand why lift equal to one matters
Lift of 1 means the observed confidence matches the consequent’s overall support under the calculation. Values above 1 indicate positive co-occurrence relative to that baseline; values below 1 indicate a negative association relative to it.
Lift is sensitive to how rare the consequent is, and rare combinations with very few supporting transactions can produce striking values that fail to repeat in new data.
Do not read lift as a causal effect size, a guaranteed forecast or a measure of business profitability.
Step 13: What the algorithm cannot infer
Apriori does not know that a product is useful, that a customer wants an add-on or that placing two items together will increase sales.
A supermarket may observe an association because both products are common during a festival, because of discounts, store layout or because the same shopper segment tends to buy them. These are alternative explanations.
To test a causal question such as whether changing shelf placement increases purchases, a properly designed experiment is more appropriate than an association rule alone.
Step 14: Why a pattern can vanish tomorrow
A rule learned from a small, seasonal or nonrepresentative sample may fail on a new week. Promotions, stock-outs and store changes alter what gets recorded.
Check how support, confidence and lift behave on a later, held-out time period. Record absolute transaction counts and business context, and be wary of treating one time window as a permanent habit.
Selection of thresholds after repeatedly inspecting outcomes can make patterns look more stable than they are. Report the exploration process honestly.
Step 15: More frequent patterns are not always more useful
An extremely frequent product might appear in many rules but provide little useful extra information, because it appears with nearly everything.
A low-frequency itemset could matter operationally in a specialist setting, yet lack enough support for a trustworthy pattern estimate. There is no universal rule that higher support automatically means greater value.
Business costs, availability, stock constraints, customer benefit and proper consent matter when moving from descriptive findings to actual recommendations.
Step 16: Privacy and responsible pattern finding
Transaction records can reveal personal interests. Work with appropriately authorised and minimised information, avoid publishing identifiable combinations and follow applicable privacy requirements.
Even a pattern derived from anonymous receipts can be misleading if the sample excludes certain groups or time periods. Carefully consider which population the dataset represents.
Step 17: Comparing Apriori with machine-learning classifiers
In logistic regression, a model learns to predict a defined binary outcome using labelled examples. It can evaluate probabilities and classifications on held-out cases.
k-nearest neighbours predicts outcomes by comparing feature similarity with stored examples. Association rules instead summarise what items co-occur in transactions under chosen thresholds.
DBSCAN discovers dense regions in feature space; it does not calculate shopping-basket confidence and lift. All are useful mathematical tools, but they answer different questions.
Step 18: Apriori is not the only frequent-pattern algorithm
Apriori is famous because its pruning logic is easy to explain. Other frequent-itemset approaches, including FP-growth, can avoid generating the same breadth of explicit candidates in some settings.
Performance depends on transaction count, the number of distinct items, typical basket size and support thresholds. Choose on evidence rather than assuming the earliest named algorithm is always fastest.
Step 19: How to evaluate a real association-rule project
Write down the unit of observation, data window, number of transactions and definition of each item. Explain how missing, repeated or returned transactions are handled.
Report the minimum support used to discover itemsets and the rule thresholds used to filter candidates. Keep track of the number of candidate rules considered, not only the most dramatic rule selected.
Test on later data and compare against simple baselines. If you turn the rule into a recommendation, measure whether real customers benefit rather than assuming mathematical co-occurrence guarantees value.
Step 20: Classroom applications beyond shopping
A teacher can use simple ‘which activities occur together’ counts in a fictional timetable to demonstrate set inclusion, fractions, conditional frequencies and why rules do not always have equal confidence in reverse.
Keep educational examples synthetic. Avoid collecting identifiable information about classmates or ranking children using casual co-occurrence patterns.
The Secondary 1 Mathematics foundation behind this exercise is especially valuable: check which quantity is the numerator, which is the denominator and what the ratio actually means.
Five common mistakes
- ‘Confidence is the probability the rule is true.’ It is the observed rate of a consequent among transactions with the antecedent.
- ‘A high confidence score always means a strong association.’ Check the consequent’s baseline prevalence using lift.
- ‘Apples → Milk means apples cause milk purchases.’ Association does not identify causation.
- ‘Only the pair count matters.’ Support and confidence need the right denominators and a defined transaction unit.
- ‘A frequent rule must be useful.’ Real utility needs independent testing and domain context.
Try it with six receipts
- Draw six numbered rows, and three yes/no columns called Apples, Milk and Bread.
- Enter the six fictional receipts in the example. Count how many rows contain Apples, Milk and Bread.
- Count how many rows contain both Apples and Milk; calculate support as 3/6 = 50%.
- Calculate confidence for Apples → Milk as 3/4 = 75%.
- Calculate lift as 0.75 ÷ (4/6) = 1.125.
- Repeat for Apples → Bread. Notice the same 75% confidence but lift 0.9.
- Explain in one sentence why baseline frequencies changed the interpretation.
Frequently asked questions
What is association rule learning?
It is a method of identifying itemsets that occur together in records and scoring conditional co-occurrence patterns using measures such as support, confidence and lift.
What is Apriori?
Apriori is a frequent-itemset mining algorithm that uses the subset property of support to eliminate candidates that cannot meet the minimum threshold.
What does support measure?
It is the proportion of transactions containing all items in a specified itemset.
What is confidence for A → B?
It is the fraction of transactions containing A that also contain B; it is directional and sample-dependent.
What does lift above one mean?
The sample has more joint occurrence than the simple independence baseline suggests, but not proof of a causal relationship.
Can association rules replace A/B testing?
No. Rule mining describes co-occurrence. A well-designed experiment is needed to test whether an intervention causes a particular outcome.
The one-minute takeaway
Association rule mining begins with actual transaction counts, finds frequent itemsets and computes support, confidence and lift. Apriori saves work by pruning impossible larger candidates. A 75% confidence rate can indicate either a positive or negative association relative to the baseline, so never forget the denominator, sampling context and difference between correlation and causation.
Trusted reading and related articles
- mlxtend: Apriori Frequent Itemsets — transaction encoding, support thresholds and candidate pruning
- mlxtend: Association Rules — formal support, confidence, lift and related metrics
- How Logistic Regression Predicts Yes-or-No Outcomes — a supervised outcome prediction method
- How DBSCAN Finds Dense Groups and Detects Noise — different unsupervised pattern discovery
- How AI Is Tested Before Release — why testing on new data matters
- How AI Works: Complete Overview — related machine-learning explanations
- Secondary 1 Mathematics Tutor Clementi — fractions, ratios and clear mathematical reasoning