Association rules help us understand which items tend to appear together in transactions and how strongly the presence of one set of items is related to the presence of another. Rather than focusing on causality, association rules reveal patterns of co-occurrence that can be used to support business decisions such as cross-selling, up-selling, and bundling.
Using the Apriori Algorithm in Python
To discover association rules in practice, we commonly use the Apriori algorithm, which is implemented in Python through the mlxtend (Machine Learning Extensions) library.
The Apriori workflow consists of two main steps:
- Finding frequent itemsets
- Generating association rules from those itemsets
Step 1: Finding Frequent Itemsets
Frequent itemsets are discovered using the apriori() function. An itemset is considered frequent if its support is greater than or equal to a user-specified threshold.
Key parameters include:
min_support: the minimum proportion of transactions in which an itemset must appear.max_len: the maximum number of items allowed in an itemset.use_colnames: whether to display item names instead of column indices.
Choosing these parameters involves a trade-off. A very low minimum support or a large maximum itemset size can result in an overwhelming number of itemsets and heavy computational costs, while overly restrictive values may cause meaningful patterns to be missed.
Step 2: Generating Association Rules
Once frequent itemsets are identified, association rules are generated using the association_rules() function.
Key parameters include:
metric: the evaluation metric used to filter rules (commonly confidence).min_threshold: the minimum acceptable value for the chosen metric.
Only rules that meet or exceed the specified threshold are retained.
How Should Algorithm Parameters Be Chosen?
There is no universal “correct” set of parameters. Instead, parameter selection depends on the data characteristics, business objectives, and available computational resources.
Determining the Maximum Size of Itemsets
A practical approach is to examine how many items typically appear in a transaction:
- Count the number of items in each transaction.
- Create a frequency table of these counts.
- Use the median number of items per transaction as a guideline.
This ensures that the algorithm focuses on itemset sizes that are realistic and meaningful for the data.
Determining Minimum Support and Confidence
Support and confidence thresholds are often selected iteratively:
- Lower thresholds may be tested if resources allow, including values as small as 1 divided by the number of transactions.
- A Support vs. Confidence scatterplot is used to visualize trade-offs.
- Analysts then apply judgment to choose thresholds that balance credibility (confidence) and coverage (support).
Imaginary Store Market Basket Example
In the imaginary store dataset:
- There are 995 customers (transactions).
- The dataset contains 7,037 records, each representing a purchased item.
- Two features are used:
Customer(transaction identifier)Item(purchased product)
Understanding Maximum Support
The item Chips appears in the baskets of 698 customers, giving it a support of:
Because adding items to an itemset can only reduce support, this value represents the upper bound of support for this dataset. No multi-item rule can exceed this level of prevalence.
Choosing the Maximum Itemset Size
The distribution of items per transaction shows that the cumulative frequency first exceeds half of the total transactions at 7 items. Since the median number of items per transaction is 7, the maximum itemset size is set to 7.
Initial Support and Confidence Thresholds
The analysis begins with:
- Minimum support = 10 / 995, ensuring that rules apply to at least 10 customers.
- Minimum confidence = 0.5, meaning the rule must be correct at least half the time.
With these settings, 53,516 association rules are generated.
Interpreting Lift Values
The highest observed Lift among these rules is 3.95. To determine whether this value is meaningfully large, it must be compared against a benchmark.
Lift is defined as the ratio of observed confidence to expected confidence. Theoretically, the highest possible lift occurs when:
- Confidence equals 1, and
- Expected confidence (support of the consequent) is minimized.
In this dataset:
- Maximum confidence observed is 1
- Minimum expected confidence observed is 0.139698
This yields a theoretical benchmark lift of:
Observed lift values are evaluated relative to this benchmark.
Support vs. Confidence Trade-Offs
When visualized on a Support vs. Confidence scatterplot, association rules naturally fall into distinct regions:
- Rules with high support and high confidence are widely applicable and reliable.
- Rules with high confidence but low support are reliable but rare.
- Rules with low support and low confidence are often accidental and unreliable.
In practice, analysts typically prefer rules that apply to more customers, even if that requires a small reduction in confidence.
Stronger Filtering
When stricter thresholds are applied:
- Support ≥ 0.1
- Confidence ≥ 0.8
Only 51 rules remain. Their lift values range from 1.14 to 1.37, with a median of approximately 1.17. This reflects a smaller set of highly credible and widely applicable rules.
Focusing on Specific Items
Association rules can also be filtered based on specific business goals:
- If a business wants to promote Cereal, it can examine only rules where Cereal appears in the consequent.
- If the interest lies in understanding what customers buy after purchasing Oranges, rules can be filtered so that Oranges appear in the antecedent.
Summary
Association rules provide a structured way to uncover meaningful co-occurrence patterns in transaction data. By carefully selecting algorithm parameters and interpreting support, confidence, and lift together, analysts can move from raw transaction records to actionable insights that support decision-making in marketing, pricing, and product placement.