Associations Between Categorical and Categorical Features

To assess the relationship between two categorical variables, the analysis begins with a crosstabulation table, which summarizes how frequently each combination of categories occurs in the data.

A crosstabulation table records the observed counts $n_{ij}$, where each entry represents the number of observations that fall into category $i$ of the first variable and category $j$ of the second variable.
Row totals $n_{i+}$​ summarize counts across the second variable for each category of the first variable, column totals $n_{+j}$​ summarize counts across the first variable, and the grand total $n_{++}$ represents the total number of observations.

---

Pearson Chi-square Test for Independence

The Pearson Chi-square test is used to formally evaluate whether two categorical variables are statistically independent.

• Null hypothesis: The two categorical variables are independent.
• Alternative hypothesis: The two categorical variables are dependent.

If the null hypothesis is true, the population proportions of the second variable should be identical across all categories of the first variable. In other words, the row proportions should not vary from one row to another.

Under this assumption, the expected count for cell $(i, j)$ is calculated as

$$\hat{n}_{ij} = \frac{n_{i+} \, n_{+j}}{n_{++}}$$

These expected counts are always positive as long as the row and column totals are positive.

The Pearson Chi-square statistic is computed as

$$P^2 = \sum_{i=1}^{L} \sum_{j=1}^{K} \frac{(n_{ij} – \hat{n}_{ij})^2}{\hat{n}_{ij}}$$

When the null hypothesis is true, this statistic follows a Chi-square distribution with
$(L – 1)(K – 1)$ degrees of freedom.

If the resulting p-value is smaller than a chosen significance level (commonly 0.05), the null hypothesis of independence is rejected, indicating a statistically significant association between the two categorical variables.

---

Measuring the Strength of Association: Cramer’s V

While the Chi-square test establishes whether an association exists, Cramer’s V quantifies the strength of that association.

$$V = \sqrt{\frac{P^2}{n_{++} \times \min(K – 1, L – 1)}}$$

Cramer’s V takes values between 0 and 1:

• A value of 0 indicates statistical independence between the variables.
• A value of 1 indicates perfect dependence, where the category of one variable fully determines the category of the other.

A Cramer’s V equal to zero implies that all observed counts exactly match their expected counts under independence.
A Cramer’s V equal to one occurs when each row has nonzero counts in only one column, meaning the outcome category is completely determined by the row category.

---

Application: Pickup and Dropoff Communities in Chicago Taxi Data

The relationship between Pickup Community and Dropoff Community is evaluated using the Pearson Chi-square test and Cramer’s V.

The Pearson Chi-square statistic is 37,121.5993 with 64 degrees of freedom, and the associated p-value is effectively zero. This provides overwhelming evidence against the null hypothesis of independence.

The corresponding Cramer’s V is 0.1460. Although this value is not large, the extremely small p-value indicates that the association is not negligible. The results show that trip destinations are not randomly distributed with respect to pickup locations.

Because the pickup community is determined prior to the dropoff community, the observed association is consistent with a directional relationship in which pickup location influences destination patterns.

---

Associations Between Categorical and Continuous Features

When one variable is categorical and the other is continuous, association is assessed by comparing how the distribution of the continuous variable differs across categories.

Let $X$ denote a categorical variable with $L > 1$ categories and let $Y$ denote a continuous variable. For category $i$, there are $n_i$ observations $y_{ij}$​, with category mean $\bar{y}_i$. The overall mean across all observations is $\bar{y}$​.

---

Decomposing Variability: Sums of Squares

The corrected total sum of squares (SST) measures total variability of $Y$ around the overall mean:

$$SST = \sum_{i=1}^{L} \sum_{j=1}^{n_i} (y_{ij} – \bar{y})^2$$

The within-group sum of squares (SSW) measures variability of $Y$ within each category:

$$SSW = \sum_{i=1}^{L} \sum_{j=1}^{n_i} (y_{ij} – \bar{y}_i)^2$$

The remaining variation is attributable to differences between category means.

---

Eta-Squared as a Measure of Association

The Eta-squared statistic is defined as

$$\eta^2 = 1 – \frac{SSW}{SST}$$

Eta-squared lies between 0 and 1 and is interpreted as the proportion of total variation in $Y$ explained by the categorical variable $X$.

• If $\eta^2 = 1$, then all observations within each category have identical values, and the category completely determines $Y$.
• If $\eta^2 = 0$, then all category means are equal to the overall mean, and the categorical variable has no effect on $Y$.

---

Application: Tip Percentage by Payment Type

The association between Payment Type (categorical) and Tip Percentage (continuous) is evaluated using Eta-squared.

The estimated Eta-squared value is 0.4759, indicating that approximately 48% of the variation in tip percentage can be explained by payment type.

Customers who pay by credit card exhibit the highest tipping behavior, with the middle 50% tipping between 20.2% and 28.0% of the fare.
Customers using mobile payment methods show a lower interquartile range, between 17.1% and 22.7%.
In contrast, cash, procurement card, and unknown payment types are associated with median tip percentages of zero or near zero, with only a small number of extreme outliers.

These results demonstrate that payment method has a substantial and meaningful effect on tipping behavior.