Genre Overlap

Genre overlap is a label for applying a set-overlap measure to the genre tags of two items, or of an item and a user profile. It is not a single standardized metric, so a result should say which formula was used. In the MovieLens dataset, the MovieLens 25M README states that genres are a pipe-separated list chosen from a fixed set of genre names, with a placeholder, “(no genres listed)”, for movies that have none.

Common formulas

Let \(G_i\) and \(G_j\) be the genre sets of items \(i\) and \(j\). The usual choices are the Jaccard index \(|G_i \cap G_j| / |G_i \cup G_j|\), the overlap coefficient \(|G_i \cap G_j| / \min(|G_i|,|G_j|)\), and the Dice coefficient \(2|G_i \cap G_j| / (|G_i| + |G_j|)\). For a hypothetical movie A with {Action, Sci-Fi, Adventure} and movie B with {Action, Sci-Fi}, they give 0.667, 1.000, and 0.800. The same pair is either two-thirds overlapping or fully overlapping, depending on the formula.

Where it is used

• Item similarity in content-based recommendation, where items with many shared genres count as similar.
• Profile matching. A user’s preferred genres can be taken as the union of the genres of consumed items, and a candidate item is compared with that set.
• List redundancy. One minus the overlap serves as a distance in Intra-List Diversity (ILD), so two items with identical genre sets add no diversity.

What goes wrong

• Coarse tags. Two films tagged only Drama overlap completely and can be very different.
• Unequal tag counts. An item with many genres overlaps more often by chance, and the formulas treat that differently.
• Missing genres. If the placeholder “(no genres listed)” is handled as a real genre, any two unlabeled items overlap fully. Treat it as an empty set and apply a stated rule for empty sets; the Jaccard index entry covers that case.
• Genre is not taste. Matching genres is weak evidence that a user will like an item.

What to report

• The formula: Jaccard, overlap coefficient, Dice, or another.
• The genre source and how many genres per item are used, for example all tags or only the first.
• How empty genre sets and placeholder values are treated.
• Whether the value is averaged over item pairs, over lists, or over users.

See Diversity in Recommender Systems for how distances between items define diversity.

References:

• GroupLens, MovieLens 25M dataset README
• scikit-learn documentation, jaccard_score