Intra-List Diversity (ILD)

Intra-list diversity (ILD) measures how different the items in a single recommendation list are from one another. A common form averages a pairwise distance \(d(i,j)\) over the \(K(K-1)/2\) pairs in a list of \(K\) items: \(\mathrm{ILD}(L) = \frac{2}{K(K-1)}\sum_{i<j} d(i,j)\), where \(d = 1 – \text{similarity}\) when the similarity lies between 0 and 1. Shani and Gunawardana note that the diversity of a list can be measured by the sum, average, minimum, or maximum distance between item pairs; the formula above is the average.

A worked example

These genre sets are hypothetical. List 1 holds movie A {Action, Sci-Fi, Adventure}, movie B {Action, Sci-Fi}, and movie C {Drama}. With the Jaccard index as similarity, the similarities are 2/3 for A and B, and 0 for A and C and for B and C. The distances are 0.333, 1, and 1, so ILD = (0.333 + 1 + 1) / 3 = 0.778.

Replace C with movie D {Action, Sci-Fi}. The distances become 0.333 for A and B, 0.333 for A and D, and 0 for B and D, so ILD = 0.222. The list is nearly redundant, and the number shows it.

Choices that change the value

• The distance. Genre tags, item text, or embeddings each define a different kind of diversity. A list can be diverse by genre and uniform by another attribute. See Genre Overlap.
• The aggregation. A sum over pairs grows with \(K\), because the number of pairs is \(K(K-1)/2\), so sums from lists of different lengths are not comparable. The average keeps the scale but still depends on \(K\). The minimum reports the most similar pair.
• The user average. ILD is computed per list and then averaged over users, which can hide lists that are nearly identical.
• Rank and relevance. Vargas and Castells (2011) describe a general form, an expected intra-list distance, that discounts items at lower positions and weights pairs by relevance, so that non-relevant items count less.

Reading an ILD value

A high ILD is not good on its own. A list of unrelated items has a high ILD and can be useless, so ILD is read next to a relevance metric such as NDCG. Re-ranking a relevance-ordered list to raise ILD usually costs some relevance, which is the trade-off discussed in Diversity in Recommender Systems. Items whose attribute sets are empty make some similarity functions undefined, so the protocol needs a rule for them.

References:

• Shani and Gunawardana, Evaluating Recommendation Systems (Microsoft Research technical report)
• Vargas and Castells, Rank and Relevance in Novelty and Diversity Metrics for Recommender Systems (RecSys 2011)