Team Draft Interleaving (TDI)
Team draft interleaving builds one blended list from two rankings by letting the rankers act as team captains choosing players. In the description given by Hofmann et al. (2011), for each pair of positions a coin flip decides which ranker picks first, and that ranker contributes its highest-ranked item that is not already in the list. The method records which ranker contributed each item, and a click counts toward a ranker only if the clicked item was contributed by that ranker.
Procedure
- Start with an empty blended list and an empty assignment record.
- In each round, flip a coin to decide which ranker picks first.
- Each ranker in turn adds its top-ranked item that is not yet in the list and is recorded as the item’s owner.
- Repeat until the list has the desired length.
- After the user clicks, count the clicks on each ranker’s items. The ranker with more clicks wins the request, and equal counts tie.
Why each click has one owner
Because every item is assigned to exactly one ranker, a click cannot be credited to both. The coin flip gives each ranker the same chance to pick first, so neither has a structural advantage in the number of clicks it can earn. Hofmann et al. describe this assignment as what gives each list an equal chance of obtaining clicks.
A worked example
This example was constructed for this entry and uses the same shifted lists as the balanced interleaving entry: the first ranker returns (a, b, c, d) and the second returns (b, c, d, a). There are two rounds, so there are four coin-flip outcomes.
| Coin flips (round 1, round 2) | Blended list | Owner of a, b, c, d |
|---|---|---|
| Ranker 1, ranker 1 | a, b, c, d | 1, 2, 1, 2 |
| Ranker 1, ranker 2 | a, b, c, d | 1, 2, 2, 1 |
| Ranker 2, ranker 1 | b, a, c, d | 1, 2, 1, 2 |
| Ranker 2, ranker 2 | b, a, c, d | 1, 2, 2, 1 |
Item a is always owned by ranker 1 and item b by ranker 2, while c and d each belong to ranker 1 in half of the outcomes. If a single click lands on one of the four items with no preference between the rankers, each ranker earns 0.5 credits on average. Balanced interleaving gave the second ranker three of four credits on the same lists.
Limits
- Hofmann et al. report that team draft lacks sensitivity to specific differences in ranking, which motivated probabilistic interleaving.
- Requests with no clicks, or equal credit for both rankers, end in ties and add no information.
- It compares two rankers at a time, and its outcome is only as informative as the clicks that are credited.
Related entries: Interleaving Tests, CTR (Click-Through Rate).
References:
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.