Relevance in Recommender Systems
A ranking metric can only reward what the labels call relevant. Judged, rated, clicked, and held-out labels measure different things, and a missing label is not a negative label.
-
-
TREC (Text REtrieval Conference)
TREC gives researchers shared documents, questions, and relevance judgments so systems can be compared on one task. Its pooling method decides which documents get judged.