1. Definition

The Jaccard Index (also called Jaccard similarity coefficient) measures the similarity between two sets by comparing their intersection to their union.

$J(A, B) = \frac{|A \cap B|}{|A \cup B|}$

  • $|A \cap B|$: number of elements common to both sets.
  • $|A \cup B|$: total number of unique elements across both sets.
  • Range: $[0,1]$
    • $1.0$ → sets are identical.
    • $0.0$ → sets are disjoint (no overlap).

2. Example

  • User A’s liked movies = {Titanic, Avatar, Inception}
  • User B’s liked movies = {Avatar, Inception, Matrix}

$J(A,B) = \frac{| \{Avatar, Inception\} |}{|\{Titanic, Avatar, Inception, Matrix\}|} = \frac{2}{4} = 0.5$

So similarity = 0.5 (50%).

3. Relation to Jaccard Distance

Sometimes we want a distance measure (dissimilarity):

$d(A,B) = 1 – J(A,B)$

  • Distance = 0 if sets are identical.
  • Distance = 1 if sets are completely different.

4. Applications

  • Recommender Systems:
    • Compute similarity between users (based on items they consumed) or between items (based on users who interacted with them).
  • Document similarity: Compare sets of words (e.g., shingling for near-duplicate detection).
  • Clustering: Measure overlap between cluster label sets.
  • Search engines: Evaluate similarity between retrieved sets of documents.
  • Image segmentation: Overlap between predicted region and ground truth mask.

5. Comparison with Cosine Similarity

  • Cosine similarity: Works on vectors (counts, weights, embeddings). Sensitive to frequency and magnitude.
  • Jaccard index: Works on sets (binary presence/absence). Ignores frequency, only overlap matters.
  • Example: If you care whether two users watched the same movies (not how many times), Jaccard is a natural choice.

6. Extended Variants

  • Weighted Jaccard (Tanimoto coefficient): Handles real-valued vectors.

$J_w(x,y) = \frac{\sum_i \min(x_i, y_i)}{\sum_i \max(x_i, y_i)}$

  • Generalized Jaccard: Works with fuzzy sets or probabilities.

Summary:
The Jaccard Index is a set-based similarity metric defined as the ratio of intersection to union. It’s widely used in recommender systems, IR, and clustering, especially when features are binary (present/absent).