1. Classify/
  2. Directory/
  3. Retrieval/
  4. Re-ranking legal passages

Re-ranking legal passages

Re-rank a 30-passage BM25 shortlist with one Jev question per pair.

Retrieval
Result
Top-1 5% to 18%, top-10 38% to 62% on 40 CLERC queries

BM25 is cheap and good at building a shortlist. It is bad at naming the right passage.

This Jev retrieval use case asks one question per query-candidate pair. 1,200 pairs is still cheap enough to fire together.

The published lift is the reason this case sits in the directory: a clear before and after on a public legal set.

Pipeline

  1. Build a 30-passage BM25 shortlist
  2. Ask Jev one relevance question per query-candidate pair
  3. Sort by the Jev score and keep the new ranking

Builder: TypeSafe cookbook. Stack: jev, bm25.