Jev re-ranks 30 CLERC passages after BM25
Retrieval- Result
- Top-1 5% to 18%, top-10 38% to 62% on 40 CLERC queries
Ask Jev one Noul per query-passage pair on whether a BM25 candidate is the cited precedent.
Questions
Primitive
Noul
Evaluate how true something is
- Name
- is_cited_source
- Instructions
- The query excerpt comes from a US federal court opinion and was written immediately around a citation to a precedent; the citation itself has been removed. Could the candidate passage be from that cited precedent: does it establish the specific legal proposition the query excerpt invokes at its citation point?
- Criteria
- true
The candidate passage states or establishes the specific rule, standard, holding, or fact pattern that the query excerpt attributes to its removed citation.
- false
The candidate passage is merely on a similar topic or doctrine; it does not supply the specific proposition the query excerpt relies on.
- true