Jev dimension scores reach 0.9076 on Japanese NLI

Cost

$1.43 for 34.1M input tokens and 5,477 test rows

Latency

9 minutes 14 seconds for 20,139 extraction rows

Result

0.9076 against 0.8373 on Japanese NLI; 37.2% against 1.5% false positives on hard benigns

Ask Jev either one direct question per row or 12 to 14 scored dimensions, then fit local weights on the cached scores.

Ask Jev either one direct question per row or 12 to 14 scored dimensions, then fit local weights on the cached scores.

The cited post ran three tasks. On Japanese NLI, 14 dimensions with fitted weights reached 0.9076 against 0.8373 for one direct Choice. On a 300-row cue-aggregate task the dimensions scored 74.0 percent against 64.7 percent for the direct call. On bookkeeping lines, word-bigram naive Bayes beat the dimensions, and a stack of both reached 0.9695.

On 339 hard benign rows the direct Choice false-positive rate was 1.5 percent. The 12 dimensions flagged 37.2 percent, about 25 times more. The whole experiment used 34.1M input tokens across 5,477 test rows and cost $1.43. Extraction of 20,139 rows took 9 minutes 14 seconds with 0 failed calls.