Jev turns 2,000 wine notes into CatBoost features at 1.77 RMSE

Author: TypeSafe

Research

How the loop works

Misses rewrite the questions
  • Answers
  • Misses
1 of 5
1Note2,000 reviewsask2QuestionsClaude writes 183NumbersJev answersguess4Scoreguess critic pointsseen error 1.903hidden error 1.869lower is better5no misses yet60 notes across rangeClaude

This round's changes

New, reword, or remove
ChangeQuestionWhy
newcomplexity
newfruit_intensity
newtannin_structure
newacidity_intensity
newoak_intensity
newfinish_length
newbalance_harmony
newaging_potential
newpositive_superlative_language
newnegative_critical_language
newdrinkability_easiness
newbody_richness
newsweetness_level
newtexture_descriptors
newearthy_savory_notes
newflaw_or_defect_mentioned
newsingle_vineyard_or_prestige_signal
newvarietal_blend_detail

Classify this use case

  • Research1.00
  • Shipped0.94
  • Measured0.99
  • Usefulness0.54
  • Cookbook1.00
  • Ready0.90
  • Measured result0.56
  • Copy quality0.28
  • Creativity0.71
  • Distinct intent0.89
Result
Held-out RMSE 1.77 after five rounds, versus 3.09 for the mean baseline

Ask Jev Score and Noul questions that turn tasting notes into numeric features for CatBoost.

This Jev research use case starts from 2,000 wine tasting notes and a critic score. CatBoost can only train on numbers, so the loop has to invent Jev questions that turn each note into a row of answers.

Each round Claude writes Score and Noul questions. Jev answers them for every note. CatBoost guesses the critic score. The 30 worst misses and 30 best hits go back so Claude can add, reword, or drop questions.

Round 1 used 18 questions and reached 1.87 RMSE on 800 held-out reviews. After five rounds the set is 38 questions and the held-out RMSE is 1.77, against 3.09 for guessing the mean. The mill on the left is that loop. The original source is the cookbook.

How it works

  1. An LLM proposes add, revise, or drop actions
  2. Jev answers every new question for all 2,000 notes
  3. CatBoost trains on the encoded answers
  4. Worst and best predictions feed the next proposal round