Jev が Notra チャットを約 300 ms p50 で振り分ける
分類- レイテンシ
- ~300 ms p50 / ~550 ms p95
- 結果
- 真偽の答えをしきい値 0.5 で切る
各 Notra チャットメッセージで、複雑さの Choice と、ツールと推論が重い Noul を Jev に聞き、0.5 のしきい値でモデルを選ぶ。
質問
プリミティブ
Choice
Ask a multiple choice question
- Name
- complexity
- 指示
- How complex is the task in userMessage?
- 基準
- simple
Single-step task, quick edit, straightforward change, greeting, or simple question
- complex
Multi-step task, content creation from scratch, research, analysis, or a task requiring reasoning
- simple
プリミティブ
Noul
Evaluate how true something is
- Name
- requiresTools
- 指示
- Does fulfilling userMessage require using tools (editing the current draft, fetching data from connected sources, accessing skills, publishing or scheduling)?
- 基準
- true
Any editing, data fetching from connected sources, skills, or explicit requests to run tools
- false
Answering questions, explaining concepts, or conversation
- true
プリミティブ
Noul
Evaluate how true something is
- Name
- reasoningHeavy
- 指示
- Is the task in userMessage reasoning-heavy?
- 基準
- true
Deep analysis, long-form writing, multi-source research, architectural decisions, nuanced critique, comparative synthesis
- false
Everyday chat, quick edits, factual lookups, straightforward content generation
- true