Skip to content

Advisor question routing and slots

Before the Advisor answers, routing works out whether the question is one of the brain's published Questions. If it is, routing makes sure the facts that Question's Answer Keys depend on (its slots) are known. It sits in front of serve(), behind the advisor_question_routing flag. It runs only for callers that pass route=True.

This page is the design reference. For what an integration sends and receives, see:

Flow

ask ──► classify ──┬─ none ───────────────────────────────────► serve() as today
                   ├─ uncertain ─► NeedsInput(choose_question)
                   └─ matched ──► fill slots ─┬─ nothing required missing ─► serve() on the resolved text
                                              └─ required missing ─────────► NeedsInput(slots)
reply ──► RouteResume(token, question_id | slots | reply) ──┬─ an answer to what was asked ─► fill slots ──► …
                                                            └─ a new question ──────────────► classify (fresh)
  1. Classify. First narrow the published Questions to a shortlist; the catalog matcher (app/eval/catalog.py, match()) can do this. Then a QuestionClassifier decides among them:
  2. JEV is the classifier we intend to use. It makes a Choice over the shortlist plus no_match, and one Choice per slot over the slot type's values plus not_stated. It is called through OpenRouter's Decisions API (typesafe/jev-1.14) with the brain's own OpenRouter key. The key comes from jev.routing_key, whatever tenant_model_choice says, and the platform key is never used. (OpenRouter reports these calls is_byok: false: the key is the brain's OpenRouter account, not a provider key registered with OpenRouter.)
  3. The no_match option says it covers a general question that names none of the facts a Question's situation turns on. Without that, generic declines ("What notices do we owe when we decline someone?") matched Q-dee-48 (app/eval/question_routing/jev.md, "Generic declines").
  4. There is also a fallback classifier that needs no JEV.

The result is a RouteDecision. How it is handled depends on its status:

Status Meaning What happens
matched Confident it is this Question Fill its slots
none Confident it is no Question Answer exactly as today
uncertain Anything else Ask the person to choose

The thresholds were set from the live benchmark, not guessed (app/eval/question_routing/jev.md). A pick below accept (0.6) is uncertain, whether it is a Question or no_match. A value the ask states needs slot_accept (0.6). A value or "I don't know" read from a reply needs reply_accept (0.45). 2. Choose (uncertain only). NeedsInput(reason="choose_question") offers the top candidates and "None of these". If the person picks NONE_OF_THESE (or types "none", "neither", …), the question is answered unrouted. A typed reply that isn't a pick the router can read goes to the classifier over the offered Questions only. If it is confidently none of them, the person has asked something else: see A new question typed in reply. 3. Fill slots. Each slot's value comes from the first of these that has one: 1. what the question states (source question); 2. the org context store (org). Not yet available: the store doesn't exist yet, so this step never fills a slot today, and every such slot is asked. When it lands it is keyed by slot type. If a Question has two slots of the same type, org context fills neither, and both are asked; 3. what the person told the Advisor when asked (told); 4. an optional slot's declared default (default). This is used only after the person was asked and skipped it or said they don't know. The ask shows the default, and the answer says it was assumed.

A slot nobody has asked about is unset. A slot the person said they don't know is unknown. 4. Ask. NeedsInput(reason="slots") lists each slot to ask about: its ask_text, whether it's required, and its allowed values. Optional slots are asked at most once. 5. Answer. Fill the Question's text with the binding using the same function that produces an Answer Key's question text (app/eval/generalize.py, text.format(...) with display()). Retrieval and the cache key use that text. The reasoner gets it and the person's own words (Routed.reasoner_query), so anything more they asked ("…and how long do we keep it?") is still answered. Their words are in the cache key too, but only when they differ from the text, so an unrouted answer's key is unchanged. The answer then ends with a "This answer is based on:" note: which facts it was told, which were assumed from a default, which it hedged over, and (once org context is available) which came from the organisation's settings.

Rules

Slot set unknown ("I don't know", "unknown", "TBD", …) unset
Required used stop: say what's needed and why; never guess ask
Optional used its default, marked assumed; with no default, a hedged answer covering the values whose Answer Keys differ ask once (showing any default), then treat as unknown

Slot filling, not the surface, decides what counts as unknown. A surface passes the person's own words in RouteResume.slots. A real value that merely contains one of the phrases is not unknown.

A "stop" is a slots ask of the blocked slot. NeedsInput has no blocked shape, and reason="slots" must ask at least one slot. So a required slot answered "I don't know" is asked again, on its own, and NeedsInput.message says why the Advisor won't answer without it. A route takes at most three replies (router.MAX_TURNS). If a required slot is still missing at the last one, the Advisor replies with a plain message saying what it needs, with no needs_input, and forgets the route.

A new question typed in reply

While a follow-up is pending, a surface passes the person's next plain message on as the reply (RouteResume.reply). Sometimes it is a new question instead. Routing then forgets the pending route and routes the typed text fresh, as if no follow-up were pending. The old question is not answered.

  • Choosing. The classifier, over the offered Questions only, reads the reply as none of them (status="none").
  • Slots. All of these hold:
  • no slot value came structured with it (RouteResume.slots);
  • the extractor read none of the waiting slots from it, neither a value nor "I don't know";
  • is_unknown doesn't read it as "I don't know";
  • it ends in ?, or the classifier, over the whole catalog, matches it to a Question on its own.

Anything else is still the answer. That includes a free-text or date slot's value, which the extractor can't read, and a value that isn't one ("consmer"), which slot filling refuses and says why. Rule 4 is what keeps a free-text value from being misread: a value is rarely phrased as a question, and is never one of the brain's Questions by itself.

When routing fails

One rule: answer unrouted, logged. What is answered depends on how far routing got:

  • A new ask answers what was asked.
  • A reply, once its pending route has loaded, answers the route's original question (PendingRoute.asked), never the reply. "Delaware" is not a question. The route is forgotten. This covers every fault after the load, including a Question text that can't be filled (slots.resolve_text raises SlotError for any str.format error).
  • A reply whose route is gone (expired, unknown, another actor's) has no original question.
  • A typed reply is answered as typed.
  • A structured reply (slot values or a pick, with nothing typed) is not a question. On the WebSocket its query is only the SPA's summary ("credit purpose: consumer; …"). The Advisor replies with a plain message, with no needs_input, saying the follow-up expired and asking the person to ask again. The audit status is route_expired.

A slot whose type has gone from the registry is asked as free text and never hedged, rather than failing the route.

Pending pointers never outlive the route. The WebSocket's per-conversation pointer and the OpenAI endpoint's follow-up reference both expire with routing_state.ROUTE_TTL_SECONDS. If a pointer outlived its route, the person's next, fresh question would be read as a reply to a route that is gone.

The input guardrail sees what is answered

The pre-gate checks the typed query. It also checks a reply's own words (RouteResume.reply) and every structured slot value (RouteResume.slots) before routing reads them. Once routing has decided what to answer (the Question's text with its binding, the original question, or a new one), the pre-gate checks that too, when it differs from what was typed. A block there is the same BLOCK envelope and pre_block audit row as any other.

How surfaces recognise a reply

  • WebSocket. A follow-up's token is remembered per conversation. The conversation's next plain message is the reply. With advisor_question_routing off, a leftover pointer is forgotten and the message is routed to a verb as usual.
  • OpenAI endpoint (stateless). A follow-up's text ends with a short reference, (ref: 3fa9c2d81b7e), minted per follow-up. Its token is stored under (brain, caller, ref). The next request whose last assistant turn carries that reference is the reply, and the reference is then spent.

The caller is the API key, which every end user of an integrating app shares. A key built from the follow-up's text, or from the whole history, collided when two of them asked the same thing: one person's reply resumed the other's route and got the other's answer. - MCP. The agent passes route_token back with slots, question_id or reply.

Shapes and interfaces

  • Shapes (app/models/routing.py):
  • SlotValue
  • RouteCandidate
  • RouteDecision
  • SlotRequest
  • NeedsInput
  • RouteResume
  • NONE_OF_THESE

They are mirrored in web/brain-builder/src/lib/api/contract.ts. - Swappable parts (app/agent/counselor/routing/interfaces.py): - QuestionClassifier.classify(asked, candidates, registry=...) -> RouteDecision - SlotExtractor.extract(text, slots, registry=...) -> dict[str, SlotValue] - CandidateQuestion: what a classifier is shown. - Envelope. ResponseEnvelope.needs_input is set instead of an answer: - result then holds the follow-up question as text, so a client that ignores the field still shows something sensible; - the field is left out of the serialised envelope when it is unset, so an unrouted answer serialises as before; - SCHEMA_VERSION is 1.3. - serve(route=False, resume=None, route_store=None, require_advisor_key=None). A user-facing surface passes route=True, and resume with the person's reply. Routes paused on a reply are kept in route_store (Redis) for routing_state.ROUTE_TTL_SECONDS (30 minutes). It defaults to cache, and then to the platform Redis. The routing step itself is in app/agent/counselor/routing/router.py. require_advisor_key defaults to route; see below. - Flag. advisor_question_routing, default off, mirrored in the SPA flag registry.

The Advisor needs the brain's OpenRouter key

A brain with no OpenRouter key of its own has no Advisor. Every user-facing call refuses, whether or not the routing flag is on, and whatever the verb. The refusal is a BLOCK envelope with rule_id="advisor.openrouter_key_required" and a message telling an admin to add the key under the brain's model providers. The Advisor MCP endpoint returns that envelope as the tool result. The OpenAI-compatible endpoint turns it into an HTTP 403 OpenAI-style error with code: openrouter_key_required, since there is no answer to show.

The key check is separate from routing: serve(require_advisor_key=True). It defaults to route, so a routed call always checks. The verbs that aren't routed pass it explicitly:

  • the WebSocket's retrieve, assess and redline;
  • the Advisor MCP server's retrieve;
  • the operator MCP server's retrieve, assess and redline.

Evaluation callers pass neither, and are unaffected.

What never changes

  • route not passed: serve() behaves exactly as before.
  • Flag off, route=True, and the brain has its OpenRouter key: serve() answers exactly as before. The key check is the only difference flag-off callers see.
  • Evaluation is never routed. These callers never pass route=True:
  • Question runs;
  • question_check;
  • advisor_eval;
  • wiki "deepen".

Scores keep measuring the unrouted path. - A NeedsInput reply makes no retrieval and no reasoner call, and is never cached.

Names

app/eval/routing.py already exists and means something else: which subsystem owns a failing rubric criterion (DEE-101). Question routing lives under app/agent/counselor/routing/. Its benchmark is app/eval/question_routing_bench.py.