Schemas¶
The formats behind the Questions Framework: a Question, an Answer Key, a slot declaration and the slot type registry. For what each one means, see the concepts.
Where the files live¶
Each brain has its own store: a git repository beside its corpus repository, named
<brain_id>.answer-keys/. Questions and Answer Keys are YAML files in it, and every draft,
approval and migration is a commit attributed to whoever made it.
<brain_id>.answer-keys/
questions/<question_id>.yaml published Questions
questions/history/<id>.vN.yaml Question versions a newer one replaced
answer_keys/<id>.yaml approved Answer Keys, locked by approval.content_sha
answer_keys/history/<id>.vN.yaml Answer Key versions a newer approval replaced
drafts/<id>/question.yaml a draft Question
drafts/<id>/answer_key.yaml a draft Answer Key
drafts/<id>/review.json the draft's gate results, review and rejection
register/<id>.yaml what has been tried against each Answer Key
slot_types.yaml the brain's own slot types
shape_vocabulary.yaml the brain's own answer-shape elements
You don't need to write these files by hand. The API, the MCP tools and the web app write them for you. This page is for reading what they return, and for teams that keep Questions in git.
Question¶
A Question file, as the store holds it:
id: Q-late-fee-cap
version: 1
status: published # draft | published | retired
text: Is a late fee capped in {jurisdiction}?
intent: Decide whether our late fee needs a state-specific limit.
topic: fees
slots:
- name: jurisdiction
type: us_state
required: true
ask_text: Which state is the loan made in?
required_provisions:
always: ["1026.52(b)"]
by_binding:
jurisdiction=US-TX: ["us-tx:342.201"]
answer_shape:
must_include: [dollar_amount, citation]
valid_refusals: [coverage_missing_for_bound_slot]
answer_keys: [late-fee-cap-us-tx]
matching:
aliases: ["What's the maximum late fee in {jurisdiction}?"]
negative_examples: ["Can we charge a returned-payment fee?"]
| Field | Required | What it is |
|---|---|---|
id |
yes | The Question's id. Polaris names new ones Q-<slug>. |
version |
yes | A positive integer. Changing the text or slots of a published Question makes a new version. |
status |
yes | draft, published or retired. A Question is published when its first Answer Key is approved. |
text |
yes | The question as people ask it. Each required slot appears as {name}, and nothing else may. |
intent |
no | What the asker needs to decide, in a sentence. The API requires it when you propose a Question. |
topic |
no | The topic card the Question belongs to. Matching narrows to a topic first. The API requires it when you propose a Question. |
slots |
no | The slot declarations. See Slot declaration. |
required_provisions |
no | The provisions an answer must rest on: always, plus more per binding under by_binding (keyed by the binding key). Polaris adds every provision an approved criterion cites. |
answer_shape |
no | What kind of answer is valid: must_include and must_not_include name shape elements; valid_refusals names refusals that count as a correct answer. Checked on live answers, where it annotates and never blocks. |
answer_keys |
no | The ids of the Question's Answer Keys, one per binding. |
matching |
no | aliases (other phrasings that should match) and negative_examples (near misses that must not). |
certification |
no | Raises the certification threshold for this Question: runs, min_median_standing, judged_graded, no_penalty_cap. It can only raise the defaults (5, 0.8, true, true), never lower them. |
history |
no | Written by Polaris when a new version is published: each retired version and why. |
The retired field name cases is refused; use answer_keys.
Over the API a Question comes back as JSON with question_id in place of id, its Answer Keys
inlined, and certified_bindings (the binding keys certified at the published brain version).
The API reference has the response models.
Answer Key¶
An Answer Key is a YAML file in the store. The API returns its rubric as JSON in a different
shape, described by a JSON Schema: answer_key.schema.json. The two
carry the same content, and converting one to the other keeps the approval hash.
| On disk (YAML) | In the API (JSON Schema) |
|---|---|
question |
question_text_resolved |
gold |
answer |
gold_when |
full_credit_when |
criterion matches |
criterion pattern |
judged: true / manual: true on a criterion |
criterion tier: judged / manual (else machine) |
verdicts, default_verdict, refuses, penalties, report, fabricates |
inside scoring |
approval.approved_by, approval.approved_at, approval.content_sha |
approved_by, approved_at, content_sha |
any other top-level key, such as a draft's draft block |
inside extensions |
The JSON form also carries fields that aren't stored in the file: question_version, status
(active, superseded, needs_review) and last_scored, the latest score with its
comparability tuple. Only a Curator or Admin sees a rubric's substance (answer, criteria,
full_credit_when, scoring, extensions). Everyone else gets the Answer Key without them.
An approved Answer Key on disk:
id: late-fee-cap-us-tx
version: 1
question_id: Q-late-fee-cap
binding: {jurisdiction: US-TX}
approval:
approved_by: Dana Reyes
approved_at: "2026-11-02"
content_sha: "sha256:…"
question: Is a late fee capped in Texas?
gold: >-
Yes. …
verdicts:
- name: NOT_CAPPED
matches: "no (cap|limit) on"
- name: CAPPED
matches: "\\bcap(ped)?\\b|may not exceed"
default_verdict: UNCLEAR
criteria:
- key: A1_cap_named
weight: 40
describe: names the dollar cap
matches: "\\$\\s?\\d"
provisions: ["us-tx:342.201"]
- key: A2_cap_correct
weight: 60
judged: true
describe: states the cap and when it applies correctly
provisions: ["us-tx:342.201"]
gold_when: [A1_cap_named]
penalties:
- cap: 20
verdict: NOT_CAPPED
describe: says there is no cap
Top-level fields¶
| Field | Required | What it is |
|---|---|---|
id |
yes | The Answer Key's id. |
question |
yes | The question at this binding, slots filled in. |
criteria |
yes | The rubric: 1 to 60 criteria, each key unique. |
version |
no | A positive integer, default 1. |
question_id |
no | The Question it belongs to. Absent means a standalone Answer Key with no slots. |
binding |
no | Slot name to value. {} for a Question with no slots. |
gold |
no | The expert's reference answer, word for word. |
verdicts |
no | Ordered {name, matches} pairs; the first whose pattern matches the answer is its verdict. |
default_verdict |
no | The verdict when none matches. Default COND. |
refuses |
no | A pattern that marks the answer as a refusal. |
penalties |
no | Caps applied after scoring: {cap, verdict?, check?, describe?}. A cap limits the score; it doesn't subtract. |
report |
no | Name to pattern. Printed with a result, never scored. |
gold_when |
no | Machine criteria that together mean full credit. Judged criteria never belong here. |
fabricates |
no | {subject, claims}: patterns for an answer that asserts the subject is something it isn't. Hedged phrasings are ignored. |
supersedes |
no | The version this one replaces. Needs a change_reason. |
change_reason |
no | {kind, note?, effective_date?, triggered_by?}. kind is law_changed, new_adjudication, scope_refined or error_corrected. |
approval |
no | {approved_by, approved_at, content_sha}. Written by approval; a draft must not have one. |
draft |
no | On drafts only: who drafted it, with which model, from what source. |
Criterion fields¶
| Field | Required | What it is |
|---|---|---|
key |
yes | Unique within the Answer Key. |
weight |
yes | A whole number of points, 0 or more. |
describe |
no | What the criterion checks, in words. A judged criterion's judge reads it. |
matches |
no | Scored when this pattern matches the answer. |
absent |
no | Scored when this pattern does not match. Use it for "must not say X". |
verdict |
no | Scored when the answer's verdict equals this. |
check / negate_check |
no | Scored by a named code check (or by it being false). |
judged |
no | true: scored by the judge. A judged criterion takes no matches, absent, verdict or check. |
manual |
no | true: graded by a person only, never counted in an automated score. Can't be combined with judged. |
provisions |
no | The provision keys the criterion rests on, such as 1002.9(a)(1). |
When a criterion has more than one scorer, verdict wins, then check, then absent, then
matches. Patterns are case-insensitive regular expressions of at most 500 characters. A literal
double backslash is refused, because in a YAML folded string (>-) it is almost always a pattern
that can never match.
The approval lock¶
approval.content_sha is sha256: plus the hash of every top-level field except approval,
serialised as sorted JSON. YAML comments aren't covered, so notes in a file's header stay
editable. Change a pattern, weight, criterion, gold, question or version, and the lock
breaks. To change what counts as correct, write a new version with supersedes and a
change_reason, and have it approved. The version it replaces is kept in
answer_keys/history/.
Slot declaration¶
Each entry in a Question's slots has exactly these fields:
| Field | Required | What it is |
|---|---|---|
name |
yes | A lower-case letter, then lower-case letters, digits or _; at most 41 characters. Unique in the Question. |
type |
yes | An active slot type's name. |
required |
no | true (the default) or false. |
ask_text |
no | What the Advisor asks a person who didn't say this. It can change on a published Question without a new version. |
default |
no | Optional slots only: the value an unbound Answer Key takes. Must be a valid value of the type. |
Rules:
- Every required slot appears in
textas{name}. An optional slot never does, since leaving it unbound would leave a hole in the question. - A binding gives a value for every required slot. An optional slot left out takes its default, or stays out of the binding.
- A slot can't use a type in the
entity_profilenamespace. Those describe an organisation (size, headcount) and are never an answer input.
When you propose a Question through the API you give slots as strings: name:type, or just
name when the name is a type, or is the namespace of exactly one input type (jurisdiction
becomes us_state).
Slot type registry¶
A slot type says what a binding value can be. The registry is the platform's types plus the brain's own.
- Platform types ship with Polaris in
app/eval/slot_types/platform.yaml:us_state,corpus_refanddate. - Brain types live in the brain's store, in
slot_types.yaml. A brain type can't reuse a platform type's name.
Both files map a type name to its definition:
loan_product:
kind: parameter
base: enum
namespace: product
description: The loan product the question is about.
values: [personal_loan, auto_loan, heloc]
status: active
proposed_by: Dana Reyes
approved_by: Sam Okafor
| Field | Required | What it is |
|---|---|---|
| (the key) | yes | The type's name: a lower-case letter, then 1 to 40 lower-case letters, digits or _. |
kind |
yes | parameter (changes which rules apply) or applicability (whether the Question applies at all). |
base |
yes | enum, string, corpus_ref, number or date. |
namespace |
yes | How the value segments an organisation's data: jurisdiction, product, timing, and so on. entity_profile types are never answer inputs. |
description |
no | What the value is, in words. |
values |
for enum |
The allowed values. Matching ignores case and returns the listed spelling. |
provision_key_prefix |
no | Set when a value selects provisions: with us-, the value US-TX selects provisions keyed us-tx:…. |
status |
no | active (default) or proposed. Nothing can use a proposed type. |
proposed_by |
brain types | Who proposed it. |
approved_by |
active brain types | Who approved it. A brain's active type must record this. |
How each base checks a value: enum must be one of values; date must be an ISO date
(YYYY-MM-DD); number must parse as a number; string must not be empty; corpus_ref must not
be empty and, where the corpus is checked, must be something the corpus names.
Adding a brain type. Propose it, then have a different person approve it. The proposer can't approve their own type, because a slot type changes what every organisation's context can say. At the CLI:
bb question slot-type propose loan_product --base enum --namespace product \
--value personal_loan --value auto_loan --value heloc \
--description "The loan product the question is about." --by "Dana Reyes" --brain-id acme
bb question slot-type approve loan_product --approved-by "Sam Okafor" --brain-id acme
Approval needs an interactive terminal. Read the active types with
GET /v1/brains/{brain_id}/slot-types (scope questions:read) or bb question slot-type list.