Skip to content

Schemas

The formats behind the Questions Framework: a Question, an Answer Key, a slot declaration and the slot type registry. For what each one means, see the concepts.

Where the files live

Each brain has its own store: a git repository beside its corpus repository, named <brain_id>.answer-keys/. Questions and Answer Keys are YAML files in it, and every draft, approval and migration is a commit attributed to whoever made it.

<brain_id>.answer-keys/
    questions/<question_id>.yaml          published Questions
    questions/history/<id>.vN.yaml        Question versions a newer one replaced
    answer_keys/<id>.yaml                 approved Answer Keys, locked by approval.content_sha
    answer_keys/history/<id>.vN.yaml      Answer Key versions a newer approval replaced
    drafts/<id>/question.yaml             a draft Question
    drafts/<id>/answer_key.yaml           a draft Answer Key
    drafts/<id>/review.json               the draft's gate results, review and rejection
    register/<id>.yaml                    what has been tried against each Answer Key
    slot_types.yaml                       the brain's own slot types
    shape_vocabulary.yaml                 the brain's own answer-shape elements

You don't need to write these files by hand. The API, the MCP tools and the web app write them for you. This page is for reading what they return, and for teams that keep Questions in git.

Question

A Question file, as the store holds it:

id: Q-late-fee-cap
version: 1
status: published               # draft | published | retired
text: Is a late fee capped in {jurisdiction}?
intent: Decide whether our late fee needs a state-specific limit.
topic: fees
slots:
  - name: jurisdiction
    type: us_state
    required: true
    ask_text: Which state is the loan made in?
required_provisions:
  always: ["1026.52(b)"]
  by_binding:
    jurisdiction=US-TX: ["us-tx:342.201"]
answer_shape:
  must_include: [dollar_amount, citation]
  valid_refusals: [coverage_missing_for_bound_slot]
answer_keys: [late-fee-cap-us-tx]
matching:
  aliases: ["What's the maximum late fee in {jurisdiction}?"]
  negative_examples: ["Can we charge a returned-payment fee?"]
Field Required What it is
id yes The Question's id. Polaris names new ones Q-<slug>.
version yes A positive integer. Changing the text or slots of a published Question makes a new version.
status yes draft, published or retired. A Question is published when its first Answer Key is approved.
text yes The question as people ask it. Each required slot appears as {name}, and nothing else may.
intent no What the asker needs to decide, in a sentence. The API requires it when you propose a Question.
topic no The topic card the Question belongs to. Matching narrows to a topic first. The API requires it when you propose a Question.
slots no The slot declarations. See Slot declaration.
required_provisions no The provisions an answer must rest on: always, plus more per binding under by_binding (keyed by the binding key). Polaris adds every provision an approved criterion cites.
answer_shape no What kind of answer is valid: must_include and must_not_include name shape elements; valid_refusals names refusals that count as a correct answer. Checked on live answers, where it annotates and never blocks.
answer_keys no The ids of the Question's Answer Keys, one per binding.
matching no aliases (other phrasings that should match) and negative_examples (near misses that must not).
certification no Raises the certification threshold for this Question: runs, min_median_standing, judged_graded, no_penalty_cap. It can only raise the defaults (5, 0.8, true, true), never lower them.
history no Written by Polaris when a new version is published: each retired version and why.

The retired field name cases is refused; use answer_keys.

Over the API a Question comes back as JSON with question_id in place of id, its Answer Keys inlined, and certified_bindings (the binding keys certified at the published brain version). The API reference has the response models.

Answer Key

An Answer Key is a YAML file in the store. The API returns its rubric as JSON in a different shape, described by a JSON Schema: answer_key.schema.json. The two carry the same content, and converting one to the other keeps the approval hash.

On disk (YAML) In the API (JSON Schema)
question question_text_resolved
gold answer
gold_when full_credit_when
criterion matches criterion pattern
judged: true / manual: true on a criterion criterion tier: judged / manual (else machine)
verdicts, default_verdict, refuses, penalties, report, fabricates inside scoring
approval.approved_by, approval.approved_at, approval.content_sha approved_by, approved_at, content_sha
any other top-level key, such as a draft's draft block inside extensions

The JSON form also carries fields that aren't stored in the file: question_version, status (active, superseded, needs_review) and last_scored, the latest score with its comparability tuple. Only a Curator or Admin sees a rubric's substance (answer, criteria, full_credit_when, scoring, extensions). Everyone else gets the Answer Key without them.

An approved Answer Key on disk:

id: late-fee-cap-us-tx
version: 1
question_id: Q-late-fee-cap
binding: {jurisdiction: US-TX}
approval:
  approved_by: Dana Reyes
  approved_at: "2026-11-02"
  content_sha: "sha256:…"
question: Is a late fee capped in Texas?
gold: >-
  Yes. …
verdicts:
  - name: NOT_CAPPED
    matches: "no (cap|limit) on"
  - name: CAPPED
    matches: "\\bcap(ped)?\\b|may not exceed"
default_verdict: UNCLEAR
criteria:
  - key: A1_cap_named
    weight: 40
    describe: names the dollar cap
    matches: "\\$\\s?\\d"
    provisions: ["us-tx:342.201"]
  - key: A2_cap_correct
    weight: 60
    judged: true
    describe: states the cap and when it applies correctly
    provisions: ["us-tx:342.201"]
gold_when: [A1_cap_named]
penalties:
  - cap: 20
    verdict: NOT_CAPPED
    describe: says there is no cap

Top-level fields

Field Required What it is
id yes The Answer Key's id.
question yes The question at this binding, slots filled in.
criteria yes The rubric: 1 to 60 criteria, each key unique.
version no A positive integer, default 1.
question_id no The Question it belongs to. Absent means a standalone Answer Key with no slots.
binding no Slot name to value. {} for a Question with no slots.
gold no The expert's reference answer, word for word.
verdicts no Ordered {name, matches} pairs; the first whose pattern matches the answer is its verdict.
default_verdict no The verdict when none matches. Default COND.
refuses no A pattern that marks the answer as a refusal.
penalties no Caps applied after scoring: {cap, verdict?, check?, describe?}. A cap limits the score; it doesn't subtract.
report no Name to pattern. Printed with a result, never scored.
gold_when no Machine criteria that together mean full credit. Judged criteria never belong here.
fabricates no {subject, claims}: patterns for an answer that asserts the subject is something it isn't. Hedged phrasings are ignored.
supersedes no The version this one replaces. Needs a change_reason.
change_reason no {kind, note?, effective_date?, triggered_by?}. kind is law_changed, new_adjudication, scope_refined or error_corrected.
approval no {approved_by, approved_at, content_sha}. Written by approval; a draft must not have one.
draft no On drafts only: who drafted it, with which model, from what source.

Criterion fields

Field Required What it is
key yes Unique within the Answer Key.
weight yes A whole number of points, 0 or more.
describe no What the criterion checks, in words. A judged criterion's judge reads it.
matches no Scored when this pattern matches the answer.
absent no Scored when this pattern does not match. Use it for "must not say X".
verdict no Scored when the answer's verdict equals this.
check / negate_check no Scored by a named code check (or by it being false).
judged no true: scored by the judge. A judged criterion takes no matches, absent, verdict or check.
manual no true: graded by a person only, never counted in an automated score. Can't be combined with judged.
provisions no The provision keys the criterion rests on, such as 1002.9(a)(1).

When a criterion has more than one scorer, verdict wins, then check, then absent, then matches. Patterns are case-insensitive regular expressions of at most 500 characters. A literal double backslash is refused, because in a YAML folded string (>-) it is almost always a pattern that can never match.

The approval lock

approval.content_sha is sha256: plus the hash of every top-level field except approval, serialised as sorted JSON. YAML comments aren't covered, so notes in a file's header stay editable. Change a pattern, weight, criterion, gold, question or version, and the lock breaks. To change what counts as correct, write a new version with supersedes and a change_reason, and have it approved. The version it replaces is kept in answer_keys/history/.

Slot declaration

Each entry in a Question's slots has exactly these fields:

Field Required What it is
name yes A lower-case letter, then lower-case letters, digits or _; at most 41 characters. Unique in the Question.
type yes An active slot type's name.
required no true (the default) or false.
ask_text no What the Advisor asks a person who didn't say this. It can change on a published Question without a new version.
default no Optional slots only: the value an unbound Answer Key takes. Must be a valid value of the type.

Rules:

  • Every required slot appears in text as {name}. An optional slot never does, since leaving it unbound would leave a hole in the question.
  • A binding gives a value for every required slot. An optional slot left out takes its default, or stays out of the binding.
  • A slot can't use a type in the entity_profile namespace. Those describe an organisation (size, headcount) and are never an answer input.

When you propose a Question through the API you give slots as strings: name:type, or just name when the name is a type, or is the namespace of exactly one input type (jurisdiction becomes us_state).

Slot type registry

A slot type says what a binding value can be. The registry is the platform's types plus the brain's own.

  • Platform types ship with Polaris in app/eval/slot_types/platform.yaml: us_state, corpus_ref and date.
  • Brain types live in the brain's store, in slot_types.yaml. A brain type can't reuse a platform type's name.

Both files map a type name to its definition:

loan_product:
  kind: parameter
  base: enum
  namespace: product
  description: The loan product the question is about.
  values: [personal_loan, auto_loan, heloc]
  status: active
  proposed_by: Dana Reyes
  approved_by: Sam Okafor
Field Required What it is
(the key) yes The type's name: a lower-case letter, then 1 to 40 lower-case letters, digits or _.
kind yes parameter (changes which rules apply) or applicability (whether the Question applies at all).
base yes enum, string, corpus_ref, number or date.
namespace yes How the value segments an organisation's data: jurisdiction, product, timing, and so on. entity_profile types are never answer inputs.
description no What the value is, in words.
values for enum The allowed values. Matching ignores case and returns the listed spelling.
provision_key_prefix no Set when a value selects provisions: with us-, the value US-TX selects provisions keyed us-tx:….
status no active (default) or proposed. Nothing can use a proposed type.
proposed_by brain types Who proposed it.
approved_by active brain types Who approved it. A brain's active type must record this.

How each base checks a value: enum must be one of values; date must be an ISO date (YYYY-MM-DD); number must parse as a number; string must not be empty; corpus_ref must not be empty and, where the corpus is checked, must be something the corpus names.

Adding a brain type. Propose it, then have a different person approve it. The proposer can't approve their own type, because a slot type changes what every organisation's context can say. At the CLI:

bb question slot-type propose loan_product --base enum --namespace product \
  --value personal_loan --value auto_loan --value heloc \
  --description "The loan product the question is about." --by "Dana Reyes" --brain-id acme
bb question slot-type approve loan_product --approved-by "Sam Okafor" --brain-id acme

Approval needs an interactive terminal. Read the active types with GET /v1/brains/{brain_id}/slot-types (scope questions:read) or bb question slot-type list.