v0.8.0 — 2026-11-10¶
The Questions Framework: versioned, approval-locked Questions and Answer Keys, live evals with budgets, certification and publishing. You can drive it from a new Questions workspace in the app, or as a developer through a REST API and a remote MCP endpoint. Alongside it: Advisor question routing, tenant-chosen model providers with your own keys, and a Sentinel that flags changes in meaning rather than changes in bytes.
Added¶
- Questions and Answer Keys (DEE-123 to DEE-141). A Question is a template with slots (
bb question draftturns an expert's input into a Question and a v0 Answer Key). An Answer Key is a versioned rubric whose criteria cite provisions. A named Curator or Admin approves it, after which it is locked. Slot types live in a registry (bb question slot-type list/propose/approve). Questions and Answer Keys are stored per brain. When Sentinel confirms a change, the Answer Keys that depend on it go up for review. See the Questions Framework. - Live evals, certification and publishing (DEE-143 to DEE-161).
- Eval runs are persisted with the Advisor's claims, its citations and the judge's reasons.
- Each brain has a live-eval budget and a run cap for each call. Runs are priced before they start, and they run as background ops that report progress.
- You can schedule re-evaluation, and a re-evaluation queue holds the work.
- Certification needs a sign-off. Only signed results are trusted, and an open review blocks certification.
- A publish view lists what is ready to publish.
- People can submit grades, and the judge's agreement with them is tracked.
- The Questions workspace in the app (DEE-153, DEE-158 to DEE-161). It holds the catalog, a page for each Question, an approvals inbox, a structured rubric editor, and "Promote to Question" on an Advisor answer. It also shows quality: the chart, the runs, and each answer beside its score. Re-evaluation, schedules and certification sign-off are there as well. It sits behind
questions_workspace. -
The Questions Framework developer API (DEE-417). Outside developers can run the whole Question lifecycle with an API key:
- Read and match Questions, propose them, and draft Answer Keys.
- Request approval, or give it with a Curator or Admin key.
- Start live evals, or submit their own answers, then read standing.
- Sign off certification, publish Question versions and edit slots.
It is served over REST under
/v1/brains/{brain_id}/…and over a remote MCP endpoint at/v1/mcp/questions, with the same scopes on both. List routes page by cursor. Writes take anIdempotency-Keyheader. The published schema isdocs/api/openapi.json. It sits behindqf_developer_api(withquestion_store_db). See the walkthrough and the API reference. - New API key scopes:questions:read,questions:write,eval:run,eval:submitandgrades:submit. There are also four Curator scopes, which need a Curator or Admin key:answer_key:approve,certification:sign_off,questions:publishandslots:write. - Advisor question routing and slots (DEE-399). The Advisor matches a question to a published Question. It asks for the slot values that the Question's Answer Keys depend on, then answers once it has them. Callers get a typedneeds_inputand resume with a token. It sits behindadvisor_question_routing. See Question routing. - Model providers and bring-your-own-key (DEE-163). A tenant Admin chooses the provider and model for each model role, and can bring the tenant's own Together or OpenRouter keys. It sits behindtenant_model_choice. - Sentinel significance judging. Sentinel now asks whether the knowledge changed, not whether the bytes did. It strips its own provenance header, masks volatile page furniture, and treats an interstitial as a failed fetch. A judge ranks how much a change matters. Flags are keyed on the change, so one unchanged condition no longer mints a new flag every crawl. The curator can see why a flag is ranked where it is. It sits behindsentinel_significance. See Sentinel. - Scoping a question to a subject, over both/chat/completionsand MCP. See OpenAI API and MCP. - Tenant hosting: - ECR images built and pushed bypush-images.sh, with registry-aware compose image refs (POLARIS_IMAGE_PREFIX,POLARIS_IMAGE_TAG). - Optional HTTP Basic auth on/docs/. - A per-tenant landing page. - The docs site is built duringhost-configure.
Changed¶
- Embeddings go through OpenRouter (
EMBEDDING_MODEL=openrouter:openai/text-embedding-3-small,EMBEDDING_DIM=1024), because Together no longer serves embeddings. The width is requested explicitly, and a vector of the wrong width raises an error instead of being written. - The extractor model is a config choice.
EXTRACTOR_MODELacceptsopenrouter:ids, including ids that contain a colon. - The Advisor needs the brain's own OpenRouter key (DEE-399). See the upgrade notes.
- Answer Key approval takes one person (DEE-393). Any Curator or Admin may approve, including their own draft. Agents, bots and system identities are refused.
- A disabled Questions Framework feature returns
403 feature_disabledinstead of 404, so it can't be mistaken for a missing Question. - Validation errors use the gateway's error envelope:
422 invalid_request. - Advisor failures are coded errors.
/chat/completionsand the Advisor MCPanswertool returnmodel_provider_out_of_credit,model_provider_rate_limited,model_provider_key_rejected(503, withRetry-After),model_provider_errororadvisor_error(502), each with a correlation id, instead of the exception's text.
Fixed¶
- The operator MCP Questions tools act on the key's brain. A read-only key can no longer spend through metered tools, and metered spend now counts against the monthly budget.
- A Curator act through an API key re-checks the key's actor every time. A key minted by a platform operator, a key whose minter was deactivated or demoted, and a key whose actor is an agent or bot are all refused.
- Eval ops are private to the API key that started them.
- An out-of-range pagination cursor returns 400, not 500.
- An Idempotency-Key retry after a cancelled request doesn't act twice.
- The evidence panel agrees with the score above it.
- The judge no longer forms a confident verdict on content it can't see.
- Indexed documents are dated by when they last changed.
Security¶
- Advisor errors no longer return the provider's response body or internal model names.
- A key's scopes are validated when it is minted. Unknown scopes, and Curator scopes on a Reader key, are refused.
Upgrade notes¶
- Run migrations
0025to0031:alembic upgrade head. They add Sentinel recurrence and significance, Answer Key review, the Questions Framework and workspace tables, eval answer evidence, and provider credentials. On a tenant host,systemctl restart polaris-tenantruns them. - Give every brain an OpenRouter key before it uses the Advisor. A brain with no OpenRouter key of its own gets a
BLOCK(advisor.openrouter_key_required) on every Advisor call. A tenant Admin adds it under the brain's model providers. Evaluation runs are unaffected. openrouter_api_keyis now required for embeddings, as well as for the Advisor.- Embeddings come from a different model. Vectors written before this release came from Together's model. Mixing them with
text-embedding-3-smallvectors makes similarity between old and new rows meaningless. Plan to re-extract documents (or re-embed them) on brains that matter. - Set
PYTHONHASHSEED=0on the app tier. Compose sets it, and so doesdev.sh. A live Questions eval refuses to run without it (seed_unpinned). - Redis data now lives in the tenant data dir (
${POLARIS_DATA_DIR}/redis). Before this, it was an anonymous volume. The idempotency markers in the old volume are not carried over. - MinIO is pulled from quay.io, pinned to
RELEASE.2025-09-07T16-13-09Z, because Docker Hub stopped serving anonymous pulls. -
New flags. These are on by default:
curator_contested_candidatesandretrieve_subtree_quotes. These are off by default:topic_cards,answer_key_review,question_catalogandquestion_store_db.questions_workspace,qf_developer_api,question_slot_editingandquality_slot_facts.advisor_question_routing,tenant_model_choiceandsentinel_significance.- The other new
retrieve_*flags.
Run
bb seed-flags --brain-id <id>after upgrading. - API changes that break existing callers: -GET /v1/brains/{id}/questionsreturns a page,{items, next_cursor}. Each item hasquestion_id(it wasid), andstatus,intentandslotsare objects. -GET /questions/{id}returnsquestion_id. -POST /questions/matchvalidates{question, topic_ids}. - The catalog and coverage routes now accept Readers, and keys withquestions:read. - Keys minted withbb apikey mintact as their--actor. For a Curator act, pass a real Curator's login email.