Skip to content

v0.8.0 — 2026-11-10

The Questions Framework: versioned, approval-locked Questions and Answer Keys, live evals with budgets, certification and publishing. You can drive it from a new Questions workspace in the app, or as a developer through a REST API and a remote MCP endpoint. Alongside it: Advisor question routing, tenant-chosen model providers with your own keys, and a Sentinel that flags changes in meaning rather than changes in bytes.

Added

  • Questions and Answer Keys (DEE-123 to DEE-141). A Question is a template with slots (bb question draft turns an expert's input into a Question and a v0 Answer Key). An Answer Key is a versioned rubric whose criteria cite provisions. A named Curator or Admin approves it, after which it is locked. Slot types live in a registry (bb question slot-type list/propose/approve). Questions and Answer Keys are stored per brain. When Sentinel confirms a change, the Answer Keys that depend on it go up for review. See the Questions Framework.
  • Live evals, certification and publishing (DEE-143 to DEE-161).
    • Eval runs are persisted with the Advisor's claims, its citations and the judge's reasons.
    • Each brain has a live-eval budget and a run cap for each call. Runs are priced before they start, and they run as background ops that report progress.
    • You can schedule re-evaluation, and a re-evaluation queue holds the work.
    • Certification needs a sign-off. Only signed results are trusted, and an open review blocks certification.
    • A publish view lists what is ready to publish.
    • People can submit grades, and the judge's agreement with them is tracked.
  • The Questions workspace in the app (DEE-153, DEE-158 to DEE-161). It holds the catalog, a page for each Question, an approvals inbox, a structured rubric editor, and "Promote to Question" on an Advisor answer. It also shows quality: the chart, the runs, and each answer beside its score. Re-evaluation, schedules and certification sign-off are there as well. It sits behind questions_workspace.
  • The Questions Framework developer API (DEE-417). Outside developers can run the whole Question lifecycle with an API key:

    • Read and match Questions, propose them, and draft Answer Keys.
    • Request approval, or give it with a Curator or Admin key.
    • Start live evals, or submit their own answers, then read standing.
    • Sign off certification, publish Question versions and edit slots.

    It is served over REST under /v1/brains/{brain_id}/… and over a remote MCP endpoint at /v1/mcp/questions, with the same scopes on both. List routes page by cursor. Writes take an Idempotency-Key header. The published schema is docs/api/openapi.json. It sits behind qf_developer_api (with question_store_db). See the walkthrough and the API reference. - New API key scopes: questions:read, questions:write, eval:run, eval:submit and grades:submit. There are also four Curator scopes, which need a Curator or Admin key: answer_key:approve, certification:sign_off, questions:publish and slots:write. - Advisor question routing and slots (DEE-399). The Advisor matches a question to a published Question. It asks for the slot values that the Question's Answer Keys depend on, then answers once it has them. Callers get a typed needs_input and resume with a token. It sits behind advisor_question_routing. See Question routing. - Model providers and bring-your-own-key (DEE-163). A tenant Admin chooses the provider and model for each model role, and can bring the tenant's own Together or OpenRouter keys. It sits behind tenant_model_choice. - Sentinel significance judging. Sentinel now asks whether the knowledge changed, not whether the bytes did. It strips its own provenance header, masks volatile page furniture, and treats an interstitial as a failed fetch. A judge ranks how much a change matters. Flags are keyed on the change, so one unchanged condition no longer mints a new flag every crawl. The curator can see why a flag is ranked where it is. It sits behind sentinel_significance. See Sentinel. - Scoping a question to a subject, over both /chat/completions and MCP. See OpenAI API and MCP. - Tenant hosting: - ECR images built and pushed by push-images.sh, with registry-aware compose image refs (POLARIS_IMAGE_PREFIX, POLARIS_IMAGE_TAG). - Optional HTTP Basic auth on /docs/. - A per-tenant landing page. - The docs site is built during host-configure.

Changed

  • Embeddings go through OpenRouter (EMBEDDING_MODEL=openrouter:openai/text-embedding-3-small, EMBEDDING_DIM=1024), because Together no longer serves embeddings. The width is requested explicitly, and a vector of the wrong width raises an error instead of being written.
  • The extractor model is a config choice. EXTRACTOR_MODEL accepts openrouter: ids, including ids that contain a colon.
  • The Advisor needs the brain's own OpenRouter key (DEE-399). See the upgrade notes.
  • Answer Key approval takes one person (DEE-393). Any Curator or Admin may approve, including their own draft. Agents, bots and system identities are refused.
  • A disabled Questions Framework feature returns 403 feature_disabled instead of 404, so it can't be mistaken for a missing Question.
  • Validation errors use the gateway's error envelope: 422 invalid_request.
  • Advisor failures are coded errors. /chat/completions and the Advisor MCP answer tool return model_provider_out_of_credit, model_provider_rate_limited, model_provider_key_rejected (503, with Retry-After), model_provider_error or advisor_error (502), each with a correlation id, instead of the exception's text.

Fixed

  • The operator MCP Questions tools act on the key's brain. A read-only key can no longer spend through metered tools, and metered spend now counts against the monthly budget.
  • A Curator act through an API key re-checks the key's actor every time. A key minted by a platform operator, a key whose minter was deactivated or demoted, and a key whose actor is an agent or bot are all refused.
  • Eval ops are private to the API key that started them.
  • An out-of-range pagination cursor returns 400, not 500.
  • An Idempotency-Key retry after a cancelled request doesn't act twice.
  • The evidence panel agrees with the score above it.
  • The judge no longer forms a confident verdict on content it can't see.
  • Indexed documents are dated by when they last changed.

Security

  • Advisor errors no longer return the provider's response body or internal model names.
  • A key's scopes are validated when it is minted. Unknown scopes, and Curator scopes on a Reader key, are refused.

Upgrade notes

  • Run migrations 0025 to 0031: alembic upgrade head. They add Sentinel recurrence and significance, Answer Key review, the Questions Framework and workspace tables, eval answer evidence, and provider credentials. On a tenant host, systemctl restart polaris-tenant runs them.
  • Give every brain an OpenRouter key before it uses the Advisor. A brain with no OpenRouter key of its own gets a BLOCK (advisor.openrouter_key_required) on every Advisor call. A tenant Admin adds it under the brain's model providers. Evaluation runs are unaffected.
  • openrouter_api_key is now required for embeddings, as well as for the Advisor.
  • Embeddings come from a different model. Vectors written before this release came from Together's model. Mixing them with text-embedding-3-small vectors makes similarity between old and new rows meaningless. Plan to re-extract documents (or re-embed them) on brains that matter.
  • Set PYTHONHASHSEED=0 on the app tier. Compose sets it, and so does dev.sh. A live Questions eval refuses to run without it (seed_unpinned).
  • Redis data now lives in the tenant data dir (${POLARIS_DATA_DIR}/redis). Before this, it was an anonymous volume. The idempotency markers in the old volume are not carried over.
  • MinIO is pulled from quay.io, pinned to RELEASE.2025-09-07T16-13-09Z, because Docker Hub stopped serving anonymous pulls.
  • New flags. These are on by default: curator_contested_candidates and retrieve_subtree_quotes. These are off by default:

    • topic_cards, answer_key_review, question_catalog and question_store_db.
    • questions_workspace, qf_developer_api, question_slot_editing and quality_slot_facts.
    • advisor_question_routing, tenant_model_choice and sentinel_significance.
    • The other new retrieve_* flags.

    Run bb seed-flags --brain-id <id> after upgrading. - API changes that break existing callers: - GET /v1/brains/{id}/questions returns a page, {items, next_cursor}. Each item has question_id (it was id), and status, intent and slots are objects. - GET /questions/{id} returns question_id. - POST /questions/match validates {question, topic_ids}. - The catalog and coverage routes now accept Readers, and keys with questions:read. - Keys minted with bb apikey mint act as their --actor. For a Curator act, pass a real Curator's login email.