Production Agent Advisor

The Initiative cockpit uses a bounded Agent Advisor. It makes one model request when a Facilitator asks for a question, a gate evaluation, or intervention suggestions. It does not run an autonomous agent loop and it cannot change an Initiative without a Facilitator action.

Vercel AI Gateway

The production API can use Vercel AI Gateway through its OpenAI-compatible endpoint. Set these variables on the Vercel project that hosts Modeller.Api:

Agent__BaseUrl=https://ai-gateway.vercel.sh/v1
Agent__Model=openai/gpt-5.4-mini
Agent__TimeoutSeconds=30
Agent__MaxOutputTokens=1200
Agent__MaxPromptCharacters=24000

By default, the public cockpit requires a caller key. A model can be enabled for anonymous use with Agent__FreeModel, but only after two checks:

1. The live Gateway catalogue lists the exact model with the free tag and no token price.
2. The Vercel team has paid Gateway credits. Vercel blocks restricted free models for teams that have not moved to the paid-credit tier.

For example, Agent__FreeModel=alibaba/qwen3.8-27b met the first check on 17 August 2026 but returned HTTP 403 on the free Vercel tier; it is no longer free. On 24 October 2026 the text models with the free tag were reflection/beam-free, reflection/beam and poolside/laguna-s-2.1-free. Do not set it until the account meets both checks. When enabled, the API authenticates anonymous requests with the deployment's short-lived OIDC token and does not let the browser select a different model.

A Facilitator can enter their own Vercel AI Gateway API key to use the configured premium model. The page keeps the key only in component memory and sends it in the X-Agent-Api-Key header for explicit AI actions. The API does not put it in Redis, logs, analytics, URLs, or the Initiative record.

Keep automatic Gateway credit top-up disabled. Model prices and availability can change. If the configured free model stops being free, remove Agent__FreeModel and redeploy; the cockpit will then require caller keys.

Apply the non-secret variables to Production and Preview only where AI access is required, then redeploy the API.

openai/gpt-5.4-mini is the initial production choice because the advice is short and structured. The model remains an environment setting so it can change without a code deployment.

Local model in the browser

A Facilitator can also run the Agent Participant on their own device, with no API key. The cockpit offers this when the browser supports WebGPU (current Chrome or Edge on a desktop computer) and gives the site a GPU. Before it offers the download, the cockpit asks the browser for a GPU adapter. When there is none (the browser fell back to software after a GPU process crash, graphics acceleration is off, the GPU or driver is blocked, or the session is a remote desktop), the panel shows the steps to fix it.

• The first start downloads Llama 3.2 3B Instruct (about 1.8 GB) from Hugging Face, after the Facilitator agrees. The browser keeps it, so later visits start from the cache. It needs about 2.3 GB of GPU memory.
• The model runs in a Web Worker on the Facilitator's GPU. The Initiative text is not sent to an AI service. Only the result (a question, a selected intervention, or gate results) goes to the API, the same as manual input.
• Licence: Llama 3.2 Community License. The cockpit shows "Built with Llama" with a link to the licence; keep that notice if the panel changes.
• It uses the same prompts as the hosted model: apps/website/src/lib/localAgent/agent-prompts.json must be an exact copy of src/Modeller.Initiative.OpenAICompatible/agent-prompts.json, and a test fails if they differ. Change the canonical file, then copy it.
• WebLLM forces each answer to match a JSON schema, and the cockpit checks it with the same rules as the API.
• The local gate is a hybrid. The browser decides every "is there content?" check exactly (for example AffectedUsersNamed and RisksAreListed), and asks the model only for the judgement checks (for example PainPointsAreConcrete). The recommended next question is the example question of the first field with a gap. The Gateway gate is unchanged.
• Storage: the model is stored in the browser profile, not in a folder the site can choose. When the browser is set to clear site data on close (Chrome: "Clear cookies and site data when you close all windows"; Edge: "Choose what to clear every time you close the browser"), and in private windows, each site can store only about 300 MB, however much disk space is free (web-llm #374). The cockpit checks the quota before the download and tells the Facilitator how to add modeller.website as an exception. A failed download is deleted so it does not use the quota.
• The local prompt limit is 8,000 characters (the model has a 4,096-token context). A longer Initiative shows a message that suggests the AI Gateway.
• When the Gateway is also available, the Facilitator chooses which one to use. The local model is weaker: for example, its Non-goals questions often ask about the current process instead of the scope.
• The website CSP allows https://huggingface.co, https://*.hf.co and https://raw.githubusercontent.com in connect-src, 'wasm-unsafe-eval' in script-src, and worker-src 'self' blob:. If the model download fails after a CSP change, check these first.

Why this model

On 24 October 2026 the shared prompts were compared on the building-variations worked example, three samples per model. Llama 3.2 3B asked 1 solution-led question out of 24, Qwen3 1.7B 10, and Qwen2.5 1.5B 11. Qwen3 4B was not better and is larger. Qwen2.5 3B performed well but has a non-commercial licence. No small model judged the gate reliably, which is why the presence checks are computed exactly.

The same comparison changed the shared prompts, so they also apply to the Gateway model: the original change request is labelled as a proposed solution that is not yet agreed, the question prompt describes the target field, interventions must include at least three options with a NoAction baseline, and a failed gate check must come with a recommended question.

Cost and request controls

• The API limits each model response to 1,200 tokens.
• The API rejects Initiative prompts over 24,000 characters before it calls the provider.
• The API cancels a model request after 30 seconds.
• Configure a small AI Gateway budget and leave automatic top-up disabled until normal usage is known.
• Review usage by model in the AI Gateway dashboard. The API records model, endpoint, status, duration, and token counts through tracing. It does not record prompt or response content.

Failure and recovery

The cockpit shows whether the Agent Advisor is available. If it is not configured, AI controls are disabled and all manual paths remain available. A provider error appears as an initiative.agent.* error and does not change the Initiative.

To recover:

1. Check the API deployment logs for the HTTP status and trace. Do not log or paste prompt content.
2. Check AI Gateway usage, credit balance, and model availability.
3. Confirm Agent__BaseUrl and Agent__Model. If anonymous free access is enabled, also confirm the live price and Agent__FreeModel.
4. Change Agent__Model to another available text model if needed, then redeploy.
5. Remove Agent__BaseUrl to disable AI safely while manual facilitation continues.

For local development, use .NET user secrets or a local environment file that Git ignores. Automated tests use fake HTTP handlers and need no provider credentials.