Add a model provider

Most model additions describe an endpoint Pi already knows how to call. Start with ~/.pi/agent/models.json or an Extension ProviderConfig; build a native Provider when you need provider-owned authentication or discovery; implement ProviderStreams only for a genuinely new wire protocol.

Choose the integration route

Choose exactly one route: complete sections 1–2 for a static catalog, 2–3 for Extension discovery, 2 and 4 for a native provider, or 2, 4, and 5 for a new wire protocol. When that route is complete, skip to 6. Select and inspect the model and 7. Probe streaming, thinking, and Tools.

Do not revive the old process-global model/translator registry. Current applications own a Models collection; Coding Agent's public implementation is ModelRuntime. Built-in provider factories live under @earendil-works/pi-ai/providers/*, while API factories use @earendil-works/pi-ai/api/* subpath exports.

Prerequisites

Pi 0.85.0 requires Node.js >=22.19.0. For the TypeScript examples, use ESM and install each package you import:

npm init -y
npm pkg set type=module
npm install @earendil-works/pi-ai@0.85.0 @earendil-works/pi-coding-agent@0.85.0 @earendil-works/pi-server@0.85.0
npm install --save-dev typescript tsx @types/node

Pi 0.85.0 packaging workaround: the published Coding Agent manifest omits this runtime dependency even though its public root export loads it. This workaround is release-scoped, not a permanent dependency rule for later Pi versions.

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "noEmit": true,
    "skipLibCheck": true,
    "types": ["node"]
  },
  "include": ["**/*.ts"]
}

Start the OpenAI-compatible server at http://127.0.0.1:1234/v1 and confirm its model ID with GET /models. Substitute its real URL, ID, limits, and capabilities below.

1. Add a static OpenAI-compatible catalog

Create ~/.pi/agent/models.json. This is the smallest integration that survives restarts and remains editable without compiling an Extension:

{
  "providers": {
    "local-openai": {
      "name": "Local OpenAI",
      "baseUrl": "http://127.0.0.1:1234/v1",
      "apiKey": "$LOCAL_OPENAI_API_KEY",
      "api": "openai-completions",
      "models": [
        {
          "id": "local-model",
          "name": "Local Model",
          "reasoning": false,
          "input": ["text"],
          "cost": {
            "input": 0,
            "output": 0,
            "cacheRead": 0,
            "cacheWrite": 0
          },
          "contextWindow": 32768,
          "maxTokens": 4096,
          "compat": {
            "supportsDeveloperRole": false,
            "supportsReasoningEffort": false,
            "supportsUsageInStreaming": false,
            "supportsFinishReason": false,
            "supportsStrictMode": false,
            "maxTokensField": "max_tokens"
          }
        }
      ]
    }
  }
}

Set the credential in the process that starts Pi:

export LOCAL_OPENAI_API_KEY="replace-me"
pi --list-models local-openai

apiKey accepts a literal, $ENV_VAR, ${ENV_VAR}, or a leading !command. Prefer an environment variable or /login; command-based resolution executes a local program and should only use a trusted, fixed command. A keyless local server still needs configured auth before its models become available: use a non-secret placeholder, store a key with /login, or pass --api-key for that invocation.

Opening /model reloads models.json, so static edits do not require a restart. ModelConfig parses this file internally, but it is not a public package export; use ModelRuntime.create({ modelsPath }) when an SDK application needs a custom path.

2. Record metadata accurately

Pi uses model metadata for selection, validation, request shaping, usage, and display. Do not copy a nearby model merely because the endpoint accepts OpenAI-shaped JSON.

Common completions compatibility switches cover developer roles, reasoning_effort, streaming usage and finish_reason, the max-token field, strict/grammar Tools, replay rules for Tool results or reasoning content, thinking format, caching, routing, and session affinity. Set only flags you can demonstrate against the server. Metadata has no general streaming or toolUse boolean: every Provider streams, and Tool support is proven by an actual Tool call.

3. Discover models with provider config

Use an async Extension when the endpoint's model list changes. Validate the untrusted response and pass the supplied signal to fetch. The returned list replaces this Extension's models; if refresh throws, Pi keeps the previous in-memory list.

import type {
  ExtensionAPI,
  ProviderModelConfig,
} from "@earendil-works/pi-coding-agent";

interface ModelListItem {
  id: string;
}

function isModelListItem(value: unknown): value is ModelListItem {
  return (
    typeof value === "object" &&
    value !== null &&
    typeof (value as { id?: unknown }).id === "string" &&
    (value as { id: string }).id.trim().length > 0 &&
    (value as { id: string }).id.length <= 256
  );
}

async function discover(signal: AbortSignal): Promise<ProviderModelConfig[]> {
  const response = await fetch("http://127.0.0.1:1234/v1/models", {
    signal,
  });
  if (!response.ok) {
    throw new Error(`Model discovery failed: HTTP ${response.status}`);
  }

  const body: unknown = await response.json();
  const data =
    typeof body === "object" && body !== null
      ? (body as { data?: unknown }).data
      : undefined;
  if (
    !Array.isArray(data) ||
    data.length > 10_000 ||
    !data.every(isModelListItem)
  ) {
    throw new Error("Model discovery returned an invalid data array");
  }

  return data.map((item) => ({
    id: item.id,
    name: item.id,
    reasoning: false,
    input: ["text"],
    cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
    contextWindow: 32768,
    maxTokens: 4096,
    compat: {
      supportsDeveloperRole: false,
      supportsReasoningEffort: false,
      supportsUsageInStreaming: false,
      supportsFinishReason: false,
      supportsStrictMode: false,
      maxTokensField: "max_tokens",
    },
  }));
}

export default function discoverLocalModels(pi: ExtensionAPI) {
  const provider = "local-openai";
  const api = "openai-completions";
  const baseUrl = "http://127.0.0.1:1234/v1";

  pi.registerProvider("local-openai", {
    name: "Local OpenAI",
    baseUrl,
    apiKey: "$LOCAL_OPENAI_API_KEY",
    api,
    refreshModels: async (context) => {
      const stored =
        context.stored?.models.filter((model) => model.provider === provider) ??
        [];
      if (!context.allowNetwork) return [...stored];

      try {
        const discovered = await discover(context.signal);
        await context.publish({
          persist: {
            models: discovered.map((model) => ({
              ...model,
              provider,
              api: model.api ?? api,
              baseUrl: model.baseUrl ?? baseUrl,
            })),
            checkedAt: Date.now(),
          },
        });
        return discovered;
      } catch (error) {
        if (stored.length > 0) return [...stored];
        throw error;
      }
    },
  });
}

ProviderConfig.refreshModels() must opt into cross-session storage with the published one-argument contract await context.publish({ persist: entry }); the example persists fully resolved Model objects and returns that cache when networking is disabled or fails. A native createProvider({ fetchModels }) restores and publishes its dynamic overlay automatically.

Call await models.refresh({ allowNetwork: true, force: true, signal }) for provider-owned remote catalogs outside an Extension. Coding Agent also offers pi update --models. PI_OFFLINE disables model network access. Refresh is optional; static providers need no refresh method.

4. Build a native Provider

Use createProvider() when the provider must own authentication, filter models by credential, or dispatch one or more Pi APIs. This example reuses the published OpenAI Completions adapter.

import {
  createProvider,
  envApiKeyAuth,
  type Model,
} from "@earendil-works/pi-ai";
import { openAICompletionsApi } from "@earendil-works/pi-ai/api/openai-completions.lazy";
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";

const models: readonly Model<"openai-completions">[] = [
  {
    id: "local-model",
    name: "Local Model",
    provider: "local-native",
    api: "openai-completions",
    baseUrl: "http://127.0.0.1:1234/v1",
    reasoning: false,
    input: ["text"],
    cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
    contextWindow: 32768,
    maxTokens: 4096,
    compat: {
      supportsDeveloperRole: false,
      supportsReasoningEffort: false,
      supportsUsageInStreaming: false,
      supportsFinishReason: false,
      supportsStrictMode: false,
      maxTokensField: "max_tokens",
    },
  },
];

const provider = createProvider({
  id: "local-native",
  name: "Local Native",
  baseUrl: "http://127.0.0.1:1234/v1",
  auth: {
    apiKey: envApiKeyAuth("Local OpenAI API key", [
      "LOCAL_OPENAI_API_KEY",
    ]),
  },
  models,
  api: openAICompletionsApi(),
});

export default function nativeLocalProvider(pi: ExtensionAPI) {
  pi.registerProvider(provider);
}

envApiKeyAuth() checks a stored credential first, then the listed environment variables. For a custom resolver, implement the public ApiKeyAuth.resolve({ ctx, credential, signal }) method and read environment values through ctx.env(). Its AuthResult can return request auth, provider-scoped env, and a source label. There is no public AuthResolver type in 0.85.0; do not import or invent one. SDK callers can inspect resolved state with Models.getAuth().

Built-in factories follow the same contract. For example, openaiProvider() is exported from @earendil-works/pi-ai/providers/openai. Use a factory when its catalog, auth, and API mix already match your service; use createProvider() for your own composition.

5. Implement an API adapter only for a new protocol

An API adapter converts Pi Context messages and Tools into the remote payload, then converts the response into one AssistantMessageEventStream. Model metadata stays in Model. This method surface is a reference excerpt, not a runnable adapter:

interface ProviderStreams {
  stream(model, context, options?): AssistantMessageEventStream;
  streamSimple(model, context, options?): AssistantMessageEventStream;
  fetchDeferred?(model, handle, options?): AssistantMessageEventStream;
  cancelDeferred?(model, handle, options?): Promise<void>;
}

Source: pinned ProviderStreams. Parameter types are omitted in the excerpt; import the published interface for the exact signatures.

streamSimple() is the provider-neutral entry point: it maps Pi reasoning levels, toolChoice, and optional thinking budgets before delegating to the adapter. A production adapter must preserve ordered start, indexed text_*, thinking_*, and toolcall_* events and finish with exactly one done or error. It must also report usage, classify context overflow, keep Tool-call IDs stable across replay, invoke request/response hooks, and stop network and parser work when options.signal aborts.

Study a current adapter with the same transport before coding. Do not publish a sketch that parses arbitrary SSE lines or pushes only text deltas: that loses partial JSON Tool arguments, reasoning signatures, usage, finish reasons, error bodies, and abort behavior. Put the finished ProviderStreams object in createProvider({ api }); do not register a global translator.

6. Select and inspect the model

List the catalog, then select by the unambiguous provider/id form:

pi --list-models local-openai
pi --model local-openai/local-model --thinking off "Reply with exactly: provider ready"

In interactive mode, open /model and search for local-openai. The selector reloads models.json and refreshes configured dynamic providers. In an SDK application, resolve through the application's Models instance:

import type { Models } from "@earendil-works/pi-ai";
import { ModelRuntime } from "@earendil-works/pi-coding-agent";

const models: Models = await ModelRuntime.create({
  allowModelNetwork: false,
});
const model = models.getModel("local-openai", "local-model");
if (!model) throw new Error("local-openai/local-model was not loaded");

console.log({
  provider: model.provider,
  id: model.id,
  api: model.api,
  contextWindow: model.contextWindow,
  maxTokens: model.maxTokens,
  reasoning: model.reasoning,
  input: model.input,
  cost: model.cost,
  compat: model.compat,
});

If getModel() succeeds but the model is absent from /model, auth is not configured. Check await models.getAuth(model) or pi auth check --provider local-openai without printing the secret.

7. Probe streaming, thinking, and Tools

Run every check that applies before claiming support:

supportsMidConvoEffort belongs in AnthropicMessagesCompat and defaults to false. For built-in models in the Pi 0.85.0 generated catalog, automatic detection lowercases modelId first, then strips one optional prefix matching ^~?anthropic/ (anthropic/ or ~anthropic/). Pi auto-enables the flag only when provider is exactly anthropic or openrouter. The normalized ID must match exactly ^claude-opus-5(?:-\d{8})?$ or ^claude-(?:fable|mythos)-5(?:[.-]1)(?:-\d{8})?$. The exact supported model must still use a faithful Anthropic Messages transport; this is not support for all Anthropic-compatible providers or an API that merely imitates the Messages shape.

The accepted normalized variants are claude-opus-5, optionally followed by -YYYYMMDD; claude-fable-5.1 or claude-fable-5-1, each optionally dated; and claude-mythos-5.1 or claude-mythos-5-1, each optionally dated.

For a custom Model using anthropic-messages, set compat.supportsMidConvoEffort: true only after verifying both a faithful Anthropic Messages transport and the same exact compatible Claude model family. A provider name outside the generated-catalog allowlist does not by itself forbid manual configuration; never generalize this exception to arbitrary Anthropic-compatible providers or models.

With that flag enabled, Pi persists each response's native provider effort and reconstructs effort-only system messages when it replays the conversation. Pi also sends the thinking binding control prefix_mismatch_behavior: "drop_block"; on a prefix mismatch this safely drops the stale signed thinking block and prevents a persistent 400-response loop.

For vLLM Chat Completions, vllmPriority is a member of OpenAICompletionsCompat. Lower values are handled earlier, and the setting is meaningful only with vLLM --scheduling-policy priority; the server default is 0. It is off by default and not set on the generated model catalog, so configure it explicitly only when the server uses priority scheduling.

For an OpenAI Responses endpoint, supportsMaxOutputTokens is a member of OpenAIResponsesCompat and defaults to true. Set it to false for a gateway that rejects max_output_tokens, causing Pi to omit the field. Do not put this flag on OpenAICompletionsCompat.

The live probe uses the Models.streamSimple() path that applies auth and provider defaults. It prints incremental output and fails on the stream's terminal error message.

import {
  Type,
  type Context,
  type Models,
  type Tool,
} from "@earendil-works/pi-ai";
import { ModelRuntime } from "@earendil-works/pi-coding-agent";

const mode = process.argv[2] ?? "text";
if (!new Set(["text", "thinking", "tool"]).has(mode)) {
  throw new Error("Use: text, thinking, or tool");
}

const models: Models = await ModelRuntime.create({
  allowModelNetwork: false,
});
const model = models.getModel("local-openai", "local-model");
if (!model) throw new Error("local-openai/local-model was not loaded");

const echoTool: Tool = {
  name: "echo_text",
  description: "Return text unchanged. Use when explicitly asked to echo.",
  parameters: Type.Object(
    { text: Type.String() },
    { additionalProperties: false },
  ),
};
const prompts = {
  text: "Reply with exactly: stream ok",
  thinking: "Think briefly, then answer: what is 2 + 2?",
  tool: "Call echo_text once with the text tool ok.",
} as const;
const context: Context = {
  messages: [
    {
      role: "user",
      content: [{ type: "text", text: prompts[mode as keyof typeof prompts] }],
      timestamp: Date.now(),
    },
  ],
  tools: mode === "tool" ? [echoTool] : undefined,
};

const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 30_000);
const stream = models.streamSimple(model, context, {
  signal: controller.signal,
  reasoning: mode === "thinking" ? "low" : undefined,
});

try {
  for await (const event of stream) {
    if (event.type === "text_delta" || event.type === "thinking_delta") {
      process.stdout.write(event.delta);
    }
    if (event.type === "toolcall_end") {
      console.log("\ntool:", event.toolCall.name, event.toolCall.arguments);
    }
    if (event.type === "error") {
      throw new Error(event.error.errorMessage ?? event.error.stopReason);
    }
  }
  const result = await stream.result();
  console.log("\nstop:", result.stopReason, "usage:", result.usage);
} finally {
  clearTimeout(timeout);
}

npx tsx verify-provider.ts text
npx tsx verify-provider.ts thinking
npx tsx verify-provider.ts tool

Run text first. Enable reasoning and a truthful thinkingLevelMap only after the thinking run emits thinking content or the protocol's documented reasoning representation. Add image only after an image request succeeds. Tool support requires a toolcall_end with the right name and parsed arguments; a normal text answer does not prove it.

This deterministic fixture is optional. Use it when writing an adapter or verifying retry, error, and cancellation behavior. Save it beside the live probe. It starts a loopback OpenAI-compatible endpoint, so it needs no external server or secret.

npx tsx --test verify-provider-errors.test.ts

8. Handle errors, retry, and cancellation

Keep transport policy explicit. Direct API adapters default to no retries unless the caller sets maxRetries; the OpenAI adapters retry connection failures, HTTP 408, 409, 429, and 5xx responses unless the server's retry header says otherwise. Retry waits are abortable. Coding Agent's higher-level agent retry is separate, so avoid multiplying retries across both layers.

An aborted request terminates with an assistant message whose stopReason is "aborted"; a provider failure uses "error" and an errorMessage. Consumers should still drain or await stream.result() and persist only the transcript state their application can safely replay. Never retry authentication failures, malformed Tool calls, or deterministic validation errors without changing the input.

Troubleshooting

Security checklist

• Keep API keys and session tokens out of model metadata, source control, URLs, error text, and logs.
• Treat discovery payloads, model IDs, headers, and Tool arguments as untrusted input. Validate shape and bound sizes before storing them.
• Use HTTPS for remote providers. Pin the intended host; do not let model output choose baseUrl, credential commands, or proxy targets.
• Give credential commands and OAuth flows their own review. Use ApiKeyAuth.resolve() with the supplied AbortSignal, and return only provider-scoped values the adapter needs.
• Redact provider error bodies before exposing them to users or a model. They can contain credentials, request content, or gateway internals.
• Test cancellation and retry under failure. A timed-out stream must release sockets, parsers, and child work.

Next

Once the seven checks pass, use the model in the Quickstart agent. Chapter 4: Model invocation traces the request and stream path. If a new protocol needs a custom Tool translation layer, review Add a custom Tool alongside the source adapter you are matching.