Using OpenCode with a Fuzzball-Hosted Model

OpenCode is a coding agent that talks to any OpenAI-compatible API. A model served on Fuzzball publishes that kind of API, so OpenCode can use a cluster GPU while it edits your files.

When you reach a model endpoint with a Fuzzball token -- required at any scope but public, and accepted on a public endpoint of your own organization -- the Fuzzball endpoint proxy tells the model service who you are, and the LiteLLM proxy in front of the model admits you on that alone -- see Identifying the caller. No LiteLLM key is involved, except in two cases.

You can run OpenCode in two places:

• On Fuzzball, with the OpenCode catalog entry. The workflow reaches the model with your identity, so you hand it no credential at all. Start here.
• On your workstation, with a local install. You paste the URL and a Fuzzball endpoint token into OpenCode's config.

Before you start

You need three things:

• A running model, either of:

  • A vLLM catalog entry (or one of its presets such as GPT-OSS 20B) with its default Proxy=true. OpenCode asks for the model by its repository name: the entry's Model value without hf:// or anything after ?, for example openai/gpt-oss-20b.
  • Several models behind the LiteLLM Model Gateway entry. See Serving an AI Model.

• OpenCode installed on your workstation, for example brew install anomalyco/tap/opencode. Both paths need it -- the on-Fuzzball path attaches to the remote server from your local opencode.

• The cluster CA in PEM form, if the cluster's API uses a private certificate authority. See Trust the cluster certificate.

Get the endpoint URL

$ fuzzball endpoint list

Copy the URL and endpoint ID from the row for the model workflow's litellm service, or the gateway workflow's gateway service.

Run OpenCode on Fuzzball

$ fuzzball workflow catalog start OpenCode \
    --values Endpoint=https://ENDPOINT-URL,ServiceScope=public,MaxContextSize=MODEL-CONTEXT-SIZE

Then attach from your workstation. Take the server URL from fuzzball endpoint list and the password from the show-server job:

$ fuzzball workflow log WORKFLOW_ID show-server
$ opencode attach https://OPENCODE-URL -p PASSWORD

What the values do:

• Endpoint is optional. Left empty, the workflow registers every gateway endpoint its own identity can reach. Set it to pin one endpoint, or to reach an API outside Fuzzball.
• ServiceScope=public is what makes opencode attach and the web app work. At any other scope a request also needs a Fuzzball token. API clients can send it in FB-Authorization and the server password as Authorization: Basic BASE64(opencode:PASSWORD), which the endpoint proxy passes through to the server.
• MaxContextSize must match the model. The entry defaults to 32768, while the vLLM presets serve 131072 to 1048576. Use the model workflow's own MaxContextSize value.
• For a model endpoint that is not public, the workflow mints its own Fuzzball token at startup and the model admits it as you, the workflow's owner. It lasts seven days at most. Restart the workflow when it lapses.
• It registers every model the endpoint lists at startup. Model=openai/gpt-oss-20b picks the default. A model the endpoint gains later needs a workflow restart.

The workspace starts empty and is lost when the workflow stops. The agent works in /data/workspace on the workflow's volume, which defaults to ephemeral. Name a persistent volume with Volume= to keep files and sessions. The image is Alpine plus the OpenCode binary, with no git or compilers. Override Image if the agent needs a toolchain.

Run OpenCode on your workstation

Trust the cluster certificate

If the cluster's API uses a private CA, every client must trust it or you get:

SSL certificate problem: unable to get local issuer certificate

Get the CA in PEM form from your administrator and point both tools at it. OpenCode is a Bun binary and reads NODE_EXTRA_CA_CERTS; curl does not, and reads CURL_CA_BUNDLE:

$ export NODE_EXTRA_CA_CERTS=/path/to/cluster-ca.pem
$ export CURL_CA_BUNDLE=/path/to/cluster-ca.pem

Get an endpoint token

A model endpoint defaults to user scope, which needs an endpoint token. Mint one:

$ fuzzball endpoint generate-token ENDPOINT_ID --expiration 8h

This token is the only credential you send. A public endpoint needs no token and lets every request through; a token there only tells the model who you are, and only when you belong to the endpoint's organization.

Check with curl

This separates connection problems from configuration problems. The token goes in Authorization:

$ curl -H "Authorization: Bearer YOUR-FUZZBALL-TOKEN" \
    https://ENDPOINT-URL/v1/models

A healthy response lists the model:

{"data":[{"id":"openai/gpt-oss-20b","object":"model","created":1677610602,"owned_by":"openai"}],"object":"list"}

Configure OpenCode

Export the token, then define Fuzzball as an OpenAI-compatible provider in ~/.config/opencode/opencode.json or a project's opencode.json:

$ export FUZZBALL_TOKEN=YOUR-FUZZBALL-TOKEN

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "fuzzball": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Fuzzball",
      "options": {
        "baseURL": "https://ENDPOINT-URL/v1",
        "apiKey": "{env:FUZZBALL_TOKEN}"
      },
      "models": {
        "openai/gpt-oss-20b": {
          "name": "gpt-oss-20b on Fuzzball",
          "limit": {
            "context": 32768,
            "output": 8192
          }
        }
      }
    }
  },
  "model": "fuzzball/openai/gpt-oss-20b"
}

{env:NAME} and {file:/path} keep credentials out of the file. Set limit.context to the entry's MaxContextSize; a larger value leads to requests the model rejects.

Send a first request

$ opencode models fuzzball
fuzzball/openai/gpt-oss-20b

$ opencode run --model fuzzball/openai/gpt-oss-20b "Reply with the single word: ready"
ready

Agentic work needs tool calling. Start the vLLM entry with the right flags in ExtraArgs, for example --tool-call-parser openai --enable-auto-tool-choice for gpt-oss models. Without them OpenCode can chat but cannot read or edit files.

Small models sometimes invent tool names, such as apply_patch. OpenCode reports the error and usually recovers on the next step.

When the Fuzzball token expires, OpenCode fails to authenticate. Mint and export a new one.

When you still need a LiteLLM key

You need a LiteLLM key in two cases:

• The endpoint is public and you send it no Fuzzball token, or you are outside its organization. Fuzzball then sends the model service nothing about the caller.
• The cluster's nodes have not picked up the node extension that publishes the signing keys. LiteLLM cannot check what is sent and falls back to its own key check.

If neither applies, skip this section. Send a key only in those two cases: if you send one, LiteLLM authenticates on that key alone and ignores your identity, so a wrong or stale key is refused even when your endpoint token is valid.

Get the key. If you started the model workflow with your own, reuse it: the ApiKey or ApiKeySecret you gave the vLLM entry, or the gateway's MasterKeySecret. Otherwise the entry generated one, and it is in the workflow definition:

$ fuzzball workflow get WORKFLOW_ID | grep -oE 'sk-[A-Za-z0-9_-]+' | head -1

On your workstation, on a public endpoint, put the key in options.apiKey. Otherwise keep the Fuzzball token there and add the key, exported as LITELLM_KEY, in a header the proxy leaves alone, inside options:

"headers": {
  "x-litellm-api-key": "{env:LITELLM_KEY}"
}

On Fuzzball, store the key as a user-scoped secret of type value and name it in ApiKeySecret. If the model endpoint is public, add EndpointAuth=api-key as well:

$ printf 'sk-...' | fuzzball secret create secret://user/litellm-key --type value
$ fuzzball workflow catalog start OpenCode \
    --values Endpoint=https://ENDPOINT-URL,ApiKeySecret=secret://user/litellm-key

Fuzzball rejects group and organization secrets for environment variables, so each user stores their own copy. See Secrets in Workflows.

Troubleshooting

See also

• Serving an AI Model for starting the model this page connects to, and for the endpoint scopes and API keys it can be given.
• Workflow Endpoints for endpoint scopes, minting access tokens, and how the caller-identity assertion is verified.
• Connecting to Workflow Services for the other workstation-side route into a running service.