Use Workshop to run OpenCode with a local Gemma4 snap

For the more demanding crowd, a coding agent setup only earns its keep when it’s secure (meaning your data doesn’t egress) and safe (neither does it destroy your data). Two things have to be true for that to hold: the AI model has to run locally, and the agent must be isolated so that a stray command can’t wander off across your home directory. On Ubuntu, you don’t need to worry about either case.

Here, you will run OpenCode, an open-source, terminal-based AI coding agent, inside a workshop (an ephemeral, container-isolated dev environment) wired to the gemma4 inference snap running on your host. A workshop seals its contents off from the rest of the machine: the only way anything inside it reaches the host is through an explicit interface connection, or via the project directory where the workshop runs. We declare exactly one custom connection (a tunnel to the model) and leave the rest of the wall standing.

By the end, you will have:

  • The gemma4 snap serving an OpenAI-compatible API on the host’s localhost.
  • A workshop running OpenCode as an SDK (that’s what we chose to call Workshop’s distribution format), sealed off from everything else on the host.
  • A single tunnel bridging the two, so the boxed-in agent reaches the host model at the same localhost:8336 it would use on bare metal.
  • A finished coding task delivered to your host, the agent’s work still there after the workshop is gone, and no precious token budget spent to get it. FizzBuzz stands in for that work here, kept trivial so the focus stays on the setup rather than the code; the same flow scales to any real project you’d point the agent at.

We’ll take the shortest path that works first, then open up what a workshop can actually do once it’s running.

Total time: ~15 minutes (plus driver install and reboot, if needed).

Prerequisites

  • sudo rights for the snap installs
  • A working GPU driver if you want speed. NVIDIA needs version 575 or newer; the snap falls back to a cpu engine otherwise (slower, but functional). The driver how-to has the details.
  • jq for the smoke tests
  • No cloud key. That’s the entire point.

References used throughout:

Step 1: Install the gemma4 snap

This runs on the host, not in the workshop. The model wants your GPU, and the GPU lives on the host. (Workshop can handle that as well, but that’s a different story.)

sudo snap install gemma4

The snap registers two services that start automatically: an OpenAI-compatible API (gemma4.server) and a browser UI (gemma4.server-webui).

Verify:

gemma4 status
engine: nvidia-gpu
services:
    server: active
    server-webui: active
endpoints:
    openai: http://localhost:8336/v1
    webui: http://localhost:8337/
model:
    name: gemma4-e4b-q4-k-m

Note the openai: URL and the model: name; you need both for later. Now confirm the API actually answers:

curl -s http://localhost:8336/v1/models | jq '.data'
[
  {
    "id": "gemma4-e4b-q4-k-m",
    "aliases": [
      "gemma4-e4b-q4-k-m"
    ],
    "tags": [],
    "object": "model",
    "created": 1780694379,
    "owned_by": "llamacpp",
    "meta": {
      "vocab_type": 2,
      "n_vocab": 262144,
      "n_ctx_train": 131072,
      "n_embd": 2560,
      "n_params": 7518069290,
      "size": 5319465128
    }
  }
]

The default model, gemma4-e4b-q4-k-m, is E4B (“Efficient 4B”), a multimodal Gemma 4 that weighs about 5.3 GB on disk. It is a default that works for our purposes, so the rest of this walkthrough just uses it.

The first request loads the weights into memory, so the model can take a few seconds to wake up. To confirm it’s ready without watching for that, run gemma4 chat: it waits for the server, then opens a prompt where you type a line and read the reply back (CTRL-C to leave). Once it answers, you’re live.

Step 2: Install Workshop

Still on the host. Workshop is a classic snap and runs on top of LXD:

sudo snap refresh lxd --channel=6/stable || sudo snap install lxd --channel=6/stable
sudo snap install workshop --classic

Verify:

workshop list

An empty table (just the headers) is the expected answer on a clean install. While we’re here, confirm the OpenCode SDK exists in the SDK Store; this is the thing the workshop will install for you:

sdk find opencode
NAME      VERSION  PUBLISHER     SUMMARY
opencode  1.15.10  Dmitry Lyfar  The OpenCode SDK.

Step 3: Define the workshop

A workshop is described by a single workshop.yaml in your project directory. Create one with a shiny new Ubuntu 26.04 base:

mkdir -p ~/Documents/opencode-gemma
cd       ~/Documents/opencode-gemma

cat > workshop.yaml <<'EOF'
name: demo
base: ubuntu@26.04
sdks:
  - name: opencode
    channel: latest/stable
    plugs:
      gemma4:
        interface: tunnel
        endpoint: localhost:8336
  - name: system
    slots:
      gemma4:
        interface: tunnel
        endpoint: localhost:8336
EOF

The sdks: list pulls OpenCode into the container. The interesting part is the tunnel:

  • The plug (gemma4, on the opencode SDK) is the side that consumes the connection, where the agent, inside the container, reaches for the API.
  • The slot (gemma4, on the system SDK) is the side that provides it: the host service at localhost:8336.
  • Both name the same endpoint, so the address inside the container is identical to the address on the host. Nothing to remember, nothing to remap.

Because the plug lives on a regular SDK rather than on system, this tunnel does not connect itself automatically. That’s deliberate: you wire it up explicitly in Step 6, so nothing reaches the host until you say so.

Step 4: Point OpenCode at the tunnel

OpenCode talks to any OpenAI-compatible endpoint through the @ai-sdk/openai-compatible provider. Drop its config into the project directory; that directory mounts into the workshop at /project, so the file rides along and OpenCode finds it inside the container without you ever editing anything in there:

cat > opencode.json <<'EOF'
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "gemma4-local": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Gemma4 Local",
      "options": {
        "baseURL": "http://127.0.0.1:8336/v1",
        "apiKey": "none"
      },
      "models": {
        "gemma4-e4b-q4-k-m": {
          "name": "Gemma 4 E4B"
        }
      }
    }
  },
  "model": "gemma4-local/gemma4-e4b-q4-k-m"
}
EOF

Three things to keep aligned:

  • baseURL points at localhost:8336, the same port the tunnel forwards, the same port gemma4 status reported.
  • The model key under models must match the id from /v1/models exactly (here, gemma4-e4b-q4-k-m).
  • The top-level model is <provider-id>/<model-id>, and becomes OpenCode’s default.

The API ignores authentication, so apiKey is set to none. If a client ever insists on a key, any non-empty placeholder works.

Step 5: Launch and inspect

workshop launch demo
"demo" launched

Verify:

workshop list
workshop info demo
WORKSHOP  STATUS  NOTES
demo      Ready   -

workshop info demo prints the full picture: base image, the project directory mounted at /project, and the OpenCode SDK with its persisted config and data mounts (that’s where the SDK persists its state between workshop updates). What it does not show yet is a tunnel. The workshop is up, OpenCode is installed, and the host model is still completely out of reach. Good.

Step 6: Connect the tunnel

This is the one line that opens the wall:

workshop connect demo/opencode:gemma4 demo/system:gemma4

Verify:

workshop connections
INTERFACE  PLUG                           SLOT                NOTES
mount      demo/opencode:opencode-config  demo/system:mount   -
mount      demo/opencode:opencode-data    demo/system:mount   -
tunnel     demo/opencode:gemma4           demo/system:gemma4  manual

The tunnel row, marked manual, is the one you just made. workshop info demo now grows a tunnels: block showing gemma4 forwarding 127.0.0.1:8336 to the identical address on the host. One port through the wall; nothing else.

Step 7: Probe gemma4 through the tunnel

Run curl inside the workshop and watch it reach the host model:

workshop exec demo -- curl -s http://127.0.0.1:8336/v1/models | jq '.data[].id'
"gemma4-e4b-q4-k-m"

Same localhost:8336, same model, but the request originated from a sealed container and came back 200. The wall is bridged for exactly one port.

Step 8: Let OpenCode write a file

Now the actual job. OpenCode runs inside the workshop, talks to gemma4 over the tunnel, and writes into /project (which is your host directory):

workshop exec demo -- opencode run 'Write /project/fizzbuzz.py: a Python script that prints FizzBuzz for 1..20'
> build · gemma4-e4b-q4-k-m

← Write fizzbuzz.py
Wrote file successfully.

The very first opencode run does a one-time database migration, and the first call into a freshly-woken model reloads weights, so the debut is slow. Later runs are brisk.

Because /project is the mount, the file is already on your host:

cat fizzbuzz.py
python3 fizzbuzz.py
for i in range(1, 21):
    if i % 3 == 0 and i % 5 == 0:
        print("FizzBuzz")
    elif i % 3 == 0:
        print("Fizz")
    elif i % 5 == 0:
        print("Buzz")
    else:
        print(i)
1
2
Fizz
4
Buzz
Fizz
...

Step 9: Tear down, the work stays

Stopping a workshop keeps its state; only remove destroys it:

workshop stop demo
workshop remove demo
workshop list
WORKSHOP  STATUS  NOTES
demo      Off     -

And the payoff:

python3 fizzbuzz.py

It still runs. The agent is gone, the container is gone, the model never left your GPU, and the only thing that made it out of the box is the file you asked for. That’s the whole shape of the exercise: confine the agent, keep the model on the metal, and let exactly one file and one port cross the line.

Where to take Workshop next

The tunnel-to-a-local-model trick is one small corner of what a workshop is for. A few directions worth exploring:

  • Run agents in parallel, safely. Pair a workshop per git worktree to let several agents work on different branches at once, each in its own sealed container. See How to use workshops with Git.
  • Bring your own editor. Connect a local VS Code (or JetBrains, or a browser) to a running workshop and edit inside it with your usual tools. See How to connect VS Code.
  • Cross the wall via other interfaces. The tunnel is one of several sanctioned crossings: mount for files, tunnel for ports and sockets, plus gpu, ssh-agent, camera, and desktop. Declaring the connection in workshop.yaml makes it survive a workshop refresh. See the tunnel interface explanation section.
  • Package your own tooling. Beyond Store SDKs like opencode, you can author an in-project SDK with setup and health hooks so a teammate gets the same environment from one workshop launch. See In-project SDKs explanation section.

Other next steps

  • Track the setup under git so it’s reproducible: echo '*.lock' > .gitignore (the workshop drops a lock file you don’t want committed), then commit workshop.yaml and opencode.json.
  • Step up to a bigger model or a different quantization: snap components gemma4 and gemma4 list-engines show what your hardware can load, and sudo gemma4 use-engine <name> switches engines. Treat those commands as the source of truth, then re-run the gemma4 chat check and update the model ID in opencode.json to match.
  • Add a second inference snap from the inference-snaps family as another provider block in opencode.json, and a second tunnel in workshop.yaml.
  • Tune the nap: gemma4 set sleep-idle-seconds=1800 keeps the model warm longer between requests.
  • Cut the workshop off from the internet so nothing can phone home, while the gemma4 tunnel keeps working: lxc network acl create offline && lxc network set workshopbr0 security.acls=offline security.acls.default.egress.action=drop. The block is enforced on the host side, so nothing inside the container can lift it, and it survives stop, start, and workshop refresh. (In fact, it covers every workshop on the shared workshopbr0 bridge). See LXD network ACLs.
  • Keep reading: the OpenCode docs cover MCP servers, custom prompts, and team config.
12 Likes

Thanks for the nice tutorial!

One quick question: if I want the model to run on a beefy server on my local network, is the only thing that needs tweaking the address of the model in the system SDK, or is that not how the tunnel interface works?
Like this:

Yes, the tweak is exactly this one-liner. However, don’t forget to workshop refresh, then workshop connect the updated setup. FInally, make sure the server IP/hostname is reachable from the Workshop host system.

1 Like

The gemma4 snap doesn’t support AMD GPUs? If so, could I use the Lemonade server snap instead?

Edit:

gemma4 use-engine amd-gpu

seems to work fine (though marked as “devel” in the docs). Still, we could use the same overall method for Lemonade I guess?

Yes, I was about to suggest the same; gemma4 list-engines will show what’s available.

As for Lemonade, the setup will work with any local OpenAI-compatible inference backend with minimal tweaks, courtesy of OpenCode mostly in this scenario; Workshop simply provides the environment.

1 Like

Thanks for the tutorial. While running it I experienced an error:

  • Write a Python script to print FizzBuzz for numbers 1 through 20.

Constraints & Preferences

  • None specified.

Progress

Done

  • (none)

In Progress

  • (none)

Blocked

  • (none)

Key Decisions

  • (none)

Next Steps

  • Write the Python implementation for /project/fizzbuzz.py.

Critical Context

  • (none)

Relevant Files

  • /project/fizzbuzz.py: The Python script to be created for the FizzBuzz challenge.

The previous request exceeded the provider’s size limit due to large media attachments. The conversation was compacted and media files were removed from context. If the user was asking about attached images or files, explain that the attachments were too large to process and suggest they try again with smaller or fewer files.

Continue if you have next steps, or stop and ask for clarification if you are unsure how to proceed.

Error: request (7725 tokens) exceeds the available context size (4096 tokens), try increasing it

Any hints how to fix it?

TBH this looks like a CPU or a smaller GPU influenced the config choices for Gemma at installation. Can you please run this script and post the feedback? It’s read-only.

#!/bin/bash
# gemma4-ctx-diag.sh — read-only diagnostic for the gemma4 snap context window.
#
# Explains why a host ended up with a given "available context size" (the value
# in: "request (N tokens) exceeds the available context size (M tokens)").
# Makes NO changes: only snap/gemma4 getters, ps, curl GET, nvidia-smi query.
#
# Usage:  bash gemma4-ctx-diag.sh
#         (re-run with sudo if "snap logs" is permission-denied)

set -u
hr() { printf '\n========== %s ==========\n' "$1"; }
have() { command -v "$1" >/dev/null 2>&1; }

# ---------------------------------------------------------------------------
hr "0. snap present?"
if ! have snap || ! snap list gemma4 >/dev/null 2>&1; then
  echo "gemma4 snap not installed (snap list gemma4 failed). Nothing to diagnose."
  exit 1
fi
echo "gemma4 snap detected."

hr "1. snap identity (channel/revision — stable e4b vs beta 2.0.0 differ)"
snap list gemma4
snap info gemma4 2>/dev/null | grep -E 'tracking|refresh-date' || true

hr "2. active engine + compatibility (GPU vs CPU is the headline)"
# active engine is the row marked with '*'
ENGINES_RAW="$(gemma4 list-engines 2>/dev/null)"
echo "$ENGINES_RAW"
ACTIVE="$(printf '%s\n' "$ENGINES_RAW" | awk '/\*/{sub(/\*.*/,"",$1); print $1; exit}')"
echo "-- active engine: ${ACTIVE:-unknown}"
[ -n "${ACTIVE:-}" ] && gemma4 show-engine "$ACTIVE" 2>/dev/null | grep -E 'name:|description:|components:|score:|^- ' | head -20

hr "3. detected memory (the constraint that drives auto-sizing)"
# total-ram + GPU vram, plus only device-names that look like a GPU (skip the full PCI dump)
gemma4 show-machine 2>/dev/null \
  | grep -E 'total-ram:|total-swap:|vram:|device-name:.*(GeForce|Radeon|Instinct|Arc|Graphics|GPU)' \
  | sed 's/^[[:space:]]*/  /' || true
if have nvidia-smi; then
  echo "-- nvidia-smi:"
  nvidia-smi --query-gpu=name,memory.total,memory.free --format=csv 2>&1
else
  echo "-- nvidia-smi: not present (expected on CPU-only or non-NVIDIA hosts)"
fi

hr "4. current snap config (note: there is NO ctx-size key to set)"
gemma4 get 2>/dev/null || true

hr "5. auto-tuning decision + actual context (KEY STEP — from startup logs)"
LOGS="$(snap logs gemma4 -n 1000 2>&1)"
if printf '%s' "$LOGS" | grep -qiE 'permission denied|cannot|must be root'; then
  echo "!! could not read logs without privileges — re-run: sudo bash $0"
else
  printf '%s\n' "$LOGS" | grep -E \
    'n_parallel|kv_unified|n_ctx_train|n_ctx|n_seq_max|KV (buffer|cache)|slot .*n_ctx|out of memory|failed to allocate|cudaMalloc|unable to (allocate|load)|reduc' \
    | tail -30
  [ -z "$(printf '%s' "$LOGS" | grep -E 'n_ctx|n_parallel')" ] && \
    echo "(no context lines in recent logs — try: sudo snap restart gemma4 then re-run, to capture fresh startup)"
fi

hr "6. actual running llama-server args (is --ctx-size / --parallel injected?)"
ps -eo pid,args 2>/dev/null | grep '[l]lama-server' || echo "(llama-server not currently running — it may be sleeping; a request will respawn it)"

hr "7. live per-slot context (authoritative current value)"
HOST="$(gemma4 get http.host 2>/dev/null)"; HOST="${HOST:-127.0.0.1}"
PORT="$(gemma4 get http.port 2>/dev/null)"; PORT="${PORT:-8336}"
echo "-- querying http://$HOST:$PORT/props"
PROPS="$(curl -s --max-time 5 "http://$HOST:$PORT/props" 2>/dev/null)"
NCTX=""
if [ -n "$PROPS" ]; then
  if have python3; then
    read -r NCTX SLOTS SLEEP <<EOF
$(printf '%s' "$PROPS" | python3 -c "import sys,json
d=json.load(sys.stdin); g=d.get('default_generation_settings',{})
print(g.get('n_ctx',''), d.get('total_slots',''), d.get('is_sleeping',''))" 2>/dev/null)
EOF
    echo "  n_ctx (per slot): ${NCTX:-?}"
    echo "  total_slots:      ${SLOTS:-?}"
    echo "  is_sleeping:      ${SLEEP:-?}"
  else
    # python3 absent — crude grep fallback
    NCTX="$(printf '%s' "$PROPS" | grep -oE '"n_ctx"[: ]+[0-9]+' | head -1 | grep -oE '[0-9]+')"
    echo "  n_ctx (per slot, grep fallback): ${NCTX:-?}"
  fi
else
  echo "  no response from /props (server unreachable or different host/port)."
fi

# ---------------------------------------------------------------------------
hr "VERDICT"
TRAIN="$(printf '%s' "$LOGS" | grep -oE 'n_ctx_train[^0-9]+[0-9]+' | grep -oE '[0-9]+' | head -1)"
PARALLEL_LINE="$(printf '%s' "$LOGS" | grep -E 'n_parallel is set to|using n_parallel' | tail -1)"
[ -n "$PARALLEL_LINE" ] && echo "auto-tuning: $PARALLEL_LINE"
[ -n "$TRAIN" ] && echo "model max (n_ctx_train): $TRAIN"
echo "active engine: ${ACTIVE:-unknown}"
echo "running context (n_ctx per slot): ${NCTX:-unknown}"

if printf '%s' "$LOGS" | grep -qiE 'out of memory|failed to allocate|cudaMalloc|unable to allocate'; then
  echo ">> MEMORY-DRIVEN REDUCTION: log shows an allocation failure. Context was capped to fit"
  echo "   available VRAM/RAM. Expected on lighter GPUs and CPU-only hosts."
elif [ -n "${NCTX:-}" ] && [ -n "${TRAIN:-}" ] && [ "$NCTX" -lt "$TRAIN" ] 2>/dev/null; then
  if printf '%s' "$LOGS" | grep -qiE 'kv_unified = false'; then
    echo ">> SLOT-DIVISION: kv_unified=false and per-slot n_ctx ($NCTX) < model max ($TRAIN)."
    echo "   Total context is split across auto-chosen parallel slots."
  else
    echo ">> REDUCED CONTEXT: per-slot n_ctx ($NCTX) < model max ($TRAIN), no explicit OOM logged —"
    echo "   most likely memory headroom forced a smaller auto context. Check step 3 free memory."
  fi
elif [ -n "${NCTX:-}" ] && [ -n "${TRAIN:-}" ] && [ "$NCTX" -ge "$TRAIN" ] 2>/dev/null; then
  echo ">> HEALTHY: running at the model's full context. No context shortfall on this host."
else
  echo ">> Inspect steps 5 + 7 above. If n_ctx is small (e.g. 4096) the error is real and"
  echo "   driven by the active engine + detected memory shown in steps 2-3."
fi
echo
echo "(read-only diagnostic complete — no changes were made)"

========== 0. snap present? ==========
gemma4 snap detected.

========== 1. snap identity (channel/revision — stable e4b vs beta 2.0.0 differ) ==========
Name Version Rev Tracking Publisher Notes
gemma4 e4b+0b2585e 80 latest/stable canonical-iot-labs✓ -
tracking: latest/stable
refresh-date: 5 days ago, at 15:37 CEST

========== 2. active engine + compatibility (GPU vs CPU is the headline) ==========
ENGINE VENDOR DESCRIPTION COMPAT
nvidia-gpu* Canonical Ltd Optimized for inference on Nvidia GPUs, us… yes
cpu Canonical Ltd Optimized for a range of CPU micro-archite… yes
amd-gpu Canonical Ltd Use ROCm for inference on AMD GPUs. no
– active engine: nvidia-gpu
name: nvidia-gpu
description: Optimized for inference on Nvidia GPUs, using CUDA
components:
score: 92

========== 3. detected memory (the constraint that drives auto-sizing) ==========
total-ram: 33049907200
total-swap: 8589930496
device-name: Arrow Lake-P [Intel Graphics]
device-name: AD107GLM [RTX 500 Ada Generation Laptop GPU]
vram: “4292870144”
– nvidia-smi:
name, memory.total [MiB], memory.free [MiB]
NVIDIA RTX 500 Ada Generation Laptop GPU, 4094 MiB, 3594 MiB

========== 4. current snap config (note: there is NO ctx-size key to set) ==========
http.host: 127.0.0.1
http.port: 8336
verbose: false
webui.http.host: 127.0.0.1
webui.http.port: 8337

========== 5. auto-tuning decision + actual context (KEY STEP — from startup logs) ==========
!! could not read logs without privileges — re-run: sudo bash ./test.sh

========== 6. actual running llama-server args (is --ctx-size / --parallel injected?) ==========
10901 llama-server --model /snap/gemma4/components/80/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf --alias gemma4-e4b-q4-k-m --mmproj /snap/gemma4/components/80/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf --port 8336 --host 127.0.0.1 --no-warmup --sleep-idle-seconds 600

========== 7. live per-slot context (authoritative current value) ==========
– querying http://127.0.0.1:8336/props
n_ctx (per slot): 4096
total_slots: 4
is_sleeping: True

========== VERDICT ==========
model max (n_ctx_train): 131072
active engine: nvidia-gpu
running context (n_ctx per slot): 4096

REDUCED CONTEXT: per-slot n_ctx (4096) < model max (131072), no explicit OOM logged —
most likely memory headroom forced a smaller auto context. Check step 3 free memory.

(read-only diagnostic complete — no changes were made)

Thanks. Indeed, 4GB of GPU memory leaves almost no space for the KV cache (this is the overhead the model needs to process your requests); the model itself fits in the memory, but it has very little wiggle room. The only option left AFAIK is gemma4 use-engine cpu: this will slow down inference but will give you more context space.

Also, a few hints from the inference snaps team:

But you can force the runtime to allocate a bigger buffer. This will force part of the model to be offloaded to system memory, so the inference speed drops.

For example, to set the context buffer length to 16384 tokens for gemma4, run:

sudo gemma4 set env.llama-arg-ctx-size=16384

Learn more about setting inference snap’s environment variables: https://documentation.ubuntu.com/inference-snaps/how-to/troubleshooting/set-environment-variables/

Read about the available environment variables for llama.cpp: llama.cpp/tools/server/README.md at master · ggml-org/llama.cpp · GitHub

1 Like

@akonev Thank you for hints. But it looks like I’m using older snap version so “sudo gemma4 set env.llama-arg-ctx-size=16384” did not work giving me an error ‘Error: key “env.llama-arg-ctx-size” is not found’.
Following the documentation:

Added in version 1.0.0-beta.42.

Changed in version 2.0.0: passthrough.environment. key prefix has been replaced with env.

I managed to apply it using “sudo gemma4 set passthrough.environment.llama-arg-ctx-size=16384”