Local Docker

Run the Compute API in a container on this PC and let Studio start, stop and inspect it. The local Studio remains the UI; the container supplies the API, model hosting and training process.

What runs locally

Local Docker is optional. Chat, local inference and CPU training already work in the Studio process. Use a container when you want the same API/host contract as a remote machine, or when a local NVIDIA card should serve and train.

Build the image

Docker Desktop must be running. The local build uses the sibling SharpMind source tree, so no packages need to be published. Run from the folder that contains both repositories.

cd C:\path\to\repos
Copy-Item SharpMind.Studio\docker\local-context.dockerignore .dockerignore

# CPU Compute API
docker build -f SharpMind.Studio/docker/compute/Dockerfile.local `
  -t sharpmind-compute:local .

# GPU server image (Studio + Compute + ilgpu)
docker build -f SharpMind.Studio/docker/server/Dockerfile.gpu.local `
  -t sharpmind-server:gpu .

The GPU image is about 1.6 GB; the CPU compute image is about 250 MB. The CUDA libraries ship in the image, but the NVIDIA driver comes from the host runtime.

Add the host

1. Settings → Providers → Compute → Add host → Local Docker.
2. Name it, keep the Base URL http://127.0.0.1:8080, and press Generate for the token.
3. Set the image: sharpmind-compute:local for CPU, or sharpmind-server:gpu for a local GPU.
4. Set the port (default 8080).
5. Tick GPU only when Docker has an NVIDIA runtime and you chose the GPU image.
6. Save host.

Start and test

On the host card:

1. Check shows whether Docker Desktop is running and what the container is doing.
2. Start container recreates it with the saved image, token, port and data folder.
3. Test calls /health and loads the model list.
4. Logs shows the last lines from the container; Stop keeps the data folder.

The data folder lives under the Studio root, next to Compute\<container>. It is mounted at /data/SharpMind, so models and uploads survive a container recreation.

Local GPU

• Docker Desktop with WSL2 and the NVIDIA driver installed on Windows.
• The GPU server image built above, selected as the host image.
• GPU ticked, which adds --gpus all.
• /health should list ilgpu with a CUDA backend.

If the container fails with could not select device driver, Docker does not have the NVIDIA runtime yet. Install/repair Docker Desktop's GPU support or use a remote host. Inference is broader than training: it also covers LayerNorm and dense-GELU models.

Upload a model

1. On the host card, Browse to a packaged .smm (or convert a GGUF first).
2. Upload and use copies it to the host's Models folder and activates it.
3. Chat with the model, or send an API request to /v1/chat/completions.

Training from scratch does not need a model on the host. A fine-tune uploads its base model with the job.

Use the API

curl http://127.0.0.1:8080/health

curl http://127.0.0.1:8080/v1/models \
  -H "Authorization: Bearer <token>"

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"model":"default","messages":[{"role":"user","content":"Hello"}],"max_tokens":64}'

Streaming works with "stream": true. The host is OpenAI-compatible for chat; tool calling on the Compute API is still on the roadmap.

Problems