Training and fine-tuning

SharpMind has three training jobs. They share the data pipeline, checkpoints and export, but they train different things and use different devices.

Three modes

GPU support

The GPU training engine accepts a base model with all of these:

• RMSNorm (not LayerNorm) — both in the blocks and the final norm.
• RoPE or NoPE positional encoding.
• Gated FFN (SwiGLU/GeGLU-style), not MoE.
• An attention layout with q_proj, k_proj, v_proj, o_proj LoRA targets.
• A packaged .smm base. Convert a GGUF first.

LLaMA/Qwen-style bases satisfy this. GPT-2/BERT-style bases (LayerNorm, dense GELU) do not for training, although GPU inference now covers them.

On a supported base the log shows:

Fine-tune: 8 adapters (rank 8, alpha 16).
Engine: [Cuda] NVIDIA A16, 14962 MB, cuBLAS 12.6
Training 100 steps (float32) — full backprop (loss → gradients → AdamW)…

On an unsupported base it shows the refusal and falls back:

Accelerator 'ilgpu' declined: GPU engine (M1) does not support a
LayerNormLayer final norm — LayerNorm, only RMSNorm
Retrying on the CPU. Set SHARPMIND_ACCELERATOR_FALLBACK=false to fail instead.

The fallback is a host policy. Set SHARPMIND_ACCELERATOR_FALLBACK=false to fail the job instead of retrying.

Fine-tune a base

1. Training → Improve.
2. Base model: pick a packaged .smm. The model list is under Models; a GGUF must be converted first.
3. Corpus: use Browse. With SFT off it filters *.txt; with SFT on, *.jsonl. A folder or glob also works.
4. LoRA: rank 8–16, alpha 16–32, targets q_proj k_proj v_proj o_proj.
5. Steps: 50–200 for a tiny test, 200–500+ for a real SFT run.
6. Output: merged .smm, adapter .smlora, or both.
7. Run on: This PC or a Compute host. The base and corpus upload with the job; the export downloads back.
8. Start fine-tune and watch the engine line and loss curve.

SFT

Tick Instruction tuning (SFT) to train on chat JSONL with assistant-only loss. Each line is one record. Supported shapes:

{"messages":[{"role":"user","content":"…"},{"role":"assistant","content":"…"}]}
{"conversations":[{"from":"human","value":"…"},{"from":"gpt","value":"…"}]}
{"prompt":"…","response":"…"}
{"instruction":"…","input":"…","output":"…"}

The base supplies the tokenizer and chat template. Use Preview data before the run: it reports parsed, skipped, truncated and train/val/exam counts, plus sample rendered conversations.

• Blank lines, malformed JSON and records without an assistant turn are skipped.
• Long conversations are truncated to the SFT sequence length.
• FuseChat's original file is a root JSON array, not JSONL; convert one record per line first.

Choosing data

For a quick test, a 1 MB text file is enough. For a useful instruction tune, a few thousand chat records are enough to see behaviour; 20k+ is a small real run. Prefer data that matches how you will use the model.

Remote runs

Select a Compute host in Run on. Studio uploads the source files and base, starts the job, polls its state and logs, then downloads the export back to the path shown on the form. The host needs no chat model for a fine-tune.

• If the base is large, upload time dominates the first poll; the card logs progress in 25% steps.
• The export is written under the host's /data/SharpMind/Training/jobs/…, then downloaded.
• Rehearse 30 steps runs locally and estimates step time; it is not the remote job.
• An interrupted run may leave the host job running; cancel it from the run record or the host API.

Progress and interrupt

The run panel follows the job: status, step, loss, validation loss, curve, log tail and an Interrupt button. Starting a run scrolls it into view. The panel repaints itself while the job is active, including remote jobs.

Accelerator errors