Two PRO 6000s.
What does GLM finish?

The exact stack matters.

These measurements are from our 19 September 2026 test on a remote Linux server. They measure this runtime and quantisation; they are not a limit on what the same GPUs could do with another serving stack.

• GPUs: 2 × RTX PRO 6000 Blackwell Server Edition, 96 GB each, 600 W variant. Both GPUs were behind the same PCIe switch, on NUMA node 0.
• Host: two AMD EPYC 9355 CPUs, 503 GB available host RAM and 300 GB local disk. The agent client reached the server over a network with roughly 0.25-second round-trip latency.
• Model: Unsloth GLM-5.3-Flash GGUF, UD-IQ4_XS, five shards totalling about 156.8 GB. All five file hashes were verified before testing.
• Runtime: Unsloth’s llama.cpp glm5next/upstream branch at 86ebfef, CUDA build for SM120. Driver 595.91.07, CUDA 12.8 build/runtime libraries; the driver reported CUDA 13.2 capability.
• Generation: thinking enabled, temperature 1.0, top-p 0.95. The request’s reasoning-effort value was not recorded, so we make no “maximum effort” claim.

With 32 slots × 32K context and MTP off, recorded GPU memory use was about 88.5 GB and 85.0 GB. With MTP on it reached about 93.7 GB and 95.8 GB. The weights fitting is only the first part of the memory budget.

The installation behind this run.

This is a reconstruction of the tested Linux build and serving flags, with generic paths. It is a historical benchmark recipe, not a claim that the current upstream release needs this branch. Start with a working NVIDIA driver, CUDA 12.8 toolkit, C++ toolchain, Git, CMake and the Hugging Face CLI.

1. Pin the build. Download every shard.

git clone --branch glm5next/upstream https://github.com/unslothai/llama.cpp.git
cd llama.cpp
git checkout 86ebfef
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 \
  -DLLAMA_CURL=OFF -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build -j 16 --target llama-server

hf download unsloth/GLM-5.3-Flash-GGUF \
  --include 'UD-IQ4_XS/*.gguf' --local-dir ./models/glm53

The download command follows the model repository’s current revision. To match this run, compare all five files with the recorded SHA-256 manifest; stop if any differ. The sha256sum command below is run from the shard directory. No model weights are hosted on this site.

cd models/glm53/UD-IQ4_XS
# Save the linked manifest here as glm-iq4-sha256.txt first.
sha256sum --check glm-iq4-sha256.txt
cd ../../..

2. Start the agent configuration.

The 4-, 8- and 16-agent waves used 16 slots × 65,536 tokens, not a million tokens per agent. MTP was off. Run from the llama.cpp directory; keep the server on loopback and use an authenticated tunnel or gateway for remote access.

CUDA_VISIBLE_DEVICES=0,1 ./build/bin/llama-server \
  -m ./models/glm53/UD-IQ4_XS/GLM-5.3-Flash-UD-IQ4_XS-00001-of-00005.gguf \
  --host 127.0.0.1 --port 8080 \
  -ngl 999 -sm layer -fa on --jinja \
  -c 1048576 --parallel 16 -t 8 -b 2048 -ub 512 \
  --cache-ram 65536 --metrics

Do not add --kv-unified when reproducing these agent waves. The short-request table below used --parallel 32 with the same total context, giving 32K per slot. The 32-agent wave also used 32 × 32K; the 64-agent wave used 64 × 16K. Smaller per-slot contexts also meant smaller output limits.

3. Check one request before adding agents.

curl --fail http://127.0.0.1:8080/health
curl --fail http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  --data '{"messages":[{"role":"user","content":"What is 2 + 2?"}],"temperature":1.0,"top_p":0.95,"max_tokens":512}'

This is a connectivity check. A valid response, particularly one stopped at the token limit while still reasoning, is not a passed coding test. Our next gate was a single agent that actually wrote a file, ran a tool and passed its task check before the concurrent waves.

A setting that failed: -ub 2048 crashed this build on a longer prompt. We used -ub 512. Tensor split was unsupported for this architecture in the pinned build, so these results use layer split. Neither observation is a claim about every newer runtime.

Short requests: more total output, less speed each.

Each row is one concurrent batch with short prompts and 512 generated tokens per request, MTP off. Per-request decode speed comes from the server’s timings; aggregate throughput divides all output by batch wall time. They measure different things.

Download the request results, including the MTP comparison (CSV).

Long inputs changed the wait: a separate single request with 19,152 prompt tokens took 15.49 seconds to its first token, versus 1.70 seconds for 2,465 tokens. Those two prefill observations are not measurements of eight simultaneous long-context sessions.

Now ask it to finish something.

The agent waves used small Python, Rust and HTML tasks, with file and shell tools. External MCP integrations were disabled. Every wave had a 30-minute cap per task and post-run checks of the files produced. Python tasks used assertions; Rust tasks checked compilation and expected output; HTML checks only required a file of sufficient length containing a script. These are basic task checks, not a standard coding benchmark or a comprehensive quality evaluation.

Download the task outcomes and recorded times (CSV).

How we calculated the medians: we averaged the two central per-task times in each even-sized group. The original harness summary incorrectly chose only the upper central value. The eight-agent median is 272.0 seconds (4.5 minutes), rather than the summary’s 325.5 seconds. Times include waiting, tools and the harness’s polling/check overhead; capped or failed runs remain in the table.

Passed checks and finished agents are different counts. At 32 agents, 26 finished normally, five hit the cap and one timed out. Four of the capped agents had already produced files that passed the checks, giving 30 passes. At 64 agents, 19 finished normally, 20 hit the cap and 25 ended in errors. Some passing files came from interrupted runs.

Task mix, context allowance and output cap changed across the larger waves, so this is not a controlled scaling curve. Nor does passing a small task prove general coding quality. No comparison with BF16 establishes the quality cost of this quantisation.

Our reading: eight agents can do useful work on this stack, but “all passed” hides minutes of waiting. Increasing the slot count to 64 did not make the box a comfortable 64-person coding server.

MTP helped the speed number. Did it help the task?

Multi-token prediction, enabled here with --spec-type draft-mtp --spec-draft-n-max 2, raised single-request median decode speed from 56.1 to 78.8 tok/s. At eight concurrent short requests it fell from 22.8 to 21.0 tok/s. A single-user result was a poor guide to the busy-server result.

The separate 16-agent MTP wave still passed 16/16 checks, but recalculated median task time was 11.9 minutes versus 11.7 without MTP. It generated more tokens and finished the whole wave sooner; that does not establish a median task-latency improvement. One wave per setting is too little to distinguish a small effect from run-to-run variation.

What we would test next.

Repeat the same task set several times at four and eight agents, then compare runtimes while holding the model, task checks, prompt history and output budgets fixed. Measure first-token waits and tool-step latency alongside whether the resulting files pass. That would make a stronger case for choosing a serving stack.

These results do not establish the fastest available implementation, equivalence to higher-precision weights, production reliability or a supported customer count. The 600 W Server Edition cards and large host-memory allocation also differ from a desktop or Max-Q system. The current TQ–32 is a single RTX 5090 workstation; it is not the two-card system tested here.

The downloads contain numeric measurements, units and test settings. Use the data notes to interpret them. The upstream model and runtime are linked above; the results reported here are our own.