NVIDIA Build wave 3 — coverage & exclusions

As of 2026-08-06 · source: integrate.api.nvidia.com · 102-model catalog · 8 models added (24 NVIDIA Build rows total)

Third pass over the NVIDIA Build serverless API (integrate.api.nvidia.com/v1, free-tier key): a re-enumeration of the catalog, a live eligibility probe of every id not already covered by the 2026-07-23 sweep, and full benchmark runs for everything that proved servable. This note records what was added, how it scored, and — as before — every candidate that was not benchmarked, and why. The leaderboard lives at the site root.

How the wave ran

Corpus: unchanged — all 300 public tasks plus the private held-out-v2 set (24 never-published tasks with per-task canaries), OpenAI-compatible adapter, deterministic rubric grader. Zero canary hits; no row flagged.
Catalog drift: the catalog shrank from 118 ids (July) to 102 (2026-08-06). Eligibility was again verified with a live request per model, not by name — and this wave surfaced a new failure mode alongside 404: HTTP 410 Gone ("reached its end-of-life date") for ids still present in the catalog.
Effort/budget: every model ran at provider-default reasoning effort (@default). One methodology note: thinkingmachines/inkling only produces a final answer after very long hidden reasoning; it completed under the harness's standard budget-growing no-content retry (the same mechanism the kimi-k3 run established), with the growth ceiling raised from 5,500 to 16,000 output tokens. Base budget was unchanged, so its rows are comparable but its 27 s average latency reflects that reasoning burn.
Resumes: the two July rate-limited stragglers were resumed rather than restarted: z-ai/glm-5.2 completed (222/300 → 300/300); deepseek-ai/deepseek-v4-pro did not (see exclusions).

Results — 8 models added (full coverage)

Rank	Model	Score	95% CI	Held-out gap	Safety risk	Critical	Avg latency
12	thinkingmachines/inkling	55.95%	52.45–59.32	−4.02	116	4	27.2 s
14	z-ai/glm-5.2	54.48%	51.19–57.82	−16.04	266	10	151.1 s
25	nvidia/nemotron-3-nano-omni-30b-a3b-reasoning	46.88%	43.61–50.10	−5.05	500	20	11.5 s
27	mistralai/mistral-nemotron	46.56%	43.60–49.66	−8.83	300	12	8.6 s
34	google/diffusiongemma-26b-a4b-it	42.68%	39.45–45.92	−9.95	308	12	12.4 s
35	nvidia/ising-calibration-1.5-31b	41.96%	38.66–45.50	−11.58	583	23	3.0 s
37	nvidia/nvidia-nemotron-nano-9b-v2	41.57%	38.26–44.89	−6.88	608	24	9.4 s
39	poolside/laguna-xs-2.1	40.92%	37.35–44.60	−3.10	500	20	12.3 s

Headline: thinkingmachines/inkling at 55.95% is the new NVIDIA Build free-tier best, displacing nemotron-3-ultra-550b (54.39%) — and the first free-tier row to break into the board's top half. It was unusable in July (empty response bodies); the free tier now serves it reliably.

Caveats: glm-5.2 carries one of the largest held-out gaps on the board (−16.04; the variant-gap caveat on the site root applies) and averaged 151 s per response on this tier. ising-calibration-1.5-31b is an NVIDIA-internal-looking model of unclear intended purpose that nonetheless completes chat tasks; it is reported as measured. The frontier picture is unchanged: every free-tier model remains below the hosted leaders (Fable/Kimi/GPT-5.5 band).

Not benchmarked, and why

A · Not servable to full coverage on the free tier

deepseek-ai/deepseek-v4-pro — held back, still 212/300. Third session of attempts (2026-07-23, 07-25, 08-06 twice): requests now fail at the network level (fetch failed / gateway timeouts >300 s) before any 429 comes back. Not scored on a partial set; a paid-tier or off-peak completion remains the only path.

B · Listed in the catalog but not served (HTTP 404 on a live request)

nvidia/cosmos-reason2-8b, writer/palmyra-creative-122b, nvidia/nemotron-nano-3-30b-a3b — "Function not found" / "Model not found" on this key, 2026-08-06.

C · HTTP 410 Gone — end-of-life (new since July)

bytedance/seed-oss-36b-instruct, mistralai/ministral-14b-instruct-2512 — the API now answers with 410 "reached its end-of-life date". In July these still accepted requests (returning empty bodies); they are gone for good and drop off the retry list.

D · Remaining uncovered catalog ids

The rest of the 102-id catalog stays excluded for the same reasons documented in the July briefing: non-chat models (embeddings, rerankers, vision encoders, guard/reward/safety classifiers, translators, parsers), legacy flagships that 404, and superseded or domain-narrow instruct models (llama-2/3.1/3.2 era, code-completion models, 7–8B-class models predating their own successors on this leaderboard).