Files
club-3090/scripts/lib
0b4e694ad3 feat(llama.cpp): Structured-CoT bounded-thinking compose + grammar-dialect fix (#214)
* Add llama.cpp bounded-thinking compose

* fix(structured-cot/llamacpp): ship no-underscore grammar variant + record on-rig validation

On-rig validation found the xgrammar deepseek-scratchpad.gbnf does NOT parse under
llama.cpp's GBNF — underscored rule names (line_char/line_chars) trip
"parse: error parsing grammar: expecting newline or end at _char", and llama-server
silently falls back to UNCONSTRAINED generation (FREE==FSM token counts — silent no-fire).

Fix: deepseek-scratchpad.llamacpp.gbnf — identical language, rule names with no
underscores (llama.cpp GBNF is [a-zA-Z0-9-] only). Validated 2026-05-24 on 1x 3090:
parses, fires (reasoning_content = PLAN/NOTE/VERDICT), coexists with --spec-type
draft-mtp (draft accept 0.66-0.88). FREE->FSM 2113->1009 tok (2.1x) and 934->331 (2.8x) @ temp 0.6.

Repointed the llama.cpp compose + READMEs + STRUCTURED_COT.md llama.cpp examples to the
variant; left the vLLM grammar + refs untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* docs(structured-cot): redact pre-existing /home/wasif absolute paths

Pre-commit leak scan caught host-local /home/wasif/structured-cot/runs/... paths in
the 'What's saved on disk' section — pre-existing (2026-04-30 doc), not introduced by
this branch. Redacted to host-local relative form per no-internal-paths-in-public-docs.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: noonghunna <[email protected]>
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-24 11:40:42 +05:00
..