Closes the deeper architectural concern @snoby raised in #37: the
host-mount of /opt/ai/vllm-src/...marlin.py was a "works on the
maintainer's machine" pattern. Auto-cloning at setup time (commit
2e934ad) papered over the UX gap but the fundamental host-filesystem
dependency remained.
Vendored approach:
- New `models/qwen3.6-27b/vllm/patches/vllm-marlin-pad/` directory
with the two patched files (marlin.py + MPLinearKernel.py, ~10 KB
total) copied from the noonghunna/vllm marlin-pad-sub-tile-n branch
(commit 67f8c2b)
- Verified upstream files have NOT changed between the fork base
(cc3993b) and our pinned vLLM image SHA (7a1eb8ac2ec) — the patch
applies cleanly to today's image, no rebase needed
- README.md in vllm-marlin-pad/ documents provenance (Apache 2.0,
commit SHA, sync procedure for future image bumps)
Compose changes (4 files):
- dual.yml, dual-turbo.yml, dual-dflash.yml, dual-dflash-noviz.yml
all updated to mount `../patches/vllm-marlin-pad/marlin.py` and
`../patches/vllm-marlin-pad/MPLinearKernel.py` instead of
`/opt/ai/vllm-src/...`. Repo-relative paths — no host filesystem
dependency.
- All 4 composes validated as parsing cleanly via PyYAML.
setup.sh changes:
- Removed the WITH_MARLIN_PATCH=1 auto-clone block (40 lines) added
in 2e934ad — no longer needed since the patched files are vendored.
- Removed the WITH_MARLIN_PATCH env-var documentation from header.
- Replaced the dual-card setup hint with a one-liner that no host
clone is required.
Doc updates:
- vllm/README.md: replaced "External: /opt/ai/vllm-src/..." line with
reference to vendored vllm-marlin-pad/ directory
- vllm/patches/README.md: rewrote the "Marlin pad-sub-tile-n" section
to reflect vendored-not-cloned setup; preserved the
brittleness-note for upstream-refactor-vs-our-fork-base concerns.
Why not a runtime text-patch sidecar (like patch_tolist_cudagraph.py):
- The Marlin patch is ~120 lines of substantive code (new
_maybe_pad_n method + edits to process_weights_after_loading and
apply_weights). Text-patches work for ~5-10 line surgical changes;
a 120-line surface is brittle and hard to review.
- Vendoring two files with a clear sync procedure is the cleaner
trade. Drops out entirely (delete the directory + 4 mount lines)
when vllm#40361 lands upstream.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
