Dual composes inject PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,...
for the PCIe (custom-all-reduce OFF) path. On NVLink / patched-PCIe-P2P
rigs detect_nvlink.sh turns custom all-reduce ON, where expandable_segments
backs allocations with a cuMemMap VA range that cudaIpcGetMemHandle rejects
during graph-buffer registration -> custom_all_reduce.cuh:455 'invalid
argument', worker dies at boot.
The prior fix clobbered PYTORCH_CUDA_ALLOC_CONF to max_split_size_mb:512,
discarding any other knob the user set (custom max_split_size_mb,
garbage_collection_threshold, ...). Instead strip ONLY the expandable_segments
token on the P2P path and keep the rest; fall back to max_split_size_mb:512
when nothing remains. The PCIe path keeps expandable_segments untouched.
Strengthen test-detect-nvlink-alloc-conf.sh to assert the surgical contract
(preserve neighbors, strip in any position, no dangling commas) and add the
vllm#42609 row to docs/UPSTREAM.md (its "DP>1 AND TP>1" title under-scopes
the bug; the same cuh:455 crash fires TP-only -- cf. vllm#42583 / vllm#45604).
Validated on a 2x RTX 3090 NVLink rig (TP=2, NV4): before, boot crashes at
custom_all_reduce.cuh:455 'invalid argument'; after, the dual stack boots
clean (custom AR ON, expandable_segments stripped), /v1/models ready, serves
a completion.
Co-authored-by: Cursor <[email protected]>
