[model] feat: add qwen3-122b long seq launch script for ascend#6807
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new launch script for running GRPO training with Qwen3.5-122B using Megatron and vLLM on Ascend NPUs. A critical issue was identified where the undefined variable NDEVICES_PER_NODE is referenced, which will cause the script to crash immediately due to the set -u option. It is recommended to replace it with the defined NPUS_PER_NODE variable.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| trainer.logger='["console"]' | ||
| trainer.project_name=${project_name} | ||
| trainer.experiment_name=${exp_name} | ||
| trainer.n_gpus_per_node=${NDEVICES_PER_NODE} |
There was a problem hiding this comment.
The variable NDEVICES_PER_NODE is not defined in this script. Since set -u is enabled, referencing an unbound variable will cause the script to crash immediately upon execution. It should be replaced with NPUS_PER_NODE which is defined on line 17.
| trainer.n_gpus_per_node=${NDEVICES_PER_NODE} | |
| trainer.n_gpus_per_node=${NPUS_PER_NODE} |
References
- Enable
set -xeuo pipefailin shell scripts to ensure that the script exits on errors, treats unset variables as errors, and pipelines fail correctly.
Co-authored-by: zhouhengan1211 <zhouhengan1211@gmail.com> Signed-off-by: zjchenn <zjchenn@gmail.com>
b495e51 to
f0ea249
Compare
What does this PR do?
Add the Ascend Megatron GRPO script for Qwen3.5-122B-A10B long-sequence training.
Checklist Before Starting
[{modules}] {type}: {description}(This will be checked by the CI){modules}includefsdp,megatron,veomni,sglang,vllm,rollout,trainer,ci,training_utils,recipe,hardware,deployment,ray,worker,single_controller,misc,perf,model,algo,env,tool,ckpt,doc,data,cfg,reward,fully_async,one_step_off,like[megatron, fsdp, doc]{type}is infeat,fix,refactor,chore,test[BREAKING]to the beginning of the title.[BREAKING][fsdp, megatron] feat: dynamic batchingTest
API and Usage Example
# Add code snippet or script demonstrating how to use thisDesign & Code Changes
Checklist Before Submitting
Important
Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.
pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=alwaysci-requestchannel in theverlSlack workspace. (If not accessible, please try the Feishu group (飞书群).)recipesubmodule, please also update the reference to the submodule commit viagit submodule update --remoteorcd recipe && git pull origin main.