Skip to content

25.09 alpha rope split concat fusion #1826

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 19 commits into
base: 25.09-alpha
Choose a base branch
from
Open

25.09 alpha rope split concat fusion #1826

Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
9d35d3b
Avoid split and concat with RoPE
vasunvidia Jul 22, 2025
51a72fc
Fix formatting errors
vasunvidia Sep 5, 2025
99cecae
Fix pending lint error
vasunvidia Sep 5, 2025
a923f60
Address review comments.
vasunvidia Sep 5, 2025
5c265bd
Remove redundant in_decode_mode check.
vasunvidia Sep 5, 2025
fd6970e
Add test and minor review comment
vasunvidia Sep 9, 2025
ea26b63
Merge branch 'main' into vrengasamy/rope_split_concat_fusion
vasunvidia Sep 9, 2025
8d39ffe
Enable fused_single_qkv_rope in unit test for Attention
vasunvidia Sep 10, 2025
e56883a
Merge branch 'main' into vrengasamy/rope_split_concat_fusion
vasunvidia Sep 10, 2025
b067e0b
Fix for NoneType object is not subscriptable
vasunvidia Sep 11, 2025
31bb955
Merge branch 'main' into vrengasamy/rope_split_concat_fusion
vasunvidia Sep 15, 2025
f8cf909
Skip test if Fused QKV RoPE not available
vasunvidia Sep 15, 2025
0291c0d
Disable split_qkv check for CrossAttention
vasunvidia Sep 16, 2025
fa5082f
ADLR/megatron-lm!4013 - Undo deletion of nemotron_h_aligned template
tdene Sep 19, 2025
c223178
ADLR/megatron-lm!3991 - Fix the print err when torch is not intialized
gdengk Sep 19, 2025
e5bc924
ADLR/megatron-lm!3855 - Enabling mixing SWA with full attention [gpt-…
cuichenx Sep 19, 2025
8479eb3
ADLR/megatron-lm!4019 - [Flux] Fix Full Iter CUDA Graph Issues
alpha0422 Sep 20, 2025
1864b53
Merge branch megatron-lm:main into vrengasamy/rope_split_concat_fusion
vasunvidia Sep 22, 2025
f3a6983
Merge branch 'vrengasamy/rope_split_concat_fusion' into 25.09-alpha_r…
vasunvidia Sep 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

These merge commits were added into this branch cleanly.

There are no new changes to show.