Skip to content

Integration from base #2033

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 69 commits into from
Closed

Integration from base #2033

wants to merge 69 commits into from

Conversation

@hsubramony
Copy link

@hsubramony hsubramony commented Oct 14, 2025 •
edited by github-actions bot
Loading

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results

Purpose

Test Plan

Test Result

shepark and others added 30 commits July 1, 2025 18:36
@shepark
merge ace w/o merged_prefill_for_v1 changes
251d173
@bukeao
first nixl version
35c22a2
@bukeao
add logger.info to indicate the completion of kv transfer
5605422
@bukeao
add perf_counter to kv cache transfer
f1fd3a0
@bukeao
disable caching in kv_manager to fix second identical request failed …
5702cc8 
…error
@bukeao
Revert "disable caching in kv_manager to fix second identical request…
e4e1418 
… failed error"

This reverts commit 5702cc8.
@skaulintel
Merge branch 'habana_main' into nixl2_buke
03a7284
@skaulintel
remove lmcache folder
39f9409
@skaulintel
add nixl connector from upstream
188ae95
@skaulintel
add KVTransferParams class back to base.py and change block handling …
c6ac725 
…in nixl connector
@skaulintel
modify kv buffer for hpu
e1404f3
@skaulintel
add hpu functions
12949c0
@bukeao
adding indexing function
094c136
@bukeao
add block_size
d015bbe
@bukeao
change position of block_size
b52daf0
@bukeao
change position of block_size
7cdd181
@bukeao
change dtype of indices
a5f845a
@skaulintel
change enumerate dict to dict.items
b2065a2
@skaulintel
add test hpu disagg accuracy script
e2e76ab
@skaulintel
modify hpu accuracy test
79b7a2f
@skaulintel
original nixl acc script
e7b89b5
@skaulintel
hpu nixl accuracy script changes
28430ac
@skaulintel
remove unnecessary stuff
39ffb32
@bukeao
fix accuracy issue
641af13
@bukeao
move the interface functions from nixl to the their appropriate locat…
cec85fd 
…ions within hpu_model_runner
@skaulintel
temporarily comment out asserts
31ba1df
@bukeao
temperary code to fix the input_batch.req_type has the right flag; te…
f0eb4d2 
…sted in high concurret requests
@libinta
Fix D node processing prompt
0786b65
@libinta
change the chunked prefill max to 8k
61cb50d
@libinta
add requirement and modification to accuracy.sh
390cae5
libinta and others added 28 commits August 29, 2025 20:13
@libinta
test
1be8753
@yeonsily
Added missing import
f1131b1
@bukeao
enable GDR
3c103da
@bukeao
indent typo
4594ca0
@bukeao
first commit for heterogeneous PD: accommodate block_size gap
ed7fe6b
@bukeao
fix indexing bug and enable the 1P1D mode, where P uses TP=1 and D us…
40a53a5 
…es TP=2
@bukeao
add backend_name check
1744167
@libinta @IT-Forrest
fix assertion for non-pd case
257aeb7
@libinta @IT-Forrest
add hpu p2d4
b089409
@bukeao @IT-Forrest
Resolve conflict file after merge buke_hetero_PD2 and libint/debug_ttft
35ee88e
@bukeao @IT-Forrest
fix indexing bug and enable the 1P1D mode, where P uses TP=1 and D us…
c476cb8 
…es TP=2
@IT-Forrest
Add two flags to enable and disable block_size and heterogeneous test
fd9625d
@IT-Forrest
WA crash in block id index but still accuracy issue
44115d5
@bukeao @IT-Forrest
fixed accuracy issue caused by repeat requests
6516466
@IT-Forrest
Fix P/D accuracy issue between Gaudi devices
ea943de
@IT-Forrest
WA accuracy issue caused by inconsistent block ids of hetero devices
fe92235
@IT-Forrest
Integrate Libin's patch for contiguous pa related accuracy issue
af874ef
@IT-Forrest
Merge Libin's contiguous pa patch needs vllm-hpu-extension ce0e48
4cf91b7
@bukeao
Optimize indexing operation and reduce the time for permute, shorten …
30173cb 
…the TTFT
@bukeao
Enable the cpu host buffer in hetero system
62342fb
@IT-Forrest
Remove typo and hard coded script in cpu host buffer support
7d58f17
@libinta @IT-Forrest
Update requirements.txt
c21e9f7
@libinta @IT-Forrest
pull PR 1711 and 1738 for performance improvement
34bd4d8
@yeonsily @IT-Forrest
Fix conflict when merging Disable debug message
8d256d4
@yeonsily @IT-Forrest
Cache the blocks for the request only if enabled.
cf768d2
@yeonsily @IT-Forrest
Disable debugging log
bd91a2a
@yeonsily @IT-Forrest
Fix conflict after merging Disable debug message in the test
03c3de9
@IT-Forrest
Revert "Optimize indexing operation and reduce the time for permute, …
5d5e495 
…shorten the TTFT"

This reverts commit 30173cb.
@PatrykWo
Copy link

Starting from v1.23.0, the vLLM fork will reach end-of-life (EOL) and be deprecated in v1.24.0, remaining functional only for legacy use cases until then. At the same time, the vllm-gaudi plugin will be production-ready in v1.23.0. Contributing to HabanaAI:habana_main is closed.

@PatrykWo PatrykWo closed this Oct 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kzawora-intel kzawora-intel Awaiting requested review from kzawora-intel kzawora-intel will be requested when the pull request is marked ready for review kzawora-intel is a code owner

@madamczyk-intel madamczyk-intel Awaiting requested review from madamczyk-intel madamczyk-intel will be requested when the pull request is marked ready for review madamczyk-intel is a code owner

@michalkuligowski michalkuligowski Awaiting requested review from michalkuligowski michalkuligowski will be requested when the pull request is marked ready for review michalkuligowski is a code owner

@mgawarkiewicz-intel mgawarkiewicz-intel Awaiting requested review from mgawarkiewicz-intel mgawarkiewicz-intel will be requested when the pull request is marked ready for review mgawarkiewicz-intel is a code owner

@vivekgoe vivekgoe Awaiting requested review from vivekgoe vivekgoe will be requested when the pull request is marked ready for review vivekgoe is a code owner

@afierka-intel afierka-intel Awaiting requested review from afierka-intel afierka-intel will be requested when the pull request is marked ready for review afierka-intel is a code owner

@xuechendi xuechendi Awaiting requested review from xuechendi xuechendi will be requested when the pull request is marked ready for review xuechendi is a code owner

@jikunshang jikunshang Awaiting requested review from jikunshang jikunshang will be requested when the pull request is marked ready for review jikunshang is a code owner

@mswiniarsk mswiniarsk Awaiting requested review from mswiniarsk mswiniarsk will be requested when the pull request is marked ready for review mswiniarsk is a code owner

@PatrykWo PatrykWo Awaiting requested review from PatrykWo PatrykWo will be requested when the pull request is marked ready for review PatrykWo is a code owner

@wpyszka wpyszka Awaiting requested review from wpyszka wpyszka will be requested when the pull request is marked ready for review wpyszka is a code owner

@deepvars deepvars Awaiting requested review from deepvars deepvars will be requested when the pull request is marked ready for review deepvars is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@hsubramony @PatrykWo @libinta @shepark @bukeao @skaulintel @yeonsily @IT-Forrest