[Refactor]Abstract Platform Interface for Distributed Backend and Add xccl Support for Intel XPU #19410
Conversation
dbyoung18
requested review from
WoosukKwon,
robertgshaw2-redhat,
njhill,
ywang96,
comaniac,
alexm-redhat,
zhuohan123 and
youkaichao
as code owners
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @dbyoung18, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request enhances the Intel XPU platform support by integrating the xccl distributed backend, which is available in PyTorch 2.8+. It refactors the platform abstraction to include a dist_backend attribute, enabling dynamic selection of the communication backend based on the environment and PyTorch version, and updates the distributed initialization process to utilize this selected backend.
Highlights
- XPU Distributed Backend: Added support for the
xccldistributed backend for Intel XPU platforms, leveraging PyTorch versions 2.8 and above. - Platform Abstraction: Introduced a
dist_backendattribute to the basePlatformclass and its subclasses (CudaPlatformBase,XPUPlatform) to abstract the distributed communication backend (e.g.,nccl,ccl,xccl). - Auto-detection and Configuration: Implemented logic in the XPU platform initialization to auto-detect the preferred backend (
xcclfor PyTorch >= 2.8 if available, otherwiseccl) and allow users to override this via theXPU_CCL_BACKENDenvironment variable. - Worker Initialization: Updated the worker initialization logic (
init_worker_distributed_environment) to use thedist_backendspecified by the detected/configured platform instead of hardcoding a backend likecclfor XPU.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configureGemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
|
/gemini review |
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
dbyoung18
changed the title
dbyoung18
changed the title
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
/gemini review |
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
* add supports_xccl check to utils * switch by export XPU_CCL_BACKEND=ccl | xccl * default dist_backend=ccl(<2.8), xccl(>=2.8) Signed-off-by: dbyoung18 <[email protected]>
Signed-off-by: Kunshang Ji <[email protected]>
Signed-off-by: dbyoung18 <[email protected]>
Signed-off-by: dbyoung18 <[email protected]>
…worker Signed-off-by: dbyoung18 <[email protected]>
Signed-off-by: dbyoung18 <[email protected]>
Signed-off-by: dbyoung18 <[email protected]>
Signed-off-by: dbyoung18 <[email protected]>
dbyoung18
force-pushed
the
dbyoung/enable_xccl
branch
from
e6d20d0 to
9140083
Compare
|
/gemini review |
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
This PR adds
xcclsupport for XPU Platform since pytorch supports xccl as xpu distributed backend since 2.8 release:dist_backendtoCudaPlatformBasewhich can better abstract distributed backend and be reused duringinit_distributed_environment()supports_xccl()check underutilsdist_backendduring checking platform