This repository provides the installation and usage of various LLM Inference frameworks on Polaris and Sophia. The frameworks currently tested are:
-
Notifications
You must be signed in to change notification settings - Fork 1
This repo hosts the different ways to run vllm on ANL HPC system
License
atanikan/llm-inference-service
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
This repo hosts the different ways to run vllm on ANL HPC system
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published