File tree Expand file tree Collapse file tree 1 file changed +8
-9
lines changed Expand file tree Collapse file tree 1 file changed +8
-9
lines changed Original file line number Diff line number Diff line change 66
77- State-of-the-art LLMs support
88- Continuous Batching via [ vLLM] ( https://github.com/vllm-project/vllm )
9- - Quantization([ issue #37 ] ( https://github.com/InftyAI/llmlite/issues/37 ) )
10- - Adapter support ([ issue #51 ] ( https://github.com/InftyAI/llmlite/issues/51 ) )
11- - Streaming support ([ issue #52 ] ( https://github.com/InftyAI/llmlite/issues/52 ) )
9+ - Quantization([ issue #37 ] ( < https://github.com/InftyAI/llmlite/issues/37 > ))
10+ - Loading specific adapters ([ issue #51 ] ( https://github.com/InftyAI/llmlite/issues/51 ) )
11+ - Streaming ([ issue #52 ] ( https://github.com/InftyAI/llmlite/issues/52 ) )
1212
1313### Model Support
1414
1818| Llama-2 | Done ✅ | Yes | |
1919| CodeLlama | Done ✅ | Yes | |
2020| ChatGLM2 | Done ✅ | No | |
21- | ChatGLM3 | WIP ⏳ | Yes | |
2221| Baichuan2 | Done ✅ | Yes | |
22+ | ChatGLM3 | WIP ⏳ | Yes | |
2323| Claude-2 | RoadMap 📋 | | [ issue #7 ] ( https://github.com/InftyAI/ChatLLM/issues/7 )
2424| Falcon | RoadMap 📋 | | [ issue #8 ] ( https://github.com/InftyAI/ChatLLM/issues/8 )
2525| StableLM | RoadMap 📋 | | [ issue #11 ] ( https://github.com/InftyAI/ChatLLM/issues/11 ) |
26- | Baichuan2 | RoadMap 📋 | | [ issue #34 ] ( https://github.com/InftyAI/llmlite/issues/34 )
2726
2827### Backend Support
2928
30- | backend | State | Note |
31- | ---- | ---- | ---- |
32- | [ huggingface] ( https://github.com/huggingface ) | Done ✅ | Support by huggingface pipeline |
33- | [ vLLM] ( https://github.com/vllm-project/vllm ) | Done ✅ | |
29+ | backend | State |
30+ | ---- | ---- |
31+ | [ huggingface] ( https://github.com/huggingface ) | Done ✅ |
32+ | [ vLLM] ( https://github.com/vllm-project/vllm ) | Done ✅ |
3433
3534## How to install
3635
You can’t perform that action at this time.
0 commit comments