Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:
jobs:
pre-commit:
runs-on: ubuntu-latest
name: Run pre-commit hooks on Go, Rust, JavaScripts and Python files
name: Run pre-commit hooks on Go, Rust, JavaScripts, Markdown and Python files

steps:
- name: Check out the repo
Expand Down Expand Up @@ -45,7 +45,8 @@ jobs:
sudo apt-get install -y \
make \
build-essential \
pkg-config
pkg-config
npm install -g markdownlint-cli

- name: Cache Rust dependencies
uses: actions/cache@v4
Expand Down Expand Up @@ -81,10 +82,10 @@ jobs:
- name: Install pre-commit
run: pip install pre-commit

- name: Run pre-commit on Go, Rust, JavaScript and Python files
- name: Run pre-commit on Go, Rust, JavaScript, Markdown and Python files
run: |
# Find all Go, Rust, JavaScripts and Python files (excluding vendored/generated code)
FILES=$(find . -type f \( -name "*.go" -o -name "*.rs" -o -name "*.py" -o -name "*.js" \) \
# Find all Go, Rust, JavaScripts, Markdown and Python files (excluding vendored/generated code)
FILES=$(find . -type f \( -name "*.go" -o -name "*.rs" -o -name "*.py" -o -name "*.js" -o -name "*.md" \) \
! -path "./target/*" \
! -path "./candle-binding/target/*" \
! -path "./.git/*" \
Expand All @@ -99,7 +100,7 @@ jobs:
echo "Running pre-commit on files: $FILES"
pre-commit run --files $FILES
else
echo "No Go, Rust, JavaScript or Python files found to check"
echo "No Go, Rust, JavaScript, Markdown or Python files found to check"
fi

- name: Show pre-commit results
Expand Down
12 changes: 11 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
# Basic hooks for Go, Rust, Python files only
# Basic hooks for Go, Rust, Python And JavaScript files only
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
Expand All @@ -22,6 +22,16 @@ repos:
language: system
files: \.go$

# Markdown specific hooks
- repo: local
hooks:
- id: md-fmt
name: md fmt
entry: bash -c "make markdown-lint"
language: system
files: \.md$
exclude: ^(\node_modules/)

# JavaScript specific hooks
- repo: local
hooks:
Expand Down
34 changes: 28 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,18 +29,22 @@ Before you begin, ensure you have the following installed:
### Initial Setup

1. **Clone the repository:**

```bash
git clone https://github.com/vllm-project/semantic-router.git
cd semantic-router
```

2. **Download required models:**

```bash
make download-models
```

This downloads the pre-trained classification models from Hugging Face.

3. **Install Python dependencies(Optional):**

```bash
# For training and development
pip install -r requirements.txt
Expand All @@ -54,30 +58,35 @@ Before you begin, ensure you have the following installed:
The project consists of multiple components that need to be built in order:

### Build Everything

```bash
make build
```

### Build Individual Components

1. **Rust library (Candle binding):**

```bash
make rust
```

2. **Go router:**

```bash
make build-router
```

### Running the System

1. **Start Envoy proxy** (in one terminal):

```bash
make run-envoy
```

2. **Start the semantic router** (in another terminal):

```bash
make run-router
```
Expand All @@ -87,16 +96,19 @@ make build
### Unit Tests

1. **Test Rust bindings:**

```bash
make test-binding
```

2. **Test Go semantic router:**

```bash
make test-semantic-router
```

3. **Test individual classifiers:**

```bash
make test-category-classifier
make test-pii-classifier
Expand Down Expand Up @@ -141,32 +153,35 @@ python e2e-tests/run_all_tests.py --check-only

The test suite includes:

+ Basic client request tests
+ Envoy ExtProc interaction tests
+ Router classification tests
+ Semantic cache tests
+ Category-specific tests
+ Metrics validation tests
- Basic client request tests
- Envoy ExtProc interaction tests
- Router classification tests
- Semantic cache tests
- Category-specific tests
- Metrics validation tests

## Development Workflow

### Making Changes

1. **Create a feature branch:**

```bash
git checkout -b feature/your-feature-name
```

2. **Make your changes** following the project structure and coding standards.

3. **Build and test:**

```bash
make clean
make build
make test
```

4. **Run end-to-end tests:**

```bash
# Start services
make run-envoy &
Expand All @@ -179,6 +194,7 @@ The test suite includes:
5. **Commit your changes:**

Commit your changes with a clear message, making sure to **sign off** on your work using the `-s` flag. This is required by the project's **Developer Certificate of Origin (DCO)**.

```bash
git add .
git commit -s -m "feat: add your feature description"
Expand All @@ -197,6 +213,7 @@ The test suite includes:
Before submitting a PR, please run the pre-commit hooks to ensure code quality and consistency. **These checks are mandatory** and will be automatically run on every commit once installed.

**Step 1: Install pre-commit tool**

```bash
# Using pip (recommended)
pip install pre-commit
Expand All @@ -209,6 +226,7 @@ brew install pre-commit
```

**Step 2: Install pre-commit hooks for this repository**

```bash
# Install pre-commit hooks
pre-commit install
Expand All @@ -218,6 +236,7 @@ pre-commit run --all-files
```

### Go Code

- Follow standard Go formatting (`gofmt`)
- Use meaningful variable and function names
- Add comments for exported functions and types
Expand All @@ -228,19 +247,22 @@ pre-commit run --all-files
- The CI will automatically check that `go.mod` and `go.sum` files are tidy using `make check-go-mod-tidy`

### Rust Code

- Follow Rust formatting (`cargo fmt`)
- Use `cargo clippy` for linting
- Handle errors appropriately with `Result` types
- Document public APIs

### Python Code

- Follow PEP 8 style guidelines
- Use type hints where appropriate
- Write docstrings for functions and classes

## Submitting Changes

1. **Ensure all tests pass:**

```bash
make test
python e2e-tests/run_all_tests.py
Expand Down
8 changes: 8 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -343,3 +343,11 @@ docs-lint:
docs-lint-fix:
@echo "Fixing documentation lint issues..."
cd website && npm run lint:fix

markdown-lint:
@echo "Linting markdown files..."
markdownlint -c markdownlint.yaml "**/*.md" --ignore node_modules --ignore website/node_modules

markdown-lint-fix:
@echo "Fixing markdown lint issues..."
markdownlint -c markdownlint.yaml "**/*.md" --ignore node_modules --ignore website/node_modules --fix
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@
[![Hugging Face](https://img.shields.io/badge/🤗%20Hugging%20Face-Community-yellow)](https://huggingface.co/LLM-Semantic-Router)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
[![Crates.io](https://img.shields.io/crates/v/candle-semantic-router.svg)](https://crates.io/crates/candle-semantic-router)
![](https://github.com/vllm-project/semantic-router/workflows/Test%20And%20Build/badge.svg)
![Test And Build](https://github.com/vllm-project/semantic-router/workflows/Test%20And%20Build/badge.svg)

**📚 [Complete Documentation](https://vllm-semantic-router.com) | 🚀 [Quick Start](https://vllm-semantic-router.com/docs/getting-started/installation) | 📣 [Blog](https://vllm-semantic-router.com/blog/) | 📖 [API Reference](https://vllm-semantic-router.com/docs/api/router/)**

![](./website/static/img/code.png)
![code](./website/static/img/code.png)

</div>

## Innovations ✨

![](./website/static/img/architecture.png)
![architecture](./website/static/img/architecture.png)

### Intelligent Routing 🧠

Expand All @@ -35,6 +35,7 @@ The screenshot below shows the LLM Router dashboard in Grafana.
![LLM Router Dashboard](./website/static/img/grafana_screenshot.png)

The router is implemented in two ways:

- Golang (with Rust FFI based on the [candle](https://github.com/huggingface/candle) rust ML framework)
- Python
Benchmarking will be conducted to determine the best implementation.
Expand Down Expand Up @@ -64,6 +65,7 @@ For comprehensive documentation including detailed setup instructions, architect
**👉 [Complete Documentation at Read the Docs](https://vllm-semantic-router.com/)**

The documentation includes:

- **[Installation Guide](https://vllm-semantic-router.com/docs/getting-started/installation/)** - Complete setup instructions
- **[System Architecture](https://vllm-semantic-router.com/docs/architecture/system-architecture/)** - Technical deep dive
- **[Model Training](https://vllm-semantic-router.com/docs/training/training-overview/)** - How classification models work
Expand All @@ -90,4 +92,4 @@ If you find Semantic Router helpful in your research or projects, please conside

We opened the project at Aug 31, 2025. We love open source and collaboration ❤️

[![Star History Chart](https://api.star-history.com/svg?repos=vllm-project/semantic-router&type=Date)](https://www.star-history.com/#vllm-project/semantic-router&Date)
[![Star History Chart](https://api.star-history.com/svg?repos=vllm-project/semantic-router&type=Date)](https://www.star-history.com/#vllm-project/semantic-router&Date)
4 changes: 3 additions & 1 deletion candle-binding/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,11 @@ go test -v

- The `-v` flag enables verbose output.
- If you want to run a specific test, use:

```sh
go test -v -run TestName
```

Replace `TestName` with the name of the test function.

## Troubleshooting
Expand All @@ -46,4 +48,4 @@ go test -v
## Notes

- The Go tests depend on the native library being present and correctly built.
- Some tests may download data from the internet (e.g., from norvig.com).
- Some tests may download data from the internet (e.g., from norvig.com).
1 change: 0 additions & 1 deletion deploy/kubernetes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ The deployment consists of:

## Deployment


```bash
kubectl apply -k deploy/kubernetes/

Expand Down
4 changes: 4 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,25 @@ This Docker Compose configuration allows you to quickly run Semantic Router + En
## Install in Docker Compose

1. **Clone the repository and navigate to the project directory**

```bash
git clone <repository-url>
cd semantic_router
```

2. **Download required models** (if not already present):

```bash
make download-models
```

This will download the necessary ML models for classification:
- Category classifier (ModernBERT-base)
- PII classifier (ModernBERT-base)
- Jailbreak classifier (ModernBERT-base)

3. **Start the services using Docker Compose**

```bash
# Start core services (semantic-router + envoy)
docker-compose up --build
Expand Down
4 changes: 3 additions & 1 deletion e2e-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,13 @@ This test suite provides a progressive approach to testing the Semantic Router,
## Running Tests

Individual tests can be run with:

```
python tests/XX-test-name.py
```

Or run all tests sequentially with:

```
cd tests && python -m pytest
```
Expand All @@ -51,4 +53,4 @@ cd tests && python -m pytest

- Envoy must be running (make run-envoy)
- Router must be running (make run-router)
- Python dependencies installed
- Python dependencies installed
Loading
Loading