Click to expand
nginx-defender is a high-performance, enterprise-grade Web Application Firewall (WAF) and threat detection system engineered for modern web infrastructure. Built with Go and designed for scalability, it provides real-time threat mitigation, machine learning-based anomaly detection, and comprehensive security analytics.
graph TD
A[Incoming Traffic] --> B[Log Parser]
B --> C[Threat Detection Engine]
C --> D[ML Anomaly Detection]
C --> E[Pattern Matching]
C --> F[Rate Limiting]
D --> G[Firewall Manager]
E --> G
F --> G
G --> H[iptables/nftables/pf]
G --> I[Notification System]
I --> J[Dashboard/API]
Feature | nginx-defender | Commercial WAFs | Open Source WAFs |
---|---|---|---|
Real-time ML Detection | Native | Limited | None |
Multi-backend Firewall | 5+ backends | Proprietary | Limited |
Advanced Analytics | Built-in | Expensive addon | Basic |
Geographic Intelligence | Integrated | Licensed | Manual |
Clustering Support | Native | Enterprise only | None |
Container Ready | OCI compliant | Legacy | Configuration heavy |
High-Level Architecture Diagram
┌────────────────────────────────────────────────────────────────┐
│ nginx-defender │
├────────────────────────────────────────────────────────────────┤
│ ┌───────────────┐ ┌──────────────┐ ┌─────────────────────┐ │
│ │ Log Monitor │ │ Threat Intel │ │ Web Dashboard │ │
│ │ (Async) │ │ (Real-time)│ │ (WebSocket) │ │
│ └───────┬───────┘ └──────┬───────┘ └─────────┬───────────┘ │
│ │ │ │ │
│ ┌───────▼─────────────────▼────────────────────▼───────────┐ │
│ │ Threat Detection Engine │ │
│ │ ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐ │ │
│ │ │ Pattern │ │ ML Anomaly │ │ Rate Limiting │ │ │
│ │ │ Matcher │ │ Detection │ │ Engine │ │ │
│ │ └─────────────┘ └──────────────┘ └─────────────────┘ │ │
│ └─────────────────────────┬────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────────▼────────────────────────────────┐ │
│ │ Firewall Manager │ │
│ │ ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐ │ │
│ │ │ iptables │ │ nftables │ │ pf │ │ │
│ │ │ Backend │ │ Backend │ │ Backend │ │ │
│ │ └─────────────┘ └──────────────┘ └─────────────────┘ │ │
│ └─────────────────────────┬────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────────▼────────────────────────────────┐ │
│ │ Multi-Channel Notification System │ │
│ │ Telegram │ Slack │ Email │ Webhook │ Discord │ │
│ └──────────────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────┘
Component | Responsibility | Technology | Scalability |
---|---|---|---|
Log Monitor | Real-time log parsing and streaming | Go channels, fsnotify | Horizontal |
Detection Engine | Threat analysis and classification | Regex, ML algorithms | Vertical |
Firewall Manager | Rule application and management | System calls, netlink | Horizontal |
Notification System | Alert distribution and escalation | HTTP clients, WebSockets | Horizontal |
Web Dashboard | Management interface and analytics | HTTP server, static assets | Horizontal |
Metrics Collector | Performance and security metrics | Prometheus client | Horizontal |
Advanced Pattern Recognition System
- Behavioral Analysis: Statistical anomaly detection using sliding windows
- Attack Vectorization: Feature extraction from HTTP requests
- Model Training: Continuous learning from threat patterns
- False Positive Reduction: Adaptive threshold adjustment
# Example threat patterns
patterns:
sql_injection:
regex: "(?i)(union|select|insert|update|delete|drop|exec|script)"
severity: 9
categories: ["injection", "database"]
xss_detection:
regex: "(?i)(<script|javascript:|vbscript:|onload=|onerror=)"
severity: 8
categories: ["xss", "injection"]
directory_traversal:
regex: "(?i)(\\.\\.[\\/\\\\]|%2e%2e%2f|%252e%252e%252f)"
severity: 7
categories: ["path_traversal", "file_access"]
type RateLimiter struct {
Window time.Duration
Threshold int
Algorithm string // "sliding_window", "token_bucket", "leaky_bucket"
}
Multi-Backend Firewall System
Backend | Platform | Features | Performance |
---|---|---|---|
iptables | Linux | Legacy compatibility, wide support | ~10k rules/sec |
nftables | Linux | Modern netfilter, better performance | ~50k rules/sec |
pf | FreeBSD/macOS | Advanced filtering, built-in NAT | ~30k rules/sec |
Windows Firewall | Windows | Native Windows integration | ~5k rules/sec |
Mock | All | Testing and development | Unlimited |
type Rule struct {
ID string `json:"id"`
IP string `json:"ip"`
Action Action `json:"action"`
Duration time.Duration `json:"duration"`
CreatedAt time.Time `json:"created_at"`
ExpiresAt time.Time `json:"expires_at"`
Reason string `json:"reason"`
ThreatLevel string `json:"threat_level"`
Metadata map[string]string `json:"metadata"`
}
BLOCK
: Complete traffic blockingDROP
: Silent packet droppingREJECT
: Explicit connection rejectionRATE_LIMIT
: Bandwidth throttlingTARPIT
: Connection delay injectionALLOW
: Whitelist override
Real-time Dashboard and Metrics
- Real-time Threat Map: Geographic visualization of attacks
- Performance Metrics: Request processing and response times
- Rule Management: Interactive firewall rule configuration
- Log Analysis: Advanced filtering and search capabilities
- Alert Management: Notification configuration and history
# HELP nginx_defender_requests_total Total number of requests processed
# TYPE nginx_defender_requests_total counter
nginx_defender_requests_total{status="blocked",reason="sql_injection"} 1234
# HELP nginx_defender_response_duration_seconds Request processing duration
# TYPE nginx_defender_response_duration_seconds histogram
nginx_defender_response_duration_seconds_bucket{le="0.1"} 8932
const ws = new WebSocket('ws://localhost:8080/ws');
ws.onmessage = function(event) {
const data = JSON.parse(event.data);
console.log('New threat detected:', data);
};
Multi-Channel Alert Distribution
Channel | Features | Configuration |
---|---|---|
Telegram | Rich formatting, bot integration | Bot token, chat ID |
Slack | Channel posting, thread replies | Webhook URL, channel |
HTML templates, attachments | SMTP configuration | |
Webhook | Custom integrations, JSON payload | Endpoint URL, headers |
Discord | Server notifications, embeds | Webhook URL |
PagerDuty | Incident management, escalation | Integration key |
severity_mapping:
critical: 9-10 # Immediate response required
high: 7-8 # Urgent attention needed
medium: 4-6 # Standard monitoring
low: 1-3 # Informational only
hardware:
cpu: "1 core (x86_64/ARM64)"
memory: "256 MB RAM"
storage: "100 MB disk space"
network: "1 Mbps bandwidth"
software:
os: "Linux 4.0+, macOS 10.15+, FreeBSD 12+"
privileges: "root/administrator (for firewall management)"
dependencies: "None (statically linked binary)"
hardware:
cpu: "4+ cores (x86_64)"
memory: "2+ GB RAM"
storage: "10+ GB SSD"
network: "100+ Mbps bandwidth"
software:
os: "Linux 5.4+ (Ubuntu 20.04+, CentOS 8+)"
kernel: "nftables support enabled"
monitoring: "Prometheus + Grafana"
Metric | Value | Notes |
---|---|---|
Log Processing | 100k+ entries/sec | Single instance |
Rule Application | 50k+ rules/sec | nftables backend |
Memory Usage | <100 MB | Base configuration |
CPU Usage | <5% | Idle state |
Response Time | <1ms | Rule matching |
Concurrent Connections | 10k+ | Web dashboard |
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Client │────│ nginx- │────│ Backend │
│ Traffic │ │ defender │ │ Services │
└─────────────┘ └─────────────┘ └─────────────┘
│
┌─────────────┐
│ Firewall │
│ Rules │
└─────────────┘
# Download latest release
curl -L https://github.com/anipaleja/nginx-defender/releases/latest/download/nginx-defender-linux-amd64.tar.gz | tar -xz
# Install system-wide
sudo mv nginx-defender /usr/local/bin/
sudo chmod +x /usr/local/bin/nginx-defender
# Create configuration directory
sudo mkdir -p /etc/nginx-defender
sudo wget -O /etc/nginx-defender/config.yaml https://raw.githubusercontent.com/anipaleja/nginx-defender/main/config.yaml
# Create systemd service
sudo tee /etc/systemd/system/nginx-defender.service > /dev/null <<EOF
[Unit]
Description=nginx-defender WAF
After=network.target
[Service]
Type=simple
User=root
ExecStart=/usr/local/bin/nginx-defender -config /etc/nginx-defender/config.yaml
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
# Enable and start service
sudo systemctl enable nginx-defender
sudo systemctl start nginx-defender
# Add tap (if available)
brew tap anipaleja/nginx-defender
# Install via Homebrew
brew install nginx-defender
# Start service
brew services start nginx-defender
# Pull latest image
docker pull ghcr.io/anipaleja/nginx-defender:latest
# Run with host networking (recommended for firewall functionality)
docker run -d \
--name nginx-defender \
--network host \
--cap-add NET_ADMIN \
--cap-add NET_RAW \
-v /var/log/nginx:/var/log/nginx:ro \
-v ./config.yaml:/app/config.yaml:ro \
ghcr.io/anipaleja/nginx-defender:latest
version: '3.8'
services:
nginx-defender:
image: ghcr.io/anipaleja/nginx-defender:latest
container_name: nginx-defender
restart: unless-stopped
network_mode: host
cap_add:
- NET_ADMIN
- NET_RAW
volumes:
- /var/log/nginx:/var/log/nginx:ro
- ./config.yaml:/app/config.yaml:ro
- ./data:/app/data
environment:
- LOG_LEVEL=info
- DRY_RUN=false
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
prometheus:
image: prom/prometheus:latest
container_name: prometheus
ports:
- "9090:9090"
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--web.console.libraries=/etc/prometheus/console_libraries'
- '--web.console.templates=/etc/prometheus/consoles'
grafana:
image: grafana/grafana:latest
container_name: grafana
ports:
- "3000:3000"
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
volumes:
- grafana-storage:/var/lib/grafana
volumes:
grafana-storage:
Complete Kubernetes Manifests
apiVersion: v1
kind: Namespace
metadata:
name: nginx-defender
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-defender-config
namespace: nginx-defender
data:
config.yaml: |
firewall:
backend: "iptables"
whitelist:
- "10.0.0.0/8"
- "172.16.0.0/12"
- "192.168.0.0/16"
detection:
enabled: true
patterns_file: "pkg/patterns/common.yaml"
rate_limiting:
enabled: true
window: "1m"
max_requests: 100
server:
host: "0.0.0.0"
port: 8080
metrics:
enabled: true
prometheus:
enabled: true
host: "0.0.0.0"
port: 9090
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: nginx-defender
namespace: nginx-defender
spec:
selector:
matchLabels:
app: nginx-defender
template:
metadata:
labels:
app: nginx-defender
spec:
hostNetwork: true
hostPID: true
containers:
- name: nginx-defender
image: ghcr.io/anipaleja/nginx-defender:latest
imagePullPolicy: Always
securityContext:
privileged: true
capabilities:
add:
- NET_ADMIN
- NET_RAW
- SYS_ADMIN
ports:
- containerPort: 8080
name: web
- containerPort: 9090
name: metrics
volumeMounts:
- name: config
mountPath: /app/config.yaml
subPath: config.yaml
- name: log-files
mountPath: /var/log/nginx
readOnly: true
- name: proc
mountPath: /host/proc
readOnly: true
- name: sys
mountPath: /host/sys
readOnly: true
volumes:
- name: config
configMap:
name: nginx-defender-config
- name: log-files
hostPath:
path: /var/log/nginx
- name: proc
hostPath:
path: /proc
- name: sys
hostPath:
path: /sys
tolerations:
- operator: Exists
effect: NoSchedule
apiVersion: v1
kind: Service
metadata:
name: nginx-defender-service
namespace: nginx-defender
spec:
selector:
app: nginx-defender
ports:
- name: web
port: 8080
targetPort: 8080
- name: metrics
port: 9090
targetPort: 9090
type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-defender-ingress
namespace: nginx-defender
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: nginx-defender.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-defender-service
port:
number: 8080
Complete Configuration Reference
# nginx-defender configuration file
# Version: 2.0
# Firewall configuration
firewall:
# Backend type: iptables, nftables, pf, mock
backend: "iptables"
# Whitelisted IP addresses and ranges
whitelist:
- "127.0.0.1"
- "::1"
- "192.168.0.0/16"
- "10.0.0.0/8"
- "172.16.0.0/12"
# Default action for threats
default_action: "BLOCK"
# Rule cleanup interval
cleanup_interval: "5m"
# Maximum number of rules
max_rules: 10000
# Threat detection configuration
detection:
enabled: true
# Machine learning model file (optional)
ml_model: "/app/models/threat_detection.model"
# Threat patterns file
patterns_file: "pkg/patterns/common.yaml"
# Rate limiting configuration
rate_limiting:
enabled: true
window: "1m"
max_requests: 100
algorithm: "sliding_window"
# Geographic blocking
geo_blocking:
enabled: true
blocked_countries: ["CN", "RU", "KP"]
allowed_countries: []
geoip_database: "/app/data/GeoLite2-City.mmdb"
# Honeypot integration
honeypot:
enabled: false
boost_factor: 2.0
services: ["ssh", "http", "ftp"]
# Log monitoring configuration
logs:
level: "info"
format: "json"
output: "stdout"
# Log sources to monitor
sources:
- path: "/var/log/nginx/access.log"
format: "combined"
follow: true
- path: "/var/log/nginx/error.log"
format: "error"
follow: true
- path: "/var/log/apache2/access.log"
format: "combined"
follow: false
# Web server configuration
server:
host: "0.0.0.0"
port: 8080
# TLS configuration
tls:
enabled: false
cert_file: "/app/certs/server.crt"
key_file: "/app/certs/server.key"
# CORS configuration
cors:
enabled: true
allowed_origins: ["*"]
allowed_methods: ["GET", "POST", "PUT", "DELETE"]
# Metrics and monitoring
metrics:
enabled: true
# Prometheus configuration
prometheus:
enabled: true
host: "0.0.0.0"
port: 9090
path: "/metrics"
# Performance monitoring
performance:
enabled: true
interval: "30s"
# Notification configuration
notifications:
enabled: true
# Notification channels
channels:
console:
enabled: true
level: "info"
email:
enabled: false
smtp_host: "smtp.example.com"
smtp_port: 587
username: "[email protected]"
password: "password"
to: ["[email protected]"]
subject_prefix: "[nginx-defender]"
telegram:
enabled: false
bot_token: "BOT_TOKEN"
chat_id: "CHAT_ID"
slack:
enabled: false
webhook_url: "WEBHOOK_URL"
channel: "#security"
username: "nginx-defender"
webhook:
enabled: false
url: "https://api.example.com/webhooks/security"
headers:
Authorization: "Bearer TOKEN"
Content-Type: "application/json"
# Clustering configuration (Enterprise)
clustering:
enabled: false
node_id: "node-1"
discovery:
method: "static"
endpoints: ["node-2:8081", "node-3:8081"]
# Distributed rule synchronization
sync:
enabled: true
interval: "10s"
Variable | Description | Default |
---|---|---|
NGINX_DEFENDER_CONFIG |
Configuration file path | config.yaml |
NGINX_DEFENDER_LOG_LEVEL |
Log level (debug, info, warn, error) | info |
NGINX_DEFENDER_DRY_RUN |
Enable dry-run mode | false |
NGINX_DEFENDER_BACKEND |
Firewall backend override | - |
NGINX_DEFENDER_WEB_PORT |
Web server port | 8080 |
NGINX_DEFENDER_METRICS_PORT |
Metrics server port | 9090 |
Complete API Documentation
GET /api/v1/status
Response:
{
"status": "healthy",
"version": "2.0.0",
"uptime": "2h30m15s",
"build_info": {
"version": "v2.0.0",
"commit": "abc123",
"build_time": "2025-07-27T10:00:00Z"
}
}
GET /api/v1/rules
POST /api/v1/rules
DELETE /api/v1/rules/{id}
Create Rule:
{
"ip": "192.168.1.100",
"action": "BLOCK",
"duration": "1h",
"reason": "SQL injection attempt",
"metadata": {
"threat_level": "high",
"pattern": "sql_injection"
}
}
GET /api/v1/stats
GET /api/v1/metrics
Statistics Response:
{
"firewall": {
"backend": "iptables",
"total_rules": 1234,
"active_rules": 987,
"expired_rules": 247
},
"detection": {
"total_requests": 50000,
"blocked_requests": 1234,
"threat_detections": {
"sql_injection": 456,
"xss": 234,
"brute_force": 344
}
},
"performance": {
"avg_response_time": "0.5ms",
"memory_usage": "89MB",
"cpu_usage": "3.2%"
}
}
GET /api/v1/config
PUT /api/v1/config
POST /api/v1/config/reload
GET /api/v1/logs?limit=100&level=error
GET /api/v1/threats?since=1h&severity=high
Real-time Event Streaming
const ws = new WebSocket('ws://localhost:8080/ws');
// Threat detection event
{
"type": "threat_detected",
"timestamp": "2025-07-27T12:00:00Z",
"data": {
"ip": "192.168.1.100",
"threat_type": "sql_injection",
"severity": 9,
"blocked": true
}
}
// Rule creation event
{
"type": "rule_created",
"timestamp": "2025-07-27T12:00:01Z",
"data": {
"rule_id": "rule-123",
"ip": "192.168.1.100",
"action": "BLOCK",
"expires_at": "2025-07-27T13:00:01Z"
}
}
// System metrics update
{
"type": "metrics_update",
"timestamp": "2025-07-27T12:00:00Z",
"data": {
"cpu_usage": 3.2,
"memory_usage": 89,
"active_connections": 1234
}
}
Performance Test Results
- Hardware: Intel Xeon E5-2686v4 (4 cores), 16GB RAM
- OS: Ubuntu 22.04 LTS
- Network: 10Gbps Ethernet
- Backend: nftables
Test Scenario | Throughput | Latency (p95) | CPU Usage | Memory Usage |
---|---|---|---|---|
Log Processing | 150k entries/sec | 2ms | 15% | 120MB |
Pattern Matching | 100k requests/sec | 1ms | 25% | 95MB |
Rule Application | 75k rules/sec | 0.5ms | 10% | 85MB |
API Requests | 50k requests/sec | 5ms | 20% | 100MB |
WebSocket Events | 25k events/sec | 3ms | 12% | 90MB |
Log Processing Rate vs CPU Cores:
1 core: 50k entries/sec
2 cores: 95k entries/sec
4 cores: 150k entries/sec
8 cores: 180k entries/sec (diminishing returns)
Memory Usage Analysis
Component | Base Memory | Per Rule | Per Connection |
---|---|---|---|
Core Engine | 45MB | 64 bytes | 2KB |
Pattern Matcher | 25MB | - | 512 bytes |
Web Server | 15MB | - | 4KB |
Metrics Collector | 10MB | 128 bytes | 1KB |
Total Baseline | 95MB | 192 bytes | 7.5KB |
- 10k rules: ~97MB
- 100k rules: ~114MB
- 1M rules: ~287MB
- 10k concurrent connections: ~170MB
Security Design Principles
- Input Validation: All user inputs are validated and sanitized
- Principle of Least Privilege: Minimal required permissions
- Fail-Safe Defaults: Secure defaults for all configurations
- Complete Mediation: All access attempts are checked
- Open Design: Security through design, not obscurity
graph LR
A[External Attacker] -->|HTTP Requests| B[nginx-defender]
C[Malicious Insider] -->|Config Changes| B
D[Compromised System] -->|Local Access| B
B -->|Validated Traffic| E[Protected Services]
B -->|Blocked Traffic| F[Firewall Drop]
B -->|Alerts| G[Security Team]
Layer | Control | Implementation |
---|---|---|
Network | Traffic filtering | iptables/nftables rules |
Application | Input validation | Regex patterns, ML detection |
Authentication | API access control | JWT tokens, API keys |
Authorization | Role-based access | RBAC system |
Audit | Activity logging | Structured logging, SIEM integration |
Encryption | Data protection | TLS 1.3, AES-256 |
Security Practices
- Static Analysis: Weekly automated scans
- Dynamic Testing: Monthly penetration testing
- Dependency Scanning: Daily vulnerability checks
- Configuration Audits: Quarterly security reviews
- Detection: Automated threat detection and alerting
- Analysis: Log correlation and forensic analysis
- Containment: Automatic rule application and isolation
- Recovery: System restoration and patch application
- Lessons Learned: Post-incident review and improvements
- OWASP Top 10: Protection against common web vulnerabilities
- NIST Cybersecurity Framework: Risk management alignment
- ISO 27001: Information security management
- PCI DSS: Payment card industry compliance (where applicable)
Development Setup
# Install Go 1.21+
curl -L https://go.dev/dl/go1.21.linux-amd64.tar.gz | sudo tar -C /usr/local -xz
export PATH=$PATH:/usr/local/go/bin
# Install development tools
go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest
go install github.com/securecodewarrior/sast-scan@latest
# Clone repository
git clone https://github.com/anipaleja/nginx-defender.git
cd nginx-defender
# Install dependencies
go mod download
# Run tests
go test -v ./...
# Build binary
go build -o nginx-defender ./cmd/nginx-defender
# Build for multiple platforms
make build-all
# Run with hot reload
air -c .air.toml
# Run tests with coverage
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out
# Lint code
golangci-lint run
# Security scan
gosec ./...
Contribution Process
- Go Style: Follow effective Go guidelines
- Documentation: Comprehensive godoc comments
- Testing: Minimum 80% test coverage
- Performance: Benchmark critical paths
- Security: Static analysis and vulnerability scanning
- Fork the repository
- Create feature branch (
git checkout -b feature/amazing-feature
) - Commit changes (
git commit -m 'Add amazing feature'
) - Push to branch (
git push origin feature/amazing-feature
) - Open Pull Request with detailed description
## Bug Report Template
**Environment:**
- OS: [e.g., Ubuntu 22.04]
- Version: [e.g., v2.0.0]
- Backend: [e.g., iptables]
**Steps to Reproduce:**
1. Configuration used
2. Command executed
3. Expected vs actual behavior
**Logs:**
[Paste relevant logs here]
**Additional Context:**
[Any other relevant information]
This project is licensed under the MIT License - see the LICENSE file for details.
Open Source Dependencies
Dependency | License | Usage |
---|---|---|
gorilla/mux | BSD-3-Clause | HTTP routing |
gorilla/websocket | BSD-2-Clause | WebSocket support |
sirupsen/logrus | MIT | Structured logging |
prometheus/client_golang | Apache-2.0 | Metrics collection |
oschwald/geoip2-golang | ISC | Geographic lookup |
go-telegram-bot-api/telegram-bot-api | MIT | Telegram integration |
slack-go/slack | BSD-2-Clause | Slack integration |
nginx-defender - Protecting the modern web, one request at a time.