Here's a conceptual model of how Kubernetes, Consul, and Traefik work together in our architecture, with Traefik deployed outside Kubernetes and Consul inside Kubernetes:
┌───────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Kubernetes Cluster │
│ │
│ ┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ │ │ │ │ │ │
│ │ Microservice A │ │ Microservice B │ │ Microservice C │ │
│ │ │ │ │ │ │ │
│ └─────────┬───────────┘ └─────────┬───────────┘ └─────────┬───────────┘ │
│ │ │ │ │
│ │ │ │ │
│ │ │ │ │
│ ┌─────────▼─────────────────────────▼─────────────────────────▼───────────┐ │
│ │ │ │
│ │ Service Mesh (Consul) │ │
│ │ │ │
│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────┐ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ Consul Server │ │ Consul Client │ │ Service Discovery DB │ │ │
│ │ │ Cluster │ │ Agents │ │ │ │ │
│ │ │ │ │ (Sidecars) │ │ │ │ │
│ │ └─────────────────┘ └─────────────────┘ └─────────────────────────┘ │ │
│ │ │ │
│ └─────────────────────────────────────┬─────────────────────────────────┬─┘ │
│ │ │ │
│ │ │ │
│ │ │ │
│ ┌─────────────────────────────────┐ │ ┌───────────────────────────┐ │ │
│ │ │ │ │ │ │ │
│ │ Kubernetes Services (NodePort)◄───┘ │ External Service │ │ │
│ │ │ │ Registry (Consul) │ │ │
│ │ │ │ │ │ │
│ └─────────────────────────────────┘ └───────────────────────────┘ │ │
│ │ │
└─────────────────┬──────────────────────────────────────────────────────┬─┘ │
NodePort │ │ │
Access │ │ │
▼ │ │
┌─────────────────────────────────────┐ ┌─────────────▼─────────────────────────┐
│ │ │ │
│ Traefik API Gateway │ │ External Services │
│ (Outside Kubernetes) │ │ (VMs, Other Clusters) │
│ ┌─────────────┐ ┌───────────────┐ │ │ │
│ │ │ │ │ │ │ ┌─────────────┐ ┌─────────────┐ │
│ │ Rate │ │ Auth │ │ │ │ │ │ │ │
│ │ Limiting │ │ Middleware │ │ │ │ Legacy App │ │ Database │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ └─────────────┘ └───────────────┘ │ │ └─────────────┘ └─────────────┘ │
│ │ │ │
└───────────────┬─────────────────────┘ └───────────────────────────────────────┘
│
│ External User Traffic
▼
External Clients/Users
- Container Orchestration: Manages deployment, scaling, and lifecycle of containerized applications
- Workload Management: Schedules pods across nodes and handles self-healing
- Basic Networking: Provides pod-to-pod communication and service discovery
- Service Mesh: Manages service-to-service communication within the cluster with transparent proxies
- Service Discovery: Registers and discovers services across environments with bidirectional K8s sync
- Health Checking: Monitors service health and removes unhealthy instances automatically
- Configuration Management: Stores and distributes configuration via Consul Key-Value store
- Security: Provides service-to-service authentication and encryption (mTLS) with automated certificate management
- Access Control: Manages service access with intentions and ACL token management
- Observability: Collects and exposes metrics for monitoring service interactions
- API Gateway: Manages external access to services, running as a standalone service on a host machine
- Traffic Control: Handles routing, load balancing, and rate limiting
- Security: Provides authentication and authorization through middleware
- Monitoring: Collects metrics and logs for API traffic
- Middleware Ecosystem: Extends functionality through middleware components
- Consul Connect Injector: Automatically injects Consul sidecars into Kubernetes pods
- Catalog Sync: Bidirectional synchronization between Kubernetes services and Consul catalog
- Consul UI: Provides visibility into services across environments with authentication
- Transparent Proxy: Intercepts all traffic automatically without application changes
- ACL System: Manages access tokens automatically with Kubernetes integration
- Custom Resource Definitions (CRDs): Allows managing Consul configuration using kubectl
- NodePort Services: Kubernetes services are exposed via NodePort for Traefik to access
- External Configuration: Traefik is configured manually to route to Kubernetes services
- Independent Deployment: Traefik operates independently from the Kubernetes lifecycle
- Service Discovery Integration: Traefik can use Consul as a service discovery backend
- Dynamic Routing: Traefik can route traffic based on services registered in Consul
- Health Checking: Traefik can use Consul health checks to determine service availability
- Catalog Provider: Traefik can watch Consul catalog for service changes
- Clear Security Boundary: External Traefik creates a clear security boundary for incoming traffic
- Resource Isolation: Traefik resource usage doesn't impact Kubernetes cluster resources
- Independent Scaling: Gateway and cluster can scale independently of each other
- Enhanced Availability: Kubernetes issues don't affect the API Gateway operation
- Unified Service Management: Manage services across multiple environments (K8s, VMs, etc.)
- Advanced Traffic Management: Fine-grained control over service-to-service communication
- Multi-Environment Support: Seamless integration between containerized and non-containerized workloads
- Zero-Trust Security: Enforce authentication and authorization at both service mesh and API gateway levels
For implementing this architecture:
- External Traefik Setup: See traefik-external-setup.md for deploying Traefik outside Kubernetes
- Consul in Kubernetes: Follow the implementation guide for deploying Consul inside Kubernetes
- Service Integration: Refer to service-specific documentation for connecting services to this architecture
- HashiCorp Documentation: See official Consul Kubernetes documentation for detailed reference
This architecture provides a robust foundation for modern microservices deployments while keeping a clear separation between external traffic management (Traefik) and internal service communication (Consul within Kubernetes).