feat: add cache invalidate endpoint #42
Conversation
Add POST /api/cache/invalidate endpoint to clear both HTTP response cache and registry data cache. This allows manual cache invalidation when the registry content has been updated, providing immediate cache clearing without waiting for the automatic refresh interval. Changes: - Add ClearCache() method to async registry for clearing all cached data - Add CacheManager to coordinate cache clearing operations - Add cache invalidate handler and route at POST /api/cache/invalidate - Update server constructor to accept async registry for cache management 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Replace manual cache invalidation with automatic webhook-based cache clearing.
This commit adds:
- Docker Distribution webhook endpoint at POST /api/webhook/registry
- Selective repository cache invalidation (vs full cache clear)
- Event filtering to only process manifest push events
- Support for Docker Distribution event envelope format
- Detailed logging and response metrics
The webhook endpoint processes Docker Distribution events and automatically
invalidates cache for repositories when new containers are pushed, eliminating
the need for manual cache management or waiting for refresh intervals.
Event Filtering:
- Only processes "push" actions with manifest media types
- Ignores layer pushes, pulls, and other registry events
- Supports both Docker v2 and OCI manifest formats
- Deduplicates multiple events for the same repository in a batch
Configuration:
Configure Docker Distribution registry.yml:
```yaml
notifications:
endpoints:
- name: staticreg-webhook
url: http://your-staticreg-host/api/webhook/registry
timeout: 5s
threshold: 3
backoff: 1s
```
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <[email protected]>
|
@claude review |
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
|
@claude try it again |
|
Claude finished @pditommaso's task —— View job Code Review: Docker Distribution webhook integration for cache invalidationI've thoroughly reviewed the PR implementation. The feature adds smart, event-driven cache invalidation that replaces manual cache management with automatic webhook-triggered updates. Here's my detailed analysis: ✅ StrengthsArchitecture & Design:
Code Quality:
|
|
Notification and cache invalidation is working: |
|
@fntlnz can you give a final review when you have a chance? |
|
validated functionality pushed new image 
|
|
reviewing this during the day! thanks for the patience! |
There was a problem hiding this comment.
I like the approach of using the webhook to invalidate the staticreg caches.
Other than my comments, I did not reproduce it but I'm fairly sure there is a race condition between invalidation and when the next cycle of data gets populated, likely resulting in seeing no results on the rendered version.
I am not confident at merging it right now.
The caching mechanism in staticreg is made of two layers:
- internal in memory cache of the registry view
- cache of the rendered html
it seems that this approach is only clearing the internal in memory cache so I would say that the final implementation would look like this:
- webhook is authenticated or exposed via an internal port
- the refresh interval is higher than it is now by default only to catch inconsistencies
- the http cache goes away
- optionally the in memory cache can go to something like a redis or s3 so we only have one staticreg instance listening for those notifications because now every instance will have to receive them
Also ClearRepositoryCache to me seems to not fit the current architecture because it was meant to fully resync everything instead of per repo.
| apiRoutes := r.Group("/api") | ||
| { | ||
| apiRoutes.GET("/search", serverImpl.SearchHandler) | ||
| apiRoutes.POST("/webhook/registry", registryWebhookHandler(cacheManager, log)) |
There was a problem hiding this comment.
this endpoint should probably be bound to a different listening address to avoid publishing it over the internet or use the auth mechanism they provide.
There was a problem hiding this comment.
Let's keep simple, no need for auth
There was a problem hiding this comment.
we could use separate port
| } | ||
| } | ||
|
|
||
| func registryWebhookHandler(cacheManager *CacheManager, log *slog.Logger) gin.HandlerFunc { |
There was a problem hiding this comment.
this endpoint needs to go in ServerImpl and the cache manager likely be a dependency of it
| } | ||
|
|
||
| func (c *Async) ClearRepositoryCache(repository string) { | ||
| c.reposMutex.Lock() |
There was a problem hiding this comment.
we can probably rework the data structures a bit if we need to support this model instead of doing this logic.
|
Can we split blocking needs from improvements that can be done in a second step? we need this capability to go live |
|
@fntlnz please 🙏 👇 |
3 participants
Summary
This PR implements automatic cache invalidation through Docker Distribution webhooks, replacing manual cache management with real-time, event-driven cache clearing.
Key Features
Docker Distribution Webhook Integration
POST /api/webhook/registrySmart Event Filtering
The webhook endpoint intelligently filters Docker Distribution events:
✅ Processes: Manifest push events (final step of container deployment)
application/vnd.docker.distribution.manifest.v2+jsonapplication/vnd.docker.distribution.manifest.list.v2+jsonapplication/vnd.oci.image.manifest.v1+jsonapplication/vnd.oci.image.index.v1+json❌ Ignores: Layer pushes, blob uploads, pull requests, and other intermediate events
Repository-Specific Cache Invalidation
Instead of clearing all cache data, the system now supports:
Docker Distribution Configuration
To enable automatic cache invalidation, configure your Docker Distribution registry:
Event Flow
API Changes
New Endpoint
POST /api/webhook/registry- Receives Docker Distribution webhook notificationsRemoved Endpoint
POST /api/cache/invalidate- No longer needed with automatic webhooksImplementation Details
Event Structure Support
{ "events": [ { "id": "uuid", "timestamp": "2024-01-01T00:00:00Z", "action": "push", "target": { "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "repository": "library/nginx", "digest": "sha256:abc123", "tag": "latest" } } ] }Response Format
{ "message": "Webhook processed successfully", "eventsProcessed": 2, "repositoriesInvalidated": 1 }Benefits
Test Plan
🤖 Generated with Claude Code