Skip to content

mozilla/crlite

Repository files navigation

Build Status Docker Version

CRLite pushes the full set of WebPKI certificate revocations to Firefox clients for private querying. It replaces OCSP for most browser TLS connections and speeds up connection time without compromising on security. The system was originally proposed at IEEE S&P 2017. Significant improvements to the underlying datastructure were presented at RWC 2022, RWC 2025, and IEEE S&P 2025. The implementation here uses the Clubcard data structure described in these later works.

For more details about CRLite, Mozilla Security Engineering has a blog post series, and this repository has a FAQ.

This repository contains all of the tools needed to produce Clubcards encoding the WebPKI revocation set. It also includes an end-user tool for querying Firefox's CRLite data: rust-query-crlite.

General Structure

CRLite is designed to run in Kubernetes, with the following services:

  1. containers/crlite-fetch, a constantly-running task that fetches certificate metadata from Certificate Transparency logs and stores that metadata in a Redis cache.
  2. containers/crlite-generate, a cron job that moves certificate metadata from the Redis cache to persistent storage, fetches CRLs, creates Clubcards, and uploads artifacts to Google Cloud Storage.
  3. containers/crlite-publish, a cron job that publishes newly generated Clubcards through Firefox Remote Settings.
  4. containers/crlite-signoff, a cron job that performs some basic consistency checks and signs off on the records produced by crlite-publish.

There are scripts in containers/ to build Docker images both using Docker. There are also builds at Docker Hub in the mozilla/crlite project.

Storage

Storage consists of these parts:

  1. Redis for initial ingestion of the certificate metadata (serial numbers, expirations, issuers) used in filter generation.
  2. A local disk for persistent storage of certificate metadata and CRLs.
  3. Google Cloud Storage for storage of the artifacts when a job is completed.

Information Flow

This tooling monitors Certificate Transparency logs and, upon secheduled execution, crlite-generate produces a new filter and uploads it to Cloud Storage.

Information flow

Clubcards are built using the rust-create-cascade tool and then read in Firefox by the mozilla/clubcard-crlite package.

Local Installation

It's possible to run the tools locally. First, install the tools and their dependencies

go install -u github.com/mozilla/crlite/go/cmd/ct-fetch
go install -u github.com/mozilla/crlite/go/cmd/aggregate-crls
go install -u github.com/mozilla/crlite/go/cmd/aggregate-known

Configuration

You can configure via environment variables, or via a config file. To use a configuration file ~/.ct-fetch.ini (or any file selected on the CLI using -config), construct it as so:

certPath=/tmp/certdb/
remoteSettingsURL=""
ctLogMetadata=[...]
runForever = true

Set the ctLogMetadata variable equal to the contents of the ct-logs collection in Firefox Remote Settings (without newline characters). For testing you can trim this to a single CT log. Other configuration options are described in go/config/config.go.

General Operation

containers/build-local.sh produces the Docker containers locally.

test-via-docker.sh executes a complete "run", syncing with CT and producing a filter. It's configured using a series of environment variables.

Starting the Redis cache

Redis can be provided in a variety of ways, easiest is probably the Redis docker distribution. For whatever reason, I have the best luck remapping ports to make it run on 6379:

docker run -p 6379:7000 redis:4 --port 7000

Tools

ct-fetch Downloads all CT entry issuer-serial pairs, and associated metadata, to the Redis cache.

aggregate-crls Obtains all CRLs disclosed to CCADB, verifies them, and lists their contents into *issuer SKI base64*.revoked files.

aggregate-known Moves certificate metadata from the Redis cache to persistent storage. Lists all known unexpired certificates into *issuer SKI base64*.known files.

Credits