Online ceremony

[wzmuda]

Overview

Provide implementation of client and server for online contributions.

Review advice

I suggest to review it commit-by-commit. The first five commits just shuffle things around. They're quite trivial and have nothing to do with the online mode itself.

Unfortunately the online mode is one huge commit, because there are no logical parts to it that make sense to be committed separately. The good thing is that ~half of it are tests and comments.

Below is an short writeup describing the architecture, to make the review as easy as possible. Hit me up if something is not clear.

Architecture

gRPC API

The ceremony API is defined in online/api/ceremony.proto. It's way simpler than the previous approach. It's composed on one method only, which guarantees a session integrity. Before I proposed a few methods for various ceremony stages, which created a need of tracking if the client calls them in the right order. Now it makes no difference - one call is the entry point to the ceremony.

The magic happens in message definitions. The server streams messages to the client. First it's the queue status. Then, after reaching the beginning of the queue - the server streams initial data for the contribution. Then it waits for the client to start streaming their contribution. In the end, after receiving the contribution from the client, the server sends the verification status.

Client

online/actions/client.go is the entry point for ceremony client.

The client is defined in client/client.go. It has two methods - the constructor and the contribution method. The contribution is a message loop (defined in online/client/handlers.go) that gets messages from the stream and calls handlers dedicated for each possible message type.

After downloading the input from the server, the client must compute their contribution that will be sent back to the server. This happens in the p2.Contribute() line in handlers.go. The contribution logic is encapsulated in the online/contribution package. The client uses the Contributable interface only.

The client also uses helpers for streaming blobs to/from the server. They're defined in online/api/stream_utils/.

Server

This is the most complicated part but I hope I structured it well.

online/actions/server.go is the entry point for ceremony server. Is instantiates the server with all its dependencies and waits for ctrl+c, which is the signal for ending the ceremony and generating keys.

The server is defined in client/server . It has three methods - New, Start and Stop. Nothing interesting is happening in this part of the code, except that the constructor accepts the service parameter. This is the ceremony service.

The ceremony service

The high level ceremony service logic is defined in the only method - Contribute in online/server/ceremony_service/ceremony_service.go. This method is called each time a client connects. This handler operates on a global ceremony state.

The flow is simple:

• add the new client(== contributor) to the ceremony. This possibly happens multiple times. The contributors are queued in the ceremony internals.
• block until the contributor that called this handlers gets to the beginning of the queue.
• send them the last known state of the ceremony (a.k.a last contribution).
• receive the incoming contribution.
• verify it and remove this contributor from the queue, advancing all the waiting guys by one place in the queue. Somebody else just reached the beginning, so their handler unblocked, while the rest of the queue remains blocked.

The server also uses helpers for streaming blobs to/from the client. They're defined in online/api/stream_utils/. They're the same helpers client uses.

Managing the state of the ceremony is done by the coordinator.

Coordinator

Coordinator's responsibility is to accept contributors, hand them the input, accept the output and validate it. It doesn't care about the connection details (this is the ceremony service's job). It's defined in online/server/coordnator/coordinator.go. The interface is simple: add contribution, send it to client (a.k.a. write) and receive the incoming one from the client (a.k.a. read).

It has very little logic - all it does is to ask a dedicated service to handle contributors and ask another dedicated service to handle contributions. These responsibilities are handed respectively to contributors manager and to the contribution package.

Contributors manager

This is a glorified FIFO. It is defined in online/server/contributors_manager/contributors_manager.go. Actually the FIFO lives there too. The manager just adds a new contributor to the queue and removes the current one.

The one extra responsibility is notification. Each time a contributor is added, it gets some slot in the queue (0, 1, 2 etc). It's always contributor 0 who is removed. On removal, every contributor advances by one, so contributor 1 becomes 0, contributors 2 becomes 1 etc. This is when the manager notifies every contributor in the queue about their new position. This is done via a channel and propagated to the caller via a function that AddContributor returns. This is the function that blocks inside the ceremony service. It's unique for every contributor because the Contribute handler of the ceremony is instantiated once per an actual client connection. It's like a separate thread in the server.

Contribution

This is where the logic regarding the contribution itself lives. It's a helper package defined in online/contribution/contribution.go. It's a collection of interfaces for different occasions. One interface is meant for the coordinator, another one is meant for the contributor. They limit the functionality, so that the contributor cannot do anything more than it should.

The coordinator gets the contribution via the Contribution interface. This is the object that grows with the incoming contributions. It is initialized via the New constructor.

Coordinator pushes new contributions to it using it's AddContribution method. This method accepts also a contribution object, but limited to it's Verifiable interface - it's the incoming contribution from a client, so all we want to do with it is to verify.

In the end of the ceremony, when the server is done (i.e. we pressed ctrl+c when called from the command line), this main contribution object can be asked to generate keys, which is the final product of the ceremony

Config

This is the representation of the JSON config file the server needs to run.

Testing

This is how the usage looks like for the ceremony for the Whir circuit. The three consoles on the top are clients, the one at the bottom is the server. The interesting thing is how the queue of contributors is maintained by the server. Notice that the client input is just server's host and port. The rest of the ceremony is handled automatically.

[wzmuda]

I see this log exactly once, for the first contributor in the ceremony. The subsequent ones don't print this. Leaving this comment so I remember to investigate.

[piohei]

LGTM!