Document/encode which version of the spec we conform to

[joshuagl]

Description of issue or feature request:

Recent specification PRs introduced changes to the detailed client workflow (i.e. theupdateframework/specification#116 and theupdateframework/specification#117 merged recently, while theupdateframework/specification#112 and theupdateframework/specification#118 may land any time).

Thinking about ensuring these are implemented I realised that I can not tell whether all details of the client workflow (and wider specification) prior to those changes are (supposed to be) implemented in the reference implementation.

We should ensure the reference implementation and POUF 1 fully implement a specification version and then capture which version of the specification we believe the reference implementation conforms to.

Thanks to our versioning process improvements, where each spec PR merge must have an associated version increment and release, it's easier to track what changes occur between specification releases. We can then leverage this to ensure that we are compliant with the specification's detailed client workflow and track changes that may need to be implemented in future.

We could even have a GitHub action generating a badge (or similar) to indicate whether our latest conformed version matches the latest specification release.

Current behavior:

Difficult to tell whether we believe we adhere to the specification and which version we conform to.

Expected behavior:

Easy to tell which exact version of the specification we believe is implemented by the reference implementation.

[mnm678]

At a minimum, it would be good to list the currently supported spec version in the README. Something more automatic would be nice, but I'm not sure how to check against workflow descriptions in the spec. Are there any existing tools for that?

[joshuagl]

Apologies, the initial description was unclear (I've made some edits). I'm not meaning to propose automatic workflow checking here. The concern I'm trying to capture in this issue is that it's not clear which recent specification changes we need to ensure are implemented in the reference implementation. I'd it to be easier for us to determine which changes we need to ensure are implemented, whereas today I think we need to do a full audit of the client workflow and ensure it adheres to the detailed client workflow in the specification.

[lukpueh]

+1 for the feature request. IIUC this goes beyond the SPECIFICATION_VERSION field, which only hints at the metadata of the major spec version being supported, right?

https://github.com/theupdateframework/tuf/blob/baae9bd1a287dc6193824d6261940215fa710d66/tuf/__init__.py#L7-L13

@joshuagl, do you think of a similar field that also takes the client workflow into consideration?

We could even have a GitHub action generating a badge (or similar) to indicate whether our latest conformed version matches the latest specification release.

I like that idea and it sounds like a fun isolated task to automate the update of a custom badge based on merge that changes SPECIFICATION_VERSION or a similar field.

[joshuagl]

+1 for the feature request. IIUC this goes beyond the SPECIFICATION_VERSION field, which only hints at the metadata of the major spec version being supported, right?

https://github.com/theupdateframework/tuf/blob/baae9bd1a287dc6193824d6261940215fa710d66/tuf/__init__.py#L7-L13

@joshuagl, do you think of a similar field that also takes the client workflow into consideration?

I was thinking perhaps something like that. Though as the current code is only checking the major version, we could just use the existing SPECIFICATION_VERSION field and keep it up-to-date with minor and patch versions when we make changes to the client workflow? Do you think an additional field would be better?

[mnm678]

I was thinking perhaps something like that. Though as the current code is only checking the major version, we could just use the existing SPECIFICATION_VERSION field and keep it up-to-date with minor and patch versions when we make changes to the client workflow? Do you think an additional field would be better?

I'm in favor of using the existing SPECIFICATION_VERSION field. Maybe we can add this as a step to the release document?

[jku]

Closing this as the version is now part of Metadata API and https://github.com/theupdateframework/python-tuf/blob/develop/.github/workflows/specification-version.yml now exists