Skip to content

Clarify timestamp.json METAFILES format #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 6, 2020
Merged

Clarify timestamp.json METAFILES format #90

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
cc00f7f
LENGTH isn't optional in timestamp.json for snapshot.json
joshuagl Feb 5, 2020
0c73d44
Clarify timestamp.json METAFILES format
joshuagl Feb 5, 2020
3ca7683
Revert "97312dd" and "d97da38"
lukpueh Feb 11, 2020
e53f11c
Clarify snapshot download limit on client
lukpueh Feb 11, 2020
a5848e7
Clarify snapshot and timestamp METAFILES
lukpueh Feb 11, 2020
e360733
Fix typo in "4.6. File formats: timestamp.json"
lukpueh Mar 3, 2020
4f6ce2e
Ensure steps in the detailed workflow are consistent in use of variables
joshuagl Mar 3, 2020
d8dcf66
Bump patch version to 1.0.1
joshuagl Mar 3, 2020
081e56a
Clarify snapshot+targets metadata hash check
lukpueh Mar 5, 2020
53f6ea3
Update Last modified: **6 March 2020**
lukpueh Mar 6, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 40 additions & 33 deletions tuf-spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# <p align="center">The Update Framework Specification

Last modified: **5 March 2020**
Last modified: **6 March 2020**

Version: **1.0.0**
Version: **1.0.1**

We strive to make the specification easy to implement, so if you come across
any inconsistencies or experience any difficulty, do let us know by sending an
Expand Down Expand Up @@ -709,10 +709,9 @@ repo](https://github.com/theupdateframework/specification/issues).

* **4.4. File formats: snapshot.json**

The snapshot.json file is signed by the snapshot role. It lists the version
numbers of only the top-level targets and all delegated targets role metadata.
The metadata length and hashes are OPTIONAL for the top-level targets and
all delegated targets roles.
The snapshot.json file is signed by the snapshot role. It MUST list the
version numbers of the top-level targets metadata and all delegated targets
metadata. It MAY also list their lengths and file hashes.

The "signed" portion of snapshot.json is as follows:

Expand All @@ -732,18 +731,23 @@ repo](https://github.com/theupdateframework/specification/issues).
, ...
}

METAPATH is the metadata file's path on the repository relative to the
metadata base URL.
METAPATH is the file path of the metadata on the repository relative to the
metadata base URL. For snapshot.json, these are top-level targets metadata
and delegated targets metadata.

VERSION is listed for the top-level targets and all delegated targets roles
available on the repository.
VERSION is the integer version number as shown in the metadata file at
METAPATH.

LENGTH is the integer length in bytes of the metadata file. It is
OPTIONAL for all roles.
LENGTH is the integer length in bytes of the metadata file at METAPATH. It
is OPTIONAL and can be omitted to reduce the snapshot metadata file size. In
that case the client MUST use a custom download limit for the listed
metadata.

HASHES is the dictionary that specifies one or more hashes, including
the cryptographic hash function. For example: { "sha256": HASH, ... }. It is
OPTIONAL for all roles.
HASHES is a dictionary that specifies one or more hashes of the metadata
file at METAPATH, including their cryptographic hash function. For example:
{ "sha256": HASH, ... }. HASHES is OPTIONAL and can be omitted to reduce
the snapshot metadata file size. In that case the repository MUST guarantee
that VERSION alone unambiguously identifies the metadata at METAPATH.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to note when length and hashes should be included (ie backwards compatibility for both, using length as an exact download size)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backwards compatibility is one reason, yes, but I believe the more important reason is to protect against certain security attacks. Please see Section 5.6 of Mercury.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, if we are bringing back hashes and length here, we should add the corresponding checks in the download process.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to note when length and hashes should be included

@mnm678, @trishankatdatadog: Good idea. Could you push a commit with such a note?

Also, if we are bringing back hashes and length here, we should add the corresponding checks in the download process.

@trishankatdatadog: What's missing from the existing (L1136-1138) and newly proposed (L11160-1163) download checks?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lukpueh Those are good checks, but:

  • 3.1 doesn't mention that hashes and length are optional. It also doesn't mention checking the length, if any, of the snapshot metadata.
  • When we check the targets role or its delegations, do we mention checking optional hashes and length from the snapshot metadata?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing these out, @trishankatdatadog!

  • 3.1 doesn't mention that hashes and length are optional. It also doesn't mention checking the length, if any, of the snapshot metadata.

Is another length match check in 3.1. necessary? We already say in 3. that the client should download snapshot.json up to the length specified in timestamp.json (if specified).

Regarding optionality of hashes, I agree we could add a "(if any)" note in 3.1, as we do for targets in 4.1.

  • When we check the targets role or its delegations, do we mention checking optional hashes and length from the snapshot metadata?

For top-level targets we do say in 4.1. "The hashes (if any), and version number of the new targets metadata file MUST match the trusted snapshot metadata."
I think the "(if any)" is slightly misplaced, because the hashes it refers are computed by the client. It would be more accurate to say: "The hashes and version number of the new targets metadata file MUST match the hashes (if any) and version number listed in the trusted snapshot". Let me push a commit that clarifies this.

Regarding length, similar to timestamp-snapshot in 3., we say in 4.1. that the client should download targets.json up to the length specified in snapshot.json (if specified). Do you think we need another length match check in 4.1.?

Regarding delegated targets metadata, we currently don't provide details about verification. #86 adds those details and adopts the "(if any)" wording used in 4.1. If we accept my suggestion for 4.1. above, we should adopt this for delegated targets metadata in #86.

Similarly, if we add a length match check for top-level targets metadata, we should adopt this for delegated in #86 too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarified optionality of hashes (checks) in 0bdb99c. Let me know if we should add another optional length check...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a second length check isn't necessary as the length is already used when downloading.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we check the targets role or its delegations, do we mention checking optional hashes and length from the snapshot metadata?
FYI, @trishankatdatadog: As promised above, I just pushed a commit to #86 that notes that delegated targets metadata hashes might be missing in snapshot metadata.


A snapshot.json example file:

Expand Down Expand Up @@ -958,8 +962,8 @@ repo](https://github.com/theupdateframework/specification/issues).

* **4.6. File formats: timestamp.json**

The timestamp file is signed by a timestamp key. It indicates the
latest versions of other files and is frequently resigned to limit the
The timestamp file is signed by a timestamp key. It indicates the latest
version of the snapshot metadata and is frequently resigned to limit the
amount of time a client can be kept unaware of interference with obtaining
updates.

Expand Down Expand Up @@ -1072,8 +1076,8 @@ repo](https://github.com/theupdateframework/specification/issues).
* **1.1**. Let N denote the version number of the trusted root metadata file.

* **1.2**. **Try downloading version N+1 of the root metadata file**, up to
some X number of bytes (because the size is unknown). The value for X is set
by the authors of the application using TUF. For example, X may be tens of
some W number of bytes (because the size is unknown). The value for W is set
by the authors of the application using TUF. For example, W may be tens of
kilobytes. The filename used to download the root metadata file is of the
fixed form VERSION_NUMBER.FILENAME.EXT (e.g., 42.root.json). If this file is
not available, or we have downloaded more than Y number of root metadata files
Expand Down Expand Up @@ -1125,9 +1129,9 @@ repo](https://github.com/theupdateframework/specification/issues).
* **1.10**. **Set whether consistent snapshots are used as per the trusted
root metadata file** (see Section 4.3).

**2**. **Download the timestamp metadata file**, up to Y number of bytes
(because the size is unknown.) The value for Y is set by the authors of the
application using TUF. For example, Y may be tens of kilobytes. The filename
**2**. **Download the timestamp metadata file**, up to X number of bytes
(because the size is unknown). The value for X is set by the authors of the
application using TUF. For example, X may be tens of kilobytes. The filename
used to download the timestamp metadata file is of the fixed form FILENAME.EXT
(e.g., timestamp.json). The client MUST write the file to non-volatile storage
as FILENAME.EXT.
Expand All @@ -1149,8 +1153,10 @@ as FILENAME.EXT.
file. If the new timestamp metadata file has expired, discard it, abort the
update cycle, and report the potential freeze attack.

**3**. **Download snapshot metadata file**, up to the number of bytes specified
in the timestamp metadata file. If consistent snapshots are not used (see
**3**. **Download snapshot metadata file**, up to either the number of bytes
specified in the timestamp metadata file, or some Y number of bytes. The value
for Y is set by the authors of the application using TUF. For example, Y may be
tens of kilobytes. If consistent snapshots are not used (see
Section 7), then the filename used to download the snapshot metadata file is of
the fixed form FILENAME.EXT (e.g., snapshot.json). Otherwise, the filename is
of the form VERSION_NUMBER.FILENAME.EXT (e.g., 42.snapshot.json), where
Expand All @@ -1159,10 +1165,10 @@ the timestamp metadata file. In either case, the client MUST write the file to
non-volatile storage as FILENAME.EXT.

* **3.1**. **Check against timestamp metadata.** The hashes and version
number of the new snapshot metadata file MUST match the hashes and version
number listed in timestamp metadata. If hashes and version do not match,
discard the new snapshot metadata, abort the update cycle, and report the
failure.
number of the new snapshot metadata file MUST match the hashes (if any) and
version number listed in the trusted timestamp metadata. If hashes and
version do not match, discard the new snapshot metadata, abort the update
cycle, and report the failure.

* **3.2**. **Check signatures.** The new snapshot metadata file MUST have
been signed by a threshold of keys specified in the trusted root metadata
Expand Down Expand Up @@ -1207,11 +1213,12 @@ VERSION_NUMBER is the version number of the targets metadata file listed in the
snapshot metadata file. In either case, the client MUST write the file to
non-volatile storage as FILENAME.EXT.

* **4.1**. **Check against snapshot metadata.** The hashes (if any), and
version number of the new targets metadata file MUST match the trusted
snapshot metadata. This is done, in part, to prevent a mix-and-match attack
by man-in-the-middle attackers. If the new targets metadata file does not
match, discard it, abort the update cycle, and report the failure.
* **4.1**. **Check against snapshot metadata.** The hashes and version
number of the new targets metadata file MUST match the hashes (if any) and
version number listed in the trusted snapshot metadata. This is done, in
part, to prevent a mix-and-match attack by man-in-the-middle attackers. If
the new targets metadata file does not match, discard it, abort the update
cycle, and report the failure.

* **4.2**. **Check for an arbitrary software attack.** The new targets
metadata file MUST have been signed by a threshold of keys specified in the
Expand Down