Proposal: make required fields with known atoms concrete on on export

[rogpeppe]

As I've been thinking a bunch about discriminator fields, I've come to a realisation about something that I've been finding quite awkward recently.

Consider an API containing a schema that can be a range of possible types, determined with a type field, an extremely common use case:

#Msg: #MsgA | #MsgB

#MsgA: {
	type!: "a"
	foo?: bool
}

#MsgB: {
	type!: "b"
	foo!: string
	bar?: int
}

This is good to use for validating existing concrete data: in that case we need the type field to exist, and the type!: value fields are important in that use case.

However, the story is different when exporting data. If someone wants to write some CUE that creates some JSON containing one of the above messages, they have to do something like:

#MsgA & {
	type: _
	foo: true
}

i.e. they need to know about the discriminator and explicitly instantiate it, because otherwise there will be an error

This is noisy and error-prone: something that the user has to do and know about but isn't adding any value.

Existing CUE users often use a regular field for discriminator fields because of this issue. But this means that the schema isn't "pure" and can't be used for validating messages received.

An idiom I've toyed with is defining an extra top level template for each such generic message type:

Msg: {
	type: _
}

so to export the type, instead of explicitly mentioning the type field, someone would do:

Msg & #MsgA & {
	foo: true
}

but that's still awkward, and would end up polluting the top level namespace of every package.

Proposal

I propose that by default, CUE in export mode will make required fields
that are concrete "atoms" behave as if they're concrete. An "atom" is defined
as any CUE value other than a struct or a list.

So this test (or something like it) would pass:

exec cue export foo.cue
cmp stdout golden.json

-- golden.json --
msgs: [{
	type: "a"
}, {
	type: "b"
	foo:  "something"
}]
-- foo.cue --
msgs: [
	#MsgA,
	#MsgB & {foo: "something"},
]

#Msg: #MsgA | #MsgB

#MsgA: {
	type!: "a"
	foo?: bool
}

#MsgB: {
	type!: "b"
	foo!: string
	bar?: int
}

In general, any reference to a required known-value field would succeed
and produce the value of the field.

A new cue vet mode would be needed to explicitly check that a given CUE value is genuinely concrete (the mode that cue vet -c works in as is). This is because it's still necessary to be able to check that some concrete data (for example a JSON or YAML file) does actually conform to the schema.

Discussion

Although this might seem strange at first, I think this makes sense.
CUE has two main use cases:

• to check that some data is valid with respect to a schema (validation)
• to generate some data (templating)

Of course, CUE being what it is, both of those use cases are somewhat overlapping, but things are much clearer from a user point-of-view, I think. That is, as a user, if I'm validating some data that I'm going to send somewhere else on the network, I absolutely want to check that all those required fields really exist and are present in the data.

However, if I'm in the business of generating data with CUE templating, I can't see why I would care at all whether those fields have been explicitly specified as regular fields or not. In fact, in general we don't want the configuration to mention their value explicitly in multiple places because that hinders maintainability. So we end up with code like the above Msg: {type: _} that's essentially just noise.

ISTM that by relaxing the requirement to mention these fields explicitly everywhere, we make CUE considerably more helpful; for example, it's easy to define discriminator fields and let users just unify with the CUE schema types and have the right thing "just happen".