Skip to content

Conversation

@rcooke-warwick
Copy link
Contributor

Previously, you could define several json schemas for each test that could be used to determine whether or not a test should be ran. For example, you could do this, to make sure a test only ran for devices with HDMI:

module.exports = {
	title: 'Balena boot splash tests',
	deviceType: {
		type: 'object',
		required: ['data'],
		properties: {
			data: {
				type: 'object',
				required: ['hdmi'],
				properties: {
					hdmi: {
						type: 'boolean',
						const: true,
					},
				},
			},
		},
	},
        run: async() function { /* test code */}

This is quite unwieldly to write and is not simple to extend. This PR lets you instead add a contract to a test using our contract syntax, and then uses contrato to evaluate the tests contract and determine whether or not to do the test. This makes it easier to use, and in the future I think it will be quite useful as we extend the use of contracts (e.g os contracts etc)

You can re write the above syntax as:

module.exports = {
	title: 'Balena boot splash tests',
	contract: {
		type: 'test.test',
		requires: [
			{
				type: 'hw.device-type',
				data: {
					hdmi: false
				}
			}
        ]
	},
        run: ////

Change-type: patch
Signed-off-by: Ryan Cooke <[email protected]>
Change-type: patch
Signed-off-by: Ryan Cooke <[email protected]>
@rcooke-warwick rcooke-warwick added the versionbot/pr-draft Draft PR - Don't merge this PR automatically label Nov 2, 2021
@klutchell
Copy link
Contributor

This is awesome!

Can you squash the existing two commits and add a second commit to lint the files? Otherwise this looks great.

@rcooke-warwick
Copy link
Contributor Author

@klutchell just thought that we need to time this one well actually, as at the moment in meta-balena we have a few tests that use the existing "contracts" schema method to determine whether or not to run the tests. This PR will make it so those tests won't run, until the tests are updated

@klutchell
Copy link
Contributor

This PR will make it so those tests won't run, until the tests are updated

So this is somewhat breaking? Is it something we can save for the new helpers repo and just keep this PR open as draft for now with a label that it needs to move to helpers?

@vipulgupta2048
Copy link
Member

This change is a bit different from the change that the helpers package will be making. Do you mean to make both changes to the test at the same time inorder to reduce the breaking changes?
If yes, then I agree that's exactly what we should be doing.

@klutchell
Copy link
Contributor

Do you mean to make both changes to the test at the same time in order to reduce the breaking changes?

@vipulgupta2048 Actually I was thinking this change would get merged to the helpers instead of leviathan, but that may not be possible?

If this change has to live in leviathan, then yes we need to lump all the breaking changes together.

@resin-jenkins
Copy link
Contributor

Can one of the admins verify this patch?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

versionbot/pr-draft Draft PR - Don't merge this PR automatically

Projects

None yet

Development

Successfully merging this pull request may close these issues.

use contrato instead of a json schema to validate whether a test should run

4 participants