Skip to content

Add "test against gold" feature #103

@ChandlerSwift

Description

@ChandlerSwift

We have a UI mocked out for this (test_against_gold.html), but it was removed from the menu in 5881885 since it wasn't yet functional.

Open questions (from Slack):
@pahp:

  • In your mind, would this be simply for running gold to see what it does, or run it like an attack, where we print the output of both gold and the team's executable, and then compare them? I think the first might be slightly easier, but they should be pretty similar, since we've already got all the logic worked out anyway.
  • Since users probably want this for fairly quick feedback, do you think it makes sense to prioritize these runs above the "normal" runs (that is, the normal "attack vs. team code" runs). Does that make sense, or can you think of any problems that might cause?
  • I figure I'd rate limit these, probably 6-ish per minute, similarly to attacks. I can't think of any issues we'd have without rate limiting other than potentially denial of service, but it seems the safer bet. And 6 per minute is fairly generous, I think.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions