-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Description
We have a UI mocked out for this (test_against_gold.html), but it was removed from the menu in 5881885 since it wasn't yet functional.
Open questions (from Slack):
@pahp:
- In your mind, would this be simply for running
goldto see what it does, or run it like an attack, where we print the output of bothgoldand the team's executable, and then compare them? I think the first might be slightly easier, but they should be pretty similar, since we've already got all the logic worked out anyway. - Since users probably want this for fairly quick feedback, do you think it makes sense to prioritize these runs above the "normal" runs (that is, the normal "attack vs. team code" runs). Does that make sense, or can you think of any problems that might cause?
- I figure I'd rate limit these, probably 6-ish per minute, similarly to attacks. I can't think of any issues we'd have without rate limiting other than potentially denial of service, but it seems the safer bet. And 6 per minute is fairly generous, I think.
Metadata
Metadata
Assignees
Labels
No labels