Inference time

[qguo96]

Hello,
I’m working on the EfficientQA project. I’m wondering do you have a time limit of inference?
It might take 48h+ hours to do inference in my smaller model.
Thank you so much.

[soheeyang]

Hi, I'm just a participant in the challenge, but yes, the description in https://ai.google.com/research/NaturalQuestions/efficientqa/participate states that there is a time limit for the leaderboard submissions:

Test attempts are given 20 minutes to run, and official attempts are given 6 hours.

In the case of the unconstrained track, there are a few days of time for inference, as stated in https://efficientqa.github.io/:

All submissions to the memory constrained tracks must be made to the EfficientQA leaderboard before 2020/11/14. Once the leaderboard is closed, the test set inputs will be released and participants will be given until the end of 2020/11/17 to submit predictions to the unconstrained track.