[FEATURE] Validation Frequency

[t0278611]

There is no parameter for validation frequency in the train script as-is. For usual runs like imagenet1k skipping validation isn't really something one would consider, but for test runs on small datasets, having the option to only run the validation/checkpointing logic every few epochs could save a lot of time

It is quite trivial to add a validation frequency parameter. I am happy to provide a pull request

[rwightman]

@t0278611 open to a PR, it's not hard but a few things to consider, by default, to save checkpoints with a consistent metric you'd need to save the 'epoch' based checkpoints only when eval is done, if you wanted to save checkpoints at a more fine-grained interval than the evals then you'd have to revert to train loss for the checkpoint saving metric...

[t0278611]

That is a good point. I only thought about the case of skipping the entire validation logic including checkpointing to save time. I think that would be the most used behavior, but it might not be clear based on the name. Do you have a suggestion?

[rwightman]

@t0278611 yes, the simplest is to leave the checkpoint saving tied to the validation frequency so only validation epochs are saved

[t0278611]

I created a PR

I was wondering why the choice was made to not log the last training step per epoch like in other framework or like it is done in the validation loop. I suppose it is because not every loader will have a length? Would it make sense to print it out for those that do have a length?

[rwightman]

@t0278611 feel free to add that, for the train dataloader the LR schedule does depend on knowing or having an estimate and enforcing a fixed # of steps per epoch so can do update_idx % args.log_interval or last_batch .. I swear it was like that at one point (or possibly in another related codebase).... either way seems harmless enough to add.

[t0278611]

so can do update_idx % args.log_interval or last_batch

@rwightman great, I added that

[rwightman]

@t0278611 great thanks, will do a quick test this afternoon

[rwightman]

@t0278611 looks good, made a small change so that the final epoch will get an eval