-
Notifications
You must be signed in to change notification settings - Fork 705
Description
Is your feature request related to a problem? Please describe.
Currently, TimeXer
and NBeats
are non-conformant with the expected output contract of the v1 estimators tested under TestAllEstimators
, but the tests are passing for these metrics.
Describe the solution you'd like
Improve the current testing framework for 2d and 3d tensor returns from the model. The current testing API checks if the output is either 2d or 3d. We can change the tests to only look for 3d tensor returns and a list of 3d tensors (in case of multi-target). This way we can strengthen the standardization in the output contract for models, especially with respect to the the 3rd loss_dim
dimension of the output tensors.
The code in question is
We can change this test to check for a 3d tensor and skip the tests for any model which is non-conformant. One PR #1936 is dependent on the PR which will eventually be raised on this issue. The model's actual output format can be changed in their respective PRs and the skips can be removed as and when the change is implemented.