Skip to content

[ENH] improve _integration in TestAllEstimators to address non-conformace of TimeXer and NBeats #1966

@PranavBhatP

Description

@PranavBhatP

Is your feature request related to a problem? Please describe.
Currently, TimeXer and NBeats are non-conformant with the expected output contract of the v1 estimators tested under TestAllEstimators, but the tests are passing for these metrics.

Describe the solution you'd like
Improve the current testing framework for 2d and 3d tensor returns from the model. The current testing API checks if the output is either 2d or 3d. We can change the tests to only look for 3d tensor returns and a list of 3d tensors (in case of multi-target). This way we can strengthen the standardization in the output contract for models, especially with respect to the the 3rd loss_dim dimension of the output tensors.

The code in question is

https://github.com/sktime/pytorch-forecasting/blob/21a46a21d1a2c9728736438f0ad2827712ae9188/pytorch_forecasting/tests/test_all_estimators.py#L342C9-L357C72

We can change this test to check for a 3d tensor and skip the tests for any model which is non-conformant. One PR #1936 is dependent on the PR which will eventually be raised on this issue. The model's actual output format can be changed in their respective PRs and the skips can be removed as and when the change is implemented.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions