Add/Build type-hints for python package

[austinwitherspoon]

Feature Request

Add type hints to the python package, for both native python code and compiled code.

Description

Type hints are incredibly value in modern python, particularly for library code that may be unknown to developers! Type hints allow our IDEs to auto-complete, catch errors, and make simplify code discovery. I know not everybody loves type hints, but as a library, I believe OpenTimelineIO would benefit a lot from them in terms of automatic tooling integration and developer onboarding!

This is obviously pretty easy in the native python code, but in the pybind11 code it's slightly more tricky. Theoretically you can use mypy stubgen to do this with compiled modules? Is there a native pybind11 way of doing this? I'm not a C++ developer and have never touched pybind, so unfortunately I don't know for sure what the ideal workflow is here.

At my job, we have a mypy validated package that uses OpenTimelineIO, and I've spent a few hours trying to set up my own stub files for OTIO using mypy stubgen. It works fairly well, but it's not perfect. Possibly good enough for release though!

Perhaps this is a two parter -

1. Add type hints to external facing python code (easy)
2. At package build/deploy time, use a command like mypy stubgen -m opentimelineio to generate .pyi files to be included in the package.

Context

Add any other context here, such as simulated output, or screenshots, that might help clarify the request.

[JeanChristopheMorinPerso]

One issue with adding type hints today is that the metadata keyword argument is of type object or any today. This makes it very hard to have useful type hints. I created a PR to fix this a while back, see #1490.

Another issue is the serializable_field decorator, which again makes it hard to have useful type hints. This decorator is a beast when it comes to type hints at least it was a couple of months ago. I think I have a somewhat working version in my fork or locally.

There is also the read_from_file and friends functions that accept keyword arguments which are not simple to type since it's the adapters that define the arguments they accept.

So all in all, it's possible, but it'll be more complex than simply adding type hints to python files and generating stubs with stubgen in order to have workable and useful type hints.

[austinwitherspoon]

I think it would be preferable to have Any type hints than none at all - for instance, currently, read_from_file has no return type.

Instead, it could be something like:

def read_from_file(
    filepath: str,
    adapter_name: str | None=None,
    media_linker_name: media_linker.MediaLinkingPolicy =media_linker.MediaLinkingPolicy.ForceDefaultLinker,
    media_linker_argument_map:Any=None, # not sure what this one takes?
    **adapter_argument_map: dict[str, Any]
) -> Timeline:

By intentionally adding Any, we're communicating that "you're going to need to read more about this, we can't type hint it."

Everywhere else, we're saying "This is definitely what it takes."

In that example, we may not know what every argument is, but we can inform your IDE that you'll receive back a Timeline object.

This example is actually the one that made me write up an issue! I'm currently writing my own stub manually that tells mypy/pylance what the return time is of read_from_file().

And now that I've done that locally, I get autocomplete and all of that great stuff with read_from_file()

[reinecke]

I'd support adding type hints and docstrings as we go, starting to get these populated will help set the example going forward and let people know it's a work in progress.

[jminor]

Having type hints would be very helpful!

One detail: the specific case of read_from_file returns any SerializableObject not just Timeline since an OTIO file might have just one Clip, or a SerialzableCollection, or whatever. All of those are subclasses of SerializableObject. This is a common point of confusion since the majority of OTIO use cases only support Timeline at the top level. Having type hints, and better doc strings, would help to clarify this.

[drunkenQCat]

Hi @austinwitherspoon , if possible, could you please share your stub file and stub generating steps in a repo? It will save a lot of efforts and be a nice begin to do more about type-hints.

[drunkenQCat]

One issue with adding type hints today is that the metadata keyword argument is of type object or any today. This makes it very hard to have useful type hints. I created a PR to fix this a while back, see #1490.

Another issue is the serializable_field decorator, which again makes it hard to have useful type hints. This decorator is a beast when it comes to type hints at least it was a couple of months ago. I think I have a somewhat working version in my fork or locally.

There is also the read_from_file and friends functions that accept keyword arguments which are not simple to type since it's the adapters that define the arguments they accept.

So all in all, it's possible, but it'll be more complex than simply adding type hints to python files and generating stubs with stubgen in order to have workable and useful type hints.

object it's not a severe problem to me. in fact, i've noticed that most of the metadata can be parsed and used as dictionary. just hint them as dict, everything may be fine.