Integrator for Trait based representations

[antirotor]

Changelog Description

Integrator plugin that will integrate all trait based representations based on current generic integrator.

Note

This already includes #979

Warning

The loader part needs ynput/ayon-python-api#225 and ynput/ayon-backend#552 - from the server side, you need at least 1.7.5

Representations and traits

Introduction

The Representation is the lowest level entity, describing the concrete data chunk that
pipeline can act on. It can be specific file or just a set of metadata. Idea is that one
product version can have multiple representations - Image product can be jpeg or tiff, both formats are representation of the same source.

Brief look into the past (and current state)

So far, representation was defined as dict-like structure:

{
    "name": "foo",
    "ext": "exr",
    "files": ["foo_001.exr", "foo_002.exr"],
    "stagingDir": "/bar/dir"
}

This is minimal form, but it can have additional keys like frameStart, fps, resolutionWidth, and more. Thare is also tags key that can hold review, thumbnail, delete, toScanline and other tag that are controlling the processing.

This will be "translated" to similar structure in database:

{
    "name": "foo",
    "version_id": "...",
    "files": [
        {
            "id": ...,
            "hash": ...,
            "name": "foo_001.exr",
            "path": "{root[work]}/bar/dir/foo_001.exr",
            "size": 1234,
            "hash_type": "...",
        },
        ...
    ],
    "attrib": {
        "path": "root/bar/dir/foo_001.exr",
        "template": "{root[work]}/{project[name]}...",
    },
    "data": {
        "context": {
            "ext": "exr",
            "root": {...},
            ...
    },
    "active": True
    ...

}

There are also some assumptions and limitations - like that if files in the
representation are list they need to be sequence of files (it can't be a bunch of
unrelated files).

This system is very flexible in one way, but it lacks few very important things:

• it is not clearly defined - you can add easily keys, values, tags but without
  unforeseeable
  consequences
• it cannot handle "bundles" - multiple files that needs to be versioned together and
  belong together
• it cannot describe important information that you can't get from the file itself, or
  it is very expensive (like axis orientation and units from alembic files)

New Representation model

The idea about new representation model is obviously around solving points mentioned
above and also adding some benefits, like consistent IDE hints, typing, built-in
validators and much more.

Design

The new representation is "just" a dictionary of traits. Trait can be anything provided
it is based on TraitBase. It shouldn't really duplicate information that is
available in a moment of loading (or any usage) by other means. It should contain
information that couldn't be determined by the file, or the AYON context. Some of
those traits are aligned with OpenAssetIO Media Creation with hopes of maintained compatibility (it
should be easy enough to convert between OpenAssetIO Traits and AYON Traits).

Details: Representation

Representation has methods to deal with adding, removing, getting
traits. It has all the usual stuff like get_trait(), add_trait(),
remove_trait(), etc. But it also has plural forms so you can get/set
several traits at the same time with get_traits() and so on.
Representation also behaves like dictionary. so you can access/set
traits in the same way as you would do with dict:

# import Image trait
from ayon_core.pipeline.traits import Image, Tagged, Representation


# create new representation with name "foo" and add Image trait to it
rep = Representation(name="foo", traits=[Image()])

# you can add another trait like so
rep.add_trait(Tagged(tags=["tag"]))

# or you can
rep[Tagged.id] = Tagged(tags=["tag"])

# and getting them in analogous
image = rep.get_trait(Image)

# or
image = rep[Image.id]

Note

Trait and their ids - every Trait has its id as a string with
version appended - so Image has ayon.2d.Image.v1. This is used on
several places (you see its use above for indexing traits). When querying,
you can also omit the version at the end, and it will try its best to find
the latest possible version. More on that in Traits

You can construct the Representation from dictionary (for example
serialized as JSON) using Representation.from_dict(), or you can
serialize Representation to dict to store with Representation.traits_as_dict().

Every time representation is created, new id is generated. You can pass existing
id when creating new representation instance.

Equality

Two Representations are equal if:

• their names are the same
• their IDs are the same
• they have the same traits
• the traits have the same values

Validation

Representation has validate() method that will run validate() on
all it's traits.

Details: Traits

As mentioned there are several traits defined directly in ayon-core. They are namespaced
to different packages based on their use:

namespace	trait	description
color	ColorManaged	hold color management information
content	MimeType	use MIME type (RFC 2046) to describe content (like image/jpeg)
	LocatableContent	describe some location (file or URI)
	FileLocation	path to file, with size and checksum
	FileLocations	list of FileLocation
	RootlessLocation	Path where root is replaced with AYON root token
	Compressed	describes compression (of file or other)
	Bundle	list of list of Traits - compound of inseparable "sub-representations"
	Fragment	compound type marking the representation as a part of larger group of representations
cryptography	DigitallySigned	Type traits marking data to be digitally signed
	PGPSigned	Representation is signed by PGP
lifecycle	Transient	Marks the representation to be temporary - not to be stored.
	Persistent	Representation should be integrated (stored). Opposite of Transient.
meta	Tagged	holds list of tag strings.
	TemplatePath	Template consisted of tokens/keys and data to be used to resolve the template into string
	Variant	Used to differentiate between data variants of the same output (mp4 as h.264 and h.265 for example)
	KeepOriginalLocation	Marks the representation to keep the original location of the file
	KeepOriginalName	Marks the representation to keep the original name of the file
	SourceApplication	Holds information about producing application, about it's version, variant and platform.
	IntendedUse	For specifying the intended use of the representation if it cannot be easily determined by other traits.
three dimensional	Spatial	Spatial information like up-axis, units and handedness.
	Geometry	Type trait to mark the representation as a geometry.
	Shader	Type trait to mark the representation as a Shader.
	Lighting	Type trait to mark the representation as Lighting.
	IESProfile	States that the representation is IES Profile.
time	FrameRanged	Contains start and end frame information with in and out.
	Handless	define additional frames at the end or beginning and if those frames are inclusive of the range or not.
	Sequence	Describes sequence of frames and how the frames are defined in that sequence.
	SMPTETimecode	Adds timecode information in SMPTE format.
	Static	Marks the content as not time-variant.
two dimensional	Image	Type traits of image.
	PixelBased	Defines resolution and pixel aspect for the image data.
	Planar	Whether pixel data is in planar configuration or packed.
	Deep	Image encodes deep pixel data.
	Overscan	holds overscan/underscan information (added pixels to bottom/sides).
	UDIM	Representation is UDIM tile set.

Traits are Python data classes with optional
validation and helper methods. If they implement TraitBase.validate(Representation) method, they can validate against all other traits
in the representation if needed.

Note

They could be easily converted to Pydantic models but since this must run in diverse Python environments inside DCC, we cannot
easily resolve pydantic-core dependency (as it is binary written in Rust).

Note

Every trait has id, name and some human-readable description. Every trait
also has persistent property that is by default set to True. This
Controls whether this trait should be stored with the persistent representation
or not. Useful for traits to be used just to control the publishing process.

Examples

Create simple image representation to be integrated by AYON:

from pathlib import Path
from ayon_core.pipeline.traits import (
    FileLocation,
    Image,
    PixelBased,
    Persistent,
    Representation,
    Static,

    TraitValidationError,
)
    
rep = Representation(name="reference image", traits=[
    FileLocation(
        file_path=Path("/foo/bar/baz.exr"),
        file_size=1234,
        file_hash="sha256:...",
    ),
    Image(),
    PixelBased(
        display_window_width=1920,
        display_window_height=1080,
        pixel_aspect_ratio=1.0,
    ),
    Persistent(),
    Static()
])

# validate the representation

try:
    rep.validate()
except TraitValidationError as e:
    print(f"Representation {rep.name} is invalid: {e}")

To work with the resolution of such representation:

try:
    width = rep.get_trait(PixelBased).display_window_width
    # or like this:
    height = rep[PixelBased.id].display_window_height
except MissingTraitError:
    print(f"resolution isn't set on {rep.name}")

Accessing non-existent traits will result in exception. To test if
representation has some specific trait, you can use .contains_trait() method.

You can also prepare the whole representation data as a dict and
create it from it:

rep_dict = {
        "ayon.content.FileLocation.v1": {
            "file_path": Path("/path/to/file"),
            "file_size": 1024,
            "file_hash": None,
        },
        "ayon.two_dimensional.Image": {},
        "ayon.two_dimensional.PixelBased": {
            "display_window_width": 1920,
            "display_window_height": 1080,
            "pixel_aspect_ratio": 1.0,
        },
        "ayon.two_dimensional.Planar": {
            "planar_configuration": "RGB",
        }
}

rep = Representation.from_dict(name="image", rep_dict)

Addon specific traits

Addon can define its own traits. To do so, it needs to implement ITraits interface:

from ayon_core.pipeline.traits import TraitBase
from ayon_core.addon import (
    AYONAddon,
    ITraits,
)

class MyTraitFoo(TraitBase):
    id = "myaddon.mytrait.foo.v1"
    name = "My Trait Foo"
    description = "This is my trait foo"
    persistent = True


class MyTraitBar(TraitBase):
    id = "myaddon.mytrait.bar.v1"
    name = "My Trait Bar"
    description = "This is my trait bar"
    persistent = True

    
class MyAddon(AYONAddon, ITraits):
    def __init__(self):
        super().__init__()

    def get_addon_traits(self):
        return [
            MyTraitFoo,
            MyTraitBar,
        ]

Usage in Loaders

In loaders, you can implement is_compatible_loader() method to check if the
representation is compatible with the loader. You can use Representation.from_dict() to
create the representation from the context. You can also use Representation.contains_traits()
to check if the representation contains the required traits. You can even check for specific
values in the traits.

You can use similar concepts directly in the load() method to get the traits. Here is
an example of how to use the traits in the hypothetical Maya loader:

"""Alembic loader using traits."""
from __future__ import annotations
import json
from typing import Any, TypeVar, Type
from ayon_maya.api.plugin import MayaLoader
from ayon_core.pipeline.traits import (
    FileLocation,
    Spatial,

    Representation,
    TraitBase,
)

T = TypeVar("T", bound=TraitBase)


class AlembicTraitLoader(MayaLoader):
    """Alembic loader using traits."""
    label = "Alembic Trait Loader"
    ...

    required_traits: list[T] = [
        FileLocation,
        Spatial,
    ]

    @staticmethod
    def is_compatible_loader(context: dict[str, Any]) -> bool:
        traits_raw = context["representation"].get("traits")
        if not traits_raw:
            return False

        # construct Representation object from the context
        representation = Representation.from_dict(
            name=context["representation"]["name"],
            representation_id=context["representation"]["id"],
            trait_data=json.loads(traits_raw),
        )

        # check if the representation is compatible with this loader
        if representation.contains_traits(AlembicTraitLoader.required_traits):
            # you can also check for specific values in traits here
            return True
        return False

    ...

Usage Publishing plugins

You can create the representations in the same way as mentioned in the examples above.
Straightforward way is to use Representation class and add the traits to it. Collect
traits in list and then pass them to the Representation constructor. You should add
the new Representation to the instance data using add_trait_representations() function.

class SomeExtractor(Extractor):
    """Some extractor."""
    ...

    def extract(self, instance: Instance) -> None:
        """Extract the data."""
        # get the path to the file
        path = self.get_path(instance)

        # create the representation
        traits: list[TraitBase] = [
            Geometry(),
            MimeType(mime_type="application/abc"),
            Persistent(),
            Spatial(
                up_axis=cmds.upAxis(q=True, axis=True),
                meters_per_unit=maya_units_to_meters_per_unit(
                    instance.context.data["linearUnits"]),
                handedness="right",
            ),
        ]

        if instance.data.get("frameStart"):
            traits.append(
                FrameRanged(
                    frame_start=instance.data["frameStart"],
                    frame_end=instance.data["frameEnd"],
                    frames_per_second=instance.context.data["fps"],
                )
            )

        representation = Representation(
            name="alembic",
            traits=[
                FileLocation(
                    file_path=Path(path),
                    file_size=os.path.getsize(path),
                    file_hash=get_file_hash(Path(path))
                ),
                *traits],
        )

        add_trait_representations(
            instance,
            [representation],
        )
        ...

Developer notes

Adding new trait based representations in to publish Instance and working with them is using
set of helper function defined in ayon_core.pipeline.publish module. These are:

• add_trait_representations
• get_trait_representations
• has_trait_representations
• set_trait_representations

And their main purpose is to handle the key under which the representation
is stored in the instance data. This is done to avoid name clashes with
other representations. The key is defined in the AYON_PUBLISH_REPRESENTATION_KEY.
It is strongly recommended to use those functions instead of
directly accessing the instance data. This is to ensure that the
code will work even if the key is changed in the future.

Closes #911

[antirotor]

btw tests are failing because they need changes to be implemented here https://github.com/ynput/pytest-ayon/tree/chore/align-dependencies

[iLLiCiTiT]

Suggested change

log = None

[antirotor]

this is actually helping with mypy and hints - ITrayAddon is using self.log.warning() but it is not defined there.

[BigRoy]

Thanks for the call that demo'ed this PR.

Use traits during publishing to improve data flow (and clarity on how publishing works)

I want to emphasize once more that I think where we would currently gain the most of traits (or at least structured dataflow) is during publishing. Being able to know what kind of data I can or should add during publishing (and before publishing/integrating) is what we're currently struggling with the most. Less so with the "loading" side of things after the data is ingested (because usually that loading part is just easier overall).

As such, I'd LOVE to see more examples of using the traits to actually improve the publishing process itself.

Here I'd love to see:

• How multiple plug-ins would easily build a representation with its traits? This is where we want to avoid storing into some magical instance.data["myRepresentations"] data structure that is undefined and instead rely on higher level API functions - this is exactly the part of the publishing that needs streamlining/improving.
• How to find the right representation to act upon (e.g. Extract Review).

Because the loading logic is usually relatively trivial I'm actually more worried that "traits" would solely make them more complicated.

Preferably, the data flow improvements also allow streamlining (or offloading to the farm) e.g. the full review process. So that all we'd do I store the "Traits" of data, for then another farm job to act upon. Just so we have structured data to act on, instead of instance.data with ANY untyped data.

Backwards compatibility

With "Loaders" intended to start relying on Traits data that'd also mean that those loader logics wouldn't be backwards compatible with existing publishes. Meaning that those loaders wouldn't be valid for AYON deployments for years to come to remain backwards compatible. Unless we either implement fallbacks in the code (which means suddenly we now need to maintain BOTH the old and new logic on loaders, for a very long time OR find a way to automatically apply traits to existing publishes.)

If we want our loaders to use Traits data from the published representations I think the only way forward is to automatically compute trait data for existing publishes. Because maintaing BOTH code logic in Loaders forever (since loading should be VERY backwards compatible, as much as we can, forever) seems to just double the workload to no benefit but only adding complexity.

---

Some other questions:

• How do I represent an animated UDIM sequence? Currently it seems a sequence is either UDIM or not.
• How does traits work with a Maya look publish that generates a .ma file, a .json and ANY amount of resources? I assume each "resource" needs to be tracked now with traits (does that mean that these files MUST now become representations each?)
• With FrameRanged trait, what is in and out versus start and end? I have no idea.

Some notes:

• I feel like KeepOriginalName and KeepOriginalLocation isn't really a "trait" of the published data. I feel the trait is named wrong maybe? Maybe it should be UsesOriginalName? But even then, is this really a "trait" of the published data? or just a rule during ingesting/integrating? I think these are somehow off in what traits are supposed to be?

[BigRoy]

Might be me - but not entirely sure what "Lighting" trait would apply to?

[antirotor]

Sure, this one serves more like an example of type trait. Maybe superceded by IntendedUse? Imagine compound representation using Bundle publishing fbx with lights, IES profiles in separate file, etc. Might be used to tag "generic" fbx that it contains Lighting? It is overlapping with the product itself, for sure.

[BigRoy]

FPS should likely be a separate trait, no?

Also, what happens if something would have a variable frame rate?

[jakubjezek001]

Also I am not sure why is fps in string and not float?

[antirotor]

It is part of the original OpenAssetIO trait, that is why it is there. If variable frame rate is ever needed then yes, it should be separate trait and variable frame rate can be described by another trait with lists? Dunno

[BigRoy]

If a new version of this trait would get created, does that mean we'd need a different class? Or how would we actually version traits in practice?

I assume somehow we'd need to keep the old trait class implementation around?

[jakubjezek001]

Issue is that versioning make sense in OTIO schemas or the original OAIO where we deal with json schema files, right? This way we will need to add version even to the class name, right?

[antirotor]

Whenever you want to de-serialize data that are using older version of trait, upgrade() method on newer trait definition is called to reconstruct new version (downgrading isn't possible). That means when you encounter trait like ayon.2d.Image.v1 in data and your runtime Image trait is v2, some method will call upgrade() method on newer trait version if implemented (for example in cases where newer version knows how to provide missing data or do conversion, etc.)

[antirotor]

For reference - upgrade() is called with old trait as an argument on .from_dict() calls and when you get traits by id and you omit version specifier (and old trait different than the runtime version is found)

[jakubjezek001]

I have tested it as it is and nothing had been broken

[antirotor]

Thanks for your points and questions. Regarding

Preferably, the data flow improvements also allow streamlining (or offloading to the farm) e.g. the full review process. So that all we'd do I store the "Traits" of data, for then another farm job to act upon. Just so we have structured data to act on, instead of instance.data with ANY untyped data.

I see representation traits as the lowest level building block for this. Once you can describe the results of the publishing, you can build API and structures on top of it. For example - you could have create_review(source_representation) that will use validate if the representation has all traits needed for createing review and it will return representation with review - just stupid oversimplification, but you get it. You'll then know exactly what traits and data you need to produce review.
Additionaly you can build structure to instance - instance is describing published product and since products share some common aspects, you could have instance-level traits.

How multiple plug-ins would easily build a representation with its traits?

class ProcessDiffuseTexture(pyblish.api.InstancePlugin):
    ...
    def process(self, instance: pyblish.api.Instance) -> None:
        ...
        diffuse_texture_traits = [
            Image(),
            PixelBased(
                display_window_width =1920,
                display_window_height = 1080,
                pixel_aspect_ratio = 1.0 
            )]
        add_trait_representation(instance, Representation("foo", diffuse_texture_traits))
        ...

Then plugin to fill in lets say planar configuration:

class CollectTextiureConfiguration(pyblish.api.InstancePlugin):
    ...
    def process(self, instance: pyblish.api.Instance) -> None:
        ...
        # add planar configuration to all Image representation that are missing it.
		for repre in get_trait_representations(instance):
            if repre.has_trait(Image) and not repre.has_trait(Planar):
                repre.add_trait(Planar(planar_configuration="RGB"))
        ...

After that you want to modify pixel aspect just on representation foo:

class ChangeFooPAR(pyblish.api.InstancePlugin):
    ...
    def process(self, instance: pyblish.api.Instance) -> None:
        ...
        repre = next(rep for rep in get_trait_representations(instance) if rep.name == "foo")
        if repre and repre.has_trait(PixelBased):
            repre[PixelBased.id].pixel_aspect_ratio = 1.0

We might have helper function get_trait_representation_by_name(name: str) or get_trait_representation_by_traits(traits: list[TraitBase]).

---

Backwards compatibility

So far the traits can help where there is ambiguity - where loaders can't do better than guess. Then we can introduce new class of loaders that will just work with trait based representation. And if that is not an option, loader with fallback logic if traits are not present. I think in most cases the loaders will use traits if there is no other way around or in cases where you need to load compound type representations - then you can have one loader using traits (because there is no other way) invoking old-style loaders on subrepresentations for example. But that is per-use case. I agree that dropping backwards compatibility in loader for traits needlessly isn't really an option.

How do I represent an animated UDIM sequence? Currently it seems a sequence is either UDIM or not.

Honestly not sure right now. Something to find out definitely.

How does traits work with a Maya look publish that generates a .ma file, a .json and ANY amount of resources? I assume each "resource" needs to be tracked now with traits (does that mean that these files MUST now become representations each?)

You are right that everything needs to be tracked. This is necessary for better site sync functionality etc. anyway. Resources are logically bound to some representation - i.e. they are needed for working with loaded product - like the look you've mentioned.

In this case you would use Bundle trait as follows:

maya_file = [
    FileLocation(file_path=Path("/path/to/look/shaders.ma")),
    MimeType(mime_type="application/maya"),  # or whatever?
	Tagged(tags=["maya", "shaders"])
] 
relations = [
    FileLocation(file_path=Path("/path/to/look/relations.json")),
    MimeType(mime_type="text/json"),
    SourceApplication(
        application="Maya",
        version="2026",
        platform="Windows x86_64")
]
diffuse_texture = [
        Image(),
        PixelBased(
            display_window_width=1920,
            display_window_height=1080,
            pixel_aspect_ratio=1.0),
        Planar(planar_configuration="RGB"),
        FileLocation(
            file_path=Path("/path/to/diffuse.jpg"),
            file_size=1024,
            file_hash=None),
        MimeType(mime_type="image/jpeg"),
    ]
bump_texture = [
        Image(),
        PixelBased(
            display_window_width=1920,
            display_window_height=1080,
            pixel_aspect_ratio=1.0),
        Planar(planar_configuration="RGB"),
        FileLocation(
            file_path=Path("/path/to/bump.tif"),
            file_size=1024,
            file_hash=None),
        MimeType(mime_type="image/tiff"),
    ]

bundle = Bundle(items=[maya_file, relations, diffuse_texture, bump_texture])
representation = Representation(name="look", traits=[bundle])

This will create single representation with all those files. If you want to create more loose relationship, you can create individual representations and then in main one use Fragment trait that is taking id of the other representations.

With FrameRanged trait, what is in and out versus start and end? I have no idea.

This is taken from OpenAssetIO FrameRanged trait and are in essence inclusive handles. It is duplicating Handles trait so up to the debate whether or not to remove it (or remove Handles trait).

KeepOriginalName and KeepOriginalLocation

Admittedly this is primarily used to drive integration but also keeping the information that the original name and locations were used and not modified by integrator itself. Might be useful for archival tool for example. Naming is difficult as usual, I stayed with what we are already using but I am definitely open to suggestions. UsesOriginalName is actually good one.