When do you use a custom data source?

[brettcannon]

I just finished reading https://toga.readthedocs.io/en/stable/topics/data-sources.html . It has the following paragraph:

A custom data source enables you to provide a data manipulation API that makes sense for your application. For example, if you were writing an application to display files on a file system, you shouldn’t just build a dictionary of files, and use that to construct a TreeSource. Instead, you should write your own FileSystemSource that reflects the files on the file system. Your file system data source doesn’t need to expose insert() or remove() methods - because the end user doesn’t need an interface to “insert” files into your file system. However, you might have a create_empty_file() method that creates a new file in the file system and adds a representation to the data tree.

OK, but then widgets that use a Source like Tree say:

A Source will be used as-is. It must either be a TreeSource, or a custom class that provides the same methods.

That seems to be in conflict with each other if you're encouraged to leave off methods you don't need but which the widgets say are needed consistently across the widgets based on using the built-in data source classes as protocols.

I'm sure I'm misunderstanding something here (I haven't done much GUI programming and I'm still reading the Toga docs). Maybe custom data sources are meant to be for listening to the built-in ones provided as per:

For example, a second data source might register as a listener to an initial source to implement a filtered source. When an item is added to the first data source, the second data source will be notified, and can choose whether to include the new item in it’s own data representation.

?

[freakboy3742]

OK, but then widgets that use a Source like Tree say:

A Source will be used as-is. It must either be a TreeSource, or a custom class that provides the same methods.

That seems to be in conflict with each other if you're encouraged to leave off methods you don't need but which the widgets say are needed consistently across the widgets based on using the built-in data source classes as protocols.

The widget using a source only views the information that the source contains. If a row is added to the source, the widget is notified that a row has been added so that it can be displayed (if necessary); but the widget won't ever call insert() on a source directly, because the Tree widget doesn't modify the data in a source it's displaying.

Some GUI action (including an action on the tree) might lead to new data being created. In that case, the handler for that GUI action might call insert() on the source to add that new entry - but the API that is used is entirely at the discretion of the person writing the Source. In the filesystem example, you wouldn't have an insert() call - you'd have a create_new_file() call which creates a file on disk in a given location; the source would then detect the new file (via file system monitoring, or through an explicit refresh() call), add the representation for the new file to the source, and notify any widget viewing the source that there's a new row to display.

Strictly, the only thing that a Source should need to implement is the Source protocol - which provides the mechanism to "listen" to changes; plus data retrieval methods like __getitem__(), __iter__(), __len__() and index().

However, the fact that I can't point you at a clear documentation page (or even Protocol specification) for this, or to a clear definition of the "listener" methods that might be invoked on Source, indicates there's a gap in our documentation. I'll open a ticket to highlight the need to clarify this topic.

For example, a second data source might register as a listener to an initial source to implement a filtered source. When an item is added to the first data source, the second data source will be notified, and can choose whether to include the new item in it’s own data representation.

The table_source example demonstrates this idea. Visually, there's 2 tables - both displaying a list of movies. The table on the left is showing a "MovieSource" - a vanilla list of movie entries. The table on the right is showing a filtered movie source, only showing "good" movies.

The source for the table on the right doesn't actually store any data - it's implemented as a listener on the first table's source. When a row is added to the first table's source, the second table's source is notified, and the second table's source makes a decision as to whether the new entry should be part of the second table source's data. If it is, it passes on the notification to the second table; if it isn't, the second table does nothing.

This is, admittedly, an artificial example. The idea it's trying to highlight is the separation between a source of data, and a widget displaying that data. The widget should be thought of a view over some collection of data; but that "view" doesn't necessarily mean that it's a concrete store of a list/tree - it could be an abstraction over another source of data (including one that isn't even stored in the app itself - like a filesystem, or a network service API)

[brettcannon]

Strictly, the only thing that a Source should need to implement is the Source protocol - which provides the mechanism to "listen" to changes; plus data retrieval methods like __getitem__(), __iter__(), __len__() and index().

That's what I figured, but the docs seemed fairly specific about wanting the full protocol. But ...

However, the fact that I can't point you at a clear documentation page (or even Protocol specification) for this, or to a clear definition of the "listener" methods that might be invoked on Source, indicates there's a gap in our documentation. I'll open a ticket to highlight the need to clarify this topic.

I almost filed a bug instead, but I felt like that was skipping a step. 😅