Rank-typing

[jorenham]

Shape-typing is often described as being able to distinguish between axes or their their sizes. But in Python's current type-system, that isn't realistically possible. If you remove the individual axes from a shape, you end up with just a rank, a.k.a. ndim. At runtime it's trivially simple, because it's just an int. But helpfully describing it using static typing is quite the opposite, and is probably the biggest typing-challenge I've ever had to solve.

For a long time, I thought that integer tuples would be the only option we could use for this. But for many situations, that would be very impractical to use, and limited in its expressiveness. I'm not sure if it would have even been worth it.

But I recently figured out an overpowered trick that I've been calling "static embedding", with which I was able to encode the complete set of NEP 50 promotion rules within the scalar types. That made it possible to write a couple of Casts* type-aliases (powered by typing.Protocol), which, when applied to the binary scalar operator methods, reduced __init__.pyi by over 5,000 LOC (!). That made me realize that the underlying "static embedding" principle could also be used for the broadcasting rules, i.e. the main use-case of shape- rank-typing.

And so far, it's looking pretty promising. It may also provide a way to simplify the array-like typing aliases for specific ndim, by associating e.g. Sequence[Sequence[T]] with the 2-d rank-type, Rank2.

This issue is intended as a tracker for the rank-typing progress. See the PR's below for the juicy details:

• ✨ towards a broadcasting-aware rank-typing framwork #538
• 💥 use the new Rank* types in _numtype.Array* #544
• 🎨 prefer Rank* over Shape* and replace npt.NDArray with Array #545
• 🤖 _numtype rank test generation #550
• ✨ HasRank{LE,GE} type aliases for for broadcasting shaped things #552
• ♻️ variadic associated rank types #559