Supporting other number types?

[timholy]

In https://github.com/HolyLab/ThickNumbers.jl I've implement a ThickNumber abstract type, meant to summarize things like intervals (using interval arithmetic). The canonical ThickNumber subtype looks like Interval(lo, hi), so it's a lot like Complex{Float64}. Is there a straightforward path to supporting such number types with GTPSA, or is that a limitation baked into the C library?

[mattsignorelli]

Unfortunately this is a limitation baked into the C library, which supports Float64 and ComplexF64.

While I will likely never have the time to, the dream would be to rewrite all functions in the C GTPSA library fully in Julia in a generic way, so that any number type can be supported. With the way I have structured this package, it would not be complicated to then swap out the C library for the Julia equivalent: the task would be to basically fill in all function calls in the low_level directory with generic Julia equivalents. The C library can be found here

[timholy]

Thanks for the answer!

WRT a Julia implementation, I imagine you're aware of TaylorSeries.jl and TaylorDiff.jl. I came here because neither seems to be supported by DifferentiationInterface, which might indicate that GTPSA is superior, but just thought I'd mention them.

I'll close this given that it doesn't seem useful to keep it open.

[gdalle]

WRT a Julia implementation, I imagine you're aware of TaylorSeries.jl and TaylorDiff.jl. I came here because neither seems to be supported by DifferentiationInterface, which might indicate that GTPSA is superior, but just thought I'd mention them.

Just passing by to say that it's not about being superior. DI is focused on first- and second-order autodiff, so TaylorSeries, TaylorDiff and GTPSA are all slightly off-target and thus not a priority in terms of support. Therefore, I decided to leave it to package devs whether or not they want to contribute an extension. Sofar only the GTPSA people did, TaylorDiff started a PR but never put anything in it.

[mattsignorelli]

For first-order autodiff and possibly even hessian vector products, the other backends in DI will likely be faster. In cases where the entire hessian is materialized, GTPSA will likely be faster. To third order I expect GTPSA to far outperform any other AD backend (e.g., GTPSA is x20 faster than ForwardDiff in my example).

As for which of GTPSA, TaylorDiff, and TaylorSeries is superior, I think it depends highly on the application. To my knowledge, TaylorDiff only gives directional derivatives, and they also state similar performance as ForwardDiff at second order. I find GTPSA to be ~x4 faster than ForwardDiff at second order in my example.

Because TaylorDiff's documentation implies much better performance than TaylorSeries, I have not bothered with benchmarking GTPSA against TaylorSeries.

GTPSA was born in the accelerator physics community, where we need to compute high order maps that represent the particle transport. As such, it has been highly optimized for performance at high order and with a large number of variables/parameters. GTPSA also has bindings with DiffEqBase.jl so that one can integrate and obtain a differential algebraic map.