ArrayFire

[clive-g-brown]

Have you thought about an ArrayFire back end [as well as CUDA) ? would make Flux usable on AMD devices [Mac].

[bhvieira]

What would need to be done? ArrayFire extends AbstractArrays, it looks like it could work out of the box. Have you tested it? I sadly don't own an AMD gpu to test it.

[clive-g-brown]

dunno, not my area of expertise but im happy to fork and have a go.

[clive-g-brown]

I tried the examples https://fluxml.ai/Flux.jl/stable/gpu/#

but using ArrayFire equivalents - all fine. difficult to tell if its doing anything on GPU [yet].

[bhvieira]

Have you seen #938 as well? It's based on ROCArrays, it gives a good idea on how to implement things and to test if they work

[bhvieira]

difficult to tell if its doing anything on GPU [yet].

Do some huge, repeated, matmul. Like a Dense(500, 500) on a 500x1000 random dataset or something similar and check gpu utilization, will give you a good idea if it works.

[clive-g-brown]

did something like that. AFarrays() seem to work ! but |>gpu, not so sure.
ROCArrays doesn't compile (OSX) - other packages also seem defunct.

[bhvieira]

gpu is simply syntactic sugar, if you can define your layers on AFArray params and they work then it's a good sign.

[clive-g-brown]

got it working. as you say, define the layers with ArrayFire, but also have to define the datasets as array fire. AF only support sigmoid AF. but if I do all that - works on AMD, 0.00017s for a given dataset versus 0.034 on cpu.(|>gpu doesn't do anything). Some of the speed difference there is undoubtedly the lack of latency on the async GPU calls.

[bhvieira]

0.0001s for a given dataset versus 0.04 on cpu

Cool!

|>gpu doesn't do anything

Yeah, you'd need to change a bit of code for it to work. gpu sends your arrays to Cuda, basically, that's all it does.

[DhairyaLGandhi]

We could get it to offload to AF, but yeah, just sugar above cuda

[bhvieira]

We could get it to offload to AF, but yeah, just sugar above cuda

I've been thinking of setting up GPU backends for Flux, either baking them into the package (a la Plots, with the many backends) or splitting it up into lightweight libraries (eg FluxCuda, FluxArrayFire and FluxROCm) for example. Then it's up to the user to load the correct one, and then the gpu method would be the correct one as well.

[bhvieira]

By the way @clive-g-brown, I'm not sure how AFArrays compare to CuArrays, but you might need to reconvert them to Arrays to save it to disk safely.

[clive-g-brown]

Great idea on the multiple back ends. You’d reach Mac users like me.

[DhairyaLGandhi]

I've been thinking of setting up GPU backends for Flux, either baking them into the package (a la Plots, with the many backends) or splitting it up into lightweight libraries (eg FluxCuda, FluxArrayFire and FluxROCm) for example. Then it's up to the user to load the correct one, and then the gpu method would be the correct one as well.

Or just do it through Requires? Similar to CUDAdrv.jl which might be cleaner and not require maintaining as many parallel glue packages.

[bhvieira]

I'll be honest here and say that I had never read into Requires 😋
But looking the package repository, that seems like it @dhairyagandhi96
Not sure about the intelligent way to make it work if the user has CuArrays and ArrayFire installed at the same time