16 Months Later: Still No Universal App for Apple Silicon

[Lucas]

@Savay

Just take a read how complicated it actually is to use the nVidia Tensor Cores via DirectML…and those have a much greater installed userbase.
And just because there is inferencing going on it doesn’t mean that those matrix units are necessarily really the most sufficient way for that, depending on the data that needs to be processd.
If FP32 datasets are used, which DeepPrime seems to be for the most part hence therefore probably also the dependence on the fp32 throughput, most matrix units are out of consideration anyway.
(The only Matrix Units that i’m aware of that can process FP32 Datasets are AMDs Matrix Cores from their most recent CDNA Professional GPGPUs.)

We actually do use NVIDIA Tensor Cores (found in RTX GPUs) since PL5, that’s how it became faster on Windows. This is done by using fp16 which also speed-ups processing on latest AMD GPUs.

@Savay

I still think they must have done some optimizations for at least some steps of the processing in this regard, in any other way the up to 4 times improvements on the Apple Silicon can’t really be explained…

We adapted our algorithm to be executable on Apple Neural Engine rather than only M1’s GPU.