4

Tensorflow came out with the XLA compiler which compiles the backend C++ tensorflow targeting LLVM. My understanding about XLA was that it was a step towards supporting generic accelerated devices, so long as there was LLVM -> Device support.

Tensorflow lite was more recently released, replacing Tensorflow Mobile, and appears to be where the work is focused on targeting embedded and mobile devices with an apparent focus on embedded DSP and GPUs as optional processors common in these environments. Tensorflow lite appears to hand off operations to the Android NNAPI (neural network API) and supports a subset of the tensorflow OPs.

So this begs the question: which direction is Google going in to support non CUDA based devices? And are there use cases for XLA beyond what I described?

Noam Hacker
  • 4,671
  • 7
  • 34
  • 55
David Parks
  • 30,789
  • 47
  • 185
  • 328
  • I'm not sure XLA is all that much about supporting accelerated devices. Although portability to new hardware is mentioned in [Why did we build XLA?](https://www.tensorflow.org/xla/overview#why_did_we_build_xla), I think it's mostly about time/memory performance, and also ease of integration (for AOT). TF Lite is more about minimising resources, it's another implementation really, although I suppose Lite and XLA could end up converging somehow. Wrt platform support, although only CPU and CUDA is officially supported (well and Cloud TPU), SyCL and ROCm support are available to compile. – jdehesa Dec 11 '18 at 14:32
  • In any case, if you want to know about future plans maybe you can get more authoritative answers in one of the [mailing lists](https://www.tensorflow.org/community/lists). – jdehesa Dec 11 '18 at 14:35

1 Answers1

6

I work on XLA. The XLA compiler has three backends: for CPU, GPU, and TPU. The CPU and GPU ones are based on LLVM and are open source, and the TPU one is closed source.

I don't know what the plans are for XLA for mobile devices, so I can't comment on that.

A benefit you get by using XLA with your TF model, instead of executing the model directly, is that XLA fuses a lot of ops for you. See this post for instance.

dimvar
  • 561
  • 5
  • 14
  • Did anything change regarding XLA for mobile devices? Especially in direction of Edge TPUs/Coral – jnnks Aug 04 '22 at 07:35