Running Machine Learning Models Efficiently on Android Devices
Running machine learning directly on an Android device sounds straightforward: load a model, pass in some data, and return a prediction. In reality, mobile inference is a balancing act. A model that runs beautifully on a desktop GPU may be far too large, power-hungry, or slow for a mid-range phone. Real Android devices have limited memory, thermal constraints, different chipsets, and very different combinations of CPUs, GPUs, DSPs, and NPUs. That is why running machine learning models efficiently on Android ...
