A Study on the Battery Usage of Deep Learning Frameworks on iOS Devices

As machine learning continues to establish its presence on mobile platforms, there arises a need to evaluate model resource usage across a variety of devices and frameworks. In this paper, we measure the performance and battery usage of inference for three significant machine learning models, MobileNetV2, ResNet50, and BERT QA, when employing different deep learning frameworks (CoreML, Tensorflow, and PyTorch), iOS devices (iPhone 8 Plus, iPhone 11 Pro, and iPad Air 4), and threading configurations. Throughout our study, we systematically assessed key metrics: battery usage, inference duration, and accuracy rates. Our findings challenge some conventional beliefs; for instance, an increase in thread count did not always guarantee faster model execution, even when there are physical cores available. Similarly, a quick inference time was not always synonymous with higher energy efficiency. In addition, our study shows no single best framework for all cases. CoreML is more energy-efficient for MobileNetV2 and ResNet50 but sometimes slower, especially on older devices. TensorFlow Lite excels in energy and performance for BERT QA, even on newer hardware. While multithreading often helps, its benefits are limited, especially for CoreML beyond two threads. These results emphasize the need to tailor machine learning implementations to specific hardware and model characteristics, indicating room for improvement in existing frameworks.