Understanding Large Language Models in Your Pockets: Performance Study on COTS Mobile Devices

Understand

As large language models (LLMs) increasingly integrate into every aspect of our work and daily lives, there are growing concerns about user privacy, which push the trend toward local deployment of these models.

• There are a number of lightweight LLMs (e.g., Gemini Nano, LLAMA2 7B) that can run locally on smartphones, providing users with greater control over their personal data.
• As a rapidly emerging application, we are concerned about their performance on commercial-off-the-shelf mobile devices.
• To fully understand the current landscape of LLM deployment on mobile platforms, we conduct a comprehensive measurement study on mobile devices.