Apple Introduces OpenELM, Tiny AI Language Models for Smartphones
Apple’s latest AI language models: OpenELM
In the world of AI, small language models have been gaining popularity for their ability to operate directly on a smartphone without the need for powerful cloud-based servers. This week, Apple unveiled a collection of small yet powerful AI language models called OpenELM. Though they are currently in the proof-of-concept stage, these models have the potential to form the foundation of future on-device AI initiatives by Apple.
OpenELM: Fast, Efficient, and Accessible
The newly introduced OpenELM models, available on the Hugging Face platform under the Apple Sample Code License, provide a source-available solution for AI language processing on smartphones. While some debate exists regarding their classification as open source, the source code for OpenELM is made available. Apple’s OpenELM models, offering remarkable language comprehension abilities, have varying parameter ranges, from 270 million to 3 billion, spread across eight distinct models. This sets them apart from larger language models, like Meta’s Llama 3 boasting 70 billion parameters or OpenAI’s GPT-3 with 175 billion parameters.
Better Performance on Fewer Tokens
What distinguishes OpenELM from other small AI models is Apple’s unique “layer-wise scaling strategy.” Through this strategy, parameters are allocated efficiently at each layer, resulting in not only improved model performance but also reduced computational resource requirements. According to Apple’s white paper, OpenELM achieved 2.36% higher accuracy than Allen AI’s OLMo 1B model while utilizing fewer pre-training tokens, showcasing Apple’s commitment to optimizing on-device AI capabilities.
Enlarge – A table comparing OpenELM with other small AI language models in a similar class. (Image: Apple)
Transparency and Engagement with the AI Community
Apple’s commitment to openness and transparency is further pronounced with the release of not only the OpenELM source code, but also the model weights and training materials. This move, aimed at providing the open research community with vital resources, aligns with Apple’s mission to involve and enrich AI research. However, Apple does caution that the publicly sourced training data creates a possibility for inaccurate, biased, or objectionable outputs in response to user prompts.
A Glimpse into the Future
While OpenELM has not yet been integrated into iOS consumer devices, there are strong indications that the forthcoming iOS 18 update, which is expected to be announced at WWDC in June, will contain new AI features that leverage on-device processing to enhance user privacy. Moreover, there is speculation that Apple may collaborate with Google or OpenAI to handle more complex, off-device AI processing, offering enhanced capabilities to its virtual assistant, Siri.
