Apple’s AI developers say they have made a breakthrough in deploying LLM (Large Language Models, like ChatGPT) on iPhones and other Apple devices with limited memory, thanks to a new innovative technology for using flash memory.
LLMs like ChatGPT and Claude usually require large amounts of memory to run, which is a challenge for iPhones that have limited memory capacity. To address this issue, Apple developers have come up with a new technology that uses flash memory — the same memory your apps and photos live in — to store the AI model’s data, Macrumors writes.
In a new research paper titled LLM in a flash: Efficient Large Language Model Inference with Limited Memory, the authors write that flash storage is larger on mobiles than working memory, which is traditionally used to run LLM. Their new method circumvents the limitation with two key techniques that minimize data transfer and maximize flash storage throughput:
Windowing: Think of this as a recycling method. Instead of loading new data each time, the AI reuses some of the data it has already processed. This reduces the need for constant memory retrieval, making the process faster and smoother.
Row-Column Bunting: This technique is like reading a book in larger chunks instead of one word at a time. By grouping data more efficiently, it can be read faster, speeding up AI’s ability to understand and generate language.
The combination of these methods allows AI models to use twice as much working memory as the phone actually has.
The breakthrough opens up new possibilities for future iPhones, such as more advanced Siri features, real-time language translation, and sophisticated AI-powered features in photography and augmented reality.
Apple is reportedly in the process of developing its own generative AI called “Ajax”.
2023-12-21 12:05:24
#Apple #develops #method #run #LLMs #iPhones
