Apple researchers have developed new methods for training large language models on text and images, enabling more powerful and flexible AI systems, in what could be a major advance for the company’s future AI and products.
This week, the researchers published a paper titled: “MM1: Methods, Analytics, and Insights from Pre-Training of Large Multimodal Linguistic Models.”
This paper published this week shows how combining different types of training data and model architectures may lead to state-of-the-art performance on a range of AI benchmarks.
By training models on a diverse dataset including visual and linguistic information, MM1 models were able to excel at tasks such as image captioning, visual question answering, and natural language inference.
Apple researchers also found that the choice of image format conversion software and the resolution of the input images had a significant impact on model performance.
“We showed that image format conversion software combined with image resolution and the number of image tokens have a significant impact, while the design of the vision language connector has relatively little importance,” they said.
This suggests that continued expansion and improvement of the visual components of these multimodal models may be key to achieving further gains.
The large 30 billion-parameter MM1 model demonstrated strong contextual learning capabilities, allowing it to perform multi-step reasoning over input images using a few-shot train of thought.
This suggests the potential for large multimodal models to address complex and open-ended problems that require a basic understanding of language and its generation.
The MM1 research comes at a time when Apple is working to intensify its investments in artificial intelligence in an attempt to catch up with competitors who have rushed to integrate generative artificial intelligence capabilities into their products.
Apple is on track to spend $1 billion annually on developing artificial intelligence, according to a recent Bloomberg report.
MM1’s research shows that Apple has the talent and resources to make cutting-edge advances in order to keep pace with the escalating AI arms race.
(ait news)
2024-03-17 04:00:32
#Apple #making #significant #breakthroughs #artificial #intelligence
