Apple Partners with University of Santa Barbara to Develop Open-Source AI Model for Image Editing
In a surprising move, Apple has joined forces with the University of Santa Barbara to venture into the world of artificial intelligence (AI) and open-source technology. The tech giant from Cupertino has collaborated with the university to create an AI model called Multimodal Large-Language Model-Guided Image Editing (MGIE), which aims to revolutionize image editing based on natural language interactions.
Unlike traditional AI models that focus solely on either text or images, MGIE is a Multimodal Large Language Model (MLLM) that can process both text and images simultaneously. This unique capability allows MGIE to understand complex instructions and work in a wide range of situations. For instance, the model can analyze a specific photo, interpret a text instruction, and then generate a new image by removing or adding elements based on the given command.
To achieve this level of image editing, MGIE relies on various capabilities such as generative text, generative image, segmentation, and CLIP analysis. By integrating these functions into a single process, Apple has brought its AI system closer to the conversational interactions offered by OpenAI’s ChatGPT Plus. Users can now engage with AI models using natural language commands to create customized images.
The introduction of MGIE means that users can transform images simply by providing detailed instructions in natural language. For example, a user can start with a photo of a blonde person and turn them into a redhead by saying, “make this person a redhead.” The model understands the instruction, segments the person’s hair, generates a command like “red hair, highly detailed, photorealistic, ginger tone,” and then executes the changes via inpainting.
Apple’s approach aligns with existing tools like Stable Diffusion, which already offers text-guided image editing. By leveraging third-party tools like Pix2Pix, users can interact with the Stable Diffusion interface using natural language commands and witness real-time effects on edited images. However, Apple’s MGIE proves to be more accurate and efficient than similar methods.
In addition to generative AI, MGIE can also perform other conventional image editing tasks such as color grading, resizing, rotations, style changes, and sketching. This comprehensive set of capabilities makes MGIE a powerful tool for both AI artists and developers.
But why would Apple make MGIE open source? The answer lies in the company’s strategic move to leverage the worldwide pool of developers and boost its strength and flexibility. By using open-source models like Llava and Vicuna, Apple can tap into the collective expertise of developers and accelerate the evolution of MGIE. This collaboration not only fosters innovation but also enhances Apple’s reputation among developers and tech enthusiasts.
Releasing MGIE as open-source software also gives Apple a head start in setting industry standards for AI and AI-based image editing. By providing a solid foundation for AI artists and developers, Apple has paved the way for more accurate and efficient image editing techniques. Moreover, this move allows users to integrate MGIE into various Apple products, such as smartphones, computers, and immersive headsets, enhancing their overall functionality.
For technically savvy AI developers, MGIE is readily available on the project’s GitHub repository. This accessibility ensures that developers can start utilizing MGIE’s capabilities immediately.
In conclusion, Apple’s partnership with the University of Santa Barbara to develop an open-source AI model for image editing marks a significant milestone in the field of artificial intelligence. With MGIE, Apple has not only created a powerful tool for image editing but has also demonstrated its commitment to collaboration and innovation within the open-source community. As AI continues to advance, MGIE sets a new standard for accuracy and efficiency in image editing, empowering users to bring their creative visions to life.
