Yann LeCun, the chief AI scientist at Meta and one of the godfathers of AI, has criticized OpenAI’s new video generation model, Sora. While Sora has gained significant attention since its release, LeCun believes that OpenAI’s claims about building “general purpose simulators of the physical world” are misguided.
LeCun takes issue with OpenAI’s approach to creating a “world simulator” through pixel generation. He argues that this method is inefficient and doomed to fail, comparing it to the abandoned concept of “analysis by synthesis.” In simpler terms, LeCun believes that these models are trying to infer too many irrelevant details instead of focusing on essential factors like mass and velocity.
LeCun’s criticism stems from the debate between generative models and discriminative models in machine learning. He believes that generative models, which generate pixels based on explanatory latent variables, are inadequate for dealing with the uncertainty that arises in complex predictions within a 3D space.
While LeCun acknowledges that generative models have been successful with language models like ChatGPT, he argues that simulating the world requires dealing with much more than just a few characters. In response to OpenAI’s approach, LeCun has developed his own model called the Video Joint Embedding Predictive Architecture (V-JEPA), which was recently unveiled by Meta.
Meta claims that V-JEPA, unlike generative approaches, has the flexibility to discard unpredictable information, leading to improved training and sample efficiency. Although LeCun’s work may not receive the same level of hype as OpenAI’s flashy image and text generation products, it is intriguing to see a prominent AI researcher diverging from the current approaches being developed by OpenAI and its imitators.
In conclusion, Yann LeCun’s criticism of OpenAI’s video generation model highlights the ongoing debate between generative and discriminative models in AI. While Sora has garnered attention, LeCun argues that its pixel generation approach is inefficient and fails to understand the complexities of the physical world. His alternative model, V-JEPA, offers improved training and sample efficiency by discarding unpredictable information. As AI continues to evolve, it is fascinating to witness prominent researchers like LeCun challenging existing approaches and pushing the boundaries of AI development.