MUNICH (IT BOLTWISE) – Meta presents new AI tools that help robots better understand and interact with the physical world. The release of Sparsh, Digit 360 and Digit Plexus is intended to significantly improve human-robot interaction.
Meta made several significant announcements about robotics and embodied AI this week, including benchmarks and artifacts for improving robotics in the physical world. Newly introduced innovations include the Sparsh, Digit 360 and Digit Plexus models, all aimed at tactile perception, robot dexterity and closer human-robot interaction. Meta also launches PARTNR, a benchmark tool for assessing planning and decision-making capabilities in human-robot collaboration.
Meta is increasingly relying on the use of Large Language Models (LLMs) and Vision-Language Models (VLMs) to enable robots to perform complex tasks that require planning and problem solving.
Tactile perception with Sparsh
The Sparsh model, developed in collaboration with the University of Washington and Carnegie Mellon University, represents a family of encoder models for tactile perception. This technology enables robots to have touch-based perception, such as the ability to apply the correct pressure to objects to exercise. Sparsh is trained using self-supervised learning (SSL), eliminating the need for large, manually labeled data. The model has currently been trained with more than 460,000 tactile images collected from various datasets. Sparsh shows a 95.1% improvement over traditional end-to-end models, each suitable only for specific tasks and sensors.
Der neue Touch sensor Digit 360
Meta also introduces the Digit 360 hardware – an artificial, finger-shaped tactile sensor that has 18 different sensor features and 8 million taxels. This sensor allows robots omnidirectional and highly detailed touch detection. The AI models are integrated directly on the sensor, which means data processing takes place locally and latency times are minimized. This technology has potential applications in areas such as medicine, prosthetics, virtual reality and telepresence.
In addition to Digit 360, Meta introduces Digit Plexus, a platform that integrates multiple tactile sensors and efficiently transmits collected data to a computer. The open source sharing of code and design for both products aims to promote research and innovation in the robotics community.
Evaluation of human-robot interaction
With PARTNR, Meta presents a benchmark environment that analyzes collaboration between humans and robots in domestic tasks. PARTNR uses 100,000 language tasks in a simulated environment and includes more than 5,800 unique objects. This is intended to assess the performance of LLMs and VLMs in following human instructions. The benchmark joins a growing number of projects that are advancing the use of language models in robotics.
With these developments, Meta hopes to bring robotics one step closer to fluid human-robot interaction, aiming for a future in which robots can handle complex tasks in real-world environments.
Meta presents new AI tools to improve tactile perception in robotics (Photo: DALL-E, IT BOLTWISE)
A notice: Parts of this text may have been generated using artificial intelligence.
Please send any additions and information to the editorial team by email to de-info[at]it-boltwise.de
