Nvidia’s Avatar Cloud Engine for Games with Generative AI Allows Realistic Conversations with NPCs

Nvidia will show its Avatar Cloud Engine for Games technique with generative AI during Computex. Players should be able to have realistic conversations with NPCs with this. The technology can run locally and in the cloud, the company reports.

Nvidia’s ACE for Games allows users to talk to npc’s in games through their microphone. The technique uses a generative AI language model that can generate responses. Nvidia provides for that its NeMo framework, which allows developers to create, modify and use language models in their games. Developers can customize the language models with character lore and backstories and prevent inappropriate responses with NeMo Guardrails.

ACE for Games also includes Nvidia Riva, the company’s text-to-speech and speech-to-text technology. This converts the questions that players ask into text. That text is fed into the language model to generate a response, and then Riva is used again to convert that response into a verbal response.

An Nvidia Omniverse Audio2Face model, in turn, can generate realistic facial expressions and animations. This should, among other things, ensure that the mouth movements correspond to the answers that the npc gives. That AI technique already existed and is being used, among other things, for facial animations in the upcoming game Stalker 2: Heart of Chernobyl, although it is without generative AI.

The company showed during Computex a demo of ACE for Games, which the company created in collaboration with AI company Convai and has been rendered in Unreal Engine 5. In the demo, a player talks to an NPC named Jin, who owns a noodle shop. The player asks questions through his microphone and Jin gives appropriate answers. Finally, Jin gives the player a quest, in which a powerful criminal boss must be stopped.

According to Nvidia, ACE for Games’ neural networks can be optimized for different systems, allowing developers to make trade-offs around size, performance and quality. The models can be run in the cloud or locally on PCs. According to Nvidia, the models are optimized for as little latency as possible.