NVIDIA ACE - Generative AI Characters

Source: https://www.youtube.com/watch?v=QSWzSRnGEFo

COMPUTEX—NVIDIA today announced NVIDIA Avatar Cloud Engine (ACE) for Games, a custom AI model foundry service that transforms games by bringing intelligence to non-playable characters (NPCs) through AI-powered natural language interactions.

Developers of middleware, tools and games can use ACE for Games to build and deploy customized speech, conversation and animation AI models in their software and games.

“Generative AI has the potential to revolutionize the interactivity players can have with game characters and dramatically increase immersion in games,” said John Spitzer, vice president of developer and performance technology at NVIDIA. “Building on our expertise in AI and decades of experience working with game developers, NVIDIA is spearheading the use of generative AI in games.”

Pioneering Generative AI in Games Building on NVIDIA Omniverse™, ACE for Games delivers optimized AI foundation models for speech, conversation and character animation, including:

NVIDIA NeMo™ — for building, customizing and deploying language models, using proprietary data. The large language models can be customized with lore and character backstories, and protected against counterproductive or unsafe conversations via NeMo Guardrails. NVIDIA Riva — for automatic speech recognition and text-to-speech to enable live speech conversation. NVIDIA Omniverse Audio2Face™ — for instantly creating expressive facial animation of a game character to match any speech track. Audio2Face features Omniverse connectors for Unreal Engine 5, so developers can add facial animation directly to MetaHuman characters. Developers can integrate the entire NVIDIA ACE for Games solution or use only the components they need.

‘Kairos’ Offers a Peek at the Future of Games NVIDIA collaborated with Convai, an NVIDIA Inception startup, to showcase how developers will soon be able to use NVIDIA ACE for Games to build NPCs. Convai, which is focused on developing cutting-edge conversational AI for virtual game worlds, integrated ACE modules into its end-to-end real-time avatar platform.

In a demo, called Kairos, players interact with Jin, the purveyor of a ramen shop. Although he is an NPC, Jin replies to natural language queries realistically and consistent with the narrative backstory — all with the help of generative AI. Watch the demo, which is rendered in Unreal Engine 5 using the latest ray-tracing features and NVIDIA DLSS.

“With NVIDIA ACE for Games, Convai’s tools can achieve the latency and quality needed to make AI non-playable characters available to nearly every developer in a cost-efficient way,” said Purnendu Mukherjee, founder and CEO at Convai.

Deploy NVIDIA ACE for Gaming Models Locally or in the Cloud The neural networks enabling NVIDIA ACE for Games are optimized for different capabilities, with various size, performance and quality trade-offs. The ACE for Games foundry service will help developers fine-tune models for their games, then deploy via NVIDIA DGX™ Cloud, GeForce RTX™ PCs or on premises for real-time inferencing.

The models are optimized for latency — a critical requirement for immersive, responsive interactions in games.

Generative AI to Transform the Gaming Experience Game developers and startups are already using NVIDIA generative AI technologies for their workflows.

GSC Game World, one of Europe’s leading game developers, is adopting Audio2Face in its upcoming game, S.T.A.L.K.E.R. 2: Heart of Chernobyl. Fallen Leaf, an indie game developer, is using Audio2Face for character facial animation in Fort Solis, a third-person sci-fi thriller that takes place on Mars. Charisma.ai, a company enabling virtual characters through AI, is leveraging Audio2Face to power the animation in its conversation engine. ... https://www.youtube.com/watch?v=RQyxJWd4Vgc