Memoir: Sony has passe generative AI to turn Aloy into an unsettling digital animatronic
By Reports indicate that Sony is investigating how generative AI tools, such as GPT, could be integrated with characters from key PlayStation franchises—and the results are intriguing to say the least.
Images shared with The Verge by an anonymous informer showcase a generative AI-enhanced version of Aloy from Horizon Forbidden West engaging in conversations with a user.
The video, which was previously available on YouTube but has been pulled down due to a copyright claim, was narrated by Sony Interactive Entertainment’s Director of Application Engineering, Sharwin Raghoebardajal, who stated that the demonstration offers a “sneak peek at what’s achievable.”
While the original demo may no longer be viewable, you can still catch a clip from it within this article, which is rather eerie due to a blend of exaggerated facial expressions and limbs that appear oddly stiff.
The footage (which has also been seen by Game Developer) shows Aloy communicating using a synthesized voice and facial movements while answering questions, powered by OpenAI’s speech recognition and problem-solving technologies, Affirm and GPT-4.
Meta’s newly released language model, Llama 3, was also utilized to enhance the demo—along with Sony’s proprietary Emotional Voice Synthesis (EVS) speech technology.
Raghoebardajal noted that the synchronization of audio to facial animations was made possible through Sony’s internal Mockingbird technology.
He refrained from addressing some pressing queries at the end of the video, such as why gamers would want to converse with Aloy or other PlayStation characters (and who exactly they might be) or how this technology could impact voice actors and developers.
Concerns about Gen-AI in Gaming
Generative AI continues to permeate the game industry, eliciting a varied response thus far.
Executives and leaders at major studios seem eager to champion this technology as a revolutionary force capable of reshaping production and gameplay paradigms.
Conversely, individuals such as voice actors and developers working on the ground have consistently expressed concerns regarding the implications this technology could have on job security and issues around copyright and authorship.
About the Author
News Editor, GameDeveloper.com
Chris Kerr, the editor for Game Developer, is an award-winning journalist and reporter with over a decade of experience in the gaming industry. His byline has appeared in prominent print and online media including Edge, Stuff, Wireframe, World Commerce Review, and PocketGamer.biz. Throughout his career, Chris has covered major industry events such as GDC and PAX Australia.