Sony is reportedly exploring how generative AI tools like GPT could be integrated with characters from major PlayStation franchises—and the results are weird to say the least.
Internal footage sent to The Verge by an anonymous tipster shows an genAI-imbued version of Aloy from Horizon Forbidden West holding conversations with a player.
The video, which was published on YouTube but has now been removed due to a copyright claim, was narrated by Sony interactive Entertainment director of software engineering Sharwin Raghoebardajal who said the demo provides a “glimpse of what is possible.”
You can, however, see a clip from the demo in the original article and it’s all rather uncanny thanks to a combination of over-exaggerated facial expressions and limbs that remain strangely motionless.
The footage (which has also been viewed by Game Developer) shows Aloy speaking with a synthesized voice and facial movements while responding to queries with the help of OpenAI speech recognition and problem solving technologies Whisper and GPT-4.
Meta’s open source large language mode, Llama 3, was also used to power the demo—alongside Sony’s own internal Emotional Voice Synthesis (EVS) speech generation system.
Raghoebardajal explained audio to face animation was handled by Sony’s internal Mockingbird technology.
He didn’t answer some rather pressing questions during the video, such as why players would need to chat with Aloy or other PlayStation protagonists (who they usually embody) or how the tech would impact voice actors and developers.
Gen-AI concerns in games
Generative AI continues to encroach on the video game industry and has so far garnered a mixed response.
Executives and leadership figures at major studios seem intent on heralding the technology as a game changer capable of rewriting the rules of production and gameplay.
Others, such as voice actors and developers in the creative trenches, have regularly raised concerns about the impact the technology could have on job security and the potential for legal issues in relation to copyright and authorship.
