Fortnite UEFN Introduces AI-Enhanced NPC Conversations with Gemini 3.1 Flash Lite and ElevenLabs

Fortnite Unveils AI-Powered NPC Conversations Amid Controversy

The gaming community is still buzzing about Fortnite’s AI-driven Darth Vader incident, which prompted SAG-AFTRA to lodge an unfair labor practice charge against Epic Games. In light of this, the studio has introduced a groundbreaking update to its technology.

Introducing “Conversations”: A Revolutionary AI Feature

Previously known as the “Persona device, ”this innovative technology is now referred to as “conversations.”This system empowers Fortnite developers to create AI-driven characters that possess unique personalities and voices, capable of engaging in dynamic dialogue with players. These characters can respond, react, and evolve based on player interactions, directly impacting the gameplay experience.

Latest Update: Features and Capabilities

Unveiled with the v40.20 update on April 16, 2023, this feature is integrated within the Unreal Editor for Fortnite (UEFN). It enables creators to develop NPCs that engage in unscripted, real-time conversations with players. Using as little as 20 lines of text, developers can define various aspects of a character’s personality, knowledge base, and behavior. The underlying technology is powered by large language models (LLMs), ensuring that NPCs generate context-aware responses based on player dialogue and actions.

Dynamic Gameplay Interactions

According to Epic Games, the NPCs are far more than mere talkers. They can remember player actions and leverage that information to influence gameplay outcomes. For instance, an NPC acting as a guard could be persuaded to allow access to restricted zones, potentially changing the course of in-game events. These characters can dynamically adjust difficulty levels or act as adaptive narrators, providing a richer gameplay experience.

Technical Framework Behind the Feature

Epic’s development of the conversations feature relies on advanced frameworks such as Scene Graph and the Verse API. Notably, the technology harnesses Google’s Gemini 3.1 Flash Lite model for processing audio inputs and generating textual outputs, along with ElevenLabs for voice synthesis. These models are recognized as leading choices in their respective fields.

Privacy and Developer Guidelines

Epic Games has assured players that it will not store any player audio data. The introduction of the conversations feature has also prompted updates to the Fortnite Developer Rule set, which now includes the following stipulations:

Do not create characters intended to provide medical or mental health advice.
Do not create personas that simulate or impersonate romantic partners or intimate companions.
Do not attempt to bypass Epic’s safety measures, including content restrictions.

Violations of these rules may result in enforcement actions, including the possibility of account bans.

Current Status: Experimental Phase

It’s worth noting that conversations remain in the Experimental phase. Consequently, Fortnite developers currently cannot publish islands featuring AI NPCs until the feature enters its Beta phase. The voice models are still under refinement, and the response times of the LLMs need improvement. During this period, Epic aims to enhance voice quality, minimize latency, and fine-tune NPC behavior.

Future Considerations

While Epic is currently absorbing the costs associated with using both Gemini 3.1 Flash Lite and ElevenLabs, it remains uncertain if this practice will continue once the feature launches publicly.

Additional Resources for Developers

For Fortnite developers interested in leveraging AI-powered NPCs, Epic has made available a comprehensive four-page documentation. You can access it for more detailed guidance and insights.

Source & Images