Exploring the Sonic Universe of Avatar: Frontiers of Pandora
Sport Developer Deep Dives is an ongoing series aimed at illuminating specific design, artistry, or technical aspects within a video game to illustrate how seemingly straightforward yet critical design choices are not actually simple at all.
Previous editions cover topics such as how camera effects, sound effects, and VFX shaped a smooth and high-energy movement system in Echo Point Nova, the technical journey behind bringing The Cycle: Frontier to Unreal Editor for Fortnite, and how the creators at ROAR Games crafted three distinct, overlapping game worlds within a single cohesive narrative in Tenet of the Spark.
Hello, I’m Sylvain Livenais, a senior sound designer at Wide Entertainment: A Ubisoft Studio, currently working on Avatar: Frontiers of Pandora. Avatar: Frontiers of Pandora is a first-person, action-adventure game set in the Western Frontier of Pandora. Players assist their character in reconnecting with their lost heritage and discovering what it truly means to be Na’vi.
Pandora is a vibrant world filled with thousands of diverse plants, animals, and insects. I had the chance to shape the sound for many of these. In this piece, we’ll examine two particular examples: The Grab Vine and the Veilswarm.
The Grab Vine
Audio direction
The Grab Vine blurs the lines between plant and animal. It is what the Avatar universe refers to as “zooplantae” – they might resemble exotic plants, but they can exhibit strange, playful interactive behaviors. They can grasp you when you get close (as shown with the “Helicoradian” in the Avatar film), electrocute you as you explore the area, or in the case of the Grab Vine: lift you up a ledge when you grab onto them.
Certain characteristics of the Grab Vine look like those of an animal, while others resemble a plant. For instance, the tail of the vine seems very much like a reptilian or bird claw, whereas the tip looks like a plant with leaves and branches. We aimed to highlight both aspects by creating organic sound textures, to ensure it felt grounded and believable, and to blend well with the overall natural soundscape in the game world. However, since they have gameplay functionality, we also ensured they have sharp transients to make them easily noticeable for the player within the busy mix of sounds in the rainforest. The Grab Vine also features several variations with gameplay differences that we will discuss later. Given its frequent presence in the game, scattered across much of the open world, the sound couldn’t be overly intrusive, repetitive, or loud. We wanted the sound to be engaging and captivating enough for players to want to utilize the Grab Vine.
Gameplay
-
Allows the player to ascend or descend ledges after grabbing the vine.
-
The vine isn’t accessible in polluted environments. The player cannot use it.
-
A second gameplay variant of the Grab Vine features the vine within a bulb (the spherical part at the top), rendering it inaccessible for the player. They must shoot the bulb to free the vine.
-
A third variant of the Grab Vine also hides the vine, making it inaccessible for the player. They need to climb and interact with the small vegetation atop the ledge to release the vine.
Audio design
The sound of the Grab Vine comprises a mix of sound libraries and natural recordings. I recorded various props, such as a leather belt, jacket, deflated balloon, different greens, etc., and processed them in Reaper.
Grab Vine in Wwise. Image and caption by courtesy of Ubisoft.
The source sounds are implemented in Audiokinetic Wwise, utilizing random containers with sound variations. We paid significant attention to detail in the implementation. Each and every instance of the Grab Vine was crafted to have its own unique, carefully placed sound, not just when the player interacts with it but also when it remains still in the open world. For example, when the Grab Vine is not in use, we can hear the vine or the claw slightly rustling. The leaves in the vegetative layer above it also softly react according to the wind’s strength in the environment. All these little details deserve a sound.
Let’s explore these components in detail, starting with the vine. The sound of the vine is characterized by leathery and creaky textures. I recall being in the recording studio, twisting an old, massive rope in every imaginable direction to capture as many variations as possible, which turned out to be quite enjoyable. Two audio emitters are positioned along the vine, with their distance from one another varying based on the vine’s length.
On the left, a vine. In the center, a 15-meter vine length, and on the right, a 30-meter vine length. Image and caption courtesy of Ubisoft.
The vine can measure just a few meters long, or extend up to 30 meters. The placement of the audio emitters adjusts according to this length to align with the vine. This the emitter positioning variation is programmed in the Take Hold Vine’s map within Snowdrop, our proprietary game engine. We determine the actual vine length by subtracting the Y axis world position of the top from the tail. We then divide the difference by a constant. This quotient is added to the tail’s Y axis world position, resulting in the Y axis for the audio emitter.
The sounds of the vine creaking cease to play when the environment it occupies has been contaminated by human adversaries in the game. Visually, when it becomes polluted, the vine disappears, the bulb and vegetative platform turn grey and appear sickly. We utilize the audio emitter’s position and assess its pollution level at runtime. When it exceeds a certain threshold, we set a Boolean to false, preventing the sound from playing.
How we dynamically adjust the audio emitter position according to the vine length in Snowdrop. Image and caption courtesy of Ubisoft.
Contaminated Take Hold Vine. Image and caption courtesy of Ubisoft.
Snowdrop map showcasing the pollution level node. Image and caption courtesy of Ubisoft.
The idle sound of the claw, when it is stationary and open, consists of rope creaks and gravel friction to create a “clicky” organic sound. This gives the claw a distinct animalistic quality. We didn’t want it to sound too plant-like, as it closely resembles a monstrous claw. Thus, this element needed to differ significantly from the rest of the vegetation. Additionally, as mentioned earlier, these sharp transient click sounds are designed to attract the player’s attention.
Take Hold Vine claw. Image and caption courtesy of Ubisoft.
The upper section of the Take Hold Vine consists of the bulb and the vegetative platform. Typically, the bulb contains the vine within it (one of the gameplay variations). Visually, when it does, we can see that the bulb is quite large and spherical, inflated. The idle sound of the bulb remains very organic but emphasizes the fact that it holds something inside. It also plays perfectly when players employ their “Na’vi senses” to listen, but we’ll delve into that later.
Inflated bulb containing the vine within it. Image and caption courtesy of Ubisoft.
In this scenario, the player must shoot the bulb to release the vine. When destroyed, we trigger a bulb burst and a vine release sound with organic undertones. We aimed for the bulb explosion to have a puffy, grainy, almost smoky quality to match the large brown seed cloud VFX that is released. The sound of the vine being set free needed to evoke moisture, as we envisioned that inside the bulb, there existed some form of lifeblood, even though it remains unseen. We extrapolated the sound from the visuals to give the plant even more vitality.
Bulb Explosion. Image and caption courtesy of Ubisoft.
Next, we encounter the third gameplay variation: the player needs to climb and interact with the small flora atop the Take Hold Vine to release it. We created a unique one-time sound for the plant retracting, maintaining the same vegetative and organic feel.
Plant to interact with. Image and caption courtesy of Ubisoft.
Image courtesy of Ubisoft.
When the player ascends or descends using the Take Hold Vine, we trigger an “in-ear” wind sound to convey a better sense of speed. The idea is to replicate a sensation we’ve all experienced in reality: the subtle, fluttering sound of air rushing past our ears when there’s significant wind or rapid movement. This is one of the ways we try to leverage the game’s first-person perspective, allowing the player to genuinely feel as though they are present—more immersed in the world and connected to their character. This sound is presented in 2D, meaning it shifts with the player’s position rather than being anchored in the environment. Consequently, we need to dynamically switch between a stereo and a 4-channel (quad) setup, depending on whether the player utilizes a surround sound system.
In-ear sound configuration in Wwise. Image and caption courtesy of Ubisoft.
Having the 2D in-ear wind sound emitter linked to the player introduced a challenge. In co-op mode, we didn’t want this sound to activate when a teammate uses the Take Hold Vine because it wouldn’t be logical for the local player to hear this proximity sound when a remote player interacts with the vine.
Snowdrop node. Image and caption courtesy of Ubisoft.
To address this, we established a condition with a Boolean to determine if the player using the Take Hold Vine is the local player or a remote player from the server. This was just one of many considerations made to ensure a seamless co-op experience.
works for every enthusiastic gamer.
As mentioned previously, let’s examine the player’s personality trait known as “Na’vi Senses.” Consistently, the player can enter a “highly focused mode” by pressing a button on the controller, which aids in navigation throughout the game. This mode visually and audibly emphasizes important elements and allows the player to discover pertinent information. (During early development, this feature was referred to as Master Hunter Senses, which is the reason you might notice the abbreviation MHS in these images).
Wwise swap container. Image and caption courtesy of Ubisoft.
When Na’vi Senses are activated, and the lightbulb is large and rounded, with the vine inside and inaccessible, we initiate an additional layer of organic and crisp looping sounds within the bulb. We also implement some dynamic mixing with azimuth and elevation RTPCs to make it more perceptible when the player looks directly at it. Each of these elements is quite subtle on their own. However, when combined, they guide the player towards the Gather Vine in the dense and busy open world without becoming overpowering.
Na’vi Senses activated to guide the player. Image and caption courtesy of Ubisoft.
Dynamic mixing with Wwise Azimuth and Elevation RTPCs. Image and caption courtesy of Ubisoft.
Now that we’ve covered all the sounds emitted from the Gather Vine, clearly, the player will be able to grab onto it as part of the gameplay mechanics. Therefore, we required some “foley” sounds to accompany the character’s hands interacting with the zooplantae. To ensure the sound aligns with what the player sees on screen, we needed to work on the Gather Vine graphical object as well as in the player animation system to integrate these subtle sounds.
Wwise implementation for player grabbing the vine sound. Image and caption courtesy of Ubisoft.
Audio integration in the Snowdrop animation system. Image and caption courtesy of Ubisoft.
Working on the Gather Vine was incredibly fascinating as it is a highly interactive element, leading us to engage with a variety of distinct sounds, including the sound of the Gather Vine itself as well as the player’s foley sounds and 2D in-ear wind effects. This also required navigating through various graphs and systems within the game engine, creating solutions for dynamic spatialization and movement for audio emitters in both Snowdrop and Wwise. It was immensely satisfying to weave all these different aspects of this feature together and utilize sound to create a cohesive experience for the player.
The Veil Swarm
Image courtesy of Ubisoft.
Audio direction
The Veilswarm is a group of tiny creatures that the player’s Ikran (the winged mount and companion) can consume to replenish its stamina. They resemble butterflies, but are significantly larger compared to Earth butterflies, prompting us to include medium-sized fluttering sounds. We avoided any type of tonal singing or bird-like whistling sounds, as the Veilswarm is omnipresent in the open world, and hearing the same type of sound repeatedly would become tiresome. Something a bit more understated was sufficient. Furthermore, the open world already features a variety of tonal sounds from different flying animals, so there was no need to overwhelm this aspect. Naturally, we aimed for the sound to convey a swarm surrounding the player, especially needing to be believable when the player flies through the swarm and they all scatter. Therefore, we executed a variety of work on these fluttering sounds through range and texture, as well as through implementation, positioning, and movement.
Image courtesy of Ubisoft.
Gameplay
If the player is soaring on their Ikran and navigates through the Veilswarm, the swarm will scatter, but the Ikran will catch and eat several of the creatures to regain stamina.
Image courtesy of Ubisoft.
Audio design
The swarm disperses in all directions when the player flies through it. We play a whooshing windy sound as it disperses to give the player a feeling of speed and dynamism. We trigger this based on the player’s velocity and proximity to the Veilswarm. In addition to the whoosh sound, when it disperses, we trigger a one-time frantic fluttering sound. Another layer consists of an audio loop within a combination container using various sources based on the distance to the player. This helps them to locate the Veilswarm while flying with the Ikran in the open world, while keeping the sound of the swarm grounded and authentic at varying distances.
Image courtesy of Ubisoft.
Image courtesy of Ubisoft.
If the player remains inside the swarm while flying slowly, the swarm will envelop them. In this case, a 2D stereo or quad frantic Veilswarm fluttering sound plays (2D meaning not specifically positioned in the environment, but surrounding the player, similar to the in-ear wind sound mentioned earlier). We alternate between the stereo or quad format depending on the player’s speaker configuration. This 2D sound provides a strong sense of being encircled by the swarm by filling the entire audio field with a broad, custom audio asset.
As the player moves within the swarm, the tiny creatures will swiftly dart to the sides, escaping from the Ikran’s attempts to consume them. We dynamically adjust the sound using Wwise’s real-time parameter controls (RTPC), based on camera movement, player proximity to the object, player speed, and azimuth. We modify the volume, pitch, and spatialization using these RTPCs. The fluttering sound dynamically increases in volume or pitch and stays synchronized with the visuals as they flare out to the sides, since we use the same information inputs as the visuals in the graphical object.
Wwise mixing RTPCs for
The swarm disperses sound while present within it. Image and caption by the Ubisoft project.
To enhance clarity during cooperative play, accurate enjoyment with the Grasp Vine is essential; we want to eliminate the 2D sound from the local player’s perspective if the remote player triggers it.
Wwise swap container for local and remote players. Image and caption by the Ubisoft project.
The Veilswarm comprises two audio emitters. The first operates at the initial location of the graph object. The second one adjusts its position dynamically based on the player’s location. It reflects the player’s position and its axis is anchored to the starting point of the Veilswarm object. This setup enhances the sound, making it more vibrant when the player remains inside the swarm and moves around.
When the player shifts to the left, the upper emitter shifts to the right. Image and caption by the Ubisoft project.
When the player moves to the right, the upper emitter shifts to the left. Image and caption by the Ubisoft project.
This positional mirroring turned out to be crucial in conveying the illusion of being enveloped by countless pairs of flapping wings, achieving our goal of making the player feel as if the world of Pandora is continuously responding to them.
Snowdrop audio emitter calculation reflecting the player’s position. Image and caption by the Ubisoft project.
The Veilswarm was also a collective effort. The player’s Ikran activates a “feeding” animation when passing through it, along with some visual effects when the small creatures are consumed by the Ikran (Pandora is a perilous place…). The player character might respond with a line of dialogue as the Ikran feeds, and the UI stamina bar replenishes with a small animation. Thus, the Veilswarm not only involved a variety of different sound designers and narrative designers but also allowed us to collaborate with all these diverse departments working on this aspect of the experience.