So far we have had over 5000 views of our virtual 360 firework display across our YouTube and Facebook posts.

This was a new challenge for us, our first full 360 experience and our first look at 3D ambisonic sound.

We used a variety of tools, plugins, and software to recreate the excitement of bonfire night. Here are the steps we took to complete the show:

1 – Matterport scan

We started with a Matterport scan of the Bloom Building. We imported this into Blender and did some cleaning up of rogue vertices and unneeded rooms.
This was then exported as an fbx file and imported into the Unreal Engine.

2 – Particle Effect Fireworks!

We downloaded the Fireworks Particle Effects asset pack which is currently free on the Epic Games Marketplace. We customised the provided blueprint to alter the curvature of some explosions, added sound effects and custom colours, then created our own logos in Blender. These logos were imported into the Unreal Editor, rigged up with skeleton physics, and then used in custom firework explosions.

3 – Sound Effects

We purchased the Real Fireworks asset pack from the Epic Games Marketplace and selected a few sounds for launch and edited a few samples for the different firework explosions. These were added to the firework launcher blueprints to auto play on launch and when exploding.

We enabled Volume Attenuation on these sounds to change the volume based on the distance between the user/camera and the audio source.
We chose to Enable Spatialisation and set this to Binaural. This interprets the sound in a 3D environment, allowing the user to detect the direction of sound through just two channels (eg Headphones, Left and Right).

We found that we needed to set up sound concurrency sound groups as we found many sounds playing at the same time caused the audio to crackle. We applied the rule of stopping the oldest sound if a newer one of the same grouping starts to play.

We also used the Resonance Audio SDK to implement reverberation, this created a virtual “room” around our world that would reverberate the sound to add realism to each blast.

4 – The Show!

We used the Sequencer in the Unreal Engine to animate the walls and ceiling falling away and added trigger events for each firework launcher.

In total there are 44 different firework launchers placed around the scene. Each of these has a unique position, speed, rotation, lifespan, burst type and colour. They also have a set of triggers on the timeline, telling them when to launch.

We were provided an audio track and used the sequencer (and a spreadsheet!) to time some of the explosions to the beat of the song.

5 – 360 Degrees

We purchased the 360 Camera pack from the Unreal Marketplace to allow us to export 360-degree videos and images. This allows a regular camera to be placed in the scene, then a 360 camera recorder. This 360 camera recorder converts the regular camera feed to a 360 monoscopic view. We then recorded this 360 feed to an 8k video, this took about 3 hours per minute of footage.

6 – Post Production

Although the audio sounds 3d when viewing the video with earphones facing forward, when played in VR or in a 360 movable video, if the user was to turn around 180 degrees, the audio would not change to reflect this. So, as soon as the user stops facing forward, the audio sounds incorrect.

This is where Reaper Audio came in. We used the ATK (Ambisonic ToolKit) plugin to create an ambisonic, 4 track version of the audio. You can visualise the sound within this awesome application, even choose the size of the head that is hearing the sound, You can also move the sound 360 degrees around the virtual head, to give the impression of realistic movement.

Exporting this sound to an ambisonic, 4 track file means that the audio changes, depending on the direction of the user, even once the video is rendered.

7 – The Final Render

The Ambisonic Firework Audio file, the Unreal Video, and Music Track are were loaded into Premiere Pro. We set the sequence to an 8k ambisonic VR video with ambisonic 4.0 sound. This was then uploaded and processed for VR on YouTube and Facebook.

8 – Data Response from YouTube and Facebook

Our current YouTube view count was great. With no paid advertisement on the platform, we gained over 700 views, with 6.8 hours watch time.

On Facebook, we spent only £20 over 4 days to boost the post. This and the great content had:

3299 post engagements,

969 playthroughs,

3290 3 second playthroughs (this ain’t amazing, but it teaches us that viewers have the attention span of 3 seconds, and the 360 experience starts a bit too slow, to build ambience. It only highlights that people need moving pictures).

and reached 5882 people.