Broadcast AR a key element at Singapore’s NDP

To celebrate Singapore’s 56th birthday, its annual National Day Parade returned over the weekend. But for the first time, AR was used as a key part of the parade to cater to viewing audiences at home via broadcast partner Mediacorp. These included the virtual projection of rain effects, a levitating mirrorball, and the dance of a hummingbird. 

Anomalyst Studio, a local multimedia and art tech company, was tasked to pull off the AR show, and began prepping six months in advance. Benjamin Kee, creative director at the agency, told Campaign Asia-Pacific that the team took a page from 3D game design by utilising the Unreal Engine platform to run the real-time content for the broadcast AR. Unreal Engine, which is used in many AAA games as well as in major Holloywood productions, provides graphics performance utilising ray-tracing technology.

The lifelike visual illusions were created in an outdoor floating stage area covering 5,000 square meters. The team needed to have 1-to-1 scale models of a virtual stage and the physical stage to start. To do this, they used a lidar (light detection and ranging, or laser radar) scanner to extract and translate an accurate scale model of the physical stage.

“Mobile AR, which most people are familiar with, utilises built-in sensors like gyroscope and accelerometer," said Kee. "For a broadcast format, we needed to augment the broadcast cameras to provide us with the data such as positional, pan, tilt, zoom, lens, etc. We need to be able to match and control the physical camera and lens to [match] the 'cameras' in our virtual environment. We initially tested with infrared camera trackers, but opted to use a camera robotic arm system, which is programmable and provides consistent choreography.”

To match the lighting in the environment and to translate the proper reflection of the physical stage to the virtual content, the team used 360 cameras to capture recording of the various lighting states, which they used to sample the lighting conditions of the environments. Once the content design was done, animated and programmed, the system was integrated to sync with the overall show’s time cue system.

Broadcast AR, according to Kee, took a while for everyone to warm up to, especially given the legacy live format of the NDP. But as confidence built among the team and client, the challenge for Kee was to push for the visual quality that made the AR feature appear believable and seamless with the show—instead of it coming across as a gimmick.

“After a year of largely virtual events, and the general trend towards hybrid events, people are getting accustomed to watching events live streaming," Kee said. "However, there are plenty of challenges to keep the engagement of viewers over their screen devices, it is easy to be distracted if the content is not engaging."