From magic to immersion, experiential marketing gets an AI upgrade

In the reception area at Google’s office in New York City, Deeplocal, an experience design agency, has suspended 160 electromechanical flowers from the ceiling. The kinetic installation, visible from the street, reacts to visitors and onlookers using real-time sensing and machine learning.

"It’s subtle yet surprising—people can’t help but smile when they make the connection that the flowers bloom in response to their movement," says Patrick Miller, Deeplocal's chief innovation officer. "It feels magical."

While flowers that bloom in response to human movement is one of Deeplocal's current AI-enabled activations, for almost ten years the agency has used AI to create innovative events and activations for clients like Google, Adult Swim, and National Geographic.

"AI has been a key tool in our work for years, even before generative AI took off," says Miller. "Our work lives in the real world, it’s sensory and tactile, so AI and machine learning have been great tools for bringing the digital world into the physical one."

AI wizardry in brand experiences

At experiential marketing agency Jack Morton Asia, AI is enabling new ways of storytelling.

"Production teams actively test and scour new AI-powered chatbots and assistants for events, and creative teams infuse experiences such as AR/VR with AI-led immersive technologies, creating impactful storytelling experiences," says Ilma Afzal, strategy director, Asia, Jack Morton Worldwide.

In one recent example, Jack Morton Worldwide worked on building a superfan experience for viral mobile game, Merge Mansion, that used AI tools to amplify the physical set builds, creating a never-before-seen activation that allowed fans to be completely immersed in the story. 

"AI was used not only in strategic brainstorming but also to create artwork, personal documents, clues, and other detailed fan touchpoints that further immersed guests in the game's world—all aiming to evoke the feeling of stepping into a lived-in mansion and rifling through personal documents," says Afzal.

"For example, artwork using Midjourney were seen on postcards from the fictional town of Hopewell Bay," adds Afzal. "The popular AI tool was also used to come up with designs for all the props and labels in the mansion’s kitchen. ChatGPT, meanwhile, generated storytelling copy seen throughout the mansion, on fan invitations, and in a voicemail recording that played when guests called-in before the event."

Geoff Renaud, CMO and co-founder at global brand experience agency, Invisible North, says that AI is being used at all stages from expedited design and render creation in planning stages, using some of NVIDIA and Adobe's new tools, to faster creation of virtual extensions of physical events using tools like Atlas.

"Rendering both physical and virtual experiences is becoming exponentially easier for creatives and producers adept at using these nascent tools," says Renaud.

But while generative AI tools like ChatGPT, Midjourney, and Atlas are in many cases helping to elevate experiential marketing and deliver new ways of working, some warn of the risks at this early stage.

"Whilst these tools are incredibly exciting, they all come with risks around copyright, bias, privacy and transparency," says James Bennett, director, creative technologies, APAC, Project Worldwide. "All things we believe will eventually be sorted out, but for now, not suitable for the vast amount of client experiences we deliver."

Bridging physical and virtual 

You only need to look at the rate at which generative AI has accelerated over the past nine months to understand just how quickly AI is developing. Experts in experiential marketing say that as the technology continues to accelerate, we will increasingly see the seamless bridging of the physical and virtual.

"In a few years when you go to an event it will be a truly mixed-reality experience," says Renaud. "If you look at all the amazing mixed-reality experiences at Coachella this year (Gorillaz, etc) you will get an idea of how fast the digital and physical worlds are becoming one. 

Gorillaz debut an AR YouTube experience at Coachella this year with a seemingly real-time AR installation that levelled up their live set for at-home viewers. Photo: Coachella

We are going to see some amazing digital visuals happening at live events and some of the tools like MXR or Atlas that are being built will allow both artists and producers to easily create amazing experiences with no code needed."

Along with live events, AI is also revolutionising the Blockchain, Metaverse, NFT, and Web3 brand experience realms.

"Utilising AI algorithms and predictive analytics to synthesise audience data and insights, brands and organisations are now able to create highly personalised Web3 campaigns," says Scott Cullather, president and CEO of  global live brand storytelling agency INVNT GROUP. "These Web3 campaigns are backed by user behaviour data that will support experiences that deliver exclusive utilities, NFTs, and content that authentically resonates with a brand’s audience – driving maximum impact and engagement."

Hyper-personalisation is another component of events and brand experience that AI looks set to enhance.

"We will be able to capture and track data across so many channels that hyper-personalisation for consumers will become the norm," says Renaud. "When you go to a business conference or a music festival, you will have a hyper-personalised journey readily mapped out for you."

Unleash AI's creative potential, but with transparency

While marketers are now able to harness generative AI-driven tools to power strategies that add new dimensions and create radically immersive, transformative brand experiences, there's no denying that there are some considerable risks right now around generative AI and copyright, bias, privacy and transparency.

"We don't have visibility on how these models were trained, notably what was used and where that information came from," says Bennett. "All of these things are going to come to courts around the world in the coming months/years. Notably, the EU is readying legislation around AI as they did for Personal Data (GDPR)."

Cullather of INVNT Group says that while AI can enhance creative and experiential campaigns, it’s paramount that fact-checking and verification processes be a part of your strategy and execution.

"Additionally, the execution of a brand experience could be skewed towards the wrong direction if it were ideated completely via insights driven by globally targeted AI technologies that are still developing," says Cullather.

As a solution, Cullather recommends never disregarding the audience data that you have captured through previous events, virtual experiences, brand experiences. “Rather, use AI to layer onto the information and analytics, to create the most curated, solution-oriented brand experience possible."

Comparing experiential marketing to other media such as performance and growth marketing, etc., measurement has always been a challenge. But through the use of cutting-edge methods like facial recognition and brainwave tracking, it is possible to monitor customer emotions in great detail.

"Finding acceptable ways to engage consumers and create healthy standards will be a challenge the industry will need to figure out," says Renaud. "If you look at what’s happened with Madison Square Garden in NYC and facial recognition tools (who used facial recognition to identify, accost, and remove lawyers involved in lawsuits against it) you can see that it’s very easy to get it wrong."

AI and human creativity must work together, not exclusively

In the long term, according to Miller of Deeplocal, the consumer-facing use cases for AI in experiential work include hyper-personalisation, endless original content, and seamless tactile and sensory interaction.

But we needn't worry about humans becoming obsolete anytime soon. While the technology is developing at an alarming rate, most experts in the field agree that humans will remain an essential part of the equation when it comes to experiential work and AI.

"It’s the human element that makes things matter and why people care," says Miller. "An AI-generated song might sound technically perfect. But, knowing that a song is expressing a person’s emotions and experiences is a lot more meaningful. To take the example a step further, being able to hear that song live and in a room with other people feeling those same things, is powerful."

In one recent example, Miller and his team at Deeplocal made an AI-powered musical instrument that combined Google Magenta’s Piano Genie model with a physical interface for the Flaming Lips Concert at Google I/O.

To get the audience involved, Deeplocal filled giant inflatables with sensors that signalled the software when touched, letting the crowd compose a song with the band in real-time.

"AI helped make an already powerful human experience a singular, unforgettable one," says Miller. "AI has helped crystalize that it’s human care and experience that make things meaningful—artificial representations of the human experience won’t cut it.”

The human desire to gather at live experiences is only likely to be enhanced by AI tools believes Invisible North's Renaud.

"One thing I’ve found consistent throughout many recent hype cycles is that even the heaviest digital natives absolutely love live experiences,” says Renaud.

“Gamers love conferences, live physical tournaments, etc.  Gen Z is attending festivals and conferences more than ever, stadium tours are selling out within minutes. So I think it’s an exciting time to work in live experiences as I only see massive augmentation coming from these new AI tools."