Behind the Build: Bringing Murals to Life in AR with Wall Bop

Wall Bop is an audiovisual WebAR experience that transforms a mural by Kel Brown in Austin, Texas into an interactive musical memory game. Developed by Lauren Schroeder, the game invites players to tap on different sections of the mural, triggering animations and sounds, and challenging them to repeat a musical sequence correctly. Using Niantic Studio and Niantic VPS, Lauren mapped the game precisely to the mural’s location, blending the real and digital worlds in an engaging new way.

We spoke with Lauren about the creative process behind Wall Bop, the challenges of integrating WebAR with physical art, and how Niantic Studio helped bring the project to life.

What inspired you to create Wall Bop?
I’ve recently been creating audiovisual XR experiences and wanted to explore these concepts in AR. I was particularly interested in integrating a mural with an interactive sound-based experience, where players could explore sounds through the visuals of the artwork. This idea evolved into a musical memory game, where the mural itself becomes a dynamic, playable instrument.

How did Niantic Studio help bring your vision for this project to life? Were there specific tools or features that were essential?

I used Niantic Studio’s Niantic VPS functionality to anchor the experience to a specific real-world location. This allowed me to precisely map AR elements onto the mural, ensuring that when players interact with different sections, their taps align with the artwork. VPS made it possible to create an experience that seamlessly integrates with the real-world mural.

Were there any challenges along the way? How did you overcome them?

One of the biggest challenges was getting the mural VPS-activated. To successfully activate the location, I needed high-quality scans taken at different times of the day to account for lighting variations. It took me multiple trips to refine my scans and ensure good coverage and accuracy. Once activated, though, everything worked seamlessly.

What were the main steps you took to make this project that would be helpful to share with other developers?

1. Collaborating with artists: I reached out to public art creators to see if they’d be interested in having an AR experience built around their work. Some artists were excited, while others preferred to keep their murals separate from technology. I wanted to respect each artist’s vision while finding a great mural to work with.
   
2. Niantic VPS scanning & testing: I conducted multiple scans of the mural to ensure it would work well with Niantic VPS. High-quality, well-lit scans were crucial for precise AR placement.
3. Creating 3D & audio assets: I designed 3D elements inspired by the mural’s 2D art and sourced audio clips to remix into interactive sound samples for the game.
4. Building & scripting the experience: I used Niantic Studio’s scripting tools to implement touch-based interactions, animation triggers, and sequence-based gameplay mechanics.
   
5. Playtesting & refinement: I ran several tests to fine-tune gameplay, timing, and UI elements, ensuring an intuitive and fun user experience.
   

What advice would you give other developers interested in building 3D web, real-world games or AR experiences with Niantic Studio?
Start simple and just give it a shot. Niantic Studio has a ton of example projects that can help you get started. Experiment with interactivity and build on what inspires you.

Where do you see the future of AR going, and how does Niantic Studio fit into that vision?

I think AR will continue to evolve as tools and hardware become more advanced. What I find exciting about  Niantic 8th Wall and Niantic Studio is how they’re making developer-created features more accessible—helping lower the barrier to entry for creating immersive AR experiences. In the future, I see AR becoming even more seamless and intuitive, allowing for spontaneous, interactive, real-world experiences without complicated setup or equipment.

How Wall Bop works

Wall Bop uses a combination of audio, animation, and VPS tracking to create an engaging, site-specific game.

• Mural-based gameplay: Players interact with the physical mural by tapping different areas to trigger sounds and animations.
• Musical memory challenge: Players must watch and listen to a sequence played by the game, then repeat the sequence in the correct order to progress.
  
• VPS anchoring: The game is precisely mapped to a real-world mural using Niantic VPS, ensuring accurate AR placement.
  
• Dynamic audio & animation: Custom 3D elements and sounds bring the mural to life with interactive feedback on every tap.
• Winning the game: Once players successfully complete the final sequence, a special animation and song play to celebrate their achievement.
  

Pro tips for developers inspired by Wall Bop

• Choose a compelling location:
  If you’re working with VPS, make sure the location is scan-friendly and visually distinctive.
  
• Test Your VPS Activation Early:
  Multiple scans at different times of day will help ensure strong tracking accuracy.
  
• Use Simple, Clear Interactions:
  A touch-to-play mechanic makes gameplay intuitive and accessible.
  
• Mix Visual & Audio Feedback:
  Combining animations and sound cues enhances user engagement and interaction.
  

Play Wall Bop and start building today
Wall Bop is a fantastic example of how Niantic Studio and Niantic VPS can be used to blend art, music, and interactivity in the real world.
Interested in building your own location-based AR game? Get started with this Niantic Studio sample project:

• Studio: VPS Bespoke: This example project demonstrates a bespoke VPS experience (location-specific animations) using a VPS location 

Sign up for your free account to start creating with Niantic Studio today.