As recipient of the inaugural 3M Art and Technology Prize at the Minneapolis Institute of Art, Luxloop was tasked with developing a new way that museum visitors could engage with the permanent collection.

Our primary goal was to create an experience that would add a layer on top of a visitor’s existing interaction with the museum instead of pulling focus away from the most important thing: the art.

By examining common social interactions in museums, we discovered a tendency for museum goers to listen-in to the conversations around them in the gallery spaces. Building upon the familiar interaction of a museum audio guide, we decided to develop a custom audio-based augmented reality app that would allow visitors to explore this impulse to eavesdrop.

Similar to immersive theater plays such as Sleep no More and Hopscotch, Overheard lets users listen-in on the conversations of fictional characters that are embedded in specific sites throughout the museum, allowing users to explore stories in their own way, at their own pace.

Combining Technology and Narrative

We did extensive research on the demographics and motivations of Mia’s museum visitors so that we could create characters that were authentic to the context. We then worked with a screenwriter to craft the dialogue of five 15-minute-long narratives that would be embedded in the museum with location-specific audio in 97 rooms, representing over 3000 years of art.

We did extensive research on the demographics and motivations of Mia’s museum visitors so that we could create characters that were authentic to the context. We then worked with a screenwriter to craft the dialog of five 15-minute-long narratives that would be embedded in the museum with location-specific audio in 97 rooms, representing over 3000 years of art.

After considering several options for tracking users locations, we deployed a network of Bluetooth Low Energy beacons across the entire museum. We developed a custom algorithm for tracking user location based on the specific needs of Overheard that increased accuracy over 20% compared to existing solutions. Finally, working with the museum’s Media/Technology and Curatorial departments we created a custom content framework that could scale and change to accommodate the museum’s ongoing operations.

Overheard checks Mia’s collections database and local weather conditions in realtime, so the app can dynamically change in response to individual works of art changing or moving, as well as specific weather conditions. This way, the fictional characters are always reacting to and addressing the same real-life environment that the user is in.

By organically incorporating site-specific narrative into an existing experience, we created a new way for visitors to engage with and relate to both the art on view and the museum context as a whole. Though fictional, the characters and their motives are relatable and provide a window into a variety of opinions and points of view, allowing visitors to validate and strengthen their own relationships with the museum.