EchoVision

Introduction

EchoVision is an immersive art installation that allows participants to experience the world of bats using sound visualization and mixed reality technology. With a custom-designed, bat-shaped headset from the open-source HoloKit project, users can simulate echolocation, the natural navigation system bats use in the dark. They do this by using their voices and interpreting the returned echoes with the mixed-reality visualization.

The exhibit adjusts visual feedback based on the pitch and tone of the user's voice, offering a dynamic and interactive depiction of how bats perceive their environment. This installation combines scientific learning with empathetic engagement, encouraging an ecocentric design perspective and understanding between species. EchoVision educates and inspires a deeper appreciation for the unique ways non-human creatures interact with their ecosystems.

"What Is it Like to Be a Bat?"

In "What Is it Like to Be a Bat?" Thomas Nagel explores the unique, subjective experience of consciousness, using bats as a primary example due to their use of echolocation, a sensory perception vastly different from human experiences. He argues that understanding what it is like to be another creature is inherently limited by our inability to fully grasp their subjective experiences. This concept directly relates to EchoVision that allows participants to simulate the echolocation of bats using sound visualization and mixed reality technology. By adjusting visual feedback based on the user's voice, EchoVision bridges the gap between human and bat perception, offering a deeper, empathetic understanding of non-human sensory worlds and emphasizing the profound differences in how various species interact with their environments.

Misconception of Bats

People often harbor misconceptions about bats, viewing them as dangerous, dirty, or associated with negative superstitions. Common myths include the belief that bats are blind, that they commonly get tangled in human hair, or that they are primarily carriers of diseases (especially during the COVID-19 pandemic). These misconceptions contribute to a general fear and misunderstanding of bats, overshadowing their ecological importance and unique biological features.

Bats are ecologically vital, providing essential benefits such as pest control by consuming large quantities of insects, pollination of numerous plants, and seed dispersal that aids in forest regeneration. Their guano enriches soil fertility, contributing to nutrient cycling. These roles help maintain ecosystem balance and biodiversity.

EchoVision aims to challenge and transform these misconceptions by providing participants with an immersive, empathetic experience of the world through a bat's perspective. By simulating echolocation using sound visualization and mixed reality technology, EchoVision allows users to understand how bats navigate and perceive their environment with remarkable precision. This interactive experience highlights the sophistication and elegance of bat echolocation, fostering a greater appreciation for their role in ecosystems, such as pest control and pollination.

Through EchoVision, participants can see bats not as menacing creatures but as fascinating and integral parts of the natural world. The installation encourages scientific learning and empathetic engagement, promoting an ecocentric design perspective that values the diverse ways non-human creatures interact with their surroundings. By offering a new perspective, EchoVision helps dispel myths, reduces unfounded fears, and inspires a deeper connection to and respect for bats and their vital ecological contributions.

Echolocation

Bat echolocation is a sophisticated biological process that allows bats to navigate and hunt in the dark by emitting high-frequency sound waves and listening to the echoes that bounce back from objects in their environment. When a bat emits a sound, it travels through the air and reflects off surfaces, returning to the bat as an echo. By analyzing the time delay and changes in the frequency of the returning echoes, bats can determine the distance, size, shape, and texture of objects, as well as the speed and direction of moving prey.

In "An Immense World," Ed Yong describes how bats' echolocation abilities are finely tuned and highly specialized. He explains that bats can emit sounds at frequencies higher than the human ear can detect, often using calls that range from 20 to 200 kHz. The bat's brain processes these echoes with remarkable speed and accuracy, allowing it to create a detailed auditory map of its surroundings. This process is so efficient that bats can detect tiny insects in complete darkness and avoid obstacles with precision. Yong also highlights the diversity among bat species in their echolocation strategies, noting that different bats have evolved various adaptations to suit their ecological niches. Some bats have highly directional calls for pinpointing prey, while others use more omnidirectional calls to navigate cluttered environments like dense forests. These adaptations demonstrate the incredible versatility and effectiveness of echolocation as a sensory tool for survival.

In addition to bats, several other animals use echolocation to navigate and hunt, including dolphins, porpoises, whales (such as sperm and beluga whales), certain species of shrews, and swiftlets. One notable human who uses echolocation is Daniel Kish. Blind since infancy, Kish developed the ability to navigate his surroundings by making clicking sounds with his tongue and interpreting the echoes that bounce back from objects around him. This technique, known as human echolocation, allows him to "see" the world through sound, enabling him to walk, ride a bike, and engage in various activities independently. Kish has also founded the organization World Access for the Blind to teach echolocation to other visually impaired individuals.

LiDAR Sensor VS Echolocation

The relationship between LiDAR sensors and bat's echolocation lies in the fundamental principles of how they both detect and map their surroundings through the reflection of emitted signals.

Both LiDAR sensors and bats emit a signal that interacts with objects in the environment and then capture the reflected signal to gather information—LiDAR emits laser pulses, while bats emit ultrasonic sound waves. The time it takes for the signals to return provides data about the distance and shape of objects. Both technologies use the returned signals to create a map or image of their surroundings; LiDAR creates detailed 3D models of environments by calculating the distance between the sensor and objects, while bats interpret the time delay and intensity of the echoes to construct a mental map of their surroundings, aiding in navigation and hunting in the dark. Both methods allow for high accuracy and precision in detecting objects and their positions, with LiDAR measuring distances with great accuracy and producing high-resolution 3D models, and bats detecting tiny insects and avoiding obstacles in complete darkness.

EchoVision leverages this similarity by using sound visualization through LiDAR sensor’s reconstruction of the environment to simulate the echolocation experience, drawing a parallel between human-made technology and natural biological processes.

The Contemporary Austin - Host: Fusebox Program

We premiered EchoVision at The Contemporary Austin as part of the Fusebox Festival. The Congress Avenue Bridge in Austin, Texas, famously hosts the largest urban bat colony in the world, with roughly 1.5 million Mexican free-tailed bats residing beneath it every summer. Our performance started under the bat bridge with our collaborator, the bat conservationist Merlin Tuttle, who saved the 1.5 million bats under the Austin Bridge from extermination. About 500 people came. They listened to Merlin’s captivating stories with bats, watched the huge number of bats flying out of the bat bridge after sunset, and experienced EchoVision to be like a bat and to have a rare glimpse into the bat's perceptual universe.

Then, the adventure progressed with a walk straight down Congress Avenue to The Contemporary Austin Museum, where Jiabao Li and Matt McCorkle performed a live bat-inspired soundscape event called "Nocturnal Fugue." Nocturnal Fugue uses bat vocalizations—from social calls to mating rituals—to artistically create a mesmerizing sonic experience. This soundscape is combined with visual projections that represent the environment in which the specific bat call is being made or in response to. At the rooftop, we served Tequila. Bats are agave pollinators. They are part of the key process that turns agave flowers into tequila.

Photo credit: Merlin Tuttle's Bat Conservation (https://www.merlintuttle.org/)

Download App

EchoVision on Apple Store for iOS

EchoVision on Apple Store for visionOS

Publications

Botao Amber Hu, Jiabao Li*, Danlin Huang, Jianan Johanna Liu, Xiaobo Aaron Hu, and Yilan Elan Tao. 2024. "Becoming Bat with EchoVision: Experiencing More-than-human Perception through Mixed Reality". In Proceeding of SIGGRAPH Asia 2024 (SA '24). Art Paper. [→]

Botao Amber Hu, Jiabao Li*, Danlin Huang, Jianan Johanna Liu, Xiaobo Aaron Hu, and Yilan Elan Tao. 2024. "EchoVision: Experiencing Bat Echolocation via Mixed Reality". In Proceeding of SIGGRAPH Asia 2024 (SA '24). Extended Reality. [→]

Jiabao Li, Matt McCorkle, and Botao Amber Hu. 2024. "Becoming Bat". In Proceedings of the Halfway to the Future Symposium 2024 (HTTF '24). Pictorial. [→]

Botao Amber Hu*, Jiabao Li*, Danlin Huang, Jianan Johanna Liu, Xiaobo Aaron Hu, and Yilan Elan Tao. 2024. "EchoVision". 2024 IEEE VIS Arts Program (VISAP). Art Exhibition. [To appear]

Botao Amber Hu*, Jiabao Li*, Danlin Huang, Jianan Johanna Liu, Xiaobo Aaron Hu, and Yilan Elan Tao. 2024. "EchoVision: A Handheld Mixed Reality Mask for Experiencing Bat Echolocation". In Adjunct Proceedings of the 2024 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2024 ACM International Symposium on Wearable Computing (UbiComp/ISWC '24 Adjunct). Design Exhibition.[→]

Exhibitions

SIGGRAPH Asia. 2024. XR. "EchoVision". Tokyo, Japan

West Bund Art Festival. 2024. "EchoVision". Shanghai, China.

TANK Art Festival. 2024. "EchoVision". Shanghai, China.

International Symposium on Wearable Computers (Ubicomp-ISWC), Design Exhibition. 2024. "EchoVision". Melbourne, Australia.

Vancouver International Film Festival. 2024. "Nocturnal Fugue". Vancouver, Canada. [→].

Ars Electronica. 2024. "Nocturnal Fugue - Becoming Bat with EchoVision". Linz, Austria. [→].

Sheffield DocFest. 2024. "Nocturnal Fugue". Sheffield, UK. [→].

Omotesando interactivité. 2024. "Nocturnal Fugue". Tokyo, Japan. [→]

Host: Fusebox Program, The Contemporary Austin. "Nocturnal Fugue". Austin, US. [→]

@inproceedings{10.1145/3675094.3681951,author = {Hu, Botao Amber and Li, Jiabao and Huang, Danlin and Liu, Jianan Johanna and Hu, Xiaobo Aaron and Tao, Yilan Elan},title = {EchoVision: A Handheld Mixed Reality Mask for Experiencing Bat Echolocation},year = {2024},isbn = {9798400710582},publisher = {Association for Computing Machinery},address = {New York, NY, USA},url = {https://doi.org/10.1145/3675094.3681951},doi = {10.1145/3675094.3681951},abstract = {What is it like to be a bat? "EchoVision" is a mixed reality interactive experience that immerses participants in the world of bats through sound visualization and mixed reality technology. With a custom-designed, handheld, bat-shaped, optical see-through mixed reality mask based on the open-source HoloKit project, participants can simulate echolocation, the natural navigation method bats use in the dark. Participants use their voices and interpret the returned echoes. The visual feedback is dynamically adjusted based on the user's voice pitch and tone, as well as the 3D shape of the surrounding environment detected by LiDAR on the mask, offering an interactive and dynamic depiction of how bats perceive their surroundings. Compared to other mixed reality headsets, the pop-up exhibition in the bat habitat demonstrates that this unique mask design can effectively accommodate a large volume of audiences in-the-wild, offering them scientific education with empathetic engagement. This work promotes an ecocentric design perspective and fosters understanding between species, educating and cultivating a deeper appreciation for the unique ways non-human creatures engage with their ecosystems.},booktitle = {Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing},pages = {355–359},numpages = {5},keywords = {augmented reality, echolocation visualization, ecocentric design, handheld mixed reality mask, non-anthropocentric design},location = {Melbourne VIC, Australia},series = {UbiComp '24}}