City of Sparkles

"City of Sparkles" is an interactive virtual reality experience that immerses participants in an AI's perspective, visualizing a city of human memory fragments through spatialized Twitter data.

Overview

We envision blockchain evolving into an unstoppable "nature," a potential substrate for self-sovereign, self-sustaining AI-based artificial life forms. We're intrigued by these godlike AIs inhabiting the "Mnemosyne Sea"—a vast, infinite expanse of human memory fragments. These data shape the AI's being. We ask:

How might an AI perceive our city?

To better convey the sense of loneliness resulting from the significant asymmetry between human and artificial life, we created an immersive virtual reality piece "City of Sparkles" on Apple Vision Pro. Within this experience, the audience can embody our AI protagonist, Zoe, who wanders, reads, and collects scattered fragments of human memories. This provides an experiential understanding of how artificial life perceives the myriad memory fragments produced by human society. To construct this data landscape, we accumulated all geographically-tagged Twitter data over four years, resulting in an interactive visualization of vast human memories. This approach enables us to use non-fiction data to tell a fictional story from a reversed non-anthropocentric perspective.

In this experience, a virtual artistic representation of New York City is dynamically shaped with sparkles sampled from real photogrammetry data of the actual city. Each sparkle represents a real Twitter message from a New York City resident, with its color and motion reflecting the emotional mood and geographical location of the message. Audiences can freely fly in the virtual city, discover the messages left within, and sense the solitary souls. The experience is divided into several chapters, each with its own storyline and emotional mood. Audiences can interact with the sparkles and feel the emotions of the messages. In accordance with the emotion, abstract visual effects and motion occur, accompanied by adaptive contemporary music and sound, to enhance the audience's experience.

"City of Sparkles" is originally a data visualization artwork made on HTC Vive, featured at the SIGGRAPH 2019 Immersive Pavilion. This year, we've recreated the entire experience with Vision Pro, incorporating new bare-hand flying interaction, as the visualization part of author Botao 'Amber' Hu's latest science fiction, "Composable Life: On the Suicide of Zoe" (https://reality.design/writing/on-the-suicide-of-zoe). This science fiction aims to envision the first suicide event of a speculative AI-based artificial life form, inhabiting the vast human-generated data. "City of Sparkles" enables us to use non-fiction data to tell a fictional story from a reversed non-anthropocentric perspective.

Publications

Botao Hu*, Yang Liu, and Ran Duan. 2019. "City of Sparkles." In ACM SIGGRAPH 2019 Virtual, Augmented, and Mixed Reality (SIGGRAPH ’19), Immersive Pavilion [→]

Exhibitions

SIGGRAPH 2019 Immersive Pavilion. "City Of Sparkles". Los Angeles, U.S..

@inproceedings{10.1145/3306449.3328815,author = {Hu, Botao and Liu, Yang and Duan, Ran},title = {City of sparkles},year = {2019},isbn = {9781450363204},publisher = {Association for Computing Machinery},address = {New York, NY, USA},url = {https://doi.org/10.1145/3306449.3328815},doi = {10.1145/3306449.3328815},abstract = {Through using authentic Twitter messages, City of Sparkles portrays storytelling through a virtual reality experience in New York City and San Francisco. The audiences can feel different emotions through exploring through the world of tweets.},booktitle = {ACM SIGGRAPH 2019 Virtual, Augmented, and Mixed Reality},articleno = {9},numpages = {1},location = {Los Angeles, California},series = {SIGGRAPH '19}}