Somaesthetic Realities
Course Description
XR Soma Design is an interdisciplinary course that explores the convergence of extended reality (XR) technologies with embodied cognition. In this course, the body is viewed as the essential medium through which consciousness engages with the world, affirming that our understanding of time, space, and sensory experience is fundamentally rooted in our physicality. XR serves as a “glue medium” that seamlessly fuses reality with human experience, thereby expanding the interfaces between our bodies and the surrounding world.
Through hands-on experimentation with ambiguous interface design, states of cognitive dissonance, and improvisational creative practices, students will work with biofeedback sensors, wearable systems, and interactive media installations to push the boundaries of perception and technology. This exploration encourages a reimagining of sensory experiences and the deepening of emotional resonance, while critically engaging with contemporary debates in transhumanism, posthumanism, and technological entanglement.
By questioning whether our increasing reliance on language-based intelligence might render us as mere “ghost shells” or lead us to become integral parts of complex “entangled assemblages” with technology, the course invites students to rethink the intricate relationships among body, mind, and digital mediation.
Week 1: Introduction
- Day 1: Introduction to XR Soma Design.
- What's XR?
- Fiction vs Reality.
- Reality Continuum.
- What's Soma?
- Body vs Mind.
- Dualism.
- What's XR Soma Design?
- Case Studies.
- What's XR?
- Day 2: What's Soma Design? (Course overview, project expectations, tool demonstrations)
- Assignment**:
- Lecture: Somaesthetics and Soma design (Theory and case studies)
- Assignment: Read and respond to Designing with the Body: Somaesthetic Interaction Design by Kristina Höök
- Tool Introduction:
- Soma Bit, Body Map.
- Day 3: Lecture: What is Mixed Reality Performance?
- Read and respond to Performing Mixed Reality
- Tool Introduction:
- HoloKit.
Week 2: Introduction
- Day 1:
- What's Concept?
- Why Design?
- Progressional Design vs Frictional Design .
- Concept is the king.
- Tension is the core.
- Good concept vs Bad concept.
- Aesthetics vs Ethics.
- What's Work?
- Artifact vs Paper.
- Exhibition vs Conference.
- What's SIGGRAPH XR?
- What's Ars Electronica?
- What's Concept?
- Day 2: How to Prototype? Day 3: Ambiguity & Uncomfortable Interactions
Day 2: Lecture: What's Soma Design? (Course overview, project expectations, tool demonstrations)
Assignment**:
Lecture: Somaesthetics and Soma design (Theory and case studies)
Assignment: Read and respond to Designing with the Body: Somaesthetic Interaction Design by Kristina Höök
Day 3: Lecture: What's Soma Design? XR Case Studies.
- Find
- Play Beat Saber, Super Nature.
Week 2: Ambiguity & Uncomfortable Interactions
- Review Case Studies.
- Reading.
- Activities: Group discussion, brainstorm XR interventions for “uncomfortable” moments
- Assignment: Read and respond to Ambiguity as a Resource for Design and Uncomfortable Interactions by Steve Benford et al. Prepare a design concept for Doing, Feeling, and Naming.
Week 4: Concept Development
- Activities: Playing Doing, Feeling, and Naming (Game) and engaging in paper prototyping
- Assignment: Submit a project proposal
Week 5: Developing on HoloKit
- Activities: Technical tutorials and a “hello world” prototype
- Assignment: Develop a small interactive prototype
Week 6: Multi-modal feedback in XR
- Activities: Technical tutorials and development of a prototype with at least one feedback input (e.g., biofeedback, acoustic feedback, haptics, or thermal feedback)
- Assignment: Create a small interactive prototype with feedback input
Week 7: Developing collocated multiplayer on HoloKit
- Activities: Technical tutorials and creation of a “hello world” multiplayer prototype
- Assignment: Develop a multiplayer interactive prototype
Week 8: Intermediate Prototype & Critique
- Activities: Prototype demonstrations and small-group critique sessions
Week 9: Workshop
- Lecture: Exploring avenues for academic conferences and exhibitions
Week 10: Academic Writing for Exhibition and paper
- Lecture: Academic writing techniques for exhibitions and publications
- Activities: Project check-ins, debugging, and performance improvements
- Assignment: Read and respond to Strong Concepts: Intermediate-Level Knowledge in Interaction Design Research by Kristina Höök et al.
Week 11: Refinement & Final Production
- Activities: Final project refinement, debugging, and polishing
Week 12: Final Presentations & Critique
- Activities: Public or in-class presentations with external reviewers
- Assignment: Write a paper detailing the project
XRSoma examines how extended reality (XR), multi-modal feedback, and somaesthetic approaches reshape our experiences of self, others, and media. Students will explore a mix of phenomenology, design research, HCI, and media theory, developing projects that embrace ambiguity, invite interpretation, and provoke new emotional or bodily insights.
In an age where technology seamlessly entangles with human existence, our traditional notions of mind and body are ripe for reimagination. Mind and body are not separate entities but co-constitutive partners—together forming what we call the “soma.” Equally, robotics and artificial intelligence are converging, signaling a shift toward “somabolitics,” in which human embodiment and machine intelligence fuse to expand our potential.
As our reality becomes increasingly mediated—by AI shaping our thoughts, and wearable devices shaping our physical experiences—our sense of self evolves into a fluid assemblage of organic and inorganic elements. We are no longer singular beings but interconnected nodes that actively entangle with others, technology, and the environment. Extended Reality (XR), as the ultimate wearable medium, mediates what we see and hear, pushing us toward an era of deeper integration—one step before neural implants.
XRSoma stands at this threshold, urging us to reconsider who we are when our senses and cognition are augmented, and how somaesthetic principles might guide our collective evolution. Through artistic exploration and design practice, XRSoma seeks to embrace ambiguity, invite improvisation, and foster defamiliarization—challenging how we perceive and perform within these new techno-human ecologies. My work aims to translate these insights into dynamic, human-centered experiences in play, performance, well-being, education, and cultural heritage. Together, we will envision the future of an entangled assemblage of mind, body, AI, and robotics—dissolving boundaries and forging novel pathways for creativity, connection, and somaesthetic becoming.
Course Objectives
- Understand the art foundation of somaesthetics in the context of XR and principle of artistic research in HCI.
- Gain practical skills in designing and implementing somaesthetics XR systems.
- Create innovative projects that contribute to the fields of media art, design, and human-computer interaction.
Teaching Team
- Botao Amber Hu
- Rem Rungu Lin
Course Dates
- Mar 8 - May 5, 2025
Syllabus
Week 1: Introduction
- Lecture: What is XRSoma? (Course overview, project expectations, tool demonstrations)
Week 2: Somaesthetics and Soma design
- Lecture: Somaesthetics and Soma design (Theory and case studies)
- Assignment: Read and respond to Designing with the Body: Somaesthetic Interaction Design by Kristina Höök
Week 3: Ambiguity & Uncomfortable Interactions
- Activities: Group discussion, brainstorm XR interventions for “uncomfortable” moments
- Assignment: Read and respond to Ambiguity as a Resource for Design and Uncomfortable Interactions by Steve Benford et al. Prepare a design concept for Doing, Feeling, and Naming.
Week 4: Concept Development
- Activities: Playing Doing, Feeling, and Naming (Game) and engaging in paper prototyping
- Assignment: Submit a project proposal
Week 5: Developing on HoloKit
- Activities: Technical tutorials and a “hello world” prototype
- Assignment: Develop a small interactive prototype
Week 6: Multi-modal feedback in XR
- Activities: Technical tutorials and development of a prototype with at least one feedback input (e.g., biofeedback, acoustic feedback, haptics, or thermal feedback)
- Assignment: Create a small interactive prototype with feedback input
Week 7: Developing collocated multiplayer on HoloKit
- Activities: Technical tutorials and creation of a “hello world” multiplayer prototype
- Assignment: Develop a multiplayer interactive prototype
Week 8: Intermediate Prototype & Critique
- Activities: Prototype demonstrations and small-group critique sessions
Week 9: Workshop
- Lecture: Exploring avenues for academic conferences and exhibitions
Week 10: Academic Writing for Exhibition and paper
- Lecture: Academic writing techniques for exhibitions and publications
- Activities: Project check-ins, debugging, and performance improvements
- Assignment: Read and respond to Strong Concepts: Intermediate-Level Knowledge in Interaction Design Research by Kristina Höök et al.
Week 11: Refinement & Final Production
- Activities: Final project refinement, debugging, and polishing
Week 12: Final Presentations & Critique
- Activities: Public or in-class presentations with external reviewers
- Assignment: Write a paper detailing the project
Assessment Overview
- Participation & Engagement: 20%
- Short Reflections & Prototypes: 10%
- Final Project & Presentation: 40%
- Paper: 30%
Reading
Kristina Höök, Designing with the Body: Somaesthetic Interaction Design, https://direct.mit.edu/books/monograph/4131/Designing-with-the-BodySomaesthetic-Interaction