Design New Realities (CAA Fall 2023)
Design New Realities
"Design New Realities" is an innovative and practical course offered at the China Academy of Art. This course aims to equip students with the necessary knowledge and skills to create immersive stereoscopic mixed reality experiences using the iPhone and HoloKit. HoloKit team has been invited by the China Academy of Art to deliver a course centered around the utilization of HoloKit. By utilizing HoloKit as an alternative to expensive equipment like HoloLens, students are provided with an affordable and accessible solution for rapidly prototyping their mixed reality ideas. Through a combination of theoretical instruction, hands-on exercises, and practical tactics, students will develop the ability to design captivating mixed reality concepts, explore the intricacies of the design process, optimize workflows for efficient prototyping, and proficiently document their works.
Venue
School of Design and Innovation, China Academy of Art
Room B4-140
Tuesday, Wednesday, Thursdays from 1pm-4pm
Course Objectives
- Understand the fundamentals of mixed reality and its applications.
- Explore the elements that make a mixed reality concept successful.
- Learn practical tactics and best practices in designing mixed reality experiences.
- Utilize paper prototyping and speculative design to conceptualize mixed reality ideas.
- Develop an optimized workflow for quick prototyping using the iPhone and HoloKit.
- Acquire skills in documenting and presenting mixed reality projects effectively.
Teaching Team
- Main Lecturers: Tongzhou Yu, Yongru Lu, Botao Amber Hu
- Teaching Assistants: Yucheng Zhang, Sizheng Hao
Syllabus
Week 1: Introduction
- Lecture: What's spatial computing?
Week 2: Presence and Co-Presence
Lecture: Presence and Co-Presence
- Video: https://youtu.be/QokUhb8gqQs
Assignment: Paper prototypes for your first AR design.
Week 3: Developing on HoloKit
- Get familiar with development environments: Mac, iPhone, Xcode, Unity
- Learn the basic of Unity, C#, ShaderGraph and Visual Effect Graph
- Resources:
- Github repo: HoloKit Unity SDK
Week 4: Developing collocated multiplayer on HoloKit
- Learn how to develop a collocated multiplayer mixed reality
- Resources:
Week 3 - 4: Workshop
Topics:
- HoloLive
- HoloLife
- HoloLove
Week 5:
Lecture: What is a good concept for mixed reality?
Resources:
Mixed reality creation workflow
Bibcam Workflow
In this course, we offer an enhanced workflow for working on Augmented Reality (AR) using Bibcam. This new approach differs from the traditional AR creation process, which typically involves developing AR content in Unity, compiling it for iPhone, and then capturing it on location. In the traditional method, if the visual outcome is unsatisfactory, the iteration process needs to be restarted, leading to significant time consumption.
In contrast, our new workflow streamlines the process. It begins by shooting the scene using Bibcam format, which encompasses the camera video stream, ARKit tracking information, depth graph, and human stencil data, all consolidated into a single frame. Subsequently, the AR work is created and the visual aspects are iterated and refined in Unity without compiling into phones, based on the composition of the video background and AR objects. This approach allows creators to envision and pre-visualize the AR content more effectively before compiling it onto the phone, resulting in a more efficient and intuitive workflow.