Implicit Space
Design Research, Mixed Reality Design
Tools: Unity, Video, Illustration, Installation, Interactive Prototype, Publications
Awards:
- Muse Creative Awards 2024,
- Vega Digital Awards 2024
How do designer extend social etiquette of body motion to augmented reality design?
Implicit Space uses virtual reality(VR) to explore private augmented reality(AR) interaction designcont in public. This project presents examples of AR content that meets social etiquette and uses body motions to categorize and visualize how other people recognize our space usage in public. And create a prototype of evaluation tool to help designer design for augmented reality base on proxemics research. This set of design tool and guidelines build courteous communication between individuals and helps avoid space conflict.
This is a one year individual design research. Learn more about the full version design and research process and editor picks article on UX Collective.
Create design tool and examples for augmented reality social etiquette
With three different simulation scenes and mocap characters, designer can easily bring their augmented reality design to this system for user test to see what kind of space impacts it has from designer view. In user view from VR headset, it provides real world public augmented reality simulation.
The demo also contains examples of how to design augmented reality interactions base on the intention that showed by user’s motion to meet social etiquette.
Video record from actual user test ➡︎
An instruction video for how others read our motion and how to use it for augmented reality
Through the research, 5 kinds of space should be considered differently when design interactions in augmented reality. The video provides not only visual instructions of how people’s motion be interpreted by others but also potential augmented reality design examples for those space.
Synthesize an augmented reality design guideline for better user experience via third-person perspective
Unlike the current design guideline that only cares about the user experience for the user, this hypothetical design guideline tells designer how to think about user experience for other people in public from third-person perspective. There is not only one design solution for courteous augmented reality design, but there is a common way to approach public-friendly design.
An integrated prototype installation for user test
Combining the VR prototype system, video instruction and guidebook for augmented reality, this all-in-one installation give designer a chance to get closer to this design future. With feedbacks from designers, research is moving forward for the next phase.
Research Process Overview (Click here for full research process)————————————————————————————————————————————————————————————————————————————
Entry point: we disturb others when we use AR — and vice versa
When you play an AR game in public like Pokemon Go or Minecraft World, if there are many people in the same place, you might block other’s views or be hit by other people when all of the people are so focus to interact with virtual objects via screen. In this case, AR creates a new conflict between people that never happen before. You disturb others when you use AR — and vice versa.
Space impacts from our interactions with virtual objects
We notice the space impacts are generated when we use our hands to interact with the digital world through the device. The impacts disturb other people potentially. But what do impacts look like and how to quantify them with proper visuals?
Synthesizing from an experimental prototype, this series of drawings indicate the various possibilities of space impacts created by different gestures (green geometry) comparing to the same augmented reality content (dot square). What is that mean to augmented reality? What if those space impacts do have physical effects.
Proxemics study — the social code behind our motions
The previous study is base on the same scenario: A single person in public space. But in reality, space impacts from our body motion are likely to be found in a social environment. What if there are two or more people?
To explore the hidden rule behind interpersonal communication, A performance simulation experiment was conducted in a dark room and use led light and infrared camera to examine how people behave under different interpersonal social situations. The insights that inspired me with the concept of implicit space between people.
Click here for the analysis sample.
Prototype in VR: how others learn our intentions from hand to the whole body motion
We know space impact comes more from other people’s understanding of our motions in public. The idea of using VR system to simulate AR environment came up to my mind. With the iterative design process, couple VR prototypes were created to visualize the understanding of our motions from the perspective of others.
From hand to the whole body, single people to a couple of people, simple algorithm to composite algorithm, those VR prototype were getting closer to real scenarios. Based on this system, new inspiration appeared in the next step.
Adaptive AR concepts based on implicit meaning of our motions
Inspired by feedbacks, I start to introduce implicit space as a dynamic stage for AR design. Like the screen as media for mobile app, body motions can imply different available space that can be the stages for different AR interactions or content.
To primarily verify the idea of adaptive AR design. GoPro and After Effects gave us a chance to prototype some potential scenarios about how AR content change when user’s motions provide indications. Notice that these motions are very common in daily life and not specially adapted for AR device, which show the possibilities of augmented reality that meet social etiquette.
Some high level findings from user test
The scope of this project evolve from a simple visualization tool to a set of tools and examples for designer as a guide for AR design. And here are some high level guidelines:
User controls their motions for different purposes and in different places with different personalities to maintain social etiquette. So do others.
User’s motions give others expectations that how the user will use space potentially. So do others.
User’s motions can also tell the AR system the expectation that how it should adapt its content to meet social etiquette.
Current design tools and design guide don’t provide guidance for AR designer to build an adaptive AR app that meets social etiquette.
A simulated environment and crowd for AR
To test AR design in VR system, high fidelity real-world environments are must-have. Not only that, for testing AR in public, mocap characters are also brought into the system. Designers can load their AR design to those characters and user to see how the design respond to behaviors
Design tool for AR designer
Further design on this prototype, I used virtual reality (VR) to help AR designers explore private augmented reality (AR) interactions in public.
User can test AR app in a real-world simulated-scene by using VR headset. They can only see their AR content just like how it is in the real world. There are three different scenes in this prototype with pre-recorded mocap characters.
For designer, the designer view provides a “god view” in which designer can see everyone’s AR content in the test. Also, the designer view provides the visualization of implicit space that main user claim, indicate, own and occupy in real-time. And how their adaptive AR design performs for individuals as well as for public social etiquette.
Design examples for AR designer
Building on the concept, we can design a system for AR app to treat different implicit space in different ways:
Claim Space
User’s limb motions including hand gestures show the temporary space that they can reach further and use in public. For AR app, the display layer can adjust their layout to follow the changes of claim space. In this prototype, claim space is showed as grey cube.
Indicate Space
User’s trunk dynamic, for example, how they lean and walk toward, show the space that they are about to pass or arrive in public. For AR app, content with lower interactivity like notification can respond to this space, which provides a brief and no disturbing interaction. In this prototype, indicate space is shown as blue cube.
Occupy Space
User’s stay has the power to show their space occupation to some extent, the longer they stay the stronger their space occupation is in public. For AR app, content with higher interactivity like the function layer should stay in this space. The display layer can increase their visibility and interactivity to take full advantage of this occupation. In this prototype, indicate space is shown as blue and yellow sphere.
Own Space
User’s ownership of a space doesn’t reflect by their body motion but a consensus from common sense or regulation. Or even just a monopoly space that pre-defined by user. For AR app, it just likes how we use space for AR content today, none of the social etiquettes need to be followed. In this prototype, own space is shown as white dots.
Iterative design process
After rounds of user tests, I turn the feedback into the next round of iteration. Most of the feedbacks point out two things:
1. visual effects are unclear and don’t show their unique properties of different space
2. In the future of AR era, regulation of space usage can be common which is missing in my implicit space category
In this iteration I adjust the category of the implicit space, add one more space “Define Space” to present the space that is behavior are defined by authority or convention. New logos are designed for print material and introduction video. There is also an update for the prototype with more representative visual effects and a new algorithm.
Develop instruction video and prototype installation
To sum up the explorations in this project in a more deliverable way, I also make a hypothetical AR design guideline and instruction video for designer, along with a prototype installation that contain live demo of the AR design tool and examples.
Research round up and next step
Spatial computing uses real-world space as their “screen”. But real-world space is a public resource for not only AR content but also physical objects as well as real people. This project assumes people will follow the rule of the real world: morality, etiquette, and social code. But still, when immersive technology needs people to occupy space that everyone shares, technology has the power to guide people to do better. Instead of learn from world, new design can guide the world. After all the process of exploring, observing, prototyping and iterating, this project trying to tell designer to consider user experience from third-person view for a bigger scope and jump out of the small scope that only comes from the user’s first-person view. In the era of immersive technology, user experience is not only for the user who uses your app but also for the people around your user.
