GRADUATE SCHOOL · Mixed reality


Splatroit is part of the Detro Tune Exhibiton. The Detro Tune exhibition is conceptualized as an audio-visual virtual voyage through music history, subcultural anatomy, and authentic Detroit city identity. To achieve such a unique experience, we expose the audience to multisensory stimuli powered by various media technologies such as virtual and mixed reality. This eternal Detroit placed in a timeless virtual space aims to immerse us into cultural habitus, which shaped a fruitful era of fresh and influential ideas and innovations.

Splatroit is an immersive mixed-reality game that intertwines Detroit's iconic techno music with its cityscape. The heart of this Mixed experience lies in the Monument to Joe Louis, 'The Fist,’ Which is Detroit's most popular symbol and location. Players can color and interact with 'The Fist' while enjoying Detroit's techno music. This symbolizes Detroit and the players becoming one. In the space surrounding the player, colorful balls and balloons envelop them. Within this setup, the player is immersed in a virtual environment of Detroit, where 'The Fist' is located. Outside this virtual realm, they can interact with the balls and balloons, experiencing a blend of mixed reality. As players engage with the pulsating techno rhythms, they dynamically color this emblematic landmark, creating a visual spectacle synchronized with the soundtrack. Splatroit embodies Detroit's spirit, allowing players to interact with its music and art.

Splatroit is featured at the upcoming Detroit Month of Design and Ars Electronica events.

- 70% of Prototyping with Unity, C#
- 50% of Research and Ideation
- Unity
- C#
- Hsieh Avery
- Cheng Boyun
- Mai Nguyen
- JAN 2023 ~ APRIL 2023

About My Role
Our team consisted of four members and collaborated well from the initial research to the testing. My particular role was prototyping using Unity and C#, and after launching this project at Ars Electronica and Detroit Month of Design, I mainly handled updating it with user feedback.

Splatroit: Detroit & Techno Music

Splatroit at Ars Electronica

Splatroit at Detroit Month of Design


When we worked on the Splatroit, we followed the below design process from defining project goal to launching.


The goal of the UX DESIGN STUDIO II course was 1) to represent Detroit and its techno music in virtual reality using Unity and 2) to showcase Splatroit at 'Ars Electronica' in Linz, Austria.

* 'Ars Electronica' is an international platform that explores and celebrates electronic art and technology. This event, which started in Linz, Austria, in 1979, continues to be held annually, gathering people from various fields, such as artists, scientists, engineers, scholars, and developers, to explore the intersections of contemporary society, technology, and art.


After recognizing our goal, we analyzed more than 20 virtual reality-related contents and discussed with our team members to decide on how to express Detroit's techno music and culture in virtual reality. The concept revolves around the user interacting with Detroit while listening to Detroit techno music. To explain a bit more specifically, various colored balls appear in sync with Detroit's techno music, and users hit those balls to color symbols of Detroit

[Concept sketch for the ideation]


In this phase, we determined through discussion what to use as symbols of Detroit, we decided to use the Monument to Joe Louis , 'The Fist'(Link) which is a famous and iconic symbol in Detroit, how the balls will be changed according to the music, what effects will occur when users hit the balls, how the map in which the user plays the game will be designed, how many balls will be used, and how the buildings will be designed, among other considerations.

After deciding on this, we created a storyboard to visualize our ideas and shared it with the team members. The storyboard greatly assisted in facilitating understanding among team members and in clarifying the direction of the design and the user experience.

1. In a white space with nothing, there is a white target object (a statue in the shape of a fist).

2. Techno music starts playing, and paintballs (red, blue, green) start rotating around the player, forming a circle.

3. The size of the paintballs increases and decreases according to the beat of the music.

4. When the player hits a paintball with a fist, the paintball flies in the direction in which it was hit.


We created an immersive virtual environment where users can engage with our content, using Unity and C#.

1. This is the first prototype. We materialized the storyboard from the initial ideate phase using Unity and C#.

2. For the Second prototype, We designed the environment around the monument to Joe Louis to resemble a more urban setting.

3. And in the third prototype, We expanded the originally confined space and brightened the atmosphere of the Scene.

4. In the fourth prototype, effects that synchronize with the light and music were introduced, aiming to elevate user immersion in the virtual space.

feedback loop
after launching

The process went smoothly up until ideation. However, after launching the project at Ars Electronica and the Detroit Month of Design and receiving feedback from users, we experienced a rough process. But this phase provided us with valuable opportunities for learning and growth in the project development process. This is because we updated and refined Splatroit based on user behavior patterns observed and direct feedback from users. Throughout this process, we repeatedly went through a cyclical process including ideation, prototyping, and testing, which was an opportunity to adjust our work to the real world through the precious opinions of users.

Users often asked me, 'Is there more?' and 'Is this the end?' while playing Splatroit, which hinted that achieving goals in Splatroit felt too easy for them. Therefore, I thought that increasing the difficulty of Splatroit and making the ball-grabbing action more enjoyable could be a solution. Consequently, I modified the code so the ball bounces when it hits the floor or buildings.

Previously, users sometimes played while standing in one place. The reason this is problematic is that Splatroit's intention was for users to enjoy Splatroit more by wandering around Spltroit and interacting with balloons and balls in reality, imprinting Detroit and Detroit techno music. I wanted to have the user move around and interact with the balls and balloons in the real world. So I added arrows to the bottom of the Splatroit scene to encourage user action to move around the space and interact with balls and balloons. As a result, the rate of users interacting with balls and balloons while moving around Splatroit increased by 60% compared to before, and users who enjoyed this process showed an increase in positive evaluations after experiencing Splatroit.

Originally, users were able to grab and throw the ball, but this was not intuitive and caused confusion. Instead, I modified it so that users can punch as soon as they grab the ball.

Despite initial explanations, some users were unclear on how to play Splatroit.  Constantly asking how to play Splatroit can not only diminish players' interest but also exhaust those guiding the game. In the long run, this can lead to poor service and give a negative impression of Splatroit.. To mitigate this, I embedded a play guide directly within the Splatroit Scene to facilitate self-guided learning during gameplay. As a result, the frequency of user questions decreased by more than 80% compared to before.




In Splatroit, when the ball flies too high in the sky, users cannot hit it. To prevent this, I modified the code to constrain the ball's flight within the bounds of the buildings.

mixed reality

The goal of Splatroit was to provide a more immersive experience by combining virtual reality and real world. I structured real-world space with numerous balls and balloons to deliver mixed reality. Users can enjoy both the balls and balloons in real world while enjoying Splatroit in the virtual space.


At Ars Electronica and Detroit Month of Design, Splatroit engaged more than 300 visitors in an immersive mixed-reality experience, seamlessly integrating music, art, and technology and providing a unique user experience.

different user experiences
between the ars electronica
and detroit month of design

How to implement mixed reality?: Employing actual balloons and balls to immerse users into Splatroit was a significant point of consideration. The spaces allocated to us at Ars Electronica and Detroit Month of Design had different sizes and conditions. Deciding on the placement of the balloons and balls while considering user flow in these conditions was challenging.

I aimed to allow Splatroit users to fully experience mixed reality by maximizing the advantages of different environments and compensating for their disadvantages. 1) The room at Ars Electronica was narrow but had a low ceiling, enabling balloon installations. The confined space limited users' mobility but allowed them to immerse more deeply into Splatroit, engaging with the floor and the air. However, 2) the space at Detroit Month of Design was broad but had too high a ceiling for balloon installations. Users could thoroughly enjoy their pathways while experiencing Splatroit, yet couldn’t interact with balloons as they could at Ars Electronica. The varying features of these two locations created similar but distinctly different user experiences.

[Ars Electronica]

[Detroit Month of Design]

Next step

Currently, Splatroit features only the 'Monument to Joe Louis'. However, I want to create a more immersive experience for users to delve into Detroit and its techno music by adding various symbols representing Detroit, such as 'The Spirit of Detroit,' and incorporating a wider variety of Detroit's techno music than is available now.

What I've learned

1) I have acquired the ability to create a 3D environment using Unity and C#. I now have the ability to test users, gather feedback, and create projects in 3D environments as well as 2D environments like screens.

2) I realized that the initial prototype might not always be perfect. However, I learned that by quickly addressing failures and refining the solution based on user feedback, we can achieve a better outcome for the project.

3) Throughout the project, I considered a lot about how to convey the Splatroit through visuals, auditory, and tactile experiences and amplify its joy and value to people. This experience allowed me to go beyond screen-based user experiences, learning ways to create connections with users in environments where reality and virtuality converge. As a UX designer, this was an excellent opportunity to better understand user experiences and interactions.