Muse
A tech-based wearable system that empowers Hearing AIDS users and individuals to appreciate sound in a creative aesthetics
Project Background
Duration 2022.3-2022.5 Restart 2023.9-2023.11
My Role UX Resaercher, Product Designer
Preview
we “stop treating people with different abilities as members of an out-group, and we start treating them as part of our own diverse in-group.”
When we treat all of our users as whole people, regardless of their abilities, then we are able to approach accessibility as just another solvable—valuable—technical challenge to overcome.”
This independent project was undertaken with the aim of expanding the accessibility of interactive and multi-sensory hearing experiences, particularly for individuals utilizing hearing aids. By employing new aesthetics and user-friendly interactions via gloves, the goal was to enable people to participate and share their diverse experiences.
Problem
1.Sharing music is challenging to the people with different music taste
As a devoted pop-music enthusiast, I've often faced rejection when trying to introduce my fav music to others, who frequently deem it "too strong". These recurring disappointments got me to think is there a way for individuals with different musical tastes to share their passions and, find some empathy beyond just listening?
2.Music can be a noisy to the people with hearing AIDS
When I tried to share my headphones with a friend who wears hearing AIDS, I was rejected again. But not for the same reasons above. She explained that to her ears, all music appeared as noise and "excessively loud". She struggles to understand the meaning of different rhythms, but remains puzzled by our fascination with music. It was at that moment that I realized that there are some people who don't understand music at all.
Research And Finding
Lack of music education and the sound distortion of the hearing AIDs can lead the music to the noise
Why some people with hearing AIDS can't enjoy the music?
Therefore, I interviewed a doctor in the Otolaryngology, who is expertise in deafness, tinnitus and vertigo, and asked her about the potential reasons.
First, the lack of music education from childhood.
"Many individuals who use hearing aids lost their hearing in childhood and missed out on early music education. As they grow up and regain some hearing ability, understanding music remains a challenge. While doctors suggest music education, like piano or violin, to improve sound perception, not everyone can afford these specialized lessons."
Second, the sound distortion of the hearing AIDS.
In simple terms, a microphone captures the sound, an amplifier makes it louder, and a receiver (similar to a speaker) outputs the amplified sound to the wearer's ears.
Research And Finding
People with hearing AIDs want to feel the music like normal people
After learning this information, I interviewed 9 participants(including normal people, people with hearing AIDS/cochlear implants, family members and friends) to gain an understanding of how they regard sound and process these sounds.
Meanwhile, to better understand the hearing AIDS users wants and needs. I also conducted a more detailed interview of Wendy, a girl who received cochlear implants and hearing AIDS, to find out how she deal with different sounds in her life.
From these study, I still see that many people want to listen to and feel music, and they also find their own identity in music, trying to accept all parts of the world.
Therefore, I raised the problem
“How Might We help people to share or understand music in a different way?”
Mission
Using the hands as the tool to interact with music visually
Based on the previous findings, I came up with the concept map below:
Therefore, the main features is allow users to use their touch and motions to interactive with the sound, and get different visualizations by differnt gestures.
Design
Multi-system Design
1 Test
I first using python to programming the interaction and visualization to test my idea.
Matplotlib and Pydub are used to analyze the sound input, OpenCV and color classification to identify the hand motions, based on those value, I use pillow to generate the circles.
And through this shape grammar, I’ve got different output results :
2 Interaction Flowchart
Therefore, based on the previous coding test logic, as well as to craft the best experience for all the users, I first design this ideal userflow as guidance for my prototypes:
Based on the flow chart, I first focus on the most complicated part, the wearable gloves, which can read the message of user’s hand and send those to the website.
The FSR and Bend sensors are used to collect data from different fingers, and will be transform into different elements in the final visualization.
4 Sound Analysis and Visualization Through Touch Designer
Before developing the patches, I first created a moodboard for the patterns. I want to using the bright color from nature, and organic and fluid shape.
Sound Analysis
Through audiofilter and audiospect, the sound is seperated into 2 frequency part, one is for the higher range, one is for the lower stage, and convert them into data.
Hand Motion Input Data Analysis
Using SerialIn to import the arduino input to the Touch Designer, and convert those message into math data. Through DATto, convert those numbers into 5 channels, corresponding to 5 fingers. Those numbers in the channels are managed in different math methods based on the final visual effects, for examples, multiply and reset range, and then assigned those managed numbers to different grammar windows.
Generative Art Grammar
I have designed 10 different scenes for music visualizations, and users can choose to switch between different scenes, and customize the details.
These are some connection sketches I worked on in touch designer.
5 Share Platform
I first used Bluetooth to send the input captured by the glove to the laptop.
Final Delivery
See the music, Interact with music, Share with other
Impact and Takeaways
we “stop treating people with different abilities as members of an out-group, and we start treating them as part of our own diverse in-group.”
When we treat all of our users as whole people, regardless of their abilities, then we are able to approach accessibility as just another solvable—valuable—technical challenge to overcome.”
Through empathetic design with accessibility, sustainability and social equity, I aim to foster communication and dialogue between different community.
