Published on | Prototyping: From UX to Front End — Medium Aarushi Abraham

Mentor | Craig Sampson

Music is an outlet, an escape, a universal language. It is an art form that tells stories, stirs emotions and connects people. I grew up surrounded by people who taught me to love and appreciate music.

For my thesis project as a master’s student in human-centered design at Northwestern University, I decided to focus on musical performance and music curation. It was partly an excuse for me to spend time exploring a space I love and am familiar with, but more importantly, a place where I saw an opportunity to create positive change.

Audio Editing and Equalization

Some tracks sound quiet and dreamy. Some might sound creepy and dark. We often associate adjectives like funky, shrill, calm, jarring, etc, to describe what we hear. Once music is recorded, part of the audio editing process involves adjusting the volume of specific frequencies within each track. This is called equalization (EQ) and changing the EQ curve alters how individuals interpret the music. Any standard audio production software has an equalizer built in.

Equalizer in Audacity (source: google images)

The Problem

Integration of technology with music has led to the development of Digital Audio Workstations (DAW’s), which have greatly influenced how artists dream and bring their work to life. All this exciting software relies heavily on visual feedback. Where does this leave someone who can’t see?

The mouse is inaccessible to someone who is completely blind because they can’t see the pointer on the screen. Blind people use computers and technology with the assistance of screen readers like NVDA or JAWS. Screen readers are software programs that read the text that is displayed on the computer screen with a speech synthesizer or braille display. Here is a clip of what that interaction is like.

What challenges a screen reader?

Anything that is not TEXT.

How is a screen reader going to read an image?

A graph?

An Equalizer?

Screenshot of the EQ mixer interface on the Logic Pro X (source: google images)

Design Challenge

The secondary research and user interviews done helped me frame the challenge as follows:

How might we design a Tangible Equalizer that provides visually impaired musicians an intuitive way to understand and manipulate the Equalization curve on a standard DAW?

To approach the problem, I followed a human-centered design process.

Design Research

Once I identified potential stakeholders, I started reaching out to them. Ethnographic research gives me a better understanding of what it is like for visually impaired musicians to interact with DAW’s. In addition to going to eight different kinds of musical performances;

I spent time with:
12 performing musicians
10 blind musicians
8 sound engineers

Below is a short snippet of a blind artist sharing some of his experiences with audio production software.

<a href="https://medium.com/media/17bfce5205d58705c9b25ce2d2a015c0/href">https://medium.com/media/17bfce5205d58705c9b25ce2d2a015c0/href</a>

Professor Bryan Pardo (Interactive Audio Lab, Northwestern), and his student, Aaron Karp have already explored this space and come up with what they call a Haptic Equalizer. My work is in collaboration with them, to see how we might bring this product to life by defining and building the user experience and, designing the product around these ideals.

Synthesis

Sound: At a high level, to a musician, the quality of sound that reaches their audience is everything. They spend hours with their bands, or on their own curating their music. They either share their music live, or through recordings. DAW’s provide a means for musicians to control the sound and edit it before it is shared. Lets look at this EQ graph:

Screenshot of the EQ mixer interface on the Pro Tools (Source: google images)

Visual Feedback: The colors mark different frequency bands, the line graph is easily manipulated with the click and drag of a mouse and the feedback in the sound is immediate. Scrolling a mouse will adjust the parametric curve on the graph.

The biggest issue is that the graph is an image. Images are inaccessible to a completely blind person.

Navigation: For both sighted and blind people, a standard DAW interface is intimidating. This is not unusual for sophisticated software platforms. There is a lot happening on the screen, and navigation relies heavily on visual feedback. Moreover, colors are often used for differentiation.

Ideation and Low-Fidelity Prototypes:

I gathered some creative minds in my program to brainstorm ideas on interactions and physical solutions to EQ.

Brainstorming solutions to building a tangible equalizer

A few of these ideas were thought through in further detail …

I quickly built some low fidelity prototypes in order to test the different interactions that we had brainstormed.

Vertical representation of the EQ curveUsing optical light sensors to create a works like prototypeUsing a magnetic board with a chain to represent the EQ Curve

User Testing:

These prototypes were put in front of 8-10 users to understand what made sense to them. I tested with one visually impaired musician and, to get more data points, with sighted people as well. For sighted people, they would be blind folded, given a short explanation of what to expect with an EQ curve and some directed tasks to perform with the prototypes. It was important that none of the sighted people had seen any of the prototypes prior to testing.

I was trying to understand how texture and 3 dimensional indicators helped users with spatial awareness on the physical EQ graph. The graph could be manipulated using a continuous medium like the chain or discrete points of control like the magnets. Tasks were designed to test the different affordances provided in each of the prototypes.

User testing different iterations of prototypes

Listening to users and observing their interactions with the prototypes have shown me the areas that need improvement. These insights have allowed a few rounds of iteration. The latest prototypes I have created are aimed at providing users a way to understand their relative location on a graph using textural variations and swell paper. Swell paper is special paper on which images/text can be printed and heated to create a tactile representation of what is printed.

User testing the updated prototypes

Next Steps

I plan to spend my next quarter at Northwestern, building higher fidelity prototypes that incorporate the insights gathered to create the best possible user experience for a visually impaired musician. I would appreciate any feedback on the project. If you know a musician or someone with visual impairment I should talk to, let me know. If there is anyone interested in working with me on the electronics/programming side, I would love to talk to you!

Please feel free to reach out at: aarushi@u.northwestern.edu

(see other work | www.aarushik.com)

Audio Production for Visually Impaired Musicians was originally published in prototypr on Medium, where people are continuing the conversation by highlighting and responding to this story.

Audio Production for Visually Impaired Musicians

Cool stuff, every week

More Articles