CS-247 Project

User Study

Introduction

Motivation and goals

Our goal is to create a platform that can facilitate critique among musicians who are trying to improve their abilities. There are musicians who spend countless hours practicing on their own, but can’t easily find anyone to give them a quality second opinion. I could just ask my next door neighbor what he thinks of my new guitar piece, but since he’s not familiar with the instrument or music he can’t really provide constructive or helpful criticism. Our hope is to therefore create the opportunity for amateur musicians to perform, discuss, and learn from one another.

Driving Questions

How will musicians find other people to critique them on a regular basis?
Will musicians feel comfortable practicing in front of people with this remote interface?

Are there any parts of the interface that make it uncomfortable for the musician?

What kind of feedback will listeners provide in real-time?

Will listeners use the quick-critique buttons? Are there certain categories that they use/don’t use?
What kind of feedback will they provide in the text comments?
What kind of additional feedback, if any, will they provide in the critique session?

How do musicians/listeners interact with the recorded playback?

How do they interact with the video controls? Who takes control?
How do the real-time listener comments add or detract to the conversation between the musician and listeners?
How do they interact with the real-time comments?

Will musicians be able to get valuable feedback from using the app?

Will they receive additional insight that they didn’t have before from the listener?
Will they be able to self-reflect on the playback to discover something new about their practice?
Will listeners be able to get additional insight into their playing from critiquing?

Is the application intuitive and easy to use for both users?

Do both users understand what “phase” of the session they are in?
Does the musician understand how to start and end the recorded practice session?
Does the listener know how to note feedback using buttons/text?
Do both users understand what the playback video is/how to use it?

Hypotheses

Users will want to connect and exchange feedback with individuals whom they already know the playing ability for.
Users are excited to share their work with others, and find value in practicing for an audience.
Users may not realize how to officially start a recorded practice session, and will play prematurely.
Users will initially be confused by the button critique interface and will primarily rely on the text commenting feature.
Users will want a customized critique button set for their particular instrument.
When reviewing the playback, users will want to jump to/replay problem spots.
The more experienced user (typically listeners) will want to take control of the playback video.
When specific pieces of critique show up in real-time, listeners will expand and explain their comment, and musicians will self-reflect and discuss this piece of feedback.
Users will refer to specific pieces of critique in real-time playback video to spark further discussion on this feedback.
Musicians will be able to self-reflect and hear things about their practice that they didn't realize before.
Critiquing with this structured format remotely will lead to more honest critiques from the listener other than positive comments.
Comment review over video chat will spark further discussion about their music and artists in general.

Methods

User Recruitment

We are recruiting users by directly contacting musicians and setting up times to meet with them over the next few days. Each of us know a few musicians, and we are seeing if they have any friends who would be interested in participating. We have also emailed out our respective email lists that we are on, including ones with a very large reach across campus, about our study with the promise of free Jamba Juice. We are making a particular effort to find pairs of musicians who play the same instrument and are on similar experience levels so that they can give each other constructive feedback.

Surveys

We are planning on having our users on taking surveys both before and after testing. Beforehand we will ask them questions about their general habits as musicians (i.e. how often do you play for an audience, how often to you ask someone for feedback, etc). Afterwards we will ask them questions about their experience with the app (what did they like the most/least, etc). We also intend to interview them afterwards to find out what kind of people they could see themselves using the app with, to get a better sense of how best to go about connecting the musicians with the listeners.

Critical Tasks

We will conduct testing by asking users to complete the following representative asks and observe them while doing so:

Registering for an account, including entering the instrument they play
Exploring homepage
Initiating a call (musician)/ Answering the call (listener)
Critiquing a performance

Starting/ending the recorded portion (musician)
Using button interface (listener)
Typing in comments (listener)

Reviewing playback together

Using video controls (listener)

Data

As mentioned above, we will give each user a pre-post survey about their experiences as a musician/listener on our app. We will also be taking observational notes while the users are performing the critical tasks above, focusing on answering the driving questions. On top of general observations we have about user behavior, we will analyze the following specific data points:

Time spent registering
Time spent figuring out navigation/homepage
Number of critiques made total
Types of critiques made (button vs. typed comments)
Observed errors or points of confusion
Length of post-practice critique exchange
Frequency listeners refer to specific points in playback video during critique exchange