Hackathon
 

Check My Hearing

Alexa Skill // Voice Design // Conversational UI


launcher_icon_512px_x_512px_v3_480.png
userArtboard 2.png
 

Together with five teammates at Horizontal Integration, we created and launch an Alexa Skill over the course of two days. Our skill, Check My Hearing, brings preliminary hearing testing into the home in an effort to reach more individuals struggling with this common condition.


CHALLENGE

Horizontal Integration organized a two day hackathon to stimulate new ideas, cultivate collaboration, encourage exploration, and develop capabilities within the agency.

Our finished products would be presented and judged on the following:

  1. Innovation: How original is your idea and does it innovatively utilize the underlying technologies?
  2. Functionality: Does the prototype function as described and adhere to user experience conventions?
  3. Presentation: Was the team able to explain their idea and what the app actually did?
  4. Relevance: Does it accomplish the goal(s) of the Hackathon and can we pitch the idea to potential clients?

Team

Our team was well-balanced in representing different areas of the agency. As a result, roles were naturally defined and we worked well together to concept and execute our idea.

Project Manager: Rachel Carlin

UX Designer: Megan Burke

UI Designer: Kathryn Deming

Strategist: Arturo Mendiola

Front End Developer: Cullan Luther

Back End Developer: Rohit Keshwani


PRELIMINARY RESEARCH

The Hackathon kickoff provided a nice introduction to the technology for those of us who had little experience with voice interaction and natural language processing. Due to the time restriction of the Hackathon we would be focusing on the core functionality, but needed a good understanding of the capabilities and limitations of the technology. We also needed a target audience and evidence that our skill would provide value to users.

Voice Interface Research:

  • Voice is the quickest way to remove barriers between users and tools. 
  • Nearly half of Americans use digital voice assistants as of 2017
  • Sales of standalone units from Google and Amazon are expected to increase 3x in 2017.
  • Only 3% of a skills' users remain active after the first week
  • 69% of the skills available for Amazon Alexa have 0 or 1 star reviews
  • Alexa skills typically fall into 4 buckets:
    • Doing (performs a task)
    • Searching (identifies specific info)
    • Telling (provides a quick reference point)
    • Browsing (gives information on a broad subject)

Hearing Loss Research:

 

  • Health and wellness as a trend is infiltrating Alexa skills, but none target hearing loss
  • Hearing loss is the third most common physical condition after arthritis and heart disease
  • 20% (48 mil.) Americans suffer from some form of hearing loss
  • 1/3 of these individuals are over 65
  • 15% of school age children have hearing loss
  • 2.5 of every 1,000 babies are born with congenital hearing loss
  • 80% of people with hearing loss do nothing about it

Identify Purpose & Capabilities

As a group, our team brainstormed skill ideas that relate to the types of clients we work with at Horizontal Integration. We wanted to keep our scope small enough that we could accomplish the core functionality in the time allotted while ensuring the product was innovative. Horizontal Integration counts a hearing aid manufacturer among its clients and the audible nature of this opportunity was immediately appealing.

Our idea was this: take the traditional preliminary hearing test offered by many sites online and translate it into Alexa skill. 

What is the purpose of the skill? Why will people want to use it?

To test a user's hearing and, if needed, provide contact information for a hearing professional. People who think they may be losing their hearing would want to use a simple, free test before scheduling an appointment. 

What will the person be doing before, during, and after interacting with the skill?

They will likely be in their home, alone or with a loved one. They may be researching hearing loss, come across a link, or be recommended the test.  After they interact with the skill they can set up an appointment if needed. 

What will people get from the skill that they cannot get another way?

Hearing tests online feel very formal - like a test. Alexa is a great opportunity to put a user in a similar situation where they may have experienced difficulty hearing in the past. 

All Photos by Horizontal Integration


Identify User Stories

  Photo by Horizontal Integation

What can a user do, or not do, with the skill?

The user can answer a series of audio quiz questions to learn if they show signs of hearing loss. They man then ask to receive the contact information of a hearing professional. The user cannot learn details about their hearing or information about hearing loss.

What information is the person expected to have available?

None. They are simply repeating back what they hear.

What are the ways a user can invoke the skill?

They can ask Alexa to test their hearing, or ask to open the app and Alexa will initiate the hearing test.

What features directly support the purpose?

Alexa plays audio files and listens to see if the user can identify a word or phrase within them. Alexa keeps track of whether the responses are right or wrong and makes a recommendation at the end.


Interaction Model, Flow, And Script

Once we had an idea of what we were creating and why, it was time to put together the interaction model and create a script so development could begin. This meant not only exploring invocations, intents, and utterances, but also creatively working on the tone and personality. 

Develop Flow:

  • Outline the shortest route to completion
    • Alexa plays three audio files, the users answers them all correctly, and Alexa says the user has great hearing.
  • Outline alternate paths and decision trees (branching logic)
    • Alexa plays three audio files, the user gets two or more wrong, Alexa asks if they would like to contact a hearing professional, the user says yes or no, Alexa does/doesn’t provide contact information.
  • Outline behind-the-scenes decisions the system logic will have to make
    • Know the correct language in the audio file and decide if what the user said has matched or mismatched with it.
    • Keep track of whether the user gets the answers right or wrong.
    • Can make a recommendation based on how many answers were right or wrong.
    • Provide contact information for a hearing professional based on location.
  • Outline how the skill will help the user
    • The questions make more sense in an audio format rather than in an online for, because audio is what is being tested. Providing audio clips that reflect real world situations shows empathy for the user and allows them to more easily connect with past similar experiences.
  • Outline the account linking process, if present
    • None. May need zip code input to suggest a local hearing professional.
  • Outline how Alexa will capture information
    • TBD - not is scope for Hackathon
  • Outline error cases
    • User begins to answer the question with something like, “I’m not sure, but I think...” or “I heard…”, etc.
    • User begins to answer before clip ends.
    • User asks Alexa to repeat the clip or asks for the correct answer.

Create a Script:


    Meet Check My Hearing

    Our developers were incredible and it was so exciting when Alexa was first able to respond to us and work as intended. 

    Check it out:

    Slide10.png

    Presentation & Next Steps

    We put together a deck to present our case for the Alexa Skill and did a demonstration of the skill in action. As exciting as it was to have created a skill in just two days, it was equally as exciting to talk about the potential of the skill and how it could grow into a valuable resource for consumers.

    Beyond The Prototype

    Ideas to extend the experience:

    • Integrate with insurance provider APIs to help consumers find a hearing aid professional in network.
    • Expand user base to family members, friends, and caregivers by making it fun and easy to use.
    • User machine learning to prompt the Check My Hearing Alexa skill to initiate the test.
    • Share user data anonymously for research and improved test quality.
    • Work with hearing professionals to incorporate more robust and sophisticated hearing test questions.
    • Extend the skill to incorporate product recommendations or smart devices like being able to check a hearing aid battery.

    Everyone's a winner, but especially us!

    Based on the judgement criteria, our team won! We were each rewarded with an Alexa to continue exploring and learning about this technology.

    1. Innovation: There is no other hearing skill currently on the market, and we thought utilizing this technology would be a great way to destigmatize hearing loss and reach more people.
    2. Functionality: Functionally, our skill has a way to go. We were unable to get the logic working in time for the presentation but put together a compelling proof of concept that we would eventually demo to our client.
    3. Presentation: By thinking through our users and their needs, we were able to connect on an emotional level with the judges.
    4. Relevance: Since we developed this skill with a specific Horizontal Integration client in mind, our skill and presentation was tailored for use as a pitch. Several months after the Hackathon, we were able to pitch this idea to our client and get them to start thinking about new technologies.

    all images by Horizontal Integration


    Pitching The Client

    In addition to presenting the Alexa skill concept, I also created a new home page concept that would elevate their design and bring interactive tools to the forefront. The goal of my design was to simplify messaging and provide users with a clear starting point as they begin or continue on their hearing loss journey.

    Existing Site

    Home Page Redesign