Voyce_complete

“The scammer threatened to arrest heR."

When we interviewed Kevin, he wasn't in a good mood - a scammer who claimed to be an IRS official called his 80-year-old mom.  "My mom couldn't get in touch with me in time when that scammer called." Kevin played a voice message from his mom. We heard a woman's shivering voice. "It is not just the money she lost. It took her a long time to recover from her emotional turmoils and self-blames."

Kevin's mom is not alone. Each year, elderly Americans lose as much as $37 billion to financial scams. How could we empower senior citizens to protect themselves from scammers?

Keywords: Voice User Interface, Smart Home, Fraud Detection

Collaborators: Elsa Ho, Kasturi Dani

Sponsor: BlinkUX

My contributions: secondary user research, fraud experts interviews, senior user interviews, experience map, flowchart, storyboards, usability tests, product ecosystem, design specifications.

User Research

Research methods

1. Secondary Research:  Read through reports from Federal trade commission, U.S. Department of Justice, Financial Fraud Research Center and Bureau of Justice Statistics.
2. Subject Matter Experts: interviewed five subject matter experts.
3. Field Visit and In-depth Interviews: conducted field visits and in-depth interviews with five senior citizens and five family members of senior fraud victims.

SUMMARY OF FINDINGs

After 8 weeks of user research, we produced a detailed research report. Below is a quick glimpse of what we have learned from user research. 

AHA MOMENT

Gradually, we noticed a pattern in our user research data: most of seniors’ family members play an important role in guarding seniors against phone fraud.

This is where the gap appears:

• Seniors desire independence and do not want to be a burden to their family
• Family members lack the time to constantly keep an eye on senior citizens and to make sure they are not falling victim to scams.

EXPERIENCE MAP

We decided to focus on phone scam. For two reasons:

• Our secondary research indicates that most of the financial scam came via phone. For example, on 2015, there were 1.5 million fraud complaints filed by Americans. 75% came via phone (source: Federal Trade Commission).
• We studied fifteen most popular senior scams such as investment scams and Medicare scams. Phone stood out to be the main channel.

An experience map was created to understand scammers’ actions and seniors’ reactions, to visualize the deficiency of current solutions, and to ideate new solutions.

Design themes

INDEPENDENCE: Providing a sense of independence to senior citizens.

EFFICIENCY: Delivering timely alerts to family members, without burdening them with excessive details.

ACCESSIBILITY: Respecting seniors citizens’ routines, constraints and behavioral preferences.

ADAPTABILITY: Adapting to new types of phone scams quickly and effectively.

IDEATION

1. Ideated around different stakeholders (seniors, caregivers, banks, peers, communities, etc.)
2. Ideated around user journey (pre-scam, during-scam, after scam).
3. Ideated based on competitor’s products or related services.
4. "Wishing" - “If you have unlimited money, what service would you develop?”
5. "Worst Idea Ideation" - Generated "worst ideas" and turn them into good ones.

Evaluation and Selection

We narrowed down more than 50 ideas into six directions:  scammer biometric characteristics identification, AI assistant, senior well-being monitoring, bank accounts catered for seniors, senior peer support and easy fraud report.

We identified “AI system” as the most promising direction to prototype

Prototype

Based on our design themes from user research (Independence, Efficiency, Accessibility and Adaptability), we prototyped an AI system “Voyce” that identifies scammers, cautions seniors and notifies family members. "Joyce" is the Artificial assistant that lives inside of "Voyce".

IDENTIFY SCAMMERS

Voyce uses three strategies to identify scammers:

1. Check if the incoming phone number is in the scam database.
2. Direct unidentified callers (callers who are not in the senior's contact list, neither do they come from credible organizations) to leave a voice message, and check if the voice message is fraudulent.
3. During a phone call, check if the senior’s current phone conversation is fraudulent.

If you want a deep dive of how Voyce handles different scenarios, check out the flowchart below.

CAUTION SENIORS

Through a voice based interaction, the artificial assistant Joyce cautions seniors and prompts seniors with next steps. Below is a few sample clips.

NOTIFY FAMILY MEMBERS

For senior’s family members, the system provides an app to guide them through the on-boarding process, helps them customize settings, and sends notifications when seniors are having conversations with potential scammers or receiving suspicious voicemails.

The AI system gives each call a “ScamScore” to help family member judge if a phone call is fraudulent.

Usability Test

We tested the audio prototype on seniors, and the app prototype on family members.

Audio prototype main improvements

Tailored to seniors’ specific needs, we added additional voice interaction design guidelines:         

SIMPLICITY: To simplify the interaction, ask only yes or no questions. If seniors have to make a choice, limit the options to two items.                        

AFTER                    

Joyce: “Do you want to end the conversation?”

Senior: "Yes."

PROGRESSIVE DISCLOSURE: Progressively guides the senior by presenting only one task at a time.                    

AFTER

Joyce: “What can I do for you?”

Senior: “Add contact.”

Joyce: “Sure. What is the name of the person you would like to add?”

Senior: “Nancy.”

Joyce: "What is Nancy's phone number?"

Senior: "It's 111-111-111.

APP prototype main improvements

We provided more information to help family members decide if they want to block a phone number, including fraudulent trigger words appeared in phone conversations, reasons for flagging, and more suggested actions.

Users commented on the onboarding experience being too long and having to make too many decisions at once. As a result, based on the level of security chosen, the system will decide the notifications and settings for them, which they can later customize.         

We also simplified some terms on the app. For example, we changed “Trusted ID” to “Contacts”.

Product Ecosystem

The chart below gives a list of secondary stakeholders.

• Voyce hosts its database on the cloud through cloud computing services.
• Voyce collaborates with existing scammer blocking services to enrich its scam database.
• Voyce works with Research Centers to update its fraud detection algorithms.
• Voyce extracts public companies' information from seg.gov to evaluate if the call is coming from credible organizations.

MOVING FORWARD

Voyce could do much more than phone fraud, through collaborating with other service platforms such as home surveillance systems and email service providers. For instance, if a senior clicked a potentially fraudulent link or opened a scammer's email, Voyce could caution the senior and notify his family members. 

Q & A

We pitched Voyce to our sponsor, user-experience consulting firm BlinkUX and other industry representatives. Below are a collection of questions inquired by our audiences.