Leveraging usability research to improve a mobile online application for disaster relief
Challenge
As a founding member of a new Digital Customer Experience team at FEMA, we were tasked with building a usability research practice in the agency to improve the experience of survivors applying for disaster assistance. The main online application in development was approaching launch but had never been tested with members of the public. Additionally, most of the application had been designed for desktop, despite over 80% of survivors accessing the service on mobile devices. Our team was tasked with conducting usability research and making recommendations to improve the user experience ahead of a rapidly approaching public launch.
Part of the project was to advocate for the importance of usability testing as part of the product design process.
Process
Our team conducted an in-person, moderated usability test of the application on mobile devices with disaster survivors in the field. It was important to our team to compensate our research participants for their time and to develop a testing protocol that was trauma-informed, neither of which were common at FEMA or within the federal government. We collaborated with members of the US Digital Service on the research protocol and socialized our plan with stakeholders across FEMA to gain the necessary approvals.
The research sessions revealed multiple critical issues with the mobile user experience that would have prevented survivors from completing their application or led to them incorrectly applying for benefits. In addition, there were several questions that caused anxiety and uncertainty among participants who felt like they were “being tested” by FEMA. Our recommendation including fixing usability issues and revising the language of several questions to better communicate with survivor and to leverage plain language best practices.
Outcome
The recommendations from the research were adopted in time for the public launch of the application. We also built trust among the product team in our testing approach and the benefits of testing with real users in their preferred format. The improvements from this and future rounds of testing reduced median application time for disaster survivors from 34 to 23 minutes (a 32% decrease). It also increase satisfaction scores across several customer experience metrics. The project was also selected for a Department of Homeland Security IT Award for Customer Experience, and helped establish a usability testing practice at FEMA.
The updated application showed improvements in several customer experience satisfaction metrics over previous baselines.
Project team: Gail Swanson, Sam Power, Ben Peters, Jyoti Sauna, and Suzy Porterfield.