Civic User Testing
Client : Dearborn Administrative Center
Team
Iris Young
Mansanjam Kaur
Phil Mendez
Sreelakshmi SB
Role
Developed Testing protocol, conducted user testing sessions, developed solutions.
Timeline
Tools
Figma
Miro
Skills
Usability Testing
User research
Problem Solving
Product thinking
October 2022- November 2022
Project Description
The project aimed at testing an already functioning digital visitor management system for Dearborn Administrative Center (DAC) to come up with design recommendations that strive to improve the efficiency of the system and enables a fast and intuitive check-in process for the visitors coming to DAC.
Project Impact on Users
The project helped to improve the check-in process for almost over 7000 visitors each month at the DAC
What is Greetly?
Greetly is a visitor management system that can welcome visitors to an office, allow them to match their needs to the people who can help, and alert relevant staff to their request.
Visitors check-in using Greetly to indicate the purpose of their visit and get directions for next steps and where to go. Greetly runs on designated hardware ( ipad) on a stand, like a digital kiosk. Greetly also has an employee interface and reporting tools, but this user test focused on the visitor interface.
Visitor Management System of
Greetly
Upon arrival, visitors check themselves in. through the digital kiosk.
Real-time notification alerts are sent to the host employee upon the visitor's check-in.
The check-in process is complete and the visitor and employee can get down the business.
Background
The City of Dearborn recognizes that it can be difficult for residents to find the services and support they need at the DAC. There are many departments, and their titles don’t clearly match with the way residents think about their business.
The City tried to address this by staffing a kiosk in the lobby, to help direct people. But staffing is expensive, and visitors were frustrated by feeling “passed off” to another person. This summer, UMSI students worked on an implementation of Greetly, a visitor management system. Based on user research, they established an information hierarchy and built out the content for the user interface.
Now it is time to test it with users!
Client Needs
DAC mostly have internal feedback from employees regarding the functioning of their visitor management system, while they absolutely lack feedback from the end users (i.e. the residents of Dearborn)
DAC needed a solution which doesn't change the system much for the staff but also caters to the residents efficiently. That is a low effort and high impact solution.
Problem Statement
Are visitors to the Dearborn Administrative Center (DAC) able to use Greetly to get to the right place for their business with the City?
What confusion, or delays, might the Greetly kiosk cause for visitors?
Is it meeting visitor expectations for fast and efficient service?
Test Participants
Age group
Used Greetly before
Language
Comfort with tech
Dearborn Employee
20s
R1
Yes
Arabic, English
High
No
30s
R2
No
Arabic, English
High
Yes
60s
R3
No
English
Low
Yes
20s
R4
No
Arabic, English
High
No
40s
R5
Yes
Arabic, English
High
No
30s
R6
Yes
Arabic, English
High
No
20s
R7
No
Arabic, English
High
Yes
30s
R8
No
Arabic, English
High
No
Majority of participants were
Comfortable with technology
Young ( between 20-30 years)
Bilingual (fluent in English & Arabic)
Usability Test Protocol
To brief the user about the process for usability testing. And, to remind them that,
"If the system doesn't work for you, then it doesn't work!"
Introduction
(2 mins)
Test Structure
Tasks
(20 mins)
A total of 5 tasks were given to the user and after each task, they were asked about any hard stops/ challenges and any other observations.
To understand if the user had any previous interactions with the Dearborn Administrative Center. What type of interactions?
Pre-test questions
(3-4 mins)
Post-test questions
(3-4 mins)
Follow up questions were asked to the user about the overall process of checking in through the digital kiosk.
The Tasks
We used the data from DAC's visitor log and identified the most frequent tasks performed through the digital kiosk. The visitor log gave us all the data till 10/31/2022.
5 Main tasks were identified to test with the participants. These are listed below:
Get a copy of your birth certificate
Get a building permit
Get a query regrading property taxes sorted
File a complaint for missed trash service
Book an appointment with the mayor
2 areas of interest based on our initial client meeting.
Data Analysis
After conducting the usability tests, the notes and the user quotes were translated into assertions. These assertions were then put on sticky notes and grouped to establish common issues within the system. Since the project had to completed within the 5 weeks, we had to prioritize on the issues that we wanted to address and build upon.
After detailed discussions, we decided to narrow down on four major findings (as labeled below). These were chosen on the basis of frequency with which these issues were faced by the users during the testing.
.jpg)
Brainstorming Solutions
As a next step, we set a timer for 15 minutes to come up with potential solutions for each of the four findings as chosen above. These solutions were put under their respective finding names. Additionally, we realized finding 3 and finding 4 could be clubbed together.
Primary Findings
Finding 1 : Information Overload on the Screen
The screen is very crowded
Instead of 12 categories, 6 would be preferable
There are just too many options!
It's hard to read the text, I would prefer a bigger font size
I would prefer a mix of technology and in-person interactions
Identifying Sub-issues
01
Users were overwhelmed by the number of categories
02
Users faced readability issues
03
Users wanted human assistance
Solution
Adding a Help button
Help can provide FAQs and an option to reach out to an employee if users still have queries.
Reducing the number of categories on homescreen
Frequently used categories can be accessed directly from the homescreen
The rest can be grouped into an Others tab
Ordering categories by frequency
Categories can be ordered by Frequency, so that users, on average, take the least amount of time to locate the category they need
Bigger Font Size
Finding 2 : Confusing Categories
Finances makes more sense than assessor with that dollar icon and all
What is assessor?
Grouping of Appointments and Delivery looks a little off to me.
Engineering’ and ‘Permitting’ both go hand in hand. Why are there two options for these?
What is licensing?
I am not gonna press sanitation even though it has the icon of a garbage can. That’s not the first place my mind goes to.
I don't know what to click!
Sub-issues
01
Users did not understand the meaning of some categories and what services those categories entailed.
02
During our client meetings, we found out that currently visitors misunderstand the check-in kiosk to be a tool to book an appointment with the mayor. We asked users to book an appointment with the mayor through the kiosk.
4 out of 8 participants went ahead with the task prompt without realizing that they could not actually book an appointment with the mayor.
03
Users had different interpretations of different category names.
Solution
Adding Pop-up Cards for each category
Instead of making the user open and check each category, they could click on each category and check what services are offered in it. The user can then see for themselves if they want to go ahead and check-in or explore a different category.
Solution
Scenario 1
If the visitor has already booked an appointment with the mayor.
Scenario 2
If the visitor does not have a pre-booked an appointment with the mayor
Solution
Adding the option to do a keyword search
The visitor can use the search feature and type in the keyword for their purpose of visit. As they press enter, the categories would be filtered and only the ones that are relevant to that keyword would be displayed for the user.
This would save the visitor from the hassle of going through each category name and it would also address the issue of information overload on the screen since only the relevant categories would be shown to them now.
Finding 3 : Lengthy Check-in process
Wow I need to take a moment to read through -(Appointment button)
They will ask my address at the counter anyways…
I usually just put in a letter (in the First Name box). It does not matter.
Oh we need to scroll down?
Why are they asking me if I am a homeowner or a contractor here? Aren't they going to ask this anyways at the counter?
“See, I can still submit this form without answering the asterisk question”
Sub-issues
01
Some information is collected at the kiosk but is again asked at the counter of the concerned department.
02
Users are not patient to fill in the questions at the kiosk. They are not motivated either because they do not know what those questions are for.
Solution
Cutting the check-in process short
-
the user will be prompted tp enter their First Name and last name on the top of check-in homepage
-
Users can click on each tab to see details of the service
-
Check-in is done via the button on homepage by a single tap.
Effort v/s Impact
1- Reducing the number of categories on homescreen
2 - Order categories by Frequency
3 - Help Feature
4- Pop up cards
5 - Adding another page in ‘delivery/appointment’
6- Search feature and top search results
7- Consolidate check-in to home screen
These were based on our discussions and assumptions as a team regarding the system capabilities of Greetly and DAC resources.
Conclusions
Currently, residents find there way to do business in the DAC using Greetly, by using work arounds such as human assistance.
The number and wording of categories create some confusion and delays
Greetly is mostly meeting expectations for fast service but it can benefit from some improvements
Client Feedback
The Client was extremely happy with the recommendations. She found almost all of them to be easy to implement.
However, for the recommendation of having pop up cards explaining descriptions of each category, she felt that would require more amount of technological resources. However, she plans to implement that solution in the near future.
As of now, she has already implemented the recommendation for adding another page in 'delivery/ appointment' and has had a positive response. The video above shows a small snippet of the client's implementation of our recommendation.
Next steps, Limitations and Future Research
Implement recommendations and monitor responses
We tested with only young, tech-savvy participants. It would be important moving forward to include participants who are older adults or only speak Arabic.
We didn't focus on issues of accessibility (such as blindness) due to limited time of the project. Ergo, some considerations need to be made to make the kiosk inclusive for different user groups.
My Learnings
-
This project helped me understand the importance of usability testing in defining the success of a project. At the very start of the project, I had some whacky ideas to improve the workflow of the digital kiosk. However, after conducting the usability tests, I realized how simple design modifications could do the job of improving the user experience of the system efficiently.
-
Getting to lead the usability tests and working on developing the testing protocol, I realized how one needs to change the testing protocol and adapt it to different users to break the monotony and the robotic delivery of the testing protocol.
-
One thing that I constantly had to struggle with was when the participant would turn to me to seek validation for their actions for a certain task. I had to constantly remind myself that helping users complete the task was never the motive of the testing, rather it was to simply observe the user's actions and thoughts as they went about performing a task.
-
Lastly, working on this project made me prioritize some issues over the others due to time, budget and technological limitations of the client .