CIVIC USER TESTING
@Dearborn Administrative Center
Project Description
The project aimed at testing an already functioning digital visitor management system for Dearborn Administrative Center (DAC) to come up with design recommendations that strive to improve the efficiency of the system and enables a fast and intuitive check-in process for the visitors coming to DAC.
Team
Mansanjam Kaur, Iris Young, Phil Mendez, Sreelakshmi SB
Role
Developed Testing protocol, conducted user testing sessions, designed solutions.
Tools
Figma, Miro
Skills
Usability Testing, User Research, Problem Solving, Product thinking
Timeline
October 2022 - November 2022
Design Process
1
Exploratory
Understand the problem space
Generative
Identify opportunities
Evaluative
Generate and evaluate concepts
2
3
Why Our Solution makes a difference?
Our recommendations accelerated check-ins for 7,000 monthly visitors and saved DAC significant staffing costs for check-in assistance..
What is Greetly?
Greetly is a visitor management system using an iPad as a digital kiosk to welcome office visitors, match them with relevant staff, and guide them on next steps.
Greetly also has an employee interface and reporting tools, but this user testing focused on the visitor interface.
How Does the Check-in Work for a Greetly User?
Upon arrival, visitors check themselves in through the digital kiosk.
Real-time notification alerts are sent to the host employee upon the visitor's check-in.
The check-in process is complete and the visitor and employee can get down the business.
Client Needs
01
It was difficult for residents to find services they needed using Greetly. There were many departments, and their titles on Greetly didn’t clearly match with the way residents thought about their purpose.
02
The City tried to address this by staffing a kiosk in the lobby, to help direct people. But staffing is expensive, and visitors were frustrated by feeling “passed off” to another person again and again.
03
DAC mostly had internal feedback from employees regarding the functioning of Greetly, while they absolutely lacked feedback from the end users (i.e. the residents of Dearborn)
Hence, it was time to test the system with users and discover their specific pain points!
Why User Testing?
To find answers to the following questions >
Are visitors able to use Greetly to get to the right place for their business at DAC?
What confusion, or delays, might the Greetly kiosk cause for visitors?
Is it meeting visitor expectations for fast and efficient service?
Test Participants
Age group
Used Greetly before
Language
Comfort with tech
Dearborn Employee
20s
R1
Yes
Arabic, English
High
No
30s
R2
No
Arabic, English
High
Yes
60s
R3
No
English
Low
Yes
20s
R4
No
Arabic, English
High
No
40s
R5
Yes
Arabic, English
High
No
30s
R6
Yes
Arabic, English
High
No
20s
R7
No
Arabic, English
High
Yes
30s
R8
No
Arabic, English
High
No
Majority of participants were
Comfortable with technology
Young ( between 20-30 years)
Bilingual (fluent in English & Arabic)
Usability Test Protocol
To brief the user about the process for usability testing. And, to remind them that,
"If the system doesn't work for you, then it doesn't work!"
1. Introduction
(2 mins)
Test Structure
3. Tasks
(20 mins)
A total of 5 tasks were given to the user and after each task, they were asked about any hard stops/ challenges and any other observations.
To understand if the user had any previous interactions with the Dearborn Administrative Center. What type of interactions?
2. Pre-test questions
(3-4 mins)
4. Post-test questions
(3-4 mins)
Follow up questions were asked to the user about the overall process of checking in through the digital kiosk.
The Tasks
Get a copy of your birth certificate
Get a building permit
Get a query regrading property taxes sorted
+
File a complaint for missed trash service
Book an appointment with the mayor
We used the data from DAC's visitor log and identified the most frequent tasks performed through the digital kiosk. The visitor log gave us all the data till 10/31/2022.
3 user testing tasks were identified based on the data from the visitor log
2 user tasks were identified based on our initial client meeting
The Day of Testing!
Synthesizing Data from User Testing
After conducting the usability tests, the notes and the user quotes were translated into assertions. These assertions were then put on sticky notes and grouped to establish common issues within the system. Since the project had to completed within the 5 weeks, we democratically prioritized on the issues that we wanted to address and build upon.
After detailed discussions, we decided to narrow down on four major findings (as labeled below). These were chosen on the basis of frequency with which these issues were faced by the users during the testing.
.jpg)
Brainstorming Solutions
As a next step, we set a timer for 15 minutes to come up with potential solutions for each of the four findings as chosen above. These solutions were put under similar themes. Additionally, we realized finding 3 and finding 4 could be clubbed together.
Primary Findings
Finding 1 : Information Overload on the Screen
What caused information overload?
01
Users were overwhelmed by the number of categories
02
Users faced readability issues
03
Users wanted human assistance
The screen is very crowded
It's hard to read the text, I would prefer a bigger font size
Instead of 12 categories, 6 would be preferable
There are just too many options!
I would prefer a mix of technology and in-person interactions
Our Recommendations
Adding a Help Button
To provide FAQs and an option to reach out to staff if users still have queries.
Placing Categories by Frequency
Categories can be organized by frequency to minimize the average search time for users, placing less common ones under an "Others" tab.
Finding 2 : Confusing Categories
How were the categories confusing users?
01
Users did not understand the meaning of some categories and what services those categories entailed.
02
We discovered that half of the participants mistakenly believed the check-in kiosk could book appointments with the mayor, despite it not having that capability. We asked users to book an appointment with the mayor through the kiosk.
4 out of 8 participants went ahead with the task prompt without realizing that they could not actually book an appointment with the mayor.
03
Users had different interpretations of different category names.
Finances makes more sense than assessor with that dollar icon and all
What is assessor?
Grouping of Appointments and Delivery looks a little off to me.
Engineering’ and ‘Permitting’ both go hand in hand. Why are there two options for these?
What is licensing?
I am not gonna press sanitation even though it has the icon of a garbage can. That’s not the first place my mind goes to.
I don't know what to click!
Variation 1 | Pop-up Cards
Users can click on categories to see available services, allowing them to decide whether to proceed or explore other options.
Our Recommendations
Variation 2 | Keyword Search
The search feature allows visitors to enter a keyword and press enter to filter and display only relevant categories, eliminating the need to browse each category one by one.
Additional Recommendation
Enhancing Clarity in Scheduling Mayor Appointments
Scenario 1
When the visitor has already booked an appointment with the mayor.
Scenario 2
If the visitor does not have a pre-booked appointment with the mayor.
Finding 3 : Lengthy Check-in process
What made the check-in process lengthy?
01
Some information is collected at the kiosk but is again asked at the counter of the concerned department.
02
Users are not patient to fill in the questions at the kiosk. They are not motivated either because they do not know what those questions are for.
I usually just put in a letter (in the First Name box). It does not matter.
They will ask my address at the counter anyways…
Oh we need to scroll down?
“See, I can still submit this form without answering the asterisk question”
Why are they asking me if I am a homeowner or a contractor here? Aren't they going to ask this anyways at the counter?
Wow I need to take a moment to read through -
(for appointment category)
Our Recommendations
Check-in with s single tap
The check-in homepage prompts users for their first and last names. Users can click on each tab to see details of the service, and then check-in on the home page itself with a single tap.
Effort v/s Impact
DAC sought a solution that minimally disrupts staff while efficiently serving residents, prioritizing low effort, high impact options. Based on team discussions and assumptions about Greetly's and DAC's capacities, we recommended immediate implementation of solutions 1, 5, and 7 for their optimal balance of ease and effectiveness
Our Recommendations
1- Reducing the number of categories on homescreen
2 - Order categories by Frequency
3 - Help Feature
4- Pop up cards
5 - Adding another page in ‘delivery/appointment’
6- Search feature and top search results
7- Consolidate check-in to home screen
Conclusions
Currently, residents do not just rely on Greetly but use work arounds such as human assistance.
The number and wording of categories create some confusion and delays
Greetly is mostly meeting expectations but it can achieve fast service with some improvements
Client Feedback
I was extremely pleased with the recommendations you guys provided. These are straightforward and easy to implement!
Regarding the suggestion to introduce pop-up cards for category descriptions, I do see the value in it. However, it appears to require a bit more in terms of technological resources than we anticipated. Nonetheless, it's definitely something we plan to implement in the near future.
They already went ahead with the recommendation to add an additional page in the 'delivery/appointment' section, and the response was overwhelmingly positive. The video below offers a glimpse into how they applied our solution!
Next steps and Future Research
Some recommendations need to be implemented yet and their response needs to be monitored.
Testing involved only young, tech-savvy participants; future tests should include older adults and Arabic speakers.
The project, limited by time, overlooked accessibility issues like blindness, necessitating adjustments for a more inclusive kiosk
My Learnings
-
This project helped me understand the importance of usability testing in defining the success of a project. At the very start of the project, I had some whacky ideas to improve the workflow of the digital kiosk. However, after conducting the usability tests, I realized how simple design modifications could do the job of improving the user experience of the system efficiently.
-
Getting to lead the usability tests and working on developing the testing protocol, I realized how one needs to change the testing protocol and adapt it to different users to break the monotony and the robotic delivery of the testing protocol.
-
One thing that I constantly had to struggle with was when the participant would turn to me to seek validation for their actions for a certain task. I had to constantly remind myself that helping users complete the task was never the motive of the testing, rather it was to simply observe the user's actions and thoughts as they went about performing a task.
-
Lastly, working on this project taught me how to prioritize some issues over the others due to time, budget and technological limitations of the client .