top of page
Talent Assessment Tool
Client: A Large USA Based Talent Assessment Company
Team
Tools
Project Description
Role
UX researcher
Figma
Miro
Skills
The project aimed at conducting contextual inquiry to improve the workflow of the company's talent assessment tool.
Amit Das
Anantika Sethi
Jialun Sun
Mansanjam Kaur
Shiyu Wu
Interaction design
User research
Affinity mapping
User Interviews
Timeline
August 2022- December 2022
Defining the Problem
The company wanted to foster an intuitive workflow between the employee, manager, and the organization by enforcing a more efficient functioning of their current AI based talent assessment software. The goal was to enhance the utilization of their talent management services by these three different user groups.
Background Research
To develop an understanding of talent assessment and its integration with Artificial Intelligence, we conducted some background research on this area. And, before diving into research we formulated our research questions (as stated below)
minimum qualifications, education or the minimum experience required for a job role
Suitability
What factors contribute to the creation of an effective and efficient talent assessment tool?
The aptitude and behavioral aspect of the employee which would be fruitful for that particular job role
Eligibility
Conducting an Integrated assessment
Includes the employee’s motivations that ultimately decides their capability and willingness to succeed in that particular job role.
Motivations and Interests
What are some of the Good practices in talent evaluation?
Make suitability efficient
The employee might give fake answers to fit in the desired suitability aspects.
Some recommendations: Cross referencing Ranking rather than multiple choice questions, include positive statements, so that all statements are regarded as desirable by the employee, considerably minimizing the likelihood of them giving fraudulent responses.
Know your workforce
Adapt the talent assessment tool according to the technology literacy of the workforce.
For instance, an aging workforce might not be very comfortable with high tech tools as compared to the younger generations, and enforcing a digital talent assessment might end up putting this chunk of the workforce at a disadvantage.
Remove employer biases
Quantify the results of the skill assessment to make employer evaluation more fair. Because generally employers tend to be partial towards employees who are more like them in terms of personality.
Thus, to get rid of this bias, suitability tests can be evaluated in numbers.
Technological Literacy
Apply AI in ways that are simple to understand and intuitive to use for an ageing workforce or non frequent users of technology.
.png)
Time Factor
AI opens up a wide variety of possibilities to evaluate the employee on, but this shouldn't translate into an excessively long review.
Factors to consider when pairing AI with talent assessment
Fluidity of job roles
Industry trends are blurring the divide between different job roles implying fluidity in the skill requirements for these job roles.
.png)
Skepticism of managers
Managers are skeptical about the accuracy of the AI recommendations, and hence, do not fully utilize them.
Reduced diversity in workforce
AI based tools work on a fixed set of job characteristics to determine the potential of its candidates. For instance, if a candidate applying for a position possesses a non-typical work experience such as where they didn’t work in that particular job role per say, but worked in other related job roles that helped them build relevant skills pertaining to this role. The AI tool might end up vetting those candidates out, thus, making the recruitment process biased
Recommendation
Ethical considerations for using AI based talent assessment tools
Algorithmic bias
The assumption that AI tools are always free of biases, since they derive results from large data sets does not always hold true.
The fairness of these tools is also determined by the type of data we are feeding them with.
Recommendations
Exclude parameters such as gender, age, cultural and social identities from the data sets
Incorporating a feedback system to increase transparency for negative evaluation.
AI based talent assessment tool can expand its database. Instead of solely relying on the skills of previous employees to do the profiling for a job role, the tool can extend to other suitability and eligibility criteria that might be relevant to that job role.
Ethical considerations for using AI based talent assessment tools
Privacy Concerns
integrating talent assessment with AI demands the employees to put in-depth accounts of their work performance on a digital platform, or provide details into their personal life such as their social media accounts, and so on. This might end up putting the organization at a much higher power than the employee, where the employees personal data can be used for anything
Recommendation
More transparency could be provided to the employees. This implies that all the data that needs to be gathered for an employee's evaluation should be validated with the purpose for which it shall be used in the talent assessment.
Accuracy
AI based tools work on a fixed set of job characteristics to determine the potential of its candidates. For instance, if a candidate applying for a position possesses a non-typical work experience such as where they didn’t work in that particular job role per say, but worked in other related job roles that helped them build relevant skills pertaining to this role. The AI tool might end up vetting those candidates out, thus, making the recruitment process biased
Recommendation
Talent assessment tools should be requested to make the working of their tools transparent so that the organizations can understand the logic behind the recommendations of these tools.
User Interviews
User Interview Goal
What do stakeholders think about the usability of the current assessment software?
What are the flaws found in the current solution?
What kind of solutions are they looking for or expecting?
Defining the Job Roles
Engineering Manager
The engineering manager would be the head of the app development team. Since their main role lies in developing and testing out the tool we aim to gather where the difference is lying to further propose points that can help make the revamped tool free of those errors.
Sales/ Marketing Manager
Marketing manager includes users from the marketing and sales department of the company. By interviewing this user group, we aim to analyze market patterns about the particular aspects of the software that worked well or didn’t work well in the market.
Application User
.png)
Application user is the group who directly interacts with the application. Their challenges are the priority that the client should consider. By interviewing application users, we can find some misunderstandings between the client and the users, as well as some problems that the client has not seen.
Interview Protocol
Overarching Question
From a technical viewpoint what’s the current assessment of the software?
What are the technical flaws found in the current solution?
What kind of improvements can be made?
Structure of Questions
Engineering Manager
Software Development
Problems/ Challenges
Potential solutions/ MVP's of product
-
technological limitations of the tool
-
What features were recently incorporated and why?
-
challenges in the current version of the talent assessment tool
-
Features they would like to retain
-
Suggestions to make the tool intuitive
Overarching Question
What do marketing and sales managers think about the usability of the current assessment software?
What are the flaws found in the current solution?
What kind of solutions are they looking for or expecting?
Structure of Questions
Sales/ Marketing Manager
Software
Usage
Problems/ Challenges
Potential solutions
-
Features of the tool used & preferred by clients.
-
Features of the tool that are underutilized by clients.
-
challenges in the current version of the talent assessment tool from a sales perspective
-
Features they would like to retain
-
Suggestions to make the tool intuitive
Overarching Question
.png)
Application User
What do application users think about the usability of the current assessment software?
What are the flaws found in the current solution?
From the perspective of a user, what kind of improvements are they looking for or expecting?
Structure of Questions
Software
Usage
Problems/ Challenges
Potential solutions
-
Steps and process followed for using the tool.
-
Commonly used features of the tool.
-
challenges in the current version of the talent assessment tool from a sales perspective
-
Features they would like to retain
-
Suggestions to make the tool intuitive
Affinity Mapping
The findings derived from the interviews were turned into self contained assertions. these assertions were arranged into broader themes, (orange sticky notes). These themes were further clubbed under parent themes (denoted by blue notes). By doing this, we got a good sense of what issues we had to focus on for the next stage.
.jpg)
bottom of page