top of page

Talent Assessment Tool

Client: A Large USA Based Talent Assessment Company

Team

Tools

Project Description

Role

UX researcher

Figma

Miro 

Skills

The project aimed at conducting contextual inquiry to improve the workflow of the company's talent assessment tool. 

Amit Das

Anantika Sethi

Jialun Sun

Mansanjam Kaur

Shiyu Wu

Interaction design

User research

Affinity mapping

User Interviews

Timeline

August 2022- December 2022

Defining the Problem

The company wanted to foster an intuitive workflow between the employee, manager, and the organization by enforcing a more efficient functioning of their current AI based talent assessment software. The goal was to enhance the utilization of their talent management services by these three different user groups.

problem graphic.png
br 4.jpg

Background Research

To develop an understanding of talent assessment and its integration with Artificial Intelligence,  we conducted some background research on this area. And, before diving into research we formulated our research questions (as stated below)

icons.png

minimum qualifications, education or the minimum experience required for a job role

Suitability

What factors contribute to the creation of an effective and efficient talent assessment tool?

The aptitude and behavioral aspect of the employee which would be fruitful for that particular job role

Eligibility

Conducting an Integrated assessment

Includes the employee’s motivations that ultimately decides their capability and willingness to succeed in that particular job role.

Motivations and Interests

What are some of the Good practices in talent evaluation?

icons.png

Make suitability efficient

 The employee might give fake answers to fit in the desired suitability aspects. 

Some recommendations: 
Cross referencing Ranking rather than multiple choice questions,  include positive statements, so that all statements are regarded as desirable by the employee, considerably minimizing the likelihood of them giving fraudulent responses.

Know your workforce

Adapt the talent assessment tool according to the technology literacy of the workforce.

For instance, an aging workforce might not be very comfortable with high tech tools as compared to the younger generations, and enforcing a digital talent assessment might end up putting this chunk of the workforce at a disadvantage. 

Remove employer biases

Quantify the results of the skill assessment to make employer evaluation more fair. Because generally employers tend to be partial towards employees who are more like them in terms of personality.

Thus, to get rid of this bias, suitability tests can be evaluated in numbers.

icons - Copy.png
Untitled-3.png

Technological Literacy

Apply AI in ways that are simple to understand and intuitive to use for an ageing workforce or non frequent users of technology.

icons 1 - Copy (2).png

Time Factor

AI opens up a wide variety of possibilities to evaluate the employee on,  but this shouldn't translate into an excessively long review. 

Factors to consider when pairing AI with talent assessment

icons 1 - Copy.png

Fluidity of job roles

 Industry trends are blurring the divide between different job roles implying fluidity in the skill requirements for these job roles.

icons 1 - Copy (3).png

Skepticism of managers

 Managers are skeptical about the accuracy of the AI recommendations, and hence, do not fully utilize them.

Reduced diversity in workforce

AI based tools work on a fixed set of job characteristics to determine the potential of its candidates. For instance, if a candidate applying for a position possesses a non-typical work experience such as where they didn’t work in that particular job role per say, but worked in other related job roles that helped them build relevant skills pertaining to this role. The AI tool might end up vetting those candidates out, thus, making the recruitment process biased

Recommendation

icons - Copy.png

Ethical considerations for using AI based talent assessment tools

Algorithmic bias

The assumption that AI tools are always free of biases, since they derive results from large data sets does not always hold true.

The fairness of these tools is also determined by the type of data we are feeding them with.

Recommendations

Exclude parameters such as gender, age, cultural and social identities from the data sets​

Incorporating a feedback system to increase transparency for negative evaluation.

AI based talent assessment tool can expand its database. Instead of solely relying on the skills of previous employees to do the profiling for a job role, the tool can extend to other suitability and eligibility criteria that might be relevant to that job role.

icons - Copy.png

Ethical considerations for using AI based talent assessment tools

Privacy Concerns

integrating talent assessment with AI demands the employees to put in-depth accounts of their work performance on a digital platform, or provide details into their personal life such as their social media accounts, and so on. This might end up putting the organization at a much higher power than the employee, where the employees personal data can be used for anything

Recommendation

More transparency could be provided to the employees. This implies that all the data that needs to be gathered for an employee's evaluation should be validated with the purpose for which it shall be used in the talent assessment.

Accuracy

AI based tools work on a fixed set of job characteristics to determine the potential of its candidates. For instance, if a candidate applying for a position possesses a non-typical work experience such as where they didn’t work in that particular job role per say, but worked in other related job roles that helped them build relevant skills pertaining to this role. The AI tool might end up vetting those candidates out, thus, making the recruitment process biased

Recommendation

Talent assessment tools should be requested to make the working of their tools transparent so that the organizations can understand the logic behind the recommendations of these tools.
user interviews graohic.png

User Interviews

User Interview Goal

What do stakeholders think about the usability of the current assessment software?
What are the flaws found in the current solution?
What kind of solutions are they looking for or expecting?

Defining the Job Roles

icons 2 - Copy.png

Engineering Manager

The engineering manager would be the head of the app development team. Since their main role lies in developing and testing out the tool we aim to gather where the difference is lying to further propose points that can help make the revamped tool free of those errors.

icons 2.png

Sales/ Marketing Manager

Marketing manager includes users from the marketing and sales department of the company. By interviewing this user group, we aim  to analyze market patterns about the particular aspects of the software that worked well or didn’t work well in the market.

Application User

icons 2 - Copy (2).png

Application user is the group who directly interacts with the application. Their challenges are the priority that the client should consider. By interviewing application users, we can find some misunderstandings between the client and the users, as well as some problems that the client has not seen.

Interview Protocol

Overarching Question

icons 2 - Copy.png

From a technical viewpoint what’s the current assessment of the software?
What are the technical flaws found in the current solution?
What kind of improvements can be made?

Structure of Questions

Engineering Manager

Software Development

Problems/ Challenges

Potential solutions/ MVP's of product

  • technological limitations of the tool

  • What features were recently incorporated and why?

  • challenges in the current version of the talent assessment tool

  • Features they would like to retain

  • Suggestions to make the tool intuitive

Overarching Question

icons 2.png

What do marketing and sales managers think about the usability of the current assessment software?
What are the flaws found in the current solution?
What kind of solutions are they looking for or expecting?

Structure of Questions

Sales/ Marketing Manager

Software
Usage

Problems/ Challenges

Potential solutions

  • Features of the tool used & preferred by clients.

  • Features of the tool that are underutilized by clients.

  • challenges in the current version of the talent assessment tool from a sales perspective

  • Features they would like to retain

  • Suggestions to make the tool intuitive

Overarching Question

icons 2 - Copy (2).png

Application User

What do application users think about the usability of the current assessment software?
What are the flaws found in the current solution?
From the perspective of a user, what kind of improvements are they looking for or expecting?

Structure of Questions

Software
Usage

Problems/ Challenges

Potential solutions

  • Steps and process followed for using the tool.

  • Commonly used features of the tool.

  • challenges in the current version of the talent assessment tool from a sales perspective

  • Features they would like to retain

  • Suggestions to make the tool intuitive

user qoutes.jpg

Affinity Mapping

The findings derived from the interviews were turned into self contained assertions. these assertions were arranged into broader themes, (orange sticky notes). These themes were further clubbed under parent themes (denoted by blue notes). By doing this, we got a good sense of what issues we had to focus on for the next stage.

K010J Young Wolverines (2).jpg

Recommendations

Work in Progress

bottom of page