Ryan Zhou, Warwick Business School finalist in the CoBS 2024 Student CSR Article Competition, takes an in-depth look at how AI recruiting software works, its pros and cons, and how legislation is attempting to safeguard candidates’ rights.

AI in Hiring Decisions: Electronic efficiency vs emotional intelligence by Ryan Zhou.

AI has many uses, from writing reports and essays with the assistance of ChatGPT to automating simple code writing with Github’s CoPilot. In fact, I used ChatGPT to help come up with this title. In these cases, the potential harm is quite limited, but what about when the stakes are much higher, for example, in healthcare or hiring decisions?

The thought of some unknown algorithm calculating whether you deserve your dream job with an abstract algorithm is a scary thought, and you would be right to think so. However, the reality of virtual interviews, psychometric testing and AI CV screening is already a reality of today. Around 65% of recruiters already use AI in the recruitment process, compared to only 34% of candidates that suspect that AI was used in their recruitment process (Stefanowicz, 2024). To mitigate rising concerns, New York City passed a law in 2023 that allows job seekers to opt out from having their CVs and job applications from being reviewed by AI. A nice touch, sure, but I will subsequently address why this may fall short of its intended purpose.

This article will help to provide more clarity on the current usage of AI, flaws in the current system and what is being done to mitigate these issues.

Hiring Decisions

Why do you want to work at xxx? You have 90 seconds to record your response… For those applying to new roles, the video interview stage is a common occurrence. HR supply a set of questions and expected response times, then the software feeds you the questions and records your responses. Despite a handful of firms having human involvement in assessing these responses, for many firms this is impractical and time-consuming. Their solution is to employ AI to analyse your video response, assessing it on body language, key word usage, tone and of course the content of your response. Hirevue is a firm name that has become synonymous to the word video interview, especially amongst banking applicant pools. Firms offering the highest paying entry level roles in the world, such as JP Morgan and Point72, are just several of the 1,593 firms using Hirevue (6sense, 2024). As a barrier to these lucrative career paths, interviewees have natural concerns about the potential for bias, lack of explanation of results and the technology itself.

Lost in Translation: Word error rates

Microsoft classifies a good speech-to-text (STT) model to have a Word Error Rate (WER) of 5-10% and advertises its own WER at 5.1%, in comparison, Google’s is 4.9% and a human transcriptionist’s is 4% (SmartAction, 2021); this has been corroborated by third parties (Ferraro et al, 2023). However, if you were to take any book or article you have at hand and read a section to Apple or Google’s speech-to-text- software, you will likely find a much larger number of errors, perhaps several rather egregious ones. This due to advertised WER being benchmarked against clean datasets, i.e no background noise. What happens if you take these out of the lab environment and introduce background noise similar to a quiet library (40dB)?

Google’s WER soars to 20%, Microsoft’s increases to 11.11% (Xu et al, 2021) and it would be reasonable to believe that background noise of a library would have little to no impact on a human transcriptionists’ WER. Without delving deeper into STT evaluation metrics, it is apparent that video interview AIs are unlikely to get a clean picture of your speech and given the nuanced nature of English language, much less the meaning. Hence, interviewees’ concerns are seemingly well founded. However, Nathan Morndragon, Hirevue’s chief psychologist mentioned in a 2019 interview, that the system even picks up details as granular as ‘false bravado, memorized answers and abnormal levels of eye contact’. (efinancialcareers, 2019). So perhaps, the actual content of the speech has less weighting than an applicant might think…

Nevertheless, given this material error rate, how do companies like Hirevue actually train and calibrate their AI to process applicants for the roles its clients offer? In the same interview, Nathan Morndragon mentions how they examine the competencies and behaviours of the employer’s top performers and then builds an ideal job profile on which candidates are assessed against. However, is the sufficient, or does this pose substantial risks to the employer?

AI’s Potential to Discard Top Talent

To identify whether an AI selection model works well, we first need to identify the wishes of the employer. Whilst many firms now use AI in their hiring process, rarely does AI dictate the entire application process. According to a survey on HR professionals, 43% said that the most challenging task is applicant screening (Stefanowicz, 2024). This is represented accordingly with majority of AI use cases as the CV and video interview screening stages, with human interviews at subsequent stages of the process. Employers use AI to narrow down the applicant pool as much as possible, with the main risk of AI rejecting talented individuals without a human set of eyes ever reviewing the application.

Here, we are more willing to make Type 1 errors than Type 2 errors. In layman’s terms, it is better to include an incompatible interviewee into the short list than incorrectly deem a potentially star employee as incompatible. The idea is that employers may reject incompatible applicants in later rounds, however, will never recover the talent they potentially missed. In Machine Learning, what we would look at is the Receiver Operating Characteristic Curve (ROC), which plots the true positive rate against the false positive rate (blue dotted line in figure 1).

An employer can alter their desired characteristic thresholds for keyword usage, eye contact and so on, to allow for a certain rate of false positives. The grey area under the ROC curve is aptly abbreviated as AUC and can be interpreted as the probability that the AI ranks a good candidate above a poor candidate. Employers should aim to maximise the AUC and have the opportunity from its AI provider to constrain their false positive rate to a board-approved threshold. Moreover, it may be beneficial for future AI powered recruitment processes to display their false positive rates and AUC to improve transparency and promote better model classification.

Amazon’s Fulfilment Flop

When using AI in screening applicants, the major reputational risk stems from bias or discrimination by the AI. Amazon began developing an AI tool in 2014 to review CVs and rank applicants, but by 2015 found that the AI was, put bluntly, sexist. Amazon used a model with unsupervised learning to spot patterns in CVs submitted to the firm over the previous 10 years. However, most of the applicants were male, hence the AI taught itself to penalise CVs with the words ‘Women’ and discriminated graduates from all women’s colleges (Dastin, 2018). Amazon discovered the discriminatory bias in 2015 and endeavoured to remedy the issue, however, they disbanded the project in 2017, failing to fulfil its original target.

The main takeaways from Amazon’s attempt are that the training dataset is incredibly important and there needs to be some level of testing to detect bias. However, latest tools such as Hirevue, compare applicants to characteristics of the top performing candidates, as previously mentioned. At large firms, there can be an element of politics in determining promotions and identifying top performers. Calibrating the model to these top performers makes it susceptible to the inherent bias management may have, hence, not irradicating the possibility of bias.

There may also be implicit discrimination during the hiring process against those with greater strengths in soft skills that are unquantifiable. For the fans of the TV show, suits, I am sure you can remember when an employee was tasked with implementing cost cutting measures for the firms and devised an algorithm to rank employees. She was about to fire the bottom performing employee before it was brought to her attention that the top performers on her list all sat around him. Hence, it is also vital to use more holistic datasets when training their AI models.

In the current age of social media, reputational risk for firms’ activities are increasingly large. Well-meaning activities such as Bud Light’s 2023 Advertising campaign, knocked their place as the #1 selling beer in the US, with US sales down 26.1% (Torrenzano, 2023) and Bud Light’s producer Anheuser-Busch InBev lost $27 billion in market capitalisation in the subsequent months (Thaler, 2023). If the aforementioned fiasco was to be brought to light in the current environment, it would likely have a larger impact on Amazon than in 2014. Firms need to be careful about the impact of their automation efforts and regulators are closely following…

The Spearhead in AI Regulation

New York’s Local Law 144, detailing regulation on employment related AI, was passed in 2021, and is now enforceable since 5^th July 2023 (Mobley, 2023). One of the first AI laws in the US, it states that firms using AI in the hiring process need to submit themselves to annual independent audits to prove absence of bias, and each individual violation yields a potential fine of $1,500 (Ryan-Mosley, 2023). Moreover, firms are required to disclose to applicants that AI is being used and provide an option to request an alternative selection process, if available.

However, this is an empty rhetoric, as it doesn’t require the employer to do so. This now becomes some sort of Prisoner’s Dilemma Game, where people may opt in or out. Unfortunately, everyone opting out is not a Nash Equilibrium. Firms would much rather minimise costs and use AI for screening purposes, so opt in applications would be reviewed before those who opted out, posing a significant advantage for rolling deadline applicants.

Despite its flaws, it is a step in the right direction and requires firms to better understand its technology and possible resulting discrimination while hiring, even if they do not necessarily have to do much. Moreover, given many international firms are headquartered in New York, firms are likely to use the same AI for all their hiring processes, which benefits applicants globally.

Gold at the End of the Rainbow?

AI clearly has a long way to go before full autonomy, but it is irrefutable that it is currently useful to access wider talent pools and minimise hiring costs. It also benefits applicants who are currently employed, by providing them with the flexibility of completing interviews at a convenient time, rather than dodging away during the workday.

Whether AI based systems in hiring are more or less biased than fully human run processes is difficult to measure and conclude. However, the increased pickup in AI based application screening for the last 4-5 years indicates that AI is performing well. If there are no apparent drawbacks to this solution and allows firms to increase their chances of finding a good match, then it would be crass to suggest firms to stop.

Regardless of its performance relative to humans, we need to ensure that the datasets that we feed to our AI models are without implicit bias and that we provide better transparency to applicants on the application process. Finally, regulatory bodies need to ensure they maintain their stance on AI regulation to ensure that firms take suitable steps to fully understand the technology they use and avoid repercussions to themselves and job-seekers. Given these terms, maybe, just maybe, we get to the gold at the end of the rainbow…

References used in this article.

Useful links:

• Link up with Ryan Zhou on LinkedIn
• Read this article and other student articles on Global Voice magazine #30
• Read a related article: People Analytics: The risks behind the promises
• Discover Warwick Business School
• Apply for the Warwick MBA.

Learn more about the Council on Business & Society

The Council on Business & Society (The CoBS), visionary in its conception and purpose, was created in 2011, and is dedicated to promoting responsible leadership and tackling issues at the crossroads of business and society including sustainability, diversity, ethical leadership and the place responsible business has to play in contributing to the common good.  

• Follow the CoBS on X and LinkedIn
• Download this magazine and others from the CoBS website downloads page.

Member schools of the Council on Business & Society.

• ESSEC Business School, France, Singapore, Morocco
• FGV-EAESP, Brazil
• School of Management Fudan University, China
• IE Business School, Spain
• Keio Business School, Japan
• Monash Business School, Australia, Malaysia, Indonesia
• Olin Business School, USA
• Smith School of Business, Queen’s University, Canada
• Stellenbosch Business School, South Africa
• Trinity Business School, Trinity College Dublin, Ireland
• Warwick Business School, United Kingdom.