'The bot will see you now': why AI could be the new HR

Ever wondered if you appear nervous in job interviews, if you speak too fast or if someone will realise you have fudged experience on your resume?

In the future, generative artificial intelligence technology will be able to tell ... but it will alert your potential employer first.

AI tools are increasingly being deployed in recruitment and human resource fields, with one Australian start-up already ranking potential job candidates based on pre-screening interviews hosted by AI software.

But these uses have been identified as potentially high-risk in the federal government's consultation into mandatory guardrails for AI, and technology experts say its deployment could be "troubling".

If the technology is used without appropriate human oversight to correct bias, as it has been in some cases overseas, they warn lawsuits will follow.

Human resources has been a popular target for technology for years but its use has ramped up following the development of generative AI.

Brisbane start-up Zeligate launched its technology after identifying its potential to assist in the recruitment process both in Australia and abroad.

"We saw a hiring assistant as having global application because, obviously, resumes, interviews, all of that is pretty much a standard process wherever you go in the world," director Rob Olver said.

"There's a lot of high quality candidates that apply for jobs but (the) business hasn't got enough resource or time to respond to everyone, so a lot of people just drop out of the hiring process."

The company, which has raised $5.5 million in seed funding, uses its AI recruitment assistant Zeli to score job candidates based on their resumes and how they perform in video interviews hosted by AI.

A score out of 100 and a transcript is delivered to the recruiter following the AI interview and candidates are judged, based not only on their experience but their mannerisms.

"We do a little bit of sentiment analysis where we assess the confidence of their voice, whether they're speaking too fast, their general demeanour," he said.

"As we move further into the technology, we'll begin to introduce Myers-Briggs and other (personality) assessment tools that the candidate can go through."

The AI assessments do not ultimately decide whether someone gets hired, Zeligate chief executive Denver Naidoo says, as recruiters can read the interview transcript or watch its recording to look for "some kind of charisma or star quality that isn't being measured".

But he says one of the AI tool's biggest benefits is its focus on facts over feelings.

"What's really good is that there's no bias - it's based on maths," Mr Naidoo told AAP.

"It only works on the facts so if you're looking to hire someone with five years' experience in marketing and they only have three, it's very clear that they don't meet the requirements exactly and these are the kinds of facts that Zeli would speak to."

But the use of AI in employment has been identified as risky in other countries and flagged as a potential target for mandatory guardrails in the government's Safe and Responsible AI consultation paper.

The review, which is open to submissions until October 4, says organisations should carefully consider the risk of bias before using AI to make hiring assessments.

"For example, an automated CV-scanning service that makes a determination of an individual's suitability for a job would be considered a high-risk system," the consultation paper said.

"This is because it has the potential to impact a person's access to work as well as discriminate against certain groups."

One example cited in the paper is an AI tool that was used by Amazon until it discovered the software was discriminating against female candidates.

RMIT University's Centre for Human-AI Information Environments director Lisa Given says the AI tool developed its bias after learning from the company's previous hiring patterns.

"If a dataset has an in-built bias – in this case, a very high percentage of applicants who are male - the system is just replicating that when it starts to filter out future applications," she said.

"It would have to be effectively trained to ignore gender."

Tech-based discrimination could also apply to a candidate's location, race, cultural background or age, Prof Given says, and could affect promotion opportunities or training if AI tools were used in other human resource areas.

Before deploying AI tools, she says, companies need to ensure they are being created and overseen by more than just software developers.

"It can be done safely," Prof Given said.

"We need to work with a team that has IT people, HR folks and people who understand equity issues."

University of NSW AI Institute chief scientist Toby Walsh says employment is one field that should be classified as high risk due to the "significant consequences" of AI systems making poor decisions.

"The use of AI in HR troubles me and I think it should be troubling a lot more people than it does," he said.

While existing laws should guard against discrimination, he said companies considering AI employment tools should use them only to assist human workers and ensure people were ultimately held responsible for their decisions.

"I do fear that we're actually going to see some really significant, serious, class-action lawsuits in the next year or two against people using HR systems for recruitment," he said.

"Whenever we make critical, consequential decisions that impact on people's lives, whether it be HR or immigration or welfare or medicine, we ultimately need a human in the loop."