COMPUTERS software may accidentally sort job applications based on race or gender

  • Lots of firms now use computer software to sort through job applicants
  • Computer scientists at the University of Utah have devised a test, which reveals whether an algorithm could be biased like a human being
  • If the test can predict a person’s race or gender based on hidden data in their CV there is a potential problem for bias, but it can be fixed
  • Currently it’s a proof of concept but could one day be used by recruiters 

Sarah Griffiths for MailOnline

You might think that computer software operates without bias because it uses code to reach conclusions.

But scientists have found that programs used to sort job applications and loans could be just as prejudiced as humans.

Machine learning algorithms used in some CV scanning software may make generalisations based on data in an application that accidentally mimic human discrimination based on race or gender, they warn.

Worrying: Scientists have found that programs used to sort job applications and loans could be just as prejudiced as humans. A stock image of a man working on a loan application is shown

Worrying: Scientists have found that programs used to sort job applications and loans could be just as prejudiced as humans. A stock image of a man working on a loan application is shown

Today it’s common for companies to use software to sort through job applications, which sometimes look for key words or grades to make the recruitment process faster.

But there’s growing concern that such programs don’t just identify unsuitable applications, but also discriminate against people.

The problem could occur if software learns how to analyse data beyond a person’s key skills or grades, for example. 

A team of computer scientists at the University of Utah, University of Arizona and Haverford College in Pennsylvania have devised a test, which reveals whether an algorithm could be biased like a human being. Suresh Venkatasubramanian, who led the research is pictured

A team of computer scientists at the University of Utah, University of Arizona and Haverford College in Pennsylvania have devised a test, which reveals whether an algorithm could be biased like a human being. Suresh Venkatasubramanian, who led the research is pictured

FACEBOOK ADDS GENDER CONTROLS TO ONLINE VIDEOS 

Facebook is giving users more control over who can see their videos and for how long. 

Page owners can now select which age groups and genders can view the clips, as well as set the videos to private. 

The age restrictions could prevent young children seeing adult content, for example, but it is not clear why Facebook has added gender controls. 

It could mean videos about products aimed at women, such as sanitary products, don’t appear as irrelevant posts in the news feed of men. 

But in theory the move could promote sexism by letting brands or groups from seeing videos based purely on their gender. 

Similar options are already available to Page owners where they select which ages, genders and locations see their promoted adverts.

Machine learning algorithms can change and adapt like humans so they can better predict outcomes.

For example, Amazon uses similar algorithms to learn about its customers buying habits so it can show them the most tempting adverts.

However, too much learning could prove unhelpful when sifting through applicants.

Suresh Venkatasubramanian, an associate professor in the University of Utah’s School of Computing, said: ‘The irony is that the more we design artificial intelligence technology that successfully mimics humans, the more that AI is learning in a way that we do, with all of our biases and limitations’

A team of computer scientists at the University of Utah, University of Arizona and Haverford College in Pennsylvania have devised a test, which reveals whether an algorithm could be biased like a human being. 

The technique determines if software programs discriminate unintentionally and violate the legal standards for fair access to employment, housing and other opportunities.

Professor Venkatasubramanian presented the team’s findings last week at the Association for Computing Machinery’s SIGKDD Conference in Sydney, Australia and showed it’s possible to test for bias and well as fix the problem. 

If the test can predict a person’s race or gender based on the data being analysed, even though race or gender is hidden from the data, then there is a potential problem for bias based on the definition of disparate impact. It's intended to make the job application process fairer. A stock image of interviewees is shown

If the test can predict a person’s race or gender based on the data being analysed, even though race or gender is hidden from the data, then there is a potential problem for bias based on the definition of disparate impact. It’s intended to make the job application process fairer. A stock image of interviewees is shown

If the test, which uses another machine-learning algorithm, can predict a person’s race or gender based on the data being analysed, even though race or gender is hidden from the data, then there is a potential problem for bias based on the definition of disparate impact.

‘I’m not saying it’s doing it, but I’m saying there is at least a potential for there to be a problem, the Professor said.

If it detects a possible problem, Professor Venkatasubramanian said the data can be redistributed to prevent the algorithm from seeing any information that can be used to create the bias.

‘It would be ambitious and wonderful if what we did directly fed into better ways of doing hiring practices. But right now it’s a proof of concept,’ he said.  

COULD WEBSITES BLOCK USERS ACCORDING TO RACE? 

Websites could soon block people from accessing their content due to their race or gender following the development of software that checks the genetic profile of users.

The Genetic Access Control application works by requesting access to information compiled by DNA analysis firms like 23andMe before allowing users can access a website.

This would allow the website to check the user’s ethnicity or gender to limit or deny them access, according to the programme’s creator.

The software, which has already been blocked by 23andMe, relies upon the user having already had their DNA profile analysed by genetic firms. 

The Genetic Access Control could have had some uses such as helping to ensure individuals on dating sites are the gender they claim to be and could also have helped provide ‘safe places’ for female only ‘victim groups’.

The Genetic Access Control used information stored by 23andMe to grant access to online content. Although now blocked by the company, it would have allowed websites to limit or deny access to people according to their ethnicity or gender

The Genetic Access Control used information stored by 23andMe to grant access to online content. Although now blocked by the company, it would have allowed websites to limit or deny access to people according to their ethnicity or gender

However, the creator says it could also have been used to restrict access to ethnoreligious sects, like Jews who carry specific maternal genes.

Writing on the collaborative software repository Github, the creator of the software said: ‘Using the 23andme API it is now possible to utilize genetic profile information and likely phenotypes in custom applications.

‘This means you can restrict access to your site based on traits including sex, ancestry, disease susceptability, and arbitrary characteristics associated with single-nucleotide polymorphisms (SNPs) in a person’s genotype.’

Genetic testing is growing in popularity as costs for the analysis have plummeted to around £125 for a kit.

Saliva samples can be sent off to commercial firms which then analyse them for sequences that indicate ancestry and genetic predisposition to certain diseases.

This information is then posted into a personal profile, accessed via a username and password, online. This information can be shared with third parties by the user.

The Genetic Access Control exploited this by requesting access to the genetic information and using a temporary access ticket to look for specific genetic information.

However, 23andMe said it blocked the app immediately after it was released.

It said it had a policy to not allow access to its profiles from applications that build ‘hate materials or materials urging acts of terrorism or violence’.

A spokesman for 23andMe said: ‘This app clearly violates our API policy. We’ve shut down the application and this developer no longer has access to our API.




Source: COMPUTERS software may accidentally sort job applications based on race or gender

Via: Google Alert for ML