Back to Articles
I Tricked an AI Age-Verification Tool into Letting a Child 'Buy' a Knife

Crikey

SKIPPED

Details

Date Published
14 June 2024
Priority Score
3
Australian
Yes
Created
8 Mar 2025, 02:41 pm

Authors (1)

Description

Governments are looking at using AI to estimate people's age using a selfie. I fooled it using a stock image and an old aging filter.

Summary

The article reveals vulnerabilities in AI-based age verification systems, specifically those using facial analysis technologies. The author demonstrates how these systems can be fooled using simple photo-editing techniques, raising concerns about their reliability and the potential risks they pose if misapplied. While intended to prevent underage access to restricted items, such as knives, the flaws highlighted suggest these systems may inadequately protect against malicious manipulation. The analysis has implications for global AI safety policy, especially in the context of Australia's ongoing discussions about age verification methods. The findings emphasize the need for robust AI governance frameworks to ensure these technologies do not inadvertently facilitate access to dangerous items or content by minors.

Body

NOTE: The Press Council has decided this article has breached its Standards of Practice. Read the full adjudicationhere.A demo of a tool used to estimate age using facial analysis that is being promoted as a way to stop underage children from accessing social media or online pornography can be fooled using an aging filter on a popular photo-editing app.Raising the minimum age for social mediahas public support, as governments around the world arestrugglingwith ways to restrict children from accessing restricted content online because of theprivacyconcerns about uploading a government identification to prove their age.Yoti is among the world’s best-known companies for handling online age and identity verification. Its technology is already being used by some of the world’s most popular platforms including Meta — owner of Facebook, Instagram and Threads — and hasworked closelywith the Australian government as it worked on age verification policy.One of Yoti’s methods for age estimation, which Opposition Leader Peter Duttonspruikedjust yesterday — is “facial analysis”, which can calculate someone’s age using a selfie. Yoti says it doesn’t retain these selfies after analysing them, promising this technology offers a way of proving someone’s age without recording their identity. Meta isalready using thisin Australia ahead of a global rollout.Critics argue thatage estimation through facial scansis flawed and vulnerable to being tricked. Yoti offerssome measuresto stop people from fooling its system, but it doesn’t change the underlying technology’s reliance on easily falsifiable information.Related Article Block PlaceholderArticle ID: 1161621AI bot with ‘IQ of 155’ to advise real estate agents on ethicsCam WilsonI know because I tricked Yoti’s age estimation into letting me “buy” a fixed-blade knife using the photograph of a 10-year-old that I had put through an aging filter on a photo-editing application.How a Google image search and a free app fooled age verification techYoti offers anonline demonstrationof its technology that allows anyone to test the ways people can prove their age online. The company notes its “anti-spoofing measures” aren’t enabled in the demo (more on this later).The demo is of a fake online store that includes products including a knife, a large drone, alcohol and vapes that are restricted by age. When trying to buy them, users are prompted to confirm their age by uploading their ID, using the Yoti app or through its facial analysis technology.I selected the selfie option. The website accessed my webcam and asked me to put my face in the camera. I found astock imageof someone who is described as a “9- to 10-year-old girl”, pulled it up on my phone and held it up to the camera. Yoti recognised the face, identified it as someone who was younger than 13, and wisely declined my knife purchase.I then downloaded FaceApp, a free, popular photo-editing smartphone application that was launched in 2016 with millions of users, and used it to apply an “old” filter to the child’s image. Using the subsequently aged image, I went through the same process.Except this time, Yoti said I — well, this aged-up child — was old enough to buy the knife. And in another attempt, I was able to purchase a drone.A different attempt, this time to buy a drone, using the images on Yoti (Image: Supplied)As mentioned, Yoti has worked on how to stop people from exploiting its system. A spokesperson for Yoti, Nikki Alvey, toldCrikeyin an email about its various anti-spoofing measures including liveliness detection methods, which try to catch people using images or even a mask, and technologies that can detect if someone isn’t really using their selfie camera. Yoti’s customers can use these measures or choose to provide their own.I wasn’t able to test these measures in the demo and have asked Yoti for access to a demonstration.The company reports it is 96.99% likely to correctly identify that a child aged betweensix and 12 years old is under 13 years of age. Last year, Yotireportedcorrectly determining whether an image was of a real person 97% of the time after three attempts using its passive MyFace liveness system on a mobile phone, while incorrectly identifying a non-real image as real 1% of the time.Even with these measures, age and identity verification may still be vulnerable to exploitation via methods like real-timedeepfake face-swap technology, which is widely accessible through consumer apps like Snapchat. A 2022 reportfoundthat nine out of 10 liveliness checks were fooled by deepfakes.Yoti’s Alvey said that bypassing its checks is extremely difficult but acknowledged it’s not impossible.Related Article Block PlaceholderArticle ID: 1161505Revealed: Australian police and criminals were guinea pigs for the world’s biggest sting operationCam Wilson“It would require significant resources, time and expertise to beat the system. In most use cases where an age check is required, this effort would outweigh the reward,” she said.While the industry’s attempts to stop people from circumventing their products is a constant battle, the limitations endure regarding the underlying facial analysis technology. Age estimation through facial images remains fraught. Astudypublished this month by Australian researchers raises how these systems use indicators like wrinkles, hair and facial features, which can be altered or are not always indicative of age. This presents a risk to individuals whose age might be incorrectly estimated, restricting them from accessing things that they are legally allowed to access.Age verification through facial analysis is sometimes presented as a way to use technology to protect kids without the privacy drawbacks, which have been the fatal flaw of previous pushes to restrict online content. In practice, the technology is not reliable enough and must still be used in tandem with old school methods. Don’t take my word for it: the other two age-verification methods from Yoti rely on providing real ID.Any policy debate that accepts the hype and ignores the limitations of facial age estimation is doomed — unless you’re fine with 10-year-olds buying knives.Clarification: This story was amended to included the words ‘demo’ and ‘a demo of a tool’ in the headline and opening paragraph.Do you have concerns about proposed age-verification technology? Let us know your thoughts by writing toletters@crikey.com.au. Please include your full name to be considered for publication. We reserve the right to edit for length and clarity.