New Zealand’s Privacy Commissioner has today announced the trial by Foodstuffs North Island of live facial recognition technology to have been in compliance with the Privacy Act, but improvements needed.
Privacy Commissioner Michael Webster has found that the live facial recognition technology model trialled in 25 Foodstuffs North Island (FSNI) supermarkets is compliant with the Privacy Act.
His Inquiry report released today, however, shows that any business considering or using FRT needs to make sure it sets things up right to stay within the law.
“While the use of FRT during the trial was effective at reducing harmful behaviour (especially reducing serious violent incidents) it has also shown that there are many things that need to be taken into account,” said Mr Webster.
“FRT systems have potential safety benefits, but they do raise significant privacy concerns, including the unnecessary or unfair collection of people’s information, misidentification, technical bias which can reinforce existing inequities and human bias, or the ability to be used for surveillance”.
“These issues become particularly critical when people need to access essential services such as supermarkets. FRT will only be acceptable if the use is necessary and the privacy risks are successfully managed”.
The Privacy Commissioner’s Inquiry into Foodstuffs North Island’s trial use of live FRT set out to understand its privacy impacts, its compliance with the Privacy Act, and to evaluate if it was an effective tool in reducing serious retail crime compared with other less privacy intrusive options.
“There is still some work to do to increase the safety and effectiveness of FRT software use in the New Zealand context, as FRT technology has been developed overseas and has not been trained on the New Zealand population.”
Privacy safeguards make the difference
The Inquiry found while the level of privacy intrusion was high because every visitor’s face is collected, the privacy safeguards used in the trial reduced it to an acceptable level.
“Foodstuffs North Island designed the privacy safeguards used in the trial with feedback from my Office,” said the Privacy Commissioner. “This has provided some useful lessons for other businesses which may be considering using FRT.”
Key privacy safeguards in place during the trial included
- Images that did not result in a positive match were deleted immediately, as recommended by OPC – this meant there was very little privacy impact on most people who entered the trial stores.
- The system was set up to only identify people who had engaged in seriously harmful behaviour, particularly violent offending.
- Staff were not permitted to add images of children or young people under 18, or people thought to be vulnerable, to the watchlist.
- There was no sharing of watchlist information between stores.
- During the trial, the operational threshold that triggered an FRT alert was raised from 90% to 92.5% likelihood of the images matching, reducing the chances that people would be misidentified while managing down the “computer says yes” risk.
- Match alerts were verified by two trained staff, ensuring that human decision making was a key part of the process.
- Access to the FRT system and information was restricted to trained authorised staff only.
- Images collected were not permitted to be used for training data purposes.
- Systems were reviewed and improved during the trial where misidentifications or errors occurred.
Improvements needed
“There is still some work to do to increase the safety and effectiveness of FRT software use in the New Zealand context, as FRT technology has been developed overseas and has not been trained on the New Zealand population,” he said.
“As a result, we can’t be completely confident it has fully addressed technical bias issues, including the potential negative impact on Māori and Pacific people. This means the technology must only be used with the right processes in place, including human checks that an alert is accurate before acting on it.”
“Some improvements will also need to be made by FSNI before the use of FRT is made permanent or expanded to more stores. These focus on ensuring the documented processes and system settings are updated to match what happens in practice, including ongoing review of the use of FRT to make sure its use is justified as an effective tool for reducing serious harm offending.
“I also expect that Foodstuffs North Island will put in place monitoring and review to allow it to evaluate the impact of skin tone on identification accuracy and store response, and to provide confidence to the regulator and customers that key privacy safeguards remain in place.
“The trial findings will help other businesses to ask the right questions about whether FRT is necessary and appropriate for them and to understand what they would need to do to set FRT up and run it in a privacy protective way.”
Minister welcomes result
The Government has welcomed the report, with Justice Minister Paul Goldsmith calling it “great news”.
“[The report] found the technology is effective at reducing harmful behaviour towards retailers, especially serious violent incidents,” he said. “This is great news for businesses that are considering using the technology as a means to protect their livelihoods.
He added that the report notes that privacy concerns must be carefully safeguarded.
“I expect our Ministerial Advisory Group will continue to look at this technology as an option to be used more widely and engage with the sector on it. I’ll be encouraging the MAG to take this report into serious consideration.”
The FSNI FRT trial started on 8 February and ended on 7 September 2024 and was conducted in 25 supermarkets. During the trial, 225,972,004 faces were scanned (includes multiple scans of the same person). 99.999% of these deleted within one minute, and there were 1,742 alerts of which 1,208 were confirmed matches. OPC is currently developing a Biometric Processing Privacy Code, which applies to biometric information, including a photo of someone’s face used in a Facial Recognition System. The new Code is expected to be published in mid-2025.
Be the first to comment