You've seen it in the movies for years: Security cameras find a face in a crowd, and — Enhance! — a computer comes up with a name. In real life, facial recognition was too error-prone to work that quickly, especially with live video streams.
But now the Hollywood fantasy is coming true.
Facial recognition accuracy rates have jumped dramatically in the last couple of years, making it feasible to monitor live video. China recently announced real-time facial recognition covering "16 provinces, cities and autonomous regions," and similar systems are being tried in Russia, India and even the United Kingdom.
"I've been in this industry since the mid-2000s, and I've never seen people gravitate toward face recognition like the past two years," says Benji Hutchinson, a vice president with NEC Corporation of America, a major supplier of the systems. "It's night and day."
Improved algorithms mean the technology can recognize faces in motion, even when viewed by security cameras positioned at odd angles.
The companies that make the systems say they're now able to compare an image against billions of faces, which makes it technically feasible to track the residents of a whole city — or country.
For autocratic societies such as China, the appeal is obvious. But police in western democracies are also showing interest.
"The question is, has the horse already bolted, with a lot of this technology?" says Tony Porter, the surveillance camera commissioner for England and Wales.
He's a kind of privacy ombudsman, and he worries that British police are running "trials" of real-time facial recognition before elected officials can properly debate what the rules should be.
"Regulation is a year, two years behind the technology," he says. "And that allows bad practice to become embedded."
There are similar concerns in the U.S. Privacy advocates say earlier forms of facial recognition technology have been adopted by American police without serious oversight, and the same may happen with real-time recognition.
For the moment, police in the U.S. seem to be holding off. Los Angeles County, for instance, has one of the most recently-updated facial recognition systems in the country, but it doesn't come with real-time capabilities.
"We don't do video on the move, we don't have cameras... capturing what the public is doing," says Lt. Derek Sabatini, a manager for the Sheriff's Department on the Los Angeles County Regional Identification System. "If there's a video of a suspect then the investigator needs to take a still from that video," which is digitally compared to the county's collection of booking photos.
Sabatini doubts L.A. County will rush to try the instantaneous facial recognition being introduced in Asia and the UK. "We don't want to do anything that the public doesn't want us to do," he says. "Me, personally — I value personal privacy."
The American jurisdiction best known for using facial recognition in the field is San Diego, where for the last few years several local law enforcement agencies have shared a system that allows officers to try to put a name to suspects who aren't carrying photo ID.
"They take a forward-facing picture of the individual from their mobile device," says San Diego County Sheriff's Department CIO Ashish Kakkad, "and that picture gets transferred to our mug shot system," which will return a "top 10" list of possible matches.
That's a far cry from real-time facial recognition, but Kakkad says the companies are eager to sell San Diego an upgrade.
Clare Garvie, who researches police use of facial recognition for Georgetown Law's Center on Privacy and Technology, says she's seen the same sales push.
"Every major company that sells to law enforcement in the U.S., advertises the ability to do real-time," Garvie says. "And we've seen a fair amount of interest on the part of law enforcement in purchasing these systems."
The appeal for American police departments would be more efficient detection of wanted people as they walk down the streets, the same way patrol cars equipped with automatic license plate readers are now able to find stolen cars.
Garvie says a few police departments have sought grants to buy the systems, or try them out.
NEC's Hutchinson confirms that his company has been getting inquiries about real-time facial recognition from American police departments.
"I can't say which ones, but it's been the large cities that you might imagine. It's some of the large coastal areas," Hutchinson says. "Some of those cities have expressed interest because they want to manage crime more effectively, or they want to manage counter-terrorism efforts."
Hutchinson says there are still some technical barriers to real-time facial recognition in the U.S. Cities here have fewer of the required high-definition digital surveillance cameras, especially when compared to China or the UK. That could be remedied with camera upgrades, or by linking facial recognition to the new generation of high-def body cameras worn by police officers.
"I believe the first thing you're going to see is obfuscation of faces," he says, meaning facial recognition that automatically finds the faces of people not implicated in a crime, to blur them.
"The second step is... they're going to look at ways to implement facial recognition against known or suspected terrorists or people that are wanted. And it is a pretty simple leap."
Jonathan Turley, a civil libertarian and law professor at George Washington University, worries that this kind of incrementalism will eventually lead to a "fishbowl society," in which it will be impossible to walk down the street without being identified.
"Unfortunately, it could happen in the United States. There's not a lot standing between instantaneous facial recognition technology and its ubiquitous use by police departments or cities," Turley says.
He says people shouldn't assume the courts will limit police use of facial recognition, especially if the real-time ID systems are first "normalized" in private settings. (NEC says it has already sold real-time facial recognition to private customers in the U.S., though it won't name them.)
Turley says the courts' attitude will likely be determined by people's everyday expectations of privacy, which will erode as people get used to these systems. "As businesses recognize you coming into stores and coffee shops, at what point do our expecations fall to the point that the extension of the government into the area becomes less problematic?"
In other words, when people come to expect facial recognition in the shopping mall, they're more likely to expect it from the police.
You hear the same prediction from the industry — but with a positive spin. Terry Hartmann, V.P. with the German company Cognitec, says they're beginning to sell real-time facial recognition to private buyers in the U.S., and he says as the public becomes aware of that, it'll be easier for the police to follow suit.
"If organizations like casinos [and] security at stadiums are putting in this technology at a level that embarrasses the police, where they don't have it themselves, that puts pressure on them," he says.
And Hartmann says people will demand that police start using the kind of real-time facial recognition that their fictional counterparts have long been using on TV.
MARY LOUISE KELLY, HOST:
Police in the U.S. have been using facial recognition software for years, usually after a suspect is caught on camera during a crime. Now real-time facial recognition is on the horizon. In China, authorities are touting a new system's ability to spot people as they're walking down the street. Similar software is being tried by police in Russia, in India, even the United Kingdom. So when might it reach American streets? NPR's law enforcement correspondent Martin Kaste went to find out.
MARTIN KASTE, BYLINE: This is connect:ID. It's a convention for the biometrics industry that's held in Washington. And the vendors' exhibit hall is just what you'd expect. Everywhere you look, there are big screens with live views of your face as you go by as computers track you and categorize you by age and sex. If they're connected to the right database, they could also guess your name. Terry Hartmann is at the booth for a German company called Cognitec. And what's new, he says, is how good these systems are getting at recognizing faces in real-world conditions.
TERRY HARTMANN: You can see all of these matches are different poses. The people aren't facing the camera straight on. You've got people with glasses. You've got the lady looking down. She's matched looking in a different direction.
KASTE: Link that to a national photo database, and it's pretty much the end of anonymity in public places. That's clearly the appeal in autocratic societies like China. But these systems are also being pitched to Western governments. Police in the United Kingdom are now scanning crowds for known troublemakers or wanted criminals. And the same tech is being offered to American police, says Clare Garvie. She tracks this issue for the Center on Privacy and Technology at Georgetown Law.
CLARE GARVIE: Every major company that sells to law enforcement in the U.S. advertises the ability to do real time. And we've seen a fair amount of interest on the part of law enforcement in purchasing these systems.
KASTE: American police aren't buying the real-time systems yet, but Garvie says a few departments have requested funding to try them out. If police are hesitant, it's because the systems are pricey, and there's also the risk of a public backlash. But when it comes to legal barriers, that's less of a concern.
JONATHAN TURLEY: There's not a lot standing between instantaneous facial recognition technology and its ubiquitous use by police departments or cities.
KASTE: Jonathan Turley is a civil libertarian and law professor at George Washington University. He just gave a speech here at this convention appealing to the industry to set up better privacy protections. That's because he's not sure he can count on the courts to limit how police use facial recognition, especially if Americans get used to this tech in other settings such as on their phones or in shopping malls.
TURLEY: As businesses recognize you coming into stores and coffee shops, at what point do our expectations fall to the point that the extension of the government into the area becomes less problematic?
KASTE: In other words, why would Americans expect not to be scanned by the police if they come to expect it from, say, the customer loyalty program at their favorite coffee chain? This prediction about the gradual normalization of facial recognition is something you hear from the industry, too. Here again is Terry Hartmann from Cognitec.
HARTMANN: If organizations like casinos, security at stadiums are putting in this technology at a level that embarrasses the police where they don't have it themselves, that puts pressure on them.
KASTE: The flashiest booth at this convention belongs to NEC Corporation of America. It's a major supplier of still photo facial recognition. They say they have not yet sold any real-time systems to American police, but executive Benji Hutchinson says a few departments are starting to window shop.
BENJI HUTCHINSON: I can't say which ones. But it's been the large cities that you might imagine, some of the large coastal areas.
KASTE: One sticking point, Hutchinson says, is that American cities often don't have enough of the kind of high-def digital security cameras that you need for facial recognition. Though he says that may not matter in the long term. As if to illustrate his point, a body camera clipped to a nearby mannequin starts beeping.
(SOUNDBITE OF BEEPING)
KASTE: Is that me, or is that...
HUTCHINSON: Oh, that's this thing.
KASTE: Oh, OK.
HUTCHINSON: Yeah. Yeah.
KASTE: (Laughter) I never know what's beeping. Oh, it's matching someone. OK.
A body camera that NEC has on display has recognized a passing face. Hutchinson predicts this is how the technology will come to American streets, by piggybacking on the thousands of new body cameras that departments are buying for their officers. At first, he says, it'll just be used as a way to recognize the faces of innocent people and blur them out. But it won't stop there.
HUTCHINSON: I think the second step is obviously they're going to look at ways to implement facial recognition against known or suspected terrorists or people who are wanted. That'll be the obvious next step. And it is a pretty simple leap.
KASTE: The question is, which law enforcement agency will be the first to take that leap? Hutchinson predicts it'll likely be a deep-pocketed federal department such as Customs and Border Protection, which is already testing facial recognition at passport control in airports. And as he mentions this, two CBP officials stop by the NEC booth, and the company's sales team scrambles to show them what the body cameras can do. Martin Kaste, NPR News, Washington. Transcript provided by NPR, Copyright NPR.