Facial Scanning Is Making Gains in Surveillance

The New York Times article

WASHINGTON — The federal government is making progress on developing a surveillance system that would pair computers with video cameras to scan crowds and automatically identify people by their faces, according to newly disclosed documents and interviews with researchers working on the project.

The Department of Homeland Security tested a crowd-scanning project called the Biometric Optical Surveillance System — or BOSS — last fall after two years of government-financed development. Although the system is not ready for use, researchers say they are making significant advances. That alarms privacy advocates, who say that now is the time for the government to establish oversight rules and limits on how it will someday be used.

There have been stabs for over a decade at building a system that would help match faces in a crowd with names on a watch list — whether in searching for terrorism suspects at high-profile events like a presidential inaugural parade, looking for criminal fugitives in places like Times Square or identifying card cheats in crowded casinos.

The automated matching of close-up photographs has improved greatly in recent years, and companies like Facebook have experimented with it using still pictures.

But even with advances in computer power, the technical hurdles involving crowd scans from a distance have proved to be far more challenging. Despite occasional much-hyped tests, including one as far back as the 2001 Super Bowl, technical specialists say crowd scanning is still too slow and unreliable.

The release of the documents about the government’s efforts to overcome those challenges comes amid a surge of interest in surveillance matters inspired by the leaks by Edward J. Snowden, the former National Security Agency contractor. Interest in video surveillance was also fueled by the attack on the Boston Marathon, where suspects were identified by officials looking through camera footage.

In a sign of how the use of such technologies can be developed for one use but then expanded to another, the BOSS research began as an effort to help the military detect potential suicide bombers and other terrorists overseas at “outdoor polling places in Afghanistan and Iraq,” among other sites, the documents show. But in 2010, the effort was transferred to the Department of Homeland Security to be developed for use instead by the police in the United States.

After a recent test of the system, the department recommended against deploying it until more improvements could be made. A department official said the contractor was “continuing to develop BOSS,” although there is no sign of when it may be done. But researchers on the project say they made progress, and independent specialists say it is virtually inevitable that someone will make the broader concept work as camera and computer power continue to improve.

“I would say we’re at least five years off, but it all depends on what kind of goals they have in mind” for such a system, said Anil Jain, a specialist in computer vision and biometrics engineering at Michigan State University who was not involved in the BOSS project.

The effort to build the BOSS system involved a two-year, $5.2 million federal contract given to Electronic Warfare Associates, a Washington-area military contractor with a branch office in Kentucky. The company has been working with the laboratory of Aly Farag, a University of Louisville computer vision specialist, and the contract was steered to the firm by an earmark request in a 2010 appropriations bill by Senator Mitch McConnell of Kentucky, the Republican leader.

Significant progress is already being made in automated face recognition using photographs taken under ideal conditions, like passport pictures and mug shots. The Federal Bureau of Investigation is spending $1 billion to roll out a Next Generation Identification system that will provide a national mug shot database to help local police departments verify identities.

But surveillance of crowds from a distance — in which lighting and shadows vary, and faces tend to be partly obscured or pointed in random directions — is still not reliable or fast enough. The BOSS research is intended to overcome those challenges by generating far more information for computers to analyze.

The system consists of two towers bearing “robotic camera structures” with infrared and distance sensors. They take pictures of the same subject from slightly different angles. A computer then processes the images into a “3-D signature” built from data like the ratios between various points on someone’s face to be compared against data about faces stored in a watch-list database, the documents show.

The Homeland Security Department hired the Pacific Northwest National Laboratory to test the BOSS system at an arena in Kennewick, Wash. The plan, according to a “privacy impact assessment,” was to use 30 volunteers whose facial data would be mingled in a database among 1,000 mug shots to see whether the system could reliably recognize when any of the volunteers were present.

The agency set up six tests to determine the technology’s overall accuracy, determining afterward that “it was not ready for a D.H.S. customer” — meaning that police departments should not buy it.

In interviews, Ed Tivol of Electronic Warfare Associates and Dr. Farag both suggested that as computer processing becomes ever faster the remaining obstacles will fall away.

Mr. Tivol said the goal was to provide a match with an 80 percent to 90 percent certainty from a range of up to 100 meters, something “that has never been done.” While the system continued to have problems with light and shading in some tests, he said, in others the goal had been achieved at closer distances. Farther away, he said, the accuracy has fallen to 60 percent to 70 percent.
“The results were increasingly positive,” he said. There was a “significant improvement” in speed, too, he said. At first, it took the system six to eight minutes to process images, but it now takes under 30 seconds.

Still, he and Dr. Farag said, the officials overseeing the testing wanted a quicker turnaround. That might be easier with the more powerful computers available to the military, they said, but the government wanted them to use processors available off the shelf for civilian applications.

Several independent biometric specialists, given a description of the project’s test results, agreed that the system was not yet ready. They said 30 seconds was far too long to process an image for security purposes, and that its accuracy numbers would result in the police going out to question too many innocent people.

Several of the specialists also suggested that similar technology may be progressing more quickly in other laboratories that have not received taxpayer financing. A spokesman for Mr. McConnell stressed that while he requested that the contract go to Electronic Warfare Associates, it was “competitively bid.” Federal records show the firm was the only one to submit a bid.

Ginger McCall, a privacy advocate who obtained the documents under the Freedom of Information Act and provided them to The New York Times, said the time was now — while such technology is still maturing and not yet deployed — to build in rules for how it may be used. (Ms. McCall was at the Electronic Privacy Information Center at the time of her information act request.)

“This technology is always billed as antiterrorism, but then it drifts into other applications,” Ms. McCall said. “We need a real conversation about whether and how we want this technology to be used, and now is the time for that debate.”

In particular, she said, there should be limits on whose faces are loaded into them when they are ready for deployment. Ms. McCall said it would be acceptable to use it for terrorism watch lists, but she feared any effort to systematically track everyone’s public movements by using a comprehensive database of driver’s license photographs.

Still, Dr. Farag said, that kind of system is still very far off because it would take far too much computer processing power to load millions of images into a system and try to identify everyone at once, as opposed to sorting images in search of only a comparatively small number of faces on a watch list.

“Disappointments come when you are overambitious,” he said.

-----------------

Next we are gonna have machines predicting crimes like in the movie Minority Report. I guess this must not bother people too much though.. I don't see as much a stink about it as I do with gun control..lol

People have been doing the same for years.

Like wheels and fire, man is making progress and using his innovations to "automate" tasks.

Haven't the Vegas casinos been using facial recognition to keep certain "undesirables" out for years?

Haven't the Vegas casinos been using facial recognition to keep certain "undesirables" out for years? Originally Posted by chefnerd

Thats a good point!

Haven't the Vegas casinos been using facial recognition to keep certain "undesirables" out for years? Originally Posted by chefnerd

Do you agree that there is a DIFFERENCE in a casino, using this technology ON THEIR OWN PRIVATE PROPERTY for their own private purposes, and the Government using such technology on Legal, Constitutional "Freedom of Assembly" gatherings of citizens?

Actually I do agree there is a difference and a need for controls on it's usage. Granted, given any government's obsession for power (regardless of the type of government), that is much easier said than done. Especially since we are already living in at least a semi-Orwellian society.

Haven't the Vegas casinos been using facial recognition to keep certain "undesirables" out for years? Originally Posted by chefnerd

My brothers work in Vegas casinos and tell me the whole security camera thing is still a lot of bluster. Remember the guy who stole and took off on a motorcycle with a bunch of $25K chips? The chips didn't even have RFIDs in them.

The facial recognition software last I checked a few years ago was still a long way off. You had to have a database of known cheats and they couldn't be disguised that well.

My guess the people at Google will make it part of their Glass product but its will have limitations.