|
Facial Recognition: Who’s Tracking You in Public?
The mall is crowded, including the department store that keeps your family supplied with everything from handbags to business suits. Moments after you enter, a saleswoman walks up holding a tablet. She smiles and greets you by name. Are you shopping for yourself or your spouse today? We’ve moved things around since you were here in December—let me help you find your way, she says.
This is how customer service works in a few high-end stores in Europe, and vendors are now marketing the underlying technology to retailers in the U.S. The experience relies on facial recognition—and whether it sounds appealing or intrusive depends on your perspective.
Here’s how facial recognition works. As shoppers enter the store, security cameras feed video to computers that pick out every face in the crowd and rapidly take many measurements of each one’s features, using algorithms to encode the data in strings of numbers. These are called faceprints or templates. The faceprints are compared with a database, and when there’s a match, the system alerts salespeople—or security guards if anyone previously caught shoplifting in the store is spotted walking the aisles.
A company called Herta Security, based in Barcelona, Spain, is one vendor of the technology. Its system is being used in casinos and expensive shops in Europe, and the company is preparing to open offices in Los Angeles and Washington, D.C.
Retailers that use the Herta system receive alerts through a mobile app when a member of a VIP loyalty program enters the store—the customers have previously agreed to have their photos entered into the retailer’s database. The screen displays the shopper’s name, a photo just taken from the video feed, shopping preferences, and other details.
For now, security is a bigger business than customer service, however. Herta’s software was used at the 2014 Golden Globe Awards at the Beverly Hills Hilton to scan for known celebrity stalkers. The company’s technology may soon help bar known criminals in soccer stadiums in Europe and Latin America. Police forces and national security agencies in the U.S., the United Kingdom, Singapore, South Korea, and elsewhere are experimenting with facial recognition to combat violent crime and tighten border security.
Beyond Photo Tagging
Facial recognition is more firmly established online than in the physical world. Facebook has used it to help users tag photos since 2010. Last spring Google launched a photos app that helps users organize their pictures by automatically identifying family members and friends. (The company suffered a public relations humiliation when the system labeled a photo of two black people as gorillas. The search giant rushed to apologize—and fix its algorithms.)
Looking ahead, MasterCard is experimenting with a system that lets users validate purchases by snapping a selfie. Like fingerprint scanners and other biometric technologies, facial recognition has the potential to offer alternatives to passwords and PINs.
Those applications can make photo-sharing faster and more fun, and they can add security and convenience to real-world venues. However, the technology has been evolving fast, with little public debate or regulation.
In that regard, facial recognition today is reminiscent of the World Wide Web of the mid-1990s. Back then, few people anticipated the day when the details of everything we read, watch, and buy online would become commodities traded and used by big business—and frequently stolen by hackers.
Two decades on, many of us have become numb to the privacy intrusions of the Web. But at least we know we’ve gone online and can control whether or not we have social media accounts and what we share through them.
Facial recognition has the potential to move Web-style tracking into the real world, and can erode that sense of control. That’s what alarms privacy experts such as Alvaro Bedoya, the executive director of Georgetown Law’s Center on Privacy & Technology, and the former chief counsel to the Senate’s subcommittee on privacy, technology, and the law.
“People would be outraged if they knew how facial recognition” is being developed and promoted, Bedoya says. “Not only because they weren’t told about it, but because there’s nothing they can do about it. When you’re online, everyone has the idea that they’re being tracked. And they also know that there are steps they can take to counter that, like clearing their cookies or installing an ad blocker. But with facial recognition, the tracker is your face. There’s no way to easily block the technology.”
No Talk, No Action
Facial recognition is largely unregulated. Companies aren’t barred from using the technology to track individuals the moment we set foot outside. No laws prevent marketers from using faceprints to target consumers with ads. And no regulations require faceprint data to be encrypted to prevent hackers from selling it to stalkers or other criminals.
You may enjoy Facebook’s photo-tagging suggestions, but would you be comfortable if every mall worker was jacked into a system that used security-cam footage to access your family’s shopping habits, favorite ice cream flavors, and most admired superheroes?
Like it or not, that could be the future of retail, according to Kelly Gates, associate professor in communication and science studies at the University of California, San Diego and author of “Our Biometric Future: Facial Recognition Technology and the Culture of Surveillance.”
“Regardless of whether you want to be recognized, you can be sure that you have no right of refusal in public, nor in the myriad private spaces that you enter on a daily basis that are owned by someone other than yourself,” Gates says. “You give consent by entering the establishment.”
In 2014 the Commerce Department’s National Telecommunications and Information Administration started to address those issues by organizing talks between trade groups, individual companies, and privacy advocates. The goal was to come up with voluntary standards to allow facial recognition to expand while protecting consumer privacy. But the talks stumbled badly last June.
Bedoya had been participating in the meetings since they began in 2014. He says that privacy advocates had started worrying at a previous meeting, when trade groups refused to commit to encrypting facial recognition data. “It’s such a basic safeguard that we thought it would sail through,” he says.
Then, at the June meeting, Bedoya says that privacy advocates asked a hypothetical question about user consent: Let’s say a citizen is walking down a public street. And then a company he’s never heard of wants to snap his photo and check a database to identify him by name. In that case, the company would clearly have to ask first, right?
“That was an edge case, the most extreme example,” Bedoya says. “But not a single company in the room would agree to it. Stakeholders were meeting in a conference room about two blocks west of the White House, in Washington, D.C. In the afternoon the group took a break, and the privacy advocates didn’t come back. A few days later they announced that they would no longer participate in the talks. “We said, ‘We’re not going to play this game. We’re withdrawing from negotiations, and we’re going to tell the world what’s happening.’” The NTIA meetings have continued—but to date no code of conduct has been adopted.
Of Staterooms and Church Pews
Although facial recognition is still used largely for security, other applications are spreading, particularly in the hospitality industry. On Disney’s four cruise ships, photographers roam the decks and dining rooms taking pictures of passengers. The images are sorted using facial recognition software so that photos of people registered to the same set of staterooms are grouped together. Passengers can later swipe their Disney ID at an onboard kiosk to easily call up every shot taken of their families throughout the trip.
Kelly Shanahan-Carson, who co-founded a Disney-travel blog called The Main Street Moms, is a fan of the technology. “In the past, they’d print every single shot and place them in racks lining the wall in Shutters, the photo store onboard. You’d have to look through hundreds of photos to find yours. By the last day, it would be nuts.” Disney’s system is built by a company called The Image Group, which also partners with Royal Caribbean, Celebrity Cruises, and other companies.
Starting in 2010, the 1,200-room Hilton Americas-Houston in Texas employed a facial recognition system created by a company called 3VR. Though the system was designed mainly as a security tool, early on the hotel experimented with using the system to identify VIP guests who could be greeted by name by hotel staff, according to 3VR. The hotel wouldn’t comment on whether that program is still active. But facial recognition companies are actively marketing their systems to hotels.
A surprising use of facial recognition was revealed in the summer of 2015 when a company called Churchix said it had installed a facial recognition system in dozens of churches around the world to track which congregants were attending services. Company founder Moshe Greenshpan declined to put Consumer Reports in touch with any clients, saying that the technology received a “wave of bad publicity, and our clients got a little scared.”
However, he defended his product. “Tracking members means that churches know who is a regular attendee, and might be open to giving a donation, for example,” he says. “It also means they can know whether a regular attendee suddenly stops coming. The church can call to make sure everything is okay.”
Surveillance in the pews may seem particularly off-putting, but there’s evidence that facial recognition tends to make people uncomfortable wherever it appears. In a recent study of 1,085 U.S. consumers by research firm First Insight, 75 percent of respondents said they would not shop in a store that used the technology for marketing purposes. Notably, the number dropped to 55 percent if it was used to offer good discounts.
The aversion people feel to facial recognition may decline as it becomes more familiar, especially if retailers offer enough incentives. Meanwhile, not every intelligent camera system is looking to identify you as an individual. Facial recognition can also help marketers determine the age, sex, and race of shoppers.
In Germany, the Astra beer brand recently created an automated billboard that noted when women walked past. The billboard approximated the women’s age, then played one of several prerecorded ads to match.
Retailers can use facial recognition systems to see how long people of a particular race or gender remain in the shop, and adjust displays and the store layout to try to enhance sales.
Using related technology, some high-end retailers in the U.S. have experimented with “memory mirrors” that perform tricks such as storing images of what shoppers tried on so that they can be revisited, or emailed directly to friends for feedback.
A Database of Billions
If a company wants to tap into a list of thousands of consumers who like stout beers and sports cars, it can do that through a big data broker. But, according to facial recognition vendors and customers, privacy experts, and lawyers we interviewed, marketers that want to combine faceprints with personal data are amassing the information themselves, one customer at a time.
That’s a slow process, and the customer databases are relatively small. The scale is entirely different online. In 2014, Facebook published a paper on a research project it calls DeepFace (read “How Facial Recognition Works: The Ghost in the Camera”), a system said to be 97.35 percent accurate in comparing two photos and deciding whether they depicted the same person—even in varied lighting conditions and from different camera angles. In fact, the company’s algorithms are now almost as adept as a human being at recognizing people based just on their silhouette and stance.
How did Facebook get so good? Partly by harnessing the photos uploaded and manually tagged by many of its 1.5 billion users. And some privacy experts consider that a misuse of personal data.
“Entities like Facebook hold vast collections of facial images,” says Gates, the UC, San Diego professor. “People have voluntarily uploaded millions of images, but for their own personal photo-sharing activities, not for Facebook to develop its facial recognition algorithms on a mass scale.”
Last spring Carlo Licata, a resident of Illinois, sued Facebook, claiming that the company broke a state law, the Biometric Information Privacy Act, by failing to get his consent to storing, using, and sharing the data. Two other men later joined the suit, which is still progressing through the legal system.
It’s not apparent what effect such lawsuits might someday have on Facebook and other companies that use facial recognition. What is clear, though, is that just a couple of states have been ahead of the rest of the country in grappling with the implications of the technology. “Illinois is on the forefront,” Licata’s lawyer, Jay Edelson, says. “Texas has a similar statute, although it doesn’t allow consumers the right to bring lawsuits if their rights are violated. Unless there is a new law that’s enacted, people in other states don’t really have many rights protecting the collection and use of their faceprints.”
And there’s no way to determine what deals online companies may someday forge with walk-in businesses. Could Facebook or another Web-based company use its vast database of faceprints to power real-world facial recognition? Hypothetically, a tech giant wouldn’t need to share the faceprints themselves. It could simply ingest video feeds from a store and let salespeople know when any well-heeled consumer walked through the door.
The Surveillance Economy
Nearly all technologies that come with privacy risks are developed for legitimate and even beneficial purposes. Facial recognition is no exception, but it deserves attention and debate. Simple facial detection could surround you in a bubble of billboards and electronic store displays shown only to people of your race, sex, and age.
More importantly, facial recognition has the potential to erode the anonymity of the crowd, the specific type of privacy you experience when you stride through a public space, near home or on vacation, and refreshingly, no one knows your name. Marketers already can see every article we read online; do we need to let them record every shop window we gaze through?
According to privacy advocates, this is the time to consider policy changes, while facial recognition is still ramping up. One step advanced by stakeholders at the NTIA meetings would be to require an opt-in before people are entered into a facial recognition database, with reasonable exceptions for safety and security applications. That idea has already been implemented by some leading technology companies.
For instance, users of Microsoft’s Xbox gaming system can access their profiles using facial recognition, but only if they choose to turn on that feature.
Second, regulations could require companies to encrypt faceprints or institute other strong data protections—after all, a compromised PIN can be replaced, but there’s no ready solution if someone steals your biometric files.
Special rules could prevent children under the age of 13 from being targeted by facial recognition systems in stores. And consumers should have the right to know who has a copy of his or her faceprint, how it is being used, and who it is being shared with.
Those are just a few of the proposals that can be debated, and should be. Because right now, there are virtually no consumer protections at all.
Editor’s Note: This article also appeared in the February 2016 issue of Consumer Reports magazine.
Consumer Reports has no relationship with any advertisers on this website. Copyright © 2006-2015 Consumers Union of U.S.
|
.JPG)