The New Ways Retailers Are Watching You Shop

NEW YORK, United States — In Mayfair, one of London's top luxury shopping districts, customers hoping to catch a glimpse of the latest collections from Celine, Louis Vuitton and Ralph Lauren could miss a significant update to the area: The streets are watching them.

In May, Colliers, the brokerage that manages the real estate along some of Mayfair’s toniest thoroughfares, teamed up with Gyana, a human mobility tech firm. Combining Gyana GPS location data from smartphones with sensor technology installed by telecoms companies like O2, Colliers is monitoring shopping behaviour along several London high streets.

Gyana uses the location tracking features embedded in countless apps on shoppers’ smartphones to follow along when they enter a store and monitor how long they spend there. That information can even be paired with a customer’s age, occupation and any other details they chose to enter into their apps. Even privacy-minded customers who block location tracking are caught up in the sweep every time their phone pings nearby telecom towers.

Stores have monitored their customers since the advent of the video camera, mainly to catch shoplifters, and CCTV cameras are ubiquitous in London. But over the last few years, retailers have begun to install more sophisticated surveillance apparatuses designed to monitor customers’ every move.

Consumers and regulators have raised concerns about how personal data is collected and used. Often shoppers aren’t aware they’re being tracked, or what rights brands have to the information they gather.

Historically, what built retail businesses was piling stuff high in stores, but now it's data that helps stores understand their consumer and build loyalty.

But for brick-and-mortar retailers, knowing who is entering their stores and how they shop is information too precious to ignore. Fashion brands are facing off with digital rivals like Asos, Boohoo and Revolve, which routinely collect enormous amounts of data from their customers, from where they click to where they live to what unpurchased items are in their shopping cart. That data is then used to create tailored marketing pitches and develop new products.

Traditional retailers see collecting similar information on shoppers in real life as crucial for levelling the playing field, informing decisions ranging from what items to stock to how many salespeople to employ and where they should be stationed within a store.

“Historically, what built retail businesses was piling stuff high in stores, but now it’s data that helps stores understand their consumer and build loyalty,” said Richard Kestenbaum, a retail analyst at Triangle Capital.

How Stores Use Surveillance Technology

Tracking shoppers through their smartphones is just one way retailers are watching their customers.

Saks Fifth Avenue and Adidas are among the brands that have hired Axis Communications to install smart cameras inside their stores. The devices watch movement throughout the store, and can also estimate a customer’s age, sex and more.

Retailers analyse this information to make decisions about staffing and inventory, among other uses, said Hedgie Bartol, Axis Communications’ business development manager.

“For a long time, retail ran with an analog approach. Businesses would come up with plans and say, ‘I think this will work,’” he said. “But today, it’s as simple as watching customer flow to see if the cashier lines are too long, and that area needs help.”

RetailNext, a surveillance technology company used by UnTuckit, Express, Club Monaco, Richemont and Ulta, among others, places its devices on top of every store fixture, from tables to rolling racks. Once a customer steps into a zone, a camera records how they are interacting with products — logging time spent with a product or which items were ignored. Stores analyse how specific customer segments interacted — or didn't interact — with merchandise and tweak their inventory accordingly, said Rachelle Benson, a senior manager with RetailNext.

Alberto Corral, the senior director of marketing at UnTuckit said such surveillance "has allowed us to measure the effectiveness of our retail stores in a similar way we measure our online business while using those insights to optimise and measure our local marketing efforts." (Other brands mentioned in this story did not respond to or declined requests for comment.)

Gyana, RetailNext and Axis Communications say they anonymise the data they collect, so retailers can analyse shopping behaviour without identifying who specifically entered their stores. But many customers say the tracking itself is an overreach.

People don't really know they are giving up information about their movements, for free, so that it's easier for these companies to make money off of them.

“What if I don’t want to be a part of your system?” said Scott Urban, a Chicago-based privacy activist who has designed anti-facial recognition glasses that block facial recognition and eye tracking. “What if I don’t want to give you information about me that will help you sell me a product better?"

"Even if stores claim their technology isn’t nefarious," Urban went on, "People don’t really know they are giving up information about their movements, for free, so that it’s easier for these companies to make money off of them. That’s pretty invasive.”

There are some, though, who would agree to be tracked if it meant better store experiences.

“Surveillance has to get personalised for bigger stores to succeed and compete, and I think consumers will make the tradeoff if it means a better shopping experience,” said Kestenbaum, the retail analyst at Triangle Capital.

The Next Frontier: Facial Recognition

Until recently, facial recognition technology, where cameras can recognise individual people, was mostly used to combat crime. Retail brands lose about $48.9 billion annually due to theft, according to the National Retail Federation. Walmart, Macy’s and other retailers have used facial recognition to thwart shoplifters by comparing faces of store patrons against a database of criminals in real time.

Some stores are turning to this technology to get a better sense of their legitimate customers.

RetailDeep, which works with retailers in the US, Canada and China, uses facial recognition to track customers’ age and sex, as well as features like hair colour and height, said Hai Hu, the Halifax start-up’s founder.

Hu said facial recognition allows retailers to provide services to customers on an individual basis — and argues that consumers will happily give up some privacy for a better shopping experience. RetailDeep doesn’t retain photos or pass them along to brands, and notifies customers they are being watched via stickers placed in shop windows.

“Understanding loyal customers is crucial to a retailers’ business, and the benefits of this would be double-sided,” said Hu. “The pros want to be recognised, and we believe they would sacrifice their privacy at the expense of convenience. No one wants to walk into a store and wait around for 30 minutes to find the right person.”

We believe they would sacrifice their privacy at the expense of convenience.

Retailers that deploy technology from Facenote, a start-up based out of Buenos Aires, offer shoppers discounts if they agree to take a selfie and let the store keep tabs on their shopping behaviour.

“It’s like tracking shopping data online, the way cookies do,” said Facenote Chief Executive Eduardo Rivara.

Facenote has worked with Philosophy, the Coty-owned cosmetics brand, as well as Melissa, a chain of shoe stores in California and Florida. Beatriz Camacho, Melissa’s business development director said the business sees high conversion rates with shoppers who opt into Facenote’s technology.

“The feedback we’ve gotten is that people are happy our sales associates have their information,” said Camacho. “It eliminates the whole charade of small talk because we already know her shopping behaviours, what she has in her cart online, and what she’s already bought.”

It eliminates the whole charade of small talk because we already know her shopping behaviours, what she has in her cart online, and what she's already bought.

Rivara said Facenote eventually wants to process payments with facial recognition, a technology already available in China. Alipay, owned by Alibaba, rolled out its “Smile to Pay” devices in 2017 and is now used in over 100 cities in China. Last month, WeChat parent Tencent said it was also experimenting with facial recognition payments with a service called Frog Pro.

Facial recognition technology has also been linked to privacy and human rights violations in China. The data is being used to track minority groups, according to The New York Times. And the technology is integrated into the country's social credit system, where the government watches and ranks citizens based on behaviours including shopping, smoking, and following laws.

Customer Backlash and Best Practises

In September, Walmart announced it had created a new position to supervise the retail giant’s use of data and technology. The executive will advise Walmart on issues regarding privacy, data and emerging technology.

Walmart may be trying to get out in front of any backlash sparked by the rise of customer data gathering. Shoppers are sceptical of how their data is being used, after tech giants like Facebook were caught mismanaging personal information. Amazon has come under fire from employees and investors alike for selling its facial recognition software Rekognition to the US government, which has reportedly used it to target undocumented immigrants.

The technology, according to the Massachusetts Institute of Technology's Media Lab, also has racial biases. Algorithms from three different facial recognition technology companies were able to identify white men correctly 99 percent of the time while women with dark skin were given wrong identified 34 percent of the time, per MIT's research. Such errors could result in inferior treatment of shoppers of colour.

Jay Stanley, a senior policy analyst with the American Civil Liberties Union, said retailers walk a fine line when they deploy surveillance technology inside stores, especially when they fail to fully disclose how they use the information they gather.

Businesses might say they are collecting data anonymously, but even the most sophisticated companies can't guarantee how that information will be used in the future. Brands that go out of business or get acquired, for example, often sell off data along with other assets, and there's no guarantee that the new owners will offer the same level of privacy.

“The technology has become so sophisticated, and the most intimate details about people can be tracked that the limits of what is being collected are largely unknown,” Stanley said. “We have no idea what the boundaries of this technology are.”

The most intimate details about people can be tracked that the limits of what is being collected is largely unknown.

The issue is underscored by the fact that this type of surveillance is largely unregulated in the US. San Francisco became the first city in the US to outlaw facial recognition in May, and about 20 more states are working on similar privacy legislation, according to the ACLU.

Experts say if retailers want to track their customers, they should at least make sure consumers are looped in first. Though that means adding an extra step to the shopping experience, it can also build trust by demonstrating transparency.

“We're in a weird time with technology where there’s this idea that everything is bad and is trying to hurt you, and so it’s important to tell customers what’s going on,” said Lance Ulanoff, editor-in-chief of Lifewire and veteran tech expert. “Explain that technology will mean they will spend less time looking through product they don’t want ... and people will feel comfortable. If you don’t, it could be a disaster.”

Editor's Note: This article was revised on 24 September, 2019. An earlier version of this article misstated how Gyana's technology works. The tech firm uses sensor technology to collect human mobility data, not video cameras. The article also misstated Gyana installed the technology on the streets. The devices were installed by several telecoms providers.

Related Articles:

[ Is Surveillance the Future of Service?Opens in new window ]

[ Live Stream Apps Are Changing the Way People ShopOpens in new window ]

[ How Can New Technologies Help Make Fashion More Sustainable?Opens in new window ]