Would You Let an AI ‘Agent’ Buy Your Next Handbag?

Ask the AI-powered “answer engine” Perplexity what’s the best handbag under $1,500 and you won’t see a grid of sponsored results or links to Reddit posts asking the same question.

Instead, you’ll get a concise selection of five bags that fit the description, such as Chloé’s Kiss Small bag and Strathberry’s Mosaic, culled from online stories, blog posts and YouTube videos on the subject, as well as prices and a brief rundown of each bag’s key features. Perplexity provides links to where some bags can be purchased, and in certain instances, through its “Buy with Pro” option, can even complete checkout for you.

The new shopping feature, available to paying subscribers, launched in November, and the results aren’t perfect. A query about the best zip boots for men returned one result with stiletto heels that it described as a women’s boot. Sometimes the products to buy aren’t the same as those highlighted in the research summary, and as TechCrunch discovered, letting Perplexity handle checkout can be slow, taking hours in one instance.

But it illustrates how tech companies are thinking about the next way AI could transform the way consumers find and buy products online — via AI “agents” that can carry out complex tasks for users. Think of them like self-driving cars, but more open-ended in their abilities.

In recent months, agents have become the next topic of excitement in AI. Sam Altman, OpenAI’s chief executive, called them “the thing that will feel like the next giant breakthrough” during a question-and-answer session on Reddit. Several of the industry’s largest players are actively developing agents, including ones to help consumers with shopping, or even do it for them.

“Imagine taking a photo of something that you like, having the agent understand the style, find similar items across retailers and complete the purchase,” said Vince Koh, head of global solutions for digital commerce at Amazon’s Web Services division, or AWS. Koh described agents that could do things like use computer vision to help organise a user’s closet while learning their style through the visual data to make better product recommendations. “The biggest opportunity, I would say, is in creating seamless experiences that combine all these different types of interactions,” he added.

While Perplexity took a lead in the field with its shopping launch, others aren’t far behind. Google and OpenAI are looking into potential uses of agents. Amazon is in the process of prototyping AI agents that could suggest products on its site, or even add them to a user’s shopping cart, Wired reported. In a sign of how valuable the company believes agents could be, AWS aims to provide agents as a service to customers, including brands and retailers. Salesforce already introduced its own platform for the purpose, called Agentforce, with Saks Fifth Avenue among its clients.

If agents catch on, they have the potential to reshape online shopping. Many consumers are suffering from information overload and looking for solutions that can do the work of researching and selecting products for them. A report by Salesforce found that AI chatbots helped boost US online sales over the holidays by nearly 4 percent versus the prior year, suggesting consumers are using this simpler form of AI assistance.

Agents would offer even more robust abilities. The consultancy Gartner forecasts that, by 2027, just over half of consumers will routinely shop for goods and services using AI shopping agents funded by companies that produce and distribute products.

Before that happens, tech companies need to prove agents can actually improve shopping — and convince consumers to use them. It’s easier to see how someone might hand off buying commodity items like batteries, or purely functional ones like microwaves, to a bot versus fashion or beauty, where personal preferences and brand are so important.

The Next Big Thing in AI

The idea of AI that shops for you sounds like science fiction, but in a sense it’s an evolution of the efforts retailers have made to remove friction from commerce, such as the one-click checkout Amazon pioneered decades ago.

According to Dmitry Shevelenko, Perplexity’s chief business officer, the first phase for the company was changing search by giving people answers to their questions rather than links to read and digest themselves. With shopping, it’s taking the next step by letting consumers transact directly from those answers. Shevelenko said the percentage of shopping queries Perplexity gets has increased by “several multiples” since the launch, though he declined to provide exact numbers.

Eventually, Perplexity envisions an agent that can behave proactively, knowing enough about a user to anticipate their needs and wants.

“I don’t think the first manifestation of that is we’re just going to go buy something for you and it just shows up at your door,” Shevelenko said. “They start to come in the form of these nudges and notifications and pushes where it’s like, ‘Hey, you should really look at this’ … and then once you have that presented to you, you then have that one-click action [to purchase].”

Some people actually enjoy shopping, however, particularly when it comes to fashion or beauty. They might not want to outsource the job to a bot. Shevelenko said the goal for agentic AI isn’t to automate away the joy in shopping but to remove the annoying parts.

Google has agents in mind as well. They won’t be the solution for every problem, said Sean Scott, the company’s vice president and general manager of consumer shopping, in an emailed statement. The future of shopping is “assistive, personalised and seamless — in whatever form is most helpful,” he noted. But Google is pondering possibilities for agents like checking nearby stores for you to see if an item is in stock, or initiating a return and arranging a package pickup.

Amazon, for one, believes they’ll prove valuable enough to shoppers that brands and retailers will want their own. The company is exploring how AWS can support clients in building agents, said Justin Honaman, head of worldwide retail, restaurants and consumer goods business development at Amazon. Agents don’t just have to be customer-facing, either. They could be focused on internal tasks, like analysing social-media trends or chatter and running visual analysis of inventory to find products that match, to take just one example.

Obstacles to Overcome

Implementing them isn’t simple, however. For an agent to check if a product is available nearby, for example, it would need real-time inventory data for every store.

“One of the things that we find challenging with legacy retailers is their systems are either not connected or not set up to enable the agent to get access to data,” Honaman said.

That’s not the only challenge they face. The large language models that underlie agents work by making probabilistic predictions, without any genuine understanding of the world or their source content. Perplexity’s CEO, Aravind Srinivas, admitted to Fortune that the company doesn’t fully understand how its AI ranks and recommends products.

That probabilistic nature makes LLMs prone to errors known as hallucinations. Some experts believe they’re an inherent side effect of how LLMs function, which would mean agents always display some rate of error. An agent could potentially give a shopper incorrect information, or try to purchase something that doesn’t exist. Amazon’s Koh said companies he’s spoken to are concerned about the issue, though he feels as the technology progresses the rate of hallucinations will decline.

“I don’t view it as a principal barrier, and I certainly don’t view it as something that has to get managed to zero,” said Jason Goldberg, a retail expert and chief commerce strategy officer for the communications giant Publicis Groupe.

Goldberg thinks companies will have to implement measures to mitigate hallucinations, but that shouldn’t prevent them from embracing the technology, which he believes is poised to have a dramatic impact on retail. Replenishment categories, like toilet paper, will be the first affected, but fashion and beauty aren’t beyond reach. An agent that could scrape social media and create a design brief based on the data would be valuable for brands, while one that could sift through the troves of information online to provide an item’s carbon footprint would be desirable for consumers.

One unanswered question about shopping agents, however, is how effective they might be in categories where purchases are often emotional and not purely rational.

“There’s been a lot of debate about this,” said Gartner analyst Andrew Frank. “I happen to be on the side of the debate that says, as AI learns more about these intangible preferences that people have for brands, it will learn how to reflect those.”

One of the superpowers of LLMs is their ability to decipher relationships between nebulous qualities from smells to music styles, which is part of what makes them effective recommendation engines. But Frank sees other issues to be aware of if agents proliferate.

“You wonder whose side these agents are really going to be on,” he said.

Companies like Google, Amazon and others that shape shopping online make vast sums from advertising, which often means putting advertisers’ sponsored products at the top of search results. If agents are doing the filtering, will they do so to serve consumers or the advertisers?

The answer might depend on the specific company and agent. Perplexity’s Shevelenko said, for now at least, the company isn’t even taking affiliate fees on sales because it doesn’t want users to feel like it’s pushing shopping simply to make money.

“All it takes is one bad experience for people to give up on this technology,” he said.