A Shopify conversion agency spent a week in September sending an AI shopping agent at 200 real retailers and got refused at 29 of them — not because the stores said no, but because bot protection did, even with robots.txt wide open for ChatGPT, Perplexity, and Claude. Blocking AI shopping agents, it turns out, is mostly happening by accident.
At WebEpex we manage ecommerce builds for D2C brands across the GCC, Europe, USA, Canada, and India, and this is exactly the kind of gap we check for before a client's holiday traffic shows up. Amazon, the same week, opened a free portal putting any eligible store's catalog into Amazon search and Alexa, no Amazon selling required.
What Actually Happened#
Blend Commerce, a Shopify-focused CRO shop, tested 200 UK and US retailers across 20 categories between September 8 and 15, scoring each on whether an AI agent could reach, read, understand, and transact with a product page (the audit). Average score: 65 out of 100.
Twenty-nine retailers refused the agent outright, Home Depot, Best Buy, Macy's, Tesco, Argos, and Next among them. Four — Saks Fifth Avenue, Bergdorf Goodman, Anthropologie, and JD Williams — blocked even a request for their own robots.txt file. Blend's co-founder Adam Pearce put it plainly: most of these retailers had likely told the AI companies their agents were welcome. The security layer sitting in front of the site just never got the memo.
Five days earlier, Amazon launched Shop Direct Merchant Workspace on November 4 — a self-serve way for any eligible store to list products in Amazon search and Alexa for Shopping without becoming an Amazon seller. No fees. Merchants keep their own pricing, fulfillment, and customer support. It's the second major agentic-shopping move in two weeks, after Shopify's own push into agentic commerce off the back of its Q3 numbers.
Why Is Your Bot Protection Blocking AI Shopping Agents?#
Usually it isn't a decision anyone made on purpose. It's a WAF or bot-mitigation vendor set to "challenge all unverified traffic" by default, from before anyone on the team had heard of an AI shopping agent. Robots.txt says yes. The firewall in front of it says no. Nobody reconciled the two.
If you're running under a couple hundred orders a month from organic or chat-assisted traffic, this one's genuinely skippable — go check your December promo calendar instead. If a real slice of your traffic already comes from Google or from someone asking ChatGPT "where should I buy X," the Blend numbers say something uncomfortable: the thing standing between you and that sale might be your own security settings, not your product page.
The sector split backs this up. Shopify stores with a live agent checkout endpoint averaged 91; everyone else averaged 59. That's not Shopify being smarter about AI — it's fewer custom security layers sitting in front of a Shopify checkout than in front of a decade-old enterprise stack with six vendors bolted on top of each other. It also pairs oddly with what we flagged after the "only 15% of buyers trust AI shopping agents" study: buyers don't trust the agent, and a chunk of retailers won't let it through the door either, for different and mostly unexamined reasons.
The Honest Trade-Offs#
Here's what the audit glosses over: the same bot protection blocking a shopping agent is also keeping credential-stuffing bots and scraper swarms off your checkout. Loosening it for "AI agents" means trusting a self-declared user-agent string — and any script can fake one. Amazon's Shop Direct, on the other side, hands Amazon your catalog and some margin visibility in exchange for discovery you don't control and can't fully measure yet; the portal shows which products are live, not impressions or sales.
We don't think either trade-off is obviously right. Most stores haven't actually made this choice. Their security vendor made it for them, by default, months ago, and nobody's looked since. It's the same blind spot from the other direction we wrote about when ChatGPT itself started quietly deprioritizing sites — the gatekeeper changes, the "nobody checked" part doesn't.
How We're Handling It#
We run client sites on our own stack — Next.js, self-hosted VPS, Nginx in front — specifically so we hold the rule set directly instead of inheriting a vendor's factory defaults. At WebEpex, the week Blend's audit went up, we pulled the robots.txt and the firewall logs on every ecommerce client we manage and checked one against the other, not against the paperwork.
Two clients had a mismatch: allowed on paper, challenged in practice. We fixed both the same day — explicit allow rules for the named AI crawlers, logged separately from general bot traffic, so a future scraper spike doesn't quietly reopen the hole. That check now runs before launch on every new build, not after a client tells us a chatbot couldn't find their product.
What Should You Check This Week?#
Pull your robots.txt and your bot-protection dashboard side by side — that's the whole check. Three things, in order:
- Does robots.txt allow GPTBot, PerplexityBot, or ClaudeBot?
- Is your WAF set to challenge all unverified agents regardless? If both are true, you've got the exact gap Blend found.
- On Shopify already? You're probably fine on reachability — spend the ten minutes on your returns policy and GTINs instead, since that's where Blend found Shopify stores actually falling short.
It takes ten minutes. Do it before any Black Friday prep starts, not after.
If you want a second pair of eyes on whether your setup actually lets an agent in, send me the URL and I'll check it myself. Two minutes, no pitch. [cal.com/webepex/growth-review]
