More 5-Star Reviews Won't Fix What AI Says About Your Shopify Store. Here's Why.

0:08I was shopping on a site the other day I landed on a product page I'm using Google Chrome and all of a sudden at the top, a little thing came down right beside the URL bar that says, would you like to see reviews about this product? This wasn't the store asking me, this wasn't a pop-up they had, this was Chrome suggesting that I can see reviews about that product. So now a shopper can land on your Shopify product page and ask Google what everybody else thinks about your product before they read a single word you wrote. So I tried it on a pair of these athletic shorts that I was looking at. Little prompt pops up in Chrome, says analyze reviews.

0:45So obviously I clicked it. I figured it would summarize the reviews sitting right there on the product page, but it actually didn't. It came back with a whole pros and cons list. And some of what it told me wasn't coming from the brand site at all. And my first thought was, okay, this changes reviews because most merchants are gonna see something negative in that summary and immediately think, We need more five-star reviews. And that could be completely wrong. You can't fix the answer until you know where the answer came from. So that's going to be the 1 % win we focus on today. I want to take one claim that AI makes about one of your products and trace it back to the source.

1:31One claim. That's it. because by the end of this episode, I want you to be able to tell the difference between four very different problems. So you've got a product problem, a data problem, an expectation problem, or an actual reputation problem. And they don't have the same fix. So I'm going to give you four moves, how to audit it, trace it, feed it, and change it. change it. And I'll come back to those shorts that I was looking at later because what happened when I tested this is really a good example of why just reading an AI summary isn't enough. OmniSend just shipped something that I've always wanted.

2:17You can now connect OmniSend to ChatGPT or Claude and just say, where can my email and SMS campaigns be making more money? And it actually figures it out. It reads your store data, all your campaign details, your store history, and it tells you and the insights are legit good. And even better, you can then say, okay, let's fix that or build that campaign. And it actually does it right from your chat. It's wild. Now, if you're one of our listeners, you can get 30 % off for three months on OmniSend. Just go to the show notes. There's a link there. Or you can just use the code J30, J-A-Y-3-0 when you sign up.

2:51Now, if you're using another email and SMS platform and you want to switch to OmniSend, which is a great idea, they will do the migration for you. Just give them five days and then you show up with everything set, ready to go. Now they've migrated thousands of stores from Klaviyo and on average, they're saving 35 % on their email and SMS bills. If you're listening to this show, you clearly care about your Shopify business. If your store isn't backed up and protected, it could all be for nothing. That's why I tell every store owner I know who cares about their business, you 100 % need Rewind installed.

3:22It's a non-negotiable for me. One bad CSV import, One app that edits your store the wrong way, which has happened to basically everyone I know. One bulk edit gone wrong. And now, AI making changes for you. Listen, Shopify does not have an undo button. Rewind is the undo button. Go to shopify1percent.com slash rewind for 30 days free on us. Links in the show notes too. Last year, I went to Commerce Roundtable in San Diego and it was so good. I'm going back again this year. It's September 21st and 22nd. And honestly, you should come to. It's the one e-commerce event that I recommend to everybody without blinking.

4:02It's not a trade show. There's no booth demos. It's really just operators on stage sharing what's actually working in their stores right now. And it's basically a room full of all founders. I was actually scrolling through my camera roll from last year, and I have over 100 pictures of slides from people's talks. It's I learned so much. It's that kind of an event. Now, I have five passes to give away. for a 30 % off ticket. It's first come first serve. So if you're interested, email me jay at shopify1percent.com and I will hook you up. They will go fast. So email if you're interested. And if you're going, come say hi.

4:37Let's grab a coffee or a drink during happy hour. I'd love to meet you.

4:44Okay, so why this matters. First, scale. So according to StatCounter, Chrome has just under 70 % of the worldwide browser share in June, out of June, 2026, about seven out of every 10 browsing sessions globally are on Chrome. Even in the U S where Safari is obviously stronger because of the iPhone Chrome is still over half 52.26 % of the exact number. So this isn't some weird, uh, experimental browser being tested on 14 nerds in Mountain View. this is all your customers. A lot of your customers are using Chrome and the reviews are already deeply baked into how people shop. There was a bright local 2026 consumer review study that found 41 % of consumers now say they always read reviews when they're considering purchasing a product.

5:4041 % always read a review. Here's what's crazy though. That was this year, 2026, the year before it was 29%. So the number of people reading reviews is going up. And a lot of that's just because it's gotten easier. Now, small disclaimer, because I hate using stats for something that I didn't personally measure, but that was a also including a local business research as well, not just e-commerce. So it blends all commerce, but so I'm not telling you for sure that 41 % of every Shopify customer reads a review, but use this as general guidance. People already rely heavily on reviews. That's the bottom line.

6:19And now AI can compress hundreds or thousands of individual opinions into just a few sentences. Now that compression is what really interests me because a recurring complaint used to be scattered across pages and pages of review and you had to click through to see them. There could be some five-star reviews, some three-star reviews. A couple people mentioned the waistband. Another person said the sizing feels weird. Somebody else says that it was tighter than expected. A shopper had to hunt and find the pattern, but now AI can do that work for them. It can take a messy pile of customer feedback and literally turn it into one single bullet.

6:59For example, customers frequently mention inconsistent sizing. Now, that's a very different shopping experience. And I honestly don't know exactly how Google weighs everything. Does a review from last week matter more than one from two years ago? Probably. I mean, Google's smart. How much more? I don't know. Google hasn't given us a ton of data on this saying it's 32 % recency, 18 % come from Reddit, some is baked in from the site. We don't exactly know that. So don't pretend we know the algorithm. them. We don't need to, but we can work the inputs that we can actually see. So before we get into the four moves I want to talk about, I need to separate three things because it's really easy to mash all of this together and they're related.

7:48They're not the same product. Layer one is Gemini in Chrome. That's what you're seeing when you land on the site. And I have a screenshot of this. I'll put it in the show notes and the blog post for this episode of what that actually looked like. Google says Gemini in Chrome gets the content from your current browser tab by default. So on desktop, you can also share up to 10 open tabs with it. So the shopper is looking at shorts. They can ask Gemini about the shorts. They can potentially open your competitor in another tab and then ask about both. So that's one layer. Now, layer two is Chrome store reviews.

8:34This is a different feature. Google launched this just about a year ago and you can click the icon right beside the web address and Chrome can show an AI generated summary about the merchant. Customer service, product quality, shipping, pricing, returns, all of it. And Google says that feature draws from the Google shopping and other popular review sites. And as I'm recording this, Google still documents the store review feature as requiring the user to be located in the United States. And then layer three is Google's broader commerce data. So the merchant center, product rating, store ratings, the shopping graph, different systems, all kinds of different inputs.

9:17Fixing one doesn't automatically fix all three. So just keep that distinction in your head because we're going to touch all three. So let's get into move number one, audit it. Start stupidly simple. Just go to your best-selling product on your site in Chrome and run the analyze reviews. It will likely pop up when you're there, but if not, I've noticed it doesn't do it every single time. You can click in the URL bar and then click on the side where it says, it will either say AI mode or analyze reviews. It's different in different countries, but either way, click on the URL bar and you can get to it.

9:57You're going to do it on your best product, then do it on your highest returned product. That's the one I would probably be the most curious about, because if you're selling apparel and one SKU has a 14 % return rate while everything else is 6%, I want to know whether the AI can see the same issue your returns data is telling you. then open your closest competitor and the closest equivalent product they have. And then also run it there. Three products, same thing. Your best seller, your highest return product, and the closest competitor. You're going to run this on three products. And don't stop with the default analyze reviews button.

10:39Actually ask buying questions. Say, what are the biggest complaints about this product? Or who is this best for? Does it run true to size? Who should, what should I know before buying this? Compare this with the competitor I have in the other open tab. Ask all the questions that a customer would be, would potentially ask. You're trying to recreate the questions somebody asks when they're almost ready to buy, but they're looking for maybe a reason not to. And that's where this gets super useful. You're not auditing your average star rating. You're auditing the story that AI tells somebody when they're interrogating your product.

11:14And if you sell something internationally, remember that Google's reputation data is market specific. So Google actually says that store ratings need enough unique reviews within a country for it to establish a rating for that country. So don't assume that the reputation data that Google holds for your business in Canada or the US is going to be identical somewhere else. So now I wouldn't spend three hours on this, spend 15 minutes on it, ask five or six prompts, take screenshots of all of them, and then let's move on because the useful part is going to start right now, which is the second move, tracing it.

11:52So let's take the first meaningful negative claim that you might have seen and write one word beside it. Source. It's where did this come from? This sounds obvious, but it completely changes what you're going to do next. Let's say Gemini says, Okay, scenario one. You go through your reviews on Shopify store and there it is. Again and again and again. customer after customer says they fit tight through the thighs. So well, congratulations, your review strategy isn't broken. The shirts, the shorts fit tight through the thighs. You have information. Now, maybe that's a product development issue. Maybe the next manufacturing run needs to change.

12:44Maybe it's completely intentional. Maybe it's completely intentional. Either way, you know what the problem you're solving is. Now, scenario two, you search your own reviews and almost nobody actually says that in the reviews on your page. So then you have to find a Reddit thread where maybe 20 people are talking about it. Now, that's very different. Getting another 200 five-star reviews into your review widget doesn't make the Reddit thread disappear. You need to understand why the off-site conversation differs from the on-site conversation happening in your reviews. Maybe they're older reviews, or maybe a previous version of the product fit differently, or maybe your current customers really do disagree.

13:32Maybe the people reviewing on your site are disproportionately happy because you have a review collection process. I don't know. But at least you're looking now in the right place. Now, scenario three is the AI says something that is just factually wrong. Returns aren't accepted, but your website clearly says 30 day returns. Now that's an unflattering opinion. The AI is wrong. Now your job is to find that stale information. It's there and we just have to find it. So is there an old outdated return policy or URL floating around? Did your policy change, but another page still has the old wording?

14:14Is the merchant center, Shopify's merchant center, sorry, Google's merchant center carrying outdated information? Is there a third party site or blog or something listicle of products describing your products incorrectly? Find the source and fix it and then retest it. And it might take a couple of days before the AI picks it up. And then scenario four is one that probably nobody likes. Let's say the AI says your shipping is slow because maybe let's face it, your shipping is slow. Now your review app can't fix slow shipping. That has nothing to do with the review of no email asking customers to share their experience is going to fix an operations problem.

14:56You need to fix the problem now. And I think this is the part where merchants are going to get themselves a little bit into trouble because reviews feel like marketing. So if the AI says something bad, marketing gets handed the problem. Sometimes marketing has absolutely nothing to do with it. And that's the point I want you to remember from this episode. You don't have a review problem until you know which problem you have. It might be a product problem. It might be a data problem. It might be an expectation problem. Maybe the description is off. Maybe how you're marketing it is off. Or it could just be an actual reputation problem.

15:40Those require completely different fixes. At the beginning of the show, I told you why everyone serious about their Shopify business should have their store backed up and protected with Rewind. There is no excuse not to. There literally is no undo button for Shopify. Shopify's own terms actually say that your data is 100 % your responsibility. Your products, your theme, your customizations, your settings, everything. They don't back up any of it. And no, a CSV export does not cover it. With CSV exports, there's no images, no meta fields, no themes, settings, etc. Nothing. I was talking to the people at Rewind about a brand where one team member actually ran a bulk update and accidentally overwrote 11 ,000 products.

16:29Orders, customers, contacts, all of it gone in a single action. Luckily though, with Rewind, it was a simple one-click rollback. It could be anything. A bad CSV import, a random app you just installed that messes everything up, or all of these AI tools that everyone's using to manage their store now. I have heard so many horror stories. Rewind just gives you peace of mind no matter what. If you don't have Rewind installed, you absolutely should have it. Go now and get it. Shopify1percent.com slash Rewind. You'll get an exclusive 30 days free on me. The link is also in the show notes. At the top of the show I mentioned, OmniSend can now talk to ChachiBT and Claude.

17:10And I want to tell you why that's even bigger than it sounds. So if you missed it, quick catch up. OmniSend is what I use for email and SMS on every Shopify store I work with. plus for this show as well. And you can now connect your AI cloud chat GPT straight to your OmniSend account, your data, your campaigns, your flows, everything. You open a chat and you ask, what should I focus on this week to make more money? And it will come back with real answers about your store, which flow is underperforming, which automation needs improvement, which customers are going quiet, what's missing, everything.

17:44My favorite part though, is AI segments. I used to build these by hand clicking through filters forever. Now I just type something like customers who bought twice but ignored my last campaign and it just builds it. And anything it writes already sounds exactly like your brand because it pulls from all your previous emails from your website, your logo, your color, your voice, everything. I've been doing this for 15 years and this is the closest I've ever felt to having a full-time marketer sitting right beside me. I have a link for you in the show notes that will give you 30 % off for your first three months or you can just use code J30 when you sign up, J-A-Y-3-0.

18:20If you're ready to migrate from Klaviyo to OmniSend like thousands of others, they will do everything for you. You just show up and all your campaigns and automations from Klaviyo will be set up and running in OmniSend, ready to go. And usually at 35 % less what you were paying Klaviyo.

18:39Okay, now move number three is what we call feed it. And so this is a little bit of the boring plumbing, but I love this stuff because boring plumbing is what quietly screws up a shocking amount of e-commerce. You can have 2000 beautiful Shopify reviews sitting in your store and still be doing a terrible job feeding Google. Those are two separate jobs. So there are three tests that I would run. Test number one is what I would call the page test. Just simply take one of your important product pages and run it through Google's rich results test. I'll put links to this in the show notes as well to where you can actually do that.

19:19What you're checking for is to see if Google sees your product structure data properly. And where it makes sense, you're also looking for a review and aggregate rating information. Now, you don't need to become a schema expert. You're just looking for the obvious holes and errors here. If you don't see any reviews and you have reviews on your page, that means something's wrong. And you can look in your review app and potentially fix it or talk to a developer, something is wrong on your page. If Google can't understand the basic structured information about the product, you need to fix that before doing anything else and dreaming up your whole reputation strategy.

19:57That's number one. Now, step number two is what I call the source test. And only bother with this if something looks wrong in that first one. So you're going to look at the source and figure out where the product structure data is actually being generated. And Google recommends putting the product structure data in the actual HTML when merchants are trying to optimize for shopping results. So if it's in JavaScript and it's, this is going to get a little bit technical, but maybe being rendered on load rather than ahead of time, Google might not see it. So it's ideal if it's in the actual HTML and you can talk to a developer if you need help with that, but Google needs to be able to see it.

20:39Now, I was going to say just simply make sure Google can see reviews. And actually, that's not quite the right way to say it. The more interesting problem is actually whether Google can correctly understand and associate all of this information with the right product, because that's what gets us to test number three here, which is what I call the distribution test. So for this, you're going to go to merchant center, Google merchant center, by the way, I know, I think I mentioned that earlier merchant center, Google merchant center. Now you might not even know that your products are in merchant center.

21:15Um, it might not be something that you update regularly, but if you've ever created a Google product feed, YouTube product feed, Google ads, your products are likely in Google's merchant center. So go there and look at your product rating setup, look at your diagnostics. There's a whole section there where it'll tell you diagnostics on your product ratings and on your products. And look whether the review data that you're expecting Google to have is actually getting there. Google's current product ratings documentation says you need at least 50 reviews on all of your products to participate. So even after meeting that minimum, Google says that sometimes onboarding can still take up to a couple of weeks.

21:58And this next detail kind of matters a lot here, but Google says that the most important factor for matching reviews to products are globally unique product identifiers, which is GTI ends. And those are in Shopify in your product pages. And if you're missing them, or if they're wrong, Google will try weaker matches using a skew, but different brands can have the same skew. They might try brand plus an MPM, MPN, or even just a product URL. But Google explicitly says that those alternatives often result in fewer reviews being correctly associated with products. So picture this, you've got 500 reviews, they exist, customers can see them, you're feeling great about yourself.

22:49The reviews are very positive, informative, and helpful, but the product identifiers in your review data don't line up properly with the product information that Google has. So collecting another 500 reviews doesn't fix that. You don't need more reviews. You need cleaner data. Google also says that a complete and accurate product ratings review feed needs to be uploaded monthly to maintain eligibility. So that's something a lot of brands miss as well too. So if you're submitting that data yourself, that's your job. If your review provider is submitting it for you, then you need to make sure they are and understand what they actually support and whether the integration is working.

23:30And don't assume the phrase connects to Google means that much because product ratings, store ratings, Chrome store reviews, all different things. Your review app supporting one doesn't automatically tell you what it supports across every other Google system. Ask them, reach out to the support, read the documentation. And then when you find out, put the links for that in your team's SOPs, standard operating procedures. So this is it. It's not super sexy work, but it is very, very meaningful. It's also the sort of thing that somebody at a$20 million Shopify brand can assume another department might be taken care of.

24:12And it's not. Well, everyone assumes the same thing, nobody actually owns it. Okay, now move number four, the last one is change it. So once you've audited what AI says, you've traced where it comes from, and made sure that your data is actually getting where it needs to go, then you earn the right to change something. Now, there are two parts. First, fix the ask. And most review requests, automations are, they're lazy. And so what it will do is if you just install a review app, it will do something like when after order is delivered, wait X amount of days, and then send an email with something like, hey, Jay, how are you enjoying your order?

24:58Five stars, check the five stars, submit, done. Great, You got five stars. Meaningless to Google. The amount of time somebody needs before they have a useful opinion depends completely on the product. So if it's a pair of shorts, is it a customer probably knows pretty quickly if it's a good fit or not. Is it a mattress? Well, maybe don't ask me for a review on the mattress the first day. I haven't slept on it. Maybe I need a month. Is it protein powder? You know, I'd rather hear from somebody after they've used the whole tub of protein powder for a few weeks than someone's who's just review is package arrived fast.

25:39Haven't tried it yet. Five stars. A plus. Those reviews are pointless. If you're using Shopify's own shop reviews app, Shopify currently lets you set the request from one, I think all the way up to 180 days. So think about that. What makes the most sense for your product? The setting to do this is just in Shopify admin, sales channels, shop settings, shop reviews. Just an important qualifier. That's Shopify's shop reviews. That setting doesn't control whatever workflow you've built with JudgeMe, Account, anything else. For those, you have to go check the actual app. Then look at the question that you're asking.

26:22Now, I want useful information. These reviews are going to feed AI. So don't just say, how is it going? You want to ask questions like, how did the fit compare with what you expected? What almost stopped you from buying and why did you buy? Who would you recommend this product for? Because then, well, I can list more. I wrote a few down here. How are you actually using it? What surprised you after buying it? Why did you come back and buy one again if it's a repeat buyer? so those questions will get good answers they'll say the thing I love about this the most is let's just say it's a I've used this example in the podcast before like a purse or a travel bag the thing I love about this the most is I am allowed it's the largest size I can take in addition to my carry-on when I travel you know that might not be something you list in your product description but I might go and ask ChatGPT, I'm looking for the biggest bag I can take in addition to my carry-on, for example.