Blog · Measurement
How to measure whether your AI sales agent works: 6 numbers that matter
To find out whether your store's AI agent works, track six things: first response time, how many leads it classifies, how many follow-ups bring a conversation back to life, how many chats it hands to a person, how many sales you can attribute to it and, most important, how many of those sales you would not have made without it. Those six numbers tell you whether it sells and whether it is worth what it costs, without relying on a gut feeling that "it seems to answer well". Almost all of them come from conversations and orders you already have, and you can track them in a spreadsheet before peak season arrives.
Why isn't "how many messages it answered" enough?
Because answering messages is activity, not results. An agent can reply to a thousand chats and close none if its answers do not move people toward buying, while a person can close plenty of sales with a few well-aimed replies. What your business cares about is that more of the people who ask end up buying, and that this earns more than it costs. So measure a chain: the customer arrives, gets a fast answer, you identify who is ready to buy, you follow up if they cool off, you pass to a person when needed, and an order appears at the end.
Another common mistake is judging the agent only by a satisfaction survey or only by the ROAS of one campaign. Both are useful, but each looks at one piece. For the survey, see how to run a customer satisfaction survey on WhatsApp, and for campaigns, what a WhatsApp campaign really costs and how to tell if it paid for itself. This article covers what sits in the middle: the agent answering, qualifying and closing.
The six numbers at a glance
Before the detail, this table sums up what each number measures, where you get it and what question it answers. Copy it into a spreadsheet and fill it in once a week, nothing fancier. What matters is comparing against your starting point: how things looked in the weeks before you turned the agent on, so you are not judging from memory.
| # | Number | Where it comes from | What it tells you |
|---|---|---|---|
| 1 | First response time | Time of the customer's first message and time of the first reply | Whether you stop losing people who got tired of waiting |
| 2 | Classified leads | Chats tagged by interest (ready to buy, browsing, support only) | Whether you know who to attend first |
| 3 | Follow-ups that reactivate | Chats that replied after a follow-up | Whether you recover conversations that were going cold |
| 4 | Handoffs to people | Chats the agent paused and passed to your team | Whether it knows when not to continue alone |
| 5 | Attributed sales | Orders from customers who talked to the agent | How much money passed through its conversations |
| 6 | Incremental sales | Comparison against a period or group without the agent | How much extra you sell thanks to it |
How do you measure first response time?
Take a sample of chats from the week and note, for each one, how long passed between the customer's first message and the first useful reply. Split by time of day: daytime, night and weekends, because that is usually where the difference lives. An agent that answers 24/7 should show similar times at 3 a.m. and 3 p.m.; if your night replies used to take hours, that change is the first sign something is improving. Count only replies that actually answer the question, not a "hi, we'll be with you shortly".
This metric alone does not prove you sell more, but it is the easiest to verify and the one you feel most in peak season, when messages pile up. For the night side, see how to handle WhatsApp after hours. Also check quality: a five-second answer that is wrong is worse than a two-minute answer that is right, so read a handful of full chats every week.
What do you look at in classified leads and follow-ups?
Look at what share of your chats got a tag and whether the tags match your own judgment. Super Agent classifies every lead; your job is to review a sample and confirm that "ready to buy" really is. If it marks people who only asked your opening hours as hot, tighten the instructions; if it leaves cold the ones who asked for the payment link, fix that too. This classification helps you decide whom to message yourself first when volume spikes.
For follow-ups, the math is simple: of all the chats where the agent wrote again because the customer went quiet, how many replied and how many ended in an order. The agent follows up inside WhatsApp's 24-hour window and stops as soon as the customer answers, so it does not keep pushing. It is normal for many not to reply; what you want is for the ones who do to justify the effort. To tune the tone, read how to follow up with a customer who went quiet without being pushy.
Are handoffs to people good or bad?
Neither on its own: what matters is whether the handoff happens when it should. When a conversation needs a person, the agent pauses, tells the customer and passes your team the full thread, so nobody has to ask everything again. A very high share of handoffs can mean the agent lacks information about your store; a share near zero can mean it is trying to handle things it should not, like delicate complaints or special negotiations.
Do two checks. First, read the handed-off chats and sort the reason: missing information, special order, complaint, or a decision only you can make. Second, measure how long your team takes to pick the chat back up, because a fast handoff from the agent is useless if the customer then waits hours. If the same reasons repeat, add that information to the agent and the number drops by itself. To set it up, see how to hand a WhatsApp chat from the bot to a person.
How do you know how many sales came from the agent?
Start with attributed sales: orders from customers who talked to the agent before buying. Match the phone numbers or names in the conversations against your Shopify or WooCommerce orders from the same week. Keep in mind that orders arrive through a periodic sync, not instantly, so do not compare hour by hour. For WhatsApp campaigns, the sales they produced show up in the dashboard itself, which saves the manual matching for that part.
Attribution has a trap: a customer who was already going to buy and only asked about shipping counts as an "attributed sale" even though the agent did not cause it. That makes this number a ceiling, not proof. Use it to see how much money flows through the agent, and leave the verdict on whether it is worth the cost to the next number. To compare against the price of the service, see what an AI WhatsApp chatbot costs for an online store.
How do you calculate incremental sales?
Compare what you sell with the agent against what you would have sold without it. The simplest way is by period: take the weeks before turning it on and the weeks after, making sure they are comparable (do not set a normal week against a Buen Fin week). Another way is to leave some channels or hours without the agent for a while and compare. The difference, after removing the effect of the season and your ads, is your incremental sale.
As a real reference, one store that already uses Super Agent sells 22% more since it started, and another gets 55 times what it invested back in orders when it messages its own customers on WhatsApp (55× return). Those are those stores' results, not a promise for yours: they depend on your catalog, your customer base and how it is set up. The useful part is the method, which you can run on your own data to see what applies to you.
How do you decide before Black Friday?
Use the free plan to measure before you spend. The Free plan costs US$0/month, needs no card and includes 250 WhatsApp campaign messages a month; enter your website and we write your first agent, live the same day. We do the initial setup with you, free, in a 30-minute call. With that you can fill in the table above for a couple of weeks and see real numbers before moving up a plan.
- Write down your starting point from recent weeks: response time, unanswered chats and orders from WhatsApp and Instagram.
- Turn on the agent and review a sample of full chats each week.
- Fill in the six numbers in your sheet and compare against the starting point.
- Work out whether incremental sales exceed what you pay for the plan and credits.
- Decide whether to move to Go (US$39/month), Starter (US$99/month) or stay where you are.
If you do not have a peak-season plan yet, lean on the 3-week WhatsApp plan for Black Friday. And when you want to start, create your free account; no card needed.
Measuring your AI sales agent: FAQ
How long does it take to know if the agent works?
Two to four weeks of conversations give you a useful sample. Compare against comparable earlier weeks and avoid mixing Buen Fin or Black Friday days with normal weeks.
Which of the six numbers matters most?
Incremental sales, because they answer whether you sell more thanks to the agent. The other five explain why that result goes up or down.
Can I measure this without analytics skills?
Yes. A weekly spreadsheet with the six numbers and a read of a few full chats is enough to start.
Can I try it without paying before peak season?
Yes. The Free plan is US$0/month, needs no card and includes 250 WhatsApp campaign messages a month. Create your account and measure with your own chats.
Does the agent change anything in my store?
No. It reads catalog, prices, stock and order status from Shopify or WooCommerce, but never changes them.
Remember that no single number decides anything. A good response time without incremental sales is convenience; attributed sales without a comparison can be an illusion. Look at all six together, read real chats every week and decide with your own data, not another store's. You may also find this useful: AI sales agent or human seller: which store conversations to give each one. You may also find this useful: How to calculate your online store's cart abandonment rate: formula, example and what to do with it. You may also find this useful: How to choose an AI sales agent for your online store: 8 questions to ask before you sign up.
Measure your AI agent with the free plan
Enter your website and we write your first agent, live today. No card needed. Track the six numbers for a couple of weeks before peak season.
Start for freeFree plan, no credit card. See plans and pricing.
Keep reading
- All the guides on the blog
- How to choose an AI sales agent for your online store: 8 questions to ask before you sign up
- How to calculate your online store's cart abandonment rate: formula, example and what to do with it
- AI sales agent or human seller: which store conversations to give each one
- How much does a WhatsApp campaign cost, and how do you know it paid for itself?
- What an AI WhatsApp chatbot costs for an online store, with no small print
- How to hand a WhatsApp chat from the bot to a person, without the bot talking over them
- Black Friday for your online store: a 3-week WhatsApp plan to sell more without burning out your customers
- Missed-reply revenue calculator
- Online store self-audit
- WhatsApp campaign ROAS calculator
- Free tools for online stores