How to build a customer health score for an e-commerce SaaS
Pick four to six product usage signals, weight them from past churn cohorts, attach an action to each band, and check the score against real churn.
In short: A customer health score is a handful of product usage signals, weighted by how well each one separated the merchants who churned from the merchants who stayed, rolled into a band with an action attached. For an e-commerce SaaS, build it from four to six signals like data flowing, integration connected and a result seen in the last 30 days, not logins, and check it against real churn every quarter.
What is a customer health score for?
A health score has one job: tell a small customer success team who to talk to this week, and what to say. It is a triage list, not a prediction model. If it does not change what someone does on Monday morning, it is a dashboard, and dashboards do not retain merchants.
I set up health scores and the success guides that went with them as Head of Customer Success at Social Snowball, where a team of three looked after 1,200 or so merchants. Three people cannot read every account every week, so the score decided who got a human that week and the success guides decided what that human did. If you have 1,000 merchants and a CS team you can count on one hand, you have the same constraint.
Which signals should go into the score?
Start with what a merchant does inside your product, not whether they showed up. A login tells you a tab was opened, not that the app did anything in the store. The signals that matter are data moving between your app and the store, and results the merchant can see.
Pick four to six from a list like this:
| Signal | What it looks like in an e-commerce app | Why it predicts churn |
|---|---|---|
| Data flowing | Orders synced, affiliate clicks tracked, creator posts pulled, emails sent in the last 7 days | If nothing is flowing, the app is not doing its job, whether or not the merchant noticed yet |
| Key integration connected | Theme app embed on, Klaviyo connected, TikTok Shop authorised, pixel firing | The one connection without which the app cannot produce a result |
| Result seen in 30 days | An attributed sale, a published review, a creator who posted, a recovered cart | The merchant has something to point to when the invoice arrives |
| Breadth of use | Number of distinct jobs done: programs created, campaigns sent, creators contacted | Merchants who use two features leave less often than merchants who use one |
| Billing state | Failed charge, frozen subscription, downgrade in the last 30 days | Money problems show up before the cancel |
| Support tone | Open ticket about billing, a ticket about "how do I cancel", or a ticket that went silent unresolved | The merchant has told you, in words, that something is wrong |
If your app bills through Shopify, the subscription status is a signal you can read directly. Shopify's AppSubscriptionStatus reference defines FROZEN as "The app subscription is on hold due to non-payment", and says it re-activates after payments resume. A frozen subscription is not churn yet, but it belongs in the red band.
Keep plan tier and store status out of the weighted score and treat them as context. A merchant on a Pause and Build plan has a store customers can browse but not buy from. Shopify's help page on pausing a store says all apps remain active after a pause while checkout is deactivated across all sales channels, so your data stops flowing for a reason that has nothing to do with you. Flag paused stores so CS does not chase them.
How do you weight the signals from past churn?
Weights come from your own churned merchants, not a template. The method fits in a spreadsheet.
- Pull every merchant who churned in the last six to twelve months, and a similar-sized group who stayed.
- For each signal, look at the state 30 days before the churn date. Was data flowing? Was the integration connected? Had they seen a result?
- For each signal, compare the share of churned merchants in the bad state with the share of retained merchants in the bad state. The gap is the signal's power.
- Set the weights in proportion to the gaps.
Say 70% of churned merchants had no data flowing in the 30 days before they left, against 15% of retained merchants. That is a 55 point gap. If open support tickets split 20% to 15%, that is a 5 point gap. The first signal deserves roughly ten times the weight of the second. The figures are hypothetical, the arithmetic is not.
If you use PostHog, dynamic cohorts do most of this for you. PostHog's cohorts documentation describes conditions like "Did not complete user signed up in the last 2 months" and "Completed watched video 5 times in the last 2 weeks", and lets you use a cohort as a filter in retention insights. Build one cohort per signal, then compare retention across them.
If you have fewer than about fifty churned accounts, weight the signals equally, write down the date, and revisit when you have more data. A score built on judgement and labelled as such is fine. One built on judgement and labelled as data is how teams lose trust in it.
What action goes with each band?
This is the part most health scores skip, and the only part that retains anyone. Every band needs an action, an owner and a deadline. Here is the shape I would use for a Shopify app or a TikTok Shop seller tool.
| Band | What it means | Action | By when |
|---|---|---|---|
| New | Installed under 14 days ago | Handled by onboarding, not the score | Day 1 and day 3 check-ins |
| Red | Paid plan, no data flowing for 14 days, or billing frozen, or a cancel ticket | Personal message from a named person asking one question, plus the fix for the most likely cause | Within 24 hours of turning red |
| Amber | Integration disconnected, or no result in 30 days, or breadth dropped to one feature | One email with one link to the one thing to reconnect or try | Within the week |
| Green | Data flowing, result seen, two or more features in use | Ask for a review, or show the next plan if they are near a limit | Monthly batch |
| Paused | Store on Pause and Build or marked seasonal | Nothing until the store reopens, then a reconnect check | On reopen |
The red action is the one to get right. A merchant with no data flowing on a paid plan has usually hit a disconnect, a theme change or a store migration. Name the likely cause and include the fix, rather than asking them to "reach out if you need anything". For the green band, the guide on getting Shopify App Store reviews covers when to ask.
Write the actions down as success guides: one page per band, with the message template, the diagnostic steps, and what to do if the merchant does not reply. A new CS hire should be able to run the Monday review from those pages.
Where should the score live?
Start in a spreadsheet, and stay there longer than feels respectable. A weekly export of the signals per merchant, a formula for the weighted score, a column for the band, a column for the owner. You will change the weights three times in the first month, which is much cheaper in a sheet.
Move it into your helpdesk or CRM once the team acts on it every week without being reminded. The band should appear on every ticket, so whoever answers a "how do I export my data" question can see the merchant turned red last Tuesday. Most helpdesks take a custom attribute you can push from a script.
Build it into the product last, and only when the actions are automated. An in-app "reconnect your theme embed" banner that fires on the amber condition beats a score column in an admin view nobody opens.
How do you keep the score honest?
A health score decays. Events get renamed, a feature launch changes what "breadth" means, a billing migration breaks the frozen flag. Two checks keep it useful.
The quarterly look-back. Take every merchant who churned last quarter and look up their band 30 days before they left. If most were green, the score is wrong, and you need new signals, not new thresholds. Track two numbers: the share of churned merchants who were red or amber a month before (how much churn the score caught), and the share of red merchants who actually churned (how much CS time the score wasted). Both should improve each quarter until they plateau.
The weekly cadence. E-commerce apps are mostly month to month, which means a merchant can be fine on the 1st and gone on the 28th. SaaS Capital's 2025 retention benchmarks, from a survey of over 1,000 private B2B SaaS companies, put month-to-month contracts at 100% median net revenue retention and 89% gross, slightly below the overall medians of 101% and 91%. The gap is small, but the timing lesson is not: a score that updates quarterly cannot catch a monthly churn. Update it weekly, and read it weekly.
One more rule. Do not make the score a target for the CS team. If the team is measured on how many merchants are green, the thresholds will drift until everyone is green. Measure the team on churn caught and actions closed on time. The guide on churn signals lists the usage patterns worth testing first, and the customer success weekly rhythm covers where the Monday review fits in a small team's week.
Where to start
- Export every merchant who churned in the last six months, with install date, plan, last data event, integration status and open tickets at the time.
- Pick four signals you can compute from your database today, even if the breadth and support signals have to wait.
- Compare each signal's bad state across churned and retained merchants, and set weights from the gaps, or equal weights if the sample is small.
- Put the score in a sheet with a band and an owner per merchant, write one success guide per band, and run the first Monday review this week.
- Put the quarterly look-back in the calendar now, before anyone starts trusting the score.
Frequently asked questions
How many signals should a customer health score have?
Four to six. Fewer than four and one noisy signal dominates. More than six and nobody on the team can explain why a merchant is red, which means nobody acts on it. Add a signal only when the quarterly look-back shows the score missed churn the new signal would have caught.
Should logins or NPS be in the score?
Not as weighted signals. A login tells you a tab was opened, not that the app did anything in the store. NPS arrives after the merchant has formed an opinion and is better used to check the score than to feed it. Use product events that show data moving and results produced.
How often should the health score update?
Weekly for a monthly e-commerce product, with billing state and cancel tickets updating daily. A merchant on a month-to-month plan can go from healthy to gone inside one billing cycle, so a quarterly refresh will miss most of the churn it was built to catch.
What if we have too few churned customers to weight the signals?
Weight them equally, label the score as judgement-based, and revisit it when you have around fifty churned accounts to compare. An honest unweighted score your team uses beats a precise-looking one built on five data points.