Your CRM has 100,000 contacts. But how many are actually usable?
You've invested in a CRM, built up a contact base, set up integrations and launched campaigns. The results, however, still disappoint. Campaigns underperform, personalisation falls flat and sales keeps complaining that 'the data is wrong.'
It's not because you have too little data. The problem is that you don't know how much of that data you can actually use. Research shows that poor (customer) data quality costs organisations an average of 12.9 million dollar annually (Gartner, 2020). People change jobs, companies merge and email addresses change. Without active maintenance, your database loses roughly a quarter of its value every year. Simply because of date you're not keeping track of.
In another blog we introduced the PPIT framework (People, Process, Information, Technology), where Information is the factor that most often causes automation projects to stall. This blog goes deeper into exactly that: how do you measure data quality, what do you do with it and how much of your data is actually usable for what you want to achieve?
)
)
Available versus usable
There's a big difference between having data and being able to use it. A contact record with a name and email address is available. A contact record with a valid email address, linked to the right company, with a job title filled in, a known source and a recent interaction is usable. That distinction determines whether you can automate or not.
Data quality is the foundation on which everything rests – a core concern rather than a technical detail. Organisations that successfully deploy automation start by honestly taking stock of what they have, rather than collecting more data.
To determine whether data is usable, it helps to apply six concrete criteria:
Validity: is the email address deliverable?
Completeness: are the essential fields filled in?
Recency: is the information up to date?
Consistency: are fields filled in the same way across records?
Connectability: can this record be connected to other systems?
Origin: do you know where this record came from?
)
What goes wrong when you don't measure data quality
An international company wanted to implement personalisation based on their "rich customer data" in BigQuery. They had invested in a modern data platform and were convinced the technical foundation was in place.
Analysis told a different story. The industry field – critical for segmentation – was empty for 34 per cent of records. The company size field had 847 unique values, including variants such as "small", "tiny", "1-10", "<10", "fewer than 10" and unintended entries like "yes". The email bounce rate on the existing database was 23 per cent, meaning nearly a quarter of emails would never arrive.
Before any automation could begin, three months of data cleansing and standardisation were needed. That's telling – because this kind of work never appears in the original project plan, and it's rarely something budgets are set aside for. Without that investment, however, any automation built on this foundation would be unstable.
)
The automation reality we see time and again
Unfortunately, the example above is not an exception. We see the same patterns come up repeatedly. A few examples:
Non-usable data in the warehouse. It's there, but in a format that can't be used directly for automation. A data team is needed to build every query. Real-time simply isn’t an option.
Definitions that differ by team. What is an 'active customer'? Marketing says it’s someone who opened an email in the past 90 days. Sales says it’s someone who made a purchase in the past 12 months. Finance says it’s someone with no outstanding invoices older than 60 days. Three definitions, three versions of the truth – and alignment between teams is often nowhere to be found.
Historical data with inconsistent field structures. Two years ago, the field was called 'industry', last year 'sector' and now ‘line of business’. Technically three fields, semantically still one. A query spanning five years of data? That requires mapping nobody has ever documented.
Match rates that fall short. System A and System B are integrated, but only 60 per cent of records match, simply because email addresses aren't consistent or company names are spelled varyingly.
)
Five steps to make data quality measurable
Improving data quality doesn't start with a large clean-up project. There's an essential step before the big clean-up: measuring. From there, you build a realistic approach to maintaining your database on an ongoing basis. Below is a step-by-step plan for a clean-up project.
Step 1: Measure before you automate.
Run an audit on the fields that are critical to your automation: email validation, completeness of fields, percentage of duplicates, recency of updates. This gives you a baseline and prevents surprises halfway through an implementation.
Step 2: Clean up before you build.
Data cleansing isn't sexy, but it is the foundation everything needs to be built on. Remove duplicates, validate email addresses and standardise field values. It takes time, but you'll win back that time by running effective automation on data that's genuinely usable.
Step 3: Define and document core concepts.
What is a lead? An opportunity? A customer? Write it down. Make sure all systems use the same definitions and update the documentation when definitions change. This may sound bureaucratic, but it prevents the chaos of five competing versions of the truth.
Step 4: Set up governance.
Who can create records? Who can edit fields? What are the mandatory fields and permitted values? Governance isn't fun, but it is necessary. Without it, data degrades faster than you can clean it up.
Step 5: Plan for ongoing maintenance.
Data decays. People change jobs, companies merge and email addresses change. Address this by scheduling regular maintenance: removing bounced emails, archiving inactive records and checking field consistency. Treat your database as a living system that needs attention and upkeep – not something you set up once and leave be.
)
From volume to valuable data
The question many organisations ask is: "How do we get more data into the system?" They see volume as an end in itself. And that's not difficult – you can simply add more fields, build more integrations or connect more sources.
The question you should be asking is: "How do we make the data we already have usable?" Quality over quantity. A hundred thousand clean records beat a million dirty ones. Five consistently filled in fields beat fifty fields that are empty or inconsistent.
In practice, this means measuring data quality on an ongoing basis rather than as a one-off exercise. Define what data you need for automation and focus on that. Remove fields that aren't being used. Build cleansing in as a continuous activity and be willing to delete data that doesn't contribute.
Data quality isn't an IT issue. It's a business issue.
Organisations that successfully deploy automation don't treat data quality as a technical by-product. They treat it as a strategic prerequisite. They measure, define, clean up and maintain. Not once, but continuously.
Want to think through the data quality challenges in your organisation? Feel free to get in touch. Our experts are happy to help.
)
Get started with data quality automation
Looking for a complete approach to data quality and business automation, including the PPIT framework and a readiness check? Download the whitepaper CTRL+SHIFT: The four factors that determine whether business automation succeeds or fails.
Download the whitepaper