Back up everything first, then let your CRM's own duplicate tool find and merge the obvious duplicates. Use an AI assistant on an exported spreadsheet to standardise company names, job titles and phone numbers and to pull useful facts out of free-text notes. Re-import in small batches, archive dead records, and add required fields so it stays clean.
For a CRM of around 5,000 records, allow two to four working days, most of it checking rather than doing. AI is good at the judgement calls a formula can't make, such as deciding that "Example Trading Ltd" and "EXAMPLE TRADING LIMITED" are the same buyer while "Example Tiles" is not. It's poor at knowing which of two conflicting phone numbers is current; that needs a person, or a quick call. The example throughout is an illustrative import-export business whose CRM holds buyers, suppliers, freight agents and ten years of half-finished deals.
Measure the mess before touching it
Export your contacts, companies and deals to spreadsheets and count what's wrong. You need a baseline to know when you're finished, and the counts tell you where to spend the time. Here's the import-export firm's audit:
| Check | How to count it | Result |
|---|---|---|
| Contacts / companies / open deals | Row counts | 6,240 / 2,110 / 340 |
| Companies that look duplicated | CRM duplicate tool, or AI on the export | About 14% |
| Contacts with no email | Blank email column | 9% |
| Contacts whose last email bounced | Bounce status field | 6% |
| Open deals with no activity for 120+ days | Last activity date | 212 of 340 |
| Records with no owner | Blank owner | 18% |
| Different spellings in "Contact type" | Count unique values | 37 variants of what should be 5 options |
| Industry left blank | Blank industry | 55% |
You can get most of these counts by asking an AI assistant with data analysis to profile the file. Remove columns it doesn't need first (personal phone numbers, notes with sensitive details) and use a business plan that doesn't train on your data; whether it's safe to put customer data into ChatGPT explains the settings. A prompt that works:
This is an export of our CRM companies table.
For every column: count blanks, count unique values, and list
the 10 most common values. Flag columns that look like they
should be a fixed list (under 20 real options) but have many
spellings. Then list groups of rows that may be the same
company, with the reason. Don't change the data.
An illustrative result flagged the "Contact type" column ("Buyer", "buyer", "Customer", "Cust.", "Client - active" and 32 more), found 290 possible duplicate groups, and noted that a "Country" column mixed full names, two-letter codes and blanks. Check a sample of each finding before acting on it; the profile is a map, not a verdict.
Back up, and decide the merge rules before any change
Take a full export of every object, including each record's unique ID, and store it somewhere safe and dated. The backup checklist before connecting AI tools covers what to keep. This matters more than usual because some clean-up actions can't be reversed. In HubSpot, for example, merging is permanent: its knowledge base states plainly that "It's not possible to unmerge records". When two HubSpot records merge, the primary record's property values are generally kept, and the secondary record's values fill only the primary's blanks (see HubSpot's article on merging records).
So decide in advance which record survives a merge. The import-export firm's survivor rules:
| If the duplicates differ on… | The survivor is the one that… |
|---|---|
| Activity | Has the most recent email, call or meeting logged |
| Deals | Has an open deal (deals are re-associated, not lost) |
| Owner | Is owned by the person who currently manages the account |
| Contact details | Has the verified, non-bounced email |
| Two different billing entities | Don't merge: link them under a parent company instead |
Write the rules down and give them to whoever does the merging, so two people don't make opposite choices on the same afternoon.
Duplicates: the CRM finds them, AI judges the hard ones
Use the CRM's built-in tools for the bulk of the work, because they merge records properly: activities, deals and emails come across with them. A spreadsheet can't do that.
- HubSpot already stops some duplicates at the door: it matches new contacts by email address and new companies by their domain name. For the rest, its Manage duplicates tool suggests pairs by comparing names, email, phone, company and other properties, and recalculates as records are created. It's available on Professional and Enterprise plans (Marketing, Sales, Service, Data Hub or Smart CRM), per HubSpot's duplicates article. On free or Starter plans you merge pairs one at a time from the record.
- Pipedrive has a Merge duplicates tool under Tools and apps that lists possible duplicate people and organisations, using the same matching rules as its import tool. Only admins, or users given the permission, can merge.
The CRM tools are cautious, which is right, and they leave the ambiguous pairs to you. That's where AI earns its place. Export the suggested or suspected pairs and ask for a verdict with reasons:
For each pair of company records below, decide:
SAME (one business), DIFFERENT, or CHECK (can't tell).
Use name, domain, phone, address and contact names.
Legal suffixes (Ltd, Limited, LLC, Inc) and capitals don't matter.
Different domains or different billing addresses usually
mean DIFFERENT or CHECK. Give a one-line reason.
Return a table: pair_id, verdict, reason.
Illustrative output for three of the firm's pairs:
| Pair | Records | Verdict | Reason given |
|---|---|---|---|
| 112 | "Example Trading Ltd" / "EXAMPLE TRADING LIMITED" | SAME | Same domain and phone; suffix and capitals differ |
| 113 | "Example Trading Ltd" / "Example Tiles" | DIFFERENT | Different domains and product lines |
| 114 | "Sample Foods Group" / "Sample Foods (Retail) Ltd" | SAME | Same domain, similar name |
Pair 114 is the realistic trap. The two records share a group website but are separate legal entities with separate billing and separate buyers; the retail arm pays on 30 days, the group on 60. Merging them would have mixed two credit histories into one record, and in HubSpot it couldn't have been undone. The fix was a line in the prompt ("If names suggest a group and a subsidiary, answer CHECK") and a rule that a person reviews every SAME verdict where the names aren't nearly identical. Treat the AI's verdicts as a sorted to-do list: the obvious SAMEs go through quickly, and your attention goes to the CHECKs.
Standardising fields on an export
Once duplicates are merged, export again and fix formats. This is where AI saves most time, because the rules are fuzzy. Before and after for four of the firm's rows:
| Field | Before | After |
|---|---|---|
| Company name | EXAMPLE TRADING LIMITED | Example Trading Ltd |
| Job title → Role (fixed list) | "Head Buyer - Chilled & Ambient" | Buyer |
| Job title → Role | "MD / Owner" | Owner or director |
| Phone | "00 [country code] (0) 555 0123 ext 12" | +[country code]5550123, with extension 12 in its own field |
| Contact type | "Cust.", "Client - active", "buyer" | Customer |
The phone format in the "after" column is the international standard (a plus sign, the country code, then the number with no spaces or leading zero), which lets the CRM's dialler and any texting tool use it. A prompt for the job-title column:
Map each job title to exactly one Role from this list:
Owner or director | Buyer | Logistics or shipping | Finance |
Operations | Sales (their side) | Other
If a title fits two, choose the one about buying decisions.
If it's unclear, answer Other and add "?" so I can check.
Return: row_id, original_title, role.
Spot-check fifty rows before accepting the rest. In the firm's run, the AI mapped "Shipping Manager" to Buyer on a few rows because those people also placed orders; the rule was clarified ("Role is their job, not every task they do") and the column re-run. For purely mechanical fixes, such as trimming spaces or splitting names, spreadsheet functions are quicker and more predictable than AI; cleaning messy data in Excel covers those.
Filling blank industries from company websites
More than half the firm's companies had no industry or product category, which made it impossible to answer simple questions such as "which buyers take spices?". Guessing from the company name is unreliable, so the AI was given each company's own website text (fetched by a research tool, or pasted in for smaller batches) and asked to choose from the firm's category list:
Using only the WEBSITE TEXT below, choose the categories this
company trades in, from: dried fruit, nuts, pulses, spices,
grains, oils, packaging, freight and logistics, other.
Also say whether they are an importer, distributor, retailer,
manufacturer or service provider.
If the text doesn't make it clear, answer "unknown".
Quote the phrase that supports each answer.
The "quote the phrase" line is what makes the output checkable. On a sample of forty, the AI was right on thirty-four, answered "unknown" on four (sites that were just a logo and a phone number), and was wrong on two, both companies whose websites described a parent group rather than the branch in the CRM. Those two errors are why the column was imported as "suggested category" first and promoted to the real field only after account managers had a week to object. If the website can't be found at all, that's useful information too: a company with no working website and no activity in three years is a candidate for archiving.
Rescuing facts buried in notes
Ten years of notes hold information nobody can filter on. An import-export CRM is full of lines like this one:
"Spoke to their buyer early Feb. They prefer FOB, usually 2 x 40ft a quarter, mostly dried fruit and nuts, sometimes pulses. Pay 30% deposit. Best on WhatsApp, not email. Might switch from current supplier if we can do organic cert."
Ask the AI to extract only fields you'll use, and to leave personal comments behind:
From each note, extract these fields if clearly stated:
incoterm_preference, containers_per_quarter, product_categories
(from: dried fruit, nuts, pulses, spices, grains, oils),
payment_terms, preferred_channel, open_opportunity (short phrase).
Use null if not stated. Don't copy names, opinions about people
or anything unrelated to trading.
Illustrative output for that note: incoterm FOB; 2 containers a quarter (40ft); dried fruit, nuts, pulses; 30% deposit; WhatsApp; "wants organic-certified supply". That last field became a list of 23 buyers to contact when the firm added an organic-certified supplier, a list nobody could have produced from the notes before. Check extracted numbers against the note on a sample, since "2 x 40ft" occasionally came back as 240.
Notes also contain things you shouldn't keep, such as comments about a person's health or opinions of individuals. Use the clean-up to delete them; keeping customer data private when your team uses AI covers the wider habits.
Stale deals and dead contacts
Agree the rules, then apply them in bulk:
- Open deals with no activity for 120 days: close as lost with the reason "No response (clean-up)", so reports stop counting them. The firm closed 212, and its pipeline value fell to a believable figure overnight.
- Hard-bounced emails: mark the email as invalid and set a task to find a current address if the company is still a target.
- People who left the company: update the record (or create a new one at their new company, which is often a sales lead in itself).
- Records with no activity for several years and no open business: decide whether to archive or delete. Data-protection law such as the GDPR expects you not to keep personal data longer than you need it, so set a retention rule and ask your data-protection adviser if you're unsure where the line is.
Re-importing without making it worse
The most common clean-up disaster is an import that creates thousands of new duplicates. Avoid it:
- Keep the CRM's record ID column in every file, and map it as the unique key so the import updates existing records rather than creating new ones.
- Make sure dropdown values match the CRM's options exactly, including capitals; a mismatched value is usually rejected or ignored.
- Import a test batch of 200 rows, then open ten of those records and check every changed field.
- Compare record counts before and after. If the total went up, stop and find out why.
- Import the rest in batches of a thousand or so, checking counts each time.
A near miss from the firm's day four shows why the test batch matters. The AI's standardised file had put the cleaned "Role" values in a column still headed "Job title", and the import mapped it straight onto the job-title field. On the 200-row test, "Head Buyer - Chilled & Ambient" became "Buyer" and the detail was gone. It was caught in the ten-record check and fixed from the backup in minutes; across 6,000 contacts it would have wiped every job title. Rename columns to match their destination field exactly before importing, and keep original values in their own column.
The import-export firm's clean-up, day by day
One operations manager did the work across four days, with an hour of each account manager's time for checking their own accounts. Illustrative timings and results:
| Day | Work | Hours |
|---|---|---|
| 1 | Export, back up, audit, write survivor rules | 4 |
| 2 | Merge obvious duplicates in the CRM; AI verdicts on the rest; review CHECKs | 6 |
| 3 | Standardise titles, types, phones and names; extract fields from notes | 5 |
| 4 | Close stale deals, archive dead records, re-import, set up rules | 5 |
| Measure | Before | After |
|---|---|---|
| Companies | 2,110 | 1,820 (290 merged) |
| Open deals | 340 | 128 |
| Contact type variants | 37 | 5 |
| Industry or product category filled | 45% | 81% |
| Records with no owner | 18% | 0% |
The quick sum: 20 hours of one person's time plus four hours of account managers' checking, and no new software, because the firm already had a business AI plan. Done by hand, the same job had been estimated at two to three weeks and kept being postponed, which is how the CRM got messy in the first place. For the same kind of exercise on customer records more generally, cleaning up customer records before you add AI takes a wider view.
Rules that keep it clean afterwards
- Fixed lists instead of free text for contact type, role, industry and lead source. Free text is where the 37 variants came from.
- Required fields when a deal is created: owner, company, source, and expected value.
- Automatic deduplication on the way in: new records from forms and imports matched on email and domain. HubSpot does this by default, and Pipedrive's import tool checks for likely duplicates too.
- A monthly 20-minute audit: rerun the profiling prompt on a fresh export and compare against last month's counts. A jump in duplicates or blanks usually points to one new form or one new colleague.
- A named data owner who decides merge questions and can change the fixed lists. Without one, everyone adds their own option and the mess returns within a year.
Clean data is also what makes the next steps possible: scoring leads, segmenting emails and letting calls update the CRM automatically all depend on fields that mean the same thing on every record. Preparing your business data for AI covers the same discipline for the rest of the business's data.
Further reads
- Is Your Business Data Ready for AI? A Clean-Up Checklist — A broader checklist once the CRM itself is clean.
- How to Update Your CRM Automatically After Every Sales Call — Keep records current by logging calls automatically.
- HubSpot vs Pipedrive for AI Sales Automation in a Small Business — If the clean-up shows you've outgrown your CRM.
- What Is an AI CRM and Which Features Actually Save Time? — Features that only work well on clean data.
- What Is AI Lead Scoring and Does a Small Business Need It? — Scoring leads becomes possible once fields are consistent.
- How to Segment Your Email List With AI, Even a Small One — Put standardised fields to work in targeted emails.
- Using AI to Spot Remortgage Opportunities in Your Client Bank — Turn an untidy client bank into a watchlist of deal-end dates, early-repayment-charge dates and loan-to-value moves, with AI filling the gaps.
- AI Service Plan Reminders for Heating and Cooling Customers — Clean the due-date list, time reminders to the heating and cooling seasons, write messages about each customer's actual system and let AI sort replies.
- What It Costs to Set Up an AI CRM for a Small Team — Itemised AI CRM costs, licence and AI credit prices, a four-person delicatessen costed three ways, and the hidden costs that double year two.
- How Much Does It Cost to Automate Lead Follow-Up With AI? — What each part of an automated follow-up costs, why the AI drafting is the smallest line, a costed furniture-maker example and the one-extra-sale test.
- Signs Your Business Is Ready to Automate Sales Follow-Ups With AI — A 16-point readiness checklist for automating sales follow-ups, each item with how to verify it, plus a filled-in example and a scoring table.
- How to Find B2B Prospects With AI Research Tools — A seven-step method for turning your best customers into a checked list of look-alike companies, named buyers and timely reasons to get in touch.
- How to Create Customer Personas With AI From Real Data — Build personas from booking, order and review data instead of AI guesswork: prompts, a persona template with evidence, and four businesses' results.
- AI Call Summaries: Log Every Phone Enquiry in Your CRM — Turn every inbound phone enquiry into a CRM record with named fields and a callback date, using a phone system that summarises calls for you.
- HubSpot AI Agents and Breeze: What Small Teams Should Switch On — Which HubSpot Breeze features a small team should switch on first, what Customer Agent and Prospecting Agent cost in credits, and how to cap the bill.
- Is It Worth Switching CRM Just for Better AI Features? — Where CRM AI features sit on each price list, the migration costs switching quotes leave out, and a three-year worked sum for a small sales team.
- AI Tools and AI Development: The Complete 2026 Guide — the AI hub, including every tutorial in the AI-for-business series.
Sources: HubSpot knowledge base articles on deduplication, merging records and managing duplicates; Pipedrive knowledge base articles on Merge Duplicates. Checked September 2026.