RapidFuzz: flag possible duplicate leads for review

Put similar names into a review queue

A lead intake workflow may receive a company name with different capitalization, punctuation or a small typo. Before creating another CRM record, it can be useful to show a reviewer the closest existing names and their record IDs. RapidFuzz supplies that string-matching step without requiring a language-model call.

This AI-assisted editorial guide tests a small candidate finder for an AI-enabled lead workflow. Every company name and record ID in the example is fictional. The script reads an in-memory dictionary and prints review suggestions. It does not connect a CRM, merge records, contact prospects or generate new leads.

1. Pin RapidFuzz and define the review policy

We verified RapidFuzz 3.14.6 on October 31, 2026. The package uses the MIT license and requires Python 3.11 or later. Our local test used Python 3.13.15 on Windows. Use a separate environment so this exercise does not change an existing application’s dependencies. [1]

The fixture maps stable CRM-style IDs to original company-name strings. Its policy returns at most two candidates with a WRatio score of at least 85. Those numbers are demonstration settings; they have not been calibrated against a real lead dataset. All outputs retain the decision needs_identity_review, including an empty candidate list.

Run these PowerShell commands in a fresh folder. Save the next block as lead_candidates.py before the final command. Installation needs internet access; the example itself uses only local fixture data. If Windows reports a DLL load error, the official installation guide explains the Visual C++ runtime requirement. [2]

mkdir lead-candidate-demo
cd lead-candidate-demo
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install "rapidfuzz==3.14.6"
# Save lead_candidates.py below first.
.\.venv\Scripts\python.exe lead_candidates.py

2. Match names while retaining the original records

process.extract accepts a dictionary of choices. Each returned tuple contains the original choice, its score and the dictionary key. We keep that key as crm_id rather than converting a position in a result list into a record identifier. limit bounds the returned list, and score_cutoff filters lower similarity results for this scorer. [3]

Preprocessing is explicit: processor=utils.default_process. The utility normalizes non-alphanumeric characters, surrounding whitespace and case. The original query and choice strings remain in our output for inspection. This normalization is useful for the fixture, but it can erase distinctions that matter in real company names. [4]

The input checks are our application policy: require a string of 1–120 characters with some content after preprocessing. This function uses a fixed, controlled choice dictionary. A production adapter would also need to validate its source records and access scope.

import json
import platform

import rapidfuzz
from rapidfuzz import fuzz, process, utils

# Fictional fixture, not Sociologix clients or a live CRM.
CRM = {
    "AC-101": "Demo Harbor Property Group",
    "AC-102": "Demo Harbor Property Group East",
    "AC-103": "Demo Juniper Robotics",
    "AC-104": "Demo Meridian Yoga",
}


def review_candidates(company):
    if not isinstance(company, str) or not 1 <= len(company) <= 120:
        raise ValueError("Company must be 1 to 120 characters")
    if not utils.default_process(company):
        raise ValueError("Company needs letters or numbers")
    matches = process.extract(
        company, CRM, scorer=fuzz.WRatio,
        processor=utils.default_process, score_cutoff=85, limit=2
    )
    return {
        "incoming_name": company,
        "decision": "needs_identity_review",
        "candidates": [
            {"crm_id": record_id, "original_name": name,
             "similarity": round(float(score), 2)}
            for name, score, record_id in matches
        ],
    }


if __name__ == "__main__":
    print(json.dumps({
        "versions": {"python": platform.python_version(),
                     "rapidfuzz": rapidfuzz.__version__},
        "examples": [
            review_candidates("DEMO HARBOR PROPERTY GRUP!"),
            review_candidates("  DEMO JUNIPER ROBOTICS!!!  "),
            review_candidates("Quartz Aeronautics"),
        ],
    }, indent=2))

3. Read the observed matches carefully

For DEMO HARBOR PROPERTY GRUP!, the local run returned AC-101, Demo Harbor Property Group, with a score of 98.04, followed by AC-102, Demo Harbor Property Group East, at 89.29. Both are plausible strings to inspect; the shared wording does not tell us whether the records describe the same legal business.

The capitalization-and-punctuation example returned AC-103, Demo Juniper Robotics, at 100.0. That score reflects the normalized string comparison in this fixture. It does not verify a company’s identity, ownership, website or relationship to another record. WRatio calculates a weighted similarity using other ratio algorithms and returns values from 0 to 100. [5]

Quartz Aeronautics returned an empty candidate list. That means this four-record pool produced no returned candidate at the chosen cutoff. It does not establish uniqueness across a full CRM, another spelling or another data source.

Ten local checks passed. They verified ranked CRM IDs, preservation of the original choice and incoming text, the normalized 100-point fixture, the empty result, and the review decision on all three outputs. The remaining checks rejected blank, punctuation-only, nonstring and overlong inputs. Scores above are rounded to two decimal places by our output adapter.

4. Evaluate the threshold with real review outcomes

Before using this in a live intake flow, build an authorized evaluation set containing both confirmed duplicates and similar names belonging to different businesses. Include abbreviations, spelling mistakes, branches and names in the languages your team actually handles. Record whether the correct candidate appears in the returned list and how often reviewers receive misleading suggestions.

Check whether two results are enough for your workflow. A top-two list can omit a useful alternative even when that alternative passes the cutoff. A cutoff of 85 can also exclude a genuine duplicate whose name differs substantially. Keep the scorer, preprocessing, cutoff and result limit together in the workflow’s versioned configuration.

Ask reviewers to use additional approved evidence, such as an existing account identifier, company website and business location. Preserve an explicit unresolved outcome. Do not treat the highest score, or even a normalized 100, as permission to merge records, overwrite contact details or start outreach.

For a larger CRM, measure the lookup against a representative approved dataset before setting capacity targets. This article is a functional test of four choices, not a performance benchmark. If you narrow the search pool before matching, evaluate whether that step accidentally excludes the correct record.

5. Give an AI worker a clear handoff

A proposed AI-enabled intake flow can attach the incoming name, candidate IDs, original names, scores and matching settings to a review item. An AI worker could summarize the visible similarities, while a person checks the underlying records and decides what to do. This article did not generate or evaluate such a model summary.

Keep access limited to the approved records for the business using the workflow. Name matching does not establish permission to inspect customer data. Store the reviewer’s decision and the source IDs so the team can revisit an incorrect association without losing the original evidence.

After approval, handle any CRM write as a separate, tested action with its own identity checks and retry behavior. Keep no-candidate and unresolved cases visible to the team. Candidate ranking helps organize attention; the business still needs a defined process for deciding whether two leads belong together.

Sources & further reading

1. Official RapidFuzz 3.14.6 package metadata: Python requirement and MIT license
2. RapidFuzz installation: packages, Windows runtime and troubleshooting
3. RapidFuzz process.extract: mapping keys, ranking, cutoff and result limit
4. RapidFuzz default_process: explicit string preprocessing
5. RapidFuzz WRatio: weighted similarity and score range

Make AI lead intake easier to review

Sociologix can help connect lead intake, CRM workflows and AI workers with clear review queues and traceable decisions. Bring one recurring intake process and the rules your team uses to recognize an existing account.