Patent data normalisation: purpose and research boundary
Patent data normalisation prepares inconsistent raw records for defensible analysis. It should preserve the original fields, maintain a traceable transformation ledger and avoid treating a cleaned name or family identifier as an authoritative legal fact.
Illustrative research scenario
Two records may identify the same applicant with abbreviations and language differences. A normalization routine can group them provisionally, but reviewers should preserve both original values and a confidence measure. Similarly, two publication identifiers should not be merged unless their country, number and kind codes have been interpreted correctly.
Raw data preservation
Save the untouched supplier extract with a timestamp, declared encoding, source license, retrieval method and checksum. Transformations should produce new versions rather than overwriting the evidence.
Identifier normalization
Standardize country, publication, application, priority and kind-code fields without dropping meaningful distinctions. Identifier formats vary and blind stripping of letters may merge legally distinct records.
Applicant and inventor names
Apply organization dictionaries, aliases and entity-resolution thresholds, while preserving individuals separately and considering privacy. Similar names should trigger human verification before they are consolidated.
Date and temporal fields
Label priority, application, publication, grant and legal event dates separately. Convert calendar formats carefully and store precision when an original value lacks a complete day or month.
Family and classification columns
Record the exact family definition and classification version used. Retain multi-valued IPC/CPC fields rather than reducing every record to a single technology label.
Quality assurance
Use null-rate reports, duplicate candidates, field-distribution comparisons and reversible sample checks. Summaries should disclose filtered records and uncertainty introduced by missing source fields.
Additional methodological safeguard 1
A reproducible cleaning notebook should report how many records are lost at every rule and preserve samples of changed identifiers.
Additional methodological safeguard 2
Resolve ambiguous organizational aliases through a confidence workflow rather than replacing every spelling with a presumed parent company.
Additional methodological safeguard 3
Patent family IDs from different suppliers may not be interchangeable; never join tables on unverified identifier equivalence.
A traceable patent-data transformation ledger
Patent analytics often fails at the identifier layer. Applications, publications, grants and families can refer to the same invention without sharing an identical visible number. Preserve the original record and create explicit normalized fields for office, kind code, number, filing date and priority identifier. Keep uncertain normalization separate from verified equivalence. For assignees, retain the as-published names and a separate mapped corporate-entity field so a later correction does not erase historical evidence. Document the matching rules and provide a manual review state for ambiguous records.
Choose duplicate removal based on the analytical question. Counting publications, priority families and granted patents are different metrics; applying one deduplication rule across all three will create systematic errors. A simple family merge based solely on shared priority may combine related technical subject matter that should remain distinct for some landscape questions. Log the selected family algorithm, data-provider implementation and exception cases. Examine improbable date order, missing country codes, inconsistent classifications and publication-kind anomalies before making charts.
A quality report should quantify missingness, identifiers that fail validation, corporate mappings requiring manual approval and records excluded from the output. Compare a random sample against official source documents to estimate transformation errors. Store source access dates and schema versions, and preserve a record of every code or spreadsheet transformation. Do not report a clean dataset merely because the script terminated successfully: a stable pipeline can consistently produce wrong family counts. Good cleaning enables transparent reuse and correction when raw patent-office records change.
Primary sources and scope of verification
EPO worldwide bibliographic data (DOCDB) is a relevant original institutional reference for the narrow claims described in this article. Open the linked publication to check the precise version and its definitions; do not attribute this article's recommended workflow to the publisher as a prescribed legal test unless the source explicitly states it.
EPO Open Patent Services overview is a relevant original institutional reference for the narrow claims described in this article. Open the linked publication to check the precise version and its definitions; do not attribute this article's recommended workflow to the publisher as a prescribed legal test unless the source explicitly states it.
Interpretation and responsible use
This research method is presented as a reproducible framework, not a completed jurisdiction-specific assessment or a claim about a particular client matter. A research outcome depends on the disclosed problem, the database and legal source editions, the date of collection and the underlying documentary evidence. Where source access or policy verification is incomplete, the open item should be recorded instead of filling the gap with an invented proposition. Readers should compare the cited authority with the current official record before relying on time-sensitive conclusions.
Illustrative decision and validation example
An implementation example should show a raw record alongside the standardized result. Suppose one source supplies a publication number without its kind code while another supplies a formatted office prefix. Do not merge merely because the numeric suffix matches; preserve the authority, complete number, document type and uncertainty flag. For company normalization, two applicant strings may have a shared commercial brand but refer to distinct legal entities. If the analyst chooses group-level consolidation, retain both source names and the rule supporting the mapping. Record the count before and after each deduplication stage, including the number of ambiguous rows held for review. A test set with known identifiers should be rerun after mapping-rule changes. Version the output schema and document any record exclusions caused by malformed fields. The purpose is not to produce a visually tidy spreadsheet alone; it is to ensure that a reader can trace every charted count back to a defensible source record and transformation decision.





