An Ensemble Blocking Scheme for Entity Resolution of Large and Sparse Datasets

Understand

Entity Resolution, also called record linkage or deduplication, refers to the process of identifying and merging duplicate versions of the same entity into a unified representation.

• The standard practice is to use a Rule based or Machine Learning based model that compares entity pairs and assigns a score to represent the pairs' Match/Non-Match status.
• However, performing an exhaustive pair-wise comparison on all pairs of records leads to quadratic matcher complexity and hence a Blocking step is performed before the Matching to group similar entities into smaller blocks that the matcher can then examine exhaustively.
• Several blocking schemes have been developed to efficiently and effectively block the input dataset into manageable groups.