Author : Rabia Turan
Publisher :
ISBN 13 : 9781124522081
Total Pages : 228 pages
Book Rating : 4.5/5 (22 download)
Book Synopsis High Quality Entity Resolution with Adaptive Similarity Functions by : Rabia Turan
Download or read book High Quality Entity Resolution with Adaptive Similarity Functions written by Rabia Turan and published by . This book was released on 2011 with total page 228 pages. Available in PDF, EPUB and Kindle. Book excerpt: Real-world datasets often contain missing, erroneous, and duplicate data. If such problems with dataset are not corrected, the analysis results on it might lead to wrong decisions. Due to practical significance of the data quality problem, many creative techniques have been proposed in the past to address such problems. In this thesis, we address one such data cleaning challenge, called entity resolution that deals with ambiguous references in data and whose task is to identify all references that co-refer. In this thesis, we exploit additional information sources to improve the disambiguation quality and overcome the limitations of feature-based approaches. Implicit relationships between entities is one such information source. We exploit relationship analysis. The approach we utilize views data as an entity-relationship graph and rely on measuring the connection strength (CS) among various entities in the graph by using a connection strength model. We propose a new adaptive similarity function that improves the quality of these approaches by adaptively learning the CS measure using the available training data. Another information source is the web. We propose an approach that utilizes web querying to measure the correlation information between entities. We also develop a classifier that converts the web-based correlation statistics into ``co-refer'' or ``do-not-co-refer'' decisions. The classifier is based on skylines and leverages the fact that the classification results are utilized in clustering. Our extensive experiments show that the proposed techniques have significant improvement over the state-of-the-art approaches. Entity resolution solutions often produce results consisting of objects whose attributes may contain uncertainty. This uncertainty is frequently captured in the form of a set of multiple mutually exclusive value choices for each uncertain attribute along with a measure of probability for alternative values. However, the applications built on top of such data requires deterministic answers. Thus, we propose a linear time algorithm that finds a deterministic answer set, which maximizes the expected $F_\alpha$ measure of selection queries on top of such a probabilistic representation. The proposed solution gets near-optimal results.
