Abstract
integration. Most state-of-the-art record matching methods are supervised, which requires the user to provide training data. Thesemethods are not applicable for the Web database scenario, where the records to match are query results dynamically generated onthe-fly. Such records are query-dependent and a prelearned method using training examples from previous query results may fail onthe results of a new query. To address the problem of record matching in the Web database scenario, we present an unsupervised,online record matching method, UDD, which, for a given query, can effectively identify duplicates from the query result records ofmultiple Web databases. After removal of the same-source duplicates, the “presumed” nonduplicate records from the same source canbe used as training examples alleviating the burden of users having to manually label training examples. Starting from the nonduplicateset, we use two cooperating classifiers, a weighted component similarity summing classifier and an SVM classifier, to iteratively identifyduplicates in the query results from multiple Web databases. Experimental results show that UDD works well for the Web databasescenario where existing supervised methods do not apply.Index Terms—Record matching, duplicate detection, record linkage, data deduplication, data integration, Web database, query resultrecord, SVM.
1 INTRODUCTION
TODAY, more and more databases that dynamicallygenerate Web pages in response to user queries areavailable on the Web. These Web databases compose the deepor hidden Web, which is estimated to contain a much largeramount of high quality, usually structured information andto have a faster growth rate than the static Web. Most Webdatabases are only accessible via a query interface throughwhich users can submit queries. Once a query is received,the Web server will retrieve the corresponding results fromthe back-end database and return them to the user.To build a system that helps users integrate and, moreimportantly, compare the query results returned from multipleWeb databases, a crucial task is to match the differentsources’ records that refer to the same real-world entity. Forexample, Fig. 1 shows some of the query results returned bytwo online bookstores, booksamillion.com and abebooks.com, inresponse to the same query “Harry Potter” over the Titlefield. It can be seen that the record numbered 3 in Fig. 1a andthe third record in Fig. 1b refer to the same book, since they have the same ISBN number although their authors differsomewhat. In comparison, the record numbered 5 in Fig. 1aand the second record in Fig. 1b also refer to the same book ifwe are interested only in the book title and author.1The problem of identifying duplicates,2 that is, two (ormore) records describing the same entity, has attractedmuch attention from many research fields, includingDatabases, Data Mining, Artificial Intelligence, and NaturalLanguage Processing.3 Most previous work4 is basedon predefined matching rules hand-coded by domainexperts or matching rules learned offline by some learningmethod from a set of training examples. Such approacheswork well in a traditional database environment, where allinstances of the target databases can be readily accessed,as long as a set of high-quality representative records canbe examined by experts or selected for the user to label.In the Web database scenario, the records to match arehighly query-dependent, since they can only be obtainedthrough online queries. Moreover, they are only a partialand biased portion of all the data in the source Webdatabases. Consequently, hand-coding or offline-learningapproaches are not appropriate for two reasons. First, thefull data set is not available beforehand, and therefore, goodrepresentative data for training are hard to obtain.


Download full report
http://ieeexplore.ieeeiel5/69/4358933/04...er=4840347