ranking model adaptation for domain specific search
Posts: 14,118
Threads: 61
Joined: Oct 2014
We describe an adaptation process to adapt a classification model constructed for a broad based search engine for use with a domain-specific classification model. It is difficult to apply the broad-based classification model directly to different domains because of domain differences, to create a unique classification model for each time-consuming domain for training models. This algorithm only requires prediction of existing classification models, rather than their internal representations or auxiliary domain data. The classification model is adapted to Use in a search environment centered on a specific segment of online content, for example, a specific topic, a type of medium or a genre of content. A domain-specific classification model reduces search results to the data of a specific domain that are relevant to the search terms entered by the user. The sort order can be determined by reference to a given numerical score, ordinal score or binary judgment as "relevant" or "irrelevant."
Recently, there are several domain-specific search engines, which are restricted to specific topics or document formats, and vertical to broad-based search. Simply applying the trained classification model for the broad search to the verticals can not achieve a sound performance due to domain differences, while constructing different classification models for each domain is laborious to label enough training samples and the Training process. In order to address the above difficulties, we investigated two problems: (1) whether we can adapt the classification model learned for the search of existing or vertical web pages, to the new domain, so that the amount of data tagged and the cost Training is reduced, while the performance requirement is still met; And (2) how to adapt the classification model of the auxiliary domains to a new destination domain. We address the second problem of the regularization framework and propose an algorithm called classification adaptation. Our algorithm is flexible enough that we only need the prediction of the existing classification model, rather than the internal representation of the model or the data of the auxiliary domains. The first problem is addressed by the proposed classification adaptability measure, which quantitatively estimates whether an existing classification model can adapt to the new domain. Extensive experiments are performed on the set of Letor reference data and two large-scale datasets are traversed from different domains through a commercial search engine on the Internet, where the classification model learned for one domain will be adapted to the other . The results demonstrate the applicability of the adaptation algorithm of the proposed classification model and the measurement of classification adaptability.
It can be understood in the following video: