[attachment=14715]
Abstract
A major assumption in many machine learning and data mining algorithms is that the training and future data must be
in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold.
For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another
domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases,
knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data labeling
efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on
categorizing and reviewing the current progress on transfer learning for classification, regression and clustering problems. In this survey,
we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask
learning and sample selection bias, as well as co-variate shift. We also explore some potential future issues in transfer learning
research.
Index Terms—Transfer Learning, Survey, Machine Learning, Data Mining.
1 INTRODUCTION
Data mining and machine learning technologies have already
achieved significant success in many knowledge engineering
areas including classification, regression and clustering (e.g.,
[1], [2]). However, many machine learning methods work well
only under a common assumption: the training and test data are
drawn from the same feature space and the same distribution.
When the distribution changes, most statistical models need to
be rebuilt from scratch using newly collected training data. In
many real world applications, it is expensive or impossible to
re-collect the needed training data and rebuild the models. It
would be nice to reduce the need and effort to re-collect the
training data. In such cases, knowledge transfer or transfer
learning between task domains would be desirable.
Many examples in knowledge engineering can be found
where transfer learning can truly be beneficial. One example
is Web document classification [3], [4], [5], where our goal
is to classify a given Web document into several predefined
categories. As an example in the area of Web-document
classification (see, e.g., [6]), the labeled examples may be
the university Web pages that are associated with category
information obtained through previous manual-labeling efforts.
For a classification task on a newly createdWeb site where the
data features or data distributions may be different, there may
be a lack of labeled training data. As a result, we may not be
able to directly apply the Web-page classifiers learned on the
university Web site to the new Web site. In such cases, it would
be helpful if we could transfer the classification knowledge
into the new domain.
The need for transfer learning may arise when the data can
be easily outdated. In this case, the labeled data obtained in
one time period may not follow the same distribution in a
later time period. For example, in indoor WiFi localization
Department of Computer Science and Engineering, Hong Kong University of
Science and Technology, Clearwater Bay, Kowloon, Hong Kong
Emails: {sinnopan, qyang}[at]cse.ust.hk
problems, which aims to detect a user’s current location based
on previously collected WiFi data, it is very expensive to
calibrate WiFi data for building localization models in a largescale
environment, because a user needs to label a large
collection of WiFi signal data at each location. However, the
WiFi signal-strength values may be a function of time, device
or other dynamic factors. A model trained in one time period
or on one device may cause the performance for location
estimation in another time period or on another device to be
reduced. To reduce the re-calibration effort, we might wish to
adapt the localization model trained in one time period (the
source domain) for a new time period (the target domain), or
to adapt the localization model trained on a mobile device (the
source domain) for a new mobile device (the target domain),
as done in [7].
As a third example, consider the problem of sentiment
classification, where our task is to automatically classify the
reviews on a product, such as a brand of camera, into positive
and negative views. For this classification task, we need to
first collect many reviews of the product and annotate them.
We would then train a classifier on the reviews with their
corresponding labels. Since the distribution of review data
among different types of products can be very different, to
maintain good classification performance, we need to collect
a large amount of labeled data in order to train the reviewclassification
models for each product. However, this datalabeling
process can be very expensive to do. To reduce the
effort for annotating reviews for various products, we may
want to adapt a classification model that is trained on some
products to help learn classification models for some other
products. In such cases, transfer learning can save a significant
amount of labeling effort [8].
In this survey article, we give a comprehensive overview of
transfer learning for classification, regression and clustering
developed in machine learning and data mining areas. There
has been a large amount of work on transfer learning for
reinforcement learning in the machine learning literature