ABSTRACT
We propose a new “content-free” image retrieval methodwhich attempts to exploit certain common tendencies thatexist among people’s interpretation of images from user feedbacks.The system simply accumulates records of user feedbackand recycles them in the form of collaborative filtering.We discuss various issues of image retrieval, argue forthe idea of content-free, and present results of experiment.The results indicate that the performance of content-freeimage retrieval improves with the number of accumulatedfeedbacks, outperforming a basic but typical conventionalcontent-based image retrieval system.
1. INTRODUCTION
A picture is said to be worth a thousand words. If this statementis true, it is no wonder that computerized image retrievalis a challenging task. Many efforts have been madein the last decade [1], and they reveal that a key to a capableimage retrieval system is how to extract and describe theimage contents.One obvious approach is to describe the image contentsverbally, typically keywords. Once the verbal descriptionsare obtained, text search techniques can be applied to retrieveimages in the database allowing query-by-keyword.However, this assumption is seldom met; manual labelingis too expensive and automatic methods are not reliable forthe moment. Limited success is reported in automatic imageclassification [2]. Only few objects, such as faces orcars can be recognized reliably from general images. Recentattempts to automatically learn the relations betweenimage regions and keywords have not yet achieved satisfactoryresults [3]. Some researchers turned to alternative informationsources. For images on web pages, the use of filenames, path names, and surrounding text has been proposedand deployed by commercial search engine companies. Ahnet al [4] has proposed a novel approach to combine manualimage labeling and network games.The other approach is to represent images with nonverbaldescriptions which can be reliably computed fromimages. Typical such descriptions are image features basedon color, shape, and texture[1]. Conventional content-basedimage retrieval (CBIR) methods use these image features todefine image similarity. Finding a good set of features isvery critical since the rest is built upon it. However, sincewe have not revealed human visual perception mechanisms,proposed features and image similarity measures are rathercomputer-centric. Interestingly, even such features workreasonably well in some occasions, although they achieveseverely limited success for most of cases. There is a differencebetween what image features can distinguish and whatpeople perceive from the image. This difference, or the “semanticgap,” is the core of the limitation. Human perceptionof images is complex and seems to be dependent on context,purpose, and individual cases. Image representations needto reflect such characteristics of human visual perception.Besides seeking for more suitable image features, manyresearchers have reported that improved results are obtainedby incorporating user feedbacks into the content-based imageretrieval system [5][6]. Typically, as the system showsthe retrieved images to the user, he/she tells the system whichimages in the output are more relevant or less relevant to thequery. Given relevance feedbacks from a user, the systemdetermines which image features are to be used to duplicatethe user’s decision and make changes to the parameters orweights in the underlying model of image similarity. Thefeedback procedures are repeated as necessary.We have proposed a new approach to image retrievalthat uses user feedbacks in the form of interpretation ratherthan through image features, thus directly utilizing humanperceptive power [7]. Relations among images are exploitedrather than the image “contents”. We adopt collaborative filteringtechniques to accumulate feedbacks of all users anduse them to help future users. By bypassing image features,the performance improvement will not be restricted by thepredefined capabilities of feature selection or object recognitionperformance. We will name our approach “content- free” image retrieval (CFIR) in order to illustrate the pointthat it does not analyze image pixels. Naturally, the traditional“content-based” approach must be combined in the finalsystem, but we will explore and emphasize the “contentfree”aspect throughout this paper.
2. CONTENT-FREE IMAGE RETRIEVAL
2.1. Content-free Concept
Relevance feedback methods have proven that humans canplay an important role in the success of image retrieval;even simple user feedbacks help improve the performanceof content-based image retrieval methods. The fundamentalreason for this is that human can provide consistent andreliable judgment of whether presented images are relevantto what he/she is looking for. By receiving the teaching signals,content-based methods can learn how to respond to thequery. However, we observe two different types of limitationin this scheme. Firstly, the selection of image featurelimits the capability of model-fitting. Secondly, several iterationsof feedbacks will not provide enough data to traina complex vision model. To utilize knowledge from usersmore effectively, we omit image features and use the human’sperceptual decisions themselves.Note that relevance feedbacks are tolerable amount ofmanual labor enforced on users to achieve their goal. Becauseof this nature, each feedback carries little but reliableinformation regarding how images are related to each other.We believe that an effective image retrieval system can berealized using only the usage history of users. We recordall of these feedbacks from all of the users. The aggregatedfeedbacks should work as asynchronous voting on relationsamong images in the database. Once enough feedbacks areaccumulated, the system can learn and summarize those relationsin a certain form. Subsequently the system retrievesrelevant images for a new query from a new user using thelearned relations, and the result is expected to agree with themajority’s perception. Unlike the content-based approach,this scheme lets all image processing and perception tasksbe done by a population of users, and uses the learned relationsfrom them to do the retrieval task. Hence the name:“content-free” approach.Some research efforts have been conducted in similarconcept. However, they used the accumulated user feedbacksin content-based frameworks so that the performanceis restricted by image features [


Download full report
http://citeseerx.ist.psu.edu/viewdoc/dow...1&type=pdf