Ten years of relevance score for content based image retrieval

After more than 20 years of research on Content-Based Image Retrieval (CBIR), the community is still facing many challenges to improve the retrieval results by filling the semantic gap between the user needs and the automatic image description provided by different image representations. Including the human in the loop through Relevance Feedback (RF) mechanisms turned out to help improving the retrieval results in CBIR. In this paper, we claim that Nearest Neighbour approaches still provide an effective method to assign a Relevance Score to images, after the user labels a small set of images as being relevant or not to a given query. Although many other approaches to relevance feedback have been proposed in the past ten years, we show that the Relevance Score, while simple in its implementation, allows attaining superior results with respect to more complex approaches, can be easily adopted with any feature representations. Reported results on different real-world datasets with a large number of classes, characterised by different degrees of semantic and visual intra- e inter-class variability, clearly show the current challenges faced by CBIR system in reaching acceptable retrieval performances, and the effectiveness of Nearest neighbour approaches to exploit Relevance Feedback.