ImageGrouper: a group-oriented user interface for content-based image retrieval and digital image arrangement

In content-based image retrieval (CBIR), experimental (trial-and-error) query with relevance feedback is essential for successful retrieval. Unfortunately, the traditional user interfaces are not suitable for trying different combinations of query examples. This is because first, these systems assume query examples are always added incrementally. Second, the query and the result display are done on the same workspace. Once the user removes an image from the query examples, the image may disappear from the user interface. In addition, it is difficult to combine the result of different queries. In this paper, we propose a new interface for Content-based image retrieval named ImageGrouper. ImageGrouper is a Group-Oriented user interface in that all operations are done by creating groups of images. This approach has several advantages. First, the users can interactively compare different combinations of the query examples by dragging and grouping the images on the workspace (Query-by-Group). Because the query results are displayed on another pane, the user can quickly review the results and modify the query. Combining different queries is also easy. Furthermore, the concept of “Image Groups” is also applied for annotating and organizing many images. The Annotation-by-Groups method relieves the user of tedious task of annotating textual information on a large number of images. This method realizes a hierarchical annotation of the images as well as Bulk Annotation. The Organize-by-Group method lets the users manipulate the image groups as “Photo Albums” to organize the images. Finally, the usability of the system is compared with the traditional user interfaces. By incorporating the lessons from the experiments, the usability of ImageGrouper is further improved.