Modeling uncertainty in video retrieval: a retrieval model for uncertain semantic representations of videos

The amount of multimedia content increases at a tremendous speed every day -- enabled by ever simpler production and processing techniques. Content based Multimedia Information Retrieval should help the user find information in these growing collections. However, the retrieval performance often prohibits real life application. The reason is the mismatch between the user's information need and the available low-level features. A lot of research aims to detect more meaningful representations of the multimedia objects, such as the occurrences of concepts or spoken words. However, the detection results are still far from being perfect. On the other hand, for most detectors it is possible to obtain a probability distribution over the possible representations of the object. This paper proposes the Expected Probability of Relevance Ranking Framework. A relationship between the objects representation and relevance is established by the probability of relevance given the representation, abstracting from the uncertain knowledge of this representation. The objects are then ranked by these probabilities weighted by the probability that the object has this representation, the expected probability of relevance. After introducing the framework we present previous work which estimates the probability of relevance given the binary occurrences of concepts as a reperesentation without prior relevance information. Then, following extensions are proposed: (1) a way to integrate the spoken content in the representation, (2) ways to improve the knowledge over the representation of individual shots and (3) a way to simulate detectors of a given performance to see what detector quality is needed.