Extending a DBMS to Support Content-Based Video Retrieval: A Formula 1 Case Study

Content-based retrieval has been identified as one of the most challenging problems, requiring a multidisciplinary research among computer vision, information retrieval, artificial intelligence, database, and other fields. In this paper, we address the specific aspect of inferring semantics automatically from raw video data. In particular, we present the Cobra video database management system that supports the integrated use of different knowledge-based methods for mapping low-level features to high-level concepts. We focus on dynamic Bayesian networks and demonstrate how they can be effectively used for fusing the evidence obtained from different media information sources. The approach is validated in the particular domain of Formula 1 race videos. For that specific domain we introduce a robust audio-visual feature extraction scheme and a text recognition and detection method. Based on numerous experiments performed with DBNs, we give some recommendations with respect to the modeling of temporal dependences and different learning algorithms. Finally, we present the experimental results for the detection of excited speech and the extraction of highlights, as well as the advantageous query capabilities of our system.