Adaptive video indexing and automatic/semi-automatic relevance feedback

This paper presents adaptive methods for content-based retrieval in video database applications. A new adaptive video indexing (AVI) technique based on a template-frequency model, together with a self-training retrieval architecture, is proposed, to allow full use of temporal information. AVI takes into account spatio-temporal information for relevance feedback analysis of the dynamic content of video data. The AVI indexing method can be effectively adapted for video shot, scene, and story queries, in order to facilitate multiple-level access to a video database. Our system incorporates this indexing structure to a self-training neural network which implements automatic adaptive retrieval, through its signal propagation process. This greatly reduces the search time for video transmissions over the Web because relevance feedback is implemented in automatic and semi-automatic fashions. The AVI structure not only works well in fully automatic mode, but is also effective in the user-interaction interface system, to achieve a user-friendly environment. Experimentally, we demonstrated the proposed indexing technique and automatic relevance feedback for retrieval of CNN news videos. We also investigated the resilience of the system with a user-controlled interaction process and applied this to an automatically indexed database of 20 h of Hollywood movies.