Class Feature Pyramids for Video Explanation

Alexandros Stergiou, Georgios Kapidis, Grigorios Kalliatakis, Christos Chrysoulas, Ronald Poppe, Remco Veltkamp

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

9 Citations (Scopus)

Abstract

Deep convolutional networks are widely used in video action recognition. 3D convolutions are one prominent approach to deal with the additional time dimension. While 3D convolutions typically lead to higher accuracies, the inner workings of the trained models are more difficult to interpret. We focus on creating human-understandable visual explanations that represent the hierarchical parts of spatio-temporal networks. We introduce Class Feature Pyramids, a method that traverses the entire network structure and incrementally discovers kernels at different network depths that are informative for a specific class. Our method does not depend on the network's architecture or the type of 3D convolutions, supporting grouped and depth-wise convolutions, convolutions in fibers, and convolutions in branches. We demonstrate the method on six state-of-the-art 3D convolution neural networks (CNNs) on three action recognition (Kinetics-400, UCF-101, and HMDB-51) and two egocentric action recognition datasets (EPIC-Kitchens and EGTEA Gaze+).

Original languageEnglish
Title of host publication2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
PublisherIEEE
Pages4255-4264
Number of pages10
ISBN (Electronic)978-1-7281-5023-9
ISBN (Print)978-1-7281-5024-6
DOIs
Publication statusPublished - 5 Mar 2020
Externally publishedYes
Event17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019 - Seoul, Korea, Republic of
Duration: 27 Oct 201928 Oct 2019

Publication series

NameIEEE International Conference on Computer Vision workshop
PublisherIEEE
Volume2019
ISSN (Print)2473-9936
ISSN (Electronic)2473-9944

Conference

Conference17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019
Country/TerritoryKorea, Republic of
CitySeoul
Period27/10/1928/10/19

Keywords

  • n/a OA procedure
  • Spatio-temporal-cnns
  • Visual-explanations
  • Saliency-visualization

Fingerprint

Dive into the research topics of 'Class Feature Pyramids for Video Explanation'. Together they form a unique fingerprint.

Cite this