Leaping Into Memories Space-Time Deep Feature Synthesis

Alexandros Stergiou, Nikos Deligiannis

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Downloads (Pure)

Abstract

The success of deep learning models has led to their adaptation and adoption by prominent video understanding methods. The majority of these approaches encode features in a joint space-time modality for which the inner workings and learned representations are difficult to visually interpret. We propose LEArned Preconscious Synthesis (LEAPS), an architecture-independent method for synthesizing videos from the internal spatiotemporal representations of models. Using a stimulus video and a target class, we prime a fixed space-time model and iteratively optimize a video initialized with random noise. Additional regularizers are used to improve the feature diversity of the synthesized videos alongside the cross-frame temporal coherence of motions. We quantitatively and qualitatively evaluate the applicability of LEAPS by inverting a range of spatiotemporal convolutional and attention-based architectures trained on Kinetics-400, which to the best of our knowledge has not been previously accomplished.
Original languageEnglish
Title of host publicationIEEE/CVF International Conference on Computer Vision
Subtitle of host publicationICCV 2023
PublisherIEEE
Pages1966-1976
Number of pages22
ISBN (Print)979-8-3503-0718-4
DOIs
Publication statusPublished - 17 Mar 2023
Externally publishedYes
Event2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023 - Paris, France
Duration: 1 Oct 20236 Oct 2023

Publication series

NameProceedings
PublisherIEEE
ISSN (Print)1550-5499
ISSN (Electronic)2380-7504

Conference

Conference2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023
Abbreviated titleICCV
Country/TerritoryFrance
CityParis
Period1/10/236/10/23

Keywords

  • cs.CV

Fingerprint

Dive into the research topics of 'Leaping Into Memories Space-Time Deep Feature Synthesis'. Together they form a unique fingerprint.

Cite this