The Wisdom of Crowds: Temporal Progressive Attention for Early Action Prediction

Alexandros Stergiou; Dima Damen

doi:10.1109/CVPR52729.2023.01413

The Wisdom of Crowds: Temporal Progressive Attention for Early Action Prediction

Alexandros Stergiou, Dima Damen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

8 Citations (Scopus)

Abstract

Early action prediction deals with inferring the ongoing action from partially-observed videos, typically at the outset of the video. We propose a bottleneck-based attention model that captures the evolution of the action, through progressive sampling over fine-to-coarse scales. Our proposed Temporal Progressive (TemPr) model is composed of multiple attention towers, one for each scale. The predicted action label is based on the collective agreement considering confidences of these towers. Extensive experiments over four video datasets showcase state-of-the-art performance on the task of Early Action Prediction across a range of encoder architectures. We demonstrate the effectiveness and consistency of TemPr through detailed ablations. † † Code is available at: https://tinyurl.com/temprog

Original language	English
Title of host publication	2023 IEEE/CVF Conference On Computer Vision And Pattern Recognition
Subtitle of host publication	CVPR 2023
Publisher	IEEE
Pages	14709-14719
Number of pages	11
ISBN (Print)	979-8-3503-0129-8
DOIs	https://doi.org/10.1109/CVPR52729.2023.01413
Publication status	Published - 2023
Externally published	Yes
Event	IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 - Vancouver, Canada Duration: 18 Jun 2023 → 22 Jun 2023 https://cvpr2023.thecvf.com/

Publication series

Name	Proceedings
Publisher	IEEE
ISSN (Print)	2575-7075

Conference

Conference	IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023
Abbreviated title	CVPR 2023
Country/Territory	Canada
City	Vancouver
Period	18/06/23 → 22/06/23
Internet address	https://cvpr2023.thecvf.com/

Keywords

n/a OA procedure

Access to Document

10.1109/CVPR52729.2023.01413

Cite this

@inproceedings{2cacef5230eb4b0595ab7811b2a6cfaf,

title = "The Wisdom of Crowds: Temporal Progressive Attention for Early Action Prediction",

abstract = "Early action prediction deals with inferring the ongoing action from partially-observed videos, typically at the outset of the video. We propose a bottleneck-based attention model that captures the evolution of the action, through progressive sampling over fine-to-coarse scales. Our proposed Temporal Progressive (TemPr) model is composed of multiple attention towers, one for each scale. The predicted action label is based on the collective agreement considering confidences of these towers. Extensive experiments over four video datasets showcase state-of-the-art performance on the task of Early Action Prediction across a range of encoder architectures. We demonstrate the effectiveness and consistency of TemPr through detailed ablations. † † Code is available at: https://tinyurl.com/temprog",

keywords = "n/a OA procedure",

author = "Alexandros Stergiou and Dima Damen",

year = "2023",

doi = "10.1109/CVPR52729.2023.01413",

language = "English",

isbn = "979-8-3503-0129-8",

series = "Proceedings",

publisher = "IEEE",

pages = "14709--14719",

booktitle = "2023 IEEE/CVF Conference On Computer Vision And Pattern Recognition",

address = "United States",

note = "IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, CVPR 2023 ; Conference date: 18-06-2023 Through 22-06-2023",

url = "https://cvpr2023.thecvf.com/",

}

Stergiou, A & Damen, D 2023, The Wisdom of Crowds: Temporal Progressive Attention for Early Action Prediction. in 2023 IEEE/CVF Conference On Computer Vision And Pattern Recognition: CVPR 2023. Proceedings, IEEE, pp. 14709-14719, IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, British Columbia, Canada, 18/06/23. https://doi.org/10.1109/CVPR52729.2023.01413

TY - GEN

T1 - The Wisdom of Crowds

T2 - IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023

AU - Stergiou, Alexandros

AU - Damen, Dima

PY - 2023

Y1 - 2023

N2 - Early action prediction deals with inferring the ongoing action from partially-observed videos, typically at the outset of the video. We propose a bottleneck-based attention model that captures the evolution of the action, through progressive sampling over fine-to-coarse scales. Our proposed Temporal Progressive (TemPr) model is composed of multiple attention towers, one for each scale. The predicted action label is based on the collective agreement considering confidences of these towers. Extensive experiments over four video datasets showcase state-of-the-art performance on the task of Early Action Prediction across a range of encoder architectures. We demonstrate the effectiveness and consistency of TemPr through detailed ablations. † † Code is available at: https://tinyurl.com/temprog

AB - Early action prediction deals with inferring the ongoing action from partially-observed videos, typically at the outset of the video. We propose a bottleneck-based attention model that captures the evolution of the action, through progressive sampling over fine-to-coarse scales. Our proposed Temporal Progressive (TemPr) model is composed of multiple attention towers, one for each scale. The predicted action label is based on the collective agreement considering confidences of these towers. Extensive experiments over four video datasets showcase state-of-the-art performance on the task of Early Action Prediction across a range of encoder architectures. We demonstrate the effectiveness and consistency of TemPr through detailed ablations. † † Code is available at: https://tinyurl.com/temprog

KW - n/a OA procedure

UR - https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=utwente-ris&SrcAuth=WosAPI&KeyUT=WOS:001062522107004&DestLinkType=FullRecord&DestApp=WOS

U2 - 10.1109/CVPR52729.2023.01413

DO - 10.1109/CVPR52729.2023.01413

M3 - Conference contribution

SN - 979-8-3503-0129-8

T3 - Proceedings

SP - 14709

EP - 14719

BT - 2023 IEEE/CVF Conference On Computer Vision And Pattern Recognition

PB - IEEE

Y2 - 18 June 2023 through 22 June 2023

ER -

The Wisdom of Crowds: Temporal Progressive Attention for Early Action Prediction

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this