DEDGraph: Delay Embedding of Dynamic Graph for Temporal Action Segmentation

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In real-world interactive applications, where videos are generated in real-time and require immediate feedback, online segmentation has practical advantages over offline inference. Many excellent previous models have been developed for offline scenarios, while real-time prediction for temporal action segmentation (TAS) is a difficult task. Some interactive applications can tolerate a certain amount of delay. In this paper, we propose a node delay embedding of a dynamic graph for real-time TAS. We transform the video stream into a dynamic graph stream that evolves over time. We define past, current, and future nodes to construct sub-graphs at each step. Specifically, future nodes are sampled using our proposed node delay method. A graph model is utilized to aggregate past, current, and future node information to update the representation of current nodes and predict their labels. To the best of our knowledge, it is the first real-time TAS graph model with delay embedding. Experiments show that delay embedding enhances node representation and improves performance. Overall, our proposed approach provides a promising solution for real-time TAS.

Original languageEnglish
Title of host publicationProceedings - 2023 6th International Symposium on Computer, Consumer and Control, IS3C 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages155-158
Number of pages4
ISBN (Electronic)9798350301953
DOIs
Publication statusPublished - 2023
Event6th International Symposium on Computer, Consumer and Control, IS3C 2023 - Taichung City, Taiwan
Duration: 2023 Jun 302023 Jul 3

Publication series

NameProceedings - 2023 6th International Symposium on Computer, Consumer and Control, IS3C 2023

Conference

Conference6th International Symposium on Computer, Consumer and Control, IS3C 2023
Country/TerritoryTaiwan
CityTaichung City
Period23-06-3023-07-03

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Human-Computer Interaction
  • Control and Optimization

Fingerprint

Dive into the research topics of 'DEDGraph: Delay Embedding of Dynamic Graph for Temporal Action Segmentation'. Together they form a unique fingerprint.

Cite this