Structural-RNN: Deep learning on spatio-temporal graphs

963Citations
Citations of this article
1.3kReaders
Mendeley users who have this article in their library.
Get full text

Abstract

Deep Recurrent Neural Network architectures, though remarkably capable at modeling sequences, lack an intuitive high-level spatio-temporal structure. That is while many problems in computer vision inherently have an underlying high-level structure and can benefit from it. Spatiotemporal graphs are a popular tool for imposing such high-level intuitions in the formulation of real world problems. In this paper, we propose an approach for combining the power of high-level spatio-temporal graphs and sequence learning success of Recurrent Neural Networks (RNNs). We develop a scalable method for casting an arbitrary spatio-temporal graph as a rich RNN mixture that is feedforward, fully differentiable, and jointly trainable. The proposed method is generic and principled as it can be used for transforming any spatio-temporal graph through employing a certain set of well defined steps. The evaluations of the proposed approach on a diverse set of problems, ranging from modeling human motion to object interactions, shows improvement over the state-of-the-art with a large margin. We expect this method to empower new approaches to problem formulation through high-level spatio-temporal graphs and Recurrent Neural Networks.

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Cite

CITATION STYLE

APA

Jain, A., Zamir, A. R., Savarese, S., & Saxena, A. (2016). Structural-RNN: Deep learning on spatio-temporal graphs. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Vol. 2016-December, pp. 5308–5317). IEEE Computer Society. https://doi.org/10.1109/CVPR.2016.573

Readers' Seniority

Tooltip

PhD / Post grad / Masters / Doc 679

77%

Researcher 139

16%

Professor / Associate Prof. 49

6%

Lecturer / Post doc 19

2%

Readers' Discipline

Tooltip

Computer Science 682

75%

Engineering 193

21%

Mathematics 18

2%

Physics and Astronomy 16

2%

Save time finding and organizing research with Mendeley

Sign up for free