Viewpoint invariant collective activity recognition with relative action context

Abstract

This paper presents an approach for collective activity recognition. Collective activities are activities performed by multiple persons, such as queueing in a line and talking together. To recognize them, the action context (AC) descriptor [1] encodes the "apparent" relation (e.g. a group crossing and facing "right"), however this representation is sensitive to viewpoint change. We instead propose a novel feature representation called the relative action context (RAC) descriptor that encodes the "relative" relation (e.g. a group crossing and facing the "same" direction). This representation is viewpoint invariant and complementary to AC; hence we employ a simplified combinational classifier. This paper also introduces two methods to accelerate performance. First, to make the contexts robust to various situations, we apply post processes. Second, to reduce local classification failures, we regularize the classification using fully connected CRFs. Experimental results show that our method is applicable to various scenes and outperforms state-of-the art methods. © 2012 Springer-Verlag.