Subspace Clustering for Action Recognition with Covariance Representations and Temporal Pruning

21 Jun 2020  ·  Giancarlo Paoletti, Jacopo Cavazza, Cigdem Beyan, Alessio Del Bue ·

This paper tackles the problem of human action recognition, defined as classifying which action is displayed in a trimmed sequence, from skeletal data. Albeit state-of-the-art approaches designed for this application are all supervised, in this paper we pursue a more challenging direction: Solving the problem with unsupervised learning. To this end, we propose a novel subspace clustering method, which exploits covariance matrix to enhance the action's discriminability and a timestamp pruning approach that allow us to better handle the temporal dimension of the data. Through a broad experimental validation, we show that our computational pipeline surpasses existing unsupervised approaches but also can result in favorable performances as compared to supervised methods.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Skeleton Based Action Recognition Florence 3D Temporal Spectral Clustering + Temporal Subspace Clustering Accuracy 95.81% # 4
Skeleton Based Action Recognition Gaming 3D (G3D) Temporal K-Means Clustering + Temporal Covariance Subspace Clustering Accuracy 92.91% # 2
Skeleton Based Action Recognition HDM05 Temporal Subspace Clustering Accuracy 89.80% # 1
Skeleton Based Action Recognition MSR Action3D Temporal K-Means Clustering + Temporal Subspace Clustering Accuracy 88.51% # 2
Skeleton Based Action Recognition MSR ActionPairs Temporal Subspace Clustering Accuracy 98.02% # 1
Skeleton Based Action Recognition MSRC-12 Temporal Subspace Clustering Accuracy 99.08% # 1
Skeleton Based Action Recognition UT-Kinect Temporal Subspace Clustering Accuracy 99.50% # 1

Methods