Title: 3D-ACT – 3D Action Recognition Using Refinement and Invariance Strategies for Reliable Surveillance
Funding source: FNR CORE, Luxembourg
Partner: POST Telecom, Dublin City University
Principal investigator: Prof. Björn Ottersten
VPI: Dr. Djamila Aouada
Researchers: Dr. Enjie Ghorbel, Dr. Kassem Al Ismaeil, Konstantinos Papadopoulos, Renato Baptista
Starting date/ Duration: 01/06/2016 – 36 months

logo

Visual surveillance systems are more than ever required to increase their performance in order to better detect, and eventually prevent, criminal as well as terrorist attacks. Indeed, in the aftermath of the Charlie Hebdo and the kosher grocery store terrorist attacks in Paris, it is clear that today no one is immune to such threats. At the local level, a 2012 Eurostat study showed a scary increase in criminality, ranking the City of Luxembourg as “the third most dangerous capital in Europe”. Automatically detecting abnormal or undesirable behaviours would certainly provide a decisive support in case of contingency, and could be a good deterrent. Action recognition systems have been extensively researched by the computer vision community, and interesting results have been achieved using images or videos captured with conventional 2D cameras. However, there is still no system that can robustly and effectively perform under real-world conditions, where there is a constant change in illumination, texture, occlusions and viewpoint. Our goal, in the proposed effort, is to lift these four limitations. By using RGB-D cameras, from which 3D information can be coupled with colour information, sensitivity to illuminations and textures can be largely reduced. Our research problem may then be casted as a 3D action recognition problem. In order to tackle sensor specific properties, and define a system that is invariant to occlusions and viewpoint variations, we will investigate three distinct and complementary research axes:1)3D trajectories for action recognition from RGB-D data,2)Selection and refinement strategies for improved trajectory-based action recognition,3)Selection and refinement strategies for improved skeleton-based action recognition.We will develop theoretical models for each axis, and target a final unified framework to be integrated in one 3D action recognition system. While the main objective of 3D-ACT is to automatically detect abnormal and suspicious behaviour of humans from surveillance videos, it is also necessary to maintain the balance between security in critical infrastructures and privacy of individuals at the same time. We will go after such a balance by directly incorporating privacy in the design of our systems, in line with our work on privacy-preserving pattern recognition.

Publications

Temporal 3D Human Pose Estimation for Action Recognition from Arbitrary Viewpoints
Adel Musallam, MohamedBaptista, RenatoAl Ismaeil, KassemAouada, Djamila
in 6th Annual Conf. on Computational Science & Computational Intelligence, Las Vegas 5-7 December 2019 (2019, December)

VIEW-INVARIANT ACTION RECOGNITION FROM RGB DATA VIA 3D POSE ESTIMATION
Baptista, RenatoGhorbel, EnjiePapadopoulos, KonstantinosDemisse, GirumAouada, DjamilaOttersten, Björn
in IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, UK, 12–17 May 2019 (2019, May)

A View-invariant Framework for Fast Skeleton-based Action Recognition Using a Single RGB Camera
Ghorbel, EnjiePapadopoulos, KonstantinosBaptista, RenatoPathak, HimadriDemisse, GirumAouada, DjamilaOttersten, Björn
in 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Prague, 25-27 February 2018 (2019, February)

Localized Trajectories for 2D and 3D Action Recognition
Papadopoulos, KonstantinosDemisse, GirumGhorbel, EnjieAntunes, MichelAouada, DjamilaOttersten, Björn
in Sensors (2019)

Two-stage RGB-based Action Detection using Augmented 3D Poses
Papadopoulos, KonstantinosGhorbel, EnjieBaptista, RenatoAouada, DjamilaOttersten, Björn
in 18th International Conference on Computer Analysis of Images and Patterns SALERNO, 3-5 SEPTEMBER, 2019 (2019)

Pose Encoding for Robust Skeleton-Based Action Recognition
Demisse, GirumPapadopoulos, KonstantinosAouada, DjamilaOttersten, Björn
in CVPRW: Visual Understanding of Humans in Crowd Scene, Salt Lake City, Utah, June 18-22, 2018 (2018, June 18)

A Revisit of Action Detection using Improved Trajectories
Papadopoulos, KonstantinosAntunes, MichelAouada, DjamilaOttersten, Björn
in IEEE International Conference on Acoustics, Speech and Signal Processing, Calgary, Alberta, Canada, 15–20 April 2018 (2018)

Enhanced Trajectory-based Action Recognition using Human Pose
Papadopoulos, KonstantinosGoncalves Almeida Antunes, MichelAouada, DjamilaOttersten, Björn
in IEEE International Conference on Image Processing, Beijing 17-20 Spetember 2017 (2017)