Find Us On:

Linked In


Join Our Mailing List


We won't spam you or ever share your information.

ADSC researchers win international action recognition competition


Advanced Digital Sciences Center researchers Bingbing Ni and pei [dot] yongatadsc [dot] com [dot] sg (Yong Pei) won the 2012 Human Activities Recognition and Localization (HARL) competition, held in conjunction with International Conference on Pattern Recognition (ICPR), the leading conference on pattern recognition. Ni and Pei will present their winning method at ICPR in Tsukuba, Japan next month.

The HARL competition invited participants to submit algorithms to automatically detect, classify and localize activities taking place in Kinect video. This emerging video modality outputs both color and depth image sequences, the latter showing the distance of each point in a scene to the camera. HARL is the world’s first competition that focuses on recognizing and localizing complex human activities in footage from a Kinect camera.

ADSC Researchers Bingbing Ni, left, and Yong Pei, right, won the 2012 Human Activities Recognition and Localization competition in September. They will present their winning method during the November 2012 International Conference on Pattern Recognition (ICPR).

HARL participants were given benchmark datasets of video clips showing complex activities taking place in a typical office setting. Participants created algorithms that could analyze a long video clip and precisely determine when and where the activity took place in the video.

“Previous activity recognition research focused primarily on classification, which means determining what the human is doing,” Ni said. “But in HARL’s more realistic and challenging setting, we had to determine when, where and what the humans were doing. Knowing when and where is more important and difficult than knowing what.”

According to Ni, by winning this award, the team has demonstrated that their algorithm and method for detecting activities is state-of-the-art and is the best option for recognizing actions in Kinect footage.

“Our team’s method is based on fusing the depth image with the grayscale image in every step of the processing pipeline, including human key pose extraction, object detection, interaction contextual modeling and a Bayesian network based action detector,” Ni said.

Harl Discussion photo
Harl Discussion photo
Ni and Pei developed algorithms to detect, classify and localize activities in Kinect video. They used their algorithms to identify actions such as a discussion between two people, see left, handshaking or talking on the phone.

Of the teams submitting results, Ni and Pei ranked first in both classification and localization of activities. Visit HARL for more information about the competition results.

Ni and Pei work with University of Illinois at Urbana-Champaign Electrical and Computer Engineering Professor Pierre Moulin on the Multi-modal Visual Analytics research project at ADSC. The project aims to develop methods for detection, recognition and localization of events and actions in depth video. This research has the potential to significantly impact video-based rehabilitation and sports training, video surveillance and many other areas.

TheAdvanced Digital Sciences Centeris aUniversity of Illinois at Urbana-Champaignresearch center in Singapore. It is led by Electrical and Computer Engineering and Computer Science faculty at the University of Illinois. ADSC focuses on breakthrough innovations in information technology.