Dual-source deep neural networks for human pose estimation Xiaochuan Fan, Kang Zheng, Yuewei Lin, Song Wang CVPR 2015
Most of the previous works on human pose estimation are based on the
two-layer part-based model. The first layer focuses on local (body) part
appearance and the second layer imposes the contextual relations between
local parts. These pose estimation methods using part-based models are usually
sensitive to noise and the graphical model lacks expressiveness to model
complex human poses. Furthermore, most of these methods search for each
local part independently and the local appearance may not be sufficiently
discriminative for identifying each local part reliably.
In this work, we propose a dual-source CNN (DS-CNN) based method for human pose estimation without using any explicit graphical model. This proposed method utilizes both the local part appearance in image patches (part patch) and the holistic view of the full human body (body patch) for more accurate human pose estimation. By adding a binary mask to the body patch, we novelly integrate global and local information to one input (body patch). By taking both the local patch and the body patch as inputs, the proposed DS-CNN performs a unified learning to achieve both joint detection, which determines whether an object proposal contains a body joint, and joint localization, which finds the exact location of the joint in the object proposal.
About PerformanceIn the paper, we evaluated our work using the PCP metric which considers a part prediction correct if its endpoints are closer to their ground-truth locations than half the length of the limb on average over the two endpoints. Moreover, multiple candidate predictions from models with different input dimensions were also used to match the ground-truth. However, some previous methods were evaluated using single candidate and stricter PCP. In order to compare our work with state-of-the-art methods fairly, we provide predictions and the comparison of strict PCP results and PDJ results using single candidate on the Leeds Sport Pose (LSP) dataset with Person-Centric (PC) annotations. Thank Pishchulin et al. for their evaluation framework.
Strict PCP (our predictions)
|Method||Torso||Upper leg||Lower leg||Upper arm||Forearm||Head||Full body|
|Pishchulin et al., ICCV'13||88.7||63.6||58.4||46.0||35.2||85.1||58.0|
|Tompson et al., NIPS'14||90.3||70.4||61.1||63.0||51.2||83.7||66.6|
PDJ value (normalized distance equal to 0.2) (our predictions)
|Pishchulin et al.||87.2||56.7||46.7||38.0||61.0||57.5||52.7||57.1|
|Tompson et al., NIPS'14||90.6||79.2||67.9||63.4||69.5||71.0||64.2||72.0|