2018

Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmark

Liang, Xiaodan, Gong, Ke, Shen, Xiaohui et al.

Understand

Human parsing and pose estimation have recently received considerable interest due to their substantial application potentials.

However, the existing datasets have limited numbers of images and annotations and lack a variety of human appearances and coverage of challenging cases in unconstrained environments.
In this paper, we introduce a new benchmark named "Look into Person (LIP)" that provides a significant advancement in terms of scalability, diversity, and difficulty, which are crucial for future developments in human-centric analysis.
This comprehensive dataset contains over 50,000 elaborately annotated images with 19 semantic part labels and 16 body joints, which are captured from a broad range of viewpoints, occlusions, and background complexities.

Built on

Pictorial structures revisited: People detection and articulated pose estimation
M. Andriluka, S. Roth, and B. Schiele · 2009
Earlier work this paper cites.
The pascal visual object classes challenge 2010 (voc2010) results, 2010
M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman · 2010
Earlier work this paper cites.
Clustered pose and nonlinear appearance models for human pose estimation
S. Johnson and M. Everingham · 2010
Earlier work this paper cites.
Learning effective human pose estimation from inaccurate annotation
S. Johnson and M. Everingham · 2011
Earlier work this paper cites.
Articulated pose estimation with flexible mixtures-of-parts
Y. Yang and D. Ramanan · 2011
Earlier work this paper cites.
Parsing clothing in fashion photographs
K. Yamaguchi, M. Kiapour, L. Ortiz, and T. Berg · 2012
Earlier work this paper cites.
Human pose estimation using body parts dependent joint regressors
M. Dantone, J. Gall, C. Leistner, and L. Van Gool · 2013
Earlier work this paper cites.
A deformable mixture parsing model with parselets
J. Dong, Q. Chen, W. Xia, Z. Huang, and S. Yan · 2013
Earlier work this paper cites.
Towards understanding action recognition
H. Jhuang, J. Gall, S. Zuffi, C. Schmid, and M. J. Black · 2013
Earlier work this paper cites.
Getting the look: clothing recognition and segmentation for automatic product suggestions in everyday photos
Y. Kalantidis, L. Kennedy, and L.-J. Li · 2013
Earlier work this paper cites.
Modec: Multimodal decomposable models for human pose estimation
B. Taskar et al · 2013
Earlier work this paper cites.
Paper doll parsing: Retrieving similar styles to parse clothing items
K. Yamaguchi, M. Kiapour, and T. Berg · 2013
Earlier work this paper cites.
Unsupervised salience learning for person re-identification
R. Zhao, W. Ouyang, and X. Wang · 2013
Earlier work this paper cites.
2d human pose estimation: New benchmark and state of the art analysis
M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele · 2014
Earlier work this paper cites.
Detect what you can: Detecting and representing objects using holistic models and body parts
X. Chen, R. Mottaghi, X. Liu, S. Fidler, R. Urtasun, et al · 2014
Earlier work this paper cites.
Articulated pose estimation by a graphical model with image dependent pairwise relations
X. Chen and A. Yuille · 2014
Earlier work this paper cites.
Towards unified human parsing and pose estimation
J. Dong, Q. Chen, X. Shen, J. Yang, and S. Yan · 2014
Earlier work this paper cites.
Microsoft COCO: common objects in context
Original
T. Lin, M. Maire, S. J. Belongie, L. D. Bourdev, R. B. Girshick, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick · 2014
Earlier work this paper cites.

Similar

Parsing semantic parts of cars using graphical models and segment appearance consistency
W. Lu, X. Lian, and A. Yuille · 2014
Cited alongside, same era.
Multi-source deep learning for human pose estimation
W. Ouyang, X. Chu, and X. Wang · 2014
Cited alongside, same era.
A High Performance CRF Model for Clothes Parsing
E. Simo-Serra, S. Fidler, F. Moreno-Noguer, and R. Urtasun · 2014
Cited alongside, same era.
Deeppose: Human pose estimation via deep neural networks
A. Toshev and C. Szegedy · 2014
Cited alongside, same era.
Deformable part model based multiple pedestrian detection for video surveillance in crowded scenes
L. Wang, X. Ji, Q. Deng, and M. Jia · 2014
Cited alongside, same era.
Segnet: A deep convolutional encoder-decoder architecture for image segmentation
V. Badrinarayanan, A. Kendall, and R. Cipolla · 2015
Cited alongside, same era.
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2015
Cited alongside, same era.
Deep human parsing with active template regression
X. Liang, S. Liu, X. Shen, J. Yang, L. Liu, J. Dong, L. Lin, and S. Yan · 2015
Cited alongside, same era.
Towards computational baby learning: A weakly-supervised approach for object detection
X. Liang, S. Liu, Y. Wei, L. Liu, L. Lin, and S. Yan · 2015
Cited alongside, same era.
Human parsing with contextualized convolutional neural network
X. Liang, C. Xu, X. Shen, J. Yang, S. Liu, J. Tang, L. Lin, and S. Yan · 2015
Cited alongside, same era.
Matching-CNN Meets KNN: Quasi-Parametric Human Parsing
S. Liu, X. Liang, L. Liu, X. Shen, J. Yang, C. Xu, L. Lin, X. Cao, and S. Yan · 2015
Cited alongside, same era.
Fully convolutional networks for semantic segmentation
J. Long, E. Shelhamer, and T. Darrell · 2015
Cited alongside, same era.

Then

Human pose estimation via convolutional part heatmap regression
A. Bulat and G. Tzimiropoulos · 2016
Later among the works it cites.
Human pose estimation with iterative error feedback
J. Carreira, P. Agrawal, K. Fragkiadaki, and J. Malik · 2016
Later among the works it cites.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
Original
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2016
Later among the works it cites.
Attention to scale: Scale-aware semantic image segmentation
L.-C. Chen, Y. Yang, J. Wang, W. Xu, and A. L. Yuille · 2016
Later among the works it cites.
Concepts not alone: Exploring pairwise relationships for zero-shot video activity recognition
C. Gan, M. Lin, Y. Yang, G. de Melo, and A. G. Hauptmann · 2016
Later among the works it cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Later among the works it cites.
Semantic object parsing with local-global long short-term memory
X. Liang, X. Shen, D. Xiang, J. Feng, L. Lin, and S. Yan · 2016
Later among the works it cites.
Deepfashion: Powering robust clothes recognition and retrieval with rich annotations
Z. Liu, P. Luo, S. Qiu, X. Wang, and X. Tang · 2016
Later among the works it cites.
Stacked hourglass networks for human pose estimation
A. Newell, K. Yang, and J. Deng · 2016
Later among the works it cites.
Convolutional pose machines
S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh · 2016
Later among the works it cites.
Zoom better to see clearer: Huamn part segmentation with auto zoom net
F. Xia, P. Wang, L.-C. Chen, and A. L. Yuille · 2016
Later among the works it cites.
Pose-guided human parsing by an and/or graph using pose-context features
F. Xia, J. Zhu, P. Wang, and A. Yuille · 2016
Later among the works it cites.
End-to-end learning of deformable mixture of parts and deep convolutional neural networks for human pose estimation
W. Yang, W. Ouyang, H. Li, and X. Wang · 2016
Later among the works it cites.
Multi-context attention for human pose estimation
X. Chu, W. Yang, W. Ouyang, C. Ma, A. L. Yuille, and X. Wang · 2017
Later among the works it cites.
Attribute and-or grammar for joint parsing of human pose, parts and attributes
S. Park, X. Nie, and S.-C. Zhu · 2017
Later among the works it cites.
Proposal-free network for instance-level object segmentation
X. Liang, L. Lin, Y. Wei, X. Shen, J. Yang, and S. Yan · 2018
Closest in time.
Active self-paced learning for cost-effective and progressive face identification
L. Lin, K. Wang, D. Meng, W. Zuo, and L. Zhang · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…