2019

High-Resolution Representations for Labeling Pixels and Regions

Sun, Ke, Zhao, Yang, Jiang, Borui et al.

Understand

High-resolution representation learning plays an essential role in many vision problems, e.g., pose estimation and semantic segmentation.

The high-resolution network (HRNet)~\cite{SunXLW19}, recently developed for human pose estimation, maintains high-resolution representations through the whole process by connecting high-to-low resolution convolutions in \emph{parallel} and produces strong high-resolution representations by repeatedly conducting fusions across parallel convolutions.
In this paper, we conduct a further study on high-resolution representations by introducing a simple yet effective modification and apply it to a wide range of vision tasks.
We augment the high-resolution representation by aggregating the (upsampled) representations from all the parallel convolutions rather than only the representation from the high-resolution convolution as done in~\cite{SunXLW19}.

Built on

Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization
M. Köstinger, P. Wohlhart, P. M. Roth, and H. Bischof · 2011
Earlier work this paper cites.
Face alignment by explicit shape regression
X. Cao, Y. Wei, F. Wen, and J. Sun · 2012
Earlier work this paper cites.
Interactive facial feature localization
V. Le, J. Brandt, Z. Lin, L. D. Bourdev, and T. S. Huang · 2012
Earlier work this paper cites.
Face detection, pose estimation, and landmark localization in the wild
X. Zhu and D. Ramanan · 2012
Earlier work this paper cites.
Constrained local neural fields for robust facial landmark detection in the wild
T. Baltrusaitis, P. Robinson, and L. Morency · 2013
Earlier work this paper cites.
Localizing parts of faces using a consensus of exemplars
P. N. Belhumeur, D. W. Jacobs, D. J. Kriegman, and N. Kumar · 2013
Earlier work this paper cites.
Robust face landmark estimation under occlusion
X. P. Burgos-Artizzu, P. Perona, and P. Dollár · 2013
Earlier work this paper cites.
300 faces in-the-wild challenge: The first facial landmark localization challenge
C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic · 2013
Earlier work this paper cites.
Overfeat: Integrated recognition, localization and detection using convolutional networks
Original
P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun · 2013
Earlier work this paper cites.
Deep convolutional network cascade for facial point detection
Y. Sun, X. Wang, and X. Tang · 2013
Earlier work this paper cites.
Supervised descent method and its applications to face alignment
X. Xiong and F. D. la Torre · 2013
Earlier work this paper cites.
Learn to combine multiple hypotheses for accurate face alignment
J. Yan, Z. Lei, D. Yi, and S. Z. Li · 2013
Earlier work this paper cites.
Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model
X. Yu, J. Huang, S. Zhang, W. Yan, and D. N. Metaxas · 2013
Earlier work this paper cites.
Extensive facial landmark localization with coarse-to-fine convolutional network cascade
E. Zhou, H. Fan, Z. Cao, Y. Jiang, and Q. Yin · 2013
Earlier work this paper cites.
Semantic image segmentation with deep convolutional nets and fully connected crfs
Original
L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2014
Earlier work this paper cites.
Occlusion coherence: Localizing occluded faces with a hierarchical deformable part model
G. Ghiasi and C. C. Fowlkes · 2014
Earlier work this paper cites.
One millisecond face alignment with an ensemble of regression trees
V. Kazemi and J. Sullivan · 2014
Earlier work this paper cites.
The role of context for object detection and semantic segmentation in the wild
R. Mottaghi, X. Chen, X. Liu, N. Cho, S. Lee, S. Fidler, R. Urtasun, and A. L. Yuille · 2014
Earlier work this paper cites.
Face alignment at 3000 FPS via regressing local binary features
S. Ren, X. Cao, Y. Wei, and J. Sun · 2014
Earlier work this paper cites.
Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment
J. Zhang, S. Shan, M. Kan, and X. Chen · 2014
Earlier work this paper cites.
Facial landmark detection by deep multi-task learning
Z. Zhang, P. Luo, C. C. Loy, and X. Tang · 2014
Earlier work this paper cites.
Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation
J. Dai, K. He, and J. Sun · 2015
Earlier work this paper cites.
Random cascaded-regression copse for robust facial landmark detection
Z. Feng, P. Huber, J. Kittler, W. J. Christmas, and X. Wu · 2015
Earlier work this paper cites.
Fully convolutional networks for semantic segmentation
J. Long, E. Shelhamer, and T. Darrell · 2015
Earlier work this paper cites.
Learning deconvolution network for semantic segmentation
H. Noh, S. Hong, and B. Han · 2015
Earlier work this paper cites.
Faster R-CNN: towards real-time object detection with region proposal networks
Original
S. Ren, K. He, R. B. Girshick, and J. Sun · 2015
Earlier work this paper cites.
U-net: Convolutional networks for biomedical image segmentation
O. Ronneberger, P. Fischer, and T. Brox · 2015
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. S. Bernstein, A. C. Berg, and F. Li · 2015
Earlier work this paper cites.
Holistically-nested edge detection
S. Xie and Z. Tu · 2015
Earlier work this paper cites.
Interlinked convolutional neural networks for face parsing
Y. Zhou, X. Hu, and B. Zhang · 2015
Earlier work this paper cites.
Face alignment by coarse-to-fine shape searching
S. Zhu, C. Li, C. C. Loy, and X. Tang · 2015
Earlier work this paper cites.
Higher order conditional random fields in deep neural networks
A. Arnab, S. Jayasumana, S. Zheng, and P. H. S. Torr · 2016
Earlier work this paper cites.
Human pose estimation via convolutional part heatmap regression
A. Bulat and G. Tzimiropoulos · 2016
Earlier work this paper cites.
Attention to scale: Scale-aware semantic image segmentation
L. Chen, Y. Yang, J. Wang, W. Xu, and A. L. Yuille · 2016
Earlier work this paper cites.
The cityscapes dataset for semantic urban scene understanding
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele · 2016
Earlier work this paper cites.
Approaching human level facial landmark localization by deep learning
H. Fan and E. Zhou · 2016
Earlier work this paper cites.
Laplacian pyramid reconstruction and refinement for semantic segmentation
G. Ghiasi and C. C. Fowlkes · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
Recombinator networks: Learning coarse-to-fine feature aggregation
S. Honari, J. Yosinski, P. Vincent, and C. J. Pal · 2016
Earlier work this paper cites.
Efficient piecewise training of deep structured models for semantic segmentation
G. Lin, C. Shen, A. van den Hengel, and I. D. Reid · 2016
Earlier work this paper cites.
Stacked hourglass networks for human pose estimation
A. Newell, K. Yang, and J. Deng · 2016
Earlier work this paper cites.
A recurrent encoder-decoder network for sequential face alignment
X. Peng, R. S. Feris, X. Wang, and D. N. Metaxas · 2016
Earlier work this paper cites.
Face alignment via regressing local binary features
S. Ren, X. Cao, Y. Wei, and J. Sun · 2016
Earlier work this paper cites.
Convolutional neural fabrics
S. Saxena and J. Verbeek · 2016
Earlier work this paper cites.
Mnemonic descent method: A recurrent process applied for end-to-end face alignment
G. Trigeorgis, P. Snape, M. A. Nicolaou, E. Antonakos, and S. Zafeiriou · 2016
Earlier work this paper cites.
Wider or deeper: Revisiting the resnet model for visual recognition
Original
Z. Wu, C. Shen, and A. van den Hengel · 2016
Earlier work this paper cites.

Similar

Robust facial landmark detection via recurrent attentive-refinement networks
S. Xiao, J. Feng, J. Xing, H. Lai, S. Yan, and A. A. Kassim · 2016
Cited alongside, same era.
WIDER FACE: A face detection benchmark
S. Yang, P. Luo, C. C. Loy, and X. Tang · 2016
Cited alongside, same era.
Occlusion-free face alignment: Deep regression networks coupled with de-corrupt autoencoders
J. Zhang, M. Kan, S. Shan, and X. Chen · 2016
Cited alongside, same era.
Unconstrained face alignment via cascaded compositional learning
S. Zhu, C. Li, C. C. Loy, and X. Tang · 2016
Cited alongside, same era.
Segnet: A deep convolutional encoder-decoder architecture for image segmentation
V. Badrinarayanan, A. Kendall, and R. Cipolla · 2017
Cited alongside, same era.
Binarized convolutional landmark localizers for human pose estimation and face alignment with limited resources
A. Bulat and G. Tzimiropoulos · 2017
Cited alongside, same era.
How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230, 000 3d facial landmarks)
A. Bulat and G. Tzimiropoulos · 2017
Cited alongside, same era.
Rethinking atrous convolution for semantic image segmentation
Original
L. Chen, G. Papandreou, F. Schroff, and H. Adam · 2017
Cited alongside, same era.
Adversarial posenet: A structure-aware convolutional network for human pose estimation
Y. Chen, C. Shen, X. Wei, L. Liu, and J. Yang · 2017
Cited alongside, same era.
Joint multi-view face alignment in the wild
Original
J. Deng, G. Trigeorgis, Y. Zhou, and S. Zafeiriou · 2017
Cited alongside, same era.
Dynamic attention-controlled cascaded shape regression exploiting training data augmentation and fuzzy-set sample weighting
Z. Feng, J. Kittler, W. J. Christmas, P. Huber, and X. Wu · 2017
Cited alongside, same era.
Residual conv-deconv grid network for semantic segmentation
D. Fourure, R. Emonet, É. Fromont, D. Muselet, A. Trémeau, and C. Wolf · 2017
Cited alongside, same era.

Then

Wing loss for robust facial landmark localisation with convolutional neural networks
Z. Feng, J. Kittler, M. Awais, P. Huber, and X. Wu · 2018
Later among the works it cites.
Stacked dense u-nets with dual transformers for robust face alignment
J. Guo, J. Deng, N. Xue, and S. Zafeiriou · 2018
Later among the works it cites.
Rethinking imagenet pre-training
Original
K. He, R. B. Girshick, and P. Dollár · 2018
Later among the works it cites.
Relation networks for object detection
H. Hu, J. Gu, Z. Zhang, J. Dai, and Y. Wei · 2018
Later among the works it cites.
Adaptive affinity fields for semantic segmentation
T. Ke, J. Hwang, Z. Liu, and S. X. Yu · 2018
Later among the works it cites.
Parallel feature pyramid network for object detection
S. Kim, H. Kook, J. Sun, M. Kang, and S. Ko · 2018
Later among the works it cites.
Recurrent scene parsing with perspective understanding in the loop
S. Kong and C. C. Fowlkes · 2018
Later among the works it cites.
Deep feature pyramid reconfiguration for object detection
T. Kong, F. Sun, W. Huang, and H. Liu · 2018
Later among the works it cites.
Disentangling 3d pose in a dendritic CNN for unconstrained 2d face alignment
A. Kumar and R. Chellappa · 2018
Later among the works it cites.
Cornernet: Detecting objects as paired keypoints
H. Law and J. Deng · 2018
Later among the works it cites.
Pyramid attention network for semantic segmentation
H. Li, P. Xiong, J. An, and L. Wang · 2018
Later among the works it cites.
R-FCN++: towards accurate region-based fully convolutional networks for object detection
Z. Li, Y. Chen, G. Yu, and Y. Deng · 2018
Later among the works it cites.
Detnet: Design backbone for object detection
Z. Li, C. Peng, G. Yu, X. Zhang, Y. Deng, and J. Sun · 2018
Later among the works it cites.
Look into person: Joint body parsing & pose estimation network and A new benchmark
Original
X. Liang, K. Gong, X. Shen, and L. Lin · 2018
Later among the works it cites.
Dynamic-structured semantic propagation network
X. Liang, H. Zhou, and E. Xing · 2018
Later among the works it cites.
Receptive field block net for accurate and fast object detection
S. Liu, D. Huang, and Y. Wang · 2018
Later among the works it cites.
Path aggregation network for instance segmentation
S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia · 2018
Later among the works it cites.
Devil in the details: Towards accurate single and multiple human parsing
Original
T. Liu, T. Ruan, Z. Huang, Y. Wei, S. Wei, Y. Zhao, and T. Huang · 2018
Later among the works it cites.
Macro-micro adversarial network for human parsing
Y. Luo, Z. Zheng, L. Zheng, T. Guan, J. Yu, and Y. Yang · 2018
Later among the works it cites.
Direct shape regression networks for end-to-end face alignment
X. Miao, X. Zhen, X. Liu, C. Deng, V. Athitsos, and H. Huang · 2018
Later among the works it cites.
Mutual learning to adapt for joint human parsing and pose estimation
X. Nie, J. Feng, and S. Yan · 2018
Later among the works it cites.
Megdet: A large mini-batch object detector
C. Peng, T. Xiao, Z. Li, Y. Jiang, X. Zhang, K. Jia, G. Yu, and J. Sun · 2018
Later among the works it cites.
Sequential context encoding for duplicate removal
L. Qi, S. Liu, J. Shi, and J. Jia · 2018
Later among the works it cites.
An analysis of scale invariance in object detection SNIP
B. Singh and L. S. Davis · 2018
Later among the works it cites.
SNIPER: efficient multi-scale training
B. Singh, M. Najibi, and L. S. Davis · 2018
Later among the works it cites.
Quantized densely connected u-nets for efficient landmark localization
Z. Tang, X. Peng, S. Geng, L. Wu, S. Zhang, and D. N. Metaxas · 2018
Later among the works it cites.
Improving object localization with fitness NMS and bounded iou loss
L. Tychsen-Smith and L. Petersson · 2018
Later among the works it cites.
A deeply-initialized coarse-to-fine ensemble of regression trees for face alignment
R. Valle, J. M. Buenaposada, A. Valdés, and L. Baumela · 2018
Later among the works it cites.
Multi-scale location-aware kernel representation for object detection
H. Wang, Q. Wang, M. Gao, P. Li, and W. Zuo · 2018
Later among the works it cites.
Understanding convolution for semantic segmentation
P. Wang, P. Chen, Y. Yuan, D. Liu, Z. Huang, X. Hou, and G. W. Cottrell · 2018
Later among the works it cites.
Look at boundary: A boundary-aware face alignment algorithm
W. Wu, C. Qian, S. Yang, Q. Wang, Y. Cai, and Q. Zhou · 2018
Later among the works it cites.
Unified perceptual parsing for scene understanding
T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun · 2018
Later among the works it cites.
Pad-net: Multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing
D. Xu, W. Ouyang, X. Wang, and N. Sebe · 2018
Later among the works it cites.
Deep regionlets for object detection
H. Xu, X. Lv, X. Wang, Z. Ren, N. Bodla, and R. Chellappa · 2018
Later among the works it cites.
Bisenet: Bilateral segmentation network for real-time semantic segmentation
C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang · 2018
Later among the works it cites.
Learning a discriminative feature network for semantic segmentation
C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang · 2018
Later among the works it cites.
Context encoding for semantic segmentation
H. Zhang, K. J. Dana, J. Shi, Z. Zhang, X. Wang, A. Tyagi, and A. Agrawal · 2018
Later among the works it cites.
Single-shot refinement neural network for object detection
S. Zhang, L. Wen, X. Bian, Z. Lei, and S. Z. Li · 2018
Later among the works it cites.
Single-shot object detection with enriched semantics
Z. Zhang, S. Qiao, C. Xie, W. Shen, B. Wang, and A. L. Yuille · 2018
Later among the works it cites.
Exfuse: Enhancing feature fusion for semantic segmentation
Z. Zhang, X. Zhang, C. Peng, X. Xue, and J. Sun · 2018
Later among the works it cites.
Psanet: Point-wise spatial attention network for scene parsing
H. Zhao, Y. Zhang, S. Liu, J. Shi, C. C. Loy, D. Lin, and J. Jia · 2018
Later among the works it cites.
M2det: A single-shot object detector based on multi-level feature pyramid network
Original
Q. Zhao, T. Sheng, Y. Wang, Z. Tang, Y. Chen, L. Cai, and H. Ling · 2018
Later among the works it cites.
Scale-transferrable object detection
P. Zhou, B. Ni, C. Geng, J. Hu, and Y. Xu · 2018
Later among the works it cites.
Unet++: A nested u-net architecture for medical image segmentation
Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang · 2018
Later among the works it cites.
Deep high-resolution representation learning for human pose estimation
K. Sun, B. Xiao, D. Liu, and J. Wang · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…