2020

GANHopper: Multi-Hop GAN for Unsupervised Image-to-Image Translation

Lira, Wallace, Merz, Johannes, Ritchie, Daniel et al.

Understand

We introduce GANHopper, an unsupervised image-to-image translation network that transforms images gradually between two domains, through multiple hops.

Instead of executing translation directly, we steer the translation by requiring the network to produce in-between images that resemble weighted hybrids between images from the input domains.
Our network is trained on unpaired images from the two domains only, without any in-between images.
All hops are produced using a single generator along each direction.

Built on

Liu, J., Kanazawa, A., Jacobs, D., Belhumeur, P.: Dog breed classification using part localization. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) Computer Vision – ECCV 2012. pp. 172–185. Springer Berlin Heidelberg, Berlin, Heidelberg (2012)
2012
Earlier work this paper cites.
Liao, J., Lima, R.S., Nehab, D., Hoppe, H., Sander, P.V., Yu, J.: Automating image morphing using structural similarity on a halfway domain. ACM Trans. Graph. 33
2014
Earlier work this paper cites.
Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. CoRR abs/1411.7766
Original
2014
Earlier work this paper cites.
Ni, K., Pearce, R.A., Boakye, K., Essen, B.V., Borth, D., Chen, B., Wang, E.X.: Large-scale deep learning on the YFCC100M dataset. CoRR abs/1502.03409
Original
2015
Earlier work this paper cites.
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: MICCAI (2015)
2015
Earlier work this paper cites.
Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., Abbeel, P.: Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In: Proceedings of the 30th International Conference on Neural Information Processing Systems (2016)
2016
Earlier work this paper cites.
Isola, P., Zhu, J., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. CoRR abs/1611.07004
Original
2016
Earlier work this paper cites.
Johnson, J., Alahi, A., Li, F.: Perceptual losses for real-time style transfer and super-resolution. CoRR abs/1603.08155
Original
2016
Earlier work this paper cites.
Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39
2016
Earlier work this paper cites.
Chen, L., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. CoRR abs/1706.05587
Original
2017
Earlier work this paper cites.

Similar

Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30, pp. 6626–6637. Curran Associates, Inc. (2017), http://papers.nips.cc/paper/7240-gans-trained-by-a-two-time-scale-update-rule-converge-to-a-local-nash-equilibrium.pdf
2017
Cited alongside, same era.
Kim, T., Cha, M., Kim, H., Lee, J.K., Kim, J.: Learning to discover cross-domain relations with generative adversarial networks. In: ICML (2017)
2017
Cited alongside, same era.
Lample, G., Zeghidour, N., Usunier, N., Bordes, A., DENOYER, L., et al.: Fader networks: Manipulating images by sliding attributes. In: Advances in Neural Information Processing Systems (2017)
2017
Cited alongside, same era.
Liu, M., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. CoRR abs/1703.00848
Original
2017
Cited alongside, same era.
Mao, X., Li, Q., Xie, H., Lau, R.Y.K., Wang, Z.: Least squares generative adversarial networks. In: ICCV (2017)
2017
Cited alongside, same era.
Taigman, Y., Polyak, A., Wolf, L.: Unsupervised cross-domain image generation. In: Proc. of ICLR (2017)
2017
Cited alongside, same era.
Yi, Z., Zhang, H., Tan, P., Gong, M.: DualGAN: Unsupervised dual learning for image-to-image translation. In: Proc. of ICCV (2017)
2017
Cited alongside, same era.
Zhu, J., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: International Conference on Computer Vision (ICCV), to appear (2017)
2017
Cited alongside, same era.
Zhu, J.Y., Zhang, R., Pathak, D., Darrell, T., Efros, A.A., Wang, O., Shechtman, E.: Toward multimodal image-to-image translation. In: Advances in Neural Information Processing Systems (2017)
2017
Cited alongside, same era.
Aberman, K., Liao, J., Shi, M., Lischinski, D., Chen, B., Cohen-Or, D.: Neural best-buddies: Sparse cross-domain correspondence. ACM Trans. Graph. 37
2018
Cited alongside, same era.

Then

Cao, K., Liao, J., Yuan, L.: Carigans: Unpaired photo-to-caricature translation (2018)
2018
Later among the works it cites.
Gokaslan, A., Ramanujan, V., Ritchie, D., Kim, K.I., Tompkin, J.: Improving shape deformation in unsupervised image-to-image translation. CoRR abs/1808.04325
Original
2018
Later among the works it cites.
Huang, X., Liu, M.Y., Belongie, S., Kautz, J.: Multimodal unsupervised image-to-image translation. In: ECCV (2018)
2018
Later among the works it cites.
Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)
2018
Later among the works it cites.
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 586–595 (June 2018). https://doi.org/10.1109/CVPR.2018.00068
2018
Later among the works it cites.
Mo, S., Cho, M., Shin, J.: Instagan: Instance-aware image-to-image translation. In: International Conference on Learning Representations (2019), https://openreview.net/forum?id=ryxwJhC9YX
2019
Later among the works it cites.
Wu, W., Cao, K., Li, C., Qian, C., Loy, C.C.: Transgaga: Geometry-aware unsupervised image-to-image translation. In: Proc. of CVPR (2019)
2019
Later among the works it cites.
Yin, K., Chen, Z., Huang, H., Cohen-Or, D., Zhang, H.: LOGAN: Unpaired shape transform in latent overcomplete space. ACM Trans. on Graphics 38
2019
Later among the works it cites.
Katzir, O., Lischinski, D., Cohen-Or, D.: Cross-domain cascaded deep feature translation. In: European Conference on Computer Vision (ECCV). Springer (2020)
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…