Fetching the paper…

Reading the bibliography…

StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesis · Around

2021

StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesis

Schaldenbrand, Peter, Liu, Zhixuan, Oh, Jean

Original
PDF
Post on X

Understand

Generating images that fit a given text description using machine learning has improved greatly with the release of technologies such as the CLIP image-text encoder model; however, current methods lack artistic control of the style of image to be generated.

We introduce StyleCLIPDraw which adds a style loss to the CLIPDraw text-to-drawing synthesis model to allow artistic control of the synthesized drawings in addition to control of the content via text.
Whereas performing decoupled style transfer on a generated image only affects the texture, our proposed coupled approach is able to capture a style in both texture and shape, suggesting that the style of the drawing is coupled with the drawing process itself.
More results and our code are available at https://github.com/pschaldenbrand/StyleCLIPDraw

Built on

Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2015
Earlier work this paper cites.
Style transfer by relaxed optimal transport and self-similarity
Nicholas Kolkin, Jason Salavon, and Gregory Shakhnarovich · 2019
Earlier work this paper cites.

Similar

Differentiable vector graphics rasterization for editing and learning
Tzu-Mao Li, Michal Lukác, Michaël Gharbi, and Jonathan Ragan-Kelley · 2020
Cited alongside, same era.
Clipdraw: Exploring text-to-drawing synthesis through language-image encoders
Original
Kevin Frans, LB Soros, and Olaf Witkowski · 2021
Cited alongside, same era.

Then

Learning transferable visual models from natural language supervision
Original
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever · 2021
Closest in time.
Zero-shot text-to-image generation
Original
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…