Style2Paints Research* (S2PR) is a non-commercial research group sharing the interest in artistic creation
techniques. The current S2PR group consists of 4 researchers, 4 engineers, and 22 artists. The main products
of S2PR involves tools in line drawing processing, color manipulation, illumination editing, shadow drawing,
etc. S2PR has made presentations in scientific conferences like ACM SIGGRAPH, IEEE CVPR, and
commercial exhibitions like Anime EXPO.
Goals:
- Interactive Research: To study how researchers can learn the needs of artists and how artists can learn the needs of researchers.
- Real Demand: To investigate what artists actually need and what artists do not really need.
- Constructive Feedback: To develop practical tools for artists and investigate the feedbacks of artists to further improve such tools.
- Unconditional Trust: We trust that users can work together with technology to achieve the art they want, in other words, that user commands and instructions are positive and constructive.
- Conditional Methodology: Depending on different problems, both machine learning techniques and non-machine-learning algorithms are acceptable.
- Divide and Conquer:We break down complex and difficult problems into easier sub-problems and solve them one by one, in a divide-and-conquer way.
News
2022.08.15 - Lvmin's article is accepted to SIGGRAPH ASIA 2022, journal track. 2022.06.15 - See some recent announcements of Style2Paints (Project SEPA) here. 2022.01.09 - See some recent announcements of Style2Paints (Project SEPA) here. 2021.06.09 - An article on shadow drawing is accepted to ICCV 2021 as Oral. 2021.06.01 - The Project SEPA is decided to be released before 2022. 2021.03.22 - The next version of Style2Paints will be called Project SEPA. See also the twitter post.
Product
Style2Paints
Interactivity, Creativity, and Possibility!
Style2Paints is a line drawing coloring software. This software is somewhat popular
(>14k github stars) mainly because (1) it is easy to install and artists do not need to manage
environments like CUDA, (2) the result quality is good, and (3) the outputs are layered and friendly
to many painting workflows.
Interactivity, Creativity, and Possibility!
Style2Paints is a line drawing coloring software. This software is somewhat popular
(>14k github stars) mainly because (1) it is easy to install and artists do not need to manage
environments like CUDA, (2) the result quality is good, and (3) the outputs are layered and friendly
to many painting workflows.
PaintingLight
Physical illumination and painted illumination are different!
Painting light is a project conducted to investigate how artists add illumination and lighting
effects to their artworks, and how we can simulate this procedure to assist this workflow.
Physical illumination and painted illumination are different!
Painting light is a project conducted to investigate how artists add illumination and lighting
effects to their artworks, and how we can simulate this procedure to assist this workflow.
DanbooRegion
An region segmentation dataset for illustrations and artworks!
Lots of applications in the field of creative researches use regions to process images but the
region data for artworks and illustrations are rare. Style2Paints Research presents this dataset in
the hope that several related tasks can be made a bit easier.
An region segmentation dataset for illustrations and artworks!
Lots of applications in the field of creative researches use regions to process images but the
region data for artworks and illustrations are rare. Style2Paints Research presents this dataset in
the hope that several related tasks can be made a bit easier.
Member
Lvmin Zhang Jinyue Jiang Chengze Li Xinrui Wang
Research
Sprite-from-Sprite: Cartoon Animation Decomposition with Self-supervised Sprite Estimation,
in ACM Transactions on Graphics (SIGGRAPH ASIA 2022, Journal Track).
Lvmin
Zhang, Tien-Tsin Wong, and
Yuxin Liu.
The "sprites" in real-world cartoons are unique: artists may draw arbitrary sprite animations
for expressiveness, or alternatively, artists may also reduce their workload by tweening and
adjusting contents. Can we use these properties to do a "reverse engineering" to get the
original sprites in digital animation?
Know more
...
SmartShadow: Artistic Shadow Drawing Tool for Line Drawings, in IEEE International Conference
on Computer Vision (ICCV) 2021 Oral
(3%).
Lvmin
Zhang, Jinyue Jiang, Yi Ji, and
Chunping Liu.
A flexible shadow drawing tool for line drawings, supporting interactive editing of
cartoon-style shadows.
Know more
...
Screenshots from Screen Photography, in SIGGRAPH '21: ACM SIGGRAPH 2021 Posters.
Lvmin
Zhang and
Chengze Li.
Screenshot is a frequently used tool in our daily life, while the screenshot capturing
techniques are not much discussed in computer graphics and image processing researches. Might we
be able to achieve a computer graphic solution to directly convert a screen photography to a
screenshot, which looks like as if it was taken using software?
Know more ...
Generating Digital Painting Lighting Effects via RGB-space Geometry,
in ACM Transactions on Graphics (Presented in SIGGRAPH 2020). This paper does not use machine
learning!
Lvmin
Zhang, Edgar Simo-Serra, Yi Ji, and
Chunping Liu.
A project conducted to investigate how artists apply lighting effects to their artworks, and
how we can assist such workflow. The main idea is to observe the real painting behaviors and
procedures of artists so that we can model the illumination in their artworks.
Know more ...
Generating Manga from Illustrations via Mimicking Manga Workflow,
in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021.
Lvmin
Zhang, Xinrui Wang, Qingnan Fan, Yi Ji, and Chunping Liu.
We propose a data-driven framework to convert a digital illustration into three corresponding
components: manga line drawing, regular screentone, and irregular screen texture. These
components can be directly composed into manga images and can be further retouched for more
plentiful manga creations.
Know more ...
User-Guided Line Art Flat Filling with Split Filling Mechanism,
in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021.
Lvmin
Zhang, Chengze Li, Edgar Simo-Serra, Yi Ji, Tien-tsin Wong, and
Chunping Liu.
We present a deep learning framework for user-guided line art flat filling that explicitly
controls the "influence areas" of the user colour scribbles, i.e., the areas where the user
scribbles should propagate and influence, to manipulate the colours of image details and avoid
colour contamination between scribbles, and simultaneously, leverages data-driven colour
generation to facilitate content creation. Know more ...
DanbooRegion: An Illustration Region Dataset,
in European Conference on Computer Vision (ECCV) 2020.
Lvmin
Zhang, Yi Ji, and Chunping Liu
Region is a fundamental element of various cartoon animation techniques and artistic painting
applications. Achieving satisfactory region is essential to the success of these techniques. To
assist diversiform region-based cartoon applications, we use semi-automatic method to annotate
regions for in-the-wild artworks. Know more
...
Erasing Appearance Preservation in Optimization-based Smoothing,
in European Conference on Computer Vision (ECCV) 2020 Spotlight (5%).
This paper does not use machine learning!
Lvmin
Zhang, Chengze Li, Yi Ji, Chunping Liu, and Tien-tsin Wong
Optimization-based smoothing can be formulated as a smoothing energy and an appearance
preservation energy. We show that partially "erasing" the energy facilitate the smoothing.
Know more ...
Learning to Cartoonize Using White-box Cartoon Representations,
in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020.
Xinrui Wang and Jinze Yu
By observing the cartoon painting behavior and consulting artists, we propose to separately
identify three white-box representations of cartoon images. Know more ...
Two-stage Sketch Colorization,
in ACM Transactions on Graphics (SIGGRAPH Asia 2018).
Lvmin
Zhang, Chengze Li, Tien-tsin Wong, Yi Ji,
and Chunping Liu
With the advances of neural networks, automatic or semi-automatic colorization of sketch
become feasible and practical. We present a state-of-the-art semi-automatic (as well as
automatic) colorization from line art. Our improvement is accounted by a divide-and-conquer
scheme. We divide this complex colorization task into two simplier and goal-clearer subtasks,
drafting and refinement. Know more
...
Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN,
in Asian Conference on Pattern Recognition (ACPR) 2017.
Spotlight (5%) - The most cited paper of ACPR 2017 -
Lvmin
Zhang, Yi Ji, and Xin Lin
We integrate U-net to apply exemplar style to the grayscale
sketch with auxiliary classifier generative adversarial network. The whole process is automatic
and fast, and the results are creditable in the quality of artistic style as well as
colorization.
Know more ...
Talk
Anime Expo 2018 (Industrial Talk) 
Anime Expo is the largest anime exhibition in United States and one of the two largest anime
exhibitions in the world (another one is Japan Comic Market). S2PR presented an industrial talk on
"deep learning for artists" in Anime Expo 2018.
Anime Expo is the largest anime exhibition in United States and one of the two largest anime
exhibitions in the world (another one is Japan Comic Market). S2PR presented an industrial talk on
"deep learning for artists" in Anime Expo 2018.
Contact Us
- To contact Style2Paints Research for academic/commercial collaboration, please send e-mail to lvminzhang AT acm.org or lvminzhang AT siggraph.org with "Style2Paints" in your e-mail title.
- Feel free to open issues in any of our Github repos or reply to our Twitter posts.
- If you use QQ, we invite you to join our QQ group 816096787.