DeepFaceDrawing Generates Photorealistic Portraits from Freehand Sketches

A team of researchers from the Chinese Academy of Sciences and the City University of Hong Kong has introduced a local-to-global approach that can generate lifelike human portraits from relatively rudimentary sketches.

Recent deep image-to-image translation techniques have enabled the prompt generation of human face images from sketches, but these methods tend to suffer from overfitting to their inputs. They thus achieve the most realistic results only when the source drawings have high-quality artistry or are accompanied by edge maps.

Unlike most deep learning based solutions for sketch-to-image translation that take input sketches as fixed, ‘hard’ constraints and then attempt to reconstruct the missing texture or shading information between strokes, the key idea behind the new approach is to implicitly learn a space of plausible face sketches from real face sketch images and find the point in this space that best approximates the input sketch. Because this approach treats input sketches more as ‘soft’ constraints that will guide image synthesis, it is able to produce high-quality face images with increased plausibility even from rough and/or incomplete inputs.

dn-6.4.png
Illustration of the model’s deep learning framework architecture

The system consists of three main modules — CE (Component Embedding), FM (Feature Mapping), and IS (Image Synthesis). The CE module adopts an auto-encoder architecture and separately learns five feature descriptors — left-eye, right-eye, nose, mouth, and remainder — from the face sketch data. The FM and IS modules together form another deep learning sub-network for conditional image generation, and map component feature vectors to realistic images.

The researchers also provide a shadow-guided interface, implemented based on CE, that makes it easier for users to refine the input sketches. Their system can produce high-quality realistic face images — with resolution of 512 × 512 — that faithfully respect and reflect the input sketches.

Both qualitative and quantitative evaluations show that the method produces visually more pleasing face images, according to the researchers. The system’s usability and expressiveness were also favourably confirmed in a user study.

The researchers say their tool is easy to use, even for non-artists, while still supporting fine-grained control of shape details. They are working on releasing the source code soon.

The paper DeepFaceDrawing: Deep Generation of Face Images from Sketches has been accepted by SIGGRAPH 2020 and is available on arXiv.


Journalist: Yuan Yuan | Editor: Michael Sarazen

Read More

68 Replies to “DeepFaceDrawing Generates Photorealistic Portraits from Freehand Sketches“

  1. Thanks for another informative site. The place else may I get that kind of info written in such a perfect method? I’ve a project that I’m just now working on, and I’ve been at the glance out for such info.|

  2. First off I want to say great blog! I had a quick question that I’d like to ask if you don’t mind. I was interested to find out how you center yourself and clear your head before writing. I’ve had a difficult time clearing my thoughts in getting my ideas out there. I do enjoy writing however it just seems like the first 10 to 15 minutes are usually wasted just trying to figure out how to begin. Any recommendations or tips? Thank you!|

  3. First of all I would like to say awesome blog! I had a quick question which I’d like to ask if you do not mind. I was curious to know how you center yourself and clear your head prior to writing. I’ve had a tough time clearing my mind in getting my ideas out. I truly do enjoy writing but it just seems like the first 10 to 15 minutes are generally lost simply just trying to figure out how to begin. Any ideas or hints? Thanks!|

  4. This is the right blog for anybody who really wants to find out about this topic. You realize a whole lot its almost hard to argue with you (not that I really would want to…HaHa). You certainly put a brand new spin on a subject that’s been written about for years. Excellent stuff, just great!|

  5. Hey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest twitter updates. I’ve been looking for a plug-in like this for quite some time and was hoping maybe you would have some experience with something like this. Please let me know if you run into anything. I truly enjoy reading your blog and I look forward to your new updates.|

  6. Howdy! This is my first visit to your blog! We are a group of volunteers and starting a new initiative in a community in the same niche. Your blog provided us useful information to work on. You have done a outstanding job!|

  7. Hi this is somewhat of off topic but I was wondering if blogs use WYSIWYG editors or if you have to manually code with HTML. I’m starting a blog soon but have no coding knowledge so I wanted to get guidance from someone with experience. Any help would be greatly appreciated!|

  8. certainly like your web site but you have to take a look at the spelling on quite a few of your posts. A number of them are rife with spelling issues and I to find it very troublesome to inform the reality however I will surely come back again.|

  9. Normally I do not read article on blogs, but I wish to say that this write-up very forced me to try and do it! Your writing taste has been amazed me. Thank you, quite nice post.|

  10. I loved as much as you’ll receive carried out right here. The sketch is attractive, your authored material stylish. nonetheless, you command get got an edginess over that you wish be delivering the following. unwell unquestionably come more formerly again as exactly the same nearly very often inside case you shield this hike.|

  11. When I originally commented I clicked the “Notify me when new comments are added” checkbox and now each time a comment is added I get several emails with the same comment. Is there any way you can remove me from that service? Thank you!|

  12. A motivating discussion is worth comment. I think that you need to write more on this subject, it might not be a taboo matter but typically people don’t discuss these subjects. To the next! Many thanks!!|

  13. I loved as much as you will receive carried out right here. The sketch is attractive, your authored subject matter stylish. nonetheless, you command get bought an shakiness over that you wish be delivering the following. unwell unquestionably come further formerly again as exactly the same nearly very often inside case you shield this hike.|

  14. Hey there! I just would like to offer you a huge thumbs up for the excellent information you’ve got here on this post. I am coming back to your website for more soon.|

  15. I think this is among the most vital information for me. And i’m glad reading your article. But should remark on some general things, The website style is wonderful, the articles is really nice : D. Good job, cheers|

  16. Nice post. I was checking constantly this weblog and I am inspired! Very useful information particularly the remaining part 🙂 I maintain such information a lot. I used to be looking for this certain information for a very lengthy time. Thanks and best of luck. |

  17. Very good blog! Do you have any tips and hints for aspiring writers? I’m hoping to start my own website soon but I’m a little lost on everything. Would you recommend starting with a free platform like WordPress or go for a paid option? There are so many choices out there that I’m completely confused .. Any recommendations? Thanks a lot!|

  18. I’m not sure where you’re getting your information, but good topic. I needs to spend some time learning much more or understanding more. Thanks for magnificent info I was looking for this info for my mission.|

  19. Can I just say what a comfort to uncover someone that actually knows what they are talking about online. You certainly understand how to bring a problem to light and make it important. More people really need to read this and understand this side of the story. I was surprised you are not more popular given that you most certainly possess the gift.|

  20. Does your website have a contact page? I’m having trouble locating it but, I’d like to send you an e-mail. I’ve got some ideas for your blog you might be interested in hearing. Either way, great website and I look forward to seeing it grow over time.|

  21. I’m not sure where you are getting your information, but great topic. I needs to spend some time learning more or understanding more. Thanks for excellent info I was looking for this info for my mission.|

  22. My developer is trying to persuade me to move to .net from PHP. I have always disliked the idea because of the expenses. But he’s tryiong none the less. I’ve been using WordPress on numerous websites for about a year and am nervous about switching to another platform. I have heard great things about blogengine.net. Is there a way I can import all my wordpress content into it? Any help would be greatly appreciated!|

  23. Having read this I believed it was extremely enlightening. I appreciate you spending some time and effort to put this article together. I once again find myself spending a significant amount of time both reading and commenting. But so what, it was still worth it!|

  24. I truly love your site.. Pleasant colors & theme. Did you develop this amazing site yourself? Please reply back as I’m attempting to create my own personal site and would like to know where you got this from or just what the theme is called. Thanks!|

  25. I’m not sure why but this web site is loading incredibly slow for me. Is anyone else having this issue or is it a issue on my end? I’ll check back later on and see if the problem still exists.|

  26. With havin so much written content do you ever run into any issues of plagorism or copyright infringement? My website has a lot of exclusive content I’ve either written myself or outsourced but it appears a lot of it is popping it up all over the web without my agreement. Do you know any techniques to help protect against content from being stolen? I’d certainly appreciate it.|

  27. This is really interesting, You are a very skilled blogger. I have joined your feed and look forward to seeking more of your magnificent post. Also, I have shared your web site in my social networks!|

  28. I’m no longer positive where you’re getting your info, but good topic. I needs to spend some time learning more or figuring out more. Thank you for magnificent information I used to be in search of this information for my mission.|

  29. Greetings from Idaho! I’m bored to death at work so I decided to browse your website on my iphone during lunch break. I really like the knowledge you provide here and can’t wait to take a look when I get home. I’m shocked at how fast your blog loaded on my cell phone .. I’m not even using WIFI, just 3G .. Anyways, great site!|

  30. Hey, I think your site might be having browser compatibility issues. When I look at your blog in Opera, it looks fine but when opening in Internet Explorer, it has some overlapping. I just wanted to give you a quick heads up! Other then that, great blog!|

  31. Excellent beat ! I would like to apprentice whilst you amend your web site, how could i subscribe for a blog site? The account aided me a applicable deal. I were tiny bit acquainted of this your broadcast offered vibrant clear idea|

  32. You are so awesome! I don’t think I have read a single thing like this before. So good to find somebody with a few unique thoughts on this topic. Seriously.. thank you for starting this up. This web site is something that’s needed on the internet, someone with a bit of originality!|

  33. Good day I am so happy I found your blog, I really found you by accident, while I was looking on Askjeeve for something else, Regardless I am here now and would just like to say many thanks for a marvelous post and a all round thrilling blog (I also love the theme/design), I don’t have time to browse it all at the minute but I have saved it and also added in your RSS feeds, so when I have time I will be back to read much more, Please do keep up the awesome work.|

  34. My coder is trying to convince me to move to .net from PHP. I have always disliked the idea because of the expenses. But he’s tryiong none the less. I’ve been using WordPress on a variety of websites for about a year and am anxious about switching to another platform. I have heard fantastic things about blogengine.net. Is there a way I can import all my wordpress posts into it? Any help would be really appreciated!|

  35. Hi, I do believe this is a great web site. I stumbledupon it 😉 I may revisit yet again since i have book-marked it. Money and freedom is the best way to change, may you be rich and continue to help others.|

  36. I seriously love your blog.. Great colors & theme. Did you build this amazing site yourself? Please reply back as I’m looking to create my own personal blog and want to learn where you got this from or what the theme is named. Cheers!|

  37. I’m now not sure the place you are getting your info, however great topic. I needs to spend some time studying much more or figuring out more. Thank you for great info I used to be on the lookout for this info for my mission.|

  38. My programmer is trying to convince me to move to .net from PHP. I have always disliked the idea because of the expenses. But he’s tryiong none the less. I’ve been using WordPress on a variety of websites for about a year and am nervous about switching to another platform. I have heard fantastic things about blogengine.net. Is there a way I can transfer all my wordpress posts into it? Any help would be really appreciated!|

  39. You actually make it seem really easy with your presentation however I to find this matter to be actually one thing that I feel I would never understand. It kind of feels too complex and very huge for me. I am having a look ahead to your subsequent submit, I will attempt to get the grasp of it!|

  40. I absolutely love your blog and find most of your post’s to be just what I’m looking for. Do you offer guest writers to write content in your case? I wouldn’t mind composing a post or elaborating on a lot of the subjects you write concerning here. Again, awesome blog!|

  41. Have you ever thought about including a little bit more than just your articles? I mean, what you say is fundamental and everything. Nevertheless think of if you added some great photos or video clips to give your posts more, “pop”! Your content is excellent but with images and clips, this website could definitely be one of the best in its niche. Awesome blog!|

  42. Excellent goods from you, man. I’ve understand your stuff previous to and you’re just too wonderful. I actually like what you’ve acquired here, really like what you’re saying and the way in which you say it. You make it enjoyable and you still care for to keep it wise. I cant wait to read far more from you. This is really a tremendous web site.|

  43. My brother recommended I might like this web site. He was once totally right. This post actually made my day. You cann’t consider just how much time I had spent for this information! Thanks!|

Leave a Reply

Your email address will not be published.