Comparison between after correction and before correction of light source
foreword
This article will discuss a problem that is often encountered when producing realistic or real-life pictures, that is, the light source is too bright.
On many popular models, due to the use of too many pictures of daytime models or studio highlight pictures for training, unnatural hyper-highlights will appear when the characters are drawn, even though the place is set at night, or the entire photo is blurred. The problem that the shadow of the light source does not follow the layout of the hint word.
The Stable Diffusion prompts introduced earlier — light and shadow and Stable Diffusion advanced — light and shadow control are both feasible light and shadow control methods, and this article is aimed at more subtle control, changing the light source and shadow of the generated picture again, And it produces different degrees of light and shade so that the producer can choose the best light and shadow results.
method
Let’s first use txt2img to generate a character image in the night forest:
(masterpiece, top quality, best quality, official art, beautiful and aesthetic:1.3), extreme detailed, Hana, fighting stance, angry, clenched teeth, floating hair, wearing white kimono and white cape, long sleeves, hand holding a wooden club (weapon), night, half moon, moonlight, fantasy, forest lora:Hana:0.4
In many cases, the pictures generated by the model do not necessarily have the layout of light and shade and light and shadow according to our ideas. Sometimes it is because too many models’ studios are used to take photos during model training, or LoRA lacks diverse light and shadow during training, or Using too many prompt words to restrain each other caused some strong light and shadow effects to be canceled. In short, when you can’t produce the desired light and shadow after trying various prompt words, you can try to use ControlNet with Loopback to generate a whole batch of pictures with different levels of light and shade, and then pick the best result from them.
First, in this case, I wanted to create a moon above the center of the frame as a light source. I open any drawing software, such as GIMP, open a new canvas, select a black background, and then use a brush to make a white light source mask, and input this image into img2img, and the prompt remains the same as the original image:
Then in the setting of img2img, use a medium weight like Denoising strength 0.4:
Then open ControlNet and set up two groups. The first group uses Openpose to fix the pose of the person when the picture is generated, and the use of ContorlNet is more important:
Openpose setting of ControlNet
The second group of ControlNet can use canny (edge detection) or normal (normal map) to fix the shape and expression of the characters in the original picture. Using ContorlNet is more important:
Finally, the most important thing is the setting of Loopback. With Loopback’s looping input capability, we can slowly restore the entire image from the pitch-black basic mask. Here I set 10 rounds, and set the Final denosing strength to a high weight of 0.75, which will produce a large change in light and dark in the next few rounds:
achievement
After everything is set up, you can run the picture, and after ten rounds are finished, you can choose the picture:
It can be seen that at the beginning, AI can only barely calculate the outline of the characters near the light source under the dark mask, but as the result map is continuously input, the picture will become clearer, and the shadow distribution will follow the relative relationship of the objects and the intensity of the light source changes.
If you use ControlNet’s canny to fix the shape, some strong shadow boundaries will be sampled, thus interfering with the light and shadow distribution. If so, you can download Canny’s picture to erase these extra lines, or you can use normal to fix it shape, but may cause a complete change in clothing style.
In the end, I selected a picture that best fit my imagination of a night forest with moonlight and enough darkness, about the seventh or so:
Changed brightness and shadow results
In addition to the change of the shadows of the characters, the shadows of the background forest and the road are also completely different, which is more in line with the atmosphere of moonlight lighting and basically meets my needs!
I wish you all a happy AI calculation!
