How to use CodeFormer in Automatic1111

CodeFormer, an advanced face restoration tool, is integrated into Automatic1111. It excels in restoring faces that are more degraded than those typically handled by GFP-GAN. A key feature of CodeFormer is its adjustable parameter, ‘weight,’ which can be adjusted to level of restoration you want to apply based on the extent of damage in the input image. Setting the weight to 0 enables restoration of higher degree of damage, albeit with potential minor deviations from the original (as seen in the rightmost image). Setting the weight to 1 (the image in the middle) offers a milder restoration, preserving the face’s original appearance while enhancing its sharpness.

Here is another example. In this case, setting weight=1 is also the best choice than setting weight to 0 which changes the face.

Using CodeFormer in Automatic1111 is straightforward. First, transfer an image from either the txt2img or img2img tab to the Extras tab, or directly upload the source image in the Extras tab. Then, select the CodeFormer checkbox.

• Visibility: This setting adjusts the blend between the CodeFormer-enhanced image and the original [2]. If the restoration appears too strong, you can lower this value. However, be cautious when mixing the processed image with the original if using weight=0, as it might lead to unintended effects due to changes in face geometry.
• Weight: Set it to 1 for mild degradation and to 0 for more severe cases. Note that setting it to 0 might alter the face geometry.

CodeFormer can be used with or without upscalers, depending on your preference.

Important considerations when using CodeFormer

In this section, we will discuss several important considerations when using CodeFormer. These are the similar issues that were discussed in How to use GFP-GAN in Automatic1111 so you can skip this section if you have already read the post. These include:

• Unintended Restoration of Blurred Faces in the Background
• Issues with Disfigured Face Recovery
• Inconsistencies in the Hair Area

Unintended Restoration of Blurred Faces in the Background

CodeFormer may inadvertently restore faces that are intentionally blurred or meant to remain in the background.

In the photo shown above, the woman’s face was blurred in the original image on the left. However, CodeFormer identified and restored the face for both weight=0 and weight=1. The disparity between the restored face and her still-blurry body creates an unusual visual effect.

Issues with Disfigured Face Recovery

CodeFormer can handle slightly higher degree of face degradation than GFP-GAN.

In the above images, the right-most face is restored nicely with weight set to 0, which was different from the result for GFP-GAN as shown in How to use GFP-GAN in Automatic1111.

However, if the degree of degradation or distortion is high, then CodeFormer still cannot handle it as shown in the right-most image above. Therefore, different approach such as Inpainting or Hires. fix is required.

After applying CodeFormer, it’s advisable to carefully examine the hair area. As observed in the photo above, certain sections of the hair might exhibit finer details compared to others. Depending on your specific needs, you may need to refine these inconsistencies using photo-editing software to achieve a more uniform appearance.

[1] Zhou, S., et al. (2022). Towards Robust Blind Face Restoration with Codebook Lookup Transformer. Retrieved from https://arxiv.org/abs/2206.11253

[2] AUTOMATIC1111. stable-diffusion-webui. Retrieved from https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/scripts/postprocessing_codeformer.py#L31