How to use GFP-GAN in Automatic1111

GFP-GAN is a cutting-edge software technology designed for face restoration in images. It incorporates a U-Net-based degradation removal module that works alongside a pretrained face GAN, such as StyleGAN2 to restore a face in the image [1]. GFP-GAN represents one of the most remarkable technologies I’ve encountered. Upon its release, I used it to restore old photographs and was astounded by how vividly it made people seem as if they were photographed in the modern era. It possesses an almost magical ability to bring images to life. Another fascinating application of GFP-GAN is its use in enhancing 3D renders such as DAZ3D. Despite using advanced raytracers like Iray, achieving human-like realism in renders can be challenging. However, applying GFP-GAN transforms these renders to a level that is virtually indistinguishable from real humans. The photo below serves as an example of this facial restoration. The image on the left, generated using Stable Diffusion, mimics an old photo, while the right one has been enhanced with GFP-GAN. Notice the extraordinary restoration of details, especially in the eye area – it’s simply magical.

GFP-GAN also excels at adding those final touches that transform a photo into something convincingly real. The image on the left was generated with Stable Diffusion, while the right-hand image shows the results post GFP-GAN application. Observe how the subtle imperfections in the left image, which detract from its believability as a photo, are completely eliminated in the right image. This showcases GFP-GAN’s remarkable ability to enhance and authenticate visual content.

Utilizing GFP-GAN in Automatic1111 is straightforward. Simply transfer the photo from other tabs (such as txt2img or img2img) to the Extras tab, and select the GFPGAN checkbox. Once you click Generate, your image will be processed with GFP-GAN. You have the option to use this feature with or without Upscalers. However, it’s common to pair it with upscalers, especially when enlarging a smaller image, like one with a resolution of 512×512, to a higher resolution. You can also adjust the strength of GFP-GAN using visibility slider.

In this section, we will discuss several important considerations when using GFP-GAN. These include:

• Unintended Restoration of Blurred Faces in the Background
• Issues with Disfigured Face Recovery
• Inconsistencies at the Boundary between Restored and Unrestored Areas

GFP-GAN may inadvertently restore faces that are intentionally blurred or meant to remain in the background.

In the photo shown above, the woman’s face was blurred in the original image below. However, GFP-GAN identified and restored the face. The disparity between the restored face and her still-blurry body creates an unusual visual effect.

GFP-GAN is not suitable for restoring distorted or damaged face in terms of geometry.

The original face is shown on the left with the right image processed with GFP-GAN. Though the right image is sharper, the distortion of the original is still preserved after applying GFP-GAN. To address face distortion, you need to use a different method such as Hires. fix or Inpainting.

As GFP-GAN focuses on face restoration rather than the entire body or image, the boundary of restoration often falls within the hair region. In the images above, while the original on the left exhibits no distinct boundary in the hair, the right image, processed with GFP-GAN, shows a clear delineation. The hair closer to the forehead is more detailed, whereas the upper portion remains blurry. When using GFP-GAN, it’s crucial to carefully observe these boundaries and decide if additional steps, like photo-editing to blend the boundary or adjusting the strength of GFP-GAN, are necessary. Despite these nuances, GFP-GAN remains an indispensable tool for creating photo-realistic images in Stable Diffusion, offering significant potential to elevate your artistic creations.

[1] Wang, X., et al. (2021). Towards Real-World Blind Face Restoration with Generative Facial Prior. Retrieved from https://arxiv.org/abs/2101.04061