How to use a reference face image to control face generation in Fooocus

When you generate images of people in Fooocus, you may have noticed that you don’t have much control over the generated face. For example, below 4 images were generated using the same prompt.

Generation parameters

{
"Prompt": "A 22-year-old woman working at a burger chain, t-shirt, apron",
"Negative Prompt": "",
"Fooocus V2 Expansion": "",
"Styles": "[]",
"Performance": "Speed",
"Resolution": "(896, 1152)",
"Sharpness": 2,
"Guidance Scale": 4,
"ADM Guidance": "(1.5, 0.8, 0.3)",
"Base Model": "juggernautXL_version6Rundiffusion.safetensors",
"Refiner Model": "None",
"Refiner Switch": 0.5,
"Sampler": "dpmpp_2m_sde_gpu",
"Scheduler": "karras",
"Seed": 6317577737999617996, (Note: this is for the top-left image as seeds are different across images)
"LoRA 1": "sd_xl_offset_example-lora_1.0.safetensors : 0.1",
"Version": "v2.1.862"
}

As you may have noticed, faces can vary significantly across different images. You might wonder if it’s possible to control the type of face that appears in your target image. Specifically, if you’re working with a reference image that includes a face, is there a way to incorporate aspects of this face into your target image? Fooocus offers a feature that supports this capability. In this tutorial, we’ll provide step-by-step instructions on how to generate images using a face from a reference image as a guide. It’s important to note that the generated face might not be an exact replica of the one in the reference image. Therefore, if your goal is to create an image that perfectly mirrors the original, this feature might not meet your needs. However, if you’re looking for a degree of influence over the appearance of the target face, this feature will be quite useful. Let’s get started.

Disclaimer

Please note, before proceeding with generating an image using a face reference, ensure that your actions are in compliance with all applicable laws and regulations. Additionally, be aware of the potential for civil lawsuits arising from unauthorized or improper use of facial imagery. It’s important to consider both legal and ethical aspects in your practices.

Steps

Step 1. Reference photo preparation

First, prepare the source face. We will be using a below photo:

If you’re planning to use your own photo as a reference image for the face, I recommend adhering to the following guidelines:

Opt for a square image with a resolution of 1024×1024 pixels.
Ensure the photo is a headshot. This means it should predominantly feature the reference face, focusing on capturing facial features clearly.

Step 2. Image generation

Once you have the source image, do the following:

Set the prompt to match the image that you want to generate: A 22-year-old woman working at a burger chain, t-shirt, apron
Check the Input Image checkbox
Check the Advanced checkbox below the text prompt
Click the Image Prompt tab
Drag the source image from the output folder of Fooocus into the canvas under the Image Prompt tab
For Stop At, set to 0.9 if not set already
For Weight, set to 1
Check FaceSwap

Scroll down the screen and check Advanced.

Now take a look at the right side of the screen. Go to the Styles tab, and uncheck all styles.

Note that for generating the target, you do not need to use the same image resolution as the face reference image. For example you can specify 896 w x 1152 h. Now press Generate to generate images. Below are the 8 example images.

Although the faces generated are not exact replicas of those in the reference image, it’s evident that there is a noticeable influence. You can observe some consistency among them, indicating that the feature does capture key aspects of the original face. Utilizing this feature allows you to effectively project your inner vision onto the target image, ensuring a semblance of your intended aesthetic is maintained.

References

[1] lllyasviel. Fooocus 2.1.0 Image Prompts (Midjourney Image Prompts) #557. Retrieved from https://github.com/lllyasviel/Fooocus/discussions/557.