Image background generation

Image-guided

Generate a similar background for the main product based on a reference image. The model can naturally blend the main image into the background generated from the reference image.

# China (Beijing) region. Replace {WorkspaceId} with your Bailian workspace ID. URLs vary by region.
curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/background-generation/generation' \
--header 'X-DashScope-Async: enable' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model": "wanx-background-generation-v2",
    "input": {
        "base_image_url": "https://vision-poster.oss-cn-shanghai.aliyuncs.com/lllcho.lc/data/test_data/images/main_images/new_main_img/a.png",
        "ref_image_url": "http://vision-poster.oss-cn-shanghai.aliyuncs.com/lllcho.lc/data/test_data/images/ref_images/c5e50d27be534709817b2ab080b0162f_0.jpg"
    },
    "parameters": {
        "model_version": "v3",
        "n": 1,
        "noise_level": 300
    }
}'

Set the noise_level field

Use noise_level to adjust the correlation between the generated background and the reference image. The value range is [0, 999]. The default value is 300.

• Lower value: The generated background has a higher correlation with the reference image.

• Higher value: The generated background has a lower correlation with the reference image.

Text and image-guided

You can also combine a text prompt and a reference image to generate a background for the main product.

# China (Beijing) region. Replace {WorkspaceId} with your Bailian workspace ID. URLs vary by region.
curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/background-generation/generation' \
--header 'X-DashScope-Async: enable' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model": "wanx-background-generation-v2",
    "input": {
        "base_image_url": "https://vision-poster.oss-cn-shanghai.aliyuncs.com/lllcho.lc/data/test_data/images/main_images/new_main_img/a.png",
        "ref_image_url": "http://vision-poster.oss-cn-shanghai.aliyuncs.com/lllcho.lc/data/test_data/images/ref_images/c5e50d27be534709817b2ab080b0162f_0.jpg",
        "ref_prompt": "mountains and sunset clouds"
    },
    "parameters": {
        "model_version": "v3",
        "n": 1,
        "ref_prompt_weight": 0.5
    }
}'

Set the ref_prompt_weight field

When you specify both a text prompt and a reference image, use ref_prompt_weight to set the weight of the text prompt. This adjusts the influence of the text prompt and the reference image on the generated background. The value range is [0, 1]. The default value is 0.5.

• ref_prompt_weight < 0.5: The reference image has a greater influence on the generated background.

• ref_prompt_weight = 0.5: The text prompt and the reference image have equal weight and influence the generated background equally.

• ref_prompt_weight > 0.5: The text prompt has a greater influence on the generated background.

Text, image, and edge-guided elements (foreground/background)

This method combines a text prompt, a reference image, and edge-guided element images (foreground or background). It is suitable for complex product background generation scenarios that require detail. For edge-guided element images, a foreground image occludes the main product, while a background image layer is placed behind the main product. Note that the foreground and background images must be generated using an edge-guidance algorithm.

Why must foreground and background elements be generated with an edge-guidance algorithm?

Image background tasks often use an edge-guidance algorithm because it effectively preserves edge and structural information in an image. It also enhances detail and improves the naturalness and realism of the generated result.

How to add foreground/background elements?

• Use the edge-guided element generation method to generate foreground or background element images.

• You can set a prompt for each foreground or background element image, or leave it unset using an empty string as a placeholder.

• The order of elements in the foreground and background lists corresponds to the layer order in the generated background, from bottom to top.

In the image below, "peach blossoms" and "cute puppy" are the prompts for the foreground elements, and "leaves" is the prompt for the background element. The order of the foreground elements in the list is peach blossoms, then cute puppy. This corresponds to the layer order in the generated background: the peach blossom layer is in front of the main product, and the cute puppy layer is in front of the peach blossom layer. The background element is leaves, and its corresponding layer is behind the main product.

# China (Beijing) region. Replace {WorkspaceId} with your Bailian workspace ID. URLs vary by region.
curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/background-generation/generation' \
--header 'X-DashScope-Async: enable' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model": "wanx-background-generation-v2",
    "input": {
        "base_image_url": "https://vision-poster.oss-cn-shanghai.aliyuncs.com/lllcho.lc/data/test_data/images/main_images/new_main_img/a.png",
        "ref_image_url": "http://vision-poster.oss-cn-shanghai.aliyuncs.com/lllcho.lc/data/test_data/images/ref_images/c5e50d27be534709817b2ab080b0162f_0.jpg",
        "ref_prompt": "mountains and sunset clouds",
        "reference_edge": {
            "foreground_edge": [
                "https://vision-poster.oss-cn-shanghai.aliyuncs.com/lllcho.lc/data/test_data/images/huaban_soft_edge/6cdd13941cef1b11d885aea1717b983ae566b8efc9094-vcsvxa_fw658webp.png",
                "http://vision-poster.oss-cn-shanghai.aliyuncs.com/lllcho.lc/data/test_data/images/ref_edge/2c36cc4b7da027279e87311dac48fc2d5d784b1e72c0e-x4f1wC_fw658webp.png"
            ],
            "background_edge": [
                "http://vision-poster.oss-cn-shanghai.aliyuncs.com/lllcho.lc/data/test_data/images/ref_edge/0718a9741e07c52ca5506e75c4f2b99e22fff68a4c7d3-P9WGLr_fw658webp.png"
            ],
            "foreground_edge_prompt": [
                "pink peach blossoms",
                "cute puppy"
            ],
            "background_edge_prompt": [
                "leaves"
            ]
        }
    },
    "parameters": {
        "model_version": "v3",
        "n": 1,
        "ref_prompt_weight": 0.5,
        "noise_level": 300
    }
}'

Usage notes

Input image requirements

Main image requirements

The main image must be a four-channel RGBA image with a transparent background. This means the subject is in color and the background is transparent. The resolution of the output image matches the main image.

• Do not use a semi-transparent main image, as this may confuse the model.

• The main image should not contain text, such as large blocks of text, text in tables, or text mixed with images.

Reference image requirements

The reference image can be an RGB image or an RGBA image with a transparent background. For RGBA images, areas with an alpha channel value of 0 are not used in the generation process.

Foreground or background element image requirements

Each foreground or background image must be a four-channel RGBA image with a transparent background. Its resolution must match the main image. If the resolutions differ, the element image is automatically scaled to match the main image’s resolution.

For information on how to generate foreground or background element images, see method for generating edge-guided elements.

How to check and obtain an RGBA image

1. Check the image format. Common RGBA formats include PNG and WebP. JPG and JPEG are not RGBA formats.

2. Check if the image is an RGBA image. Common methods include the following:

   1. Right-click the image to view its file properties and check the channel information. If the properties do not show channel information, use image editing software, such as Photoshop, to check.

   2. Use a Python script to check the image properties. See the Print original image mode code in step 3.

3. Convert an RGB image to an RGBA image.

// Install the dependency package
pip install Pillow

from PIL import Image
import os

# Get the directory path of the current file
current_dir = os.path.dirname(os.path.abspath(__file__))
print(f"Current file directory path: {current_dir}")

# Open the image
image_path = os.path.join(current_dir, 'image.jpg')
image = Image.open(image_path)

# Print the original image mode
print("Original image mode:", image.mode)

# Convert to RGBA mode (to ensure an alpha channel)
if image.mode != 'RGBA':
    image = image.convert('RGBA')
else:
    print("Image is already in RGBA mode")

# --------------------- Add logic to make the background transparent ---------------------
# Set the background color to be made transparent (default is white, can be changed to other colors like (0, 0, 0) for black)
background_color = (255, 255, 255)  # RGB white
pixels = image.load()  # Load pixel data

for y in range(image.height):
    for x in range(image.width):
        r, g, b, a = pixels[x, y]
        # Check if the current pixel is the target background color (exact RGB match)
        if (r, g, b) == background_color:
            # Set the background color to fully transparent (Alpha=0)
            pixels[x, y] = (r, g, b, 0)
        else:
            # Keep non-background pixels (Alpha remains unchanged)
            pixels[x, y] = (r, g, b, a)

# --------------------- Save and display ---------------------
output_path = os.path.join(current_dir, 'output_transparent.png')  # Save to the current directory
image.save(output_path, 'PNG')
print(f"Transparent background image saved to: {output_path}")

# Display the image (optional)
image.show()