Deploy and fine-tune Stable Diffusion V1.5 for text-to-image

Prerequisites

You have created an OSS bucket. Create a bucket in the console.

1. Access model details

1. Go to the Model Gallery page.

   1. Log in to the PAI console.

   2. In the top-left corner, select a region based on your actual situation.

   3. In the left-side navigation pane, choose Workspaces, click the name of the target workspace to enter that workspace. In the left-side navigation pane, choose Quick Start > Model Gallery.

2. On the Model Gallery page, search for and click the Stable_Diffusion_V1.5 model card.

2. Deploy and test the model

1. On the model details page, click Deploy in the upper-right corner. In the Deploy panel, select a deployment resource such as ecs.gn6e-c12g1.3xlarge. Keep default values for other parameters.

2. Click Deploy. In the Billing Notification dialog box, click OK.

   The Service details page opens. In the Basic Information section, check the deployment status. When Status changes to Running, the service is deployed.

3. In the upper-right corner of the service details page, click View Web App to launch the WebUI.

4. On the WebUI, test the model.

   On the text-to-image tab, enter a prompt in the prompt area, for example: The eagle is flying in the sky, in the distance is the vast snow, and under the snow mountain is a grassland. Click Generate to view the result. For finer control, enter a prompt such as The eagle is flying in the sky, in the distance is the vast snow, and under the snow mountain is a grassland in the positive prompt text box, set sampling method to Euler a, sampling steps to 20, width and height to 512, CFG Scale to 7, and seed to -1. Click Generate. The image appears in the output area on the right.

3. Fine-tune the model

Fine-tune Stable Diffusion V1.5 to customize output for your use case.

1. Return to the model details page and click Fine-tune in the upper-right corner.

2. In the Train panel, expand the configuration sections to review the following parameters. Keep defaults for unlisted parameters. Model deployment and training.

   Configuration	Description
   Dataset Configuration	PAI provides a default dataset. To use your own dataset, prepare it as follows: Prepare training data based on the training data format on the model details page. Download the sample dataset for reference. Directory structure: |——img1.jpg |——img2.jpg |——... |——metadata.jsonl The metadata.jsonl file uses JSON Lines format. file_name specifies the image file name, and text contains the corresponding caption. Example metadata.jsonl: {"file_name": "img1.jpg", "text": "cat"} {"file_name": "img2.jpg", "text": "penguin"} {"file_name": "img3.jpg", "text": "dragon"} Upload the training data to an OSS bucket. Upload files. Update the training dataset configuration. Model deployment and training.
   Hyperparameter configuration	Keep training_method set to standard. Use defaults for other parameters.
   Output Configuration	Set Model output path to an OSS bucket path. Quick start.

3. Click Train.

   The Task details page opens.

After training completes, click Deploy in the upper-right corner of the task details page to deploy the fine-tuned model. The deployment process is the same as in II. Directly deploy and debug a model.