Deploy model services with custom images

Step 1: Set up your environment

Before you begin, make sure the following resources are ready:

• Virtual Private Cloud (VPC): EAS services, ECS instances, and ACR communicate over a VPC for stable, secure image push and pull operations.

• Container Registry (ACR): Stores and manages your custom images. See Push and pull images to set up an ACR Enterprise Edition instance with the required namespaces and repositories.

• Development environment: Create an ECS instance with the following configuration to build your image:

  • Instance type: ecs.u1-c1m2.large

  • Image: Alibaba Cloud Linux 3.2104 LTS 64-bit

  • Extension: Docker Community Edition

  • Network: Add the VPC to the ACR Enterprise Edition instance's access control settings; otherwise, image pushes to ACR will fail. For details, see Configure VPC access control for ACR.

  Note

  You can also use a local or other development environment.

  • Local development environment: Install and run Docker to build images.

  • DSW development environment: In the DSW instance's Actions column, click Create Image. The system builds a Docker image and saves it to ACR. When deploying, select Custom Image to choose the image from a drop-down list. For details, see Create a DSW instance image.

Step 2: Prepare your application files

Create a project folder (for example, my-app) with the following files:

1. requirements.txt (application dependencies)

   flask
   gunicorn

2. app.py (web application code)

   from flask import Flask
   
   app = Flask(__name__)
   
   @app.route('/hello/model')
   def hello_world():
       # Add your model inference or business logic here
       return 'Hello World from Gunicorn!'
   
   # Note: No app.run() needed — Gunicorn starts the app

1. Dockerfile (image build instructions)

   # 1. Use a lightweight official Python image as the base
   FROM python:3.9-slim
   
   # 2. Set the working directory
   WORKDIR /app
   
   # 3. Copy and install dependencies first to leverage Docker's layer cache
   COPY requirements.txt .
   RUN pip install --no-cache-dir -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
   
   # 4. Copy application code
   COPY app.py .
   
   # 5. Declare the service port
   EXPOSE 8000
   
   # 6. Define the default container startup command (can be overridden by the EAS "Run command" setting)
   CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:8000", "app:app"]

Step 3: Build and push the image

1. Open a terminal and navigate to the project folder (my-app).

2. Log in to the ACR instance. Replace the placeholders in the command with actual values.

   # Example: docker login --username=your_user my-registry.cn-hangzhou.aliyuncs.com
   docker login --username=<username> <ACR registry domain>

3. Build the image with the full tag.

   # Example: docker build -t my-registry.cn-hangzhou.aliyuncs.com/my-namespace/flask-app:v1 .
   docker build -t <ACR registry domain>/<namespace>/<repository name>:<tag> .

4. (Recommended) Verify the image locally before pushing. Run the following command to confirm the image starts correctly before pushing to ACR, avoiding broken images in your registry.

   # Replace <port> with the port declared in your Dockerfile's EXPOSE instruction (8000 in this example)
   docker run -p <port>:<port> <ACR registry domain>/<namespace>/<repository name>:<tag>

   If the application responds normally, the image is ready to push.

5. Push the image to ACR.

   # Example: docker push registry.cn-hangzhou.aliyuncs.com/my-namespace/flask-app:v1
   docker push <ACR registry domain>/<namespace>/<repository name>:<tag>

Step 4: Deploy the service

1. Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).

2. Click Deploy Service, select Custom Model Deployment > Custom Deployment.

3. Configure the following key parameters:

   • Deployment Method: Select Image-based Deployment.

   • Image Configuration: Select Image Address, then enter the full image address from the previous step.

     Because this example uses ACR under the same Alibaba Cloud account, EAS can pull the image without credentials.

   • Command to Run: gunicorn -w 4 -b 0.0.0.0:8000 app:app

     How it works: The run command entered in the EAS console overrides the CMD instruction in the Dockerfile. Define the startup command here to make it easier to debug and update later.

   • Port Number: 8000.

   • Deployment: CPU resources are sufficient for this example. Under Public Resources, select ecs.c6.large.

   • VPC: Select a VPC, vSwitch, and security group.

     Make sure the VPC you select here has been added to the ACR instance's access control settings. If not, image pulls will fail due to network connectivity issues (a common error is ImagePullBackOff).

4. Click Deploy. When the service status changes to Running, the deployment is complete.

Step 5: Test the service

After the service is deployed, get the endpoint address and token, then run the following command to test the Flask service:

# Replace <endpoint> and <token> with the actual service endpoint and token
curl <endpoint>/hello/model -H "Authorization: <token>"

If the response is Hello World from Gunicorn!, the service is deployed and accessible.

For more information about calling services, see Service invocation methods.

Q: Image pull fails with ImagePullBackOff

Troubleshoot in this order:

1. Image address: Verify that the image address, namespace, and tag are correct, and that the image was successfully pushed to the registry.

2. System disk space: If you see the error no space left on device, expand the system disk.

3. Network connectivity:

   • Using a public internet image address: Confirm that the EAS VPC has public internet access and that public access is enabled for ACR. Note that cross-region pulls over the public internet are slow and prone to network failures.

   • Using a private network image address: Cross-region pulls over the private network require ACR Enterprise Edition and VPC connectivity between regions. For details, see Access ACR Enterprise Edition instances across regions or from IDC.

4. ACR access control: Based on the type of image address you're using, check whether Configure internet access control or Configure VPC access control is configured for ACR.

5. Registry credentials: If you're using a private registry that requires authentication (not ACR under your own account), check that the credentials in the image configuration are correct.

Q: How do I use an ACR image across regions?

By default, EAS can pull images from ACR repositories in the same region over the VPC private network. If the ACR instance and EAS service are in different regions, private network access isn't available. Use one of the following approaches:

• Option 1 (recommended): Create a new ACR instance in the same region as your EAS service and push the image to that regional repository before deploying. This avoids cross-region issues.

• Option 2: Connect the VPCs across both regions using Cloud Enterprise Network (CEN) to enable cross-region private network pulls. For details, see Access an Enterprise Edition instance across regions or from a data center.

• Option 3 (testing only): Pull images over a public internet address. Make sure public access is enabled for ACR and that the EAS service VPC has a NAT gateway for internet access. This option is slow and poses security risks — use it for testing only.

Q: What does "no match for platform in manifest" mean when deploying a custom image?

This error means the image architecture (for example, arm64) doesn't match the EAS instance's runtime environment (typically amd64/x86_64). This commonly occurs when building an image on an Apple Silicon Mac (arm64) and deploying it to an EAS instance (amd64). To resolve this:

1. Confirm the CPU architecture of the target EAS instance (most EAS instances use amd64).

2. Rebuild the image locally to match the EAS instance architecture:

   # Build an amd64 image (compatible with most EAS instances)
   docker build --platform linux/amd64 -t <ACR registry domain>/<namespace>/<repository name>:<tag> .

3. Push the new image to ACR and redeploy the service.

Q: Can ACK connect to the EAS dedicated image registry (PAI official images)?

No. The EAS dedicated image registry doesn't support direct ACK connections.