Deploy model services with custom images

Step 1: Prepare the environment

Make sure the following resources are available in the same region (such as China (Hangzhou)) before you begin:

Step 2: Prepare the project files

Log in to the ECS instance, create a project folder such as my-app, and add the following three files:

1. requirements.txt (application dependencies)

   flask
   gunicorn

2. app.py (web application code)

   from flask import Flask
   
   app = Flask(__name__)
   
   @app.route('/hello/model')
   def hello_world():
       # Integrate your model inference or other business logic here
       return 'Hello World from Gunicorn!'
   
   # Note: app.run() is not needed. Gunicorn starts the application.

3. Dockerfile (image build instructions)

   # 1. Use the lightweight official Python image as the base
   FROM python:3.9-slim
   
   # 2. Set the working directory
   WORKDIR /app
   
   # 3. Copy the dependencies file and install. This layer is cached to speed up subsequent builds.
   COPY requirements.txt .
   RUN pip install --no-cache-dir -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
   
   # 4. Copy the application code
   COPY app.py .
   
   # 5. Declare the service port
   EXPOSE 8000
   
   # 6. Define the default container startup command (can be overridden by the Run Command in the EAS console)
   CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:8000", "app:app"]

Step 3: Build and push the image

1. Open a terminal and go to the project folder (my-app).

2. Log in to your ACR instance. Replace the placeholders in the command with your actual values.

   Note

   On the details page of the ACR repository, you can find login, pull, and push command examples. The examples use public endpoints. Replace them with private endpoints.

   # Example: docker login --username=your_user my-registry-vpc.cn-hangzhou.aliyuncs.com
   docker login --username=<username> <ACR private endpoint>

3. Build the Docker image and tag it with the full repository path.

   # Example: docker build -t my-registry-vpc.cn-hangzhou.aliyuncs.com/pai-demo/flask-app:v1 .
   # The full image address is <ACR private endpoint>/<namespace>/<repository name>:<tag>
   docker build -t <ACR private endpoint>/<namespace>/<repository name>:<tag> .

4. (Recommended) Verify the image locally before pushing. Run the following command to make sure the image starts correctly before you push it to the repository.

   # Replace <port> with the EXPOSE port in your Dockerfile (8000 in this example)
   docker run -p <port>:<port> <ACR private endpoint>/<namespace>/<repository name>:<tag>

   If the application responds as expected, the image is ready to push.

5. Push the image to ACR.

   # Example: docker push my-registry-vpc.cn-hangzhou.aliyuncs.com/pai-demo/flask-app:v1
   docker push <ACR endpoint>/<namespace>/<repository name>:<tag>

Step 4: Deploy the service

1. Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).

2. Click Deploy Service and choose Custom Model Deployment > Custom Deployment.

3. Configure the following key parameters:

   • Deployment Method: Select Image-based Deployment.

   • Image Configuration: Select Image Address and enter the full image address from the previous step: my-registry-vpc.cn-hangzhou.aliyuncs.com/pai-demo/flask-app:v1.

     Because this example uses ACR under the same Alibaba Cloud account, EAS can pull the image without credentials. No username or password is required.

   • Command to Run: gunicorn -w 4 -b 0.0.0.0:8000 app:app

     The Run Command in the EAS console overrides the CMD instruction in the Dockerfile. Enter the startup command here for easier debugging and modification later.

   • Port Number: 8000.

   • Deployment: This example uses CPU resources. Select Public Resources and choose ecs.c6.large.

   • VPC: Select a VPC (VPC_HZ), a vSwitch, and a security group.

     Make sure the VPC you select here has been added to the access control list of the ACR instance. Otherwise, image pull fails due to network connectivity issues. A common error is ImagePullBackOff.

4. Click Deploy. The deployment succeeds when the service status changes to Running.

Step 5: Test the service

After the service is deployed, obtain the endpoint and token and run the following command to test the Flask service:

# Replace <endpoint> and <token> with the actual service endpoint and token
curl <endpoint>/hello/model -H "Authorization: <token>"

If Hello World from Gunicorn! is returned, the service is deployed and accessible.

For more information about service invocation, see Service invocation methods.

Image registry types

EAS pulls images from an image registry when deploying services. We recommend Alibaba Cloud Container Registry (ACR) for image storage. Use VPC endpoints to pull images for better performance and security.

Image registry authentication

For registries that require authentication, provide credentials when you deploy the service.

Specify the credentials in the JSON configuration file through the dockerAuth parameter. The value is the Base64 encoding of username:password.

For example, to encode abcd:abcde12345, run echo -n "abcd:abcde12345" | base64 to get YWJjZDphYmNkZTEy****. Use this as the dockerAuth value.

{"dockerAuth": "YWJjZDphYmNkZTEy****"}

Q: Image pull fails (ImagePullBackOff)

Check the following items:

1. Image address: Verify that the image address, namespace, and tag are correct, and that the image has been pushed to the repository.

2. System disk space: If you see the error no space left on device, expand the system disk.

3. Network connectivity:

   • Using a public endpoint: Make sure EAS is configured with a VPC that has Internet access, and enable public access for ACR. Cross-region pull over the Internet is slow and prone to failure.

   • Using a private endpoint: For cross-region scenarios, only ACR Enterprise Edition is supported. You must connect the VPCs. For details, see Access an ACR Enterprise Edition instance across regions or from an on-premises data center.

4. ACR access control: Based on the image address type you use, check whether ACR Configure internet access control or Configure VPC access control is configured.

5. Registry authentication: If you use a private registry that requires authentication (not ACR under your account), verify that the credentials in Image Configuration are correct.

Q: How do I use ACR images across regions?

EAS can pull images from ACR repositories in the same region through VPC private endpoints by default. If the ACR instance and EAS service are in different regions, direct private network access is not available. Use one of the following methods:

• Method 1 (recommended): Create an ACR instance in the same region as the EAS service, push the image to the local repository, and then deploy. This eliminates cross-region issues entirely.

• Method 2: Use CEN (Cloud Enterprise Network) to connect VPCs across regions for private network image pull. For details, see Access an Enterprise Edition instance across regions or from a data center.

• Method 3 (testing only): Pull images over the Internet. Enable public access for ACR and configure a NAT gateway with Internet access for the EAS service. This method is slow and has security risks. Use it only for testing.

Q: Custom image deployment fails with "no match for platform in manifest"

This error means the image architecture (such as arm64) does not match the EAS instance runtime environment (typically amd64/x86_64). EAS cannot run this image on the selected instance type. This commonly happens when you build an image on an M-series Mac (arm64) and deploy it to an EAS (amd64) instance. To fix this:

1. Confirm the CPU architecture of the target EAS instance (most EAS instances use amd64).

2. Rebuild the image locally to match the EAS instance architecture:

   # Build an amd64 image (compatible with most EAS instances)
   docker build --platform linux/amd64 -t <ACR endpoint>/<namespace>/<repository name>:<tag> .

3. Push the new image to ACR and redeploy the service.

Q: Can ACK connect to the EAS dedicated image registry (PAI official images)?

The EAS dedicated image registry does not support direct ACK connections. ACK cannot pull images from the EAS dedicated image registry.