DevPod development environment

Step 1: Create a DevPod instance

1. In the Model Marketplace left-side navigation pane, select Custom Development.

2. Configure the following key parameters:

   Parameter	Description	Example
   Region	In the upper-right corner, select the Alibaba Cloud region where you want to deploy the DevPod instance.	China (Hangzhou)
   Select model environment	Select a pre-built deep learning framework environment, or specify a custom image.	Modelscope 1.26.0
   Model name	Set an easy-to-identify name for the instance.	tokenizer-deploy-dev
   Model description	Briefly describe the model's purpose, features, or development goals.	ModelScope tokenization model
   Model source	Enter the path information based on the model source instructions.	None (The Modelscope 1.26.0 model environment already includes the required content)
   Resource configuration	Select an Instance type and GPU Specifications based on your task requirements.	Elastic Instance > GPU Advanced
   Role name	Specify the RAM role used to access cloud resources (requires necessary permissions).	AliyunFCDefaultRole

3. Click DevPod Development and Debugging.

Step 2: Develop and debug

1. Go to the VS Code online development environment.

2. Open a Terminal and run ./run.sh to download the model to the external storage and start the application. You can view the model loading and web service startup logs in the Operational Log.

3. Open a new terminal and run curl localhost:9000 to view the tokenization result.

Step 3: Create an image and deploy the service

After development and debugging, you need to package the current environment, including its code and dependencies, into a container image and then deploy it as an online service.

Step 4: Verify the online service

1. After the deployment is successful, go to the model service details page and click Online Debugging.

2. On the API Online Debugging Interface > Manually Configure API Destination, configure the request:

   • Path: Enter /.

   • HTTP Method: Select GET.

3. Click Send Request. You can see the model's response in the Response Result area on the right.

Environment management

On the Custom Development page, you can manage the complete lifecycle of your DevPod instances:

• Create: Configure and create a new development environment based on your requirements.

• Start/Stop: Stop an instance when not in use to reduce compute resource fees. When an instance is stopped, compute resources are released, but persistent data in the /mnt directory is retained.

• Delete: Permanently delete an instance. Note that this operation is irreversible, but data in the /mnt directory is retained.

Development tools

DevPod integrates three mainstream development tools into a single interface. You can switch between them at any time on the DevPod Development and Debugging page of your instance:

• VS Code: A powerful online integrated development environment based on openvscode-server. It provides comprehensive features for code editing, debugging, Git management, and terminal operations.

• Jupyter Notebook: The de facto standard for interactive computing environments. It is ideal for data exploration, algorithm experiments, and model visualization.

• Terminal: A complete Linux Shell environment where you can perform any command-line operation, such as installing software or managing files.

Persistent storage

Each DevPod instance automatically mounts a NAS file storage directory at /mnt/<devpod_name>. Data in this directory persists even after the instance is destroyed. We recommend storing important files, such as model weights and training data, in this directory.

Default environment variables enabled in DevPod:

HF_HOME=/mnt/<devpod_name>/hf
MODELSCOPE_CACHE=/mnt/<devpod_name>/modelscope
OLLAMA_MODELS=/mnt/<devpod_name>/ollama

Features of persistent storage:

1. It is not packaged into the image during image creation.

2. Content persists even if the DevPod instance is deleted.

3. It is suitable for storing large model files, which helps control the image size.

Tip: DevPod's built-in development images include sample code files main.py and run.sh by default. You can run run.sh directly to start the application. The first time you run run.sh, the model is automatically downloaded to the persistent NAS storage. The model is downloaded to the directory specified by the MODELSCOPE_CACHE environment variable.

Custom images

If pre-built images do not meet your needs, for example, you require specific library versions, custom system tools, or a private base image, you can use a custom image to create a DevPod instance.

• How to use: When creating a DevPod instance, in the Model environment section, select Custom Image and enter the ACR address of your image.

• Image requirements:

  • Operating system: A Linux system based on the AMD64 architecture. The minimum requirement is glibc 2.28. Recommended base images include debian:10, ubuntu:20.04, centos:8, or later versions.

  • Pre-installed tools: The image must include the curl command.

  • User permissions: The default user is root.

  • Optional support: To use Jupyter, the image must have Python 3.8 or later installed and include the pip command.

  • GPU compatibility: The GPU driver version used by Function Compute is currently 570.133.20, which corresponds to CUDA user-mode driver version 12.8. For optimal compatibility, we recommend using a CUDA Toolkit version of at least 11.8, and it should not exceed the CUDA user-mode driver version provided by the platform. For more information, see [Product Change] GPU Driver Version Upgraded from 550.54.15 to 570.133.20.

Image building and ACR integration

You can package the complete runtime environment of your current DevPod instance, including code, dependencies, and configurations, into a Docker image. You can then push it to Alibaba Cloud Container Registry (ACR) for storage and reuse.

Remote access and debugging

DevPod lets you expose HTTP services running inside an instance through a secure proxy for remote debugging or collaborative frontend-backend debugging.

• Procedure:

  1. Start an HTTP service in the DevPod terminal. For example:

     python -m http.server 8000

  2. On the Quick Access page of the DevPod instance, find the proxy address for the corresponding port. The address format is usually:

     https://<devpod-id>.cn-<region>.ide.fc.aliyun.com/proxy/8000/

  3. You can use this URL to access your service from a local browser or using tools such as curl.

• Important:

  This proxy method adds a /proxy/<port> prefix to the URL. If your frontend application (such as a React/Vue SPA) uses absolute paths to reference static resources (such as /main.js), this may cause resource loading to fail. You need to modify the build configuration of your frontend application to add a public path prefix.

File transfer

• Through the IDE interface: In the file browser on the left side of VS Code or JupyterLab, you can upload and download files using the context menu or by dragging and dropping. This is the most convenient method for small files.

• Through the HTTP API:

  # Upload a file
  curl -T /tmp/app.py -u root:{vscodeToken} {url}/webdav/app.py
  
  # Download a file
  curl -u root:{vscodeToken} -o app.py {url}/webdav/app.py
  
  # Delete a file
  curl -X DELETE -u root:{vscodeToken} {url}/webdav/app.py

  Note

  On the Quick Access tab in the DevPod console, you can find the parameters for this command.

• Large file transfer: For large files, such as datasets or models, we recommend first uploading them to Alibaba Cloud Object Storage Service (OSS). Then, in the DevPod terminal, use the ossutil tool to quickly download them to the /mnt directory.

Cost optimization

• Start and stop instances promptly: Stop your DevPod instance during non-working hours. In the stopped state, you are only charged a small storage fee, which can significantly reduce compute resource costs.

• Use persistent storage effectively: Store large datasets and model files in the /mnt directory instead of packaging them into the image. This can significantly reduce the image size, which lowers ACR storage costs and reduces the time required to pull the image during model deployment.

Security best practices

• Custom excluded paths: When creating an image, use the Custom Excluded Paths feature to avoid packaging irrelevant or sensitive content, such as local caches, logs, or configuration files, into the production image.

• Use ACR Enterprise Edition for security scans: For production environments, we recommend using ACR Enterprise Edition and enabling the image security scan feature to promptly detect security vulnerabilities in your images.

Team collaboration workflow

1. Build a base environment image: The team lead or architect creates a custom image containing all common dependencies, tools, and configurations, and pushes it to the team's shared ACR Enterprise Edition repository.

2. Share the development environment: Team members create their own DevPod instances based on this shared image to ensure a consistent development environment.

3. Share data and models: Through a shared NAS file system (or OSS), team members can easily access and share large datasets and model files.

Official pre-built images

To fine-tune the official environment, you can use a FunModel pre-built image as your base image:

• China regions: serverless-registry.cn-hangzhou.cr.aliyuncs.com/functionai/vllm-openai:<tag>

• Regions outside China: serverless-registry.ap-southeast-1.cr.aliyuncs.com/functionai/vllm-openai:<tag>

Available <tag> values include v0.10.1, v0.10.2, and v0.11.0. To ensure you are using the latest environment, we recommend referring to the following link for a complete list of image versions: FunModel Pre-built Image List.

Troubleshooting (FAQ)

DevPod instance startup failure

• Symptom: The instance remains in the "Starting" state for a long time or fails to start.

• Troubleshooting steps:

  • Check the custom image: If you are using a custom image, confirm that the image can be pulled and started correctly in your local environment or other environments.

  • View instance logs: Check the logs to identify the specific error.

Image deployment service failure

• Symptom: The model service deployment times out or fails.

• Possible causes and solutions:

  • Image pull timeout: If the image is large, the underlying node may take a long time to pull it during the first deployment. You can manually trigger a redeployment on the service details page.

  • Service startup failure:

    • Check if the image meets the requirements (such as the curl and glibc versions mentioned in the image requirements section).

    • Check if the failure is caused by insufficient GPU stock and retry.

  • For more information, see GPU-accelerated instance FAQ.

JupyterLab is inaccessible

• Symptom: In a GPU-accelerated instance, JupyterLab cannot start or be accessed.

• Troubleshooting steps:

  • In the DevPod terminal, confirm that the Python version is 3.8 or later.

  • Check if the jupyter-related Python libraries are installed correctly.

  • View the JupyterLab startup logs to locate the specific error.

Billing

The fees associated with DevPod include the following:

• DevPod instance fee: The cost of the underlying compute resources (vCPU, memory, GPU), which is billed based on usage duration. Billing stops when the instance is stopped.

• Storage fee: The cost of the mounted NAS persistent storage, billed based on capacity and storage class.

• Service deployment fee: The cost of compute resources and public bandwidth incurred after deploying the model as an online service.

• Other fees: Potential fees for services such as ACR Enterprise Edition instances and OSS.