DevPod development environment

Step 1: Create a DevPod instance

1. In the navigation pane on the left of Model Marketplace, select Custom Development.

2. Configure the following key parameters:

   Parameter	Description	Example
   Region	In the upper-right corner, select the Alibaba Cloud region where you want to deploy the DevPod instance.	China (Hangzhou)
   Select Model Environment	Select a pre-configured deep learning framework environment, or specify a custom image.	Modelscope 1.26.0
   Model Name	Set an easy-to-identify name for the instance.	tokenizer-deploy-dev
   Model Description	Briefly describe the model's purpose, features, or development goals.	ModelScope tokenization model
   Model Source	Enter the path information according to the model source instructions.	None (The Modelscope 1.26.0 model environment already includes the required content).
   Resource Configuration	Select the Instance Type and GPU Specifications based on your task requirements.	Elastic Instance > GPU Advanced
   Role Name	Specify the RAM role used to access cloud resources. The role requires necessary permissions.	AliyunFCDefaultRole

3. Click DevPod Develop and Debug.

Step 2: Develop and debug

1. Enter the VS Code online development environment.

2. Open a terminal and execute ./run.sh to download the model to external storage and start the application. You can view the model loading and web service startup logs in the Operational Log.

3. Open a new terminal and run curl localhost:9000 to view the tokenization result.

Step 3: Create an image and deploy the service

After you finish development and debugging, you can package the environment with its code and dependencies into a container image. Then, you can deploy it as an online service.

Step 4: Verify the online service

1. After the deployment succeeds, go to the model service details page and select Online Debugging.

2. In the API Online Debugging Interface > Manually Configure API Destination section, configure the request:

   • Path: Enter /.

   • HTTP Method: Select GET.

3. Click Send Request to view the model's response in the Response area on the right.

Environment management

You can manage the complete lifecycle of your DevPod instances on the Custom Development page:

• Create: Configure and create new development environments as needed.

• Start/Stop: Stop instances when not in use to save on compute resource fees. When an instance is stopped, its compute resources are released, but persistent data in the /mnt directory is retained.

• Delete: Permanently delete an instance. Note that this operation is irreversible, but data in the /mnt directory is still retained.

Development tools

DevPod integrates three main development tools into a single interface. You can switch between them at any time on the DevPod Development and Debugging page of the instance:

• VS Code: A powerful online integrated development environment (IDE) based on openvscode-server. It provides full code editing, debugging, Git management, and terminal functions.

• Jupyter Notebook: The de facto standard interactive computing environment. It is ideal for data exploration, algorithm experiments, and model visualization.

• Terminal: A complete Linux Shell environment. You can run any command-line operation, such as installing software or managing files.

Persistent storage

Each DevPod instance automatically mounts a NAS file storage directory at /mnt/<devpod_name>. Data in this directory is not lost when the instance is destroyed. We recommend storing important files, such as model weights and training data, in this directory. Files outside the /mnt path are on local storage and may be lost after the instance restarts.

Default environment variables in DevPod:

HF_HOME=/mnt/<devpod_name>/hf
MODELSCOPE_CACHE=/mnt/<devpod_name>/modelscope
OLLAMA_MODELS=/mnt/<devpod_name>/ollama

Features of persistent storage:

1. It is not built into the image.

2. The content persists even if the DevPod instance is deleted.

3. This storage is suitable for storing large model files and helps control the image size.

Tip: The built-in development images in DevPod include sample code files, main.py and run.sh, by default. You can run run.sh directly to start the application. The first time you run run.sh, it automatically downloads the model to the persistent NAS disk. The download directory is the one specified by the MODELSCOPE_CACHE environment variable.

Custom images

When pre-configured images do not meet your needs, you can use your own custom image to create a DevPod instance. For example, you might need specific library versions, custom system tools, or a private base image.

• Usage: When creating a DevPod instance, set Model Environment to Custom Image, and enter the registry address from Container Registry (ACR).

• Image requirements:

  • Operating system: A Linux system based on the AMD64 architecture. The minimum requirement is glibc 2.28. Recommended base images include debian:10, ubuntu:20.04, centos:8, or later versions. alpine images are not supported.

  • Pre-installed tools: The curl command must be pre-installed.

  • User permissions: The default user is root.

  • Optional support: To use Jupyter, the image must have Python ≥ 3.8 installed and include the pip command.

  • GPU compatibility: Function Compute GPU instances currently use driver version 570.133.20, which corresponds to CUDA user-mode driver version 12.8. For the best compatibility, we recommend using a CUDA Toolkit version that is at least 11.8 but does not exceed the CUDA user-mode driver version provided by the platform. For more information, see Product Change: GPU Driver Upgrade from 550.54.15 to 570.133.20.

Image building and ACR integration

You can package the complete runtime environment of your current DevPod instance, including code, dependencies, and configurations, into a container image. You can then push the image to Alibaba Cloud Container Registry (ACR) to save and reuse the environment.

Remote access and debugging

DevPod lets you expose an HTTP service that is running inside an instance through a secure proxy. This feature is useful for remote debugging or joint frontend-backend debugging.

• Procedure:

  1. Start an HTTP service in the DevPod terminal. For example:

     python -m http.server 8000

  2. On the Quick Access page of your DevPod instance, find the proxy address for the corresponding port. The address is usually in the following format:

     https://<devpod-id>.cn-<region>.ide.fc.aliyun.com/proxy/8000/

  3. Use this URL to access your service from a local browser or with tools such as curl.

• Important:

  This proxy method adds a /proxy/<port> prefix to the URL. If your frontend application, such as a React or Vue single-page application (SPA), uses absolute paths to reference static resources, such as /main.js, this may cause resource loading to fail. To resolve this issue, you must modify the build configuration of your frontend application to add a public path prefix.

File transfer

• Through the IDE interface: In the file browser on the left of VS Code or JupyterLab, you can upload and download files using the context menu or by dragging and dropping. This is the most convenient method for small files.

• Through the HTTP API:

  # Upload a file
  curl -T /tmp/app.py -u root:{vscodeToken} {url}/webdav/app.py
  
  # Download a file
  curl -u root:{vscodeToken} -o app.py {url}/webdav/app.py
  
  # Delete a file
  curl -X DELETE -u root:{vscodeToken} {url}/webdav/app.py

  Note

  On the Quick Access tab in the DevPod console, you can find the command to retrieve the relevant parameters.

• Large file transfer: For large files, such as datasets or models, we recommend that you upload them to Alibaba Cloud Object Storage Service (OSS) first. Then, use the ossutil tool in the DevPod terminal to download them at high speed to the /mnt directory.

Cost optimization

• Start and stop instances promptly: Stop your DevPod instance during non-working hours. When an instance is stopped, you are charged only a small storage fee, which can significantly reduce compute resource costs.

• Use persistent storage effectively: Store large datasets and model files in the /mnt directory instead of packaging them into the image. This practice can significantly reduce the image size, which lowers ACR storage costs and the time required to pull the image during model deployment.

Security guidelines

• Custom exclude path: When you build an image, use the Custom Exclude Path feature to exclude irrelevant or sensitive content, such as local cache, logs, or configuration files, from the production image.

• Use ACR Enterprise Edition for security scanning: For production environments, we recommend that you use ACR Enterprise Edition and enable the image security scanning feature to promptly detect security vulnerabilities in your images.

Team collaboration workflow

1. Build a base environment image: A team lead or architect can create a custom image that includes all common dependencies, tools, and configurations, and then push it to the team's shared ACR Enterprise Edition repository.

2. Share the development environment: Team members can create their own DevPod instances based on this shared image to ensure a consistent development environment for everyone.

3. Share data and models: Team members can easily access and share large datasets and model files through a shared NAS file system or OSS.

Official pre-configured images

To fine-tune the official environment, you can use a FunModel pre-configured image as your base image:

• China regions: serverless-registry.cn-hangzhou.cr.aliyuncs.com/functionai/vllm-openai:<tag>

• Regions outside China: serverless-registry.ap-southeast-1.cr.aliyuncs.com/functionai/vllm-openai:<tag>

Available <tag>s include v0.10.1, v0.10.2, and v0.11.0. To ensure that you are using the latest environment, we recommend that you check the following link for a complete list of image versions: FunModel Pre-configured Image List.

Troubleshooting (FAQ)

DevPod instance startup failure

• Symptom: The instance is stuck in the 'Starting' state for a long time or fails to start.

• Troubleshooting steps:

  • Check the custom image: If you are using a custom image, confirm that the image can be pulled and started normally in a local or other environment.

  • View instance logs: View the logs to identify the specific error.

Image deployment service failure

• Symptom: The model service deployment times out or fails.

• Possible causes and solutions:

  • Image pull timeout: If the image is large, the underlying node may take a long time to pull it during the first deployment. You can manually trigger a redeployment on the service details page.

  • Service startup failure:

    • Check whether the image meets the requirements, such as the `curl` and glibc versions mentioned in the image requirements section.

    • Check whether the failure is due to insufficient GPU resources and retry.

  • For more information, see GPU Instance FAQ.

Cannot access JupyterLab

• Symptom: In a GPU-accelerated instance, JupyterLab cannot be started or accessed.

• Troubleshooting steps:

  • In the DevPod terminal, confirm that the Python version is 3.8 or later.

  • Check whether the jupyter-related Python libraries are installed correctly.

  • View the JupyterLab startup logs to locate the specific error.

Billing

The costs associated with DevPod include the following:

• DevPod instance fees: These are the fees for the underlying compute resources (vCPU, memory, and GPU), which are billed based on usage duration. Billing stops when you stop the instance.

• Storage fees: These are the fees for the mounted NAS persistent storage, which are billed based on capacity and storage class.

• Service deployment fees: These are the fees for the compute resources and public bandwidth that are generated after you deploy the model as an online service.

• Other fees: You may also incur fees for services such as Container Registry (ACR) Enterprise Edition and Object Storage Service (OSS).