Manage third-party libraries in DSW-Platform For AI(PAI)-阿里云帮助中心

In a DSW instance, you can use the Terminal to install, view, uninstall, and update Python third-party libraries. DSW automatically saves the updated environment to the image.

Install a third-party library

DSW development environments include Python 2, Python 3, PyTorch, and TensorFlow 2.0. By default, third-party libraries are installed in the Python 3 environment. To install a library in a different environment, you must first manually switch to that environment.

# Install in the Python 3 environment.
pip install <yourLibraryName>
# Install in the Python 2 environment.
source activate python2
pip install <yourLibraryName>
# Install in the TensorFlow 2.0 environment.
source activate tf2
pip install <yourLibraryName>

Replace <yourLibraryName> with the name of the library. For example, run pip install bottle to install the bottle library.

If you run the installation command in a Jupyter Notebook cell, prefix the command with an exclamation mark (!), for example: !pip install bottle.

Important

If a package installs successfully but is unavailable in a notebook, restart the kernel.

View third-party libraries

Run the following command to view installed third-party libraries.

pip list

Uninstall a third-party library

To uninstall a third-party library, run the following command.

pip uninstall <yourLibraryName>

Replace <yourLibraryName> with the name of the installed third-party library.

Note

You can only uninstall libraries that you have installed yourself.

Update a third-party library

Run the following command to update an installed third-party library.

pip install --upgrade numpy==<versionNumber>

Replace <versionNumber> with the target version of numpy.

View or change a pip mirror

Run the following command to view the pip configuration:

pip config list

In the output, the value of the global.index-url parameter is the global pip mirror URL. Sample output:

global.index-url='https://mirrors.aliyun.com/pypi/simple/'
global.trusted-host='mirrors.aliyun.com'
install.trusted-host='mirrors.aliyun.com'

To temporarily set a pip mirror for a single installation:

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple <yourLibraryName>
# If an SSL verification error occurs, try adding the --trusted-host parameter. For example:
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn <yourLibraryName>

To change the global pip mirror, edit the configuration file. The default path of the pip configuration file is ~/.config/pip/pip.conf. Run the following command to edit the file:

vim ~/.config/pip/pip.conf

For example, to switch to the Tsinghua University pip mirror, update the index-url and trusted-host parameters, then save and exit. The modified pip.conf is as follows:

[global]
trusted-host=pypi.tuna.tsinghua.edu.cn
index-url = https://pypi.tuna.tsinghua.edu.cn/simple/

[install]
trusted-host=pypi.tuna.tsinghua.edu.cn

FAQ

Q: After I stop a DSW instance, will the packages I installed with `pip` and the code I wrote be lost?

No. Your data is retained as long as the instance uses a cloud disk for its system disk. All disk data on the instance, including the environments in /mnt/workspace and /root, is retained. When you start the instance again, all environments and files are still there. Only deleting the instance permanently clears all data.

Q: Why is an installed third-party library not taking effect?

After you install a third-party library with the pip command, if you see a "module not found" error when you use the import command, first try restarting the service or the kernel. If the error persists, check the active environment. By default, DSW installs third-party libraries into the Python 3 environment. To install a library into another environment, you must manually activate that environment first. For example:

Install into the Python 2 environment.
source activate python2
pip install --user xxx
Install into the TensorFlow 2.0 environment.
source activate tf2
pip install --user xxx

Replace xxx with the name of the third-party library you want to install.

Q: A package installation failed in DSW with `pip install` due to dependency conflicts or version errors. What should I do?

This is usually caused by environment incompatibility. Follow these troubleshooting steps:

Switch the image. Stop the current instance, create a new DSW instance, and select a different official image. For example, if the current PyTorch 2.1 image does not work, try a PyTorch 2.3 image or an image from the modelscope series, which often provides better compatibility.
Install a specific version. Check the package's official documentation, find a version that supports your current DSW environment (Python/CUDA versions), and then run pip install package_name==x.y.z.
Switch the download source. Try using a different mirror, such as the Tsinghua mirror: pip install -i https://pypi.tuna.tsinghua.edu.cn/simple <yourLibraryName>.

Q: Why can't a library installed in the DSW Terminal be imported in a Jupyter Notebook?

The Terminal and Jupyter may be using different Python environments. You can run the which python command to check the current Python environment, or install the required library within the notebook. For example:

Q: My code reports an error indicating that the CUDA driver version is too low. Do I need to manually upgrade the NVIDIA driver in DSW?

Do not upgrade the driver version. The driver and CUDA versions in a DSW instance are pre-installed and locked. You cannot and should not modify them manually. Modifying them can corrupt the instance and make it unrecoverable. The correct approach is to switch the DSW image. Stop the current instance, create a new instance, and select an official image that includes a higher CUDA and driver version.

For example, the official image modelscope:1.9.4-pytorch2.0.1tensorflow2.13.0-gpu-py38-cu118-ubuntu20.04. Here, cu118 indicates that the CUDA version is 11.8.

Q: Can I use Docker to deploy my applications in DSW?

Docker is supported only on DSW instances created from Lingjun resources or version 1.0 of a General-purpose Resource Group. For details, see Use DockerBoard to manage Docker and Use Docker in DSW.

Q: The `unzip` or `7z` command is not available in my DSW instance. How can I extract files?

You can use the apt-get command to install the required tools.

To install unzip, run apt-get update && apt-get install -y unzip in the Terminal, and then use unzip your_file.zip.
To install p7zip (for 7z files), run apt-get update && apt-get install -y p7zip-full in the Terminal, and then use 7z x your_file.7z.

Q: Why does installing a third-party library hang or time out?

If a third-party library installation hangs, times out, or is extremely slow, it is usually a network issue. Follow these steps to troubleshoot:

Step 1: Check network connectivity

In the Terminal, run the ping www.aliyun.com command to test public internet access. If the network is unavailable, proceed to the next step to check the gateway configuration.

Step 2: Check the gateway configuration

On the DSW instance configuration page, check the type of Internet Access Gateway:

Public Gateway: DSW uses a Public Gateway by default for public internet access. You can check the gateway type on the DSW instance configuration page. A Public Gateway has limited bandwidth, and the network speed may be insufficient for downloading large files. In this case, consider using a Dedicated Gateway.
Dedicated Gateway: A Dedicated Gateway provides higher network access speeds. After you select a Dedicated Gateway, you must create an internet NAT gateway in the VPC, bind an Elastic IP (EIP), and configure SNAT. Otherwise, you cannot access the public internet. For more information, see Improve public network access speed with a dedicated gateway.

Step 3: Try a different pip mirror

DSW uses the Alibaba Cloud mirror by default, which may experience issues during peak hours or network fluctuations. We recommend switching to other mirrors:

# Install using the Tsinghua mirror (recommended)
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn <yourLibraryName>

# Install using the USTC mirror
pip install -i https://pypi.mirrors.ustc.edu.cn/simple --trusted-host pypi.mirrors.ustc.edu.cn <yourLibraryName>

# Install using the Douban mirror
pip install -i https://pypi.doubanio.com/simple --trusted-host pypi.doubanio.com <yourLibraryName>

To permanently change the default pip mirror, see View or change a pip mirror.

Step 4: Use offline installation

If the network is unavailable or unstable, use offline installation:

On your local machine (with a good network connection), download the installation package in .whl format:
```
# Run on your local machine
pip download <yourLibraryName> -d ./packages
```
Upload the downloaded .whl file to the DSW instance. For instructions, see Upload and download files.
Perform the offline installation in DSW:
```
pip install /path/to/your-package.whl
```

Q: How do I get root permissions in the WebIDE?

Most official DSW images run as the root user by default. If the Terminal prompt shows root@..., you already have root permissions. You can safely ignore the warning that appears during pip installation about running as root. If your image does not log in as root, this is by design for that image. You must switch to an image that supports root.

Q: How do I start an X server in DSW?

DSW does not support starting an X server.