EAIS package overview

Python programs

To install the eais-torch .whl package, first make sure that Python 3 and pip3 are installed in your environment. Then, follow these steps:

1. Run the following commands to set the version numbers for the EAIS and PyTorch packages.

   The version numbers that you set are automatically used in the subsequent steps.

   Important

   • EAIS_PACKAGE_VERSION: The version number of the EAIS package. Only 4.2.5 is supported.

   • pytorch_VERSION: The version number of the PyTorch package. This topic uses PyTorch 1.13.1 as an example.

   export EAIS_PACKAGE_VERSION=4.2.5
   export pytorch_VERSION=1.13.1

2. Download and install the eais-torch .whl package.

   sudo pip3 install eais-torch${pytorch_VERSION}==${EAIS_PACKAGE_VERSION} \
     -f https://eais-rel-pub.oss-cn-beijing.aliyuncs.com/packages/index.html

3. Run the following command to check the installation result.

   pip3 list | grep eais

   If the output contains information about the eais-torch package, the installation was successful.

C++ programs

1. Run the following commands to set the version numbers for the EAIS and libtorch packages.

   The version numbers that you set are automatically used in the subsequent steps.

   Important

   • EAIS_PACKAGE_VERSION: The version number of the EAIS package. Only 4.2.5 is supported.

   • libtorch_VERSION: The version number of the libtorch package. This topic uses libtorch 1.13.1 as an example.

   export EAIS_PACKAGE_VERSION=4.2.5
   export libtorch_VERSION=1.13.1

2. Obtain the package download URL.

   EAIS provides packages that depend on libtorch and are compiled with either Pre-cxx11 ABI or cxx11 ABI. The following table lists the download URLs for the eais-torch packages that include libtorch.

   ECS instance operating system	Compilation method	Package type	EAIS package download URL
   Ubuntu	Pre-cxx11 ABI	deb	https://eais-rel-pub.oss-cn-beijing.aliyuncs.com/packages/eais-torch${libtorch_VERSION}_${EAIS_PACKAGE_VERSION}_amd64.deb
   	cxx11 ABI	deb	https://eais-rel-pub.oss-cn-beijing.aliyuncs.com/packages/eais-torch${libtorch_VERSION}-cxx11-abi_${EAIS_PACKAGE_VERSION}_amd64.deb
   CentOS or Alibaba Cloud Linux	Pre-cxx11 ABI	rpm	https://eais-rel-pub.oss-cn-beijing.aliyuncs.com/packages/eais-torch${libtorch_VERSION}-${EAIS_PACKAGE_VERSION}-1.x86_64.rpm
   	cxx11 ABI	RPM	https://eais-rel-pub.oss-cn-beijing.aliyuncs.com/packages/eais-torch${libtorch_VERSION}-cxx11-abi-${EAIS_PACKAGE_VERSION}-1.x86_64.rpm

3. Download and install the EAIS package.

   • The ECS instance runs Ubuntu

     The following steps use the eais-torch .deb package that depends on libtorch and is compiled with cxx11 ABI as an example:

     1. Run the following commands to download and install the eais-torch .deb package.

        wget https://eais-rel-pub.oss-cn-beijing.aliyuncs.com/packages/eais-torch${libtorch_VERSION}-cxx11-abi_${EAIS_PACKAGE_VERSION}_amd64.deb
        dpkg -i eais-torch${libtorch_VERSION}-cxx11-abi_${EAIS_PACKAGE_VERSION}_amd64.deb
        source /etc/profile

     2. Run the following command to check the installation result.

        dpkg -l | grep eais-torch

        If the output contains information about the eais-torch package, the installation was successful.

   • The ECS instance runs CentOS or Alibaba Cloud Linux

     The following steps use the eais-torch .rpm package that depends on libtorch and is compiled with cxx11 ABI as an example:

     1. Run the following commands to download and install the EAIS torch .rpm package.

        wget https://eais-rel-pub.oss-cn-beijing.aliyuncs.com/packages/eais-torch${libtorch_VERSION}-cxx11-abi-${EAIS_PACKAGE_VERSION}-1.x86_64.rpm
        sudo rpm -ivh eais-torch${libtorch_VERSION}-cxx11-abi-${EAIS_PACKAGE_VERSION}-1.x86_64.rpm --nodeps
        source /etc/profile

     2. Run the following command to check the installation result.

        rpm -qa | grep eais-torch

        If the output contains information about the eais-torch package, the installation was successful.

Python programs

To enable eais-torch, add the following line to your Python inference script.

import eais.torch_eais

C++ programs

Assume that your C++ program for PyTorch model inference is named my_program and you run it using the following commands:

export LD_LIBRARY_PATH=/your/lib/path:$LD_LIBRARY_PATH
./my_program my_model.pt

To use EAIS to infer your PyTorch model, run your C++ program using one of the following command options. eais-torch is automatically enabled at runtime.

• Command 1:

  export LD_LIBRARY_PATH=/your/lib/path:$LD_LIBRARY_PATH
  LD_PRELOAD=libtorcheais.so ./my_program my_model.pt

• Command 2:

  export LD_LIBRARY_PATH=/your/lib/path:$LD_LIBRARY_PATH
  export LD_PRELOAD=libtorcheais.so
  ./my_program my_model.pt