Deploy privacy computing nodes

更新时间:
复制 MD 格式

After you complete the preparations, you can deploy the privacy computing nodes as described in this topic.

Cluster networking

Note

If you are using single-machine mode, you can skip this step.

Deploying nodes in a cluster helps address the performance limitations of single training nodes and increases the number of parallel tasks. A multi-replica mode improves the stability of prediction nodes. When you use docker-compose, containers on the same host can communicate, but containers on different hosts are isolated. To allow containers on different hosts to communicate, you can use a network plugin. This section describes how to set up cluster networking using Docker's built-in Overlay Network.

Environment check

Check your current firewall rules. Ensure that the following ports are open for communication between cluster nodes:

  • TCP port 2377 for cluster management communication.

  • TCP port 7946 for communication between nodes.

  • UDP port 7946 for communication between nodes.

  • UDP port 4789 for overlay network traffic.

Overlay networking

  1. Log on to a node client in the cluster. Run the following command to set the current node as a Manager node.

    docker swarm init

    The system returns a message similar to the following. Obtain and record the token value. If you lose the token, you can run the docker swarm join-token worker command to retrieve it.

    Swarm initialized: current node (zwjcr9w3zr9a72oxw39j7xxxx) is now a manager.
    
    To add a worker to this swarm, run the following command:
    
        docker swarm join --token SWMTKN-1-3nuu8r2s3z9eq2o3tl47wjgacpfuxpg64eqsc8wb5t8buijzbx-d2qw8h52loc0ithrvnosxxxxx 172.xx.xx.12:2377
    
    To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
  2. Run the following command to create an overlay network. Set the CIDR block to 11.0.0.0/8.

    docker network create -d overlay --subnet 11.0.0.0/8 --attachable nueva-exchange
  3. Log on to another node client in the cluster. Run the following command to set the current node as a Worker node. ${TOKEN} is the token value that you obtained in Step 1.

    docker swarm join --token ${TOKEN}
  4. Unzip the node deployment package. Go to the extracted folder and run the install.sh script.

    ./install.sh
  5. Run the following command to start the container and discover the network.

    IMAGE=`docker image ls | head -2 | tail -1 | awk '{print $1":"$2}'`
    echo $IMAGE
    docker run -dit --rm --net nueva-exchange $IMAGE sh
  6. Run the following command to view the network list for the current node.

    docker network ls

    The system returns a message similar to the following. The NAME is nueva-exchange, which confirms that the node successfully joined the overlay network.

    NETWORK ID          NAME                DRIVER              SCOPE
    vdqud8syxxxx       nueva-exchange      overlay             swarm

Node configuration and installation

Follow these steps to install a node for the first time.

  1. Log on to the node client and unzip the node deployment package.

  2. Go to the extracted folder and run the following command to install the node. After you run the script, the installation files are copied to the ~/nueva folder.

    ./install.sh
  3. Run the following command to go to the ~/nueva installation folder.

    cd ~/nueva
  4. Run the following command to edit the .env file. Configure the environment variables, then save the file and exit.

    vim .env

    The following list describes the environment variables to configure:

    • NAMESPACE: The identity of the current node. To obtain the node identity, see Basic configurations.

    • TOKEN: The identity authentication token for the current node. To obtain the token, see Basic configurations.

    • CENTRAL_GATEWAY: The gateway address of the central node.

    • GATEWAY_HTTP_PORT: The external HTTP access port for the gateway. The default value is 80.

    • GATEWAY_HTTPS_PORT: The external HTTPS access port for the gateway. The default value is 443.

    • GATEWAY_GRPCS_PORT: The external GRPCS access port for the gateway. The default value is 1443.

    • GATEWAY_INTERNAL_PORT: The local HTTP access port for the gateway. The default value is 1080.

  5. Run the following command to confirm that the key pair exists. The private key, named server.key, is used for identity authentication and authorization between nodes.

    ls ~/nueva/gateway/conf/
  6. Record the public key and upload it to the management console to authorize the nodes. If you use cluster mode, make sure that the RSA private key is the same for each instance. Otherwise, authorization fails. If you cannot obtain the public key, run the following command to retrieve the RSA public key.

    openssl rsa -in ~/nueva/gateway/conf/server.key -RSAPublicKey_out | base64 | tr -d "\n"