Install cloud-init

For more information about cloud-init, see the cloud-init official documentation.

Step 1: Check whether you need to install or upgrade cloud-init

1. Log on to the source server.

2. Run the following commands to check whether cloud-init is installed.

   CentOS series

   rpm -qa | grep -i cloud-init 
   pip list | grep -i cloud-init

   Ubuntu series

   dpkg -l | grep -i cloud-init 
   pip list | grep -i cloud-init

   • If no output is returned or the version is earlier than 0.7.9: You must Step 2: Install cloud-init.

     Important

     Version 0.7.9 is an early community edition of cloud-init and cannot be used to initialize ECS instances. You must upgrade to a later version.

   • If the version is 18 or later: You do not need to install cloud-init. However, cloud-init automatically configures the network during instance initialization. If the default configuration does not meet your requirements, you can disable the network initialization feature. For more information, see Disable automatic network initialization by Alibaba Cloud.

   • If the version is 19.1.21: We recommend that you upgrade to Alibaba Cloud cloud-init 23.2.2. Compared with Alibaba Cloud cloud-init 19.1.21, Alibaba Cloud cloud-init 23.2.2 provides the following improvements and features:

     • If the cloud-init version of an instance is 23.2.2, the instance supports access to metadata in security hardening mode. For more information about instance metadata, see Instance metadata.

     • Other feature enhancements, performance improvements, bug fixes, and community contributions. For more information, see the ChangeLog file.

   • Other cases: You do not need to install cloud-init. You can directly perform the operations described in What to do next.

3. To prevent data loss caused by accidental operations during the installation of cloud-init, we recommend that you back up the data of the source server first, for example, by creating snapshots.

Step 2: Install cloud-init

• Alibaba Cloud cloud-init 23.2.2: recommended. This version depends on Python 3.6 or later and uses the Aliyun data source.

• Alibaba Cloud cloud-init 19.1.21: This version depends on Python 3.6 or later and uses the Aliyun data source.

• Alibaba Cloud cloud-init 0.7.6a18: This version is required for specific early operating systems, such as CentOS 6, Debian 9, and SUSE Linux Enterprise Server 12. This version depends on Python 2.7 and uses the Aliyun data source.

  Important

  The Python community has stopped technical support for Python 2.7. We recommend that you use a later version of cloud-init whenever possible to avoid potential issues with dependency libraries.

• Community edition cloud-init: The community edition of cloud-init is maintained by the community. The latest version of Alibaba Cloud cloud-init is 23.2.2. If you need a later version of cloud-init, you can install the community edition. For more information about versions, see the cloud-init official website.

The community edition of cloud-init is the official version of the cloud-init project, whereas Alibaba Cloud cloud-init is optimized for the Alibaba Cloud platform and provides better support for Alibaba Cloud platform services. Therefore, we recommend that you use Alibaba Cloud cloud-init. Select a cloud-init edition to install based on your actual needs.

# Check and install python3-pip
if ! python3 -c 'import setuptools' >& /dev/null; then
  yum -y install python3-pip
fi
# Back up the earlier version of cloud-init
test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
# Download and decompress Alibaba Cloud cloud-init
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install cloud-init
issue_major=$( cat /etc/redhat-release | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh centos "$issue_major"

# Check and install python3-pip
if ! python3 -c 'import setuptools' >& /dev/null; then
  yum -y install python3-pip
fi
# Back up the earlier version of cloud-init
test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
# Download and decompress Alibaba Cloud cloud-init
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install cloud-init
issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh rhel "$issue_major"

# Check and install python3-pip
if ! python3 -c 'import setuptools' >& /dev/null; then
  apt-get install python36 python3-pip -y
fi
# Back up the earlier version of cloud-init
test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
# Download and decompress Alibaba Cloud cloud-init
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install cloud-init
issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh ubuntu "$issue_major"

# Check and install python3-pip
if ! python3 -c 'import setuptools' >& /dev/null; then
  apt-get -y install python3-pip
fi
# Back up the earlier version of cloud-init
test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
# Download and decompress Alibaba Cloud cloud-init
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install cloud-init
issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh debian "$issue_major"

# Check and install python3-pip
if ! python3 -c 'import setuptools'>& /dev/null; then
  zypper -n install python3-pip
fi
# Back up the earlier version of cloud-init
test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
# Download and decompress Alibaba Cloud cloud-init
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install cloud-init
issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh sles "$issue_major"

# Check and install python3-pip
if ! python3 -c 'import setuptools'>& /dev/null; then
  zypper -n install python3-pip
fi
# Back up the earlier version of cloud-init
test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
# Download and decompress Alibaba Cloud cloud-init
wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.21.tgz
tar -zxvf ./cloud-init-19.1.21.tgz
# Install cloud-init
issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
bash ./cloud-init-*/tools/deploy.sh opensuse "$issue_major"

(Optional) Step 3: Configure cloud-init

FAQ

• What do I do if dependency libraries are missing when I install cloud-init by using Python 3?

  Different images may lack different libraries. You can install the missing libraries by using pip and then install cloud-init again.

• What do I do if cloud-init runs abnormally and an error message is reported?

  If the default package manager of the system (such as yum) and the pip manager have installed different versions of dependency libraries, library version conflicts may occur and cause cloud-init to run abnormally. We recommend that you download the required dependency libraries based on the error messages.

  Error message	Cause	Troubleshooting command
  no setuptools module in python	The python setuptools module is missing	Take Python 3.6 as an example: CentOS/Red Hat: yum -y install python3-pip Ubuntu/Debian: apt-get -y install python3-pip OpenSUSE/SUSE: zypper -n install python3-pip
  File "/home/testuser/cloud-init/cloudinit/log.py", line 19, in <module> import six ImportError: No module named six )	The six dependency library is missing	pip3 install six
  File "/home/testuser/cloud-init/cloudinit/url_helper.py", line 20, in <module> import oauthlib.oauth1 as oauth1 ImportError: No module named oauthlib.oauth1 )	The oauthlib dependency library is missing	pip3 install oauthlib
  The error message does not specify the missing dependency library	The error message is not mapped	Based on the libraries listed in the requirements.txt file of cloud-init, run the following command to install all dependency libraries: pip3 install -r requirements.txt

• What do I do if a new version of Python 3 installed on the instance causes cloud-init to run abnormally?

  If you install a new version of Python 3 (such as Python 3.9) on the instance and set the new version as the default version (that is, you have created a symbolic link for the new Python 3 version, for example, ln -s /usr/bin/python3.9 /usr/bin/python3), the installed cloud-init runs abnormally. For example, running cloud-init --version reports the following error:

  $cloud-init --version
  Traceback (most recent call last):
    File "/usr/local/bin/cloud-init", line 33, in <module>
      sys.exit(load_entry_point('cloud-init==19.1.9', 'console_scripts', 'cloud-init')())
    File "/usr/local/bin/cloud-init", line 22, in importlib_load_entry_point
      for entry_point in distribution(dist_name).entry_points
    File "/usr/lib64/python3.9/importlib/metadata.py", line 524, in distribution
      return Distribution.from_name(distribution_name)
    File "/usr/lib64/python3.9/importlib/metadata.py", line 187, in from_name
      raise PackageNotFoundError(name)
  importlib.metadata.PackageNotFoundError: cloud-init

  You can use either of the following methods to resolve this issue:

  • Method 1: Reinstall cloud-init by using the new version of Python 3.

  • Method 2: Modify the cloud-init executable file to change the path of the Python interpreter to the previous version of Python 3. Take Python 3.6 as an example. Perform the following operations to modify the interpreter in the cloud-init executable file:

    1. Run the following command to open the cloud-init file.

       vim   /usr/local/bin/cloud-init

    2. Press i to enter edit mode, and change the content after #! at the beginning of the file to the path of Python 3.6.

       After the modification, the #! line is as follows:

       #!/usr/bin/python3.6

    3. Press the Esc key to exit edit mode, enter :wq, and then press Enter to save and exit the file.

• What do I do if the user data specified in the instance metadata during instance creation is not run or fails to run after cloud-init is installed?

  1. Run the following command inside the instance to check whether user-data exists in the instance metadata.

     curl http://100.100.100.200/latest/user-data

     Check result description:

     • If user-data is set, the corresponding user-data information is returned. Proceed to the next step for troubleshooting.

     • If no information is returned, user-data is not set.

  2. Use the following methods to troubleshoot why user-data is not run or fails to run.

     • Check whether the user-data format is correct.

       The user-data is run by cloud-init. You must ensure that the data format is correct. For example, the first line of user-data must start with #!. For more information, see Customize instance initialization configurations.

     • Check the user-data execution results in the cloud-init log file /var/log/cloud-init.log and troubleshoot the issue based on the error messages.

       The following example shows the information recorded in the log file:

        util.py[DEBUG]: Running command ['/var/lib/cloud/instance/scripts/part-001'] with allowed return codes [0] (shell=False, capture=False)

     • View the standard output and standard error output of the user-data execution process.

       In systemd, the Linux system initialization tool, user-data is run by the cloud-final.service system service. You can run the following command to view the standard output and standard error output of the user-data execution process and troubleshoot the issue based on the output.

       journalctl -u cloud-final.service

     • If none of the preceding methods can identify why user-data is not run or fails to run, we recommend that you copy the user-data content to an on-premises host for debugging to check whether it can run successfully.

• What do I do if the system initialization configurations and instance user data scripts cannot run as expected after cloud-init is installed?

  1. Run the following commands to check whether the four cloud-init services are set to start at system startup.

     systemctl is-enabled cloud-init-local.service
     systemctl is-enabled cloud-init.service
     systemctl is-enabled cloud-config.service
     systemctl is-enabled cloud-final.service

     If an error message or disabled is returned, cloud-init is not set to start at system startup.

  2. Run the following commands to set cloud-init to start at system startup.

     systemctl enable cloud-init-local.service
     systemctl enable cloud-init.service
     systemctl enable cloud-config.service
     systemctl enable cloud-final.service

• What do I do if the root partition is not automatically expanded inside the instance after I expand the system disk during instance creation?

  Automatic partition expansion for Linux instances is supported by cloud-init. Make sure that the growpart tool is installed in the system and the cloud.cfg file is correctly configured.

  1. Run the following command to check whether the growpart tool is installed in the system.

     which growpart

  2. (Conditionally required) If the tool is not installed, run the following commands to install the growpart tool.

     • RHEL series:

       sudo yum -y install cloud-utils-growpart

     • Debian series:

       sudo apt -y install cloud-guest-utils

  3. Run the following command to check whether the following content exists in the /etc/cloud/cloud.cfg file.

     cat /etc/cloud/cloud.cfg

     Make sure that the frequency of growpart and resizefs is once-per-instance:

     # The modules that run in the 'init' stage
     cloud_init_modules:
      - migrator
      - source-address
      - pip-source
      - seed_random
      - bootcmd
      - write-files
      - [ growpart, once-per-instance ]
      - [ resizefs, once-per-instance ]
      - set_hostname
      - update_hostname
      - update_etc_hosts
      - rsyslog
      - users-groups
      - ssh

  4. (Conditionally required) If the content does not exist, add it to the /etc/cloud/cloud.cfg file.

• What do I do if the new hostname does not take effect after I update the hostname in the console and restart an ECS instance whose cloud-init version is 0.7.6a16 or earlier?

  • Problem description

    After you update the hostname in the ECS console and restart the ECS instance, the new hostname is not set. When you run the cloud-init --version command, the cloud-init version is 0.7.6a16 or earlier.

  • Cause

    When cloud-init updates the hostname through the update_hostname module, it runs the hostname command to obtain the current hostname of the system, reads the /var/lib/cloud/data/previous-hostname file to obtain the hostname in the cloud-init data cache, and then checks whether the two are consistent. If they are consistent, the hostname is updated. If they are inconsistent, cloud-init assumes that the hostname has been manually modified and does not update the hostname.

    In Red Hat series operating systems, the line break in the /var/lib/cloud/data/previous-hostname file is processed when the file is read, but the output of the hostname command is not processed for line breaks. As a result, the two results are inconsistent and the hostname is not updated.

  • Solution

    You can use either of the following solutions to resolve this issue:

    • Upgrade the cloud-init version. For more information, see Step 2: Install cloud-init.

      • If you use CentOS 6 or earlier, use cloud-init 0.7.6a18.

      • If you use CentOS 7, use cloud-init 19.1.21.

      • If you use CentOS 8 or later, we recommend that you use cloud-init 23.2.2 or later.

    • Manually process the line break.

      Manually modify the defective cloud-init file cloudinit/distros/rhel.py (obtain the file path by running the find / -name rhel.py command). Refer to distros/rhel.py: _read_hostname() missing strip on "hostname" to process the line break in the output of the hostname command. The following example shows how to process the line break:

      diff --git a/cloudinit/distros/rhel.py b/cloudinit/distros/rhel.py
      index 0b68414e..65176e99 100644
      --- a/cloudinit/distros/rhel.py
      +++ b/cloudinit/distros/rhel.py
      @@ -143,6 +143,7 @@ class Distro(distros.Distro):
                   return util.load_file(filename).strip()
               elif self.uses_systemd():
                   (out, _err) = util.subp(['hostname'])
      +            out = out.strip()
                   if len(out):
                       return out
                   else: