Fix the "failed to leapp preupgrade" error

Updated at:

Resolve the "failed to leapp preupgrade" error that occurs during Linux OS migration with SMC.

Problem description

The "failed to leapp preupgrade" error occurs during Linux OS migration with SMC.

Cause

Possible causes:

  • Invalid package dependencies.

  • Failed to access Yellowdog Updater Modified (YUM) repositories.

  • Invalid mount configurations in the /etc/fstab file.

  • Invalid FTP configurations in the /etc/vsftpd/vsftpd.conf file.

  • Invalid file or path names in the / root directory.

  • Failed to create the specified user space.

Solution

Invalid package dependencies

  1. Connect to the source server. (Connect to an instance).

  2. Check the SMC client error logs.

    Locate the log directory (View the SMC client). Sample command:

    cat /smc/go2aliyun_client2.7.6_linux_x86_64/Logs/osm_prepare.log

    Replace 2.7.6 with your actual SMC client version.

    Run the following command to view the error details:

    cat /var/log/leapp/leapp-preupgrade.log
    2024-02-21 21:29:16.543 DEBUG    PID: 31683 leapp.workflow.TargetTransactionCheck.dnf_transaction_check: Transaction check:
    2024-02-21 21:29:16.551 DEBUG    PID: 31683 leapp.workflow.TargetTransactionCheck.dnf_transaction_check:
    2024-02-21 21:29:16.559 DEBUG    PID: 31683 leapp.workflow.TargetTransactionCheck.dnf_transaction_check:  Problem: problem with installed package lynx-2.8.8-0.3.dev15.1.al7.x86_64
    2024-02-21 21:29:16.566 DEBUG    PID: 31683 leapp.workflow.TargetTransactionCheck.dnf_transaction_check:   - cannot install the best update candidate for package lynx-2.8.8-0.3.dev15.1.al7.x86_64
    2024-02-21 21:29:16.575 DEBUG    PID: 31683 leapp.workflow.TargetTransactionCheck.dnf_transaction_check:   - lynx-2.8.8-0.3.dev15.1.al7.x86_64 does not belong to a distupgrade repository
    2024-02-21 21:29:16.582 DEBUG    PID: 31683 leapp.workflow.TargetTransactionCheck.dnf_transaction_check:   - nothing provides anolis-indexhtml needed by lynx-2.8.9-4.0.1.al8.x86_64

    The dnf_transaction_check actor found invalid dependencies during package updates. A required package may be missing from the repository, or dependencies may conflict. In this example, the anolis-indexhtml package required by lynx is not found in the repository.

  3. Identify the problematic package from the error log and uninstall it. For example, uninstall lynx:

    yum remove lynx-2.8.8-0.3.dev15.1.al7.x86_64
  4. After you resolve the network issue, run the SMC client to retry the migration. For more information, see Run the SMC client.

Failed to access YUM repositories

  1. Connect to the source server. (Connect to an instance).

  2. Check the SMC client error logs.

    Locate the log directory (View the SMC client). Sample command:

    cat /smc/go2aliyun_client2.7.6_linux_x86_64/Logs/osm_prepare.log

    Replace 2.7.6 with your actual SMC client version.

    Run the following command:

    cat /var/log/leapp/leapp-preupgrade.log

    The following error message is reported:

    Errors during downloading metadata for repository 'docker-ce-stable-migrate':
     - Curl error (28): Timeout was reached for https://mirrors.aliyun.com/docker-ce/linux/centos/8/x86_64/stable/repodata/repomd.xml [Connection timed out after 30000 milliseconds]
    Error: Failed to download metadata for repo 'docker-ce-stable-migrate': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
    
    Errors during downloading metadata for repository 'zabbix-migrate':
     - Curl error (28): Timeout was reached for http://repo.zabbix.com/zabbix/5.0/rhel/8/x86_64/repodata/repomd.xml [Connection timed out after 30000 milliseconds]
    Error: Failed to download metadata for repo 'zabbix-migrate': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

    This error indicates that repositories in /etc/leapp/files/leapp_upgrade_repositories.repo are unreachable. The failing repositories include nodejs, zabbix, and docker-ce public endpoints.

    Some ECS instances cannot access these public endpoints.

  3. Fix the error based on your ECS instance network access:

    • Your ECS instance cannot access the Internet

      In /etc/leapp/files/leapp_upgrade_repositories.repo, replace the public endpoint http://mirrors.aliyun.com/ with the VPC endpoint http://mirrors.cloud.aliyuncs.com/, or set enabled=0 for Internet-only repositories.

    • Your ECS instance cannot access VPC

      The source server must access the VPC endpoint http://mirrors.cloud.aliyuncs.com/ during migration. Verify that the server is not limited to classic network and that /etc/resolv.conf is correctly configured.

  4. After you resolve the network issue, run the SMC client to retry the migration. For more information, see Run the SMC client.

Invalid mount configurations in the /etc/fstab file

  1. Connect to the source server. (Connect to an instance).

  2. Check the SMC client error logs.

    Locate the log directory (View the SMC client). Sample command:

    cat /smc/go2aliyun_client2.7.6_linux_x86_64/Logs/osm_prepare.log

    Replace 2.7.6 with your actual SMC client version.

  3. Verify the mount configurations in /etc/fstab:

    • Each file system entry must be on a single line.

    • All file systems must be mountable. Run mount -a to verify.

    • The file must not contain Chinese characters.

  4. After you resolve the network issue, run the SMC client to retry the migration. For more information, see Run the SMC client.

Invalid FTP configurations in the /etc/vsftpd/vsftpd.conf file

  1. Connect to the source server. (Connect to an instance).

  2. Check the SMC client error logs.

    Locate the log directory (View the SMC client). Sample command:

    cat /smc/go2aliyun_client2.7.6_linux_x86_64/Logs/osm_prepare.log

    Replace 2.7.6 with your actual SMC client version.

    This error indicates the vsftpd_config_read actor failed. Check /etc/vsftpd/vsftpd.conf for issues.

    # When "listen" directive is enabled, vsftpd runs in standalone mode and
    # listens on IPv4 sockets. This directive cannot be used in conjunction
    # with the listen_ipv6 directive.
    listen=NO
    #
    # This directive enables listening on IPv6 sockets. By default, listening
    # on the IPv6 "any" address (::) will accept connections from both IPv6
    # and IPv4 clients. It is not necessary to listen on *both* IPv4 and IPv6
    # sockets. If you want that (perhaps because you want to listen on specific
    # addresses) then you must run two copies of vsftpd with two configuration
    # files.
    # Make sure, that one of the listen options is commented !!
    listen_ipv6=YES
    
    pam_service_name=vsftpd
    userlist_enable=YES
    tcp_wrappers=YES
    
    chroot_local_user=YES
    allow_writeable_chroot=YES
    
    guest_enable=YES
    guest_username=www
    virtual_use_local_privs=YES
    user_config_dir=/etc/vsftpd/vconf
    
    # Passive mode
    pasv_enable=YES
    pasv_min_port=30000
    pasv_max_port=30050
    
    ssl_enable=YES
    allow_anon_ssl=YES
    force_anon_data_ssl=YES
    force_anon_logins_ssl=YES
    force_local_data_ssl=YES
    force_local_logins_ssl=YES
    ssl_tlsv1=YES
    rsa_cert_file=/etc/vsftpd/vsftpd.pem
    
    listen_port=21
  3. Remove Chinese characters from /etc/vsftpd/vsftpd.conf. Chinese characters are not supported in this file.

  4. After you resolve the network issue, run the SMC client to retry the migration. For more information, see Run the SMC client.

Invalid file or path names in the / root directory

  1. Connect to the source server. (Connect to an instance).

  2. Check the SMC client error logs.

    Locate the log directory (View the SMC client). Sample command:

    cat /smc/go2aliyun_client2.7.6_linux_x86_64/Logs/osm_prepare.log

    Replace 2.7.6 with your actual SMC client version.

    This error indicates the root_scanner actor failed. Check the / root directory for invalid characters.

    [root@pro-3 /]# ll
    total 9488
    -rw-r--r--    1 root root 9046177 Mar  18 2022 ~
    lrwxrwxrwx.   1 root root       7 Jul  11 2019 bin -> usr/bin
    dr-xr-xr-x.   5 root root    4096 May   9 2023 boot
    -rw-r--r--    1 root root  571772 May  12 2022 check2.txt
    drwxr-xr-x    4 root root    4096 Sep  14 2022 data
    drwxr-xr-x   19 root root    2960 Jan  18 19:08 dev
    drwxr-xr-x.  92 root root   12288 Jan  18 19:55 etc
    drwxr-xr-x    3 root root    4096 Feb  25 2023 hbr
    drwxr-xr-x.   6 root root    4096 Mar  11 2022 home
    lrwxrwxrwx.   1 root root       7 Jul  11 2019 lib -> usr/lib
    lrwxrwxrwx.   1 root root       9 Jul  11 2019 lib64 -> usr/lib64
    -rwxr-xr-x    1 root root    3956 Feb  21 2022 linux_check.sh
    drwx------.   2 root root   16384 Feb  21 2022 lost+found
    drwxr-xr-x.   2 root root    4096 Apr  11 2018 media
    drwxr-xr-x    3 root root    4096 Aug  21 2020 mnt
    drwxr-xr-x.   3 root root    4096 Jun  23 2022 opt
    dr-xr-xr-x  167 root root       0 Aug   9 08:39 proc
    drwxr-xr-x   31 root root    1120 Jan  18 19:56 run
    lrwxrwxrwx.   1 root root       8 Jul  11 2019 sbin -> usr/sbin
    drwxr-xr-x    3 root root    4096 Jan  18 19:08 smc
    drwxr-xr-x.   2 root root    4096 Apr  11 2018 srv
    dr-xr-xr-x   13 root root       0 Aug   9 08:59 sys
    drwxr-xr-x    4 root root    4096 Jan  19 10:13 test
    drwxrwxrwt.  13 root root   12288 Jan  19 10:04 tmp
    -rw-r--r--    1 root root       0 Oct  30 2022 UQ?J?
    drwxr-xr-x.  13 root root    4096 Jul  11 2019 usr
    drwxr-xr-x.  19 root root    4096 Jul  11 2019 var
    [root@pro-3 /]#
    [root@pro-3 /]#
    [root@pro-3 /]#
  3. Remove unparseable UTF-8 special characters from file and path names in the / root directory.

  4. After you resolve the network issue, run the SMC client to retry the migration. For more information, see Run the SMC client.

Failed to create the specified user space

  1. Connect to the source server. (Connect to an instance).

  2. Check the SMC client error logs.

    Locate the log directory (View the SMC client). Sample command:

    cat /smc/go2aliyun_client2.7.6_linux_x86_64/Logs/osm_prepare.log

    Replace 2.7.6 with your actual SMC client version.

    Run the following command to view the error details:

    cat /var/log/leapp/leapp-preupgrade.log
    2024-02-26 19:15:22.357 INFO     PID: 22700 leapp.workflow.TargetTransactionFactsCollection: Executing actor target_userspace_creator
    2024-02-26 19:15:22.454 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Creating source overlay in /var/lib/leapp/scratch with mounts in /var/lib/leapp/scratch/mounts
    2024-02-26 19:15:22.459 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Creating mount directories.
    2024-02-26 19:15:22.463 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Done creating mount directories.
    2024-02-26 19:15:22.463 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has started: ['rm', '-rf', '/var/lib/leapp/scratch/mounts/root_/upper']
    2024-02-26 19:15:22.481 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has finished: ['rm', '-rf', '/var/lib/leapp/scratch/mounts/root_/upper']
    2024-02-26 19:15:22.485 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has started: ['rm', '-rf', '/var/lib/leapp/scratch/mounts/root_/work']
    2024-02-26 19:15:22.499 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has finished: ['rm', '-rf', '/var/lib/leapp/scratch/mounts/root_/work']
    2024-02-26 19:15:22.502 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has started: ['rm', '-rf', '/var/lib/leapp/scratch/mounts/root_/system_overlay']
    2024-02-26 19:15:22.516 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has finished: ['rm', '-rf', '/var/lib/leapp/scratch/mounts/root_/system_overlay']
    2024-02-26 19:15:22.520 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has started: ['mount', '-t', 'overlay', 'overlay2', '-o', 'lowerdir=/,upperdir=/var/lib/leapp/scratch/mounts/root_/upper,workdir=/var/lib/leapp/scratch/mounts/root_/work', '/var/lib/leapp/scratch/mounts/root_/upper,workdir=/var/lib/leapp/scratch/mounts/root_upper,workdir=/var/lib/leapp/scratch/mounts/root_work', '/var/lib/leapp/scratch/mounts/root_/system_overlay']
    2024-02-26 19:15:22.532 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: mount: unknown filesystem type 'overlay'
    2024-02-26 19:15:22.536 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Command ['mount', '-t', 'overlay', 'overlay2', '-o', 'lowerdir=/,upperdir=/var/lib/leapp/scratch/mounts/root_/upper,workdir=/var/lib/leapp/scratch/mounts/root_/work', '/var/lib/leapp/scratch/mounts/root_/system_overlay'] failed with exit code 32.
    2024-02-26 19:15:22.547 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: External command has finished: ['mount', '-t', 'overlay', 'overlay2', '-o', 'lowerdir=/,upperdir=/var/lib/leapp/scratch/mounts/root_/upper,workdir=/var/lib/leapp/scratch/mounts/root_/work', '/var/lib/leapp/scratch/mounts/root_/system_overlay']
    2024-02-26 19:15:22.553 WARNING  PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Mounting /var/lib/leapp/scratch/mounts/root_/system_overlay failed with: Command ['mount', '-t', 'overlay', 'overlay2', '-o', 'lowerdir=/,upperdir=/var/lib/leapp/scratch/mounts/root_/upper,workdir=/var/lib/leapp/scratch/mounts/root_upper,workdir=/var/lib/leapp/scratch/mounts/root_work', '/var/lib/leapp/scratch/mounts/root_/system_overlay'] failed with exit code 32.
    Traceback (most recent call last):
      File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/libraries/mounting.py", line 328, in _create
        run(['mount'] + self._mount_options() + [self.target], split=False)
      File "/usr/lib/python2.7/site-packages/leapp/libraries/stdlib/__init__.py", line 188, in run
        result=result,
    CalledProcessError: Command ['mount', '-t', 'overlay', 'overlay2', '-o', 'lowerdir=/,upperdir=/var/lib/leapp/scratch/mounts/root_/upper,workdir=/var/lib/leapp/scratch/mounts/root_/work', '/var/lib/leapp/scratch/mounts/root_/system_overlay'] failed with exit code 32.
    2024-02-26 19:15:22.557 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Cleaning up mounts
    2024-02-26 19:15:22.562 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Recursively removing scratch directory /var/lib/leapp/scratch.
    2024-02-26 19:15:22.566 DEBUG    PID: 29331 leapp.workflow.TargetTransactionFactsCollection.target_userspace_creator: Recursively removed scratch directory /var/lib/leapp/scratch/
    2024-02-26 19:15:22.576 INFO     PID: 22700 leapp: Answerfile will be created at /var/log/leapp/answerfile

    This error indicates a mount failure caused by an outdated system kernel and software packages. The overlay driver is not loaded.

  3. Update the system to CentOS 7.9 to get the latest kernel and packages:

    yum update
  4. Restart the source server and verify that the overlay driver is loaded:

    lsmod |grep overlay
  5. After you resolve the network issue, run the SMC client to retry the migration. For more information, see Run the SMC client.