Migrate full backups to the cloud

更新时间:
复制 MD 格式

To migrate an on-premises SQL Server database to an ApsaraDB RDS for SQL Server instance, upload a full backup of the database to Alibaba Cloud Object Storage Service (OSS) and then use the ApsaraDB RDS console to restore the backup to a specified ApsaraDB RDS for SQL Server instance. This approach is ideal for data backup, migration, and disaster recovery.

Prerequisites

  • The RDS for SQL Server instance must meet the following requirements:

    • The instance must have more available storage space than the size of the data file you are migrating. If storage space is insufficient, upgrade the instance storage.

    • For an instance running SQL Server 2012 or later, or an instance running SQL Server 2008 R2 with a cloud disk, ensure it does not contain a database with the same name as the one to be migrated.

    • For an instance running SQL Server 2008 R2 with a high-performance local disk, ensure you have created a database on the instance with the same name as the one to be migrated.

  • If you log on as a RAM user, you must meet the following requirements:

    • The RAM user must have the AliyunOSSFullAccess and AliyunRDSFullAccess permissions. For more information, see Control access to OSS by using RAM and Control access to RDS by using RAM.

    • Ensure that your Alibaba Cloud account has granted the official RDS service account access to your OSS resources.

      Authorization method

      1. Go to the Restoration page of the RDS for SQL Server instance and click Restore Backup Data from OSS.

      2. In the Import Guide, click Next twice to reach step 3. Import Data.

        If the message You have authorized RDS official service account to access your OSS appears in the lower-left corner of the page, authorization has been granted. Otherwise, click the Authorization URL on the page to grant it.

        image

    • You must manually create an access policy in your Alibaba Cloud account and then attach it to the RAM user.

      Policy content

      {
          "Version": "1",
          "Statement": [
              {
                  "Action": [
                      "ram:GetRole"
                  ],
                  "Resource": "acs:ram:*:*:role/AliyunRDSImportRole",
                  "Effect": "Allow"
              }
          ]
      }

Notes

  • Migration scope: This solution supports only database-level migration. If you need to migrate multiple or all databases, use the instance-level migration solution.

  • Version compatibility: You cannot migrate data from a backup file of a local SQL Server to an ApsaraDB RDS for SQL Server instance that runs an earlier version of SQL Server.

  • Permission management: When you grant the ApsaraDB RDS service account permissions to access OSS, RAM creates a role named AliyunRDSImportRole in the Role Management section of the RAM console. Do not modify or delete this role. Otherwise, the migration task fails. If you accidentally modify or delete the role, you must re-grant the permissions by using the data migration wizard.

  • Account management: After the migration is complete, the source database accounts are disabled. You must create new database accounts in the ApsaraDB RDS console.

  • OSS file retention: Do not delete the backup file from OSS before the migration task is complete. Otherwise, the task fails.

  • Backup file requirements:

    • File name restrictions: The file name must not contain special characters, such as !@#$%^&*()_+-=. Otherwise, the migration fails.

    • File name extension: The backup file must have the .bak extension. For this migration method, other extensions, such as .diff (differential backup), .trn, and .log (log backup), are not supported.

    • File type: Only full backup files can be uploaded. Differential backups and log backups are not supported.

    • File source: If your source backup is a .zip file downloaded from ApsaraDB RDS for SQL Server, decompress it to a .bak file before starting the migration.

Billing

Only OSS fees apply to this solution.

image

Scenario

Description

Uploading local backup files to OSS

No charge.

Storing backup files in OSS

OSS storage fees apply. For pricing details, see OSS Pricing.

Migrating backup files from OSS to RDS

  • Migration over an internal network is free of charge.

  • Migration over the internet incurs charges for outbound traffic from OSS. For pricing details, see OSS Pricing.

Prerequisites

Run the DBCC CHECKDB statement to verify that your database is free of allocation errors and consistency errors. The following output indicates a successful run:

...
CHECKDB found 0 allocation errors and 0 consistency errors in database 'xxx'.
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

Step 1: Back up a local database

Select a method based on the version of your Alibaba Cloud RDS for SQL Server instance.

Sql server 2012+ or 2008 R2 (cloud disk)

Note

Before you create a full backup of the local database, stop writing data to it. Data written during the backup process is not included in the backup.

  1. Run the following command to check the current recovery model of the source database.

    SELECT
        name AS DatabaseName,
        recovery_model_desc AS RecoveryModel,
        state_desc AS State
    FROM sys.databases
    ORDER BY name;

    Before you back up your database for migration to Alibaba Cloud RDS, change the recovery model to FULL:

    ALTER DATABASE [db_simple] SET RECOVERY FULL WITH NO_WAIT;
  2. Download the backup script and open it in Microsoft SQL Server Management Studio (SSMS).

  3. Modify the parameters in the SELECT statement of the script, located under YOU HAVE TO INIT PUBLIC VARIABLES HERE.

    Parameter

    Description

    @backup_databases_list

    The databases to back up. Separate multiple databases with semicolons (;) or commas (,).

    @backup_type

    The backup type. Valid values:

    • FULL: full backup.

    • DIFF: differential backup.

    • LOG: log backup.

    @backup_folder

    The local directory where the backup file is stored. If the directory does not exist, the script automatically creates it.

    @is_run

    Specifies whether to run the backup. Valid values:

    • 1: Runs the backup.

    • 0: Checks the script without running the backup.

  4. Run the backup script.

Sql server 2008 R2 (local disk)

  1. Open the Microsoft SQL Server Management Studio (SSMS) client.

  2. Log on to the source database.

  3. Run the following command to check the current recovery model of the source database.

    USE master;
    GO
    SELECT name, CASE recovery_model
    WHEN 1 THEN 'FULL'
    WHEN 2 THEN 'BULK_LOGGED'
    WHEN 3 THEN 'SIMPLE' END model FROM sys.databases
    WHERE name NOT IN ('master','tempdb','model','msdb');
    GO
    • If the value of model in the result is not FULL, proceed to step 4.

    • If the value of model in the result is FULL, proceed to step 5.

  4. Run the following command to set the recovery model of the source database to FULL.

    ALTER DATABASE [dbname] SET RECOVERY FULL;
    GO
    ALTER DATABASE [dbname] SET AUTO_CLOSE OFF;
    GO
    Important

    Setting the recovery model to FULL increases the amount of log information for SQL Server. Ensure that you have sufficient disk space.

  5. Run the following command to back up the source database.

    The following example backs up the dbtest database to the backup.bak file.

    USE master;
    GO
    BACKUP DATABASE [dbtest] to disk ='d:\backup\backup.bak' WITH COMPRESSION,INIT;
    GO
  6. Run the following command to verify the integrity of the backup file.

    USE master
     GO
     RESTORE FILELISTONLY 
       FROM DISK = N'D:\backup\backup.bak';
    Important
    • If a result set is returned, the backup file is valid.

    • If an error occurs, run the backup again.

  7. Optional: Run the following command to restore the original recovery model of the database.

    Important

    If the database recovery model is already FULL, you do not need to perform this step.

    ALTER DATABASE [dbname] SET RECOVERY SIMPLE;
    GO

2. Upload the backup file to OSS

Select a method based on the version of your ApsaraDB RDS for SQL Server instance.

SQL Server 2012 or later and 2008 R2 with cloud disk

  1. To upload the backup file to OSS, you must first create a bucket.

    • If you use an existing OSS bucket, ensure it meets the following requirements:

      • The storage class of the bucket is Standard. Other storage classes, such as Infrequent Access, Archive, Cold Archive, and Deep Cold Archive, are not supported.

      • server-side encryption is disabled for the bucket.

    • If no bucket exists in OSS, you must create one. (Make sure that you have activated OSS.)

      1. Log on to the OSS console, click Buckets, and then click Create bucket.

      2. Configure the following key parameters. You can retain the default values for other parameters.

        Important
        • This bucket is created only for this migration, so you only need to configure the key parameters. You can delete the bucket after the migration is complete to prevent data breaches and reduce costs.

        • When you create the bucket, do not enable server-side encryption.

        Parameter

        Description

        Example

        Bucket Name

        The name of the bucket. The name must be globally unique and cannot be changed after the bucket is created.

        Naming conventions:

        • The name can contain only lowercase letters, digits, and hyphens (-).

        • The name must start and end with a lowercase letter or a digit.

        • The name must be 3 to 63 characters in length.

        migratetest

        Region

        The region where the bucket is located. For internal network transfers, the bucket, ECS instance, and ApsaraDB RDS instance must all be in the same region.

        China (Hangzhou)

        Storage Type

        Select Standard. This migration supports only the Standard storage class.

        Standard

  2. Upload the backup file to OSS.

    After you back up your on-premises database, upload the backup file to an OSS bucket that is in the same region as your ApsaraDB RDS instance. If the bucket and the instance are in the same region, they can communicate over the internal network. This allows for faster uploads and avoids outbound internet traffic fees. You can use one of the following methods:

    Use ossbrowser to upload the file (recommended)

    1. Download ossbrowser.

    2. For example, on Windows x64, decompress the downloaded oss-browser-win32-x64.zip package and double-click the oss-browser.exe application.

    3. Select AK, configure the Access Key ID and Access Key Secret parameters, retain the default values for the other parameters, and then click Log On.

      Note

      An AccessKey pair verifies your identity. Keep your AccessKey pair confidential to ensure data security.

      登录ossbrowser

    4. Click the destination bucket.进入bucket中

    5. Click 上传图标, select the backup file that you want to upload, and then click Open.

    Use the OSS console to upload the file

    Note

    If your backup file is smaller than 5 GB, you can upload it directly from the OSS console.

    1. Log on to the OSS console.

    2. Click Buckets and then click the name of the destination bucket.网页进入bucket

    3. On the Files tab, click Upload File.网页上传文件

    4. You can drag a backup file to the Files to Upload area, or click Select Files to choose a file.网页扫描文件

    5. Click Upload File at the bottom of the page to upload the file to OSS.

    Use the OSS API for multipart upload (Python 3 project example)

    Note

    If your backup file is larger than 5 GB, we recommend using the OSS API to upload the file to an OSS bucket by using multipart upload.

    # -*- coding: utf-8 -*-
    """
    Alibaba Cloud OSS Python SDK v2
    Dependency: pip install alibabacloud-oss-v2
    """
    
    import os
    import sys
    from pathlib import Path
    import alibabacloud_oss_v2 as oss
    from alibabacloud_oss_v2 import exceptions as oss_ex
    
    
    def get_client_from_env(region: str, endpoint: str | None = None) -> oss.Client:
        """
        Create a v2 client from environment variables.
        - Prioritizes Region (recommended) and supports custom Endpoints (optional).
        - Compatible with both AccessKey (AK) and Security Token Service (STS) credentials:
            * AK: Requires OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET.
            * STS: Requires OSS_SESSION_TOKEN (compatible with the legacy OSS_SECURITY_TOKEN variable).
        """
        # For compatibility, map the legacy OSS_SECURITY_TOKEN variable to the v2-expected OSS_SESSION_TOKEN.
        sec_token_legacy = os.getenv("OSS_SECURITY_TOKEN")
        if sec_token_legacy and not os.getenv("OSS_SESSION_TOKEN"):
            os.environ["OSS_SESSION_TOKEN"] = sec_token_legacy
    
        ak = os.getenv("OSS_ACCESS_KEY_ID")
        sk = os.getenv("OSS_ACCESS_KEY_SECRET")
        st = os.getenv("OSS_SESSION_TOKEN")  # STS Token (optional)
    
        if not (ak and sk):
            raise ValueError("No valid AK found. Set the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables. "
                             "If using STS, also set OSS_SESSION_TOKEN (or the legacy name OSS_SECURITY_TOKEN).")
    
        # Indicate the credential type in use.
        if st:
            print("STS Token (OSS_SESSION_TOKEN) detected. Using STS credentials.")
        else:
            print("No STS Token detected. Using AccessKey (AK) credentials.")
    
        credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
        cfg = oss.config.load_default()
        cfg.credentials_provider = credentials_provider
    
        # Basic network configuration.
        cfg.region = region  # Example: 'cn-hangzhou'
        if endpoint:
            # Optional: Specify a custom endpoint (e.g., for an internal network, acceleration, or a private domain).
            cfg.endpoint = endpoint
    
        # You can also add other configurations here, such as cfg.use_accelerate_endpoint = True
        return oss.Client(cfg)
    
    
    def resumable_upload_file_v2(
        client: oss.Client,
        bucket_name: str,
        object_key: str,
        file_path: str,
        part_size: int = 1 * 1024 * 1024,
        parallel_num: int = 4,
        checkpoint_dir: str | None = None,
    ):
        """
        Implements concurrent multipart upload with resumable upload capabilities.
    
        :param client: An initialized oss.Client instance.
        :param bucket_name: The name of the destination bucket.
        :param object_key: The key of the destination object (excluding the bucket name).
        :param file_path: The full path to the local file.
        :param part_size: The size of each part in bytes. Default: 1 MB.
        :param parallel_num: The number of concurrent upload threads. Default: 4.
        :param checkpoint_dir: The directory to store checkpoint information. If set to None, resumable upload is disabled.
        """
        file_path = str(file_path)
        if not Path(file_path).exists():
            raise FileNotFoundError(f"Error: Local file not found. Check the file_path configuration: {file_path}")
    
        # Construct the Uploader. Enable resumable upload if a checkpoint_dir is provided.
        if checkpoint_dir:
            uploader = client.uploader(
                enable_checkpoint=True,
                checkpoint_dir=checkpoint_dir,
                part_size=part_size,
                parallel_num=parallel_num,
            )
        else:
            uploader = client.uploader(
                part_size=part_size,
                parallel_num=parallel_num,
            )
    
        print(f"Starting to upload file: {file_path}")
        print(f"Destination Bucket: {bucket_name}")
        print(f"Destination Object: {object_key}")
        print(f"Part size: {part_size} bytes, Concurrency: {parallel_num}")
        if checkpoint_dir:
            print(f"Resumable upload: Enabled (checkpoint_dir={checkpoint_dir})")
        else:
            print("Resumable upload: Disabled (set checkpoint_dir to enable).")
    
        # Execute the upload. The Uploader automatically selects between multipart and single-part upload based on file size.
        result = uploader.upload_file(
            oss.PutObjectRequest(bucket=bucket_name, key=object_key),
            filepath=file_path,
        )
    
        print("-" * 30)
        print("File uploaded successfully!")
        print(f"HTTP Status: {result.status_code}")
        print(f"ETag: {result.etag}")
        print(f"Request ID: {result.request_id}")
        # The SDK for Python v2 enables CRC-64 data validation by default.
        print(f"CRC64: {result.hash_crc64}")
        print("-" * 30)
    
    
    def main():
        # Before running this code, make sure the required environment variables are set.
        # On macOS/Linux:
        #   For AK authentication:
        #     export OSS_ACCESS_KEY_ID=YOUR_AK_ID
        #     export OSS_ACCESS_KEY_SECRET=YOUR_AK_SECRET
        #   For STS authentication:
        #     export OSS_ACCESS_KEY_ID=YOUR_STS_ID
        #     export OSS_ACCESS_KEY_SECRET=YOUR_STS_SECRET
        #     export OSS_SECURITY_TOKEN=YOUR_STS_TOKEN
        #
        # On Windows:
        #   In PowerShell: $env:OSS_ACCESS_KEY_ID="YOUR_AK_ID"
        #   In Command Prompt: set OSS_ACCESS_KEY_ID=YOUR_AK_ID
    
        # ===================== Parameters (modify as needed) =====================
        # Example region: 'cn-hangzhou'. We recommend prioritizing the region parameter.
        region = "cn-hangzhou"
    
        # Optional: Specify a custom endpoint (e.g., for an internal network, private domain, or accelerated access).
        # Example: 'https://oss-cn-hangzhou.aliyuncs.com'
        endpoint = 'https://oss-cn-hangzhou.aliyuncs.com'  
    
        # Bucket and Object
        bucket_name = "examplebucket"
        object_key = "test.bak"
    
        # The full path to the local file to upload.
        # Windows example: r'D:\localpath\examplefile.txt' (note the 'r' prefix)
        # macOS/Linux example: '/Users/test/examplefile.txt'
        file_path = r"D:\oss\test.bak"
    
        # Part size and concurrency
        part_size = 1 * 1024 * 1024  # Default: 1 MB. OSS requires a minimum part size of 100 KB.
        parallel_num = 4
    
        # The directory for resumable uploads. Set to None to disable. We recommend specifying a writable directory.
        checkpoint_dir = str(Path.cwd() / ".oss_checkpoints")
        # =================== End of Parameters ===================
    
        print("Script execution starts...")
        try:
            client = get_client_from_env(region=region, endpoint=endpoint)
            # If resumable upload is enabled, ensure the checkpoint directory exists.
            if checkpoint_dir:
                Path(checkpoint_dir).mkdir(parents=True, exist_ok=True)
    
            resumable_upload_file_v2(
                client=client,
                bucket_name=bucket_name,
                object_key=object_key,
                file_path=file_path,
                part_size=part_size,
                parallel_num=parallel_num,
                checkpoint_dir=checkpoint_dir,
            )
        except FileNotFoundError as e:
            print(e)
        except oss_ex.ServiceError as e:
            # An error returned by the OSS server.
            print("\nAn OSS server-side error occurred.")
            print(f"HTTP Status: {getattr(e, 'status_code', 'N/A')}")
            print(f"Error Code: {getattr(e, 'code', 'N/A')}")
            print(f"Message: {getattr(e, 'message', 'N/A')}")
            print(f"Request ID: {getattr(e, 'request_id', 'N/A')}")
            print(f"Endpoint: {getattr(e, 'request_target', 'N/A')}")
        except oss_ex.BaseError as e:
            # An SDK client-side error (e.g., local, serialization, deserialization, or credential error).
            print("\nAn OSS SDK client-side error occurred.")
            print(str(e))
        except Exception as e:
            print(f"\nAn unknown error occurred: {e}")
    
    
    if __name__ == "__main__":
        main()

SQL Server 2008 R2 with local disk

  1. To upload the backup file to OSS, you must first create a bucket.

    • If you use an existing OSS bucket, ensure it meets the following requirements:

      • The storage class of the bucket is Standard. Other storage classes, such as Infrequent Access, Archive, Cold Archive, and Deep Cold Archive, are not supported.

      • server-side encryption is disabled for the bucket.

    • If no bucket exists in OSS, you must create one. (Make sure that you have activated OSS.)

      1. Log on to the OSS console, click Buckets, and then click Create bucket.

      2. Configure the following key parameters. You can retain the default values for other parameters.

        Important
        • This bucket is created only for this migration, so you only need to configure the key parameters. You can delete the bucket after the migration is complete to prevent data breaches and reduce costs.

        • When you create the bucket, do not enable server-side encryption.

        Parameter

        Description

        Example

        Bucket Name

        The name of the bucket. The name must be globally unique and cannot be changed after the bucket is created.

        Naming conventions:

        • The name can contain only lowercase letters, digits, and hyphens (-).

        • The name must start and end with a lowercase letter or a digit.

        • The name must be 3 to 63 characters in length.

        migratetest

        Region

        The region where the bucket is located. For internal network transfers, the bucket, ECS instance, and ApsaraDB RDS instance must all be in the same region.

        China (Hangzhou)

        Storage Type

        Select Standard. This migration supports only the Standard storage class.

        Standard

  2. Upload the backup file to OSS.

    After you back up your on-premises database, upload the backup file to an OSS bucket that is in the same region as your ApsaraDB RDS instance. If the bucket and the instance are in the same region, they can communicate over the internal network. This allows for faster uploads and avoids outbound internet traffic fees. You can use one of the following methods:

    Use ossbrowser to upload the file (recommended)

    1. Download ossbrowser.

    2. For example, on Windows x64, decompress the downloaded oss-browser-win32-x64.zip package and double-click the oss-browser.exe application.

    3. Select AK, configure the Access Key ID and Access Key Secret parameters, retain the default values for the other parameters, and then click Log On.

      Note

      An AccessKey pair verifies your identity. Keep your AccessKey pair confidential to ensure data security.

      登录ossbrowser

    4. Click the destination bucket.进入bucket中

    5. Click 上传图标, select the backup file that you want to upload, and then click Open.

    Use the OSS console to upload the file

    Note

    If your backup file is smaller than 5 GB, you can upload it directly from the OSS console.

    1. Log on to the OSS console.

    2. Click Buckets and then click the name of the destination bucket.网页进入bucket

    3. On the Files tab, click Upload File.网页上传文件

    4. You can drag a backup file to the Files to Upload area, or click Select Files to choose a file.网页扫描文件

    5. Click Upload File at the bottom of the page to upload the file to OSS.

    Use the OSS API for multipart upload (Python 3 project example)

    Note

    If your backup file is larger than 5 GB, we recommend using the OSS API to upload the file to an OSS bucket by using multipart upload.

    # -*- coding: utf-8 -*-
    """
    Alibaba Cloud OSS Python SDK v2
    Dependency: pip install alibabacloud-oss-v2
    """
    
    import os
    import sys
    from pathlib import Path
    import alibabacloud_oss_v2 as oss
    from alibabacloud_oss_v2 import exceptions as oss_ex
    
    
    def get_client_from_env(region: str, endpoint: str | None = None) -> oss.Client:
        """
        Create a v2 client from environment variables.
        - Prioritizes Region (recommended) and supports custom Endpoints (optional).
        - Compatible with both AccessKey (AK) and Security Token Service (STS) credentials:
            * AK: Requires OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET.
            * STS: Requires OSS_SESSION_TOKEN (compatible with the legacy OSS_SECURITY_TOKEN variable).
        """
        # For compatibility, map the legacy OSS_SECURITY_TOKEN variable to the v2-expected OSS_SESSION_TOKEN.
        sec_token_legacy = os.getenv("OSS_SECURITY_TOKEN")
        if sec_token_legacy and not os.getenv("OSS_SESSION_TOKEN"):
            os.environ["OSS_SESSION_TOKEN"] = sec_token_legacy
    
        ak = os.getenv("OSS_ACCESS_KEY_ID")
        sk = os.getenv("OSS_ACCESS_KEY_SECRET")
        st = os.getenv("OSS_SESSION_TOKEN")  # STS Token (optional)
    
        if not (ak and sk):
            raise ValueError("No valid AK found. Set the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables. "
                             "If using STS, also set OSS_SESSION_TOKEN (or the legacy name OSS_SECURITY_TOKEN).")
    
        # Indicate the credential type in use.
        if st:
            print("STS Token (OSS_SESSION_TOKEN) detected. Using STS credentials.")
        else:
            print("No STS Token detected. Using AccessKey (AK) credentials.")
    
        credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
        cfg = oss.config.load_default()
        cfg.credentials_provider = credentials_provider
    
        # Basic network configuration.
        cfg.region = region  # Example: 'cn-hangzhou'
        if endpoint:
            # Optional: Specify a custom endpoint (e.g., for an internal network, acceleration, or a private domain).
            cfg.endpoint = endpoint
    
        # You can also add other configurations here, such as cfg.use_accelerate_endpoint = True
        return oss.Client(cfg)
    
    
    def resumable_upload_file_v2(
        client: oss.Client,
        bucket_name: str,
        object_key: str,
        file_path: str,
        part_size: int = 1 * 1024 * 1024,
        parallel_num: int = 4,
        checkpoint_dir: str | None = None,
    ):
        """
        Implements concurrent multipart upload with resumable upload capabilities.
    
        :param client: An initialized oss.Client instance.
        :param bucket_name: The name of the destination bucket.
        :param object_key: The key of the destination object (excluding the bucket name).
        :param file_path: The full path to the local file.
        :param part_size: The size of each part in bytes. Default: 1 MB.
        :param parallel_num: The number of concurrent upload threads. Default: 4.
        :param checkpoint_dir: The directory to store checkpoint information. If set to None, resumable upload is disabled.
        """
        file_path = str(file_path)
        if not Path(file_path).exists():
            raise FileNotFoundError(f"Error: Local file not found. Check the file_path configuration: {file_path}")
    
        # Construct the Uploader. Enable resumable upload if a checkpoint_dir is provided.
        if checkpoint_dir:
            uploader = client.uploader(
                enable_checkpoint=True,
                checkpoint_dir=checkpoint_dir,
                part_size=part_size,
                parallel_num=parallel_num,
            )
        else:
            uploader = client.uploader(
                part_size=part_size,
                parallel_num=parallel_num,
            )
    
        print(f"Starting to upload file: {file_path}")
        print(f"Destination Bucket: {bucket_name}")
        print(f"Destination Object: {object_key}")
        print(f"Part size: {part_size} bytes, Concurrency: {parallel_num}")
        if checkpoint_dir:
            print(f"Resumable upload: Enabled (checkpoint_dir={checkpoint_dir})")
        else:
            print("Resumable upload: Disabled (set checkpoint_dir to enable).")
    
        # Execute the upload. The Uploader automatically selects between multipart and single-part upload based on file size.
        result = uploader.upload_file(
            oss.PutObjectRequest(bucket=bucket_name, key=object_key),
            filepath=file_path,
        )
    
        print("-" * 30)
        print("File uploaded successfully!")
        print(f"HTTP Status: {result.status_code}")
        print(f"ETag: {result.etag}")
        print(f"Request ID: {result.request_id}")
        # The SDK for Python v2 enables CRC-64 data validation by default.
        print(f"CRC64: {result.hash_crc64}")
        print("-" * 30)
    
    
    def main():
        # Before running this code, make sure the required environment variables are set.
        # On macOS/Linux:
        #   For AK authentication:
        #     export OSS_ACCESS_KEY_ID=YOUR_AK_ID
        #     export OSS_ACCESS_KEY_SECRET=YOUR_AK_SECRET
        #   For STS authentication:
        #     export OSS_ACCESS_KEY_ID=YOUR_STS_ID
        #     export OSS_ACCESS_KEY_SECRET=YOUR_STS_SECRET
        #     export OSS_SECURITY_TOKEN=YOUR_STS_TOKEN
        #
        # On Windows:
        #   In PowerShell: $env:OSS_ACCESS_KEY_ID="YOUR_AK_ID"
        #   In Command Prompt: set OSS_ACCESS_KEY_ID=YOUR_AK_ID
    
        # ===================== Parameters (modify as needed) =====================
        # Example region: 'cn-hangzhou'. We recommend prioritizing the region parameter.
        region = "cn-hangzhou"
    
        # Optional: Specify a custom endpoint (e.g., for an internal network, private domain, or accelerated access).
        # Example: 'https://oss-cn-hangzhou.aliyuncs.com'
        endpoint = 'https://oss-cn-hangzhou.aliyuncs.com'  
    
        # Bucket and Object
        bucket_name = "examplebucket"
        object_key = "test.bak"
    
        # The full path to the local file to upload.
        # Windows example: r'D:\localpath\examplefile.txt' (note the 'r' prefix)
        # macOS/Linux example: '/Users/test/examplefile.txt'
        file_path = r"D:\oss\test.bak"
    
        # Part size and concurrency
        part_size = 1 * 1024 * 1024  # Default: 1 MB. OSS requires a minimum part size of 100 KB.
        parallel_num = 4
    
        # The directory for resumable uploads. Set to None to disable. We recommend specifying a writable directory.
        checkpoint_dir = str(Path.cwd() / ".oss_checkpoints")
        # =================== End of Parameters ===================
    
        print("Script execution starts...")
        try:
            client = get_client_from_env(region=region, endpoint=endpoint)
            # If resumable upload is enabled, ensure the checkpoint directory exists.
            if checkpoint_dir:
                Path(checkpoint_dir).mkdir(parents=True, exist_ok=True)
    
            resumable_upload_file_v2(
                client=client,
                bucket_name=bucket_name,
                object_key=object_key,
                file_path=file_path,
                part_size=part_size,
                parallel_num=parallel_num,
                checkpoint_dir=checkpoint_dir,
            )
        except FileNotFoundError as e:
            print(e)
        except oss_ex.ServiceError as e:
            # An error returned by the OSS server.
            print("\nAn OSS server-side error occurred.")
            print(f"HTTP Status: {getattr(e, 'status_code', 'N/A')}")
            print(f"Error Code: {getattr(e, 'code', 'N/A')}")
            print(f"Message: {getattr(e, 'message', 'N/A')}")
            print(f"Request ID: {getattr(e, 'request_id', 'N/A')}")
            print(f"Endpoint: {getattr(e, 'request_target', 'N/A')}")
        except oss_ex.BaseError as e:
            # An SDK client-side error (e.g., local, serialization, deserialization, or credential error).
            print("\nAn OSS SDK client-side error occurred.")
            print(str(e))
        except Exception as e:
            print(f"\nAn unknown error occurred: {e}")
    
    
    if __name__ == "__main__":
        main()
  3. Generate a signed URL for the backup file.

    1. Log on to the OSS console.

    2. Click Buckets and then click the name of the destination bucket.

    3. In the left-side navigation pane, choose File Management > Files.

    4. In the Actions column for the backup file, click Details. In the panel that appears, set Expiration (Seconds) to 28800 (8 hours).

      Important

      The data migration will fail if this URL expires before the migration is complete.

    5. Click Copy Object URL to copy the file URL.

      image.png

    6. Modify the URL of the backup file.

      By default, the console generates a URL with a public endpoint. To migrate data over the internal network, you must replace the public endpoint in the URL with an internal endpoint.

      For example, if the URL of the backup file is http://rdstest.oss-cn-shanghai.aliyuncs.com/testmigraterds_20170906143807_FULL.bak?Expires=15141****&OSSAccessKeyId=TMP****, you must change oss-cn-shanghai.aliyuncs.com in the URL to oss-cn-shanghai-internal.aliyuncs.com.

      Important

      The internal endpoint varies based on the network type and region. For more information, see Endpoints.

3. Import OSS backup data to RDS

Select a method based on the version of your ApsaraDB RDS for SQL Server instance.

SQL Server 2012+ and 2008 R2 (cloud disk)

  1. Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.

  2. In the left-side navigation pane, click Restoration.

  3. At the top of the page, click Restore Backup Data from OSS.

  4. On the Import Guide page, click Next twice to proceed to the data import step.

    Note
    • The first time you use the Migrate OSS Backup Data to RDS feature, you must authorize ApsaraDB RDS to access OSS. Click the Authorization URL and grant the required permissions. Otherwise, the OSS Bucket drop-down list will be empty due to insufficient permissions.

    • If you cannot find the target file on this page, check whether the file name extension of the backup file in OSS meets the requirements. For more information, see the Precautions section of this topic. Also, make sure that the ApsaraDB RDS instance and the OSS Bucket are in the same region.

  5. Configure the following parameters.

    Parameter

    Description

    Database Name

    The name of the database on the destination RDS instance. The name must comply with SQL Server naming conventions.

    Important
    • Before starting the migration, ensure that the destination instance does not have a database or an unattached database file with the same name as the database you are restoring. If neither exists, you can restore the database using the file with the matching name from the backup set.

    • The cloud migration task will fail if a database or an unattached database file with the same name already exists on the destination instance.

    OSS Bucket

    The OSS Bucket where the backup file is stored.

    OSS File

    Click the 放大镜 button on the right to perform a fuzzy search by the prefix of the backup file name. The file name, file size, and update time are displayed. Select the backup file that you want to migrate.

    Cloud Migration Method

    • Immediate Access (Only One Full Backup): Performs a full migration. This option is suitable for scenarios where you migrate to the cloud by using only one full backup file. When you select Immediate Access (Only One Full Backup), the CreateMigrateTask operation is configured with BackupMode = FULL and IsOnlineDB = True.

    • Access Pending (Incremental Backup or Log Files): Performs an incremental migration. Use this option for migrating a full backup file along with subsequent log or differential backups. When you select this option, the CreateMigrateTask operation is called with BackupMode = UPDF and IsOnlineDB = False.

    Consistency Check Mode

    • Asynchronous DBCC execution: The system does not run a DBCC CheckDB operation when the database is opened. Instead, the operation is run asynchronously after the database opening task is complete. This reduces the time required to open the database and minimizes your business downtime, especially for large databases where DBCC CheckDB operations can be time-consuming. If minimizing business downtime is a high priority and you are not concerned about the immediate results of the DBCC CheckDB operation, we recommend that you use asynchronous DBCC execution. In this case, set CheckDBMode = AsyncExecuteDBCheck in CreateMigrateTask.

    • Synchronous DBCC: Select this option to get immediate DBCC CheckDB results, which is useful for verifying data consistency after migration. This method increases the time required to open the database. In this case, the CreateMigrateTask operation uses CheckDBMode = SyncExecuteDBCheck.

  6. Click OK.

    Wait for the migration task to complete. You can click Refresh to view the latest status of the task. If the migration task fails, troubleshoot the error based on the task description. For more information, see the Common errors section of this topic.

    Note

    After the data is migrated, the system initiates a backup based on the automatic backup policy of the ApsaraDB RDS instance at the specified backup time. You can manually adjust the backup time. The generated backup set contains the migrated data. You can view the backup set on the Restoration page of the instance.

    To create a backup immediately instead of waiting for the scheduled time, you can perform a manual backup.

SQL Server 2008 R2 (high-performance local disk)

  1. Log on to the ApsaraDB RDS console and go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.

  2. In the left-side navigation pane, click Databases.

  3. Find the target database and click Migrate Backup Files from OSS in the Actions column.

  4. In the Import Guide dialog box, read the prompts and click Next.

  5. Read the prompts about uploading files to OSS and click Next.

  6. In the OSS URL of the Backup File field, enter the URL of the backup file in OSS and click OK.

    image.png

    Note

    ApsaraDB RDS for SQL Server 2008 R2 instances with high-performance local disks support only a one-time migration from a single full backup file.

4. Backup migration progress

The method you use depends on the version of your ApsaraDB RDS for SQL Server instance.

SQL Server 2012 or later, or SQL Server 2008 R2 with cloud disk

In the left navigation pane of the ApsaraDB RDS instance, navigate to the Restoration page. The Cloud Migration Records of Backup Data tab shows the backup migration records, including the task status, start time, and end time. By default, the page displays records from the last week. You can adjust the time range as needed.

image

Note

If the Task Status is Failed, check the Task Description or click View File Details for the task to find the cause of the failure. After you resolve the issue, rerun the data migration.

SQL Server 2008 R2 with high-performance local disk

In the left navigation pane of the ApsaraDB RDS instance, navigate to the Data Migration To Cloud page. Then, find the task and view its progress.

Note

If the Task Status is Failed, check the Task Description or click View File Details for the task to find the cause of the failure. After you resolve the issue, rerun the data migration.

Common errors

Each data migration record includes a task description that identifies the cause of a task failure or error.

  • A database with the same name already exists

    • Error message 1: The database (xxx) already exists on RDS. Please back it up, drop it, and then try again.

    • Error message 2: Database 'xxx' already exists. Choose a different database name.

    • Cause: To ensure data security, ApsaraDB RDS for SQL Server prevents migrating a database if another database with the same name already exists.

    • Solution: To overwrite an existing database, back up the existing data, delete the database, and then run the data migration task again.

  • Using a differential backup file

    • Error message: Backup set (xxx.bak) is a database differential backup; we only accept a full backup.

    • Cause: The provided backup file is a differential backup, not a full backup. This data migration method supports only full backup files.

  • Using a log backup file

    • Error message: Backup set (xxx.trn) is a transaction log backup; we only accept a full backup.

    • Cause: The provided backup file is a log backup, not a full backup. This data migration method supports only full backup files.

  • Backup file verification failed

    • Error message: Failed to verify xxx.bak. The backup file may be corrupted or from a version newer than the destination RDS instance.

    • Cause: The backup file is corrupted, or the SQL Server version of the source database is newer than that of the destination ApsaraDB RDS for SQL Server instance. This version incompatibility causes the verification to fail. For example, this error is reported when you try to restore a backup from SQL Server 2016 to an ApsaraDB RDS for SQL Server 2012 instance.

    • Solution: If the backup file is corrupted, create a new full backup of the source database and then create a new data migration task. If the versions are incompatible, use an ApsaraDB RDS for SQL Server instance that runs a version the same as or later than that of the source database.

      Note

      To upgrade an existing ApsaraDB RDS for SQL Server instance to a later version, see Upgrade the database engine version.

  • DBCC CHECKDB failed

    • Error message: DBCC checkdb failed.

    • Cause: A failed DBCC CHECKDB operation indicates that the source database contains an error.

    • Solution: Run the following command to fix the error in the source database and then migrate the data again.

      Important

      Running this command to fix the error may cause data loss.

      DBCC CHECKDB (DBName, REPAIR_ALLOW_DATA_LOSS) WITH NO_INFOMSGS, ALL_ERRORMSGS
  • Insufficient space 1

    • Error message: Not Enough Disk Space for restoring, space left (xxx MB) < needed (xxx MB).

    • Cause: The remaining instance storage on the RDS instance is insufficient for restoring the backup file.

    • Solution: Upgrade instance storage.

  • Insufficient space 2

    • Error message: Not Enough Disk Space, space left xxx MB < bak file xxx MB.

    • Cause: The remaining instance storage on the RDS instance is smaller than the backup file.

    • Solution: Upgrade instance storage.

  • Insufficient account permissions

    • Error message: Cannot open database "xxx" requested by the login. The login failed.

    • Cause: The account used to log on to the ApsaraDB RDS instance does not have the required permissions to access the database.

    • Solution: On the Accounts page of the ApsaraDB RDS instance, grant the account permissions to access or manage the database. For more information, see Grant permissions to an account and Permissions supported by different account types.

  • No privileged account

    • Error message: Your RDS doesn’t have any init account yet, please create one and grant permissions on RDS console to this migrated database (xxx).

    • Cause: The ApsaraDB RDS instance does not have a privileged account, so the data migration task cannot determine which user to grant permissions to. Despite this, the backup file is restored successfully, and the task is marked as successful.

    • Solution: Create a privileged account.

  • Insufficient RAM user permissions

    • Q1: In Step 5 of creating a data migration task, all configuration parameters are entered, but the OK button is grayed out. Why?

    • A1: This can happen if the RAM user has insufficient permissions. See the Prerequisites section in this topic to ensure that the required permissions are granted.

    • Q2: When I use a RAM user to grant the AliyunRDSImportRole role, a permission denied error is returned. How do I solve this?

    • A2: Use your Alibaba Cloud account to temporarily grant the AliyunRAMFullAccess permission to the RAM user.

  • The database name conflicts with a SQL Server system database

    • Error message: The database (xxx) is mssql system db, change your database name and try again.

    • Cause: The specified database name conflicts with a SQL Server system database, such as master, msdb, tempdb, or model.

    • Solution: Specify a different database name.

  • The database name conflicts with an RDS system database

    • Error message: The database (xxx) is RDS system db, change your database name and try again.

    • Cause: The specified database name conflicts with a management database of the ApsaraDB RDS for SQL Server instance, such as rdscore.

    • Solution: Specify a different database name.

  • The number of databases exceeds the limit

    • Error message: The migration of database (xxx) failed because it exceeded the database count limit (xxx).

    • Cause: The number of databases on the instance exceeds the allowed limit.

    • Solution: Reduce the number of databases on the instance and try again.

  • OSS URL issues

    • Error message 1: Failed to download backup (xxx) since OSS URL was expired.

    • Error message 2: Failed to download since could not find backup file (xxx) on OSS.

    • Cause: Possible causes include an expired OSS URL, a non-existent file, or insufficient permissions for the ApsaraDB RDS service account.

    • Solution: Verify that the OSS file exists, that its access permissions are correct, and that the required authorization has been granted. Then, retry the operation.

  • Issues with striped backups (multiple files)

    • Error message: Failed to verify (xxx.bak), error message:The media set has xxx media families but only 1 are provided. All members must be provided. VERIFY DATABASE is terminating abnormally.

    • Cause: The source database was backed up using the striped backup feature, which writes a single full backup to multiple .bak files. However, only one file was provided for the data migration to ApsaraDB RDS from OSS. ApsaraDB RDS does not support data migration from multiple files simultaneously.

    • Solution: Back up the source database to a single .bak file and then retry the operation.

Common return messages

Task type

Task status

Message

Description

Full backup file import

Success

success

Migration successful.

Failed

Failed to download backup file since OSS URL was expired.

The migration failed because the OSS download URL expired.

Your backup is corrupted or newer than RDS, failed to verify.

The migration failed because the backup file is either corrupted or newer than the ApsaraDB RDS instance.

DBCC checkdb failed

The migration failed because the DBCC CHECKDB command failed.

autotest_2008r2_std_testmigrate_log.trn is a Transaction Log backup, we only accept a FULL Backup.

The migration failed because the file is a log backup.

autotest_2008r2_std_testmigrate_diff.bak is a Database Differential backup, we only accept a FULL Backup.

The migration failed because the file is a differential backup.

API reference

API

Description

CreateMigrateTask

Creates a data migration task by restoring a backup file from OSS to an ApsaraDB RDS for SQL Server instance.

CreateOnlineDatabaseTask

Brings a database online after it is restored as part of a data migration task.

DescribeMigrateTasks

Lists the data migration tasks for an ApsaraDB RDS for SQL Server instance.

DescribeOssDownloads

Returns the backup file details for a data migration task.