Configure scheduled synchronization tasks with ossutil-Object Storage Service(OSS)-阿里云帮助中心

Configure a scheduled task to automatically synchronize files from a local server to an Object Storage Service (OSS) bucket by using ossutil. OSS does not support real-time synchronization from a server, so a scheduled task is how you keep a bucket up to date without uploading files manually.

Choose a synchronization method

OSS does not support real-time file synchronization from a server to OSS. Files on your server are not automatically uploaded to OSS. Use one of the following methods instead:

Manual synchronization — Run ossutil sync to trigger the synchronization manually. This is a one-time manual operation and is not real-time.
Scheduled synchronization — Configure a cron job to run ossutil sync periodically. The shortest scheduling interval is 1 minute, so this method is not real-time either.
Replication between buckets — If you need near real-time data replication between buckets, see Cross-region replication overview and Same-region replication (SRR). Cross-region replication (CRR) and same-region replication (SRR) are automatic and asynchronous (near real-time), and apply only between buckets. You cannot use them to synchronize files from a server to OSS.
This topic describes the scheduled synchronization method: a cron job on Linux, or a Task Scheduler task on Windows.

Scenario

Files in a folder named folder are added, modified, and deleted every day, and you want to synchronize the folder to OSS on a schedule. To improve transfer efficiency and avoid transferring large numbers of unnecessary old files, upload only the files that were created or updated within a specific period of time.

Prerequisites

OSS is activated. For more information, see Activate OSS.
A bucket is created. For more information, see Create a bucket.
ossutil 2.0 is installed. For more information, see Install the ossutil 2.0 command line tool.

Configure the scheduled task

The upload commands in the following examples filter files on both the source side and the destination side:

Source filter parameters — Use source filter parameters, such as --max-age, to select the files within a specific period of time based on your requirements. This improves file transfer efficiency and gives you precise synchronization and backup.
Destination filter parameters — Use a destination filter parameter, such as --update/-u, to exclude files that are already synchronized. This improves synchronization efficiency and reduces redundant operations and resource consumption.

Linux

On Linux, cron runs the upload on a schedule. Put the ossutil command directly in the cron entry when a single command is enough, or put the command in a shell script and call the script from the cron entry when you expect to combine several commands.

Run the upload command from a cron job

Open a terminal and run the following command:
```
crontab -e
```
Add a scheduled task. The following entry runs at 03:00 every day and uploads the files under /path/folder that were created or updated within the last 7 days.
```
0 3 * * * ossutil cp /path/folder oss://your-bucket/path/a -r -f -u --max-age 7d
```

Run a script from a cron job

Create a shell script named upload.sh and add the following content:

#!/bin/bash
ossutil cp /path/folder oss://your-bucket/path/a -r -f -u --max-age 7d

Add a scheduled task. The following entry runs the script at 03:00 every day and uploads the files under /path/folder that were created or updated within the last 7 days.
```
0 3 * * * /path/to/upload.sh
```

Windows

Create and edit the upload script.

Create a batch file, such as upload_to_oss.bat, and add the following content:
```
cd /d "C:\ossutil"
ossutil cp "C:\local\folder" "oss://your-bucket/folder" -r -u -f
```
Configure Task Scheduler.
1. Search for and open the Task Scheduler application.
2. In the left-side pane, select Task Scheduler Library, and then click Create Basic Task.
3. Follow the wizard to set the task name, the trigger (the time and frequency at which the task runs), and the action (the program or script to run, and its parameters).
4. In Actions, select Start a program, and then browse to and select the batch file (.bat file) that you created.
Test the task.
1. Double-click the batch file (.bat file) that you created to run it.
2. Log on to the OSS console and check whether the files under the OSS path specified in the script are updated.
  After the files are updated, the task automatically runs the script at the specified time and performs the ossutil upload operation.