Use Git to manage code in an Airflow instance

Updated at:

In an Airflow instance, associate a Git account and bind a code repository to enable version management and team collaboration through Git.

Overview

DMS Airflow supports two code management approaches:

  • Approach 1: Create and edit DAG code files directly in the workspace.

  • Approach 2: Bind an existing Git repository to manage code.

This topic describes the second approach. For information about how to create and edit DAG code files directly, see Step 2: Add a linked account.

Procedure

Step 1: Add an associated account

Note

Associated accounts are independent. Even if other users work in the same workspace, they cannot view the resources under your associated account.

  1. In the upper-right corner of the workspace, click your account avatar, and then click the image icon next to Associated Accounts to create an associated account.

  2. In the Create Service Provider Account dialog box, select the account Type. DMS supports the following three account types:

    • GitHub

    • Yunxiao Code Management (Codeup)

    • Self-hosted GitLab

  3. Select the Creation Method, which determines how the account is imported into the workspace. DMS supports the following two methods:

    • Username and Password

    • User Token

  4. Configure the password or token.

    • If you select Username and Password as the Creation Method, enter the Username and Password of the account.

    • If you select User Token as the Creation Method, enter the Username and Access Token of the account.

  5. Click OK.

Step 2: Create a code repository

  1. In the left-side navigation pane of the workspace, click the image icon to go to the File Explorer page.

  2. In the CODE section, click the image icon and select Add existing Git repository to create a code repository.

  3. In the Create Code Repository dialog box, configure the following parameters:

    • path: The path of the code repository. Default value: /Workspace/code. Click Edit to modify the path.

    • Directory Name (required): Enter the directory name of the code repository.

    • Git Repository URL (required): The URL of the Git repository to add. SSH and HTTP/HTTPS URLs are supported.

      Note

      If you use an HTTP/HTTPS URL, you must enter the Username and Token for accessing the repository.

    • Private Repository: Specify whether the Git repository is private. Select this option if your code repository uses a non-public Git service.

  4. Click OK. The repository is created when its name appears in the repository list.

Step 3: Repository operations and code deployment

Repository operations (Git repositories only)

  • In the CODE section, click master next to the target repository name to switch branches, create a branch, edit code, or save code (equivalent to running the git push command).

  • Move the pointer over the repository name and click the image icon. Select Repository Information. In the dialog box that appears, confirm the environment and parameter settings.

Code deployment

  1. In the upper-right corner of the CODE section, click the image button.

  2. In the dialog box that appears, select the Deployment Branch, configure the Excluded Directories, and then click Deploy.

    Note

    The dialog box indicates that deployment may take up to 10 minutes. Uncommitted changes in the current branch are automatically committed and pushed.