Workflow persistence

更新时间:
复制 MD 格式

Workflow resources in a workflow cluster are periodically cleaned up. To analyze and trace workflow execution, you can configure a persistence policy to persist the workflow to a database. This lets you view workflow logs even after the workflow or its pod is deleted. This topic uses an Alibaba Cloud RDS for MySQL database as an example to describe how to configure a workflow persistence policy.

Configure RDS

  1. Create an Alibaba Cloud RDS for MySQL instance. For more information, see Quickly create an RDS for MySQL instance.

    Important

    Ensure that the selected VPC is the same as the VPC used by the workflow cluster. Add the CIDR block of the VPC to the whitelist.

  2. Create a database and a database account. For more information, see Create a database and a database account.

  3. Save the following YAML template as argo-mysql-config.yaml. Run the kubectl create -f argo-mysql-config.yaml command to create a Secret named argo-mysql-config in the workflow cluster. The Secret stores the database account and password.

    Replace `username` and `password` with the database account and password that you created in the preceding step.
    apiVersion: v1
    stringData:
      username: database-username
      password: database-password
    kind: Secret
    metadata:
      name: argo-mysql-config
      namespace: default
    type: Opaque
  4. Edit the workflow-controller-configmap file to add the persistence configuration.

    Note
    • The workflow-controller-configmap file is located in the namespace that is named after the cluster ID.

    • Set host to the endpoint of the RDS for MySQL instance.

    • Set database to the name of the database.

    • Set archive to true.

    • archiveTTL specifies the retention period for persisted data. In this example, the parameter is set to `30d`, which specifies that the persisted workflow is saved in the database for 30 days. There is no limit on the value of this parameter.

      persistence: |
        connectionPool:
          maxIdleConns: 100
          maxOpenConns: 0
          connMaxLifetime: 0s     # 0 means connections don't have a max lifetime.
        archiveTTL: 30d
        archive: true
        mysql:
          host: rm-xxx.mysql.cn-beijing.rds.aliyuncs.com
          port: 3306
          database: argo-workflow
          tableName: argo_workflows
          userNameSecret:
            name: argo-mysql-config
            key: username
          passwordSecret:
            name: argo-mysql-config
            key: password

References

After a workflow is persisted to a database, you can use the Argo command-line interface (CLI) to view its logs, even after the workflow is deleted. For more information, see Use Simple Log Service.