Data Connection

Data synchronization rules

On the Files tab of a file-type data connector, click Sync Data Rules > Create Sync Rule in the upper-right corner to open the rule creation page.

1. Select a Category for the data synchronization.

2. Select a Sync Source:

   OSS

   Model Studio uses a service-linked role (SLR) to automatically synchronize data from OSS by accessing your buckets.

   Important

   • OSS synchronization uses a service-linked role (SLR) to access user buckets, eliminating the need to configure an AccessKey. Before first use, you must initialize the service-linked role and complete the authorization as prompted.

   • The object path must include the bucket name, for example, my-bucket/docs/ or my-bucket/docs/foo.md.

   • The current account must have read permissions for the target bucket. To grant access to Model Studio, you must also add the bailian-datahub-access tag with the value read to the target bucket. For more information, see Manage bucket tags.

   Fill in the following OSS object information:

   Parameter	Required	Description
   OSS region	Yes	Select the region where the OSS bucket is located. For a list of supported regions, see OSS Regions Overview.
   OSS object path	Yes	Enter the path of the OSS object to synchronize. The path must include the bucket name. For example, my-bucket/docs/ synchronizes all files in the directory, while my-bucket/docs/foo.md synchronizes a single file.
   Synchronization period	Yes	Select the frequency for automatic data synchronization, for example, every minute.
   Connection test	Yes	Click Connection Test to verify that the configured OSS path is accessible. You can submit the rule only after the test passes.
   Data tags	No	Configure tags for the synchronized data to help categorize it. Each tag can be up to 32 characters long and can contain Chinese characters, uppercase and lowercase letters, numbers, underscores (_), and hyphens (-). Press Enter after typing to add a tag.

   Lark

   Access Lark knowledge bases and documents through Lark application credentials to synchronize content to the Bailian platform. For instructions on creating a Lark custom app, see Lark custom app development process.

   Important

   • Before synchronizing Lark documents, you must create a custom app on the Lark Open Platform and obtain the App ID and App Secret.

   • You must enable read-only permissions for knowledge base, cloud document, and spreadsheet features, as well as the contact user ID read permission for the app.

   • You must fill in one of the following: Lark Knowledge Base ID, Lark Folder ID, or Lark Document ID.

   • Synchronization is supported for documents (doc), spreadsheets (xls), bitable, PPT, and other nodes in Lark knowledge bases. It is not supported for locally uploaded files.

   1. Create a Lark application and obtain credentials

   1. Log in to the Lark Open Platform and click Developer Console in the upper-right corner. In the console, click Create Custom App, fill in the App Name, App Description, and App Icon, and then click Create.

   2. On the app management page, find the Bot feature and click Add.

   3. In the left-side navigation pane, click Permission Management. On the tab that opens, click Bulk Import/Export Permissions. Copy the following JSON-formatted permission description code into the Import tab, click Next, Confirm New Permissions, and then click Request to Enable.

      {
        "scopes": {
          "tenant": [
            "aily:file:read",
            "aily:file:write",
            "application:application.app_message_stats.overview:readonly",
            "application:application:self_manage",
            "application:bot.menu:write",
            "cardkit:card:write",
            "contact:contact.base:readonly",
            "contact:user.employee_id:readonly",
            "corehr:file:download",
            "docs:document.content:read",
            "event:ip_list",
            "im:chat",
            "im:chat.access_event.bot_p2p_chat:read",
            "im:chat.members:bot_access",
            "im:message",
            "im:message.group_at_msg:readonly",
            "im:message.group_msg",
            "im:message.p2p_msg:readonly",
            "im:message:readonly",
            "im:message:send_as_bot",
            "im:resource",
            "sheets:spreadsheet",
            "wiki:wiki:readonly"
          ],
          "user": [
            "aily:file:read",
            "aily:file:write",
            "im:chat.access_event.bot_p2p_chat:read"
          ]
        }
      }

   4. Go to the Credentials and Basic Information page and copy the App ID and App Secret for later use.

   2. Configure knowledge base access permissions for the Lark application

   • In the Lark client, create a group chat and add the application to it.

   • As a knowledge base administrator, navigate to Knowledge Base Settings > Member Settings > Add Admin.

   • Search for the group chat containing the bot and add the group as an administrator.

   Fill in the following Lark document information:

   Parameter	Required	Description
   Lark App ID	Yes	The App ID of your Lark custom app. View instructions.
   Lark App Secret	Yes	The App Secret of your Lark custom app. Obtain it from the Credentials and Basic Information page on the Lark Open Platform.
   Lark Knowledge Base ID	One of three	You must fill in one of the following: Lark Knowledge Base ID, Lark Folder ID, or Lark Document ID. Obtain it from the Lark knowledge base URL.
   Lark Folder ID	One of three	You must fill in one of the following: Lark Knowledge Base ID, Lark Folder ID, or Lark Document ID. Obtain it from the Lark folder URL.
   Lark Document ID	One of three	You must fill in one of the following: Lark Knowledge Base ID, Lark Folder ID, or Lark Document ID. Obtain it from the Lark document URL.
   Exclude Document IDs	No	A list of Lark document IDs that you do not want to synchronize.
   Lark Document Domain	No	If your document domain is xxx.feishu.cn/wiki/..., enter xxx. This is used to populate the document source URL during upload.
   Sync Period	Yes	Select the frequency for automatic data synchronization. Options: 1 minute, 1 hour (default), and 1 day.
   Connection Test	Yes	Click Connection Test to verify that the Lark application credentials and knowledge base access are working properly. You can submit the rule only after the test passes.
   Data Tags	No	Configure tags for the synchronized data for easier categorization. Each tag can be up to 32 characters and supports Chinese, English letters (upper and lower case), numbers, underscores (_), and hyphens (-). Press Enter to confirm after typing.

   After filling in the form, click Connection Test to verify connectivity. Once the test passes, configure data tags.

   Important

   • You must set export permissions for Lark.

   • Lark does not support exporting MindNote documents.

   DingTalk

   To create a DingTalk document synchronization rule, obtain the necessary credentials by following these steps.

   1. Create a DingTalk application

   1. Log in to the DingTalk Open Platform. Select an organization for which you have developer permissions, or select an organization and then obtain developer permissions.

   2. In the top menu bar, select App Development.

   3. On the right side of the page, click Create App, fill in the application name and description, and then click Save. You are automatically taken to the application details page.

   2. Add permissions to the DingTalk application

   Before synchronizing data, on the application details page, navigate to Development Configuration > Permission Management, then search for and enable the following permissions:

   • Contact.User.Read

   • Wiki.Workspace.Read

   • Wiki.Node.Read

   • Document.Workbook.Read

   • Notable.Base.Read_All

   • qyapi_get_member

   3. Publish the application

   1. In the left-side navigation pane of the application details page, click Version Management & Release.

   2. On the right side of the page, click Create New Version and enter a version number (for example, 1.0.0) and a version description.

   3. Set the visibility range, for example, to All Employees.

   4. Click Save, and then confirm the publication.

   4. Obtain the application Client ID and Client Secret

   1. In the left-side navigation pane, click Credentials and Basic Information.

   2. Obtain the Client ID and Client Secret.

   5. Obtain the DingTalk User ID

   1. Go to the DingTalk Admin Console and log in as an organization administrator.

   2. In the left-side navigation pane, click Contacts > Member Management.

   3. In the member list, find the Employee UserID column and copy the UserID of the target user.

   Note

   You must have administrative permissions for the organization to view employee UserIDs.

   6. Enable DingTalk MCP

   Before creating a DingTalk synchronization rule, you must enable the following MCP services in the DingTalk MCP Marketplace:

   • DingTalk Docs MCP: Required to synchronize DingTalk documents.

   • DingTalk Sheets MCP: Required to synchronize DingTalk spreadsheets.

   Steps to obtain an MCP Endpoint:

   1. Go to the DingTalk MCP Marketplace, find DingTalk Docs and DingTalk Sheets, and click to open their details pages.

   2. On the details page, click to get the MCP server configuration and copy the corresponding MCP Endpoint link.

   3. Paste the copied links into the DingTalk Docs MCP Link and DingTalk Sheets MCP Link fields in the form.

   7. Obtain the DingTalk Knowledge Base ID, Folder ID, or Document ID

   You must fill in one of the following three IDs in the form to specify which content to synchronize: DingTalk Knowledge Base ID, DingTalk Folder ID, or DingTalk Document ID. You can obtain them directly from the URL:

   • DingTalk Knowledge Base ID: Go to the DingTalk knowledge base homepage, click the knowledge base you want to synchronize, and find the ID at the end of the URL in your browser's address bar.

   • DingTalk Folder ID: In the knowledge base, click the corresponding folder. The ID is at the end of the URL.

   • DingTalk Document ID: In the knowledge base, click the corresponding document. The ID is at the end of the URL.

   Note

   These three IDs determine the synchronization scope. Providing the knowledge base ID synchronizes the entire knowledge base. Providing the folder ID synchronizes that folder and its sub-documents. Providing the document ID synchronizes only that single document.

   After completing the preceding steps, fill in the DingTalk Document Information:

   Parameter	Required	Description
   DingTalk user ID	Yes	Obtain this from Contacts > Member Management in the DingTalk Admin Console.
   DingTalk application ID	Yes	Obtain this from the app management page on the DingTalk Open Platform.
   DingTalk application secret	Yes	Obtain this from the app management page on the DingTalk Open Platform.
   DingTalk knowledge base ID	One of three	You must enter one of the following: DingTalk Knowledge Base ID, DingTalk Folder ID, or DingTalk Document ID.
   DingTalk folder ID	One of three	You must enter one of the following: DingTalk Knowledge Base ID, DingTalk Folder ID, or DingTalk Document ID.
   DingTalk document ID	One of three	You must enter one of the following: DingTalk Knowledge Base ID, DingTalk Folder ID, or DingTalk Document ID.
   DingTalk Docs MCP link	Conditionally required	This value is required if you need to synchronize DingTalk documents.
   DingTalk Sheets MCP link	Conditionally required	This value is required if you need to synchronize DingTalk spreadsheets.
   Synchronization period	Yes	Select the frequency for automatic data synchronization.

   After filling in the information, click Connection Test to verify connectivity. After the test passes, configure the data tags.

   Important

   • DingTalk document synchronization consumes DingTalk development resources. View DingTalk development resource consumption and expansion options.

   • Synchronization is supported for DingTalk documents, spreadsheets, and AI sheets created in a DingTalk knowledge base. It is not supported for documents and spreadsheets uploaded from local files.

   DingTalk API call consumption

   Each synchronization consumes the API quota of the DingTalk Open Platform. We recommend that you assess your API quota in advance to ensure it is sufficient. Consumption occurs in two phases: pre-synchronization check and synchronization execution.

   Pre-synchronization check (checks connectivity, permissions, and resource existence when a task starts):

   • Account authentication: 1 call.

   • Batch document check: 1 call for every 30 documents (rounded up), plus 1 additional call.

   • AI sheet existence check: 1 call per AI sheet.

   • Knowledge base existence check: 1 call per knowledge base.

   Synchronization execution (pulls content into Model Studio):

   • Account authentication: 1 call.

   • Knowledge base content retrieval: 1 call per knowledge base.

   • AI sheet content retrieval: (Number of sheets + Number of unidirectional/bidirectional linked fields) calls per AI sheet.

   Example

   To synchronize 1 knowledge base (containing 50 documents) and 2 AI sheets (each with 3 sheets and 2 linked fields), the total API calls are calculated as follows:

   • Pre-synchronization check: 1 + (ceiling(50/30) + 1) + 2 + 1 = 7 calls.

   • Synchronization execution: 1 + 1 + 2 * (3 + 2) = 12 calls.

   • Total: approximately 19 calls.

   Note

   This is a typical estimate. The actual consumption may fluctuate slightly due to factors such as API retries and error rollbacks.

   SharePoint

   Access SharePoint documents and folders through Azure AD application registration credentials to synchronize content to the Bailian platform.

   Important

   SharePoint synchronization is currently available only to whitelisted users. To request access, contact the Alibaba Cloud Bailian team.

   Prerequisites: Register an Azure AD app and obtain credentials

   1. Log in to the Azure portal, search for Microsoft Entra ID (formerly Azure Active Directory) in the top search bar, or click Manage Microsoft Entra ID on the home page.

   2. Under Manage in the left navigation pane, click App registrations, then click New registration. Fill in the application name and select the supported account type to create the app. For detailed instructions, see Microsoft documentation: Register an application.

   3. On the app Overview page, in the Essentials section, copy the Directory (tenant) ID. This is the Tenant ID in the form.

   4. In the same Essentials section, copy the Application (client) ID. This is the Client ID in the form.

   5. Under Manage in the left navigation pane, click Certificates & secrets, switch to the Client secrets tab, and click New client secret. Fill in the description and set the expiration. Copy the Value immediately after creation (it is only displayed once and cannot be viewed after leaving the page). This is the Application Secret in the form.

   6. Under Manage in the left navigation pane, click API permissions, click Add a permission, select Microsoft Graph -> Application permissions, and add the following permissions:

      • Sites.Read.All: Read files in all site collections.

      • Files.Read.All: Read file content.

   7. After adding permissions, click the Grant admin consent for [tenant name] button at the top of the page to complete authorization. The permission status will change to "Granted for [tenant name]".

   Obtain SharePoint document or folder share links

   1. Open the document or folder you want to synchronize in SharePoint.

   2. Click the Share button in the upper-right corner, configure the sharing permissions, and obtain the share link.

   3. Copy the share link and paste it into the SharePoint document or folder share link list field in the form. Separate multiple links with line breaks.

   Fill in the following SharePoint document information:

   Parameter	Required	Description
   Tenant ID	Yes	The Directory (tenant) ID of your Azure AD app. Obtain it from the Essentials section on the app registration Overview page in the Azure portal.
   Client ID	Yes	The Application (client) ID of your Azure AD app. Obtain it from the Essentials section on the app registration Overview page in the Azure portal.
   Application Secret	Yes	The client secret value of your Azure AD app. Create it on the Certificates & secrets page in the Azure portal, under the Client secrets tab. Note: The secret value is only displayed once at creation time and cannot be viewed after leaving the page. Save it promptly.
   SharePoint document or folder share link list	Yes	Enter the share links of the SharePoint documents or folders you want to synchronize. Separate multiple links with line breaks. Obtain share links through the Share feature in SharePoint.
   Cloud Type	No	Select the cloud environment of your SharePoint account. Options: Global Public Cloud (default) and China 21Vianet. If your SharePoint account is deployed on the China 21Vianet cloud, select the corresponding option.
   Sync Period	Yes	Select the frequency for automatic data synchronization. Options: 1 minute, 1 hour (default), and 1 day.
   Connection Test	Yes	Click Connection Test to verify that the SharePoint credentials and share links are accessible. You can submit the rule only after the test passes.
   Data Tags	No	Configure tags for the synchronized data for easier categorization. Each tag can be up to 32 characters and supports Chinese, English letters (upper and lower case), numbers, underscores (_), and hyphens (-). Press Enter to confirm after typing.

   After filling in the form, click Connection Test to verify connectivity. Once the test passes, configure data tags.

   Yuque Sync

   Access Yuque documents and knowledge bases through a Yuque Token to synchronize content to the Bailian platform.

   Obtain a Yuque Token

   • Personal Token: Log in to the Yuque settings page, create a new token, and copy it.

   • Team Token: Navigate to Team Space -> Team Settings -> Developer -> Tokens, create a new token, and copy it.

   Fill in the following Yuque document information:

   Parameter	Required	Description
   Yuque Document/Knowledge Base URL	Yes	A list of complete Yuque document or knowledge base URLs. Separate multiple URLs with commas (,). Example: https://www.yuque.com/xionghuyi/xr0xqi,https://www.yuque.com/xionghuyi/xr0aax.
   Yuque Token	Yes	A Yuque access token. Both personal tokens and team tokens are supported. See the instructions above for how to obtain a token.
   Document Permission Level	No	The permission levels of Yuque documents to synchronize. Multiple values are supported. 0: Private, 1: Public, 2: Public within the organization. Example: to sync private and organization-public documents, enter 0,2. Default: 0,1,2 (this setting does not apply when the URL list specifies a specific document).
   Sync Period	Yes	Select the frequency for automatic data synchronization. Options: 1 minute, 1 hour (default), and 1 day.
   Connection Test	Yes	Click Connection Test to verify that the Yuque Token and URLs are accessible. You can submit the rule only after the test passes.
   Data Tags	No	Configure tags for the synchronized data for easier categorization. Each tag can be up to 32 characters and supports Chinese, English letters (upper and lower case), numbers, underscores (_), and hyphens (-). Press Enter to confirm after typing.

   After filling in the form, click Connection Test to verify connectivity. Once the test passes, configure data tags.

3. Configure the data tags, and then click Confirm to complete the creation.

After creation, the rule is displayed on the synchronization rule list page, where you can Enable or Disable it in the Actions column.

File connector tools

Creating a file connector automatically generates the following two tools:

Permissions and security

• What should I do if I receive a "permission for this module is missing" error when importing data?

  The root account must grant the Administrator permission to the RAM user. For more information, see Page permissions.

• Is it safe to upload confidential files through public network data connectors when creating a knowledge base?

  Data connectors transmit data through public network links, but the entire transmission uses HTTPS/TLS encrypted channels. Data is in ciphertext during transmission and cannot be interpreted even if the link is intercepted. After data reaches the platform, it is written directly to Alibaba Cloud internal network secure storage, which cannot be accessed from the public network. The public network only serves as a transport medium—data is not persisted on the public network.

  After import, files are stored as independent copies (unlinked from original data) in the platform's secure storage. Imported files are accessible only to users in the current workspace. Alibaba Cloud Model Studio does not use the data for any commercial purposes or make it publicly available.

  If you still have concerns about public network transmission, you can choose the following options for enhanced security:

  • Direct file upload: Upload local files through the file connector. Data does not pass through public network links and is written directly to Alibaba Cloud internal network secure storage.

  • Private network data source: For Alibaba Cloud RDS for MySQL or self-managed MySQL databases, you can select private network (VPC) connection. Data is transmitted through the internal network, offering better security and performance. See the Private network data source section above.

  Model Studio Standard and Enterprise editions have identical data security mechanisms: both provide user-level isolation, workspace member-only access, and HTTPS/TLS encrypted transmission. The difference between the two editions is only in storage capacity and feature limits, not security.

Import files from OSS

• How do I configure permissions to import files from OSS?

  The first time you import files from OSS, you must authorize Alibaba Cloud Bailian to access your OSS resources. The authorization process differs for a root account and a RAM user.

  Authorize a root account

  1. On the Import data page, click Authorize Now.

  2. In the dialog box that appears, click Confirm Authorization. The system automatically creates an OSS service-linked role to allow Alibaba Cloud Bailian to access your OSS resources.

     This change usually takes effect within seconds but can be delayed during peak hours.

  3. Add the tag bailian-datahub-access with the value read to the target OSS bucket.

     This tag identifies the buckets that Alibaba Cloud Bailian can access. Alibaba Cloud Bailian cannot access buckets without this tag.

     1. Go to the OSS console. In the left-side navigation pane, click Bucket list and find the target bucket.

     2. Hover over the icon next to the bucket name and click Modify (if no tags have been set) or Go to edit.

     3. On the bucket tags page, click Create tag (if no tags have been set) or Settings.

     4. Click Tag, add a tag with the key bailian-datahub-access and the value read, and then click Save.

  4. Return to the Import Data page, reselect the target bucket, and try the import again.

     Note: Alibaba Cloud Bailian does not support accessing files in the root directory of a bucket. You must select an existing subdirectory or create a new one for Alibaba Cloud Bailian to access.

  Authorize a RAM user

  1. On the Import data page, click Authorize Now.

  2. In the dialog box that appears, click Confirm Authorization. If the UI displays an Authorization failed error with the message The current user does not have permission to create a service-linked role, you must first grant this permission to the RAM user.

     1. Sign in to the RAM console as the root account. In the left-side navigation pane, choose Permissions > Policies, and then click Create policy.

     2. Click the Script editor tab. Copy and paste the following policy into the editor, and then click OK.

        {
            "Action": [
                "ram:CreateServiceLinkedRole"
            ],
            "Resource": "*",
            "Effect": "Allow",
            "Condition": {
                "StringEquals": {
                    "ram:ServiceName": "datahub.sfm.aliyuncs.com"
                }
            }
        }

     3. Enter a policy name, such as CreateServiceLinkedRole, and click OK.

     4. In the left-side navigation pane, choose Identities > Users. Find the target RAM user in the list and click Add permissions in the Actions column.

     5. Select the custom policy you created, and then click OK. This grants the RAM user permission to create the service-linked role.

  3. Authorize the RAM user to access OSS through Alibaba Cloud Bailian.

     1. Return to the Import Data page and click Authorize Now.

     2. In the dialog box that appears, click Confirm Authorization. The system automatically creates an OSS service-linked role, which is required.

        This change usually takes effect within seconds but can be delayed during peak hours.

  4. Add the tag bailian-datahub-access with the value read to the target OSS bucket.

     This tag identifies the buckets that Alibaba Cloud Bailian can access. Alibaba Cloud Bailian cannot access buckets without this tag.

     1. Go to the OSS console. In the left-side navigation pane, click Bucket list and find the target bucket.

     2. Hover over the icon next to the bucket name and click Modify (if no tags have been set) or Go to edit.

     3. On the bucket tags page, click Create tag (if no tags have been set) or Settings.

     4. Click Tag, add a tag with the key bailian-datahub-access and the value read, and then click Save.

  5. Return to the Import Data page, reselect the target bucket, and try the import again.

     Note: Alibaba Cloud Bailian does not support accessing files in the root directory of a bucket. You must select an existing subdirectory or create a new one for Alibaba Cloud Bailian to access.

• What should I do if I get error "10041495" when importing files from OSS?

  This error usually occurs because the root account has not activated OSS. To resolve this issue:

  1. Sign in to the OSS console as the root account and activate OSS when prompted.

  2. Return to the Alibaba Cloud Bailian page and authorize again.

Import MySQL data