Create a multi-physical table service unit

Updated at:

A multi-physical table service unit associates multiple physical tables and configures metadata for their joined fields. You can then use this service unit to develop APIs that access data from these tables to perform join queries. This topic describes how to create a multi-physical table service unit.

Prerequisites

  • You have created a service unit group. For more information, see Create a service project group.

  • If your data source is HBase 0.9.4/1.1.x/1.2.1/2.x, Elasticsearch, or MongoDB, you must create the required metadata. For more information, see Create metadata.

Limitations

Permissions

Project administrators and developers can create service units.

Important

Joined queries that use a multi-physical table service unit succeed only if the join key fields in the primary and non-primary tables have matching values and data types. Otherwise, the API call fails.

Step 1: Select a service unit type

  1. In the top navigation bar of the Dataphin console, navigate to Service > API Development.

  2. In the upper-left corner, select your project. In the left-side navigation pane, click Service Unit. On the Service Unit page, click + New Service Unit.

  3. In the New Service Unit dialog box, select multi-physical table service unit.

Step 2: Configure the service unit

  1. On the New Service Unit page, configure the basic information and parameters for the service unit.

    1. Configure basic information

      Parameter

      Description

      Service unit name

      The name of the service unit. The name must meet the following requirements:

      • Can contain only letters, digits, and underscores (_).

      • Must start with a letter.

      • Must be 4 to 64 characters long.

      Service unit group

      Select an existing service unit group.

      Mode

      The mode determines whether the service unit reads data from a development data source or a production data source. The following modes are available: Basic and Dev-Prod.

      • Basic: In this mode, the service unit can read data only from a data source in the production environment.
      • Dev-Prod: In this mode, the service unit can read data from either a data source in the development environment or a data source in the production environment.

      You can check the environment where a data source resides on the Data Sources page.

      Maximum number of returned entries

      The default value is 1,000. The maximum value is 10,000.

      This parameter limits the number of entries returned by a single API query to prevent system out-of-memory errors. To retrieve more data, you can modify the PageSize parameter in the SDK when you Call an API.

      Service unit description

      Enter a brief description of the service unit. The description can be up to 128 characters long.

      Add physical table

      Click Add Physical Table. In the Add Physical Table dialog box, select the data tables from your data source. You can add up to 10 physical tables. For a list of supported data sources, see Data sources supported by Data Service.

      image.png

      By default, Dataphin sets the first table you add as the primary table. You can select a different physical table as the primary table.

      The tables are combined using a left join operation. The primary table serves as the left table, and all other tables are treated as right tables. You must select a field from the left table as the join key. This field becomes a required parameter when you call the API. If the API call does not include this field, the call fails.

      If Dataphin finds existing service units that are similar to the one you are configuring, it automatically displays their names. You can review the details of these service units in the service unit list. To proceed, click Continue. This action takes you to the Configure service unit fields page.

    2. Configure the fields for the service unit.

      Parameter

      Description

      Configure service unit association fields

      Service unit field

      Specify a custom name for the service unit's association field. The name must meet the following requirements:

      • Can contain only letters, digits, and underscores (_).

      • Must start with a letter.

      • Must be 4 to 64 characters long.

      Primary table name.Associated field (primary key)

      Select a field from the primary table to use as the join key. This field, which must have values that match the corresponding fields in the non-primary tables, becomes a required parameter in API calls. If an API call omits this parameter, the call fails.

      Service unit association field type

      Select the data type of the associated field in the primary table.

      Service unit field description

      Enter a brief description for the associated service unit field.

      Configure service unit fields

      Service unit field type

      Supported data types include Double, Float, String, Date, Boolean, Int, Long, Short, Byte, BigDecimal, and Binary. Select the data type that corresponds to your business data.

      If your business data uses a different data type, we recommend selecting String.

      Service unit field

      The name of the service unit field, which maps to the underlying business data field. You can customize the field name based on your business requirements. The name must meet the following requirements:

      • Can contain only letters, digits, and underscores (_).

      • Must start with a letter.

      • Must be 4 to 64 characters long.

      Service unit field description

      Enter a brief description of the service unit field.

      After you configure the service unit fields, click Preview Data at the bottom of the page to view the corresponding business data. The preview displays a maximum of 1,000 data entries.

    3. The system automatically detects and displays any row-level security policies associated with your selected physical tables. This information includes the policy name, description, control field, data source environment, associated table, and associated field. You can also perform the following actions:

      • Enable or Disable a policy: This controls whether a row-level security policy is active and whether it appears in the policy list when you view the service unit.

      • Create Row-level Permissions: You must have permission to create row-level security policies. Clicking this option redirects you to the Management Center > Permission Management page, where you can create a new row-level security.

        Note
        • To use this feature, you must purchase the row-level security service.

        • Row-level security controls apply to the data returned by an API call. Applying different policies may therefore change the results.

        • In Basic mode, the system displays policies associated with physical tables in the production environment. In Dev-Prod mode, it displays policies for tables in both the development and production environments.

        • After a service unit with enabled row-level security is published, the associated security policies apply to any APIs created from it.

        • For APIs created from a published service unit with row-level security, any changes to the associated policies in Management Center > Permission Management take effect automatically. You do not need to republish the API.

        • A table's row-level security policies apply to all of its fields, regardless of whether the fields are used in the API or added to the service unit.

        • If multiple fields from different physical tables are associated with the same row-level security policy, the policy applies to all of them.

  2. Click Submit to create the multi-physical table service unit.

Next steps

  • On the service unit page, find the service unit you created. In the Actions column, click the image.png icon and select Publish. This publishes the service unit to the production environment. You can then create an API from it. For more information, see Create an API.

  • After you create a service unit, you can edit, publish, or delete it on the Service Unit page. For more information, see Manage a service unit.