Create an EMR table

Procedure

1. Go to the DataStudio page.

   Log on to the DataWorks console. In the target region, click Data Development and O&M > Data Development in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.

2. Hover over the icon and click Create Table > EMR > Table.

   Alternatively, find the target workflow, right-click EMR, and then click Create Table.

3. In the Create Table dialog box, configure the parameters for the table, such as the engine instance and path.

   For Engine Type, select EMR. Specify the Database and Name. Then, click Create.

4. Click Create to open the table editor.

5. In the General section, configure the parameters.

   Parameter	Description
   Level-1 Folder	The level-1 folder for the table. Note Level-1 and level-2 folders organize tables in DataWorks for easier management.
   Level-2 Folder	The level-2 folder for the table.
   Create Folder	Click Create Folder to go to the Folder Management page. On this page, you can create level-1 and level-2 folders.
   Refresh	After you create a folder, click Refresh.
   Description	A brief description of the new table.

6. In the Physical Model section, configure the parameters.

   Supported storage formats include ORC, PARQUET, SEQUENCEFILE, AVRO, TEXTFILE, JSON, RCFile, and Custom.

   Parameter	Description
   Layer	Select the layer and physical classification from the drop-down lists. To create a new layer or physical classification, contact your workspace administrator, click Create Level, and add them on the Level Management page. After you create them, click Refresh.
   Physical classification
   Partition Type	The options are Partitioned Table and Non-partitioned Table.
   Table Type	The options are Internal Table and External Table.
   Select storage format	Select a storage format for the files in the table based on your business requirements.

7. In the Schema section, configure the parameters.

   Parameter	Description
   Create Field	Click Create Field, configure the field information, and then click Save to add a new field.
   Up	Adjusts the field order for a new table. If you reorder fields for an existing table, you must delete the current table and create a new one with the same name. This operation is prohibited in a production environment.
   Down
   Field Name	The field name. It can contain only letters, digits, and underscores (_).
   Field Type	Supported data types: TINYINT, SMALLINT, INT, BIGINT, FLOAT, DOUBLE, DECIMAL, VARCHAR, CHAR, STRING, BINARY, DATETIME, DATE, TIMESTAMP, BOOLEAN, ARRAY, MAP, and STRUCT.
   Definition or Maximum Value Length	If the selected data type requires a length, specify the length in the text box.
   Description	A description of the field.
   Primary Key	Sets the field as the primary key. A primary key is a business concept that ensures record uniqueness. DataWorks does not enforce primary key constraints.
   Edit	After a field is saved, click Edit to modify its configuration, and then click Save.
   Delete	Deletes the field. Note If you delete a field from an existing table and submit the change, you must delete the current table and create a new one with the same name. This operation is prohibited in a production environment.
   Add Partition	If you set Partition Type to Partitioned Table in the Physical Model section, you must configure partition fields. You can add a partition to the current table. If you add a partition to an existing table, you must delete the current table and create a new one with the same name. This operation is prohibited in a production environment.

8. In the toolbar, click the icon to submit the EMR table to the production environment.

   If you use a workspace in Standard Mode, you must first submit the table to the development environment and then to the production environment.

   Note

   When you submit the table, you must select a scheduling resource group. If you use a serverless resource group, DataWorks submits a table creation task to the engine and prints the execution logs. You can use the execution logs to troubleshoot submission issues. If no serverless resource group is available, purchase and configure one. For more information, see Use a serverless resource group.