This tutorial guides you through building user profiles with a sample dataset in DataWorks. You will learn to perform data synchronization, processing, and quality monitoring in the China (Shanghai) region. Before you start, prepare a MaxCompute project, a DataWorks workspace, and the required data sources, compute resources, and storage resources.
Background
To improve business strategies, you need to analyze website user behavior to build basic user profiles. These profiles can include attributes like geographic location and social characteristics. By performing scheduled analysis, you can achieve granular control over website traffic.
Before you begin
To complete this tutorial, ensure you have read the tutorial overview and understand the entire user profile analysis workflow.
Notes
-
This tutorial provides the required user information and website access test data. You can use this data directly.
-
The data provided in this tutorial is for hands-on practice in DataWorks only. All data is mock data.
-
This tutorial uses the new version of Data Studio for data processing.
Prepare the MaxCompute environment
Step 1. Activate MaxCompute
This tutorial uses MaxCompute. Before you proceed, ensure that you have activated MaxCompute in the China (Shanghai) region with the following settings.
-
Region: China (Shanghai)
-
Type: Standard compute resources
Step 2. Create MaxCompute projects
A DataWorks workspace in standard mode must be bound to two MaxCompute projects. One project is for the development environment, and the other is for the production environment.
-
Go to the MaxCompute console. In the left-side navigation pane, choose .
-
Click New Project and create two separate MaxCompute projects. The following table describes the key parameters for this tutorial. Use the default values for other parameters.
Parameter
Description
Project Name
Custom. The name must be globally unique.
Examples for this tutorial:
-
Production environment:
workshop2024_01 -
Development environment:
workshop2024_01_dev
Billing Method of Computing Resources
For this tutorial, select Pay-as-you-go.
Default Quota
For this tutorial, select
os_PayAsYouGoQuota (Pay-as-you-go Standard Edition)from the drop-down list.Data Type
For this tutorial, select 2.0 Data Type (Recommended) from the drop-down list.
Storage Encryption
For this tutorial, select Not Encrypted.
-
For more information on creating a MaxCompute project, see Create a MaxCompute project.
Prepare the DataWorks environment
Before you use DataWorks, make sure you have activated the service. For more information, see Activate DataWorks.
Step 1. Create a workspace
If you already have a workspace that uses the new version of Data Studio in the China (Shanghai) region, you can skip this step.
-
Log on to the DataWorks console. In the top navigation bar, switch the region to China (Shanghai). In the left-side navigation pane, click Work space to go to the workspace list page.
-
Click Create Workspace to create a workspace in standard mode that uses the Use Data Studio (New Version), with Isolate Development and Production Environments.
NoteAfter February 18, 2025, if an Alibaba Cloud primary account activates DataWorks and creates a workspace in the China (Shanghai) region for the first time, DataWorks enables the new version of Data Studio by default.
For more information about how to create a workspace, see Create a workspace.
Step 2. Create a serverless resource group
-
Purchase a serverless resource group.
This tutorial requires a DataWorks serverless resource group for data synchronization and scheduling. You must purchase and prepare a serverless resource group.
-
Log on to the DataWorks resource group list page. In the top navigation bar, switch the region to China (Shanghai). In the left-side navigation pane, click Resource Group to go to the Resource Groups page.
-
Click Create Resource Group. On the purchase page, set Region and Zone to China (Shanghai) and specify a Resource Group Name. Configure other parameters and complete the payment as prompted. For more information about the billing of serverless resource groups, see Billing of serverless resource groups.
NoteIf no VPC or vSwitch is available in the current region, click the corresponding console link in the parameter description to create them. For more information about VPCs and vSwitches, see What is a VPC?.
-
Bind the resource group to your DataWorks workspace.
You must bind a new serverless resource group to a workspace before using it.
Log on to the DataWorks resource group list page and switch the region to China (Shanghai). Find the serverless resource group that you purchased. In the Operation column, click Associate Workspace. Then, find the DataWorks workspace that you created and click Associate.
-
Configure public network access for the resource group.
This tutorial uses test data that must be accessed from the public internet. By default, a resource group cannot access the public internet. To enable access, you must configure an Internet NAT gateway and add an EIP for the VPC bound to the resource group.
-
Log on to the VPC - Internet NAT Gateway console. In the top navigation bar, switch the region to China (Shanghai).
-
Click Create Internet NAT Gateway and configure the parameters. The following table describes the key parameters for this tutorial. Use the default values for other parameters.
Parameter
Value
Region
China (Shanghai).
Network and Zone
Select the VPC and vSwitch that are bound to the resource group.
You can go to the DataWorks console, switch the region to China (Shanghai), and then click Resource Group in the left-side navigation pane to go to the Resource Groups page. Find the resource group that you created and click Network Settings in the Operation column. In the Data Scheduling & Data Integration section, you can view the bound VPC and Switch. For more information about VPCs and vSwitches, see What is a VPC?.
Network Type
Internet NAT Gateway.
EIP
Create an EIP.
Create service-linked role
If you are creating an Internet NAT gateway for the first time, you must create a service-linked role. Click Create Service-Linked Role.
-
Click Buy Now, agree to the service agreement, and then click Activate to complete the purchase.
-
After you purchase the Internet NAT gateway, return to the console and create an SNAT entry for the new gateway instance.
NoteResource groups in the VPC can access the public internet only after you configure an SNAT entry.
-
In the row of the new instance, click Management in the Operation column. On the instance details page, click the SNAT Management tab.
-
On the SNAT Entry List tab, click Create SNAT Entry. Configure the following key parameters:
Parameter
Value
SNAT entry granularity
Select Specify VPC. This ensures that all resource groups in the VPC can access the public internet through the configured EIP.
Select EIP
Select the EIP that is bound to the current Internet NAT gateway instance.
-
After you configure the parameters for the SNAT entry, click OK to create the SNAT entry.
On the SNAT Entry List tab, wait for the Status of the new SNAT entry to become Enabled. Once the status is Available, the VPC bound to the resource group can access the public internet.
-
-
For more information about how to add and use a serverless resource group, see Use serverless resource groups.
Step 3. Bind MaxCompute projects
To process data in MaxCompute using Data Studio, you must first bind your MaxCompute projects to your DataWorks workspace as compute resources.
-
Go to the DataWorks workspace list page. In the top navigation bar, switch the region to China (Shanghai). Find the workspace that you created and click its name to go to the Workspace Details page.
-
In the left-side navigation pane, click Computing Resources.
-
Click Associate Computing Resources, select the Compute Resource Type to bind, and then configure the related parameters.
In the left menu of the Workspace management page, click Compute Resources, and then click the Bind Compute Resources button at the top of the page.
This tutorial uses MaxCompute for compute and storage. Select MaxCompute as the compute engine type and configure the parameters. The following table describes the key parameters for this tutorial. Use the default values for other parameters.
Parameter
Description
MaxCompute Project
Select the MaxCompute project to bind. For this tutorial, bind the projects that you created in Step 2 to the development and production environments respectively.
Default Access Identity
Specifies the identity used to access the MaxCompute project from this workspace.
-
Development environment: Only the Executor identity is supported.
-
Production environment: You can select an identity from the drop-down list based on your logon account. For this tutorial, select Alibaba Cloud primary account.
NoteIf you are logged on with a different type of account, see Data Studio (New): Bind a MaxCompute compute resource for specific instructions.
Computing Resource Instance Name
Enter a custom name to identify the compute engine. When a task runs, the compute engine is selected based on this name.
Connection Configuration
Select the resource group used to connect to the MaxCompute compute resources. The serverless resource groups that you created and bound to the current workspace are displayed. You must test the connectivity for both the development and production environments.
-
-
Click Confirm to complete the configuration of the MaxCompute compute resource.
For more information about how to bind compute engines, see Bind a compute engine.
Next steps
Now that you have prepared the environment, you can proceed to the next tutorial, where you will learn how to synchronize user information data and website access log data to OSS, and then use an ODPS SQL node to create tables and query the synchronized data. For more information, see Synchronize data.