Use underlying data masking with MaxCompute

Background information

MaxCompute does not natively support dynamic data masking. Instead, it uses the data masking capabilities of DataWorks Data Security Guard. To enable dynamic data masking for a MaxCompute project, you must first activate Data Security Guard.

The new underlying data masking for MaxCompute feature works as follows:

• Once configured in Data Security Guard, masking rules are automatically applied to queries on sensitive data from any MaxCompute entry point, including JDBC connections, Cloud Shell (odpscmd) connections, or local client (odpscmd) connections.

• When this feature is enabled with corresponding rules, queries on sensitive data are subject to the underlying data masking for MaxCompute rules.

This feature effectively protects sensitive information such as phone numbers, ID card numbers, bank card numbers, license plate numbers, and IP addresses. Dynamic data masking masks only the query results and does not modify the underlying stored data.

Limitations

• This feature is available only in DataWorks Professional Edition or higher and is not supported in the Basic Edition. If you use the Basic Edition, you need to upgrade. For more information, see Upgrade DataWorks.

• Underlying data masking for MaxCompute is supported only in the following regions: China (Beijing), China (Shanghai), China (Hangzhou), China (Chengdu), China (Shenzhen), China North 2 Ali Gov 1, China East 2 Finance, China (Hong Kong), Singapore, Germany (Frankfurt), Malaysia (Kuala Lumpur), US (Silicon Valley), and Indonesia (Jakarta).

• If underlying data masking for MaxCompute is enabled and its rules are configured, those rules are applied. Otherwise, the rules from your configured upper-layer masking scenario are used.

• This feature cannot mask primary key fields in MaxCompute tables.

• This feature is available only for MaxCompute projects that contain data that is more than 24 hours old.

Prerequisites

1. Prepare a MaxCompute project and the data to be masked. For instructions, see Create a MaxCompute project and Import data.

2. Activate the Data Security Guard service. For instructions, see Go to Data Security Guard.

   On the Service Agreement page, select I have read and agree to the preceding terms, and then click Activate Now.

Enable data masking

1. Select a masking scenario.

   1. Go to Data Security Guard.

   2. In the left-side navigation pane, choose Rule Configuration > Data Masking to go to the Data Masking page.

   3. In the Masking Scenarios section, select Underlying Masking Scenario > Underlying Data Masking for MaxCompute_New.

   Note

   If you want to view the masking effect on the DataWorks console, turn on Display Masking in DataStudio/Data Map. For more information about creating data masking scenarios, see Create a data masking scenario.

2. Create a data masking rule.

3. (Optional) Configure an allowlist.

   To exempt specific users from a masking rule, add them to an allowlist.

   1. On the Data Masking page, click the Allowlist tab.

   2. In the upper-right corner, click + Create Allowlist.

   3. In the Create Allowlist dialog box, select a Sensitive Data Type, User Group, and Effective Period.

      Note

      After you set an effective period, users on the allowlist can view the original data only within the specified period. When a query is run outside this period, the data is masked.

4. Configure the project.

   1. Click Underlying Data Masking for MaxCompute_New to view all compatible MaxCompute projects.

   2. Find the target project and turn on the Status switch to enable the masking rules for it.

      Note

      If underlying data masking for MaxCompute is enabled and its rules are configured, those rules are applied. Otherwise, the rules from your configured upper-layer masking scenario are used.

Verify the masking result

This example shows how to use the odpscmd client to run an SQL statement and verify the masking effect.

1. Run a query statement.

   Note

   Underlying data masking for MaxCompute is supported only at the session level.

   The following script provides an example query.

   select * from table;

2. View the masking result.

   AdnocSink1: 113  (min: 113, max: 113, avg: 113) MaxInstance: 0
          GlobalInit: 3116  (min: 3116, max: 3116, avg: 3116)    MaxInstance: 0
              Limit1: 0  (min: 0, max: 0, avg: 0)    MaxInstance: 0
            Project1: 766  (min: 766, max: 766, avg: 766)    MaxInstance: 0
         TableScan1: 0  (min: 0, max: 0, avg: 0) MaxInstance: 0
   +------------+-------------+----------------------+-------+
   | name       | phone_no    | gmt_order            | amt   |
   +------------+-------------+----------------------+-------+
   | simon      | **10***3127 | 2015-10-03 01:12:31  | 18.6  |
   | trudy      | **33***5599 | 2015-10-03 01:12:31  | 6.58  |
   | jinhang    | **30***0850 | 2015-10-03 01:12:31  | 73.34 |
   | jiangfang  | **15***2356 | 2015-10-03 01:12:31  | 30.62 |
   | lyan       | **66***0573 | 2015-10-03 01:12:31  | 4.92  |
   | lijing     | **30***3828 | 2015-10-03 01:12:31  | 56.74 |
   | frank.lizg | **58***9679 | 2015-10-03 01:12:31  | 21.09 |
   | maggie     | **60***6880 | 2015-10-03 01:12:31  | 31.17 |
   | daniel     | **63***5620 | 2015-10-03 01:12:31  | 42.42 |
   | carvin     | **60***4886 | 2015-10-03 01:12:31  | 80.75 |
   +------------+-------------+----------------------+-------+
   A total of 10 records fetched by instance tunnel. Max record number: 10000
   dsg_fin_demo>