Use MaxFrame in DataWorks

Prerequisites

Create a MaxCompute data source and associate it with your target DataWorks workspace. For instructions, see Associate a data source or a cluster with DataStudio.

Use MaxFrame

1. Create a PyODPS 3 node.

   Go to the Data Development page in DataWorks to create a PyODPS 3 node. For instructions, see Develop a PyODPS 3 task.

2. Create a MaxFrame session.

   PyODPS 3 has built-in MaxCompute user and project information, so you can create a MaxFrame session directly. The following code is an example:

   import maxframe.dataframe as md
   from maxframe import new_session
   from maxframe.config import options
   options.sql.enable_mcqa = False
   
   table = o.create_table("test_source_table", "a string, b bigint", if_not_exists=True)
   with table.open_writer() as writer:
       writer.write([
           ["value1", 0],
           ["value2", 1],
       ])
   
   # Create a MaxFrame session.
   session = new_session(o)
   
   df = md.read_odps_table("test_source_table",index_col="b")
   df["a"] = "prefix_" + df["a"]
   
   # Print the DataFrame data.
   print(df.execute().fetch())
   
   # Write the MaxFrame DataFrame data to a MaxCompute table.
   md.to_odps_table(df, "test_prefix_source_table").execute()
   
   # Destroy the MaxFrame session.
   session.destroy()
   

Verify the result

After you run the script, verify that MaxFrame is working correctly by checking the following outputs.

Script output: The script prints the following DataFrame result:

b           a
0  prefix_value1
1  prefix_value2

Table query: Execute the following SQL statement in your MaxCompute project to query data in the test_prefix_source_table table:

SELECT * FROM test_prefix_source_table;

The following result is returned:

+------------+------------+
| b          | a          |
+------------+------------+
| 0          | prefix_value1 |
| 1          | prefix_value2 |
+------------+------------+

If both outputs match the expected results above, MaxFrame is working correctly in your on-premises environment.