Developing with MaxFrame notebooks

Use cases

• Multi-step data processing: Use a single notebook to chain together a complete pipeline, including data extraction, cleaning, deduplication, data masking, creating derived fields, and writing to a result table.

• Logic that is hard to express in SQL: Use Python for more intuitive operations like bulk processing with regular expressions, complex string manipulation, custom row-level calculations, and cross-column derivations.

• Multimodal data processing: With the Blob column type, process images, audio, and video stored in tables at scale.

• AI-powered generation: Describe which table to read, what to do, and which table to write to, and have the AI generate, check, and run the complete job for you.

Features

Step 1: Create a notebook

In the left-side navigation pane, in the MaxFrame section, click the + icon. Enter a name, for example, mf_my_pipeline, and then click Create Notebook. The editor opens automatically. You can also create a notebook by selecting the MaxFrame type from the unified new file dialog.

Step 2: Generate code with AI

Click Continue with AI in the editor toolbar. The AI Assistant panel on the right switches to the MaxFrame Job Assistant. Describe your requirements in the input box, for example:

Read project_with_dr.sales_detail, remove duplicate records, and filter out rows where the amount is null.
Aggregate daily sales amount and order count, and write the results to project_with_dr.sales_daily.
Use separate cells for each step, and add step names in Markdown cells.

The AI first reads the current notebook and then generates the code.

Step 3: Confirm and apply

Before the AI modifies the notebook, a confirmation card appears, showing the proposed changes and their rationale:

• Click Apply to Notebook: The code is written to the notebook, the editor refreshes immediately, and the AI automatically performs a check.

• Click Cancel: The changes are not applied. You can then ask the AI to generate the code again.

If you are not satisfied after applying the changes, you can continue the conversation. For example, say "change the deduplication to only apply to the order ID". The AI remembers the previous context.

Step 4: Check the code

Click Check at the top of the editor. The client scans the code for common issues and displays the results in the Advisor tab of the bottom panel.

Click Go to next to an issue to jump to the corresponding line of code. You can also click Let Agent Fix All to have the Agent correct the issues.

Step 5: Run job and write to table

Click Run in the toolbar. The client executes all Python cells from top to bottom and displays the progress at the top, such as "Running cell 3/5".

After to_odps_table(...).execute() completes, the result is written to the MaxCompute table. No additional publishing or deployment steps are required. After the run completes, you can preview the output data below the cell.

If you only modified a single cell, you do not need to run all cells again. Click Run Cell in that cell's toolbar.

Cell types

A notebook is essentially a standard Python file that uses comments to define cells:

The top of the editor shows the cell structure of the current notebook, for example, 8 cells · 3 Markdown · 5 Python.

Editing operations

Your content is saved automatically. The status at the top displays Unsaved, Saving, or Saved. The notebook is always saved before being checked or run.

You can right-click a notebook in the left-side navigation pane to Add to Chat (send the code as an attachment to the AI), Copy Name, or Delete.

Code organization rules

Each cell is compiled and executed independently, but variables are shared between cells. Note the following two rules:

• Rule 1: The code block within each cell must be syntactically complete.

  Statement blocks such as try/finally, if, for, and def cannot be split across multiple cells. Doing so will result in a syntax error.

  # ✗ Incorrect: try and finally are split into two cells
  # %%
  try:
      result.execute()
  
  # %%
  finally:
      session.destroy()

  # ✓ Correct: The entire block is in the same cell
  # %%
  try:
      result.execute()
  finally:
      session.destroy()
  

• Rule 2: Variables can be used across cells.

  Variables defined in one cell, such as session and df, can be referenced in subsequent cells without redefinition.

  Code generated by the AI automatically follows these two rules. Please keep them in mind when writing code manually.

If the last line of a cell is an expression, its value is automatically displayed. This is useful for quickly viewing results and is consistent with Jupyter's behavior.

Complete example

# %% [markdown]
# ## Step 1: Create a session
# MaxFrame requires you to explicitly create a session and release it when the job is finished.

# %%
import maxframe.dataframe as md
from maxframe.session import new_session

session = new_session()

# %% [markdown]
# ## Step 2: Read and clean the table
# Deduplicate and filter out null amounts.

# %%
df = md.read_odps_table("project_with_dr.sales_detail")
clean = df.dropna(subset=["amount"]).drop_duplicates(subset=["order_id"])

# %% [markdown]
# ## Step 3: Summarize by day and write to the result table

# %%
daily = clean.groupby("stat_date").agg({"amount": "sum", "order_id": "count"})
daily.columns = ["total_amount", "order_count"]

try:
    md.to_odps_table(daily, "project_with_dr.sales_daily", overwrite=True).execute()
finally:
    session.destroy()

Usage notes:

• MaxFrame uses lazy execution. Computations are not submitted until you call execute().

• overwrite=True overwrites the entire result table with each run. If the table does not exist, MaxFrame automatically creates it.

• Wrap execute() and session.destroy() in a try/finally block within the same cell to ensure resources are released correctly.

• Do not hard-code an AccessKey in your code. Credentials are automatically injected based on the current connection. If you need PyODPS, call ODPS.from_environments() to get an authenticated entrypoint.

Execution modes

A cancel button appears at the top while the job is running. Only one job can run in the same notebook at a time. If you trigger another run, a message is displayed: This draft already has a running task. Wait for it to finish or cancel it before trying again.

Cell output

After each cell runs, the results are displayed in three tabs below the cell:

• Target table preview: The client automatically identifies the cell's output table and displays its data directly in the Target table tab. You can switch between viewing the first 20, 50, or 100 rows, refresh the data, or click the external link icon to open the full table details.

• Review history: If a cell has been run multiple times, a selector appears in the top-right corner, allowing you to switch between and view the results of previous runs by time and status. The last 10 runs for each cell are retained. The output area also displays the duration, instance ID, and an Open LogView link for the current run, allowing you to jump directly to MaxCompute for troubleshooting.

Bottom panel

There are three collapsible tabs at the bottom of the editor:

• Run details: This is divided into four sub-tabs: Summary, Standard output, Console, and DAG. The summary shows the status and duration of the current run. If a run fails, the client highlights the failure location and provides a Let Agent Fix button.

• History: Shows a complete record of all runs for this notebook, including time, mode, status, and duration. Failed runs are marked in red, and you can expand them to see details.

• Advisor: Lists the issues found by the most recent check.