Best practice: end-to-end from model development to simulation evaluation

1.1 Create a model development machine

1. On the Model Development page, click New dev environment.

2. Select the gr00t1.7 official image and a GPU specification (ADB.MLTensor.2 is recommended).

3. Configure the SSH public key and click Confirm.

4. After the status changes to Running, log on via SSH:

ssh -i ~/.ssh/id_rsa root@<SSH地址>

1.2 Verify the environment

After logging on, verify the directory structure:

ls /mnt/workspace    # 工作目录（持久化）
ls /data             # 数据集挂载
ls /official-models  # 官方预训练模型

1.3 Adapt robot data

This is the most critical step in the entire workflow -- it tells the model how to interpret your robot data. You need to prepare two configuration files.

modality.json: Defines the slice ranges for each modality in the state/action arrays, and the key mappings for video and annotation fields. The following example uses the Unitree G1 with a gripper (16-dimensional state/action):

{
    "state": {
        "left_arm":      {"start": 0,  "end": 7},
        "right_arm":     {"start": 7,  "end": 14},
        "left_gripper":  {"start": 14, "end": 15},
        "right_gripper": {"start": 15, "end": 16}
    },
    "action": {
        "left_arm":      {"start": 0,  "end": 7},
        "right_arm":     {"start": 7,  "end": 14},
        "left_gripper":  {"start": 14, "end": 15},
        "right_gripper": {"start": 15, "end": 16}
    },
    "video": {
        "cam_left_high":  {"original_key": "observation.images.cam_left_high"},
        "cam_left_wrist": {"original_key": "observation.images.cam_left_wrist"},
        "cam_right_wrist":{"original_key": "observation.images.cam_right_wrist"}
    },
    "annotation": {
        "human.task_description": {"original_key": "task_index"}
    }
}

Key considerations:

• start/end follow Python slicing rules [start:end]. Fill in the values based on the degrees of freedom of your robot joints.

• video > original_key must exactly match the column names in the parquet files.

• The file can be placed in the meta/ directory of the dataset (highest priority), or specified via the --modality-json parameter.

*_config.py: Defines how the model processes data (camera selection, action representation, prediction steps, etc.).

from gr00t.configs.data.embodiment_configs import register_modality_config
from gr00t.data.embodiment_tags import EmbodimentTag
from gr00t.data.types import (
    ActionConfig, ActionFormat, ActionRepresentation, ActionType, ModalityConfig,
)

config = {
    "video": ModalityConfig(
        delta_indices=[0],
        modality_keys=["cam_left_high", "cam_left_wrist", "cam_right_wrist"],
    ),
    "state": ModalityConfig(
        delta_indices=[0],
        modality_keys=["left_arm", "right_arm", "left_gripper", "right_gripper"],
    ),
    "action": ModalityConfig(
        delta_indices=list(range(16)),
        modality_keys=["left_arm", "right_arm", "left_gripper", "right_gripper"],
        action_configs=[
            ActionConfig(rep=ActionRepresentation.RELATIVE, type=ActionType.NON_EEF,
                         format=ActionFormat.DEFAULT, state_key="left_arm"),
            ActionConfig(rep=ActionRepresentation.RELATIVE, type=ActionType.NON_EEF,
                         format=ActionFormat.DEFAULT, state_key="right_arm"),
            ActionConfig(rep=ActionRepresentation.ABSOLUTE, type=ActionType.NON_EEF,
                         format=ActionFormat.DEFAULT, state_key="left_gripper"),
            ActionConfig(rep=ActionRepresentation.ABSOLUTE, type=ActionType.NON_EEF,
                         format=ActionFormat.DEFAULT, state_key="right_gripper"),
        ],
    ),
    "language": ModalityConfig(
        delta_indices=[0],
        modality_keys=["annotation.human.task_description"],
    ),
}

register_modality_config(config, embodiment_tag=EmbodimentTag.NEW_EMBODIMENT)

Key considerations:

• modality_keys must match the key names in the corresponding section of modality.json.

• We recommend using RELATIVE for arm joints and ABSOLUTE for grippers.

• The file must end with a call to register_modality_config(..., EmbodimentTag.NEW_EMBODIMENT).

1.4 Debug training locally

Run a small number of steps on the development machine to verify that the code runs correctly:

python /home/code/Isaac-GR00T/gr00t/experiment/launch_finetune.py \
    --base_model_path /official-models/GR00T-N1.7-3B \
    --backbone_model_name_or_path /official-models/Cosmos-Reason2-2B \
    --dataset-path /data/<dataset_name> \
    --embodiment_tag new_embodiment \
    --modality_config_path /mnt/workspace/gr00t_config/my_robot_config.py \
    --modality_json /mnt/workspace/gr00t_config/modality.json \
    --output_dir /mnt/workspace/checkpoints/debug_run \
    --max_steps 100 \
    --save_steps 50 \
    --global_batch_size 4 \
    --num_gpus 1 \
    --report_to tensorboard

Key parameters:

1.5 Verify inference locally

After a few training steps, you can start the inference service on the development machine to verify that the model output is reasonable:

python /home/code/Isaac-GR00T/gr00t/eval/run_http_server.py \
    --model-path /mnt/workspace/checkpoints/debug_run \
    --embodiment-tag new_embodiment \
    --modality_config_path /mnt/workspace/gr00t_config/my_robot_config.py \
    --modality_json /mnt/workspace/gr00t_config/modality.json \
    --port 8100

• --model-path automatically selects the latest checkpoint in the directory.

• --port corresponds to the service port configured when creating the development machine.

In another terminal, verify that the inference service is running:

curl http://localhost:8100/health
# Returns {"status": "ok"} if the service is ready

1.6 Create an image

After verification is successful, package the development environment as a custom image and push it to ACR for subsequent training and evaluation.

2.1 Create a training template

1. On the Model Training page, click Start Training.

2. For Model source, select Custom model.

3. Configure the training parameters:

   • Image: Select the custom image created in Phase 1.

   • Dataset path query: View the dataset storage path. You need to manually add it to the start command or environment variables.

   • Compute resources: Select a GPU specification.

   • Start command: Enter the complete training command.

   • TensorBoard log path (optional): Enter the log directory path.

4. Click Start Training.

2.2 Monitor training

In the model training list:

• View the training status (Training / Completed / Failed).

• View training logs and resource usage through Ray Dashboard. To view training metrics such as loss curves and learning rates, click More > View Metrics in the training task list.

2.3 Download training results

After training is completed:

• Model files smaller than 200 MB can be directly downloaded through the browser.

• For larger files, we recommend that you export them to OSS and then use ossutil to download them locally.

One-click deployment for official models

1. On the Model Deployment page, find the trained model with the Idle status.

2. Click Deploy Inference Service.

3. The deployment is completed automatically. Record the Inference Service URL, which is required for simulation evaluation later.

Manual deployment for custom models

1. Manually start the inference service on the development machine.

2. Expose the service port through port mapping.

3. Record the access URL of the inference service.

4.1 Create a simulation development machine

1. On the Simulation Development page, click New simulation environment.

2. Select the unitree official simulation image (which includes Isaac Sim 5.1 + xpra).

3. Select a GPU specification and configure the SSH public key.

4. Click Confirm. After the environment is ready, log on via SSH.

4.2 Install simulation assets

1. On the Simulation Assets page, verify that the required Unitree G1 robot model is installed.

2. If not installed, click the Install button for the target asset.

3. After installation, the asset is automatically mounted to the simulation development machine.

4.3 Simulation task and robot configuration mapping

Select the matching simulation task based on your robot and training configuration:

4.4 Verify the evaluation workflow

Run the following command on the simulation development machine to start an evaluation and verify the connectivity between the simulation environment and the inference service:

python /home/code/unitree_sim_isaaclab/adb/evaluate.py \
    --serve_url http://<inference_service_address>:8100 \
    --task Isaac-PickPlace-RedBlock-G129-Dex1-Joint \
    --task_desc "Pick up the red cup on the table." \
    --robot_type g129 \
    --enable_dex1_dds \
    --n_episodes 5 \
    --max_steps 500 \
    --output_dir /mnt/workspace/eval_output \
    --device cuda:0 \
    --enable_cameras

Key parameters:

4.5 View the evaluation report

After evaluation is completed, the report is saved as eval_report.json in the directory specified by --output_dir, and contains the following key metrics:

4.6 Create a simulation image

After verification is successful, package the simulation environment as a custom image for subsequent automated evaluation.

5.1 Create an evaluation template

1. On the Simulation Eval page, click New evaluation.

2. Configure the evaluation parameters:

   • Inference service URL: The inference service address obtained in Phase 3.

   • Image: Select the custom simulation image created in Phase 4.

   • Compute resources: Select a GPU specification.

   • Start command: Enter the evaluation script command.

   • Headless: Must be enabled. One-click evaluation on the platform requires --headless mode.

3. (Optional) Click Save as Template to save the configuration for future use.

4. Click Confirm.

5.2 Monitor evaluation

View the task status in the simulation evaluation list and monitor execution details through the Ray Dashboard.

5.3 View the evaluation report

After evaluation is completed, download the evaluation result files and analyze the model performance in the simulation environment. Based on the evaluation results, decide whether iterative optimization is needed:

• Success rate does not meet the target: Return to Phase 1 to adjust the data or training parameters.

• Success rate meets the target: Proceed to the physical machine deployment phase.