By configuring log forwarding, you can send logs from Deep Learning Containers (DLC) jobs in your workspace to a specified Logstore in Simple Log Service (SLS) for custom analysis.
Prerequisites
If you use a RAM user to configure SLS log forwarding, you must grant the AliyunLogFullAccess permission to the RAM user. For more information, see Grant permissions to a RAM user.
Procedure
Go to your workspace.
-
In the top navigation bar, select a region.
In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the workspace to manage.
Log on to the PAI console.
-
From the Configure Workspace drop-down list on the right, select Configure SLS.
The log forwarding feature supports both user logs and system logs from DLC jobs. You can forward these logs to a custom Logstore in Simple Log Service (SLS) to facilitate end-to-end monitoring, operations, and diagnostics for training jobs based on system logs. On the Configure SLS page, set the following parameters and click OK.
Parameter
Description
SLSProject
A resource management unit in Simple Log Service used for resource isolation and control. If no projects are available, click Create SLS Project to create one. For more information, see Manage projects.
LogStore
A unit in Simple Log Service for collecting, storing, and querying log data. If no Logstores are available, click Create Logstore to create one. For more information, see Manage a Logstore.
Modules that require SLS storage
Select the Deep Learning Containers (DLC) checkbox.
FAQ
Q: Empty list when configuring SLS log forwarding
Error message: An 'Unauthorized' error occurs with the message: denied by sts or ram, action: log:ListProject, resource: acs:log......
Cause: The user does not have permission to read the SLS Logstores.
Solution: Grant the required permissions to the user. Follow these steps:
Log on to the Resource Access Management (RAM) console. In the left-side navigation pane, choose Permissions > Grants, and then click Grant Permission.
Set the Principal to a RAM user or RAM role, and select the
AliyunLogFullAccesspolicy.For fine-grained SLS permission control, go to the Log Service console. In the project to which you want to grant permissions, click the More Applications icon, choose Permission Assistant, and generate a custom RAM policy.
Q: Request fails when configuring SLS log forwarding
Error message: Modify configuration failed [SLS] cannot init client for sis service: com.alibaba.pai.workspace.common.exception.ServiceExceptionV2: No Privilege error: {0}
Cause: The user does not have permission to enable or disable log forwarding for the SLS Logstore.
Solution: Log on to the Resource Access Management (RAM) console and create a custom policy to grant log forwarding permissions to the user.
Use the following custom policy. For instructions, see Create a custom policy.
{
"Version": "1",
"Statement": [
{
"Effect": "Allow",
"Action": [
"log:GetProductDataCollection",
"log:CloseProductDataCollection",
"log:OpenProductDataCollection"
],
"Resource": "*"
}
]
}For more fine-grained control, replace the value of Resource with the specific resources you want to authorize.