Log query-Dataphin(Dataphin)-阿里云帮助中心

View and download application logs

Filter and download logs

In the top menu bar of Dataphin Manager, click Application O&M.
In the navigation pane on the left, choose Application O&M > Log Query.
On the Application Log Search tab, filter logs as follows.
- Keyword: Enter a keyword with at least 5 characters and click Search. The search progress for each application subdomain is displayed below.
- Application subdomain name: Select one or more application subdomains from the drop-down list. You can also enter a keyword to search for subdomains.
- Log type: Select one or more log types. Options include application, rpc, nginx, jvm, audit, and mesos logs. The mesos log option appears only if the scheduling cluster is Mesos. By default, application, rpc, nginx, and jvm logs are selected.
- File time range: Select Last 1 day, Last 2 days, Last 3 days, or Last 5 days. The default is Last 1 day.
- Display context: Enter the number of lines to show before and after the keyword. The value must be an integer from 1 to 500.
Click Filter and Download Logs to download the filtered logs.

Full log download list

Click Download Full Logs. In the Select for Download dialog box, select an application and click OK to create a download task.
Click Download Task List. The list shows the Application Name, Pod Name, Download Time, and Download Progress for each task. In the Actions column for a task, click Download to download the application logs to your local machine.
Note
- By default, you can only download folders that are smaller than 1 GB before compression.
- Full log downloads are compressed files from the /home/admin/logs folder. You can download logs for only one application at a time.
- Logs in the download task list are stored in Object Storage Service and are kept for a maximum of 3 days.

End-to-end log search

On the End-to-end Log Search tab, enter an instance ID that starts with t to search. The instance ID is available in the instance O&M list. The results display the ordinal number, instance creation time, and last update time. The ordinal number indicates the retry count. The system retries tasks automatically based on retry settings, but a manually triggered rerun runs only once.

Click Expand next to an ordinal number, then click Execution Log, Task Scheduling Log, Resource Scheduling Log, or Executor Log to view details in the code area on the right. Click the icon next to a log type to download it. Click Download next to an ordinal number to download all log types for that instance.

Execution Log: Records task instance execution details.
Task Scheduling Log: Records task scheduling activity for the instance.
Resource Scheduling Log: Records resource scheduling activity for the instance.
Executor Log: Records executor activity while the task instance runs.

Pod log search

On the Pod Log Search tab, select a Namespace (required), a Pod (required), a Log Start Time, and a Line Limit to search for pod logs. The matching logs are displayed in the code block below.

Namespace: Namespaces are retrieved automatically from Kubernetes and include those required for Dataphin deployment. On a shared Kubernetes cluster, namespaces from other services also appear. Dataphin typically uses the following namespaces:
- Dataphin: The namespace for the Dataphin application.
- Dataphin-schedule-pool: The namespace for Dataphin task scheduling.
- stolon: The self-built PostgreSQL metadatabase for Dataphin.
- redis: The self-built Redis for Dataphin.
- rocketmq: The self-built RocketMQ for Dataphin.
- rook-ceph: The self-built Ceph Object Storage for Dataphin.
Pod: Select a pod within the specified namespace.
Log Start Time: Select a time in YYYY-MM-DD hh:mm:ss format. Only log entries after the selected time are returned. Dataphin application logs support time-based filtering. For other logs, time-based filtering works only when log lines start with a YYYY-MM-DD hh:mm:ss timestamp.
Line Limit: The number of lines to display in the log details. Enter an integer from 1 to 1000.