DocsICP FilingConsole
Documentation
Home Page Simple Log Service Technical Solutions Monitor Simple Log Service FAQ: New job monitoring dashboard

FAQ: New job monitoring dashboard

Updated at:
Copy as MD
Product
My Favorites

Common questions about the job monitoring dashboard for new jobs, including the new versions of data shipping to Object Storage Service (OSS) and MaxCompute, and data import.

Error on the job monitoring dashboard

Symptom

When you open a job's overview page in the target project, a The query failed. error appears, and the statistical reports are empty. The data shipping job status is running, but an orange Query failed error message appears in the upper-right corner of the job monitoring dashboard: key (job_name) is not config as key value config, if symbol : is in your log, please wrap : with quotation mark. In the Data Processing Traffic section, the six panels—Successful Reads, Failed Reads, Read Public Traffic (MB), Successful Writes, Failed Writes, and Write Public Traffic (MB)—all display "No Data". The processing rate line chart shows 0, and the Lagging Progress and Running Exception panels at the bottom also display "No Data".

Cause

Job operational logs are stored in a Logstore named internal-diagnostic_log within the specified project. If the index in this Logstore is deleted or outdated, Simple Log Service cannot monitor jobs in the project.

Solution

Find the job monitoring dashboard in the CloudLens for SLS application for the project that stores job operational logs, and then reset the index. To identify the project that stores the job operational logs, see Preparations.

  1. Log on to the Log Service console.

  2. In the Log Application section, click the Cloud Service Lens tab, and then click CloudLens for SLS.

  3. In the left-side navigation pane, choose Report Center > Job Monitoring.

  4. In the upper-left corner of the page, select the project where your logs are stored.

  5. In the upper-right corner of the page, click Index Reset.

  6. In the dialog box that appears, click OK.

No data on the job monitoring dashboard

Symptom

When you open the overview page for a job in the target project, no data is displayed in the statistical reports. The data shipping job status is running, but in the Statistical Report, all monitoring panels in the Data Processing Traffic section—including Successful Reads, Failed Reads, Read Public Traffic, Processing Rate, Successful Writes, Failed Writes, Write Public Traffic, Lagging Progress, and Running Exception—display "No Data", and the processing rate is zero.

Cause

Log collection for job operations is not enabled for the project. Without operational log data, Simple Log Service cannot populate the job monitoring dashboard.

Solution

Find the project that contains the job and enable job operational log collection for that project.

Use CloudLens for SLS to enable log collection for job operations in the target project.

Procedure

  1. Log on to the Log Service console.

  2. In the Log Application section, click the Cloud Service Lens tab, and then click CloudLens for SLS.

  3. If you have not enabled CloudLens for SLS, a page appears. Click Enable.

    The page prompts you to authorize a RAM role named AliyunServiceRolePolicyForSLSAudit. Simple Log Service uses this role by default to manage data collection. After you confirm the authorization, click Enable.

    After the feature is enabled, a success page appears. For more information, see Enable the log collection feature.

    In the left-side navigation pane, click Access Management. In the SLS Collection Rule List, you will find four built-in collection rules: internal_cloudlens_sls_error_log (error logs), internal_cloudlens_sls_audit_log (audit logs), internal_cloudlens_sls_monitor_metric (monitoring metrics), and internal_cloudlens_sls_project_run_log (operational logs). The Resource Matching Mode for all rules is All Resources, and their Status is Rule Enabled.

View job operational logs

After you enable log collection for job operations, Simple Log Service stores the operational logs of new jobs in the internal-diagnostic_log Logstore within your specified project. To view job operational logs in this Logstore:

  1. Log on to the Simple Log Service console.

  2. In the project list, click the target project.

    This is the project that you specified when you enabled log collection for job operations.

  3. On the Log Storage > Logstores tab, click internal-diagnostic_log.

  4. View the job operational logs.

    You can search for the operational logs of a specific job by its name. The query statement is job_name:Job Name, for example, job_name:job-1646****946.

    For a detailed description of the fields in job operational logs, see Operational logs of new data transformation, import, and shipping jobs.

    In the Logstore list on the left side of the Simple Log Service console, select internal-diagnostic_log and click Raw to view the raw logs. The diagnostic logs for a data shipping job (connector) contain a _etl_:connector_meta block with fields such as action, connector, instance, task_name, and task_type, and a _etl_:connector_metrics block with fields such as state, lags, events, failed, native_bytes, events_bytes, req_count, and rep_time. They also include common fields such as instance_id, job_name, and metric_type.

Previous：Project monitoringNext：Developer Reference