Failure Review

Failure Review Specification

Failure review covers the fault handling process, improvement analysis, and responsibility assignment. It uses a standardized review SOP with preventive action recommendations and an accountability mechanism to retrace production faults, produce review reports, and drive improvements that prevent recurrence.

The review follows this standard process:

• Process review: Apply the 5-why method to examine the incident in depth. For example: Why did this fault occur? Why was it not detected earlier? How did each team respond? Where can the process be optimized?

• Problem analysis: After the process review, perform a deeper analysis. Were there process, quality inspection, product, or system design issues? Are there stronger defense mechanisms? How can recurrence be prevented?

• Experience summary: After root cause analysis, identify practical actions—short-term mitigations, long-term fixes, and lessons learned.

• Responsibility assignment: After analyzing causes and improvement measures, determine the fault level and assign responsibility. Teams are categorized as primary responsible, secondary responsible, and testing responsible.

• Improvement tracking: Without effective implementation, review results go to waste. Define clear improvement action plans with owners and completion deadlines.

• Actions must follow the SMART principle:

  • Specific: What exactly needs to improve? Which metrics or indicators should be optimized?

  • Measurable: What acceptance criteria apply? Which metrics determine whether the improvement succeeded?

  • Attainable: Is the improvement achievable? Avoid goals that cannot realistically be implemented.

  • Relevant: Does this improvement connect to other initiatives? Avoid isolated efforts.

  • Time-bound: When should the improvement be completed? Aim to finish within three months to prevent improvements from becoming formalities.

• A complete action record includes: title, planned completion time, owner and assisting team, acceptance method and acceptor, tracker, improvement category, description, and acceptance criteria. After implementation, conduct acceptance through review or simulation. The acceptor then closes the improvement action.

A review document typically includes:

• Fault Summary: Fault overview, impact, handlers, etc.

• Fault Background: Business context at the time of the fault.

• Fault Timeline: Chronology of fault introduction, occurrence, discovery, response, recovery execution, and resolution.

• Fault Cause Analysis: One-sentence summary followed by a detailed cause analysis.

• Fault Process Analysis: Analysis covering requirements evaluation, code release, and emergency response.

• Follow-up Improvement: Improvement measures with clear direction and responsible owners.

• Fault Level/Responsibility: Based on the fault level definitions above, assign a level and identify the responsible team and owner.

Failure Data Operation

Analyze and present fault data across different dimensions and channels—report platforms, production safety reports, and production safety meetings—both online and offline. Use historical fault data to measure system stability and operational capability. The goal of fault data operations is to drive quantitative assessment that reduces overall fault occurrence.