Burning Question #4: What Happens If I Violate a Run Rule?

Welcome back to our ongoing EPA Tier 3 Informational Series. Stay knowledgeable with our previous Burning Questions related to:

See ASTM guidelines for dealing with a violated run rule.

What happens if I violate a run rule?
What is ASTM proposing and is it likely to change what I have to do?

According to D6299-17, Section 8.3, Individual and Moving Range charts are recommended along with either Strategy 1 (run rules) or Strategy 2 (EWMA). Strategy 1 and 2 are intended to detect small levels of sustained shifts or drifts of the complete analytical system. We know that individual values outside of the upper control limit (UCL)/lower control limit (LCL) range are strong indications of an out-of-control system and trigger an investigation for assignable cause(s).

What needs to be done if a Strategy 1 or Strategy 2 violation is detected?

First, let's define Strategy 1 and Strategy 2.

Strategy 1

Strategy 1 includes the following checks:

Two out of three consecutive results on the I chart that are more than 2σR’ from the center line in the same direction.
Five consecutive results on the I chart that are more than 1σR’ from the centerline in the same direction.
Nine or more points in a row above or below the centerline on the I chart.
Seven points in a row steadily increasing or decreasing.

A violation of any of these checks is interpreted as a strong signal that a change in state of the measurement system has likely occurred.

Strategy 2

Strategy 2 involves:

The calculation of an EWMA trend line using the measurement values and equations A1.16 and A1.17. Equations for UCL and LCL limits are further detailed in equations A1.18 and A1.19, respectively. If the EWMA trend line exceeds the specified UCL and LCL, again it shall be interpreted as a strong signal that a change in state of the measurement system has likely occurred.

What does the phrase “strong signal that a change in state of the measurement system has likely occurred” really mean?

The Annex to the method (A1.5.14) provides some specific language when discussing Strategy 1 and 2:

“one of the following strategies shall be used to detect changes in state of the measurement system that are considered to constitute an out-of-control situation.”

So what can you do?

Section A1.5.3 states that if the control chart data exhibit a strong signal of change in the state of the measurement system, investigate for root causes. If this investigation leads to a significant change in the measurement system (e.g., recalibration or service), reset the run rule counts or restart the EWMA. However, if the investigation does not lead to a significant change in the measurement system, continue with the current control chart but treat results as suspect and use with great caution.

ASTM Ballot D02 (17-01) issued on February 27, 2017, and closed on March 29, 2017, includes a number of proposed changes, including the interpretation of Strategy 1 and 2. While this ballot is not final and has not become part of the method at this date, it would appear that ASTM is considering the following additions/changes to D6299-17:

Explicitly stating that Strategy 1 run rule violations declare a system out-of-statistical control
Explicitly stating Strategy 2 EWMA exceedance of UCL or LCL declare a system out-of-statistical control
Addition of an EWMA run rule where 15 or more consecutive EWMA data that remain on one side of the center line without violating the EWMA control limit declare a system to be out-of-statistical-control
Addition of a Moving Range (MR) strategy where any MR value that exceeds the upper control limit (UCL) should be investigated as a cautionary event and any root cause identified documented
Addition of a MR strategy where five or more out of the past twenty MR values that exceed the UCL declare a system to be out-of-statistical-control.

Furthermore, the ballot provides a new section (A1.5.5) that describes what actions may be taken if a system is found to be out-of-statistical control” The intent is to allow more flexibility with respect to possible actions. The key additions list possible actions that include:

reassessing the system for bias and precision,
analyzing a reference material or check standard, and
considering the statistical, practical and regulatory significance of any bias.

There is acknowledgement that corrective action may depend on the relative magnitude of the bias and other considerations such as consequences for no action.

Need a refresher on what we mean by terms like "bias"? Check out our key terms and definitions of ASTM D6299.

Thanks for reading!

Chris