STAT-12: Verification/Validation Sampling Plans for Proportion Nonconforming

This is part of a series of articles covering the procedures in the book Statistical Procedures for the Medical Device Industry.

Purpose

This procedure provides tables and instructions for selecting sampling plans for FDA process validation and design verification to ensure they are based on a valid statistical rationale.  These determine the samples size and acceptance criteria.  They make confidence statements like 95% confidence the process or device is more than 99% reliable or conforming.  These sampling plans are often referred to as confidence-reliability sampling plans.  They are for the statistical property proportion conforming or nonconforming.  They require that requirements be established for individual units of product. They apply to design verification (STAT-04), process validation (STAT-03), validation of a pass/fail inspections (STAT-08) and CAPA effectiveness checks (STAT-07).

Appendices

  1. Attribute Single and Double Sampling Plans for Proportion Nonconforming
  2. Variables Single and Double Sampling Plans for Proportion Nonconforming
  3. Selecting Sampling Plans for Proportion Nonconforming using Software
  4. Lower Confidence Limit for Percent Conforming—Attribute Data
  5. Lower Confidence Limit for Percent Conforming—Variables Data
  6. Sampling Plans for Proportion Nonconforming

Highlights

Attribute Sampling Plans

  • Appendix F of STAT-12 contains tables like the one shown below for 95%/99%  – 95% confidence of more than 99% reliable or conforming.  This is equivalent to 95% confidence of less than 1% nonconforming.  This table contains attribute single and double sampling plans.

Attribute Sampling Plans

  • 95% confidence of more than 99% conformance means there is a 95% chance of rejecting a 99% conforming product/process.  99% conforming is therefore an unacceptable level of quality designed to fail.

  • All the above sampling plans, if they pass, allow the same confidence statement to be made.  They offer the same protection against a bad product/process passing.  They are all equivalent from the customer/regulatory point of view.  They differ with respect to their sample sizes and their chances of passing a good product/process.  The decision of which confidence statement to use should be based on risk and must be justified.  The choice of which sampling plan to use for a given confidence statement is a business decision.

  • The AQLs in the above table are nonconformance levels that have a 95% chance of passing the sampling plan.  They are useful in deciding which sampling plan to use.  Historical data can be used to estimate the nonconformance rate and then matched to the AQL.  If historical data is not available, data from similar products or processes can be used.  If there is no good estimate of the nonconformance rate, stay away from the top of the table.  These are the hardest plans for a good product/process to pass.

  • The top plan, n=299, a=0 offers the lowest sample size.  It minimizes the sample size.  However, it also has the lowest AQL.  It maximizes the chance a good product/process will fail.

  • The double sampling plans have a first sample size not much greater than the top single sampling plan.  They offer a good compromise between sample size and the chance of false rejection of a good product/process.  They are generally preferred to the single sampling plans.

  • Attribute sampling plans are always applicable.  For measurable characteristics, they make no assumption about the underlying distribution of the data.  However, they have higher sample sizes.  When the measurements follow the normal distribution or can be transformed to the normal distribution as described in STAT-18, variables sampling plans can be used to dramatically lower the sample size.

 Appendix F also contains tables of variables sampling plans.  They will be covered in a separate article.  Future articles will also compare these sampling plans to other approaches including normal tolerance intervals, capability analysis and confidence intervals on capability indices.

These tables and all the procedures can be licensed individually or as a group by a company so that they can use them for or as part of there company procedures.  This requires paying a 1-time license fee as described at Company Licenses.  If your company is using these tables, please make sure they have been properly licensed.  Below is a previous version of the table.

Attribute Plans with 95% Confidence

10 thoughts on “STAT-12: Verification/Validation Sampling Plans for Proportion Nonconforming”

  1. Dear,
    Can I ask what is the relationship between the RQL and PPK, as I find RQL 0.1%, with corresponds to a Ppk of 1.03 in other document. How to convert between them?

    1. The Ppk values associated with the RQLS in STAT-12 based always on the one-side case. 3*Ppk corresponds to the number of standard deviations to the nearest spec. Assuming normality, the percent nonconforming is calculated based on the normal distribution in Excel as: =100*NORM.DIST(-3*Ppk,0,1,TRUE)

      1. Hello Taylor,

        Your explanation gave me the idea and I have got it. Z bench= (USL-u)/σ=3Ppk. Then I use the Z bench to calculate the defect. Thank you.

    2. It simplifies things if α is fixed at 0.05 and β is fixed at either 0.05 (95% confidence for validation) and 0.1 (90% confidence for validation and for manufacturing). Then you just adjust the AQL and RQL to adjust the protection.

      Whenever one selects sampling plans, both the AQL and RQL are important and should be considered. When validating a process, it is important to demonstrate the process meets the AQL used in manufacturing. One would not want to have an AQL of 1% in manufacturing and pass a process running 3% nonconforming in manufacturing. It is common for validation to use an RQL set equal to the AQL in manufacturing. This gives 95% confidence the manufacturing AQL is meet. The validation sampling plan offers far greater protection. This is the difference.

      1. Hello Taylor,
        The following steps are my understanding if I want to choose a validation plan after reading your instructions. If I am wrong, please point out. Thanks. I have this question for very long time and hope to solve it under your help.

        Step 1: use the historical data or similar process or aligned AQL with customer to define the nonconforming rate, for example, about 1% non-conformance. Then 1% will be used as the RQL when defining the validation sampling plan. Meanwhile, define the β, e.g. 5%.

        Step 2: According to the non-conformance rate in step 1, and define the AQL used in validation sampling plan. Any AQL that no more than the non-conformance rate in step 1 can be used. E.g. 0.13% AQL or any rate between 0 to 1%. Meanwhile, define the α risk

        After these two steps, we can have the validation sampling plan either from your provided table or use the software to calculate.

        1. It is a two-step process as you describe. To clarify your descriptions:

          Step 1: Select Validation RQL (β=5% corresponds to 95% confidence). Select RQL to match the manufacturing AQL, which in turn is based on risk/severity. Passing the plan then demonstrates the manufacturing AQL is met and most lots should pass.

          Step 2: Select Validation AQL. Match it to an estimate of expected performance based on historical data, when possible.

          Then go to my tables.

  2. For a sampling size, if we have multiple detection methods for a single failure, can we divide the sample size by detection method? for example for 95%/95% and sample size of 60, there are 60 data points required (not sample size). If we have 2 detection methods, per say, observing damage and discoloration; can we use 30 samples?

    1. When you have multiple defect types of the same severity, the expectation is that the sampling plan be applied to the group. The 95%/95% plan would be to take n=60 samples and accept if there are zero nonconforming units. A nonconforming unit is a unit with one or more nonconformities.

  3. Hi Wayne,

    From what I understand, sampling plan using AQL/RQL is ok to be used for “process validation”. But, I am not so sure if it can be used for “design verification/validation”?

    There are several sources that advise against the use of AQL/RQL sampling plan for “design verification/validation”, unfortunately I couldn’t grasp the rationale. May I know your thoughts?

    Thanks.

    1. During design verification, you are trying to demonstrate the entire design (range for the design outputs) works. The best way to demonstrate this is to test the limits of the design space. This can be done by worst-case testing. It can also be done by analysis by performing a worst-case tolerance analysis. The issue with a sampling plan is the samples may not push the limits of the design outputs.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top