**This is part of a series of articles covering the procedures in the book Statistical Procedures for the Medical Device Industry.**

### Purpose

This procedure provides tables and instructions for selecting sampling plans for FDA process validation and design verification to ensure they are based on a valid statistical rationale. These determine the samples size and acceptance criteria. They make confidence statements like 95% confidence the process or device is more than 99% reliable or conforming. These sampling plans are often referred to as confidence-reliability sampling plans. They are for the statistical property proportion conforming or nonconforming. They require that requirements be established for individual units of product. They apply to design verification (STAT-04), process validation (STAT-03), validation of a pass/fail inspections (STAT-08) and CAPA effectiveness checks (STAT-07).

### Appendices

- Attribute Single and Double Sampling Plans for Proportion Nonconforming
- Variables Single and Double Sampling Plans for Proportion Nonconforming
- Selecting Sampling Plans for Proportion Nonconforming using Software
- Lower Confidence Limit for Percent Conforming—Attribute Data
- Lower Confidence Limit for Percent Conforming—Variables Data
- Sampling Plans for Proportion Nonconforming

### Highlights

**Attribute Sampling Plans**

- Appendix F of STAT-12 contains tables like the one shown below for 95%/99% – 95% confidence of more than 99% reliable or conforming. This is equivalent to 95% confidence of less than 1% nonconforming. This table contains attribute single and double sampling plans.

- 95% confidence of more than 99% conformance means there is a 95% chance of rejecting a 99% conforming product/process. 99% conforming is therefore an unacceptable level of quality designed to fail.
- All the above sampling plans, if they pass, allow the same confidence statement to be made. They offer the same protection against a bad product/process passing. They are all equivalent from the customer/regulatory point of view. They differ with respect to their sample sizes and their chances of passing a good product/process. The decision of which confidence statement to use should be based on risk and must be justified. The choice of which sampling plan to use for a given confidence statement is a business decision.
- The AQLs in the above table are nonconformance levels that have a 95% chance of passing the sampling plan. They are useful in deciding which sampling plan to use. Historical data can be used to estimate the nonconformance rate and then matched to the AQL. If historical data is not available, data from similar products or processes can be used. If there is no good estimate of the nonconformance rate, stay away from the top of the table. These are the hardest plans for a good product/process to pass.
- The top plan, n=299, a=0 offers the lowest sample size. It minimizes the sample size. However, it also has the lowest AQL. It maximizes the chance a good product/process will fail.
- The double sampling plans have a first sample size not much greater than the top single sampling plan. They offer a good compromise between sample size and the chance of false rejection of a good product/process. They are generally preferred to the single sampling plans.
- Attribute sampling plans are always applicable. For measurable characteristics, they make no assumption about the underlying distribution of the data. However, they have higher sample sizes. When the measurements follow the normal distribution or can be transformed to the normal distribution as described in STAT-18, variables sampling plans can be used to dramatically lower the sample size.

Appendix F also contains tables of variables sampling plans. They will be covered in a separate article. Future articles will also compare these sampling plans to other approaches including normal tolerance intervals, capability analysis and confidence intervals on capability indices.

These tables and all the procedures can be licensed individually or as a group by a company so that they can use them for or as part of there company procedures. This requires paying a 1-time license fee as described at Company Licenses. If your company is using these tables, please make sure they have been properly licensed. Below is a previous version of the table.

Qian JianDear,

Can I ask what is the relationship between the RQL and PPK, as I find RQL 0.1%, with corresponds to a Ppk of 1.03 in other document. How to convert between them?

Wayne TaylorThe Ppk values associated with the RQLS in STAT-12 are based on the one-sided case. 3*Ppk corresponds to the number of standard deviations to the nearest spec. Assuming normality, the percent nonconforming is calculated based on the normal distribution in Excel as: =100*NORM.DIST(-3*Ppk,0,1,TRUE)

Qian JianHello Taylor,

Your explanation gave me the idea and I have got it. Z bench= (USL-u)/σ=3Ppk. Then I use the Z bench to calculate the defect. Thank you.

Wayne TaylorIt simplifies things if α is fixed at 0.05 and β is fixed at either 0.05 (95% confidence for validation) and 0.1 (90% confidence for validation and for manufacturing). Then you just adjust the AQL and RQL to adjust the protection.

Whenever one selects sampling plans, both the AQL and RQL are important and should be considered. When validating a process, it is important to demonstrate the process meets the AQL used in manufacturing. One would not want to have an AQL of 1% in manufacturing and pass a process running 3% nonconforming in manufacturing. It is common for validation to use an RQL set equal to the AQL in manufacturing. This gives 95% confidence the manufacturing AQL is meet. The validation sampling plan offers far greater protection. This is the difference.

QIAN JIANHello Taylor,

The following steps are my understanding if I want to choose a validation plan after reading your instructions. If I am wrong, please point out. Thanks. I have this question for very long time and hope to solve it under your help.

Step 1: use the historical data or similar process or aligned AQL with customer to define the nonconforming rate, for example, about 1% non-conformance. Then 1% will be used as the RQL when defining the validation sampling plan. Meanwhile, define the β, e.g. 5%.

Step 2: According to the non-conformance rate in step 1, and define the AQL used in validation sampling plan. Any AQL that no more than the non-conformance rate in step 1 can be used. E.g. 0.13% AQL or any rate between 0 to 1%. Meanwhile, define the α risk

After these two steps, we can have the validation sampling plan either from your provided table or use the software to calculate.

Wayne TaylorIt is a two-step process as you describe. To clarify your descriptions:

Step 1: Select Validation RQL(β=5% corresponds to 95% confidence). Select RQL to match the manufacturing AQL, which in turn is based on risk/severity. Passing the plan then demonstrates the manufacturing AQL is met and most lots should pass.Step 2: Select Validation AQL.Match it to an estimate of expected performance based on historical data, when possible.Then go to my tables.

QIAN JIANHello

Frankly speaking, sampling is an interesting topic and also a very difficult one. Maybe I cannot understand every aspect. But what you do give me much help. Thanks.

calvin shiraziFor a sampling size, if we have multiple detection methods for a single failure, can we divide the sample size by detection method? for example for 95%/95% and sample size of 60, there are 60 data points required (not sample size). If we have 2 detection methods, per say, observing damage and discoloration; can we use 30 samples?

Wayne TaylorWhen you have multiple defect types of the same severity, the expectation is that the sampling plan be applied to the group. The 95%/95% plan would be to take n=60 samples and accept if there are zero nonconforming units. A nonconforming unit is a unit with one or more nonconformities.

ChrisHi Wayne,

From what I understand, sampling plan using AQL/RQL is ok to be used for “process validation”. But, I am not so sure if it can be used for “design verification/validation”?

There are several sources that advise against the use of AQL/RQL sampling plan for “design verification/validation”, unfortunately I couldn’t grasp the rationale. May I know your thoughts?

Thanks.

Wayne TaylorDuring design verification, you are trying to demonstrate the entire design (range for the design outputs) works. The best way to demonstrate this is to test the limits of the design space. This can be done by worst-case testing. It can also be done by analysis by performing a worst-case tolerance analysis. The issue with a sampling plan is the samples may not push the limits of the design outputs.

tonyfor design verification studies, as I understand it this methodology supports the sample size justification, but how does it affect the acceptance of the study itself? meaning, how does this compare to statistical Tolerance Limits analysis or bionomical probability? or would Ppk/Pp replace the need?

Wayne TaylorThe table includes the acceptance criteria along with the sample size. Only sampling plans for attribute data are covered here based on the binomial. The book also contains tables for variables data based on Ppk/Pp, which are equivalent to normal tolerance intervals through the relationship k = 3 Ppk.

JBQuestion regarding Table F3 of STAT-12:

for a single 90/70 attribute plan with RQL0.10 = 30% non conforming. What would a and AQL % be for a sample size of 50?

Thanks,

JB

Wayne TaylorYou can use Sampling Plan Analyzerr to answer this question. Enter a single sampling plan with n=50 and increase the accept number until RQL

_{0.10}is just below 30%. The answer is n=10 and AQL = 12.86.StephanieHello Dr. Taylor,

I am trying to validate an attribute test method. Using your tables for a a secondary method, single sampling plan, at 90% confidence and 95% reliability it directs me to n-45, a =0. AQL =0.11% and LTPD0.1 = 5%. Firstly, it suggests separate sampling plans for alpha and beta errors. Alpha must be no less than 40% of samples. Does that mean I must run two validations with 45 samples each or two validations with 18 for alpha and 27 for beta sampling plans? Secondly, a=0 means no defects allowed, but how do I interpret the AQL 0.11% and LTPD 5%? Thanks for your help

Wayne TaylorThere are two errors that can be made: (1) falsely accepting a nonconforming unit and (2) falsely rejecting a conforming unit. The 90%/95% plan n=45, a=0 means to inspect n=45 nonconforming units and accept if there are a=0 false acceptances. My book provides alternative 90%/95% plans including double sampling plans that decrease the chance of a false failure of the TMV.

My book recommends not having an acceptance criterion for false rejections. Otherwise, a separate sampling plan is required for false rejections.

The 5% in LTPD

_{0.1}= 5% is a 5% false acceptance rate meaning 95% reliability. The 0.1 is a 10% chance of an incorrect decision meaning 90% confidence. LTPD_{0.1}= 5% is the same as 90%/95%. This is an unacceptable false acceptance rate the sampling plan is designed to reject. The AQL is a false acceptance rate the sampling plan is designed to accept. 0.11% means 99.89% of nonconforming units are rejected. This is how good the inspection must be to be assured of passing the TMV.Carla QuirosHi! The company that I work for adquired your book and I was looking into one of the acceptance criteria. I just wanted to understand the relationship between the number of samples with the required ppk for each confidence/reliability. For example, how is it that we can say it is statistically valid that for a sample size of 15, with 90% reliability and 95% confidence, we require a ppk of 0.69?

Wayne Taylorn=15, Ppk=0.69 is a variables acceptance sampling plan. The more common form for a variables sampling plan is n=15 and k=2.07=3 Ppk. ANSI Z1.9 contains such plans and information can be found on calculating the OC curve. This is also the same as a normal tolerance interval with parameters n and k with an acceptance criteria of passing if the tolerance interval is inside the spec limits. They are all identical procedures.

The article at variation.com/confidence-statements-associated-with-sampling-plans/ shows how the OC curve relates to the confidence statement. Your plan has RQL

_{0.05}= 10%.Ray SmithHello Dr. Taylor,

For design verification, when establishing a variables single sampling plans for proportions, n and Ppk are two parameters considered for a one-sided specification. In Appendix B of STAT-12 there is pass criteria given for a one-sided-only lower specification limit (LSL) involving Ppk. How does this correlate to the tables in Appendix F? For example, for “95/97 Variables – One-Sided Specification” table multiple sample sizes and Ppk values are given. How do I select the appropriate parameters to compare to the pass criteria given in Appendix B? Do I start with the smallest sample size (n=15) shown?

Wayne TaylorThe 95/97 table in Appendix F and STAT-12 has two columns. The fist column is the Parameters column. It provides the sample size (n) and acceptance criteria (P

_{pk}) of eight different sampling plans that allows the 95/97 statement to be made if they pass. The second column is the AQL column. This column is used to decide on which plan to use. The AQL column tells you how high the actual P_{pk}must be to be assured of passing the sampling plan. As an example, suppose previous data estimates the P_{pk}is 0.98. Matching this to the AQL column results in the sampling plan n=40 and P_{pk}=0.81.Atul SpehiaHello Taylor,

I tried using Minitab’s Variable acceptance sampling plan feature to generate a sampling plan for double sided specification and single sided specification. But the sampling plan i.e number of units to be tested and K- factor is same for both single side and double side specification. The only difference is that i have an additional requirement of passing the MSD. Is it ok to use ?

Wayne TaylorThe n and k given by Minitab for variables sampling plans are correct for 1-sided sampling plans. They should not be used for 2-sided sampling plans. The additional MSD requirement is an assumption for the true standard deviation and is not part of the acceptance criteria for individual lots. My book Statistical Procedures for the Medical Device Industry gives 2-sided variables plans with acceptance criteria for MSD using the estimated lot standard deviation. This results in different n and k values. The other option is to use the n and k values for a 2-sided normal tolerance interval with no criteria on MSD.