STAT-04: Statistical Techniques for Design Verification

This is part of a series of articles covering the procedures in the book Statistical Procedures for the Medical Device Industry.

Purpose

Design verification studies are confirmatory studies to ensure the product design performs as intended.  They make pass/fail decisions as to whether the product’s design outputs (specifications, drawings) ensure each design input requirement (requirements definition document) is met or not.  The procedure determines which design requirements require statistical methods to verify them and then helps to select the best method.

Note:  It is the goal of Process Validation to show the process is capable of meeting the specifications for the design outputs.  It is the goal of design verification to show that the entire specification of the design output result is product that works (meets the design inputs).

Design Output

Highlights

    Sometimes Statistical Methods are not Required – When no variation

  • Rules for deciding if testing 1 unit is sufficient.  This is for requirements where there is no variation in performing the test.  One example is logic testing of software where each test is performed once because the same result will occur each time the test is performed.  In software validation, the focus is identifying all the logic paths and testing each one once as part of a test script.  This option and approach also applies to other features such as color and presence of a feature.

   There are  Several Alternatives to Sampling Plans – When there is variation

  • Demonstration by Analysis.  Examples of Demonstration by Analysis include tolerance analysis, finite element analysis, and models developed by designed experiments.  Testing is sometimes, but not always required, to demonstrate the model is predictive.  For example, a confirmation run at the optimal settings following a designed experiment.  The model is then used to demonstrate that as the design outputs vary over the design space, the product performance meets the design inputs.
  • Worst-case testing allowing 1-5 units to be tested at each worst-case setting.  Worst-case conditions are the settings for the design outputs that cause the worst-case performance of the design inputs. When worst-case conditions can be identified and units can be precisely built at or modified to these worst-case conditions, a single unit may be tested at each of the worst-case conditions. This ensures the design functions over the entire specification range.  This approach is generally preferable to testing a larger number of units toward the middle of the specification range.  When units cannot be precisely built at or modified to these worst-case conditions, multiple units may have to be built and tested.

   Instructions for Selecting Sampling Plans 

  • Sampling plans are selected based on the confidence statement that can be made if they pass.  For example, 95% confidence more than 99% of the units meet the requirement (denoted 95%/99%).  When a sampling plan is required, instructions are provided for linking the confidence statement to risk using a table like the one below.
Product Risk/Harm Level Design Verification Reduced Level (Stress Tests)
High 95% / 99% 95% / 95%
Moderate 95% / 97% 95% / 90%
Low 95% / 90% 95% / 80%
  • For each confidence statement, numerous sampling plans exist.  For example the following table from STAT-12 contains several attribute single and double sampling plans that all make the same 95%/99% confidence statement.
Type Parameters AQL
Single n = 299, a = 0 0.017%
Double n1 = 320, a1 = 0, r1 = 2, n2 = 256, a2 = 1 0.069%
Single n = 473, a = 1 0.075%
Double n1 = 327, a1 = 0, r1 = 2, n2 = 385, a2 = 2 0.094%
Single n = 628, a = 2 0.13%
Double n1 = 327, a1 = 0, r1 = 3, n2 = 582, a2 = 3 0.15%
Single n = 773, a = 3 0.18%
Double n1 = 330, a1 = 0, r1 = 3, n2 = 719, a2 = 4 0.18%

Strategies for Reducing the Sample Size – For when a sampling plan is used 

  • Variables data – having a measurement instead of attribute pass/fail results, allows variable sampling plans to be used.  They require as few as 15 samples in contrast to the minimum of 299 above.
  • Stress testing – testing a small number of units using a method that induces more failures than expected in the field.  Note the stress test column in the table above.   This can include design margin as described in Appendix A of STAT-03.
  • Multiple tests on the same unit.  A sample size is 30 means 30 tests are required.  Under certain circumstance if may be possible to test 3 units 10 times each.

Leave a Comment

Your email address will not be published. Required fields are marked *