On Conformity Testing and the Use of two Stage Procedures

Conformity testing is a systematic examination of the extent to which an entity conforms to specified requirements. Such testing is performed in industry as well as in regulatory agencies in a variety of fields. In this paper we discuss conformity testing under measurement or sampling uncertainty. Although the situation has many analogies to statistical testing of a hypothesis concerning the unknown value of the measurand there are no generally accepted rules for handling measurement uncertainty when testing for conformity. Usually the objective of a test for conformity is to provide assurance of conformity. We therefore suggest that an appropriate statistical test for conformity should be devised such that there is only a small probability of declaring conformity when in fact the entity does not conform. An operational way of formulating this principle is to require that whenever an entity has been declared to be conforming, it should not be possible to alter that declaration, even if the entity was investigated with better (more precise) measuring instruments, or measurement procedures. Some industries and agencies designate specification limits under consideration of the measurement uncertainty. This practice is not invariant under changes of measurement procedure. We therefore suggest that conformity testing should be based upon a comparison of a confidence interval for the value of the measurand with some limiting values that have been designated without regard to the measurement uncertainty. Such a procedure is in line with the recently established practice of reporting measurement uncertainty as “an interval of values that could reasonably be attributed to the measurand”. The price to be paid for a reliable assurance of conformity is a relatively large risk that the procedure will fail to establish conformity for entities that only marginally conform. We suggest a two‐stage procedure that may improve this situation and provide a better discriminatory ability. In an example we illustrate the determination of the power function of such a two‐stage procedure.