Conducted by MoniQA WG Validation of qualitative methods, this activity is a joint undertaking between IUPAC and MoniQA and aims to develop a harmonised protocol for validation of qualitative methods. Qualitative methods are understood to be those that do not provide a measurable answer to a given question (e.g., dipstick assays, PCR). Typically, these analyses render yes/ no responses testing for the presence/ absence of a substance at given limit. MoniQA aims to establish guidelines for validation of qualitative methods and the performance of validation studies. To achieve this, model studies have been performed using data from a variety of validation studies to investigate the minimum/optimum numbers of tests necessary. More specifically, two topics have been addressed; (1) method performance characteristics of qualitative methods and (2) specific challenges for the validation of qualitative methods including (a) qualitative method where the evaluation of results is based purely on a qualitative – yes/ no – basis and (b) qualitative methods based on a quantitative measurement.
Focus has been on qualitative methods using real-world examples of method validation to which statistical assessments can be applied. Special emphasis has been placed on predicting error in the percentage of correct results at estimated levels of dosage. The expected outcome is a harmonised IUPAC-MoniQA protocol that will form the basis of guidelines to validate qualitative methods. In 2009, progress towards these guidelines was made with the (1) collection of real-world examples suitable for running statistical simulations, (2) selection of appropriate statistical tools for the simulations, and (3) completion of the first statistical simulations on real-world examples as model cases. These activities were undertaken in collaboration with SAFEED-PAP, which provided the concept and statistical data from a validation study of qualitative method coordinated in SAFEED-PAP.
The aim of the SAFEED-PAP validation study was to assess whether real-time PCR was able to provide results fit-for-purpose with respect to detection of MBM in animal feed. Measures of fitness included (1) estimated false positive probability when a sample does not contain MBM and (2) estimated limit-of-detection for MBM (i.e., minimum concentration at which the probability of detection reaches a high level, 95%). Estimates were obtained by examining the relationship between concentrations of MBM in samples and the proportion of samples that produce a positive response. Confidence intervals, which reflect variation between laboratories, were also estimated and allowed prediction of likely (with 95% confidence) worst-case performance (limit-of-detection, false positives) for a competent laboratory using a given method. This was achieved by fitting a beta-distribution to a set of observed proportion of positive responses (one per laboratory) at each concentration. The 5th and 95th percentiles gave upper and lower limits for probability of a positive response, which could be expected in an individual laboratory. A linear interpolation between concentrations was used to find the concentration at which 95% probability of detection could be expected, and a confidence interval for this concentration reflects between-laboratory variation.
Contact: Christoph von Holst (christoph [dot] von-holst [AT] ec [dot] europa [dot] eu), JRC/ IRMM, BE
More information: see pdf below
Printer-friendly version