False Positives, Errors, and Assumptions

ABSTRACT

There are three main categories into which most errors fall in drug evidence evaluations. These error categories can be summarized as comprising:

      1) false positives;

      2) instrumental analysis errors; or

      3) incorrect assumptions made by analysts.

Some errors fall into more than one category on this list. I will present examples of each of these types of errors in the discussion on drug evidence below. 

Given that most drug analyses are performed by analysts called by the prosecution to testify, these types of analytical errors and assumption errors will favor the prosecution more often than they will favor the defendant. It is therefore helpful to have such analyses and assumptions re-evaluated by an independent expert before trial to bring to light areas where false-positives, instrumental errors or incorrect assumptions may have occurred.

INTRODUCTION:

The introduction of chemical evidence into a courtroom record usually requires the presence of an expert to testify about the relevance of any chemical evidence to the subject of the trial. Experts, however, are human beings and can make mistakes or err in their judgment about their techniques or their results. They are prone to blind spots in their thinking, just as the rest of humanity is. 

There are three main categories that comprise the common errors found in drug analyses.  These are: false positives; instrumental analysis errors; and incorrect assumptions made by an analyst.  This presentation shows a number of areas where chemical evidence can be flawed but still brought into a courtroom by experts who have not double-checked the results of their assumptions; or have not double-checked for errors in their instrumentation or techniques; or have not eliminated the possibility of false-positives from their analyses.

FALSE POSITIVES:

False positives occur when a drug analysis shows the presence of a certain drug in a sample when there actually is none of that drug present in that sample.  These errors can occur for a variety of reasons ranging from cross-contamination of samples with other drug samples; to decomposition of a material during an analysis which leads to the production of a controlled substance during the analysis itself.

An example of cross-contamination can arise when a sample containing a drug is analyzed on a GCMS instrument and the injector piece in the instrument is not rinsed adequately with a solvent after that sample is analyzed; if the injector needle is not cleaned before the next sample is analyzed by the instrument, then the second sample can show the presence of the drug that was in the first sample even though the second sample actually had no drug in it.  This type of error can be removed by rinsing an injector with a solvent after each sample is run and running a solvent blank before each sample of suspected chemical evidence is injected.  The possibility of error is not removed, however, if the injector is only rinsed with air and the air injection is used as a blank before the suspected sample is run.  Air does not dissolve most chemical samples so would not show that the injector needle was still dirty before the next injection was made.

An example of a false positive created by decomposition during a GCMS run is given below.  In this analysis by GCMS, a sample containing ephedrine (an isomer of pseudo-ephedrine, the active ingredient in Sudafed) was analyzed in an instrument where the injector port was too hot.  Under these conditions, a small amount of ephedrine decomposition occurred and this created a detectable amount of methamphetamine in the material that was injected onto the column of the instrument.  The instrument detected this methamphetamine and the analyst assumed that the sample had methamphetamine in it.  A subsequent retest under lower-temperature conditions showed that no methamphetamine was present in the ephedrine sample.

​INSTRUMENTAL ANALYSIS ERRORS:

Instrumental analysis errors can occur when analytical instruments such as balances, spectrometers, and other types of equipment are not set up correctly or are uncalibrated when they are used.  If the readouts from these pieces of equipment are not compared to those from standard samples of materials that are of known weight, concentration, identity, etc., before the instruments are employed to analyze unknown samples of evidence, the results that these uncalibrated instruments record can be inaccurate and show that too little or too much of a substance is present compared to what the actual amount is.  This is especially important when weighing samples on a balance; the balance must be leveled and calibrated to known weights before it is used to measure the weights of unknowns.

In other examples involving these types of errors, certain analytical instruments such as GCMS or LCMS instruments that are typically used to identify the presence of a scheduled substance in an evidence sample can give incorrect results for retention times, for example, if they are not first standardized using a set of substance standards with known identities, retention times, etc., using a given set of flow and column conditions.  Unknown samples of a controlled substance need to be compared to known, standardized samples of the suspected controlled substance which are run under identical conditions in order to measure and compare retention times and fragmentation patterns of the substances under these conditions.  Errors can occur if the methods employed for the analysis of an evidence sample and a controlled substance standard are different from one another.  Such errors can occur, for example, when the standard is run on one machine and the evidence sample is run using another machine using a different set of columns or conditions which lead to different outputs.  These changed outputs can possibly lead to mis-identification of what substances are actually present in an evidence sample; for example, an isomer of methamphetamine called N-ethylphenethylamine might be mis-identified as methamphetamine under different sets of conditions using two different instruments.  This would be an incorrect analytical result that would later be relied upon in court if no one double-checked the procedures used for this analysis.

​INCORRECT ASSUMPTIONS

By far, the largest numbers of errors involving the analysis of controlled substances arise because analysts periodically make incorrect assumptions about the utility of the methods or techniques that they employ in their work. 

Many people think that the methods they use for analysis can do things or prove things that actually cannot be proven using those methods. 

Assumptions are also made about the chemical properties of mixtures in order to estimate the volumes or weights of a given amount of substance in an evidence sample; but these assumptions, upon later inspection by an independent chemist, turn out to be incorrect.

Other areas where incorrect assumptions are commonly made involve the testing of the compositions of substances in confiscated evidence samples.   For example, an analyst might mistakenly make the assumption that four baggies of white powder are all the same substance; so, from this assumption, it would follow that the analyst only needed to analyze the mixture in one baggie in order to prove the identity of all the substances in all four of the baggies.  This is a false assumption and needs to be pointed out to the analyst.  None of the baggies needs to be related to any of the other baggies, in actuality.  In this case, if the contents of one baggie were analyzed and found to contain a controlled substance but the other three baggies were not analyzed, another expert could testify that the only amount of controlled substance that was proven to be present was that of the white powder in the baggie that was tested.  The weights of substances in the other three baggies would not have been shown to contain any controlled substance at all so their weights would have to be ignored for sentencing purposes if an independent expert testified to that in court.  There would be no proof of the contents inside the other three baggies; just a guess on the part of the analyst.

This is also true for pharmaceutical pills in many cases.  Just because one pill looks identical to ninety-nine others does not necessarily mean that it is identical; it does not have to have identical materials present inside of it. 

With the proliferation of pill-making equipment, colored powders for formulations, other ingredients and molds for shaping pills, it is now possible to create counterfeit versions of pharmaceutical pills and tablets that look identical to the actual commercially-available products.  These counterfeit products can contain other ingredients that are not controlled substances or they may contain other types of controlled substances which might mimic the effects of the actual pharmaceutical ingredients that produce the desired effects but which are not used in the commercially-available pharmaceutical product.  Testing one pill and extrapolating to predict the contents of another ninety-nine pills or more is not science.  It is being lazy.  As an expert, one must be able to ascertain the errors associated with the analytical processes one uses and one cannot do that with only one data point out of a set.  More work needs to be done in order to ascertain an error rate in a case such as that.

OTHER EXAMPLES:

Some of the other issues that can come up involve the use of presumptive tests in ways that they are not designed for.  For example, presumptive tests such as color tests can be used to show which samples of evidence might have controlled substances in them and which samples do not.  But without a more exacting analysis using GCMS or LCMS, a presumptive test cannot be considered conclusive for the presence of a controlled substance.  Even though this is true, some analysts will testify that a certain drug is present based entirely on the results of a presumptive test; this type of error should be countered by the defense using the testimony of another expert to explain to the court why such a presumptive test, by itself, was inconclusive.

SUMMARY

There are many types of scientific errors that can be found by going through the analytical printouts and laboratory notebooks in a drug case.  Some of the more common types have been presented in this discussion.  Defense attorneys should be aware of the fact that laboratory personnel do not always do perfect jobs and make mistakes from time to time when they are working in the lab.  Unless an attorney has been trained in scientific methods of analysis, he or she would be best served by hiring an independent chemistry expert to evaluate the processes and results that a forensic laboratory has summarized for the prosecution in a drug case.