The problem with forensics

If you’re a fan who’s been watching the new season of TV show ‘Line of Duty’ - or perhaps you’re just familiar with crime dramas and courtroom thrillers - you’ll know they make a big deal out of forensics. Fingerprints and blood spatters tend to catch the killer. But is real life forensic science as ironclad as it seems on screen? Duke University’s Brandon Garrett says an emphatic 'no'. His new book Autopsy of a Crime Lab is designed to expose the many ways in which forensic evidence is - in his words - “dangerously imperfect”. Chris Smith asked him what he means...

Brandon - I certainly thought that forensics was a triumph of modern technology, and certainly preferable to alternative types of evidence. In my early work I represented people who had been exonerated by DNA testing, and modern forensics corrected terrible miscarriages of justice in their cases. Over time, as I studied their cases, I realised, “well, wait a minute - a lot of these people had been convicted in the first place based on traditional forensics.” And moreover, it's not just the egregious mistakes in arson cases, or egregiously unreliable forensics like bite-mark evidence, or sort of junk science techniques; but a lot of the bread and butter of what crime labs mostly do these days still involves a lot of subjective judgment, and no statistics, no research behind these techniques, and poor quality control at the crime scene. We invest much more care and science into a strep test or a COVID test than we do in a test that could put someone in jail for the remaining days of their life.

Chris - Someone once said to me - not a criminal, I might add, a scientist who wasn't a criminal - that perhaps the best thing to do would actually be to go and find the nearest vacuum cleaner and empty the vacuum cleaner bag all over the environment, thus contaminating it with enormous amounts of DNA from all the people who've ever walked through the environment the vacuum cleaner has been used on. And in this way, you would make the life of the laboratory technician who tried to analyse that so impossible that you'd never be caught.

Brandon - And that's a reason why DNA testing just can't be used in most criminal cases. Contamination is a real problem. You don't need the dusty vacuum cleaner - just breathing on the crime scene is enough to contaminate, if you're not wearing a mask. But unfortunately most of what labs do is not DNA testing. And there are very different types of contamination: just poorly dusting and lifting the print; not knowing where to look; contamination through information, telling the fingerprint examiner, “by the way, this guy confessed, he has a record as long as my arm.” That kind of cognitive contamination we see all over the place.

Chris - Tell us about the fingerprints question then, because obviously that's one of the oldest forms of forensics. What's wrong with fingerprints then?

Brandon - There's a lot of detailed information in fingerprints. We all know it from looking at our prints. Although there's a reason why the makers of smartphones stopped using fingerprints to open phones: you put your finger on it five times, and it sometimes wouldn't open. And one reason why was that the fingerprint reader was partial. It doesn't read your entire fingerprint. Those phones have a little small surface, maybe like one fifteenth or one twentieth. But that's the problem in criminal cases - you don't have a perfect pristine fingerprint, with lots of information, you'll have a smudged partial. And so you're trying to compare something with really good information - a print from a suspect - to something at the crime scene. And how you connect those two pieces of evidence involves a lot of judgment. But no statistics! We have no idea how common it is for people to share small parts of their print. We have no idea how common it is for someone to leave a smudged print that in part looks a lot like someone else's. And so fingerprint examiners for years said their work was perfect, there was a zero error rate, because fingerprints are unique. They never addressed this question of this highly imperfect, subjective comparison process. And they never provide statistics. They said, “this print came from that person, it's a match, it's an identification.” And in the absence of any statistics, they actually did not know how rare or common it was to see some level of similarity.

Chris - So how many people then, based on what you're finding, are potentially getting the wrong result because of the use and abuse of forensics?

布兰登-从我们所知道的,从the few studies that have been conducted - under enormous pressure from the scientific community in recent years - there are really high error rates for some of these disciplines. And not error rates of one in millions, but error rates like one in dozens or one in hundreds. And given the number of cases in which there are forensics done every year, it could be enormous numbers. One of the many audits and scandals going on in the US right now is in Massachusetts, where they're reopening 70,000 cases involving drug analyses. We're about to see what will happen in Washington DC, where their firearms unit is under scrutiny and the lab has had its accreditation pulled. We have no idea how many people may have been convicted based on the work of that firearms unit. So whenever we pull threads at errors, we find thousands and thousands of cases that are tainted. We have no idea how many people's lives may have been upended by erroneous forensics.

Chris - Do you know what, I'm really quite shocked and alarmed by what you're saying, because I probably was among the group of people who would assume that when we introduce evidence to the courtroom, and tests, we would apply the same rigor to it that we would, for instance, in the hospital, with the diagnostic test. We introduce the test only when we understand exactly in the hospital, how well it performs, what it's likely to miss, what it's likely to pick up, and therefore how likely we are to be right. So why is a different criteria set applied to forensics then?

Brandon - Scientific laboratories are regulated! They'll give them blind tests, and if they don't correctly diagnose strep throat, or COVID, there may be an investigation or they'll check the equipment, they'll check whether the clinicians are well-trained, or doing their jobs correctly. There are a whole host of regulations, certainly in the United States, that regulate any clinical laboratory. And that just isn't true for crime labs. They've been sort of let go: “oh, that's policing, that's practical, applied stuff. We don't have to have quality control.”

Chris - Can a lawyer not challenge this then Brandon? Can you not then, you're in court and someone comes up with all this evidence, and someone like you would stand up and say, well, read Brandon Garrett's book. It shows that this is flawed and unreliable. So this evidence should be discounted.

布兰登- ar可能没有律师挥舞着我的书ound in court, because they get their certificate saying, “oh, the fingerprint, it's an identification.” They get no documentation surrounding how the work was done. In some disciplines, like in firearms, there is no documentation. There's no way for the lawyer to know how accurate the work was - they can't even know what the examiner did. So they get the certificate, and certainly in the US most cases are plea bargained; they won't have access to their own expert to look at the evidence, there's no second opinion. And so their client pleads guilty. The client may say, “look, I'm innocent. I don't understand. The fingerprint, the firearms - that wasn't me! And the lawyer says, “well, come on, look at what the lab said. And they're offering you three years - take it.” And so they take it. And so it's rare for this stuff to be challenged in court. And then when it is, judges have sort of had a roll over and play dead approach to forensic science.

克里斯,我really sincerely hope that they do read your book, Brandon. And that has seriously dented my trust. Brandon Garrett, and his book Autopsy of a Crime Lab is out now. Anna, did you look at any, from a materials point of view, issues related to forensics in your book? Because you must've left a few fingerprints on a few pots that you tried to make!

Anna - Yeah, definitely. So my book actually stemmed from a podcast of the same name, Handmade, where I've interviewed over 70 different craftspeople and makers. Some of them made it into the book, but one that I interviewed recently is a jeweller turned forensic scientist called Maria MacLennan, based up in Scotland. And her work is all about looking at jewellery taken from crime scenes, and trying to find identifying factors in those pieces. So it could be, the type of silver that was used is particularly prevalent in certain areas; or it could be the cultural value, where that jewellry came from; maybe a silversmith left their particular mark on a piece, which can be traced then back. So her work is really about looking at jewellery, seeing what it could say about a person in terms of perhaps a victim of a crime, and using material science and forensic techniques to be able to provide evidence to the police.