Series two of The Recommended Dose kicked off this week with polymath and poet, Dr John Ioannidis. Recognised by The Atlantic as one the most influential scientists alive today, he’s a global authority on genetics, medical research and the nature of scientific inquiry itself – among many other things. A professor at Stanford University, John has authored close to 1,000 academic papers and served on the editorial boards of 30 of the world's top journals. He is best known for seriously challenging the status quo. His trailblazing 2005 paper Why Most Published Research Findings Are False has been viewed over 2.5 million times and is the most cited article in the history of PLoS Medicine. In it, he argues that much medical research is biased, overblown or simply wrong. Here, he talks to Ray about the far-reaching implications of these findings for people both inside and outside the world of health.
While most closely associated with exploring cutting-edge conundrums across science, genomics and even economics, John is also something of a humanist. He’d be right at home with the philosophers of ancient Greece, seeking as he does to find answers to the big questions of the day in science and medicine, as well as in nature and narratives. A voracious reader himself, John has a lifelong love of ‘swimming in books’ and has penned seven literary works of his own in Greek – two of which have been nominated for prestigious literary prizes. And fittingly, he finds inspiration for his myriad of multi-disciplinary pursuits on Antipaxi, one of Greece’s most beautiful and secluded islands. He shares some of his distinctive logic, reason - and even a little of his poetry - on this very special episode of The Recommended Dose. Produced by Cochrane Australia and co-published with the BMJ.
Links to papers covered in the show:
Disclosures in Nutrition Research: Why It Is Different, JAMA, 13 February 2018
What Happens When Underperforming Big Ideas in Research Become Entrenched, JAMA, October 2016
Evidence-based medicine has been hijacked: a report to David Sackett, Journal of Clinical Epidemiology, May 2016
Is everything we eat associated with cancer? A systematic cookbook review, American Journal of Clinical Nutrition, November 2012
Why Most Published Research Findings Are False, PLOS Medicine, 30 August 2005
Links to assorted profiles of/articles about John:
The man who did the math, Tufts Now, June 2018
When evidence says no but doctors say yes, The Atlantic, February 2017
John Ioannidis has dedicated his life to quantifying how science is broken, Vox, February 2016
Lies, Damned Lies, and Medical Science, The Atlantic, November 2010
John Ioannidis: Uncompromising gentle maniac, The BMJ, September 2015
The Truth Wears Off: Is There Something Wrong with the Scientific Method?, The New Yorker, December 2010
| AAA |
Episode transcript:
John Ioannidis:
There are some settings and some locations, and some situations where we're under-utilising the best options that we have, but in most developed countries and clearly in the US, we're just spending so much and we're wasting so much that healthcare is one of the leading public dangers for health. So it's a major threat and something that everyone who's interested in health needs to fight against. I think at some point we need to fight against medicine. It's becoming really dangerous.
Ray Moynihan: Echoing that age-old adage first do no harm, that’s one of the most influential scientists on the planet, John Ioannidis.
Hello, I’m Ray Moynihan and welcome to a very special conversation that opens season two of The Recommended Dose, the podcast encouraging a healthy scepticism towards health care - funded by Cochrane Australia and co-published with the BMJ.
John Ioannidis is a professor at Stanford University in the United States. He’s a global authority on genetics, on scientific evidence and on research about research. And he has that rare capacity to speak in language we can all understand. And to deeply questions. He’s published more scientific papers than you can poke a stick at, he’s served on the editorial boards of 30 leading journals, and in his spare time he’s written seven books of literature including this poem about a man looking back at all the disasters in his life, which we’ll hear in full at the close of this podcast.
John Ioannidis: He decided to make a cool appraisal of all the disasters that had afflicted him. To attempt a lifelong evaluation. A reasonable and moderate, precise recording of calamities. First his birth, a huge disaster, an enormous woe.
Ray Moynihan: They were the opening lines of a poem that John has promised to read us at the end of this interview. Apart from disasters, John, there's been a lot of successes in your life too. One of them I want to talk about, one of your most famous papers - Why Most Published Research Findings are False. Now that's free online, anyone can see it at PLOS medicine. I checked and I think it's been viewed over two and a half million times already. There's some tough mathematics in there, but can you give us a very brief, simple summary of what you were saying in that article?
John Ioannidis: Well, I think that this paper is a success about a disaster. It is trying to understand how likely it is that a new research finding, a new study that appears in the scientific literature would be wrong. And it is using some mathematical modelling to try to account for the fact that we're performing studies that may be small, that may be trying to chase pretty subtle effects sometimes, that there are many researchers who are trying to compete against each other rather than collaborate necessarily to try to answer these questions, that there is a multiplicity of a very large number of questions and hypotheses that may be targeted. That there are biases in the process and that the final product may be pretty uncertain. So that mathematical modelling is also juxtaposed against what empirical evidence we have from how different types of designs and different types of settings of conducting research have done in terms of how often we get results that can be replicated in additional studies versus something that cannot be replicated or seems to be very exaggerated.
Ray Moynihan: So in a nutshell, what did you argue in that piece?
John Ioannidis: Practically, if you go through the calculations, you see that in most circumstances it's very difficult to get it right with the first shot. You know with the first paper, the first time that you see something being published. Especially if you have small studies. If you're chasing small effects, if you have more bias, if you have conflicts of interest, financial or other. If you have very complex databases. Lots of freedom on how to analyse the data.
All of these factors contribute towards getting you literature that would be replete with false findings. And in most scientific fields, including of course medicine, and in most medical fields, probably the majority of these first discoveries and first passes and first results, first study insights that we get are false results.
Ray Moynihan: So are you saying that a lot of the studies, a lot of the claims we hear let’s say in the media – whether it’s a link between genetic defects and an illness, or it’s about the benefit of a new drug – are you saying that they’re simply not right?
John Ioannidis: Many of these claims are indeed not right. And I think that it varies a lot from one field to another. The two examples that you mentioned, genetics and drug effects could be pretty different.
In genetics, 10 years ago or more than that, almost everything that you would hear would be wrong because people were running very small studies and they were trying to cherry pick hypotheses that had very weak statistical support. So almost everything was wrong. Moving forward currently almost everything that you hear in genetics about big studies with extensive collaboration, with multiple teams, sharing their data with rigorous statistics, with a very careful analysis and sharing of all the data that are generated, plus replication across multiple teams - almost everything that you will hear is likely to be correct once these recipes are in place. The question is still whether this is useful, but that's I think a different question.
For drug effects, I think that there is quite a gradient. There is some trials about the effects of medications that can be very liable because they're very well done. They're randomized and carefully protected from bias and they have the appropriate outcomes and there's no conflicts of interest in the background. So I would trust those. There's also a very large proportion that are highly unreliable and they can also be big disasters.
Ray Moynihan: What kind of reception did that paper get? I mean it was a very serious attack on a lot of the work that a lot of researchers do.
John Ioannidis: I think that I have avoided having people threatening to assassinate me or a really fierce opposition. The good thing about this paper and also about much of the work that I do and other people are doing in meta-research, or research on research, is that very often we're trying to have a bird’s-eye view of what is happening and not focused on trying to shame anyone or trying to claim that this is the single horrible study that the investigators need to be ashamed of. I think that as scientists, we are trained to listen to arguments and I think we can tolerate the fact that millions of papers may be wrong. It's more difficult to tolerate if we're told that our single paper, our own single paper is wrong. So I think that probably this is why I'm still alive and have not been assassinated.
Ray Moynihan: Well I’m very glad you’re still alive. One of the things you’ve argued I think is that we need to change the way we reward scientists, so we start rewarding more the quality of what’s done and not just the quantity. Is that right and if so how?
John Ioannidis: I think that this is important. The rewards and incentive system is what will drive the production of whatever we cherish, whatever we think is valuable. So if we reward people for publishing more papers, this is exactly what we will get. We will get more papers. If we reward people for being successful to get more money for their research, we'll have probably mostly managers recruited to science who will be very good at making sure that they just get all the money that there is.
If conversely, we reward people for doing rigorous work, for being very careful in thinking about biases and how to eliminate them or at least reduce them, in really thinking about hard questions and a rigorous methods to attack them I think it's likely that we will get more of that. So it's where we invest and what we prioritise and what we believe is essential. And I believe that clearly quality is more important than quantity. I have nothing against quantity, but we need secure quality of course.
Ray Moynihan: One of the things we talk about a lot on this podcast is evidence, the importance of evidence. One of the recent articles I'd like to talk about, your recent articles is called Evidence Based Medicine Has Been Hijacked. Now, before we talk about that extremely provocative title, just a quick reminder briefly, what is evidence-based medicine?
John Ioannidis: So, evidence based medicine is an effort to combine the best possible evidence with an individualised approach to the patient and to the clinician judgment and his or her interaction with a patient. It has these two components. It has a systematic approach to the external evidence, to the scientific literature, to science, to the products of research. And then you have the physician patient encounter, the interaction, which may be different in each case.
As an idea, I think it's a fabulous idea and it's something that probably antedates the coining of the word and the effort to push evidence-based medicine forwards in the last 25 years. However, it has been hijacked because there's lots of vested interests and there's lots of stakeholders who are trying to use evidence or distorted evidence to serve their agendas. And I think this is where the problem arises.
Ray Moynihan: Can you give me an example of one of the vested interests that you think are hijacking evidence based medicine?
John Ioannidis: So first of all, there's very strong financial conflicts of interest. There's lots of stakeholders in medicine and health. We're talking a multi-billion dollar market. Sometimes even for single drugs and we're talking about trillions of dollars when we come to the entire health and disease related market. There's lots of companies that even though they may have R&D agendas, they also have very strong marketing agendas, they really want to be leaders in that market. And that leads unavoidably to some very strong conflicts on what kind of evidence they will try to produce and how exactly they will try to disseminate it and use it to maximise their profits.
There's even conflicts from physicians, from all of us who are involved in healthcare. It's very difficult sometimes to disentangle from the fact that we're specialists that we have some reason of existence and our reasons of existence may be related to doing some particular procedures or doing some type of medicine that may be threatened by evidence.
How likely is it that we will design studies that will show that what we do is something that needs to be abandoned, that we need to change jobs, that, we're not relevant, we're not needed in order for people to have better health. I think that adds another layer of a conflict that is very difficult to disentangle.
Then you have lots of non-financial conflicts. You may have people who want to defend their theories. They want to defend their expertise. You have a lot of expert based medicine that might be masquerading as evidence based based medicine or lots of eminence based medicine that is trying to present itself with the tools of evidence based medicine.
Ray Moynihan: There's so much talk about. I mean, I think I'd recommend that piece to anyone who hasn't read it. Evidence Based Medicine Has Been Hijacked. It's actually, I think I'm right in saying, an extraordinary critique of how medicine has lost its way in some ways. I mean, you actually say in there that in some places we have too much medicine, that healthcare itself has become a threat to human health. Why do you say that?
John Ioannidis: Absolutely. I think that there's both a overuse and under use of medicine in healthcare. There are some settings and some locations, and some situations where we're under-utilising the best options that we have, but in most developed countries and clearly in the US, we're just spending so much and we're wasting so much that healthcare is one of the leading public dangers for health. It's possible that our society will disintegrate just because we're wasting too much on trying to do too much that has very little evidence or even has evidence that it is unnecessary. So it's a major threat and something that everyone who's interested in health needs to fight against. I think at some point we need to fight against medicine. It's becoming really dangerous.
Ray Moynihan: It's extraordinary to hear someone like you saying that. Is it time, do you think for some kind of global commission to investigate and try and reduce the harms of healthcare?
John Ioannidis: I think we need a global response and we also need a local response. Clearly there are some issues that are permeating the entire global community and when it comes to, for example, trying to accumulate the best evidence and make sense of it, summarise it and try to see what it says we need to have the global evidence. At the same time, there's local challenges in each health system has its own balances and checks and some positive aspects and some negative aspects. So how exactly to fight the war in the trenches of each healthcare system is something that's worthwhile studying beyond the level of having a global response of better evidence, more transparent evidence, more unbiased evidence, which is obviously a high priority.
Ray Moynihan: So do you have ideas on how health systems can actually start to tackle the problem of the dangers of healthcare?
John Ioannidis: I think this can be done at multiple levels. One level is to try to make sure that we have evidence that matters and evidence that is unbiased. If you look across what Cochrane has accomplished, we have all these thousands of systematic reviews. If you go back and check what are they telling us, the most common conclusion is that we don't really know. There are exceptions, but it's only a minority of the questions that do matter, that have high quality evidence and evidence that is pretty conclusive for action.
So at a first level we need evidence to be aligned with what really matters. We need useful clinical research. We need research that can make a difference and that means that there needs to be a problem rather than create a problem because very often we are creating problems that don't exist by shifting the definition of disease and making people seem that they are sick, even though they're perfectly fine. We need to see what we already know and build on that. We need studies that will give us information, what I call information gain. We need to remove conflicts of interest, we need to think about the value of the investment in getting that evidence. We need to be pragmatic, we need to listen to patients and what their priorities are rather than what our priorities are, and of course we need to protect ourselves from optimism, from futility, just designing a Towers of Babel type of studies that go nowhere and need to be abandoned or have studies that are biased.
So usefulness is one dimension and another dimension is focusing on who is going to do that. Currently, most of that research is done funded by the industry. And I have nothing against industry. We need industry and actually they need to liberate their resources to do the real research that then would need to be tested by stakeholders who are unbiased.
So the current situation is that governments are putting a lot of money trying to do translational research, to develop targets for companies then to develop drugs to test their effectiveness and their safety. Conversely, I think that it's the companies who should be doing the translational research because they would have every incentive to try to develop the very best technologies that would have the best chances and then have these technologies be drugs or devices or biologics or whatever, be tested by independent bodies with independent funding, you know, with government funding, with public funding, that can really tell us what really works and what does not work. If we have that, probably we have a more fair starting point and we can build on that.
Ray Moynihan: That recommendation echoes something that the editor-in-chief of the British Medical Journal, Fiona Godlee, told us on the Recommended Dose some time ago. I mean, she argued that it seemed to her that industry should stop funding studies of their own products. Essentially that's what you're arguing too.
John Ioannidis: I fully agree. I think that it would be good for them as well. Because it does cost them tons of money. Their agenda unavoidably is inefficient because it becomes more of a marketing agenda rather than a research agenda. So it's a huge waste for governments, it's a huge waste for the industry and of course it's a huge waste for people and for patients.
Ray Moynihan: If a study is funded by a company with an interest in the outcome, should there be a sort of an automatic red flag? I'm not saying we should ignore the results, but in your mind, should there be an automatic red flag applied to the findings of that study?
John Ioannidis: I think that we have to be a bit cautious here because it doesn't mean that every study that is funded by the industry is wrong. In fact if you look at different types of quality assessment scales that have been applied over many years, currently, research that is funded by the industry, very often scores better on average compared to research that is not funded by the industry. But this is very obvious why this is the case. If you're paying a lot of money and this is your product and you're waiting to make tens of billions of dollars out of this effort, you don't really want to have people like me or some other crazy people like me trying to come up with scales that will tell that you know, your study was flawed.
So company trials, they will do their best to check off the checklist and get a good grade in that checklist that might be available to assess their trial. However, where the difference arises is mostly on how the study is designed. What exact question it is posing. How are the comparator arms are being chosen? Are we dealing with kind of straw man comparators that are being chosen? What kind of outcomes are being chosen? Again, this could make a huge difference on whether the results are going to be quote unquote successful and eventually how the study will be disseminated and promoted in the public view. So these studies may sound perfect sometimes, but they're just not asking the questions that are of real interest and a real importance.
Ray Moynihan: While we're on conflicts of interest, John, I think you've recently argued that for scientific research in the nutrition field, there should be a different higher standard for disclosure of potential biases. So I think you've argued scientists should disclose their personal dietary biases or their personal diets. Why is that?
John Ioannidis: So what I argued in that paper in JAMA along with my colleague John Trepanowski is that we should give the opportunity to investigators to disclose very strong personal preferences. There's no way to disclose a personal preference unless one wants to do that. Exception, if someone is an advocate or an activist. If this is not personal, this is public. If there's public advocacy or activism about something, this needs to be disclosed in the paper. It means that someone feels very strongly about it and has already expressed that in the public eye. Therefore, that disclosure is necessary and it's not different from what he or she does already by being an advocate in the public domain.
In the situation where you don't have public advocacy, but you have someone having a very strong opinion about a particular diet, I think that we should encourage that person to disclose that particular preference because again, it's a situation where someone has some entrenched belief that this diet or this nutrient or this approach to nutrition is the best to go.
I'm not saying this to shame anyone or to put anyone to shame. It could be actually an act of courage and an act of getting credit down the road when everybody, who knows, down the road realises that that was the best choice and it was indeed something that everyone should adopt.
Ray Moynihan: So in other words, if I'm just quietly a vegetarian or if I'm just quietly someone who likes eating the Mediterranean diet, you know, then you're not expecting that to be disclosed.
John Ioannidis: No way.
Ray Moynihan: But if I'm on the public record as very strongly advocating the Mediterranean diet than you think that should be disclosed.
John Ioannidis: Absolutely.
Ray Moynihan: Great.
John Ioannidis: And in many cases, this even goes to indirect financial conflicts. So you have lots of public advocates of particular diets who have non-profit foundations. Supposedly they're not making money out of it, but if someone has a non-profit foundation that does receive a lot of money and has a high visibility and/or publishes books and becomes famous out of that, shouldn't really people know about it?
[MUSIC]
Ray Moynihan: You’re listening to a conversation with John Ioannidis, a professor of medicine at Stanford who may be one of the most influential scientists alive, according to a profile in The Atlantic magazine. He’s speaking with us today on The Recommended Dose from California.
[MUSIC]
Ray Moynihan: Among your key interests is genetics or genomics. For those of us who are not inside that world can you briefly explain what you mean by genomics?
John Ioannidis: So, genomics is a research agenda that is trying to understand how genes affect our lives. How they affect our health, risk of disease, the manifestations of disease, and what we can make of it. There's a sense that this is our genes and therefore maybe we should just take it for granted, but this is not the case. I mean, currently we can affect our genome, plus there's lots of environmental influences that interact with genes and therefore if we modify the environmental influences, then possibly we could affect the way that these genes affect our lives. So in principle, human genomics is a very interesting field and it could be pretty useful.
Ray Moynihan: We hear more and more enthusiastic promotion of how genomics is going to revolutionise healthcare, could bring enormous benefits. There's a lot of promises, a lot of optimism. Do you think we should all be feeling optimistic?
John Ioannidis: So I'm not necessarily a pessimist, but I'm pretty sceptical about what we have achieved. Clearly, we have achieved a lot of information gathering. We have a pretty solid knowledge base about genomics that has grown very rapidly over the last 20 years. We have learned a lot about the biology, about the complexity of many diseases. We come to realise that many of the common diseases that affect very large numbers of people are highly genetic in nature. However, they are also extremely complex. They're very convoluted. It's not that there's a one or a few targets or some low hanging fruit that we can just take care of somehow and get rid of, you know. They're very, very convoluted and very complex. Having hundreds and thousands of genes inter playing and who knows how many environmental influences.
There was a lot optimism early on that it will be an easy ride, that we will easily understand the genetic underpinnings and then just work on them and have major benefits. This hasn't happened. There are a few cases, there is a few examples, probably where a screening for our genes is useful for some diseases that have very strong genetic risk and where there's something to do about it. However, in the vast majority, that information is not really going to tell us much at the moment in terms of how to change our life or how to change our interventions or medical treatments if it comes to treating or preventing a disease.
So it's still a work in progress. I'm increasingly sceptical about its utility. It's absorbing a tremendous amount of funding and I think I have spent some of that funding myself as well (laughs). But in terms of its utility, I think that maybe we have to revisit what exactly we're doing and maybe we still need to work on some genetics, genomics theme, but a try some new avenues. Some avenues that are not explored yet. Try some high risk ideas rather than the ones that we have been pursuing for many, many years now.
Ray Moynihan: One of the articles that you published with colleagues a couple of years ago that resonated with me and I'm sure many other people, seems very relevant to mention here. It was called What Happens When Underperforming Big Ideas in Research Become Entrenched. You essentially seem to debunk the idea that the combination of genetic science and information technology that's been around for a while now is going to dramatically improve human health. Is that right?
John Ioannidis: Yeah, absolutely. I think that both of these ideas are very promising. They have many ramifications. They have many potential disciplines of science who are currently working on them. They're absorbing tens of billions of dollars in funding, but they haven't delivered yet. And I think that probably they will not deliver much unless we radically change our approach to what are we trying to accomplish. So I think that it's very difficult to change course because once you have some ideas take 50 or 60 or 70 per cent of their research budget, you have hundreds of thousands of scientists absorbed in them. This is their career, this is their labs, this is their universities, their institutions. And it's not easy to say overnight that this is it, you know, just go home (laughs). And you don't want to do that because these are extremely smart people and very hardworking people and they're really doing their best.
But how do we disengage, or how do we say, please try something that is very different? We've tried that, we have been hammering that nail again and again, and we will keep spending tens of billions of dollars getting nowhere or getting very little out of that. This is a big question and I think that we need to find ways to really make the best of this very smart workforce, but also give it the opportunity to diversify and to start attacking questions that have a higher chance of yielding something that is useful rather than the ones that we have invested so much, getting very little.
Ray Moynihan: Going back to what you said before about the dangers of medicine, the dangers of healthcare - a previous guest on The Recommended Dose, Paul Glasziou, has said that in his view, the genetic testing of healthy people is a looming disaster and could cause a tsunami of over-diagnosis. What do you think of that view?
John Ioannidis: I would think that it's more likely to be mostly useless rather than necessarily a disaster. Although I like the word disaster as you can tell, and I wouldn't be surprised if it ends up being a disaster. I would generalise it to include any type of testing, so it could be genetic, it could be any sort of testing that currently can be done in a more massive scale. When we're trying to deal with people who have no health problem, feel perfectly healthy, perfectly asymptomatic, and they're not really asking for medical health. Whatever we decide to do with these people, we have to be extremely careful, because sometimes we may find some information that we may be able to make some use of in preventive terms.
Clearly, I'm very interested in prevention. And there are some examples where screening for disease has been useful, like pap test and it was a very good idea, I think that it managed to decrease the risk of deaths from cervical cancer substantially. But we have many examples of screening tests that really did nothing for us or actually just caused a lot of overdiagnosis and lots of wasted resources and additional testing and treatments that just had no benefit and we have far more examples of that latter category of useless waste rather than the former category of this is something that was really worth it and it really helped people and saved lives.
So I think that now that we can test not just a couple of things but a couple of million of things or a couple of billion of things, obviously the risks are getting much higher that we can easily waste our efforts and our resources and our lives just collecting information that doesn't get us anywhere.
Ray Moynihan: John, it strikes me that people like you, people like Paul Glasziou, you know, some of the people listening, many perhaps listening to this podcast, have got enough information to help protect themselves from the dangers of unnecessary testing and unnecessary treatments. But a lot of others haven't really got that, you know, haven't really read all the papers that you've read. How on earth can you spread this intelligence? Can you help people unconnected to the healthcare system, help them protect themselves from the harms of healthcare and obviously enjoy the benefits of it?
John Ioannidis: This is a major challenge and I think that we really need a concentrated and coordinated effort to improve literacy and numeracy in the wider general public. And how to do that, I think we have not enough evidence to know what is the best approach. There is some evidence that probably this is something that needs to be done starting at a very young age, probably starting in elementary school. People need to be informed at a formative age and not just wait for the last moment after they had been exposed to tons of noise and tons of nonsense. Maybe it's too late by that time. There's evidence that kids, eight, 10 year old kids, they can understand experimental design. They can understand some key issues about the scientific method. I think we need to work on that. We need to work on what we are communicating to the population at large.
And obviously I will never say that we need to communicate less science. We need to communicate more science, but we also need to find ways to communicate science in the proper context and in the proper light and in the context of what it means and whether it's something that is just for your curiosity or something that you should use to change your life. We're making very poor distinction between these notions of what is just curiosity and what is something that you need to do because otherwise you will die or have some major consequences.
Ray Moynihan: Do you think that the media has some blame here, some responsibility here, if you will. I mean, it strikes me that a lot of media reporting and in fact, I and others have done studies on this. A lot of media reporting on healthcare is much more like promotion than rigorous journalism. Do you see that as a problem too? And are there prospects to change that?
John Ioannidis: I think that this is a super imposed problem that makes things worse. By default, lots of media are trying to exaggerate and they're trying to create stories that would catch attention, probably over simplify the narrative and create extreme contrast. And science is sometimes more subtle. It's about modest differences, pros and cons, balanced nuanced situations that you need to have a very temperate and a very moderate approach to how you're communicating information. I think that media could also be a major player in transforming the understanding of science and medicine and medical evidence by the population. So it's clearly a front here that we need a lot of work.
Ray Moynihan: Just to finish off our discussion about genetics and genomics, another of the phrases that we're hearing a lot of these days, a lot more of is, is personalised medicine or precision medicine. There are critics around who say that if we embrace this too prematurely, it could cause a lot of harm and waste, but the promoters are saying that it's already here and we have to embrace it. What's your view?
John Ioannidis: So I think that there is already a lot of discussion about precision medicine and personalised medicine and there's a lot of information that indeed is available. So we cannot just ignore it. We need to deal with it and we need to appraise what it means and whether it can be used and whether we can gain something out of it.
I'm neither a strong optimist nor a strong pessimist in that regard. Probably, I'm sceptical about what can be achieved and I think that most of the narratives about precision medicine really transforming medicine are infested with a lot of hype. It's an effort in the right direction in the sense that it is very much aligned with what evidence based medicine is about. As we started our discussion, you asked me what is evidence based medicine, and I mentioned that one of the two core components is that individualisation, you know, we're talking about individual patients, individual clinicians and the interaction of the individual clinician with individual patient is exactly precision. It's personalised. So this is not new and in that regard I think this is a good idea.
Where the problem arises is that if you're trying to generate evidence that is tailored only to that single patient or that single patient physician pair, then this is very difficult. It's very difficult to get reliable evidence even for the average patient running large studies with hundreds and thousands of participants. If you want to get the same level of reliability for the single individual, most of the time you will not be able to get that. You need to borrow strength from what has happened to other people, some of which, some of whom may be pretty similar, but others may be a bit dissimilar and you need to decide whether the dissimilarity is sufficient to make you think that that's not something that I want to consider as evidence that would affect my patient.
Most of the time, we still need to depend on traditional tools of randomized trials, average effects, meta-analysis, summary of results across multiple patients to inform what is the best choice for the single patient. There will be some situations where we will have highly tailored evidence relevant to single or a few participants. By definition, this is relevant to one or a few people, so each one of these successes will not really save a lot of the population level burden of disease. Precision medicine in its own roots is doomed to be insignificant. However, if you build many, many millions of single patients with personalised evidence, then you start catching up and you start getting something that could also have an impact on the burden of disease. So it's a possibility, but it's not a panacea.
Ray Moynihan: It’s fascinating to hear you reflect on this because these are such important questions for all of us in our own lives, but for the people who run health systems as well. I mean in Australia and elsewhere, it seems to me that part of the promotion of precision medicine is actually calling for us to water down existing regulations, existing evaluation regimes for the reasons that you outlined. I mean, do you see any danger that the gains that have been made in the last 30 years in trying to improve more rigorous methods of evaluation of new technologies, new drugs, do you see a danger that precision medicine is going to knock them down?
John Ioannidis: There's clearly a risk and I think that there's lots of people who are trying to remove randomized controlled trials from where they are right now, where they occupy a very central role in appraising new technologies, new drugs, new devices and so forth. I think this is risky business. I think that there are a few exceptions where probably the personalised effects are going to be so clear cut and so huge that probably we don't need randomized trials - based on what I have seen so far this is really the exception. It may become a bit more common, but it's still going to be the exception I think for many, many years and it may be the exception even at the best circumstances and at any time in the future. So I think that the efforts to dismantle randomized trials is really dangerous and it may really get us back to where we were in the 1900s, where we just had observations with a little bit of suboptimal control data that were highly misleading.
Ray Moynihan: We're going to change gear in a moment and go to a more sort of personal interview, if you will. But before we do, it strikes me there's an incredibly strong line of scepticism running through a lot of what you say and causing you to see the dangers, see the potential harms. Clearly a lot of listeners will be sympathetic to that view. But when those views are shared publicly, is there a danger that they can be seen as undermining science, as an attack on science rather than an attempt to improve science?
John Ioannidis: There are lots of people who are coming up with anti-science agendas and this is becoming more prominent over the years. We have seen, for example, climate change denialism and vaccine denialism, and HIV denialism, and lots of crazy thoughts. Obviously, you know, these people that have these views, they may utilise any statement by anyone as a weapon to promote their agendas. However, I think the best way to defend science is to stand firm on what the scientific method is about, how difficult science is, how difficult it is to get it right and why we need to do the best effort to put the best of the best together to try to get a reasonable answer that eventually would also be useful.
I don't think that we are defending science if we're telling that, oh, everything is so spectacular. Everything is working so easy. We are so powerful, everything is at the tip of our fingers. We can make you immortal and a cancer will go away within a week. You know, this is not reality. Science is about understanding and describing reality and if the reality is tough, we need to be tough and we need to be tough with ourselves. And I think that this is more likely to gain the respect of the general public. If you have dogmatists on one side who are most anti-science voices and scientists on the other side who take a more temperate, more sceptical, more healthy, sceptical approach to what they communicate to others.
[Music]
Ray Moynihan: So now, John, we're going to sort of move a little bit into your personal life if we can. Talk a little bit about your life and career. Can we go back to that birth that you might have described as a disaster I think (laughs). Although we're not sure it's your birth was a poetic birth.
John Ioannidis: This is just the hero of the text (laughs).
Ray Moynihan: So you're born in New York, but you were raised in Greece. Tell us a little bit about your childhood, the family, the place, the roots of who you are.
John Ioannidis: So I have ping ponged between continents during my entire life. I was born in New York, I grew up in Athens, then when back to the states, the east coast, then went back to Europe, then went back to the States in the west coast in California. My childhood was mostly in Athens. My parents were both physician scientists, so there was always both the element of the medical doctor around but also the element of the scientist around. And I think that that was clearly a very strong influence. Growing up in Athens was really fabulous. Greece is a very beautiful country. The people are very friendly and, I mean these are years that I believe most of us think of our childhood as blessed time unless there are some problems of course. But yeah, I think that it was really fabulous being in that place and growing up there.
Ray Moynihan: I think you had an early love affair with mathematics, is that right?
John Ioannidis: Yes. So I always loved mathematics and even when I was a three year old little kid, it was like a show that the friends of the family would gather together and they would start asking me to calculate weird calculations that I would get responses very quickly.
And then when I was eight or so, I started creating a list of who I love the most and it was a ranking that included the numbers, and numbers with decimal points and the list was being revised every week or so depending on whether my mum or my grandmother or uncles or whoever were willing to be good with me. So I would add them two points, 72 points, for every gift that they would make.
Ray Moynihan: That does sound a little bit scary, I got to say. (laughs)
John Ioannidis: (laughing) So I always wanted to quantify, you know, even love. If it could be quantified, I felt much more secure. And I think that eventually when I had that interface between mathematics and medicine, I felt that we need more mathematics. We need more numbers, more quantitation in medicine.
Ray Moynihan: Do you still think love can be quantified?
John Ioannidis: Uh, no. I think I have failed miserably in that regard. But science never has the final word. You never know. (laughing)
Ray Moynihan: You became a doctor in Greece and I think you worked briefly in the military, is that right?
John Ioannidis: Yeah, that's an interesting story. So when I was elected as faculty in a Greek medical school, all Greek medical schools are state institutions, they're public institutions. And to have an appointment in a public institution, you have to serve in the military like every other Greek citizen who lives in Greece and has a public appointment. So I was both a US and a Greek citizen and in order to take that appointment in Greece, I had to serve six months in the Navy, yes.
Ray Moynihan: So how was six months in the Navy? You don't strike me as a military man, but how was it?
John Ioannidis: I think it was kind of a surrealistic experience. Much of the time I spent in a naval base where a lot of the submarines where the frigates of the Greek Navy are stationed. And I was responsible for patients. This included lots of recruits who had mental problems. You know, I think that that was a quite a scary experience to see these young people probably under the stress of the experience of being recruited and having to do that. Losing their mind. And it was quite an interesting experience. It's made me think a lot about lots of things. About, for example, whether military is necessary, whether we could do more for peace. How is civilization investing its resources? How can we really help young people thrive and become much better than we are? It was a very valuable experience in that regard.
Ray Moynihan: I've got to say, it reminds me of the book Catch-22 a little bit too.
John Ioannidis: Of course. Yes.
Ray Moynihan: I think when you visit Greece these days you stay in a tiny cottage on a tiny island with your family. Can you tell us a little bit about that island and perhaps some of your favourite spots there?
John Ioannidis: So we love Greece and we want to spend as much time there as possible. That tiny island that you refer to probably is Antipaxi, where I have been going every summer since 2000. So for the last 18, 19 years with my wife and my daughter, it's a tiny island in the Ionian and we arranged a small cottage in the middle of the island. There's no hotels, there's only two tavernas that close at 5pm. And it's a little bit like being all alone in the middle of nowhere. Which is very refreshing (laughs).
Ray Moynihan: I bet. While we're on matters personal, just quickly, I mean, when you or your loved ones has a health problem, given your scepticism, given your knowledge, how do you interact with the system? Do you search for a systematic review? What do you do?
John Ioannidis: Clearly I try to get the most systematic and unbiased information as a start. My bias is to avoid contact with the healthcare system unless you really want it. So very often I hear my mum for example, saying that I have this and this and that, and I try to make sure that she really needs to see someone or do something because most of the time the best solution is just forget it (laughs).
But sometimes you may need to interact with the healthcare system and you could get a successful interaction out of it. I think that you need to find the best evidence and try to stick to that. And this is not easy when you are the patient or one of your loved ones is the patient. So it's difficult to be your own physician. You need, or be a physician of people who you love. You need someone who is equally trained with the same premises, with the same standards, with the same principles to be involved. And hopefully, I know several such colleagues who are very well trained and they're very good in evidence-based approaches.
Ray Moynihan: Alongside being one of the world's most influential scientists, you also write a lot of fiction. What do you write that fiction about?
John Ioannidis: So, I'm not sure that I would necessarily classify it as fiction. I think that all the critics who have written about my work, the most difficult question of for them is what exactly is John writing? And I don't think that they have come up with a good reply.
I used to like the term experimental writing or mixed techniques. So it could include fiction, it could include poetry, it could include other tools or a hybrid set of methods. And this is something that I tremendously enjoyed doing. I think it's offering me a different angle or viewpoint to my life. I write since I was a little kid. And I have published six books in Greek. The seventh one will be coming out in a few months.
Ray Moynihan: A lot of people listening, including me, want to know how on earth you are so productive. It's one of the most obvious things about you that people must find very confusing. You've served on the editorial boards of I think 30 leading journals. I think you've published close to a thousand academic papers. You're constantly, constantly flying around the world giving presentations on top of all the poetry and the fiction and so on. What is the secret?
John Ioannidis: I don't know. I think probably I'm a maniac to some extent (laughs). Or even a classic case. And I think that I enjoy having a variety of exposures in a variety of modes of expressing what I feel and what I think. I can work in very weird places. So travelling on the plane is wonderful, protected time. You can work during the entire flight. I can shift gear between working on a literary text and working on a paper or working on a protocol within the same time block. And I just enjoy enormously learning from lots of smart people who interact with me.
I think I feel entirely privileged to have the chance to interact with so many brilliant people. I have been really blessed in that regard to meet with lots of young people, young - not necessarily age - although many of them are also young in age, but young in spirit and young in mind and share ideas and brainstorm and think about what might be the next step. So I try to enjoy whatever I do. And I also try to say no to whatever I do not enjoy. And that's not easy, but I'm trying to do my best (laughs).
Ray Moynihan: I know you read a huge amount of fiction yourself. Are there any books in your life, fiction books or non-fiction for that matter that you want to share with the listenership now that you'd recommend? Real stand outs?
John Ioannidis: So it's very hard to just pick one thing. I think that my choices for modern literature are a bit too classic and too anticipated. But, for example, I love James Joyce and I love Sebald. Like, you know, Austerlitz I think is clearly a masterpiece. My taste is very kind of commonplace when it comes to literature. I like the big, great writers, you know, Dostoevsky is clearly amazing. I also like reading a lot of history. Sometimes books that are a bit esoteric about the history of little places or unknown places that nobody knows about and trying to catch little details or historical records - a papyri from ancient Egypt or inscriptions from ancient Greece. Some of these texts eventually find their ways in my own weird writings, with bits and pieces thereof. So yeah, very fragmented.
Ray Moynihan: Given that background, John, with those physicians, scientists, parents, where did this love of literature come from?
John Ioannidis: I think that probably it's the fact that there were always lots of books around me. And even as a little kid I enjoyed just swimming in books - that I would throw them on the floor and start reading multiple books at the same time and just enjoying that tremendously. It's something that I have not avoided since. So I'm still swimming in books, either physically or electronically.
Ray Moynihan: Beautiful. What a lovely image to leave - swimming in books. Thank you John. I hope our paths cross again one day. Really appreciate your time.
John Ioannidis: Thank you Ray. I hope to see you again soon. A million thanks.
Ray Moynihan: That was the amazing John Ioannidis on the Recommended Dose. If you’ve enjoyed it please let others know about it and tune in to some of the other conversations on this podcast.
Thanks to Jan Muths for the fantastic editing, and to the indefatigable Shauna Hurley and Cochrane Australia for production.
And to take us out, as promised, is John reading his poem Harvesting Disasters.
John Ioannidis:
Harvesting disasters
He decided to make a cool appraisal of all the disasters that had afflicted him. To attempt a lifelong evaluation, a reasonable and moderate, precise recording of calamities. First, his birth. A huge disaster, an enormous woe. In itself, his birth alone was sufficient to kill him. He was conceived elsewhere, born elsewhere, there mediate white letters, all-white snow, cherry blossoms, a beautiful young woman and other such inhuman, terrible, awkward and monstrous stuff. Second disaster in the first three months of his life. I keep it silent, a taboo. They had told him about it, even though he didn’t live it, so it must have been terribly real. Third disaster somewhere in his seventh to ninth year. Long-drawn calamity, with a protracted tail of five or six years before he found some relief again in his high performance in mathematics, especially in commutative and homological algebra, but also in the hyperbolic geometry of Lobachevsky. Fourth disaster at age twenty-two. This disaster in particular is responsible for his entire life. Whatever he did, he did so as not to forget – ever. And indeed, he never forgot her, while she, she didn’t even notice that she had torn him to pieces, that she had thoroughly devastated him. Fifth disaster at age twenty-five. That one he even engineered it himself. Disaster engineering. He caused it as a controlled explosion that would self-traumatize him. He was playing for years with gunpowder, TNT and poems, so he set it off with confidence. Sixth disaster at age thirty- three. It was the first time that a disaster surprised him, the first time that he already was sufficiently suspicious and alerted (from past disaster) to be surprised. He lost everything in no time without having predicted even the slightest calamity. He took his hat off to the disaster. How ingenious she had been! Seventh disaster at age forty-four. The fiercest of all, it was unthinkable that he could carry on. He woke up in summer from a lethargy, with tinnitus, in a ship sailing to Naxos, full of happy vacationers and carefree tourists. He almost lost his mind from this absurd, outrageous, wicked sight. Eighth disaster at age fifty-one. He had been expecting her. He had kept notes, he had prepared his answer, he had written his public apology. Not that he could prevent her. Let’s not fool ourselves, disasters cannot be averted. Simply this was the first time that he systematically prepared himself to be destroyed. He felt proud that they were destroying him, as if he witnessed the completion of a splendid public project. In short sequence the ninth disaster followed suite at age fifty-two. Totally unpredictable, it humiliated him, right when he thought that from now on at least he could foresee with clarity, with some superior resignation his extermination. For none of the nine disasters did he ever dare publish anything. Only minced words, cover ups, indirect mentions, obscure hints, ghosts, unsupported speculations, cravings, unfounded beliefs, reasonable betrayals, in short only his entire life – quite a disaster.
[Music]
Ends