Metanalytical Madness Over Medical Cannabis: Meta-analysing Poor Research is Uninformative
- Prof David Nutt Prof Michael Lynskey, Prof Larry Phillips, Dr Anne Katrin Schlag,
- 4 minutes ago
- 8 min read
Written by:
Prof David Nutt FMedSci, Imperial College London
Prof Michael Lynskey, Drug Science
Prof Larry Phillips, London School of Economics
Dr Anne Katrin Schlag, Imperial College London
Many of you will have seen the media reports of the recent Lancet Psychiatry paper by Wilson et al 2026 claiming that there is little or no evidence of clinical utility for medical cannabis in mental illnesses (other than possibly for cannabis dependence). This conclusion will surprise many, not least especially the estimated 100,000 or more people in the UK paying to access prescribed cannabis for health reasons, mostly for pain and mood/anxiety disorders. So how has a peer-reviewed analysis come to a conclusion that is so contradictory to the lived experience of so many patients?
The answer is complex, reflecting a range of conceptual biases plus errors in the study design, analysis and interpretation.
The first is the lack of any interrogation of patient perspectives. It is a well-established principle, long endorsed by MHRA and other regulatory bodies that proper decision making about a medicine’s efficacy must involve the patients’ voice. The preferred way to incorporate the patients’ experience is the multi-criteria decision analysis (MCDA) method developed for the EMA by Prof Larry Phillips [3]. This has been applied to compare the value of medical cannabis with other medicines for a common chronic pain syndrome, neuropathic pain and should be used to estimate clinical values for other common conditions for which patients commonly are prescribed medical cannabis such as anxiety disorders and depression.
The second is their bizarre reliance on a very small number of randomised controlled trial (RCT) data sets to come to conclusions. Once the inadmissible studies are excluded (see later) we have the remarkable situation that there are more authors on the paper than the total of relevant papers in the field of just mental illness. Surely a record for a metanalysis paper?
The authors have either ignored or were unaware of the vastly greater amount of data from patients treated in the real world for whom outcome data have been collected. It would appear that they were not aware of the famous 2008 Lancet paper authored by Sir Michael Rawlins in 2008 when he was head of NICE in which he demolished the prevalent belief that RCTs were the apex of decision estimates of medical efficacy. To quote him, “Randomised controlled trials (RCTs), long regarded at the 'gold standard' of evidence, have been put on an undeserved pedestal. Their appearance at the top of "hierarchies" of evidence is inappropriate; and hierarchies, themselves, are illusory tools for assessing evidence. They should be replaced by a diversity of approaches that involve analysing the totality of the evidence-base.”
Rawlins also made some other points relevant to medical cannabis, particularly the problem that RCTs are conducted in very unusual and selected populations that are not representative of patients in general. The Drug Science T1 initiative recruited nearly 5000 patients into medical cannabis treatment and found that less than 10% did not have a comorbid condition. Thus 90% of typical medical cannabis patients would not be eligible for a typical RCTs, one reason so few studies have been conducted. Another reason pointed out by Rawlins is that RCTs are hugely expensive and are not good value for money. Medical cannabis is still a controlled drug so expensive to research, and plant products are very hard if not impossible to protect from competition even if approved as medicines by the FDA/MHRA. Together these barriers mean the financial benefits from achieving marketing authorisation are unlikely to offset the costs of the RCTs needed to achieve it.
It is not clear why the authors chose to ignore such clear guidance to explore real world evidence (RWE) from an undisputed leader in the field? A recent survey of publications on UK medical cannabis patients using RWE reveals powerful estimates of effectiveness for anxiety [1], OCD, and PTSD with and without depression. International data from similar medical cannabis registries show equivalent effects. In other words, the majority of patients treated with medical cannabis for whom we have outcome data are in RWE registries not RCTs. That this data shows highly significant clinical effects supports the assertions of Rawlins and calls into serious doubt the utility of the RCTs for making clinical decisions.
Another major flaw in the Wilson et al conclusions on anxiety is that many of the anxiety disorder studies they included measured the clinical effect of just a single dose in patients compared to healthy volunteers. Anyone who has worked in the field of pharmacological treatment of mental illness would confirm such short treatment durations are likely to grossly underestimate clinical value. Even with fast acting anxiolytic medicines such as benzodiazepines, clinical trials show several weeks of drug administration to reveal maximal effects. To conclude medical cannabis had no effect in anxiety based on meaningless data is to grossly misrepresent the evidence. It has already led to media claims of ineffectiveness that will in turn will drive patients and possible prescribers away from what RWE reveals is a useful treatment. Perhaps they should have included an expert in the disorders they were evaluating as part of their research team, which seemed largely to be addiction researchers?
On top of this the authors conflate data on the two major active ingredients of medical cannabis - d9THC and cannabidiol. These have fundamentally different pharmacologies and medical indications, so collapsing them into a single medical cannabis category is scientifically flawed. Added to this, products made of combining single molecule extracts of cannabis (e.g. Sativex) were treated as being equivalent to whole plant products. This may not be the case given the evidence already established in childhood epilepsy that whole plant products show greater efficacy.
The paper also has number of errors, including incorrect THC vs CBD in anxiety column of Table 1 and the irregular scale in fig 1 where the y-axis is incorrect as the 0-1 interval is smaller than the others – see fig here. This mistake, which tends to give the impression of smaller effect sizes, is compounded by the use of range of SMDs from 0-5 when the largest mean difference is less than 2!
Together these errors raise concerns about scientific rigour thereby further undermining confidence in their conclusions.
Given the addiction-centred expertise of the authors one obvious conflict is the over-focus on risk of dependence. Certainly, the press rather focused on this threat even though the evidence of dependence in medical cannabis users is limited and is compromised by the use of scales such as the CUDIT to measure it. Elements of this scale used to predict dependence are terms such as daily use and inability to stop. Simple reflection will reveal these elements are precisely those required to treat the underlying medical disorder rather than being predictive of dependence. Applying such criteria to the use of medicines for epilepsy would conclude that all patients are addicted! The most reliable evidence of dependence is dose escalation and experience from our large clinical sample of medical cannabis patients reveal few patients escalate dose without evidence of illness breakthrough.
However, there are two things we do agree with Wilson et al on. The first is that the adverse effects profile of medical cannabis is low and does not differ much between patients in RCTs and RWE studies. Secondly lack of evidence e.g. for medical cannabis in depression, doesn’t mean lack of effect. But this is a reality that is misunderstood/ignored by the press.
The implications of this paper are clear. RCT data is of little use in making decisions about the clinical utility of medical cannabis for mental health disorder because there is too little RCT data. This begs the question why would referees endorse and a reputable journal publish the result of such a pointless exercise given that for pure mental health disorders only a single study met the authors criteria for moderate confidence in the results, the rest were graded to be poor or very poor. And why were they allowed to ignore publications from other much larger RWE data sets? Does Lancet Psychiatry not believe in RWE? Do the authors believe that RWE is of no value? Do they think that medical cannabis patients are lying about their positive outcomes? Do they believe these patients are continuing to medical cannabis because they are dependent? Or is this a collective misunderstanding of the validity of patient reported outcomes from RWE?
Overall, the Lancet paper is an example of mindless metanalysis, an all too frequently occurring phenomenon in mental health publications today. Disappointingly the editors rejected our request to publish this commentary! They argued that as the authors pursued a pre-registered metanalysis then the results are correct, even if meaningless. However thankfully they have corrected the error in the anxiety column in Table 1 where it stated that 5 of the 6 studies used d9THC when in fact they used cannabidiol.
We let the readers make up their own minds.
REFERENCES