When I am 64 and other false positives : The PACE Trial.
Greg Crowhurst
6th February 2013
“Much of what we think we know based on conventional
statistical studies published in the academic literature stands a good chance
of just not being so “
Roger Pielke
Jr
Imagine I am a psychiatrist and for £5 million, say ,
I want to prove that CBT and GET are a
safe and useful treatment for ME. The good news is “how
unacceptably easy it is to accumulate (and report) statistically significant
evidence for a false hypothesis, according to Nelson and Simonsohn (2011).
Their frightening paper “False-Positive Psychology,” shows how “undisclosed flexibility in data collection and analysis allows presenting anything as significant,” . In fact Nelson and Simonshon statistically “prove” that listening to “When I am 64” by the Beatles can make you a couple of years younger !
Their frightening paper “False-Positive Psychology,” shows how “undisclosed flexibility in data collection and analysis allows presenting anything as significant,” . In fact Nelson and Simonshon statistically “prove” that listening to “When I am 64” by the Beatles can make you a couple of years younger !
The bad news, for people
with ME, as Nature points out, is how :
Positive results in psychology can behave like rumours: easy to release
but hard to dispel.
http://www.nature.com/news/replication-studies-bad-copy-1.10634.
http://www.nature.com/news/replication-studies-bad-copy-1.10634.
Compounding the problem “ is the pathetic state of science reporting:
the problem of how unacceptably easy it is to publish total fictions about
science, and falsely claim relevance to real everyday life.” In the ME community we know more than most the devastating impact of that.
http://chronicle.com/blogs/linguafranca/2012/03/15/bad-science-reporting-effect/
http://chronicle.com/blogs/linguafranca/2012/03/15/bad-science-reporting-effect/
Science has the potential to do so much good - imagine if for the last 20 years the emphasis had been upon biomedical research in ME ! Instead we have all suffered grievously from psychiatry acting in the interests of massive vested interests, determined to deny the physical reality of ME. Ultimately that is terribly damaging for science.
http://www.guardian.co.uk/science/2012/sep/13/scientific-research-fraud-bad-practice
http://www.guardian.co.uk/science/2012/sep/13/scientific-research-fraud-bad-practice
Disturbingly it has been found that the odds of reporting a positive result are around "five times higher among papers in the disciplines of psychology and
psychiatry and economics and business compared with space science". We would have done far better, it seems , if ME had been ascribed to aliens !!
http://www.guardian.co.uk/science/2012/sep/13/scientific-research-fraud-bad-practice
http://www.guardian.co.uk/science/2012/sep/13/scientific-research-fraud-bad-practice
The situation is not improving. In 1959,
statistician Theodore Sterling found that 97% of the studies in four major
psychology journals had reported statistically significant positive results. When he repeated the analysis in 1995, nothing
had changed. http://www.nature.com/news/replication-studies-bad-copy-1.10634
This is especially remarkable , given that the DSM currently lists 297 official mental disorders, yet there is no lab test for any of those 297 disorders. As Jon Rapport comments : If psychiatry were nothing more than an intellectual game in a vacuum, it wouldn’t matter.
Oh, but it matters !
It matters very much to my wife and I, given she has suffered twenty untreated
years of absolute agony, while ME
funding, in its millions, has been wholly diverted to psychiatry.
The evidence is overwhelming
that CBT – designed to change "negative thinking" and GET are devastating for
people with ME, making them worse – this is shown in study after study. So, if
I was a psychiatrist, how might I manipulate the inevitable disappointing research
figures ?
There are any number of ways
:
- Do not study people with ME , in the first place – : the PACE Trial studied: “CFS defined simply as a principal complaint of fatigue that is disabling, having lasted six months, with no alternative medical explanation (Oxford criteria)”. http://www.meactionuk.org.uk/Hoopers-initial-response-to-PDW-letter.htm.
- Write your own version of the London Criteria , so that they are virtually identical to the Oxford Criteria.
- Imply that you are studying people with ME : The PACE Trial Identifier is clear: “Myalgic encephalomyelitis is thought by most to be synonymous with CFS” (PACE Trial Identifier; 2.1). http://www.meactionuk.org.uk/Hoopers-initial-response-to-PDW-letter.htm
- Make sure that almost half the participants
suffer from a psychological disorder. http://esme-eu.com/news/the-pace-trial-do-you-also-get-so-tired-of-this-article483-7.html
- Make sure that the majority of participants have exceptionally high expectations of CBT and GET ( that would definitely exclude people
with ME !) Do not admit immobilized patients, the most severely affected. http://esme-eu.com/news/the-pace-trial-do-you-also-get-so-tired-of-this-article483-7.html
- Get your results, fast-tracked and published by The Lancet - don't bother with getting the data rigorously checked by The Lancet’s own statisticians before publication.
http://www.meactionuk.org.uk/Comments-on-PDW-letter-re-PACE.htm
- Change
the way the trial is conducted . In
the original protocol activity graphs were adopted and budgeted,
but these were canceled. The patients however were allowed
to decide themselves how and when they would report on a scale that
was designed by one of the researchers. http://esme-eu.com/news/the-pace-trial-do-you-also-get-so-tired-of-this-article483-7.html
- Ensure that a participant can have a fatigue rating that is both
“normal” and “abnormal” depending on which of the Investigators’ various
definitions is applied. http://www.meactionuk.org.uk/Comments-on-PDW-letter-re-PACE.htm
- Make sure it is possible for a person to record
a poorer score on the CFQ (Chalder Fatigue Questionnaire) on
completion of the trial than at the outset, yet still be deemed to have
attained “normality” on this primary outcome measure. http://www.meactionuk.org.uk/Comments-on-PDW-letter-re-PACE.htm
- Make sure there is absolutely no data whatsoever in your paper on what most people would regard as objective markers of recovery; this data was simply ignored. http://www.meassociation.org.uk/?p=14333
- Redefine the meaning of “recovery : the PACE Trial’s infamous “post-hoc” definition of recovery set a threshold for recovered fatigue, using the SF-36 threshold for recovery , lowered from 85 to 60, which is classed as moderately disabled and which would have been regarded as abnormal or excessive fatigue in the original protocol. http://forums.phoenixrising.me/index.php?threads/recovery-from-chronic-fatigue-syndrome-after-treatments-given-in-the-pace-trial.21628/page-6
- Do not define or objectively measure "fatigue". Just as the meaning of "ME " is shifted to "CFS", shift the meaning of "recovery" to not mean physical recovery, nor to mean loss of symptoms - only the removal of a fatigue that you have not defined in the first place, or measured in any objective way.
- Make sure that your definition of recovery is not relative to a healthy person’s score by using the English population mean ( which includes elderly and disabled people) - rather than the working age mean .
The PACE Trial, surely , is a watershed. No matter how much they try and veil the results in smoke and
mirrors or spin it to the hilt, the Pace Trial confirms how redundant psychiatry has always been in the treatment
of ME; the serious neurological disease that is destroying our
lives.
The authors of this PACE trial study are using a deceptive trick to get their apparently good results.
ReplyDeleteThese researchers concluded in the abstract of their PACE study that:
"This study confirms that recovery from CFS is possible, and that CBT and GET are the therapies most likely to lead to recovery."
However, these authors are distorting the truth by using trickery of language centered on the word "recovery."
Their deceptive trick is this: these authors redefine the meaning of the word "recovery" within body of the text of their study, so that, under this new meaning of the term "recovery," many ME/CFS can be said to be, ahem, "recovered" following the GET/CBT therapies used in the PACE trial.
However, if you only read the study's abstract, there is no indication that the word "recovery" used therein has been totally redefined by the authors, and thus the causal reader will erroneously assume that the word "recovery" just carries its normal English meaning, which in the dictionary is defined as "a return to a normal state of health, mind, or strength."
Thus many people causally reading this PACE study authored by White, Goldsmith, Johnson, Chalder and Sharpe will be led to erroneously believe that GET/CBT therapies can return a ME/CFS to a normal state of health, which is not the case at all.
I think this deceptive trick used by the authors is a genuine case scientific misconduct.
If you are going to redefine your terms and the words you use, you need to make sure that the precise definitions of these terms are clearly given, so that there can be no misinterpretations.
It seems apparent that the authors of this PACE study actually want their completely redefined word "recovery" to be misinterpreted. Thus, this is not a mistake by the PACE study authors, but is deliberately deceptive.
This deceptive PACE study publication should be reported to an appropriate body within the UK National Health Service (NHS), otherwise busy NHS doctors, who only have time to scan study abstracts, may read this PACE study abstract, and then get the completely incorrect impression that the GET/CBT therapies used in the PACE trial can actually cure ME/CFS, which they cannot at all.
Thank you so much Hip, for this - it is very helpful.
DeleteI agree with you, a complaint needs to be lodged; the PACE document is profoundly misleading. I was sickened to read the response to the Countess of Mar's presentation yesterday in the House of Lords - all those Lords, reading from the same script it seemed - praising PACE and pushing for it to be funded for another 5 years. God help us all- the harm that is being done, the sophisticated propaganda campaign that is being waged : running rings around us all.
The only scientific measure in the PACE trial was the 6 min walking test. But the authors did not publish 27% of results from this test. What did those show?
ReplyDeleteIn the paper the 73% of the data published shows that CBT produced no objective benefit and GET produced no clinically significant benefit. Did these hidden results show CBT and GET to not only be no use for anyone in the trial, but also show them to be harmful? How can anyone know if they keep this hidden from peer review? How was such a study ever published?