|
|
With regards to RCTs....
|
Logged in as: Guest
|
|
Users viewing this topic:
none
|
|
Login  | |
|
With regards to RCTs.... - October 3, 2006 12:40:00 PM
|
|
|
Sebastian Asselbergs
Posts: 1211
Joined: September 29, 1999
From: Barrie, Canada
Status: offline
|
Just some interesting food for thought from research methodologists.
"Circular instead of hierarchical: methodological principles for the evaluation of complex interventions
Harald Walach1 , Torkel Falkenberg2 , Vinjar Fønnebø3 , George Lewith4 and Wayne B Jonas5
1University of Northampton & Samueli Institute – European Office, School of Social Sciences, Park Campus, Northampton NN2 7AL, UK
2Karolinska Institutet, Center for Studies of Complementary Medicine, Department of Public Health Sciences, Division of International Health (IHCAR) and Department of Nursing, Stockholm, Sweden
3National Research Center in Complementary and Alternative Medicine, University of Tromsø, Tromsø, Norway
4University of Southampton, Department of General Practice, Southampton, UK
5Samueli Institute, Alexandria VA, USA
BMC Medical Research Methodology 2006, 6:29 doi:10.1186/1471-2288-6-29
Published 24 June 2006
Abstract
Background
The reasoning behind evaluating medical interventions is that a hierarchy of methods exists which successively produce improved and therefore more rigorous evidence based medicine upon which to make clinical decisions. At the foundation of this hierarchy are case studies, retrospective and prospective case series, followed by cohort studies with historical and concomitant non-randomized controls. Open-label randomized controlled studies (RCTs), and finally blinded, placebo-controlled RCTs, which offer most internal validity are considered the most reliable evidence. Rigorous RCTs remove bias. Evidence from RCTs forms the basis of meta-analyses and systematic reviews. This hierarchy, founded on a pharmacological model of therapy, is generalized to other interventions which may be complex and non-pharmacological (healing, acupuncture and surgery).
Discussion
The hierarchical model is valid for limited questions of efficacy, for instance for regulatory purposes and newly devised products and pharmacological preparations. It is inadequate for the evaluation of complex interventions such as physiotherapy, surgery and complementary and alternative medicine (CAM). This has to do with the essential tension between internal validity (rigor and the removal of bias) and external validity (generalizability).
Summary
Instead of an Evidence Hierarchy, we propose a Circular Model. This would imply a multiplicity of methods, using different designs, counterbalancing their individual strengths and weaknesses to arrive at pragmatic but equally rigorous evidence which would provide significant assistance in clinical and health systems innovation. Such evidence would better inform national health care technology assessment agencies and promote evidence based health reform."
_____________________________
Mundi vult decipi
|
|
|
|
|
Re: With regards to RCTs.... - October 3, 2006 3:13:00 PM
|
|
|
jlharris
Posts: 477
Joined: April 12, 2006
From: Nebraska
Status: offline
|
Very interesting observation/philosophy by the author's. There is no doubt much truth in their conclusions. It'd be nice to see how this framework could actually carried out and implemented.
Thank you for the abstract.
_____________________________
Jason L. Harris, PT, DPT
My PT Blog
|
|
|
|
|
Re: With regards to RCTs.... - October 3, 2006 6:34:00 PM
|
|
|
srcase
Posts: 551
Joined: November 30, 2004
From: Michigan
Status: offline
|
I think the PT profession is already doing this. The journals are not throwing out case studies or cohort studies just because they are not "best evidence" and even Sackett states that for researching therapies, it is not adviseable or necessarily always possible to do RCT's.
THis is exactly my point I've been trying to make on other threads. Thanks Sebastian!
Sarah
|
|
|
|
|
Re: With regards to RCTs.... - October 3, 2006 10:03:00 PM
|
|
|
nari
Posts: 1568
Joined: November 14, 2003
From: Australia
Status: offline
|
Great post, Sebastian.
I agree with Sarah that the PT profession is trying to do this, and meeting with some considerable resistance.....
I also recall a researcher PT saying the same thing about 6 years ago to us; that RCTs do not have much meaning for therapies; but they can be a useful guide. In the same breath, he said that EBP is a guide, not a rule or a tool. Perhaps that is why we are more relaxed about and more confident with our clinical reasoning in Oz.
Nari
|
|
|
|
|
Re: With regards to RCTs.... - October 6, 2006 6:56:00 AM
|
|
|
mcap56
Posts: 619
Joined: October 26, 2002
From: New York, NY
Status: offline
|
Great article Sebastian. This will be required reading for my students from now on.
In defense of the hierarchical model however, I would like to offer some considerations:
1. We never teach that the model needs to be strictly interpreted. Some cohort or case series studies can be far better than RCTs. There are good and bad both ways.
2. We teach that the evidence hiearchy is to be combined with clinical reasoning. Not to be used in lieu of it.
3. we teach that each type of evidence has it's strengths. We always stress that many types of evidence, including qualitative research are really critical for total consideration of a topic.
4. The article does a good job of making this clear though and I particular like the discussion of internal v. external validity. Our system is biased in favor or internal.
5. Our system is extensively influenced by pharmaceutical research which lends itself well to the RCT.
Marc
|
|
|
|
|
Re: With regards to RCTs.... - October 6, 2006 7:45:00 AM
|
|
|
Shill
Posts: 1098
Joined: February 13, 2003
From: Madison WI USA
Status: offline
|
In addition to the new reading requirements, one might want, for the sake of grounding ourselves, to take a look at this one too.
[URL=http://medicine.plosjournals.org/archive/1549-1676/2/8/pdf/10.1371_journal.pmed.0020124-L.pdf]Most Research is False?[/URL]
I was turned on to this by a physician patient, who showed me an article about this article in "The Economist".
Its a bit hard to follow, given the multitude of complex formulas that make me run for cover, but it does make a whole lot of sense.
_____________________________
Steve Hill PT
|
|
|
|
|
Re: With regards to RCTs.... - October 16, 2006 7:04:00 PM
|
|
|
Geert Jeuring
Posts: 92
Joined: June 26, 2002
From: Möhnesee, Germany
Status: offline
|
Hello Forum, I´m not convinced:
It is inadequate for the evaluation of complex interventions such as physiotherapy:
Because we can´t manage to answer what physiotherapy consist of (because most of us, work behind closed doors and do as we please?). Even if our conducts are complicated proving evidence isn´t.
surgery: there I have a total blank. In surgery a diagnosis is needed before opening the patient. The diagnosis is or isn´t confirmed once the patient is opened. After that it shouldn´t be to hard to determine effectiveness of the treatment.
and complementary and alternative medicine (CAM).
First of all there isn´t such a thing as complementary or alternative medicine. There is medicine and if one wants to be in that boat one has to play by its rules. Those rules have arisen of al long history of epistemology and can still be changed but only with decent arguments leading to new better rules that would be compulsory for every one practising medicine.
The group of what you call complementary or alternative medicine but which I referr to as quacks mostly like it to have no rules. Everything is possible, mostly along the line of: he who heals is right. I won´t go in to that argument again.
The above doesn´t mean we shouldn´t be open for new strategies in treating patients, but considering the treatments flooding the market we should be very critical.
There is however one great flaw of the evidencehierarchy. Even if a metaanalysis tells us (for the sake of argument) that exercise is the best treatment for Diabetes Typ II, getting it in to practise is very difficult if not impossible. The same vor NSAR, which are in some case best treatment, but if 50% of the patients flush them down the drain, it might be best treatment, but not realistic.
Geert
|
|
|
|
|
Re: With regards to RCTs.... - October 17, 2006 1:57:00 AM
|
|
|
Sebastian Asselbergs
Posts: 1211
Joined: September 29, 1999
From: Barrie, Canada
Status: offline
|
Geert:
"It is inadequate for the evaluation of complex interventions such as physiotherapy:
Because we can´t manage to answer what physiotherapy consist of (because most of us, work behind closed doors and do as we please?). Even if our conducts are complicated proving evidence isn´t."
This does not make sense in the context of the article. The "complexity of physiotherapy" relates to the variables present in ANY physiotherapy treatment -= that is: one person treating another. It has NOTHING to do with what is done, just that it is by nature COMPLEX. And I have no trouble at all defining physiotherapy....
Then:
"In surgery a diagnosis is needed before opening the patient. The diagnosis is or isn´t confirmed once the patient is opened. After that it shouldn´t be to hard to determine effectiveness of the treatment."
A diagnosis often arrived at with imaging, and thus often a great surprise awaiting the surgeon when he or she is "in" the patient. And let's not forget that lovely term "exploratory surgery" - a safari into the body.
For years the arthroscopic surgery has been a strong tool for surgeons - until it was shown that sham 'scopes were equally effective....
Then CAM - well, if you don't want to accept the concept of CAM you may run into a lot of discussions. Especially with the medical establishment in many countries, where there are even curriculum courses specifically dedicated to touching on all types of CAM.
And many CAM therapies may be quackery, but they are still complex interactions and thus difficult to properly investigate in the RCT style. You are putting values on the subjects being mentioned, but the article is about research "style" for complex interactions, NOT a pro- or con for the subject matter mentioned.
_____________________________
Mundi vult decipi
|
|
|
|
|
Re: With regards to RCTs.... - October 18, 2006 12:10:00 PM
|
|
|
Geert Jeuring
Posts: 92
Joined: June 26, 2002
From: Möhnesee, Germany
Status: offline
|
Hello Sebastian,
"This does not make sense in the context of the article. The "complexity of physiotherapy" relates to the variables present in ANY physiotherapy treatment -= that is: one person treating another. It has NOTHING to do with what is done, just that it is by nature COMPLEX. And I have no trouble at all defining physiotherapy...."
The variables that you mention are present in a possible effective treatment as well as in a placebotreatment as long as it is a therapist/ Patient relationship. That is the aim of a RCT, to have an almost identical situation with only one difference: the treatment.
"A diagnosis often arrived at with imaging, and thus often a great surprise awaiting the surgeon when he or she is "in" the patient."
But then the patient would be excluded then wouldn´t he?
"For years the arthroscopic surgery has been a strong tool for surgeons - until it was shown that sham 'scopes were equally effective...."
Yes that was Moseley et al. and they favour the evidence based method very much. That is what part of evidence based medicine is all about. When you see the adds for washing powder, all of them claim to be the best in washing your clothes. The only way you can tell if they are telling the truth is testing their outcomes.
"Then CAM - well, if you don't want to accept the concept of CAM you may run into a lot of discussions. Especially with the medical establishment in many countries, where there are even curriculum courses specifically dedicated to touching on all types of CAM."
Yes of course, but that would be countries where first of all the scientific revolution hasn´t taken place as it has in western Europe (starting with people like Galileo Galilei) and people still often suffer under dictatorial regimes (China) or extreme poverty (India). In other countries like Holland which is my origin, the rise of these "therapies" started in the seventies. There are a lot of possible reasons why these "therapies" became popular. But the rise is rather illogical because in the pas 30 Years people became healthier and healthier.
"And many CAM therapies may be quackery, but they are still complex interactions and thus difficult to properly investigate in the RCT style. You are putting values on the subjects being mentioned, but the article is about research "style" for complex interactions, NOT a pro- or con for the subject matter mentioned."
Well in my eyes it is another attempt to bring weak and irrefutable arguments as an excuse for treatments that can´t stand up to the simple standards of evidence based medicine.
RCT´s should be a wonderfull method for possible real Quacktherapies. You don´t have to explain why it works, you just have to prove that it works.
Greetings from Germany
Geert
|
|
|
|
|
Re: With regards to RCTs.... - October 18, 2006 1:46:00 PM
|
|
|
Sebastian Asselbergs
Posts: 1211
Joined: September 29, 1999
From: Barrie, Canada
Status: offline
|
Geert, my reading of the article is that it is NOT making excuses for CAM or physiotherpay techniques. It is addressing the difficulty of designing "clinically" relevant RCTs - due to the nature of interaction with patients. It does not seem to promote voodoo or reiki or CST, but just focusses on the possible approach to evaluating their effectiveness and/or scientific foundations.
Your comment regarding the "scientific revolution": I came from the Netherlands and am well aware of the large CAM bloom. It is one of the western countries where CAM was introduced at universities and Hogescholen - mostly as "electives". There are colleges here in North America that teach CAM courses -there are CEUs being earned in CAM techniques.
The article is NOT judging CAM or PT either way - it offers a different slant on what to include when we talk about EBM in these fields.
Marc's post is really good and succinct. And he is a EBM supporter, but a discerning one!
_____________________________
Mundi vult decipi
|
|
|
|
|
Re: With regards to RCTs.... - October 19, 2006 8:29:00 AM
|
|
|
Geert Jeuring
Posts: 92
Joined: June 26, 2002
From: Möhnesee, Germany
Status: offline
|
Hello Sebastian, I´m repeating myself: the nature of interaction with patients is a non - specific effect in any treatment even placebotreatments. It is for this reason that a lot of rct´s have a control group, a placebo group and a treatmentgroup. The nature of interaction with patients is also highly individual and it isn´t the goal of an rct to judge on the individual patient or therapist but the treatment.
Allthough it doesn´t promote quacktherapies, it is trying to open the door. This isn´t a new tactic. It has been tried before in religious discussions. It sort of goes like this: you can´t judge us with your rules because they don´t applie to our believes/ therapies etc.
The authors don´t offer a real alternative (in the abstract)or stay very vague how this circular modell should function. Also one could think that in evidence based therapy the rct is the only evidence. I don´t think that you can find a book concerning EBM that doesn´t say that some questions are better answered with other studytypes.
If you have the complete originale I would be greatfull if you could mail it to me.
Greetings
Geert
|
|
|
|
|
Re: With regards to RCTs.... - October 19, 2006 9:02:00 AM
|
|
|
mcap56
Posts: 619
Joined: October 26, 2002
From: New York, NY
Status: offline
|
Geert:
We all know that the RCT has a lot of advantages. And, we have all seen what happens to some very popular treatments when exposed to RCTs. Suddenly, they don't seem to work as well.
But the article brings up some very valid points. You are focused solely on internal validity. Internal validity is acheived in some RCTs (especially in disciplines less suited to the RCT) to some degree by sacrificing external validity. When you must control every variable, you then must control the intervention to a very high degree. For the very best designed RCTs in physical therapy, the treatments delivered are often far too focused, non specific for each patient, and too restricted to represent what actually happens in the clinic. Take for instance the McKenzie/manipulation RCT. There are very few McK folks who would just perform McK routines on patients.
No one is saying RCTs aren't needed. We are just saying that they have limitations just as other designs do and that those limitations need to be considered particularly in some instances.
As for CAM, don't over-estimate the evidence that traditional medicine relys on. The next time you take a medication, look at the research behind it. You would be shocked at how little evidence some the most common medical practices are supported by. And then, in this country, you must account for the influence of pharmaceutical research. Take for instance a recent RCT on Glucosamine. There have been RCTs that show good support - this study demonstrated none. A careful look at the study however, and you can see that study actually showed more of an effect than the media claimed and that the study was basically designed to prove Glucosamine was ineffective.
Have a look at RCTs on back pain where the end point is return to physician and return to doctor. Does this explain what happened to the patients? Don't we need qualitative researchers to go and speak to the patients to see how they are actually doing? Perhaps they go back to work but are absolutely miserable!!!
I am not a particularly huge proponent of CAM and any of my students would tell you that I ram EBP down their throats. But.....I think the topic is a complex one and just like any complex phenomenon, it may not be best explained by a straightforward hierarchical model.
|
|
|
|
|
Re: With regards to RCTs.... - October 23, 2006 12:02:00 PM
|
|
|
Geert Jeuring
Posts: 92
Joined: June 26, 2002
From: Möhnesee, Germany
Status: offline
|
Hello Mcap, I´m well aware of the limitations of RCT´s but I´m not aware of better alternatives.
As for traditional medicine, I´m very sceptic of that as well. I tend to think that Cochrane was being optimistic when he said that only 20% of common medical therapies were proven to be effective. We have a tendency to think we can cure or influence every disease. I think that is very naive. In fact the things that have had the most influence on our health are hygiene and a relative peacefull country.
If you´ll take a look at philosophers and doctors of the past 2000 years you will see that they have reached rather high ages without the benefit of moderne medicine. That doesn´t mean old treatments are better or that moderne medicine isn´t sufficient in various cases. Just that it has it´s limitations.
I´d like to quote Carl Sagan here although I don´t recall the exact sentence.
Science has it´s limitations but for now I don´t see any alternatives that are better.
Geert
|
|
|
|
 New Messages |
 No New Messages |
 Hot Topic w/ New Messages |
 Hot Topic w/o New Messages |
 Locked w/ New Messages |
 Locked w/o New Messages |
|
 Post New Thread
 Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts |
|
0.094
|
Printable Version
All Forums >> [RehabEdge Forum] >> Open Forum >> With regards to RCTs.... 
