|
|
Re: Look at this validation study guys, worth reading!
|
Logged in as: Guest
|
|
Users viewing this topic:
none
|
|
Login  | |
|
Re: Look at this validation study guys, worth reading! - January 3, 2005 3:59:00 PM
|
|
|
Jon Newman
Posts: 1707
Joined: April 24, 2004
From: Amherst, WI
Status: offline
|
Hi SJ,
I'm openly trying to help in explaining why. I've recently come across these two links that suggest I'm making a(nother) mistake of logic. Stated more clearly, I'm confusing prediction for explanation. Still, I'm intrigued at what was not retained by the regression analysis in coming up with the prediction rule.
Here's the links.
http://pareonline.net/getvn.asp?v=7&n=2
http://ist-socrates.berkeley.edu/~fitelson/164/lecture11.pdf
Any insights?
jon
_____________________________
[URL=http://www.sonymusic.com/clips/selection/30/064887/064887_03_03_30.wav]Evidence[/URL]
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 5, 2005 2:57:00 AM
|
|
|
SJBird55
Posts: 2438
Joined: May 10, 2004
From: Michigan
Status: offline
|
What do you mean by not retained by the regression analysis?
With the prediction part... they supposedly took a lot of data on each patient. The prediction rule was created by the commonalities for those patients that had successful outcomes. At least that's what I remembering them saying, although the current study doesn't say that.
I equate a prediction more in the terms of "if this, then (blank)." And even one step further, a prediction can help in determining a prognosis.
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 5, 2005 7:51:00 AM
|
|
|
Jon Newman
Posts: 1707
Joined: April 24, 2004
From: Amherst, WI
Status: offline
|
In the original study many factors were at least considered including:
Tests for (SI) symmetry such as PSIS, pubic tubercles, etc
Motion tests such as standing and seated flexion, Gillet test, etc
Provocation tests such as Gaenslen, posterior shear, etc.
Other variable from baseline clinical exam included left/right side bending and the discrepency. Straight leg raising. Lateral shift and more
Variable from history included mode of onset, worst and best position.
After the typical statistical gymnastics done in these studies, 11 variables were considered in the regression analysis. These included the following:
Duration of sx less than or equal to 15 days
Episodes not becoming more frequent
Standing not ranked as worst position
FABQ work subscale
Symptoms in low back only
Symptoms not distal to the knee
At least one hip int rot range of >35 degrees
Hypomobility at one or more lumbar levels with spring testing
Pain at one or more lumbar levels with spring testing
Does not peripheralize with lumbar single movement testing
Negative compression/distraction test
Those variables retained have already been reported.
I guess it isn't the variables that weren't retained that I'm curious about, it is the ones that never even made it that far.
jon
_____________________________
[URL=http://www.sonymusic.com/clips/selection/30/064887/064887_03_03_30.wav]Evidence[/URL]
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 6, 2005 1:04:00 AM
|
|
|
Alex Brenner PT MPT OCS
Posts: 1057
Joined: February 29, 2004
From: Kentucky
Status: offline
|
Hi Jon. I know you have struggled with this study for a while. :)
This study has definetly changed the way I conduct a physical exam and it is interesting to talk to the authors about it too. Tim Flynn was our spine instructor when I was in PT school and he taught us an osteopathic or "palpation" model. I am pretty sure he has moved away from this model as have I. Rob Wainner currently teaches spine at my old school (Army-Baylor University) and in the past year I was able to sit in on some of his classes and I know they no longer teach the osteopathic model.
I also had the chance to ask Tim Flynn about his thoughts on the clinical predictors that "fell out" in this study. In particular, I was interested in the hip internal rotation > 35 degrees, which Flynn et al admits they are not sure what it means or why it is a clinical predictor. To me it would make more sense that a lack of hip internal rotation would be more of a predictor for success with spinal manipulation. This study certainly raises a lot of questions but good research should always do this.
In my opinion, this is a great study and has changed the way that I conduct my physical exam and how I intervene with acute low back pain. I was manipulating spines before this study was published, now I am clinically able to make better choices about who I manipulate.
_____________________________
Alex Brenner, PT, MPT, OCS
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 6, 2005 2:44:00 AM
|
|
|
Alex Brenner PT MPT OCS
Posts: 1057
Joined: February 29, 2004
From: Kentucky
Status: offline
|
Jon, Which variable that did not make the list are you most curious about?
_____________________________
Alex Brenner, PT, MPT, OCS
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 6, 2005 7:32:00 AM
|
|
|
Bournephysio
Posts: 585
Joined: April 25, 2002
From: Calgary
Status: offline
|
I still need to read this study so forgive me if I'm off. Are the starred items points that they found to be significant or are only some of them significant? What were "the typical statistical gymnastics"? That could very well have something to do with what they chose to include.
So they included a segmental mobility test?
A couple of points about regression analysis:
1. The more factors you include the lower the power of the study so they will have wanted to limit the number of items they included.
2. Ideally, you want the items included to be as independant as possible. for example you wouldn't included lumbar rom and forward flexion in the same analysis because forward flexion is partly determined by lumbar rom.
Those two factors may have had something to do with which factors they chose.
Doug
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 6, 2005 10:26:00 AM
|
|
|
Jon Newman
Posts: 1707
Joined: April 24, 2004
From: Amherst, WI
Status: offline
|
Doug,
From the original article (Spine, Vol27(24) Dec 15 2002):
Data Analysis
"Kappa coefficients were calculated to determine the interrater reliability of the special tests for SI dysfunction. Patients were dichotomized based on success or nonsuccess with respect to the treatment. Success or nonsuccess was then used as the reference standard. Individual variables from the self-reports, history, and physical examination were tested for their univariate association with the reference standard using independent sample t tests for continous variables and chi squared tests for categorical variables. Variables with a significance level of P<0.15 were retained as potential prediction variable; a more liberal significance level was chosen at this stage to avoid excluding potential predictive variables. For continous variables with a significant univariate asociation, sensitivity and specificity values were calcualted for all possible cut-off points and then plotted as a ROC curve. The point on the curve nearest the upper left corner represents the value with the best diagnostic accuracy, and this point was selected as the cut-off defining a positive test. Sensitivity,acuracy, and this point was selected as the cut-off defining a positvie test. Sensitivity, specificity, and PLR were calcualted for all potential prediction variables....We chose to foucs on the PLR as opposed to the negative LR becuase we were attempting to predict success with manipulation based on positive test results. Potential prediction variables were entered into a stepwise logistic regression equatrion to determine the most parsimonious set of predictors for success using a multivariate model. A significance of 0.05 was required to enter a variable into the model and a significance of 0.01 was requried to remove it."
Army, I'll address your questions a bit later.
jon
_____________________________
[URL=http://www.sonymusic.com/clips/selection/30/064887/064887_03_03_30.wav]Evidence[/URL]
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 6, 2005 2:36:00 PM
|
|
|
Bournephysio
Posts: 585
Joined: April 25, 2002
From: Calgary
Status: offline
|
Wow thats a lot of stats. I will ad a disclaimer that I only have a basic knowledge of multiple regression.
It looks like they used t-tests and chi squared tests to determine what to test in the final regression. They are basically saying that if a variable may have some predictive ability on its own they would try it in their multiple regression. The problem is that some variables may not have significant predictive value unless other variables are also taken into acount. It also doesn't take into account if two variables are not independant of one another. This may not always be intuitive.
Doug
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 6, 2005 2:42:00 PM
|
|
|
Jon Newman
Posts: 1707
Joined: April 24, 2004
From: Amherst, WI
Status: offline
|
Hi Army,
I was initially fascinated by the fact that this particular manipulation is claimed to be a "SI region manipulation" yet no special tests for SI dysfunction made it to the point of predicting success.
Then I became fascinated by the fact that the only biomechanical exam item that predicted success was lumbar hypomobility at any level. The questionable reliability and ubiquitous presence of this finding can be left for another discussion.
Limited spinal flexion, extension sidebending etc did not predict success. I found this interesting as only one manipulation technique was performed. It would seem that something that corrects a biomechanical fault should have a biomechanical predictor of success.
Limited flexion, extension, sidebending and rotation and the reproduction of pain are the things a clinician considering manipulation tends to look at (if I'm not mistaken) when choosing which technique is likely to be successful. That is, they look at these things to predict success.
In another thread, there was a brief discussion about sham manipulation. In a sense, I think that is what this study accomplished in an unintended sort of way. That is, there were people who likely improved with the manipulation technique provided that should not have based on their clinical presentation (the opposite also likely happened). I expect this however, based on what it is we're dealing with.
I think a brilliant study would be to perform this same sort of exam with one technique regardless of presentation. One person examines and records what they believe the correct technique would be while another, oblivious to the results, performs the preselected technique. I'm sure it would make it past the ethics committee, as that is what this study did for the most part. I think this would add to our understanding of what is relevant in a clinical exam.
I'm also interested in noting that the predictors of success can be boiled down to acute non-sciatic back pain without significant yellow flags.
Doug, they included spring testing, does that count for "segmental mobility test"?
Lastly, the terrible spelling mistakes and repeated sentence in the quoted part of my previous post are my mistakes, not the authors' (who were Flynn, Fritz, Whitman, Wainner, Magel, Rendeiro, Butler, Garber, Allison).
jon
_____________________________
[URL=http://www.sonymusic.com/clips/selection/30/064887/064887_03_03_30.wav]Evidence[/URL]
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 7, 2005 6:46:00 AM
|
|
|
BRuchin
Posts: 43
Joined: December 9, 2004
From: Atlanta, Georgia
Status: offline
|
I like and have used this CDR in clinic and it works very well. I haven't finished reading the study yet, but I plan too. I also know that St. Augustine discusses this CDR to be used.
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 7, 2005 8:59:00 AM
|
|
|
Bournephysio
Posts: 585
Joined: April 25, 2002
From: Calgary
Status: offline
|
Jon, the technique is a very generic lumbar manipulation not an si manipulation, not matter what the authors say although after hearing fritz talk, I'm sure they'd agree.
"Then I became fascinated by the fact that the only biomechanical exam item that predicted success was lumbar hypomobility at any level. The questionable reliability and ubiquitous presence of this finding can be left for another discussion."
This is truly on of the most powerful aspects of the study. No matter what the validity (for measuring spinal motion or stiffness) or reliability for this test, it has been validated as a predictive test. One of the big problems I have with reliability studies is the use of cookbook standards of what is good or poor reliability (e.g. ICC >0.8). This study shows that the test is reliable enough to be used a predictor for who will respond positively to manipulation.
A spinal PA would be considered more of a stability test in the Canadian system and more of a mobility test in the Maitland system.
Doug
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 7, 2005 2:20:00 PM
|
|
|
Jon Newman
Posts: 1707
Joined: April 24, 2004
From: Amherst, WI
Status: offline
|
Hi Doug,
You are correct in noting that the information about the intended use of the technique was from the authors and they concede the non-specific effects of it.
As far as hypomobility representing a powerful aspect of the study: Since only 4 of 5 clinical predictors need to be present to predict success, hypomobility suddenly loses its power. That is, lumbar hypomobility is not needed for the technique to work. In fact, no biomechanical pathology needs to be present at all. The best univariate predictor of success was duration of symptoms, not a biomechanical factor.
My point has always been that while manipulation works, the reason why is likely flawed. Manipulation, regardless of how conservative or how bizzare a camp you sit in, always has biomechanics as the logic for technique selection despite biomechanical factors not being requisite for success. So, while the technique works, what is a better explanation for why than biomechanics flaws?
I concede that the logic that brings me to the above question might be screwed up as stated in an earlier post, but it seems like a rational question.
jon
_____________________________
[URL=http://www.sonymusic.com/clips/selection/30/064887/064887_03_03_30.wav]Evidence[/URL]
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 9, 2005 3:02:00 AM
|
|
|
childsjd
Posts: 63
Joined: June 29, 1998
Status: offline
|
All:
I have been reading the discussion threads regarding our recently published valdation study of the manipulation prediction rule with interest. It has been good to see the dialogue. I won't take the time here to respond to each of the individual issues that have been raised, but feel free to pose the key questions, and I will be happy to address them. There seems to be some misunderstanding about what we actually did.
In terms of hands-on courses, there will be 3 lumbar spine manipulation courses in April 2005 (one each in Tucson, AZ, Concord, NH, and Mobile, AL). You can visit [URL=http://www.evidenceinmotion.com]www.evidenceinmotion.com[/URL] for details. Let me know if you have any questions.
Looking forward to the discussion. By the way, please feel free to be critical here. Don't assume you should "back off" simply because you're having a discssion with the author. The dialogue is worthwhile. It's good to see thoughtful discussions about the evidence. Take care.
John Childs
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 9, 2005 3:20:00 AM
|
|
|
PTupdate.com
Posts: 1477
Joined: October 8, 2001
From: Pittsburgh, PA USA
Status: offline
|
John,
Why did the experimental group (manipulation group) recieve ROM exercises that the control exercise group did not? Why not just keep the groups the same, and add the testing variable?
Best,
Duffy
_____________________________
John M. Duffy, PT
Board Certified Orthopaedic Clinical Specialist
www.PTupdate.com
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 9, 2005 4:54:00 AM
|
|
|
SJBird55
Posts: 2438
Joined: May 10, 2004
From: Michigan
Status: offline
|
Before I ask my question, I would first like to say thanks for what you did. I think it is pretty cool to see therapists that believe in a particular technique to actually take the time and effort to publish studies regarding their beliefs.
The most difficult aspect I'm finding with the study is the one factor dealing with onset of injury. Someone basically 2 weeks post injury is one of the indicators. That's nice to know, but in the big picture, don't most folks with back pain have a resolution of symptoms within 8-10 weeks? I guess to support the intervention and to support cost-effectiveness, to me it would have been interesting to see a control group (and I guess my definition of a control being no manipulation and no in clinic exercise intervention but a provision of a home exercise program only because that could actually be done in the real civilian world by a physician). To then add that bit to the study to see if overall, the manipulation group was the group that reached less disability on the Oswestry in a quicker amount of time and compared to all groups also had less re-entry into the medical system for further care secondary to back pain later in the future. So, what was the rationale for not having a control group?
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 9, 2005 5:29:00 AM
|
|
|
Jon Newman
Posts: 1707
Joined: April 24, 2004
From: Amherst, WI
Status: offline
|
Hello John C.,
I'm not sure of titles I ought to be using. Since you did not use any title in signing your post, I'm hoping that means I'm not making a mistake in addressing you the way I did.
I am always so impressed when the author of an article actually participates in a public forum to answer questions about their article. I'm sure time constraints, among other issues prevent this from happening more often. Too bad.
I have asked numerous questions in my posts up this point. One question that I'm still pondering is the relationship of prediction, explanation and statistics (step-wise multiple regression in this example).
I understand that something that predicts does not necessarily explain. But shouldn't it be the case that variables that explain at least also predict?
In my searching about I had come across this article
http://pareonline.net/getvn.asp?v=8&n=2
that, while interesting, only confused me as to how to figure out if these assumptions were met. I would presume that it is standard to meet these assumptions and I tried to ascertain that from the original article. Am I correct in thinking that all these assumptions were met?
Thank you for participating.
jon
_____________________________
[URL=http://www.sonymusic.com/clips/selection/30/064887/064887_03_03_30.wav]Evidence[/URL]
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 9, 2005 7:26:00 AM
|
|
|
childsjd
Posts: 63
Joined: June 29, 1998
Status: offline
|
Duffy:
Good question. The only difference between the groups was during the first week of treatment. Patients in the manip group were manipulated twice and completed a basic ROM exercise as part of a home exercise program. Patients in the exercise group immediately began the aerobic and strengthening exercises, essentially a lumbar stab program based on some of O'Sullivan and Hides' work. By the way, despite Rick Deyo's comment in his editorial (well intended), this is a legitimate comparison since it has considerable evidence for its use. We addressed this in a response that was published on the Annals website last week. Beginning on week 2, patients in the manip group stopped getting the manip and ROM exercise and initiated the same exercise program as patients in the exercise group.
The rationale for not specifically isolating the effectiveness of manipulation was that doing so would not reflect clinical practice. Therapists don't typically limit their treatment solely to manipulation (nor should they), thus limting the treatment for research purposes solely to manipulation doesn't make sense. More importantly, the evidence s clear that manipulation is most effective when it's combined with exercise. To be honest though, we're still confident that most of the effect was attributable to the manip given that the ROM exercise was simply a pelvic tilt exercise (and there is no evidence to suggest ROM exercise is all that helpful).
The other issue was that we wanted to demonstrate that you don't have to spend hours with these patients to generate large clinical change in the subgroup likely to benefit from manip. Patients in the manip group were in the clinic a total of about 10 minutes for the treatment portion of the study during the first week (all of about 1-2 minutes for the manip and the remainder to perform the ROM exercise). The drop in pain and disability in the folks positive on the rule who received manip occurred during the first week, making a compelling case that it doesn't take much early on, then you can progress to a more exercise and strengthening-based rehab program.
I think someone else asked about why not compare to a true control group (ie, no treatment). This one is simple. The evidence is clear that something is better than nothing, thus withholding at least some minimally effective treatment in a control group during research is unethical. Plus, what's the big deal about finding that manipulation for the right subgroup is better than nothing? The intervention needs to compete against a compelling alternative, which lumanr stabilization exercises are. Hope this helps. I'd be glad to answer any other questions.
John
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 9, 2005 7:44:00 AM
|
|
|
Barrett
Posts: 967
Joined: July 28, 1999
From: Cuyahoga Falls, Ohio
Status: offline
|
I spent the first several years of my career manipulating people with low back pain before I abandoned the practice. I also taught many others to do the same. Having said that, I remain troubled by the studies looking statistically at the patient’s reported response in this way-not because it can’t be defended as a reason to go ahead and manipulate given the proper predictors of response, but because it treats “back pain” as if it were an essential diagnosis. There seems to be little concern for a theoretical approach that includes a tissue at fault in a certain way, and in a way that can be changed in the right direction with manipulation. Why have we abandoned this issue?
From the healthfraud listserv recently: "evidence-based medicine - the discipline that insists on proof that time-honored medical practices and procedures are actually effective.” And then this from another regular contributor: “This seems an incomplete definition to me. I'd prefer David Sackett's definition: "the conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research."
I much prefer the second definition as well, but it seems the study we’re looking at here is satisfied with the first.
_____________________________
Barrett L. Dorko P.T.
http://barrettdorko.com
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 9, 2005 10:15:00 AM
|
|
|
childsjd
Posts: 63
Joined: June 29, 1998
Status: offline
|
Barrett:
We have only recently (at least in historical terms)understood the mechanism for how aspirin (and many other efficacious remedies) work, yet it has spared many individuals from prolonged pain for hundreds of years. Should we have waited until we knew understood the prostaglandin pathway before recommending it?
Similarly, attempts to identify pathoanatomic causes of back pain have been entirely unsuccessful, and likely account for why the management of back pain in the 20th century was a "medical disaster" (Gordon Waddell), leading to millions of dollars of unneccssary imaging studies that in many cases cause more harm than good. Do we tell patients with acute back pain to wait another 20 years until we understand why manipulation works before providing treatment, despite have good evidence to suggest it is highly effective for at least a subgroup of patients with back pain. This seems utterly preposterous. Understanding the mechanism for why particular treatments work is important; however, patients just want their pain and disability to go away. I'll take my chances and help as many patients as I can while research continues to elucidate exactly why it works.
The other issue is that anyone who adhere's to David Sackett's definition of EBP would be the first to integrate an efficiacious treatment into their practice despite not understanding the precise mechanism. "Current best evidence" implies that we don't always have all the answers. But we still have to make decisions about the care of the patient sitting in front of us with the evidence that we do have. I would be very interested if you are aware of a treatment that generates the dramatic changes we observed in the subgroup of patients who were positive on the rule and received manipulation. Bottom line is that the failure to use manipulation for at least these patients with back pain is likely synonymous with providing suboptimal care unless someone can demonstrate otherwise (with data, not a song and dance about what works best in "their hands".)
John
|
|
|
|
|
Re: Look at this validation study guys, worth reading! - January 9, 2005 10:26:00 AM
|
|
|
childsjd
Posts: 63
Joined: June 29, 1998
Status: offline
|
SJBird55:
Having symptoms for less than 16 days was indeed one of the predictors of a successful outcome from manipulation (and in fact was one of the most powerful). The logic you use in your argument is good, suggesting that perhaps folks with acute back pain will do well no matter what treatment you give them. In other words, perhaps the positive changes we observed were attributed to the passage of time rather than a specific response to manipulation. The problem is that if this were the case, we should have seen similar improvements among the patients with less than 16 days who received the exercise intervention without manipulation. This was not the case. In fact, I don't have the study in front of me, but the positive likelihood ratio among patients who were positive on the rule and recieved exercise was close to 1, suggesting that the patient's status on the rule was unable to distingusish between patients who achieved a successful outcome and those did not. Bottom line, this study (unlike with the original development study where your argument held more merit) eliminates the possibility that the improvements could be attributed to the passage of time but rather were a response to the manipulation (and ROM exercise). Hope this helps.
John
|
|
|
|
 New Messages |
 No New Messages |
 Hot Topic w/ New Messages |
 Hot Topic w/o New Messages |
 Locked w/ New Messages |
 Locked w/o New Messages |
|
 Post New Thread
Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts |
|
0.109
|
Printable Version
