Thursday, August 29, 2013

The “US effect” Biases Behavioral Research

This tight clustering went away when it came to the behavioral studies, though, which had a much greater frequency of extreme results. And here, US-based researchers showed a clear indication of bias. Rather than being evenly distributed around the typical answer, the results were much more likely to support the experimental hypothesis. Why do so many papers that produce positive results get published? Fanelli and Ioannidis suggest that it's because behavioral sciences don't have a robust set of theories, in contrast to traditional biology (which has things like evolution and genetics). Without that, researchers are able to be very flexible about the hypotheses that they propose and the methods they use to test them. They also argue that the "publish or perish" mentality that drives US scientists motivates people to report positive results. The two combine, they argue, to make "US researchers potentially more likely to express an underlying propensity to report strong and significant findings."
Not surprised actually. I think that there is also the political correct issue of not wanting to report something that would offend any number of groups. For example, when it Psychology Today reported on the findings that Black women were objectively less attractive (compared to others), a whole storm erupted. Many people, including myself took offense to the publication. However, what I took offense to was the explanation rather than the actual data. Since if we do observe the behavior of people, we cannot miss a strong negative reaction to black (as in dark skinned and tightly curled hair, thick lipped and broad of nose) women. We see it even in publications that claim to represent said women. A lot of people went after the results, when they are pretty much undeniable. The explanation fell far from the root of the problem though and that was the problem with the publications.

Today the medical journals that previously declared certain behaviors as diseases no longer do so for reasons that IMO have nothing to do with objective science but with politics. If one wishes to publish material that is contrary to the new "canon" you can expect to either not be published or to have one's career deaded quickly.

So it's not just a case of "publish or perish" that is a likely culprit but publishing the "right stuff". For example I know someone working on a dissertation in which they asserted that a particular community was segregated. I've long ago explained the difference between separation and segregation. While it may have been proper to declare certain areas segregated in say 1940 because it was clear that black people (or others) could NOT live in certain areas regardless of income, etc. it is impossible to make such a claim in 2013. if one is going to assert a place segregated in 2013 one has to prove that point. Until then you can only factually state that a particular neighborhood is primarily inhabited by a particular ethnic or racial group and that is all. That persons that far up in academia does not realize this basic issue is, in my opinion, symptomatic of the biases that are in the behavioral sciences. This can only be eradicated by enforcing a strict "math" rule to such research. In math 1+1 =2. No way around that. You can make any kind of explanation you want but if you don't show 2 as the proper answer and show your work, it isn't valid. Period.