This was likely to be a linkfest item, but I made the decision to convert it into a mini-publish.
Right here’s Jessica Hullman on whether “rigor-boosting techniques” (e.g., preregistration, huge sample sizes, open details) are a distraction–or at minimum, an ineffective gateway–to contemplating tough about harder troubles like “what are you even measuring, truly?” Features a link to Devezer et al. 2021, which statements that most arguments for bettering methodological rigor in the sciences aren’t them selves all that arduous.
I have blended thoughts on this.
On the just one hand, I’m sympathetic to the argument that absence of statistical or methodological rigor isn’t all that massive a offer in the grand plan of factors. I agree that the most common and critical troubles in scientific investigation are hard to fix, specifically mainly because they’re not amenable to narrowly-concentrated technological or procedural fixes. There’s no checklist you can follow to do superior science.
On the other hand, I’m reflexively suspicious of arguments from incremental, doable reforms, on the grounds that incremental, doable reforms are a distraction from attacking extra vital and hard issues. I’m reflexively suspicious for two motives. 1, these sorts of arguments make the fantastic the enemy of the fantastic. Two, I assume it’s commonly just not true that exertion becoming put into slender, doable reform X can be redirected in the direction of resolving massive, intractable trouble Y. All those two points aren’t interconvertible substitutes, I don’t imagine. At the very least, not to most men and women. I’m reflexively suspicious when anyone assumes that time, hard work, or income remaining put to X could be place toward Y in its place. You have to have to set up that X and Y are in truth substitutes in most people today’s eyes. Or else exhibit that any individual has the electric power to incentivize or force people to substitute Y for X.