Critical Perspectives

according to Schmidt 2009, replication/repeatedly finding the same results protects… addresses… and tests whether…

against false positives, researcher fraud, findings generalise to different populations

direct replication is the scientific attempt to…

recreate critical elements of an original study

conceptual replication is where the same/similar results are…

an indication that findings are robust to alternative research designs

what is the percentage of studies that are actually replicated?

36%

outcome switching refers to…

changing the outcomes of interest in study depending on observed results

outcome switching is known as… in the nine circles of scientific hell

p value fishing

one reason for lack of replicability due to lack of statistical power is…

the use of small samples

John et al 2012’s study surveyed over 2000 psychologists in the US about involvement in questionable research practices. What was the result?

the percentage of respondents who engaged in the questionable practices was high

Moderators is another factor that influences replicability. What does this refer to?

variables that influence the nature of an effect, for example country or culture

Doyen and colleagues were accused of what malpractice that impacts replicability?

scientific error and poor replication

publication bias is another factor influencing replicability. what does this refer to?

findings that are statistically significant are more likely to be published than those that are not

open science is considered a… to reducing ‘sloppy science’

potential solution

open science is the process of making content and the process of producing evidence…

transparent and accessible to others

open methodology is the process of…

documenting methods and process by which those methods were developed

pre-registration involves defining… before observing research outcomes

research questions, methods and the approach to analysis

pre-registration prevents… which stands for…

HARKing, Hypothesising after results are known

Protzco et al used 4 labs to replicate 16 novel experimental findings using rigour-enhancing practices: confirmatory tests, large sample sizes, preregistration and methodological transparency. replicated expected effects found in 86% of attempts. What does this show?

that preregistration improves replicability

to integrate preregistration into the publication process, registered reports must be completed, which are done in 2 stages:

reviewers and editors assess detailed protocol, following good reviews the journal offers acceptance in principle

to integrate preregistration into the publication process, open source materials and code must be used. this involves using… and making the dataset… The Data needs to be FAIR, which means…

open source technology, freely available; findable, accessible, interoperable and reusable

open access for the publication of any findings has two types, these are…

gold open access and green open access

gold open access publishing involves researchers…

paying a journal to publish their article, with the final version freely and permanently accessible to everyone

green open access publishing involves researchers…

self-archiving the articles, putting an unformatted version into a repository

open access publication works are… and given more… and facilitate…

used more, coverage by journalists, meta-research

Transparency and Openness Promotion (TOP) Guidelines were developed by Nosek et al, these are: citation standards, data transparency, analytic methods transparency, research materials transparency… what are the other 4?

design and analysis transparency, preregistration of studies, preregistration of analysis plans, replication