Research Design - Procedural

Application of concepts, complex definitions

You decided to hold a confound constant in order to limit its influence. What do you expect to happen to your results?

Internal validity is increased at the expense of external validity. Results are less generalizable because they are limited to only one level of the confound.

You want to control for testing effects. What could you do?

Implement a Solomon 4-group design, avoid reactive pre-test measures, or make administration of the test an independent variable.

Calculate Rosenthal’s Minimum Threshold for 42 studies.

Fail-safe n formula: 5n + 10

Where n = number of studies included in the meta-analysis

5(42) + 10 = 220

220 studies of null findings are needed to render the mean effect size insignificant.

Create a diagram for a standard treatment control design.

Tx: R O X O

CTRL: R O X_TAU O

Create a diagram for a placebo control design.

Tx: R O X O

CTRL: R O X_P O

Create a diagram for a dismantled control design.

Tx: R O X_full O

CTRL: R O X_minus O

Create a diagram for a pseudo-experimental design.

Tx: Non-R O X O

Create a diagram for a combined control design.

Tx: R O X_T O

CTRL: R O X_{T + A} O

Create a diagram for a quasi-experimental design.

Tx: Non-R O X O

COMP: Non-R O O

Create a diagram for an interrupted time series design.

Tx: Non-R O_i O X O O_j

Did you know that Ph.D. students are 43.17% more likely to experience burnout than Psy.D. students? Describe this in terms of relative risk.

[probability of a Ph.D. student experiencing burnout] / [probability of a Psy.D. student experiencing burnout]

Probability of Ph.D. burnout = 1.4317

Probability of Psy.D. burnout = 1

Relative risk (odds ratio) = 1.4317

Comparison of Ph.D. to Psy.D. burnout = 1.4317 - 1 = 0.4317

Conversion of comparison to % value: 0.4317(100) = 43.17%

Snow et al. just published a study on burnout in grad students. Where can you expect to find the operational definitions?

In the methods section of the study, where specific variables and measurements are described. If there is a measures subsection, it will be there.

You have finished evaluating publication bias in your meta-analysis and you run a Q statistic. You get a significant result. What do you do now?

Conduct a moderator analysis to determine if there are any variables regulating the extent to which heterogeneity influences your results.

You have elegantly designed a randomized clinical trial but you are worried about “personal style” effecting results. What should you do?

Operationalize “personal style” and incorporate it into the study design as an independent variable.

Create a diagram for a waitlist control design.

Tx: R O X O O

CTRL: R O O X O

Create a diagram for a post-test only experimental design.

Tx: R X O

CTRL: R O

Create a diagram for a simple true experimental design.

Tx: R O X O

CTRL: R O O

You want to see if program type (Ph.D. or Psy.D.) predicts burnout, but you are worried about location of the program influencing the results. What should you do?

If using a quantitative burnout questionnaire, run a hierarchical regression with location in block 1 and program type in block 2.

If trying to assess an odds ratio, run a logistic regression and include location as a moderator.