Research Methodology and Validity

The extent to which an instrument measures what it is intended to measure.

Validity

The extent to which the clinical outcome of interest (dependent variable) in a study is caused by the treatment (independent variable).

Internal Validity

What are three common threats to internal validity?

Regression, History, and Maturation

MRH

Systematic error in the estimate of effect due to procedures used to select subjects or factors influencing participation/follow-up.

Selection Bias

Changes in study outcomes due to external events occurring during the study.

History Threat

Naturally occurring changes over time that could be confused with a treatment effect.

Maturation Threat

Bias caused by differential dropout of patients between treatment and control groups in an RCT.

Attrition Bias

Bias occurring because of changes in outcomes due to repeated prior assessments.

Testing Threat

Changes in outcomes due to the instrumentation or measurement technique used.

Instrumentation Threat

Shift in initial extreme measures toward the mean in subsequent measures due to statistical variability.

Regression Threat

The extent to which study results can be generalized to other populations or settings.

External Validity

A researcher at UTEP is interested in assessing the average IQ of UTEP students. The researcher recruits 45 participants from their Theoretical Dance class. What threat to validity is the researcher most likely to commit

threat to External Validity

A researcher is investigating the impact of a new weight loss drug. Suppose that the weight loss drug is given to individuals who volunteered to be part of a weight loss program and that the comparison condition includes only individuals who were not volunteered in the weight loss program. What threat to internal validity is the researcher likely to commit:

Selection Bias

A faculty member was interested in pharmacy student's ability to react quickly in emergency situations while participating in their Objective Structured Clinical Exams (OSCE). Prior to the OSCE, the faculty member gave his students a practice problem which included time-sensitive emergency situation (similar to the situation they would encounter in their OSCE) that they needed to resolve. Which threat to internal validity is most likely to be present due to the faculty member providing a pre-test.

Testing Threat

A researcher used a physiological measure of stress (pulse oximeter) which assesses heart rate and blood oxygen levels. Midway through the study, the researcher realized that he received more precise measurements if the pulse oximeter was placed on the participants thumb instead of their index finger. What threat to internal validity does this scenario most likely represent.

Instrumentation Threat

A student earned a perfect score on their PCOA and was asked to take the test again because the student was suspected of cheating. The student did not receive a perfect score the second time that they took the PCOA, however, the student still scored extremely high. What is most likely the explanation for not receiving a perfect score again?

Regression Bias

What are the 5 main types of study bias?

Investigator, Performance, Attrition, Detection, and Selection

Technique ensuring patients, investigators, and monitors do not know treatment allocation until study end.

Blinding (or Masking)

Blinding that includes patients, investigators, and external outcome-monitoring groups.

Triple Blind

A study design where all participants, investigators, and assessors know treatment allocations.

Open Label Design

Process of assigning patients to treatment or control groups by chance alone.

Randomization

A statistical estimation of the magnitude or strength of the association between variables.

Effect Size

The likelihood of detecting an effect in a sample if the effect truly exists in the population.

Statistical Power

Suppose you are conducting your own study that investigates the impact of a new drug that reduces epileptic seizures in children. As you search the literature for other studies in the area, you find a study that is very similar to the study you are designing and thus you choose to replicate findings from the previous study. As you review the previous study, you find that the effect size was very large (r = 0.78; Cohen's d = 2.50). This finding tells you that when you conduct a Power Analysis, you will most likely be able to recruit

B. a smaller sample size, because larger effect sizes require smaller samples to detect an effect

Consists of an inert (chemically inactive) substance (such as lactose powder) that is identical in appearance (shape and form) to the active treatment.

Placebo

Type of study designed to determine the effects of the intervention under the conditions that the drug is most often used in the clinical setting.

Drug Effectiveness Study

A _____________ study is when the independent variable is varied in a series of increasing levels. The more levels of treatment that are administered, the finer the assessment can be of the functional form of dosage effects.

Dose-Response Study

A _____________ involves a periodic analysis of study results while the study is ongoing.

Interim Analysis

A __________ is an analysis of patient outcomes as if all of the subjects completed the study in their originally assigned group.

Intent to Treat Analysis

Which of the following are limitations of a Randomized Control Trial (RCT)? Please select all that apply:

High expense/time, high dropout rates, and potential Hawthorne-type learning effects.

The extent to which the results of a study can be generalized to other populations or settings.

C. External Validity

What percentage of values fall within 1 SD and 2 SDs of the mean in a normal distribution?

68% within 1 SD, and 95% within 2 SDs.