Research Exam 2

Internal validity

the degree to which a study accurately establishes a causal relationship

External validity

pertains to the generalizability of the study's findings to other settings or populations.

Does statistical significance between two treatment groups mean clinical significance?

No. Statistical significance means the result is unlikely due to chance (e.g., p < 0.05), but it does not guarantee that the change is meaningful or beneficial to the patient in real life.

What is statistical significance?

A mathematical indication that a result is unlikely to have occurred by chance, typically defined by a p-value less than 0.05.

How do you know if something is statistically significant?

Look for a p-value < 0.05 (ɑ = 0.05) Indicates <5% probability the result occurred by chance Often supported by high statistical power (e.g., 0.95)

What is clinical significance?

The practical importance of a treatment effect—whether it leads to a noticeable, meaningful improvement in a patient's condition or daily life

How do you know if something is clinically significant?

Determine if the change exceeds the Minimally Clinically Important Difference (MCID). In short: MCID > MDC confirms the change is both detectable and meaningful to the patient

What is the Minimal Detectable Change (MDC)?

The smallest amount of change that can happen in a variable before it is significant; NOT dependent on the patient

What is the Minimal Clinically Important Difference (MCID)?

The smallest change in a treatment outcome that a patient would identify as important and beneficial.

What is a placebo group?

a group receives a sham treatment that mimics the real one but lacks active ingredients. It helps control for psychological and contextual effects like patient expectations and attention

What is a no-treatment group?

A group that receives no intervention. It helps measure natural recovery, spontaneous improvement, or baseline changes over time.

What is the purpose of com paring treatment, placebo, and no-treatment groups?

Comparing all three helps isolate the true effect of the treatment by accounting for placebo responses and natural changes. This strengthens the validity of the study

How does the comparison between the treatment, placebo, and no treatment group relate to statistical significance?

If the treatment group shows greater improvement than both placebo and no-treatment groups, and the difference is unlikely due to chance (e.g., p < .05), the result is considered statistically significant

What is effect size?

a quantitative measure of the magnitude of a phenomenon; helps describe how much difference or impact a treatment or intervention has between groups.

Absolute Effect Size?

The raw difference between two group means (e.g., Mean₁ − Mean₂) Often shown in tables of results

Standardized Effect Size?

Adjusts for sample variability Commonly referred to as Cohen's d Useful for comparing effects across different studies

Small effect size (Cohen's d)

0.20-0.49

Medium effect size (Cohen's d)

0.50-0.80

Large effect size (Cohen's d)

>0.80

Number needed to treat (NNT)

the number of patients who need to receive a treatment over a specific time period to achieve one desired outcome (e.g., prevent one adverse event or produce one benefit)

What does a low NNT mean?

Low NNT = more effective treatment Fewer patients need to be treated for one to benefit Example: NNT = 1 means every patient treated will benefit—this is ideal.

What does a high NNT mean?

High NNT = less effective treatment More patients must be treated for one to benefit Indicates a smaller impact of the intervention on individual outcomes

What is "Number Needed to Harm" (NNH)?

NNH is the opposite of NNT. It represents how many patients need to be exposed to a treatment before one experiences an adverse effect.

What is statistical power?

Power is the probability that a statistical test will detect a true effect (difference or relationship) if one actually exists. It helps determine whether a study can reliably find meaningful results.

Why is power important for study validity?

Power analysis helps ensure the study is designed to detect meaningful effects

The concerns associated with power?

A study with low power (underpowered) may fail to detect real differences, leading to false negatives This threatens internal validity, especially if the study concludes "no difference" when one may exist

How is power determined?

Power is influenced by several factors, especially sample size Researchers can calculate power before the study (a priori) using pilot data or after the study (post hoc) to interpret null results.

What is the minimum recommended power?

The standard threshold is >80% power, meaning there is less than a 20% chance of missing a true effect (Type II error). This level balances feasibility with statistical reliability.

What Is Sensitivity (Sn)?

The ability of a test to correctly identify those with the condition. High sensitivity = few false negatives

What Is Specificity (Sp)?

The ability of a test to correctly identify those without the condition. High specificity = few false positives

Why Are Sensitivity and Specificity Important?

They measure a test's diagnostic accuracy Help clinicians choose the right test for ruling in or ruling out conditions Guide interpretation of test results in clinical decision-making

Sensitivity Calculation

Sn = a / (a + c)

Specificity Calculation

Sp = d / (b + d)

What Is SnNOut?

If a test is highly sensitive and the result is negative, it helps rule OUT the condition.

What Is SpPIn?

If a test is highly specific and the result is positive, it helps rule IN the condition.

Acceptable Values for Sensitivity and Specificity

> 80% is desirable Closer to 100% = stronger diagnostic confidence

Key Difference for Sensitivity and Specificity

Sensitivity focuses on detecting disease (true positives) Specificity focuses on excluding disease (true negatives).

What Is Predictive Value?

Estimates how likely a test result reflects the true disease status.

What is a positive predictive valve (PPV)?

Likelihood that a person with a positive test truly has the disease

What is a negative predictive valve (NPV)?

Likelihood that a person with a negative test truly does not have the disease

Equations for PPV and NPV

PPV = [a] / [a + b]

NPV = [d] / [d + c]

Predictive Values Acceptable Values

80% is considered good, but loser to 100% means stronger confidence in test results

Clinical Application for high NPV and low PPV

High NPV: Helps rule out conditions (e.g., appendicitis) Low PPV: Positive result may not confirm disease

Likelihood Ratios (LR)

LRs are used for assessing the value of performing a diagnostic test and reflect a test's ability to provide persuasive information.

LR+

The likelihood a positive result was obtained in a person with the condition vs. a person without (clinical concept of 'ruling-in disease').

LR-

The likelihood a negative result was obtained in a person with the condition vs. a person without (clinical concept of 'ruling-out disease').

LR Interpretation > 10

Large and often conclusive increase in the likelihood of disease

LR Interpretation 5 - 10

Moderate increase in the likelihood of disease.

LR Interpretation 2 - 5

Small increase in the likelihood of disease.

LR Interpretation 1 - 2

Minimal increase in the likelihood of disease.

LR Interpretation 1

No change in the likelihood of disease.

LR Interpretation 0.5 - 1.0

Minimal decrease in the likelihood of disease.

LR Interpretation 0.2 - 0.5

Small decrease in the likelihood of disease.

LR Interpretation 0.1 - 0.2

Moderate decrease in the likelihood of disease.

LR Interpretation < 0.1

Large and often conclusive decrease in the likelihood of disease.

LR = 1 (Clinical Meaning)

A test with LR = 1 is a coin flip (useless) in terms of identifying whether a person is likely to have the condition.

LR+ Effect

LR+ tells us how many times more likely a positive test will be seen in those with the disorder than those without.

What is a Receiver Operating Characteristic (ROC) Curve?

An ROC curve is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. It shows how well a test can distinguish between two diagnostic groups.

What does the ROC Curve evaluate?

It evaluates different cut scores or threshold values to find the optimal balance between sensitivity (Sn) and specificity (Sp).

What is the Area Under the Curve (AUC)?

AUC quantifies the overall accuracy of the test AUC = 1.0 means perfect discrimination The closer the AUC is to 1, the better the test is at distinguishing between conditions.

What is the optimal cut score?

The optimal cut score is the threshold that provides the highest combined value of sensitivity and specificity, maximizing diagnostic accuracy.

What are confidence intervals (CIs)?

Confidence intervals are ranges of scores within which the true population value is estimated to lie, based on sample data. They are expressed with a probability level (e.g., 90%, 95%)

Why are confidence intervals important?

They help assess the precision of study findings Narrow CIs suggest less variability and greater confidence in the estimate Wider CIs indicate more variability and less certainty Clinically, CIs help determine how meaningful and reliable a result may be

How do you interpret a mean with its confidence interval?

The mean is the central estimate of the sample The CI shows the range where the true population mean is likely to fall Example: If Mean = 50 and 95% CI = [45, 55], we are 95% confident the true mean lies between 45 and 55.

What other ways are errors shown in research?

Standard Deviation (SD): Measures variability within the sample Standard Error of the Mean (SEM): Estimates how far the sample mean is likely to be from the true population mean.

What is the primary goal of descriptive research?

To document the nature of existing phenomena as they occur naturally.

How is data collected in descriptive research?

Data is collected as it exists naturally, without manipulation.

What kind of changes does descriptive research aim to describe?

Changes in variables over time.

When is descriptive research commonly used?

To classify individuals, to identify relevant variables, and to generate new research questions

What studies are described as descriptive research?

Case study, developmental research (longitudinal method and cross-sectional method), normative research, qualitative research, and descriptive surveys

What is a case study in research?

A detailed description of interesting, new, or unique cases.

What is the primary purpose of a case study?

To analyze and understand factors important to the etiology, care, and outcome of the subject's problems.

When are case studies typically used?

For in-depth descriptions, to explore individual conditions, and to document individual treatments.

What is a ma jor contribution of case studies to research?

Discovery of new relationships between clinical factors.

What types of information are in cluded in a case study?

Full history Presenting problem and symptoms Prior treatment Demographics and social factors Treatment plan and justification Literature review Documentation of all interventions and responses Follow-up

What kind of data is included in a case study database?

Qualitative and quantitative data, and objective and subjective observations

What is a common clinical setting for presenting case studies?

Clinical Grand Rounds

What is a key advantage of case studies in clinical research?

They capture the totality of the experience, offering a holistic view of the subject's condition and care

How do case studies help identify impor tant variables?

By exploring unique or special circumstances in depth, they reveal factors that may influence outcomes.

What is a major limitation of case studies regarding experimental control?

They lack control over variables, making it difficult to isolate cause-and-effect relationships.

Why is generalizability limited in case studies?

Findings are based on individual cases, which may not apply to broader populations.

What are the validity concerns in case study re search?

Weak internal validity: Limited ability to establish causal relationships Limited external validity: Findings may not be applicable across settings or populations

How can quantifying observations improve case study validity?

It adds objectivity and allows for clearer comparisons and analysis.

Why are repeated measurements important in case studies?

They increase reliability and help track changes over time.

How does using multiple dependent variables enhance validity?

It provides a more comprehensive view of treatment effects and outcomes.

Why select outcomes that show large and imme diate changes?

These outcomes are easier to detect and attribute to the intervention, strength ening internal validity

How does using more than one setting improve case study validity?

It increases generalizability by showing that findings apply across different contexts

What is the value of including follow-up visits in a case study?

They help assess the durability and long-term impact of the intervention.

How does referencing literature support the validity of a case study?

It grounds the treatment approach in existing evidence and enhances credibility.

What is developmental research?

A study that describes developmental changes and the sequence of those changes over a period of time.

What are the two main classifications of developmental research?

Longitudinal method: Observes the same individuals over time Cross-sectional method: Compares individuals of different ages at one point in time

What is the longitudinal method in developmental research?

A research design that follows the same cohort of subjects over time, with repeat ed measurements at specific intervals

What do differences observed over time in longitudinal studies represent?

Developmental changes.

What types of research common ly use the longitudinal method?

Intellectual

Physical

Social

Developmental

Pathological

What are two key advantages of the longitudinal method?

Tracks the same individuals, reducing variability Reveals patterns of change over time

What are major limitations of longitudinal research?

Requires extended time commitment Demands substantial funding and resources Subjects must remain 100% compliant Subjects cannot be replaced once enrolled

What are common threats to validity in longitudinal studies?

Testing effects

Attrition

Confounding variables that coincide with developmental changes

What is the cross-sectional method in research?

A design that studies various developmental levels at one point in time using different groups at each level

What type of participant does each group in a cross-sectional study represent?

A "typical individual" at that developmental stage.

Why is the cross-sectional method used more often than the longitudinal method?

It is more efficient Requires less time and fewer resources