Research methods final

Belmont principles

• Respect for person- informed consent, protection for vulnerable populations

• Justice- equitable selection, fair distribution of burdens and benefits

• Beneficence- maximize benefits, minimize harms

Ethical approval vs. ethical research behavior

Ethical approval: Official permission from a review board to conduct a study.

Ethical research behavior: How researchers act responsibly and ethically during and after the study.

Honest error vs. research misconduct

Honest error: Unintentional mistake in research (e.g., miscalculation or typo).

Research misconduct: Intentional wrongdoing, like falsifying, fabricating, or plagiarizing data.

what does HIPAA protect

patients’ private health information (PHI) their medical records and any identifiable health data by keeping it confidential and secure.

Reliability

Consistency—getting the same results repeatedly.

Validity

Accuracy—measuring what you’re supposed to measure.

Why is reliability is necessary but not sufficient for validity

Reliability is necessary for validity because a measure has to be consistent before it can be accurate. If results keep changing, you can’t trust what it’s measuring.

Test-Retest Reliability

What it measures: Consistency of results over time.

• Example: Giving the same language assessment to a child two weeks apart and getting similar scores.

Intra-Rater Reliability

What it measures: Consistency of ratings or measurements made by the same person across multiple instances.

• Example: An SLP scoring a speech sample today and again next week. Do they give the same score?

Inter-Rater Reliability

• What it measures: Agreement between different observers or raters.

• Example: Two SLPs independently scoring a speech sample and arriving at similar conclusions.

Construct validity (most stringent)

Does the test measure the concept it's intended to?

• A standardized test designed to assess expressive language should not inadvertently measure vocabulary knowledge alone. It should capture the broader construct of expressive language ability.

Criterion validity

• Does the test correlate with a relevant outcome?

• Includes predictive and concurrent validity.

• Example: Preschool language screening scores predict later academic performance in literacy.

Content validity

• Does the test cover all relevant parts of the concept?

A speech intelligibility rating scale should include multiple speech contexts (e.g., single words, sentences,conversation) to fully represent intelligibility across settings.

Face validity (least stringent)

Does the test look like it measures what it should? Often by gathering opinions from experts

• Example: A fluency assessment that includes stuttering frequency, duration, and secondary behaviors could have high face validity to clinicians and clients.

What is the difference between internal and external validity?

Internal = can we trust the cause-and-effect conclusion?

Threats: history, maturation, testing, attrition, selection bias

External = can we generalize results to other settings/populations?

Threats: sampling bias, setting effects

What is treatment fidelity and why does it matter?

The degree to which an intervention is delivered as intended.

Without it, you can’t tell if outcomes reflect the treatment or inconsistent delivery.

Monitored through: training, checklists, observation/recording, inter-rater reliability.

non-experimental research

• Non-experimental research involves studying existing conditions without manipulating variables. It is systematic and includes a clear research question, participant selection, data collection, and analysis.

Experimental research

An experimental design is a research approach where the investigator manipulates one or more independent variables and randomly assigns participants to conditions to observe the effect on dependent variables.

Randomized vs. non-randomized designs

Randomized design: Participants are randomly assigned to groups, reducing bias and improving fairness.

Non-randomized design: Participants are assigned without randomization (e.g., by choice or existing groups), which can introduce bias.

Identify designs that provide stronger causal inference

• Strongest: Randomized experimental designs

• Weaker: Non-randomized experimental designs

• Weakest: Non-experimental designs

Population vs. Sample

• Population: Entire group you want to study.

• Sample: Smaller group actually studied, meant to represent the population.

Inclusion vs. Exclusion Criteria

• Inclusion criteria: Characteristics required to be in the study.

• Exclusion criteria: Characteristics that disqualify someone from the study

Sampling Error vs. Sampling Bias

• Sampling error: Natural difference between sample results and true population (random, unavoidable).

• Sampling bias: Systematic error from how the sample is chosen (leads to skewed results).

Probability vs. Non-probability Sampling

• Probability sampling: Everyone has a known chance of being selected (more representative, less bias).

• Non-probability sampling: Selection is not random (more convenient, but more bias).

Stratified sampling

• Population divided into subgroups (strata), then randomly sampled from each.

• Strengths: More representative, ensures all groups included.

• Drawbacks: More complex, time-consuming.

---

Convenience sampling

• Use whoever is easiest to reach.

• Strengths: Fast, cheap, easy.

• Drawbacks: High bias, not very generalizable.

Snowball sampling

• Participants recruit other participants.

• Strengths: Useful for hard-to-reach populations.

• Drawbacks: Bias from social networks, not representative.

Purpose of inferential statistics

Inferential statistics are methods that allow researchers to make educated guesses or inferences about a population based on data collected from a sample.

Null Hypothesis (H₀)

A statement that there is no effect, no difference, or no relationship between variables.

• Example: “There is no difference in speech intelligibility between two therapy methods.”

Meaning of: p-values, and α (alpha)

Tells us whether the observed results are likely or unlikely to occur by chance

• Determined using a p-value:

• If p < α, results are considered statistically significant → we reject H₀.

• If p ≥ α, results are not significant → we fail to reject H₀.

Difference between: Type I error , Type II error , Sampling error

Type I error: Saying there is an effect when there isn’t (false positive; rejecting a true null hypothesis).

Type II error: Missing a real effect (false negative; failing to reject a false null hypothesis).

Sampling error: Natural random difference between a sample result and the true population value (not a decision error, just variability).

Main effect: ANNOVA

The overall effect of one independent variable on the dependent variable, ignoring other variables

Interaction effect: ANNOVA

When the effect of one independent variable depends on the level of another variable.

Why interactions matter

• They show whether variables work together or change each other’s effects.

• Without checking interactions, you might misinterpret results.

Why interpret interactions first

• If an interaction is significant, it changes how you understand main effects.

• Main effects can be misleading when interactions are present.

The general order of inferential analysis:

1. Check assumptions

2. Run the model (including interaction)

3. Conduct post-hoc analyses only when appropriate