Research Exam 2 pt. 1

p. 18- of notes

Nonexperimental research

lacks the manipulation of an IV, random assignment of participants to conditions, or both; generally cannot link changes in IV to differences in DV; still valuable research

Experimental research is appropriate when…

there is a specific question about a causal relationship between variables and it is possible, feasible, and ethical to do the experiment

Non-experimental research is used when experimental research requirements are not met; therefore, it can be used for hypotheses/questions that can be:

1. about a single variable, rather than a statistic relationship

2. about a non-causal statistic relationship, such as correlated variables

3. about a causal relationship where the IV is not manipulated or the participants cannot be randomly assigned to conditions or orders of condition

4. broad and exploratory, or about what it’s like to have a particular experience

Distinction between experimental and non-experimental research is generally dictated by

the nature of the question; however, the two approaches can complement the same research in different ways

Types of Non-Experimental Research

1. Single variable research (quantitative)

2. Correlational or quasi-experimental research: examine relationship w/o variable manipulation (quantitative)

3. Qualitative research

Correlational research

Measures 2 variables with little or no attempt to control extraneous variables and assesses relationship

Quasi-experimental research

manipulates an IV but does not randomly assign participants to conditions

Qualitative (non-numerical data) research

rather than statistical analysis, it has a different set of analysis tools such as thematic analysis

Planning a set of procedures (sci meth step 3) is done by

choosing from a variety of study designs

File/document review (or restrospective) study design

• Conducting research with existing documentation, which may be quali or quantitative, an internal or external review, etc.

• Pros: data already exists and may be easy to obtain, especially for an internal review. Unobstrusive and relatively inexpensive, provides rich picture

• Cons: ease of review depends on organization of existing files. Incomplete or inaccurate info is common, creating biases in data, and there are no controls. Limited ability to infer causal relations

Survey study design

• qualitative or quantitative data, written or in person

• pros: quick, easy, an inexpensive, making it helpful for guiding experimental research where its valuable

• cons: limited ability to infer causal relations, randomization difficult, survey fatigue can affect results

Case study design

• example: dysphagia lusoria article, which has a left rather than right aberrant subclavian artery

• of an individual or organization, no comparison or control. Data is in-depth and collected over time

Case control study design

comparison of patients and matched controls; because groups are made to be as similar as possible, assignment is not random

pros and cons of case studies and case control studies

• pros: often possible based on routine medical care

• cons: limited ability to infer causal relations

Cross-sectional study

• different sets of sample participants at different ages to observed changes in outcome

• pros: relatively efficient to collect normative data about development or aging

• cons: no randomization, limited ability to infer causal relations

Longitudinal study design

• participants are followed over time to observe long-term changes to participant characteristics, behavior, or functioning

• pros: provide a generalizable picture of change within individuals of normative data info about development or aging; strong data

• cons: harder to collect than cross-sectional data, no randomization, limited ability to infer causal relations

Kasari article? p. 20 of notes

Experimental research

• provides the control necessary to infer causal relations

• involves an IV that is systematically manipulated by the experimenter

• for human subjects research, this typically includes random assignment to the different conditions under the IV

• the IV tested is usually a form of treatment

• we want to know what is being manipulated and what is being measured

Ebbinghaus experiments (study 1)

• took place 160 years ago

• first quantitative psychological experiments; only subject was himself

• systematic manipulation between time memorizing nonsense syllables and time recalling

• created the “forgetting curve”; longer time between study and test, more forgotten

• found it’s easier to remember words at the beginning and end due to primacy and recency

child Appraxia of Speech (study 2)

• CAS: motor planning speech disorder characterized by prosody, stress, and articulation difficulty

• RCT treatment by Murry et al. — ReST (rapid syllable transition treatment) and the Nuffield Dyspraxia Programme 3, compared percent accuracy at different time points

Independent Variable

• cause

• manipulation

• intervention

• can be with different conditions

• can be more than one per condition

Dependent Variable

• effect

• measurement

• outcome

Methods: Participants

• select a sample representative of a population through defining target pops, participant matching, and acknowledge and work around geographic restrictions

• recruitment by word of mouth, flyers, traditional and social media

• description as well as normative data are used to prove diagnoses in participants; this is very important and can be done through standardized tests and measurements

Methods: Materials

• fundamental to work with humans, but in research w/o humans as well

• human research materials can include: therapy programs, assessments, words/sentences, games, computer programs and interfaces, apparatuses put on clients

• Nonhuman research is also relevant to our discipline (may not include participants)

• Nonhuman research materials can include: biological samples, animal models such as pig larynxes, computer simulations

Methods: Procedures

• what is done in the experiment from start to finish (step-by-step recipe)

• number of sessions

• location of the study

• order of events

• how IV and outcome are implemented

• what experimenter was doing

• what participants were doing

Methods: Analysis

• explicit statements

• variables that were controlled and/or measured

• data wrangling: any cleaning and coding necessary

• statistical analyses to be used

• PICO question is repeated/restated as one or more hypotheses

Erickson + Sivasankar pig study

• simulated effects of acid reflux on excised pigs vocal folds to gain knowledge about humans

• done in this way to be ethical

• used hydrochloric acid, pepsin, and a combo

Storkel study

kids with DLD learning new words, different exposures, asked to recall and give definitions

complex studies can have more than one…

DV and IV at the same time

Constant variables

unchanging natural forces; constant properties of the environment like gravity, are not reported in an academic source

control variables

something about the experimental environment that could change but that the researcher holds constant or controls (e.g., room temperature, time of day); consistently reported in academic sources

covariate variables

properties of experimental environment that are not controlled but are accounted for, such as participant age; consistently reported

confound variables

• “mediator variables” that interfere with the ability to infer causal relations between the DV and IV

• not reported in an academic source because they typically aren’t discovered until later, wouldn’t be published if known

• often notable in poorly designed experiments, or inherent to poorly defined IVs

• regularly revealed in good research—in some sense unavoidable, but should be limited

• single most important means of reducing these: randomization

• 'may be a result of a lack of controls

[mancopes et al. review article notes p. 24]

Pre-test post-test experimental research design (ERD)

• appropriate for studies with groups of participants, ideally > or = to 10, often small groups for treatment research

• one group is given at least one treatment

• target behaviors are measured before and after treatment, main variable is time

• may be level 1b (more restricted studies) to 4; common level is 3; variation bc can be done many ways

• very susceptible to confounds, but good start for further research

ABAB ERD

• case studies and small groups of participants (< or = to 7)

• Best for when just one DV

• session-to-session data, back and forth between two conditions that are different (A: no/unrelated treat, B: treatment)

• designs can be modified to add more periods

• almost always level 3, may be lower

ABAC ERD

• appropriate for same group as ABAB

• A: almost always periods of no treatment given

• B and C: refer to unique treatments

• Can be modified to add more periods

• order of periods should vary across participants

• almost always level 3

Multiple-baselines research design (MBRD)

• case studies and groups < or = 7 when 1 or 2 treatments are being compared

• multiple dependent variables (targets) are measured, session-to-session data

• one DV may be related to another to serve as a measure of generalization (related target)

• baseline: measure of where a client is before treatment starts

• one or more treatments may be implemented, each with a baseline

• onset of each baseline treatment may be staggered; this is a benefit, proving treatment helps specific targets, so it’s what we want; all baseline measurements start at the same time to ensure improvement starts when each individual treatment starts

• alternatively, there could be one treatment for several targets simultaneously (treatment, generalization, and control targets), goal is to demonstrate generalizability outside of maturation

• level 3 on evidence hierarchy

Experimental research with small participant numbers is…

common in our field, and these lower-level experimental research designs still have value

Well-designed controlled experiment without randomization

• people are allocated to different interventions non-randomly

  • randomization may be unethical, or harder with money and time

  • researchers want to use cut-offs, such as most or least severe in a population

• may compare a participant group w/ a disorder to those w/o the disorder

• considered experimental so long as they have an IV manipulated and a DV outcome

Randomized controlled trial

• 3 necessary qualities: at least one treatment group, at least one group not receiving treatment (often control group getting usual treatment as normal), and random group assignment which may be done by an outside co.

• stages: enrollment, allocation, follow-up, analyses (group will likely have shrunk by this stage, be mindful of large dropouts)

• blinding: possible as allocation, during intervention, or analysis

  • sometimes beneficial to have multiple kinds

  • participant versus experimenter blinding

  • makes sense to blind participants in medicine testing; not so much in our field

Systematic Review

• comprehensive review of relevant studies on a clinical or health-related question (research about research, can be quantitative, is often qualitative, or both)

• no statistics: all data is secondary (nothing new), reported in peer-reviewed literature

• review should use 6 steps of “repeatable, analytical methods”

• quality depends on quality of reviewed research

Meta-analysis

• quantitative review of relevant studies; a subset of systematic reviews

• the strength of outcomes or effect sizes (outcome or DV, difference btwn groups) are assessed statistically

• all data is secondary; quality depends on that of reviewed experimental research

Quality controls

1. reliability

2. integrity/fidelity

3. validity

4. homogeneity and heterogeneity

5. narrow confidence bands in meta-analysis

Reliability

• appropriate measurements

• inter-observer: two observers/experimenters obtain the same results

• test-retest: same results obtained from one participant on multiple occasions

• parallel forms: creating two forms of a test results in similar results

• this is commonly reported for both treatment and assessment research

Integrity/Fidelity

• consistent delivery of appropriate procedures

• quality control procedure, commonly reported for treatment research

• an independent observer assesses implementation of the treatment protocol

Validity

• meaningful results; asks, “does this tell us what it’s supposed to?”

• face/content: test/results reflects what 1+ experts believe it should

• concurrent: test/result is as good or better than an existing test or result

• predictive: test/result predicts future results from another, existing test

• constructive: items or subtests should accurately assess something unique

• most commonly presented for experimental studies of an assessment instrument such as a standardized test

Homogeneity

• usually what we want in treatment research

• members are similar across various characteristics

• leads to similar outcomes

Heterogeneity

• useful when we want to show that a certain variable has no impact

• typically not what we want

• people with differing characteristics such as demographics, test results medical histories, etc. can lead to varying outcomes across participants

Narrow confidence bands in meta-analysis

• experimental research often contains heterogeneity and wide confidence bands

• meta-analyses combine results across experiments, generally resulting in a narrower confidence band (what we want, as it leads to higher confidence in results)