Research Methods Exam 2

________the process of selecting units from a population of interest.

Sampling

______allows to generalize findings from a relatively small group of individuals (sample) to a larger group of similar individuals (population)

Sampling

_______ each individual in the population has an equal chance of being selected for the sample. This type of design ensures better generalization of results and conclusions to the larger target population.

Simple random sampling

__________Here the target population is divided into a number of non- overlapping subpopulations (called strata) Once strata have been selected, the researchers select random samples from each of the subpopulations.

Stratified Sampling

______ every Nth member

systematic sampling

__________ participants are selected from the pool available due to either closer geographic proximity or for other reasons. Limitation included the restriction in researchers ability to generalize results to a larger population.

Convenience Sampling (AKA accidental sampling)n

_____ refers to total number of participants in a study. It contributes to the power of a research design.

Sample size

______ refers to the ability to detect significant treatment effects when present.

Power of a study

Within-Subjects experimental designs

One group of participants (each participant exposed to all levels of IV)

Less worry about group equivalence (assumed participants are inherently equivalent to themselves)

May have concern about unrelated change over time. (sequencing effect: what order participants experience levels of IV)

Between-Subjects Descriptive Design

Use naturally forming groups (groups defines so that participants can ONLY fall into one group) (No ambiguous membership)

Mostly comparative and cross-sectional research (should use subject matching, but may only match overall averages of the groups)

Sometimes surveys (using surveys to compare two groups)

Between - Subjects experimental design

2 or more groups of participants (each group exposed to one level of the IV)

Comparisons made BETWEEN the performances of each group (who did better or worse? did they perform all the same?)

What are the types of quantitative designs?

Cross sectional

Longitudinal

Correlational

Single subject design

Quasi experimental

_______ development over time of one group of participants

Longitudinal research

_________ looking at how multiple variables compare in a single group.

Correlational research

A _____ is a probability sampling method in which you divide a population into clusters, such as districts or schools, and then randomly select some of these clusters as your sample

Cluster

The primary objectives of a _______ experimental design is to determine whether a treatment is effective and how much treatment is needed to reveal the desired effectiveness.

Single Subject Design

_________ focus on a small sample of participants and the changes observed within each participant rather than differences among groups of participants.

_____ the clinician documents changes in an individual participants performance by measuring behaviors repeatedly and periodically over time.

Single subject design

A ________ is a type of research design that attempts to establish a cause-and-effect relationship. The main difference with a true experiment is that the groups are not randomly assigned.

Quasi-experimental design

_______ studies collect data from many subjects at a single point in time.

Cross-sectional

_______ studies collect data from many subjects at a single point in time.

_____ collect data repeatedly from the same subjects over time, often focusing on a smaller group of individuals that are connected by a common trait.

Cross-sectional

Longitudinal

______ is a common approach in various social science fields, not just anthropology. It is used not only to study distant or unfamiliar cultures, but also to study specific communities within the researcher's own society.

Ethnography

_______ is a qualitative method that investigates the lived experiences of individuals to understand a particular phenomenon from their perspective, aiming to describe the essence of these experiences.

Phenomenology

_______ seeks to understand the outside world as it is interpreted by and through human consciousness.

____ is concerned with the study of phenomena that arise from the experience of being in the world.

Phenomenology

______ a qualitative research method that enables you to derive new theories based on iterative collection and analysis of real world data.

Grounded theory

What are the steps of qualitative designs?

1. Observe events or ask questions with open ended answers.

2. Record what is observed, said or done.

3. Interpret the data

4. Return to further observe or ask more questions.

5. Repeat steps (II-IV)

6. Develop formal theories

7. Form conclusions and generate hypotheses

Qualitative Design flexibility:

Not as rigidly designed

Generate hypotheses and do not test them.

Qualitative Design: Criterion-based selection:

More important to pre plan what participants you will include, rather than how you will compare their data.

NOT random selection or assignment

Qualitative Design: Naturalistic Inquiry:

Participatory and non participatory

_____ researchers help shape the flow of the study (interviews, etc)

participatory

______researchers are completely hands-off (observation,etc)

Nonparticipatory

__________ qualitative data analyzed first, then quantitative data gathered.

Sequential Exploratory

_______ Quantitative data analyzed first, then qualitative data gathered.

Sequential explanatory

_________ Both types of data collected at same time, no prioritization.

Concurrent triangulation

__________ Both types of data collected at the same time, one only included to support the other.

Concurrent nested

________ not control, often ask participants to "check" interpretations.

Credibility

_______ refers to the extent to which findings and interpretations are considered believable and trustworthy, based on evidence and rigorous methods, ensuring that the research is perceived as accurate and reliable.

Credibility

HIgher ______ adds statistical power. You are also more likely to find results.

reliability

HIgher ______means lower effect of random error.

reliability

______ is a property of the instrument.

reliability

________ the consistency and stability of a measurement or method, meaning whether the same results are obtained when repeated under similar conditions. If a study is _____ the same findings will be produced when the same methodology is applied again.

Reliability

- Are your results and conclusions "well grounded or justifiable; being at once relevant and meaningful" Does this score measure what it claims to measure?

Validity

______is a property of the scores/interpretations in context, , not a static quality of the instrument itself.

Validity

True or False:

Something can be reliable without being valid.

Something cannot be valid without being reliable.

True

True or False:

The results of any test are valid ONLY for the construct it is meant to assess

True

________the degree to which a study's results accurately measure what it intends to measure.

Validity

________How well the results represent the truth of the population being studied.

Internal validity

______how well the results apply to similar individuals outside the study.

External validity

________ are the extraneous variables controlled (quantitative) or appropriately described (qualitative)

The fewer alternate explanations that exist, the better the ____ validity.

Required for strong study design.

Internal validity

_________ can the results of the study be generalized to the entire population. (quantitative) or transferred to other contexts /settings (qualitative)

Limited by the range of participant demographics included in the study

Required for study interpretation, but second to internal validity.

Must be proven for each new context/population.

External validity

Threats to Internal Validity

History: unexpected events during the course of the study

Maturation: natural developmental changes during the course of the study.

Attrition: Participants drop out of study and may have different characteristics from people who stay in.

Reactive Pretest: pretest functions as a practice test for post test, improving scores through familiarity.

Instrumentation: equipment not calibrated, assessments not update for current standards questions are ambiguous.

Statistical regression: participants with incredibly high or low scores are not likely to be AS extreme on every test.

Differential Subject Selection: groups are not equivalent on key factors and are not randomly assigned.

Threats to External Validity

Subject Selection: Cannot generalize to populations not well-represented in the study.

Interactive Pretest: The experience of taking pretest interacts with the IV, so individuals who were not primed with the pretest may not do so well.

Reactive Arrangement: AKA Hawthorne effect: people will perform better just because they know they are in a research environment.

Multiple-Treatment Interference: cannot generalize just one treatment if participants received multiple at once.

Individual to participant

Matter in their own live and reflect client's perception.

Personal Significance

Focused on a particular intervention

Based on research evidence and apply to general population.

Clinical Significance

_______ studies have stronger claims and better external validity. An area that has historically been neglected in publication.

Replication

______ refers to the process of repeating a previous study using similar methods and procedures to verify if the original findings can be reproduces with a new set of data.

Replication

______ using same population , setting, variables.

As few changes as possible.

Good for reliability

Direct replication

_______ changing out population, setting, variables one at a time.

-Usually done in a series (systematic)

-extends external validity beyond the original study.

Systematic Replication

What are the common types of reliability?

Inter-observer: how similarly do two people measure the same thing?

Intra-observer: how similarly does one person measure something at multiple different points?

Test/retest or alternative forms: common for pre-post-test designs

Spilt-half: researcher divides questions into two equal halves and sees if scores on the halves are correlated.

Cronbach's alpha: theoretically equivalent to the average of all possible split-half combinations.

Treatment Study Steps:

Phase 1: Does an affect exist? Are there negative effects? Case studies. small-n studies

Phase 2: What populations do better with this intervention? SSD's, small-n studies

Phase 3: Efficacy research, clinicial trials. Large-n experimental studies

Phase 4: Efficacy/practicality in clinic, field research. Large -n experimental studies

Phase 5: Efficiency studies and opinions of stakeholders. Qualitative, mixed methods

____positive changed expected from routine practice

Effectiveness

_______ length of time and other resources required to get a benefit.

Efficiency

_______reproducible results with internal validity and statistical significance.

Efficacy

_______ procedures are followed as closely as possible.

Fidelity

What is the content of the methods section?

Participants

Materials

Procedures

Factors used when describing participants:

Number recruited/retained

Ages

Race/ethnicity

SES

Language status

Health status (to the extent relevant): comorbidities, hearing/vision, neurological status

Sex/gender

________ anything that was used as stimuli, part of the treatment, or to collect data. Ex: recordings, AAC/hearing devices, assessments, etc.

Materials

Rationale for selection of ______ may be in Intro or Methods

Readers need to evaluate if IV's and DV's were appropriately selected and adequately measured. (validity and reliability)

Materials

-Manipulations of IV (usually tasks performed by the participants)

-At what point DV is measured (if not provided earlier, also "by whom"

-step-by-step like a recipe

-A variety of factors can impact protocol decisions

Research Protocol

-The initial plan for how to organize and interpret data usually goes at the end of Methods

-Includes the strategy chosen for analysis, as well as any software or other tools used.

Data analysis

Categorical Measurement includes:

Nominal and Ordinal

Continuous Measurement includes:

Interval and Ratio

Nominal Data

Data is sorted into 2+ mutually exclusive categories.

NO ranking between categories (Like a jersey number)

Analysis: # of people in each category (mode), number of categories (similar to range)

Cannot perform math on _____ scales

Ordinal Data

Data is sorted into 2 + mutually exclusive categories

WITH ranking between categories

NO equivalent distances between categories. (1st place is inherently better at a task than 2nd place, but magnitude is uncertain and is not necessarily the same magnitude difference with 3rd place)

Analysis: # if people in each category (mode), number of categories (range, "center" person if there are multiple categories (median)

Cannot peform math on _____ scales

Interval Data

Data is sorted into 2 + mutually exclusive categories.

WITH ranking between categories

WITH equivalent distances between categories.

NO true zero: O degrees celsius does not mean "no heat", and 200 IQ is not twice as intelligent as a 100 IQ because it is standardized bell curve.

EX: 27.07 All-time PER vs 24.58 all-time PER

Ratio data

Data is sorted into 2 + mutually exclusive categories.

WITH ranking between categories

WITH equivalent distances between categories.

WITH true zero: ) Hz means there are no cycles, someone who coughs 6 times has coughed twice as much as someone who coughs 3 times.

Ex: $480 M all time earnings vs $17.5 all time earnings

Regardless of level, measurements must be ______ and _______.

Reliable and Valid

______precision, getting the same results every time.

Ex: stability overt time, internally consistent

Generally, want ratings above 0.7, higher for high-stakes

Reliability