Research in Communication Disorders - Scientific Inquiry Process Part 2

Variables

The focus of interest of the researcher

Independent Variables

The variable that •has an effect (produces a change) on the dependent variable. The variable that is manipulated or observed (If X caused Y, then X is the IV)

• Intervention

• Strategy

• Method

• Program

What are some examples of an independent variable?

Co-teaching

This is a way to do speech therapy and is considered an independent variable as it is a thing that is being introduced as a mechanism to trying change the generalizability of skills

Introduction of the independent variable

What does the researcher control in regards to the types of variables?

Dependent Variable

• The variable that is affected by the independent variable (If X caused Y, then Y is the DV).

• The focus of the observations.

Constants

Does not change over time (aka constant variables)

Extraneous Variables

Unwanted and often a nuisance to the researcher as they have a potential impact to the dependent variable

As kids age, they become better at certain speech sounds and naturally develop phonetic repertoires that are closer to English

What is one example as to why is age considered an extraneous variable?

• Age

• Gender

• Length of therapy session

What are some examples of constant variables?

Observation

Some sort of measure of the behavior (e.g., test score, rating, tally, duration, % occurrence - # of correct / total # of opportunities)

Dependent Variable

Is behavior considered an independent or dependent variable?

Experimental / Treatment Group

Group that receives the treatment

Control Group

The group that receives no treatment or a treatment that is different than the experimental group

Sample

A subset of the population

Population

A group of potential participants to whom researchers want to generalize the results of the study derived from a sample drawn from the population

Notion of groupings

What is central to research as one group receives the treatment and one doesn’t?

To help us understand the impact of certain extraneous variables and the amount of natural aging that occurs

Why do we use a control group?

The population

What do we want the sample to be representative of?

• Socioeconomic status / background

• Age

• Gender

• Severity of disability

In the sample, there should be balances in these areas…

Extraneous Variables

By not having a representative sample, what is introduced into our study?

• What the population is

• Who you know you want to try this intervention or strategy

What is the job of the researcher to understand?

Baseline

The level at which the participant performs a behavior without the intervention (how they act / behavior = dependent variable without the intervention = independent variable)

Pre-testing

The baseline that is scored prior to the introduction of the independent variable

Post-testing

What is done (e.g., observation, scoring, testing) that are done and is measured at the end of the study

Random Selection

Every individual has an equiprobability of being included in the sample. Ensures initial equivalence of groups

Random Assignment

Every individual event that comprises the sample has an equiprobability of being included in each group/condition (e.g., treatment, control). Ensures absence of systematic bias in the makeup of groups

Sampling

What is random selection tied to?

Being included in the sample

With random selection, everyone has an equal chance or probability of what?

Confidence

Through random selection, what emerges?

• The sample will represent the population

• What will happen in this study will likely happen in the population

• No biases emerge as the researcher objectively went went through to pick the population

Why does confidence emerge with random selection?

Experimental Research Designs

Looks at controlling all of those extraneous variables so that they don’t infiltrate the study and don’t affect the dependent variable

Objective manner

In what type of manner is data collected din experimental (categorical term) research designs?

Operationally defined

Data is collected and is defined in this manner. In other words, it is defined in an observable manner and numbers are attached to observations

Experimental (categorical term)

The impact of all other variables are controlled in order to isolate the causal variable (achieved best in a laboratory)

Separate

In experimental research, does the research attempt to stay with or away from the participants?

Inferential Statistics

• This is designed to determine whether or not, if the study was done again with a different sample, that these results will be similar in the second study as they were in the original study

• It will tell us if the differences occurred by chance or if those numbers ar real as well as the likelihood that it will happen in the population

True Experimental Designs

• Participants should be randomly selected from the population (this seems to have relaxed)

• Should include a control group in the research design (this seems to have relaxed)

• Participants MUST be randomly assigned to at least one experimental and one control group to order and ensure equivalency of groups to avoid bias

• Participants should be randomly selected from the population (this seems to have relaxed)

• Should include a control group in the research design (this seems to have relaxed)

• Participants MUST be randomly assigned to at least one experimental and one control group to order and ensure equivalency of groups to avoid bias

What are 3 things about true experimental designs?

Experimental Research Studies

Occurs when you look at the introduction of an independent variable and measure its impact on the dependent variable

Control Groups

What kind of group helps us weed out extraneous variables?

Pretest-Posttest Control Group

A true experimental design where you have data collection or observations before the study starts and at the end and compare where people are at the beginning and end of the study

Randomized Controlled Trial Study

A pretest-posttest control group is also called what?

If the experimental and control groups are equivalent at the beginning of the study

What does the pretest tell the researcher?

Randomized Controlled Trial Study

People were randomly assigned to the experimental or control group and there is some sort of comparison between those two

Posttest-Only Control-Group

No pretest administered to determine baseline and equivalency. Equivalency is assumed since random selection is used

Equivalency

A pre-test is done to determine WHAT between the groups?

Control Group

Changes in the posttest score after treatment would be reflected in which group?

When a pretest is not practical or desirable

When is a posttest-only control-group used?

Random Selection

A posttest-only control-group is only used when?

• That we randomly selected everyone from the population

• They represent the population

• We have an even balance of individuals

Random selection in a posttest-only control group ensures what?

Pre-test sensitivity

An extraneous variable where some people are sensitive to a pre-test and it might influence it

Practice Effect

A likelihood that the participants will remember the answers or questions to the test

• Control / post-test only

• Experimental / post-test only

• Control pre/posttest

• Experimental pre/posttest

What are the 4 groups that participants can be assigned to in the Solomon four-group design?

The greatest amount of experimental control to combat extraneous variables

What does the Solomon four-group design provides us with?

Pre-test sensitivity

The Solomon four-group design allows us to feel confident about what?

To determine if there is any pre-test sensitivity and still have the ability to look at initial group equivalency because the individuals are randomly assigned to one of the four groups

How is data collection set up in the Solomon four-group study?

Quasi-Experimental

•Don’t always have random selection from population

•Don’t always have random assignment to experimental and control groups

•Allows researchers to move out of the laboratory and into the natural environment

•Usually see a control group

Causality

What does quasi-experimental research designs look at?

Variables

In quasi-experimental groups, what is sometimes investigated after the fact?

Non-equivalent group designs

• Uses matching to try to approximate equivalency (if you have an experimental group of 20 and a control group of 3 due to small sample size)

• Occurs because groups can’t be randomly assigned (see the way 2 classrooms respond to 2 different interventions)

• Treatment groups and control group are not equivalent (disordered vs. typically developing)

Nonclinical Groups

In quasi-experimental research designs, what group serves as the control group?

• Highly systematic process of data collection

• Must avoid personal biases

What are 2 reasons as to why we want to ensure the reliability of data?

• Blinding

• Interobserver Reliability

What are 2 ways to ensure the reliability of data?

Blinding

• Collecting data and performing other tasks (e.g., proving the treatment) without knowledge of the research questions, hypothesis, experimental group, etc.

• Use independent individuals to collect data and provide treatment

Single-Blinding

When the people who are collecting data don’t know what the research question or hypothesis is

Double-Blinding

When the people collecting data don’t know the research question / hypothesis as well as they don’t know who is in the experimental versus control group

Interobserver Reliability

Measuring how consistent two or more individuals are at observing or measuring the same event

90%

What percentage do we want the agreement between two observers to be?

Statistics

What can determine the % of agreement between the 2 people taking data / observing?

Internal Validity

•How confident we are that there was a relationship between the IV and the DV

•The change in the DV was associated or correlated with the implementation of the IV

Causal Research

Things we do to make a higher internal validity to say that the change in the DV was associated or due to the IV

Internal Validity

What do extraneous variables weaken?

Extraneous Variables

The presence of ____ ____ weakens internal validity

Weak internal validity

THIS means we cannot say with confidence that the IV produced an effect on the DV

Research Studies

Problems with internal validity weaken THIS

• Influence of maturation

• Spontaneous / natural recovery after a stroke

• Influence of natural development

A control group can help eliminate these things.

• Differential Selection Effects

• History Effects

• Maturation Effects

• Statistical Regression Effects

• Attrition / Mortality Effects

• Testing Effects

• Instrumentation Effects

What are the main treats to internal validity?

Differential Selection Effects

1.Situations where there are different ways that the participants were assigned to the experiment in the control group

2.Unequal of how they were assigned (e.g., age, gender, intellect, etc.)

History Effects

1.Outside extraneous variable that may influence the DV (if they are receiving therapy outside of study, extra homework at home)

2.More prevalent when studies are longer

Maturation Effects

1. Change in the child that occurs during the course of the study.

2. Change is not external, like additional speech therapy, change is that child is acquiring more language

Statistical Regression Effects

1.When you have groups of individuals in the sample who test extremely high or low on pre-test but post-test scores closer to the mean or average

2.Change is the natural regression to the mean, not due to the IV

Attrition / Mortality Effects

Loss of participants sometime in the study for whatever reason and don’t know if groups are equivalent or equal

Testing Effects

1.Something about the process of taking a test that influences the results.

2.Could affect disposition during the intervention

3.Learn some of the answers on the pre test and remember it on the post test

4.Way someone tests can affect how they respond to the IV

Instrumentation Effects

Any kind of variations in the instrumentation that are used to measure the DV

Random Assignment

How do you combat differential selection effects?

• Don’t make your studies super long

• Ask parents to not do anything additional or extra

How do you combat history effect?

• Control group to see how much was maturation and how much was IV

• Don’t make study super long as kids get bored

• Motor fatigue can influence DV

• Think about when you are doing the intervention

How do you combat the maturation effect?

Determine if there is any sort of regression to the mean and can use a control group

How to control statistical regression

• Shorten length of study by reducing participants who drop out

• Provide transportation

• If you are using tech, don’t include participants who aren’t tech savvy and will drop out

How do you control attrition / mortality effect?

• Counterbalancing where half of participants take one form of a test while the other half takes another form of the test

• Post-tests aren’t the same, but different forms

• Time in between tests and forget anxiety or answers

• Don’t use pre-test

Ways to combat control testing effect

• Calibrate equipment and is functioning properly

• Make sure people are trained

Ways to combat instrumentation effects

Statistical Regression to the Mean

• A behavior that goes from the extreme high or low point to an average level

• Many clients wait until their problem is at its worst before they seek treatment (vocal abuse)

External Validity

What is the generalizability of the results of the study? In other words, it is the confidence that we have that if someone else did our study, they found similar or the same results

1.Generalization Across Participants (Accessible and Target Populations)

2.Interaction of Personological Variables and Treatment Effects

3.Verification of the Independent Variable/Describing the Independent Variable Explicitly

4.Multiple-Treatment Interference Effects

5.Novelty and Disruption Effects

6.Experimenter Effects

7.Pretest Sensitization Effect

8.Posttest Sensitization Effect

9.Hawthorne Effect

10.Interaction of History and Treatment Effects

11.Measurement of the Dependent Variable

12.Interaction of Time of Measurement and Treatment Effects

Threats to External Validity

Threats to External Validity

1.Generalization Across Participants (Accessible and Target Populations)

2.Interaction of Personological Variables and Treatment Effects

3.Verification of the Independent Variable/Describing the Independent Variable Explicitly

4.Multiple-Treatment Interference Effects

5.Novelty and Disruption Effects

6.Experimenter Effects

7.Pretest Sensitization Effect

8.Posttest Sensitization Effect

9.Hawthorne Effect

10.Interaction of History and Treatment Effects

11.Measurement of the Dependent Variable

12.Interaction of Time of Measurement and Treatment Effects

Posttest Sensitization Effect

The administration of a posttest influences the treatment’s impact or the participant’s subsequent behavior, making the treatment effects different than they would be for someone who only receives the treatment without the posttest

Pretest Sensitization Effect

Occurs when completing a pretest that alters a participant’s awareness, motivation, or behavior, thereby influencing their response to a subsequent intervention or posttest

Novelty and Disruption Effects

A temporary, positive influence where participants respond better to a new treatment or situation or treatment where disruption is a temporary, negative influence where participants respond worse due to change

Multiple-Treatment Interference Effects

Occurs when the presence and simultaneous administration of two or more treatments make it impossible to determine if the individual effect of any single treatment, as the treatments influence each other