Module 3 Powerpoint #1: Types of Research in Communication Sciences and Disorders

Types of Research in CMD

1. Basic Research
2. Applied Research
3. Field Research
4. Laboratory Research
5. Experimental Research
6. Non-Experimental Research
7. Quasi-Experimental Research

What is Basic Research?

1. Contributes new scientific knowledge with no intention of solving a social or clinical problem.

2. Useful for constructing new theories and modifying existing theories.

3. News, Very Theoretical; Not Clinical, Don't Solve Immediately

---> Theory - general explanation that attempt to explain the relationship between effects and causes.

What is Applied Research?

1. Contributes scientific knowledge and solves asocial problem, clinical problem, or other clinical application.

2. Attempts to solve problems that require immediate solutions.

3. Literature/ Knowledge Based, Solve problems for immediate need.

---> Clinical research (we use this a lot as SLPs)

What is Field Research?

Conducted in everyday settings such as homes, schools, or clinic.

What is Laboratory Research?

Conducted in more or less contrived settings outside the mainstream of daily lives.

What is Experimental Research?

Has Two Requirements:

1. Random assignment of participants to one or more conditions.
--->Best way to ensure equivalency and eliminate bias
---> WAYS: Close eyes, Draw Numbers, etc.

2. Selection and manipulation of conditions
---> One group does something
---> CONTROL GROUP: other group does nothing.

What is Non-Experimental Research?

1. Lack measures of comparison (e.g., group or prior measurement)

2. Typically observe behavior without inferring cause and effect.

3. Collect Data: only looking at one group, ex: survey after YOGA

What is Quasi-Experimental Research?

1. Almost experimental but not quite

2. Usually lacks random assignment.

3. If an experiment doesn't meet the requirements of experimental research, it is probably quasi-experimental.

Are subject randomly assigned to groups?

1. YES: True Experiment

2. NO: Ask, Is there a control group or multiple measurements?

3. YES: Quasi-Experiment

4. NO: Non-Experiment

Types of Variables in CMD Research

1. Variable
2. Independent Variables
3. Dependent Variables
4. Active Variables
5. Assigned Variables
6. Continuous Variables
7. Categorical Variables
8. Intervening (Extraneous) Variables

What is a Variable in CMD Research?

Focus of interest for behavior scientists; concepts that take on different quantitative or qualitative values (finite or infinite # of values)

What is an Independent Variable in CMD Research?

1. Presumed cause; antecedent; conditions that cause changes in behavior.

2. In experimental research designs, researchers control and manipulate the independent variable.

3. In quasi-experimental research designs, researchers cannot control or manipulate the independent variable, because it is fixed-->(such as age).

What is a Dependent Variable in CMD Research?

1. Presumed effect; consequence; the behavior that is changed; the focus of observations; the data.

2. If X, then Y.

3. To what extent does A cause B. If A happens, then B.

Ex: Praxis Scores--> GPA at Grad Level and Undergrad Level.
UGPA= Independent Variable
GGPA= Dependent Variable

What is Bivariate research?

Includes one independent variable and one dependent variable.

What is Multivariate Research?

Includes more than one independent variable and more than one dependent variable.

What is an Active Variable in CMD Research?

1. A manipulated IV; manipulated by the researcher.

---> Ex: Types of language tests, treatment procedures, noise, and other conditions that are readily CHANGED.

What is an Assigned Variable in CMD Research?

1. Variables that are measured but not manipulated--> Human characteristics

2. CAN'T be CHANGED

---> Ex: age, gender, intelligence, SES, occupation, hearing sensitivity, disabilities.

What is a Continuous Variable in CMD Research?

1. Take on a range of variables and possess the property of order.

---> Ex: Age: 18, 16, 10, 15, 19, 30,43, 38, 50, 74, 80, 88,62

What is a Categorical Variable in CMD Research?

1. People or objects are assigned to categories based on whether they possess some characteristic or not; no order.

Ex: Age:
Young (btwn 10 - 20)
Middle (btwn 30 - 50)
Old (btwn 60 - 90)

What is an Intervening (Extraneous) Variable in CMD Research?

1. Potential nuisance variables.

2. Any variable that affects the DV that is not the IV.

3. May not be observable.

4. Can confound (compromise) the study

Operational Definitions

1. Describe the activities necessary to measure and manipulate variables.

2. Are instructions for selecting subjects, measuring behaviors, and carrying our procedures.

3. Measures used for operational definitions must be valid and reliable for the population of interest.

Example of Operational Definitions

"Fifteen of the participants were 6 years old and from low socioeconomic status backgrounds (LSES).
---"These children had mothers who did not graduate from high school".--- Often, when researchers operationally define a concept or a variable, it is based on previous studies or the research literature. So, in the Pruitt and Oetting study, the researchers cite Dollaghan (1999) as their rationale for this operational definition.

What is an F2?

F2 target frequency was operationally defined as the first occurrence of a frequency minimum or maximum following F2 onset frequency.

What is a phoneme in regards to Operational Definitions?

A phoneme was considered 'later-acquired' if the reported age of mastery exceeded a child's chronological age, and it would have been the latest acquired of errored phonemes.

Yoga Example in regards to Operational Definitions

Imagine you are conducting a study on yoga and wellness on graduate students. How might you operationally define:

1. Experienced yoga participant vs. Inexperienced yoga participant?
---> Experienced Yoga= 1+ years of experience; consistently.
---> Inexperienced Yoga= Do not do yoga consistently.

2. Full yoga session
---> Session lasts 30 minutes to 1 hour and includes stretching, yoga positions, etc.

3. Graduate student
---> Someone currently enrolled in Master's Program

4. Full-time vs. part time graduate student:
---> Enrolled in 9+ hours for current semester

5. Wellness:
---> Number of meltdowns during 1st week of Grad School

Limits of Operational Definitions

1. Some concepts are difficult to define.

2. Operational definitions may differ across research studies.

Research Data

1. Consequences of observing or otherwise gathering information for study.

2. Observations are coded

3. Usually coded in numerical data.

4. When research data are collected and coded, the researcher has to be sure that the data that are collected and coded are reliable.

What is Measurement?

Assignment of numerals to objects according to specific rules

Research Data: Reliability Procedures

1. Blinding:
---> Single
---> Double

Research Data: Interobserver and/or Intraobserver Reliability

1. Percentage of agreement
---> Dividing the smaller number of behaviors observed by the larger number.

2. Point by point agreement.
---> Examines behaviors point by point.
---> Exhibit 1.2, p. 16.

--An agreement index of below 80% is usually unacceptable.

What is an Interobserver?

1. Across People
---> EX: Same survey given to two different people; same results.

What is Intraobserver?

1. Within same person
---> EX: Survey given to me on Monday, Survey given to Me on Wednesday; same results; high agreement for intra.

What is Internal Validity?

1. GOAL OF RESEARCH: The degree to which the relationship between the independent variable and the dependent variable is observed without the influence of extraneous variables.

2. The degree to which the independent variable leads to a change in the dependent variable.

3. When internal validity is weak, researchers cannot imply that the independent variable produced the effect observed in the dependent variable.

What does Internal Validity Ask?

1. Asks: Did the IV lead to change in DV?
---> In an experiment lacking internal validity, the researcher does not know whether the experimental treatment or uncontrolled factors produced the difference between groups.

What Weakens Internal Validity?

1. Presence of confounding variables weaken internal validity.

What happens when Internal Validity is Weak?

1. When internal validity is weak, researchers cannot imply that the IV produced the effect observed in the DV.

What is the Most Common Weaknesses in Research Studies?

1. Problems with internal validity are the most common weaknesses in research studies

Possible Threats to Internal Validity

1. ATP
2. Differential Selection Effects
3. History Effects
4. Maturational Effects
5. Statistical Regression Effects
6. Attrition Effects
7. Multiple Testing Effects
8. Instrumentation Effects
9. Additive and Interaction Effects

Possible Threats to Internal Validity: ATP

1. Ambiguous Temporal Precedence (ATP) Effects

2. To infer causality, the treatment must occur before a change in the DV is observed.
EX: Causality-------> Cause and Effect
3. ATP effects are present when the direction of the relationship is not clear.

Example of ATP

Imagine you're conducting a study to see whether practicing yoga reduces stress among graduate students.However, you survey the students' stress levels after they begin practicing yoga without checking their baseline stress levels before the intervention. As a result, it's unclear whether the yoga practice reduced their stress or if students with lower stress were more likely to engage in yoga. The cause-effect relationship remains ambiguous because it's uncertain which occurred first. This is an example of ATP, where it's unclear whether the cause truly precedes the effect.

Possible Threats to Internal Validity: Differential Selection Effects

1. Occurs when subjects are assigned to experimental and control groups in a way that results in unequal distribution of the subjects' unique characteristics.

2. How to control:
---> Random assignment
---> Matching

Example of Differential Selection Effects

1. Differential Selection Effects can occur when groups in a study are not equivalent at the start due to differences in how participants were chosen or assigned.

2. For a yoga study, imagine you want to compare two groups of graduate students: one that practices yoga and one that doesn't. If the yoga group consists mostly of students already interested in wellness and self-care, while the non-yoga group doesn't, any differences in stress levels may be due to pre-existing differences between the groups, rather than the yoga practice itself. This creates a threat to internal validity through differential selection.

Possible Threats to Internal Validity: History Effects

1. Research designs that require observations over long periods of time are vulnerable to the threat of history effect.

2. Longitudinal designs are especially vulnerable particularly if there is a long span of time between pre-tests and post-tests.

3. Includes outside effects (extraneous variables) that may influence the DV during the course of the study.

4. How to control:
---> Reducing time between pre-testing and post-testing
---> Statistical procedures

Example of History Effects

1. In a yoga-related study, the history treat to internal validity refers to events outside of the study that affect participants during the research period.

---> For example, imagine a study that measures the effect of daily yoga on student stress levels over several weeks. If a stressful university event, like midterms (the outside event), occurs during the study, it could increase everyone's stress level, marking it hard to determine if changes in stress were due to yoga or the external event. This external event would be a history threat.

Possible Threats to Internal Validity: Maturational Effects

1. Occurs when there are changes in physical abilities and mental processes such as children's development of speech and language or motor skills.

2. An issue when maturation is not the IV/focus of study, as in typical in developmental research.

3. Experimental task that are long or complicated may produce maturational effects such as boredom and fatigue.

4. How to control:
---> Reduce time for completing observations.
---> Add a control group
---> Provide planned rest times or comfortable surroundings.

Example of Maturational Effects

1. In a yoga related study, the maturation threat to internal validity refers to changes that naturally occur in participants over time, independent of the intervention.

---> For example, suppose you're conducting a study that examines the effect of weekly yoga sessions on participants' flexibility over six months. If participants naturally improve their flexibility over time due to physical growth or increased activity outside of yoga, it would be difficult to determine if their improvement was due to the yoga sessions or simply the natural process of maturation.

Possible Threats to Internal Validity: Statistical Regression Effects

1. Occurs when subjects who score very high or very low on a test regress toward the mean on the next administration of the test.

2. How to control:
---> Don't select subjects who have extreme test scores.

Example of Statistical Regression Effects

1. In a yoga-related study, statistical regression(also known as regression to the mean) could threaten internal validity when participants are selected based on extreme pre-test scores.

---> For example, suppose you conduct a study on the effects of yoga on stress reduction, and you select participants with very high initial stress levels. Over time, their stress levels might naturally move closer to the average, regardless of the yoga intervention. Therefore, the observed improvement could simply be due to statistical regression rather than the effects of the yoga program itself.

Possible Threats to Internal Validity: Attrition Effects

1. Occurs when there is a loss of participants. (death; loss)

2. Studies that require a long period of time for completion.

3. How to control:
---> Researcher can add additional participants to offset possible losses.

Example of Attrition Effects

In a yoga-related study, attrition (or dropout or mortality) can threaten internal validity if participants leave the study in a non-random manner, leading to biased results.

---> For instance, suppose you are studying the effect of a 12-week yoga program on physical fitness. If the participants who find the program too challenging or time-consuming drop out before the study is completed, while only those who are more physically fit remain, the final results might inaccurately show that yoga has a greater effect on fitness than it actually does. This could occur because the remaining participants were already more likely to improve, not because of the yoga

Possible Threats to Internal Validity: Multiple Testing Effects

1. Occurs when subjects are tested more than once during an experiment.
---> Test-sensitizing effect
---> Practice effects

2. How to control:
---> Counterbalancing tests
---> Plan an interval of time between tests

Example of Multiple Testing Effects

1. In a yoga study, multiple testing effects could lead to internal validity when participants are repeatedly tested on the same measures overtime, leading to improvements that are not due to the intervention itself but rather familiarity with the test.

---> For example, if participants in a study measuring stress reduction take the same stress assessment multiple times throughout the yoga program, they may become more accustomed to the test format. Their improved scores may reflect their familiarity with the test rather than a true reduction in stress caused by the yoga intervention.

Possible Threats to Internal Validity: Instrumentation Effects

1. Occurs when there are unwanted variations in instruments used to measure human behaviors.

2. Mechanical instruments may change over time due to temperature, humidity, mechanical wear.

3. Human observers may change over time.

4. How to control:
---> Calibrations
---> Training
---> Perceptual anchors

Example of Instrumentation Effects

1. In a hypothetical yoga study, instrumentation effects could threaten internal validity if changes in the measurement tools or procedures occur during the study.

---> For example, imagine a study measuring flexibility improvements through yoga. If different instructors measure participants' flexibility at different points in time, using slightly different techniques or tools(e.g., different types of flexibility tests or methods of recording results), any observed changes may reflect differences in measurement rather than true changes in participants' flexibility caused by the yoga program itself.

Possible Threats to Internal Validity: Additive and Interaction Effects

One or more of the many potential threats to internal validity may interact with one another