RMDA Exam 2

Population

 the larger group the researcher is interested in

Sample

the small set of individuals who participate in the study 

Target Population

the group defined by the researchers specific interest

Accessible Population

a portion of the target population consisting of individuals who are accessible for recruitment as participants 

How accurately can we generalize the results from a given sample to the population depends on

The representativeness of the sample.

The goal of a research study is to examine a sample and then…

….generalize the results to the population 

Representativeness

refers to how closely the sample mirrors or resembles the population 

Biased Sample

one that has characteristics noticeably different than those of the population

selection bias/sampling bias

the sampling procedure favors the selection of some individuals over others

The likelihood of the sample being representative depends on

the procedure that is used to select participants

How big does a sample need to be?

Depends on the study!

A large sample

will be more representative than a smaller sample

Law of Large Numbers

says sample size for each condition should be between 25-30

Research ethics should take into account:

justice & autonomy

Sampling Methods

probability sampling or nonprobability sampling

Probability Sampling

the entire population is known, each individual in the population has a specifiable probability of selection, and sampling occurs by a random process based on the probabilities

Probability sampling requires

extensive knowledge of the population

 random process

every possible outcome is equally likely. 

Simple Random Sampling 

Each individual in the population has an equal chance of selection 

• 1. Clearly define the population 

• 2. List all members of the population

• 3. Use a random process to select members of pop

Sampling w Replacement 

Requires that an individual selected for the sample be recorded as a sample member and then returned to the population (replaced) before the selection is made

Sampling without replacement

Removes each selected individual from the population before the next selection is made. This method guarantees that no individual appears more than once

Stratified Random Sampling 

Populations are comprised of various subgroups. A representative sample will adequately represent each subgroup 

Stratified Random Sampling does not produce a true random sample because all individuals in the population are…

…not equally likely to be selected. Some subgroups may be overrepresented 

Proportionate Stratified Random Sampling

when researchers deliberately ensure that the composition of the sample matches the composition of the population

Cluster Sampling 

A researcher can randomly select groups instead of selecting individuals 

advantages of cluster sampling

Quick & easy to get a large sample, collect a lot of data at once

disadvantages of cluster sampling

Individuals within a cluster often have common characteristics or share common experiences that might influence the variables being measured.

 convenience sampling

researchers simply use as participants those individuals who are easy to get. People are selected on the basis of their availability and willingness to respond 

-used more often

advantages of convenience sampling

It is easier, less expensive, and more timely technique than the probability sampling techniques. 

Quota Sampling

can ensure that subgroups are equally represented in a convenience sample 

A researcher can adjust the quotas

to ensure that the sample proportions match a predetermined set of population proportions

Probability Samples:

 Simple random, Systematic, Stratified Random, Proportionate Stratified, & Cluster

Research Strategy

a general approach to research determined by the kind of question that the research study hopes to answer 

The Descriptive Strategy

• examines individual variables for a specific population 

  • Different types of relationships between variables (wake up time v. grades)

Correlational Research Strategy

• Involves measuring each participant on the two variables under consideration and seeing how scores on one measure relate to scores on the other 

  • Only attempts to describe the relationship (if it exists) it is NOT able to explain the relationship or say one causes the other 

  • The closer to 1, the stronger the correlation!

Experimental Research Strategy

• enables the answering of cause-and-effect questions regarding how two variables relate

• Conducted with rigorous control to ensure you have truly identified a cause-and-effect relationship

Quasi-Experimental Research Strategy

• Similar to experimental research strategy, except it lacks the rigorous control and you can not determine cause and effect 

• Almost, but not quite, experiments may have pre-existing groups or not be able to randomly assign participants into groups

Non-Experimental Research Strategy

• explores relationships, but has less control than experimental or quasi-experimental designs and can not determine cause-and-effect 

  • The independent variable can not be manipulated

    • If you want to look at gender/age differences

Correlational & Non-experimental methods achieve the

SAME GOALS using DIFFERENT TECHNIQUES

Research Designs

• how to conduct the study (correlational/experimental)

Research Procedures

 the details that precisely decide how the study is to be conducted

External Validity

 the extent to which the results can be generalized to the public (do the results match how the problem presents itself in the real world?)

• Need to generalize:

  • From a sample to the larger population

  • From one research study to another

  • From the research study to the real-world context

threat to external validity 

Any factor that limits the ability to generalize the results from a research study

Internal Validity

• concerned with factors in the research study that raise doubts or questions about the interpretation of the results 

a threat to internal validity

Any factor that allows an alternative explanation for the results

Threats to external validity: generalizing across participants

• Selection Bias- if a sample does not accurately represent the population, then there are serious concerns that the results obtained from the sample will not generalize to the population.

• College Students

• Volunteers vs. Non-Volunteers

• Homogeneity of Sample

• Cross-Species

Threats to external validity: generalizing across study features

• Novelty Effect- in this novel situation of a research study, individuals may perceive and respond differently than they would in the real world

• Multiple Treatment Interference- fatigue & practice effects, the results obtained from individuals who have participated in previous conditions may not generalize to individuals who do not have the same previous experience

• Experimenter Characteristics

threats to external validity: generalizing across features of the measures

• Sensitization- the process of measurement (being asked questions about a certain phenomenon) alters participants so they react differently to treatment. 

• Generality Across Response Measures

• Time of Measurement 

Threats to internal validity: extraneous & cofounding variables

• Extraneous Variables- additional variables that exist in a research study but are not directly investigated

• Confounding Variable- an extraneous variable that changes systematically along the two variables being studied. A confounding variable provides an alternative explanation for the observed relationship between the two variables, and therefore, is a threat to internal validity. 

• If a study includes a confounding variable then there is an alternative explanation and the internal validity is threatened 

• To achieve it, ensure no extraneous variable is allowed to become a confounding variable. 

Types of confounding variables

• Environmental Variables- whenever a difference in environment exists, there is an alternative explanation for the results and the internal validity of the study if threatened 

• Participant Variables- individuals in one treatment condition may be consistently smarter than the individuals in another treatment condition. 

• Time Related Variables- time between conditions/observations

Experimental Design: 4 Elements

• Manipulation 

  • The researcher manipulates one variable by changing its value to create a set of two or more treatment conditions 

• Measurement 

  • A second variable is measured for a group of participants to obtain a set of scores in each treatment conditions 

• Comparison

  • The scores in one treatment condition are compared with the scores in another treatment condition. Consistent differences between treatments provide evidence that the manipulation has caused changes in the scores

• Control

  • All other variables are controlled to be sure that they do not influence the two variables being examined

independent variable

What you manipulate

treatment condition

a situation or environment characterized by one specific value of the manipulated variable. An experiment contains two or more treatment conditions that differ according to the values of the manipulated variable

dependent variable

What you measure

We cannot establish a cause-and-effect relationship by

observing two variables.

Goals of an Experimental Study

• Demonstrate that the cause (manipulated IV) precedes the effect (measured DV)

Manipulation

• consists of identifying the specific values of the independent variable to be examined and then creating a set of treatment conditions corresponding to the set of identified values.

  • It is useful because it allows you to identify the cause of a relationship you have observed. (ice cream and temperature)

  • It also allows for the control of extraneous variables and ensure there is not a 3rd variable at play 

Control

 an experiment must identify and control any third variable that changes systematically along with the independent variable and has the potential to influence the dependent variable.

A confounding variable must

vary systematically with the independent variable. A variable that changes randomly, with no relation to the independent variable, is not a threat

An extraneous variable becomes a confounding variable only if it influences the

dependent variable

Holding a Variable Constant

• by standardizing the environment and procedures, most environmental variables can be held constant. This technique can also be used with participant variables.

  • For example, by selecting only 6yr old children to participate in an experiment, age is held constant

Drawbacks to holding a variable constant

this method may have negative consequences because it can limit the external validity of an experiment

Matching across levels of IV

 ensuring participants in each treatment condition/level of the IV are the same across variables that may be confounds (there can be variability in each group, but both groups vary in the same way)

The variety of different ways to construct a control condition for an experiment can be classified into two categories:

1. Non-treatment control conditions 

2. Placebo control conditions 

Manipulation Checks

an additional measure to assess how the participants perceived and interpreted the manipulation and/or to assess the direct effect of the manipulation. 

• There often is good reason to question the success of manipulations that are intended to affect participants

Subtle Manipulations 

In some situations, the variable being manipulated is not salient and may not be noticed by participants

Placebo Controls

As with a simulation, the effectiveness of a placebo depends on its credibility. It is essential that participants believe that the placebo is real. They must have no suspicion that they are being deceived

Simulations

The effectiveness of the simulation, however, depends on the participants perception & acceptance.

Researchers use two techniques to maximize external validity while conducting an experiment:

Simulations & Field Studies

Which of the following is the primary goal for randomly assigning participants to treatment conditions in an experiment?

Minimize the likelihood that a participant variable (such as age or gender) becomes a confounding variable 

Holding a variable constant is a technique for removing one threat to ___, but it can limit the ___ of an experiment

internal validity; external validity 

How do studies using the experimental research strategy differ from other types of research?

Only experiments can demonstrate a cause and effect relationship between variables

Which of the following characteristics are necessary for an extraneous variable to become a confounding variable?

It must change systematically when the independent variable is changed

An experiment includes a treatment condition, a no treatment control, and a placebo control. Which two conditions should be compared to determine the size of the effect that is actually caused by the experiment?

Placebo vs treatment

Between-subjects design

The different groups of scores are obtained from 2+ different groups of people

Within-subjects design

The different groups of scores are obtained from the same group of participants

• Participants are measured multiple times 

• our study

What is required in an experimental design?

1. Manipulation of one variable to create a set of two more treatment conditions 

2. Measurement of a second variable to obtain a set of scores within each treatment condition 

3. Comparison of the scores between treatments 

4. Control of all other variables to prevent them from becoming confounding variables

An experimental design can take 1 of 2 forms:

Within-subjects & Between subjects 

Advantages of Between-Subjects Designs

Since each participant is measured only once, the researcher can be confident that the resulting measurement is clean and not affected by other treatment factors (eg. no practice effects). 

Disadvantages of Between-Subjects Designs

Each score is obtained from a unique individual who has personal characteristics that are different from all the other participants.

Individual Differences of between and within subjects

personal characteristics that differ from one participant to another. Individual differences can become confounding variables. Individual differences can produce high variability in the scores, making it difficult to determine whether the treatment has any effect. 

Threats to validity in between subjects designs 

• Confounding from individual differences

• Confounding from environmental variables

Limiting confounding by individual differences

restricted random assignment, matching, and holding varaibles constant or restrict range

Restricted random assignment 

Group assignment process is limited to ensure predetermined characteristics (such as equal size) for the separate groups

Matching

Assigning individuals to groups so that a specific participant variable is balanced or matched across the groups. The intent is to create groups that are equivalent (or nearly) with respect to the variable matched

Hold variables constant or restrict range

Limits external validity

Individual Differences & Variability 

• We compare mean scores across conditions to see if there is an experimental effect 

• Variance is a statistical value that measures the size of the differences from one score to another 

• We must take into account the variance of the scores

• If the variance is large, it is harder to find a significant difference across groups.

Researchers typically try to increase the differences between…

treatments and to decrease the variance between treatments

Minimizing variance within treatments

• Standardize procedure and treatment setting 

• Limit individual differences (restrict the sample based on individual difference variables)

• Sample size (larger samples are better; not usually most efficient method)

Remember:

Any factor that allows for an alternative explanation for the research is a threat to internal validity 

Threats to internal validity in between-subjects designs 

• Confounding due to individual differences between groups 

• Confounding from environmental variables

• Differential attrition  

  • Differences in attrition rates from one group to another and can threaten the internal validity of a between-subjects design 

• Communication between groups

  • Diffusion (the true effects of the treatment can be masked by the shared information)

  • Compensatory equalization, compensatory rivalry, or resentful demoralization