PBSI 302 Exam 3 - Correlational Designs & Survey Research

What is the fundamental difference between true and quasi-experimental designs?

random assignment into groups

What are the types of quasi experimental designs?

- Nonequivalent control group designs

- Designs without control groups

What are types of nonequivalent control group designs?

- Delayed group designs

- Mixed factorial designs

What are the types of designs without control groups?

- Interrupted time-series designs

- Repeated treatment designs

Types of Correlational Designs

- Predictive

- Concurrent

- Postdictive

Correlational Designs

- measure two or more variables and assess relationship between them

- no manipulation

- low control

- no causal inferences

Survey Research

Measurement and assessment of opinions, attitudes, and so on, usually by means of questionnaires and sampling methods

Correlational designs __ ____ _____ correlation.

do not imply

Designs that have two or more variables & attempt to determine degree of relationship between them

1. No manipulation

2. Low control

3. No causal inferences

Correlational Designs

characterized by differences in the time frames if the collection of the IV and DV data

Predictive

IV data are collected before DV data with an appreciable time interval between the two

Concurrent

IV and DV data are collected at about the same time (i.e., concurrently) without any appreciable time interval between the two

Postdictive

DV has been occurring in the past before the IV are collected

What are some problems with postdictive designs?

-selective sampling

- archival data

- instances where the occurrence of the DV may/could influence one's standing on the IV

Archival data

No control over how data were collected - unknown quality

Archival Research

refers to research conducted using data that the researcher had no part in collecting

Archival data

- those that exist in public records, or archives

- the researchers simply examines or selects the data for analysis

What are limitations from Archival Research?

1. Most archival data collected for unscientific reasons by people who are not researchers and thus, may not be very useful, may be incomplete, and may be subject to bias (e.g., insurance records, police reports)

2. Because archival research is by nature carries out after the fat, ruling out alternative hypotheses for particular observed correlations may be difficult

-- reliance on post hoc explanations elevates susceptibility to alternative explanations

What is the central issue?

research validity

Quasi-experimental designs

research studies in which participants are selected for different conditions from pre-existing groups

- might also entail self-selection into groups

In true experimental designs,

participants are randomly assigned to experimental and control groups

In quasi-experiment designs,

design DOES NOT permit the researcher to control the assignment of participants to conditions or groups

_______________ _________________ to groups is the basic difference between true and quasi-experimental designs.

Random assignment

What are quasi-experimental designs characterized by?

Lower levels of control

- WHO

-WHAT

- WHEN

- WHERE

- HOW

of the study

What reduces the internal validity of a quasi-experimental design?

presence of uncontrolled or confounded variables

What must be evaluated in a quasi-experimental design?

the likelihood that confounding variables are responsible for the study outcome must be evaluated

Nonequivalent Control Group Design

research designs having both experimental and control groups but the participants are NOT randomly assigned to these groups

What is the most common type of quasi-experimental design?

Nonequivalent Control Group Design

What problems occur with Nonequivalent Control Group Design?

How to compare results between groups when they are not equivalent to begin with

What does interpretability depend on?

Depends on the pattern of results obtained can be accounted for by possible differences between the groups or by something else in the study

How can you improve interpretability?

1. Matching

2. Identifying and building extraneous variables into the design or study as moderator variable

3. Pretesting

4. if nonequivalent, as per pretest data, then can use a number of statistical control procedures to covary out or control for preexisting differences

Pretesting

empirically documenting the degree of nonequivalence

Delayed Controlled Group Designs

nonequivalent control group design in which the testing of one group is deferred

i.e., the two groups are tested sequentially with appreciable time intervals between them

Mixed Factorial Designs

have one between-subjects variable and one-within subjects variable

Between subjects variable =

preexisting

Example of Mixed Factorial Design

Study go trait [between] and state anxiety [within] and impact on test performance

Interupted Time Series Designs

these designs allow the same groups to be compared over time by considering the trend of the data before and after the treatment

Repeated Treatment Designs

this research design allows the same groups to be compared by measuring participants' responses before and after repeated treatments

Multiple Time Series Design

a variation of interrupted time-series design, which is really NOT a design without a control group. A control and experimental group are included to rule out HISTORY as a rival hypothesis

How do you measure opinions and attitudes?

means of questionnaires and sampling methods

Steps for Self Report Measures / Questionnaires

1. First Determine the purpose of the questionnaire

2. Determine the types of questions

3. Item Writing

First Determine The Purpose of the Questionnaire

- Ask the target participants for useful information

- Anticipate questions of interpretation that may arise

Determine the types of questions

a. Open-ended/constructed response

b. closed-ended

Open-ended / Constructed-response

permits the respondents to answer in their own words

Closed-ended

limits the respondents to alternatives determined in advance by the designers

Item Writing

potentially, the questions and items themselves can have a big and major influence on how people will respond

a. Determine the format of the item

b. Address a single issue per item

c. Loaded items generate or produce specified responses

d. Topic or issue may be "sensitive" which can also have a major influence on how people respond, so avoid bias. Under these conditions, effects of loading are even more pronounced

e. Effect of question order

f. make the alternatives clear

e. questions and specified responses to them are not independent - adjacent question effect

g. there are a variety of ways in which participants' own characteristics may inadvertently alter the research outcome

Determine the format of the item

- construct-response (fill in or write in)

- true / false

- multiple - choice

- likert scales

Response Styles

tendencies to respond to questionnaire items in specific ways regardless of content. Biases that are consistent across time and questionnaires.

Willingness to answers

some people will not answer items or questions they are unsure about (will leave them blank). Others will go right ahead and guess.

- can usually control for this with strong instruction to answer ALL questions

Position preference

when in doubt pick (C)

- for knowledge, ability, and other measures with a correct of known "true" score, can control for this by randomization of alternatives

Acquiescence or yea- and nay-saying

tendency to consistently agree or disagree with questionnaire statements or questions regardless of content

- controlled for by using method of matched pairs (repeat item and reverse); also controlled by using bi-directional responses

Response Sets

tendencies to respond to a questionnaire or test content with a particular goal in mind

Social desirability

tendency to present self in a socially desirable manner; tendency to choose specified responses even if they do not represent ones true tendency or opinion

Self deception

occurs when an individual unconsciously views him/herself in an inaccurately favorable light; lack of self awareness

Impression Management

refers to a situation in which an individual consciously presents him/herself falsely to create a favorable impression

Social desirability responding

tending to over-report socially desirable personal characteristics and to under-report socially undesirable characteristics

- also a tendency to present self in test taking situations in a way that makes self look positive with regard to culturally derived norms and standard

Common method variance

& collection of data from single source

- potential to inflate and confound observed relationships especially where there is a theoretically justifiable reason to expect this

Major Survey Techniques

1. Face to Face Interviews

2. Telephone Interviews

3. Mail

4. Magazine

5. Internet based surveys & sample recruiting sources

What is the typical response rate for survey research?

30%

- it is higher for target sampling of paid internet-based sampling

The quality of the data is a _________ function of the response rate.

Direct

What is the key to the meaningfulness of any survey?

sourness of the sampling procedure used to generate respondents

Types of Sampling Procedures

1. Uncontrolled

2. Haphazard sampling

3. Probabilistic sampling

Uncontrolled

researcher has no control in the selection of respondents

Haphazard sampling

sampling procedure where the researcher may have some control over selection into study but it is still basically a hit-and miss method for selecting participants

Probabilistic sampling

sampling procedures in which the researcher makes an effort to assure that each person in the population has an equal chance of being represented

Types of Probabilistic Sampling

a. Sample random

b. Stratified random

c. Cluster

Simple Random

sample chosen from an entire population such that every member of the population has an equal and independent chance of being selected

Stratified Random

sample is chosen to proportionally represent certain segments in the larger population

Cluster

sample is selected by using clusters or grouping from the population

e.g., sampling every student in 10th class rather than every 10th student (simple random)

What factor is not characterized by correlational designs?

- Manipulation

A study gathers information on participants empathy and athletic ability in the same day. This is a _______

Concurrent Correlational design

Dr. Jay finds a -0.84 correlation between time slept and irritation. Can she conclude that sleep loss causes irritations?

No; A correlation is not enough to infer causation

Closed ended questions are preferable because they allow for the subjects to answer in their own words

False

"Are you in favor of marijuana legalization, free healthcare, and decriminalizing prostitution?" What's wrong with this?

Violates single item per question rule

A study that uses correlations to analyze data does NOT necessarily make it a correlational design

True

Which of the following is NOT an example of social desirability responding?

underestimation

Correlational designs often have _____ control in a(n) ________ setting

low; natural

Quasi-experiments are characterized by

self-selection into pre-existing groups

Ceiling and floor effects can harm the ______ of results

interpretability

Selective sampling, archival data (no control over data), and possible DV influence on IV are issues that can happen in

postdictive correlational designs

If the treatment is administered multiple times, with pre- and post- test taken before and after each instance, it is a

repeated treatment design

The main difference between true and quasi-experimental designs is

random assignment

Which of the following is NOT a way in which participant characteristics can alter research outcome in surveys?

Selection bias

What is the main difference between interrupted time-series designs and multiple time-series designs?

multiple time series designs have a control group

Is it possible to get a margin of error of 0?

yes; by including every single member of the population

Which of the following is accurate in terms study strength?

experiments > q-design > correlation > survey

Which of the following is NOT a form of probabilistic sampling?

uncontrolled sampling

Which of the following is an example of malingering?

responding so that someone on trial may make an insanity plea

Likert scale questions have _____ variability when compared to binary (yes/no, true/false) questions

more