Ch. 7: Correlational Research and Survey Design (PSYC 305)

correlational study

a research design that examines whether, and how strongly, two variables are related to one another, without manipulating either varriable

correlation (or correlation coefficient)

the statistic (Pearson’s r) used to summarize the strength anddirection of a relationship

specific prediction

Qualitative research suits questions without a ________.

survey

a quantitative research srategy for systematically collecting information from a group of individuals; the information is then generalized to a larger group of interest

open-ended questions

questions that let participants answer in their own words

closed-ended questions

questions that give a predetermined set of response options

• openly invites vague answers that can be hard to compare across hundreds of students

What are the disadvantages of open-ended questions?

• if you just use one:

  • one item can be misuderstood and it can’t capture a complex variable

What are the disadvantages of closed-ended quetions?

scale

a measurement strategy for assigning a number to represent the degree to which a person possesses or exhibits the target variable

• usually a series of closed-ended items whose responses are summed into a single score

random error in any single item tends to cancel out across multiple items

Why does a scale measures more precisely than one question?

likert scale (summated ratings scale)

a scale where a participant evaluates a series of statements using a set of predetermined response options, and the responses are summed to represent the overall measurement for the variable

statements to judge, questions to answer

Likert items are phrased as _______ __ ______, not ________ __ _______.

double-barreled item

an item that asks two different questions at once but allows only a single response

participants may give a midpoint answer as a compromise or respond to only one half

• makes it difficult to tell what their answer means

What is the problem with a double-barreled item?

• split it into two items

How to fix a double-barreled item?

• keep them clear and unambiguous

  • avoid double-barreled items

• avoid jargon or terms some participants may not know

• match vocabulary to participants’ reading level

What are some strategies for writing high-quality items?

acquiescent response set (yea-saying)

a response bias where a participant tends to agree with most, if not all, of the items on a scale, regardless of what is asked

often because (based on how the items were worded), participants have caught on to what the scale is asking and agree with everything without reading

Why does an acquiescent response set (yea-saying) happen?

word some items in the opposite direction

• don’t just add “not”

  • double negatives confuse people

How do you prevent an acquiescent response set?

about half

About how many items should be worded in the opposite direction to prevent an acquiescent response set?

forced choice scale

a scale with only two response options (ex: yes/no, true/false)

• minimal sensitivity

  • ex: a “true” could mean sometimes true or always true

• participants may skip an item if they truly can’t decide

What are the problems with forced choice scales?

likert-style options

a graded range of options, typically five to nine with verbal labels

• captures degree

  • scale can tell participants apart

What is a strength of likert-style options?

error of central tendency

a response bias where a participant tends to avoid using the extreme response alternatives on a scale?

because some participants pick it just because it seems less extreme

Why does adding a middle option to a two-option item invite errors of central tendency?

• offer more response alternatives

How do you prevent the error of central tendency?

reverse-coding

a scoring strategy where more negative response alternatives are assigned higher numerical values and more positive ones lower values; used o minimize the potential for an acquiescent response set

internal consistency reliability

the degree to which the individual items in a scale are interrelated

Cronbach’s Alpha

How is internal consistency reliability assessed?

test-retest reliability

the temporal stability of a measure

• similar scores across time

correlating Time 1 and Time 2 scores

How is test-retest reliability assessed?

• because it’s not too soon and not too soon

  • too soon = people remember answers

  • too long = attitudes may truly change

Why is testing and retesting one to two weeks apart good for test-retest reliability?

alternative-form (equivalent-form) reliability

how well a measure correlates with a similar, but different, measure of the same variable

face validity

the degree to which a scale appears, on the surface, to measure the intended variable

too obvious a purpose can invite demand characteristiccs and social desirability bias

Why can having too much face validity be problematic for a study?

content validity

the degree to which items reflect the range of material the variable should cover

construct validity

the degree to which a scale actually measures the desired construct

• established by evaluating its convergent and discriminant validity

convergent validity

scores correlate with measures of other, theoretically related variables

discriminant validity

scores do NOT correlate with measures of unrelated variables

the scale taps the construct we intend

What does convergent validity show?

shows the scale isn’t measuring something else

What does discriminant validity?

criterion validity

the degree to which a measurement relates to a particular outcome or behavior

• established by evaluating its concurrent and predictive validity

concurrent validity

the scale relates to an outcome or behavior measured at the same time

predictive validity

the scale relates to an outcome or behavior that occurs in the future

shows the scale predicts real behavior

What does predictive validity show?

directional hypothesis

predicts the specific direction of a relationship

appropriate when existing theory or evidence clearly favors one direction over another

When do we create a directional hypothesis?

nondirectional hypothesis

predicts only that a relationship exists, without specifying its direction

when the literature supports competing predictions about which way the relationship should run

When do we use nondirectional hypothesis?

evaluation apprehension

anxiety or concern that participants may experience about how their answers or behaviors appear to the researcher

demographic questions go at the END

• so that a participant who quits halfway has already answered the questions about our research question

When do we ask demographic questions?

• can make study’s purpose easier to guess, which can lead to evaluation apprehension

What is the drawback of a short survey?

distractor items

items included in a scale to mislead participants as to the scale’s real purpose

• to minimize demand characeristics, social desirability, and evaluation apprehension

Why do we use distractor items?

counterbalancing the survey

changing the order participants answer the scales in for some participants

• answering questions about one scale could influence your answers on the next scale

• minimizes the potential source of bias

Why do we counterbalance the survey?

convenience sampling

recruiting whoever is easiest to reach

raises concerns about how representative the sample is

What are the drawbacks of convenience sampling?

simple random sampling

participants are chosen by chance alone from the entire population of first-year students

subgroups can still end up over- or underrepresented by chance

What are the drawbacks of simple random sampling?

stratified random sampling

split the population into subgroups by a known characteristic, then randomly sample within each to match its share of the population

What is the