Research Methods Exam 2

Survery and poll

• what validity is most important to consider when creating a poll/ survery

a method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via internet

A construct validity is most important

Open-ended questions

A survey question format that allows respondents to answer any way they like(provides researchers with spontaneous rich information) (very broad)

• the drawback is that responses must be coded and categorized which is a difficult and time consuming process

ex: asking people to name a public figure they like; departing overnight guests might be asked to comment on their experience at the hotel.

forced choice question

A survery question format in which respondents give their opinion by picking the best of two or more options. (A simple yes/no are also considered this)

ex: often used in political polls asking if they would vote democrat or republican and are also used to measure personality

Likert scale

A survey question format using rating scale containing multiple response options anchored by the specific terms (strongly agree, agree, neither agree or disagree, disagree, and strongly disagree). A scale that does not follow this format exactly is called a Likert-type scale

Semantic differential format

A survery question format using a response scale whose numbers are anchored with contrasting adjectives. Numbers carry a value with it.

ex: Rate my professor, five-star rating format sites (Yelp)

Question wording matters

Leading question

A type of question in a survery/poll that is problematic because its wording encourages one response more than others, which weakens its construct validity.

Double-barreled questions

A type of question in a survey/poll that is problematic because it ask two questions in one, which weakens the construct validity.

• people might be responding to the first half of the question, the second half, or both. The item could be measuring the first construct, the second construct, or both.

Negatively worded questions

A question in a survery/poll that contains negative phrased statements making the wording complicated or confusing potentially weakening its construct validity.

ex: “Abortion should never be restricted.”

• those who oppose abortion must think in the double negative “I disagree that abortions should never be restrictred”

• those who support abortion rights would be able to answer more easily “I agree - abortion should never be restricted”

When possible negative wording should be avoided be avoided

Response sets

A shortcut respondents may use to answer items ina long survery rather than responding to the content of each item (nondifferentiation)

• rather than thinking carefully about each question people might andwer all of them positively, negatively, or neutrally. Response sets weaken contruct validity because the respondents are not saying what they really think.

Acquescience (to yeild; yea-saying)

Answer, “yes” or “strongly agree” to every item in a survey (yea-saying)

ex: a respondent might answer “5” to every item on Diener scales.

Fence sitting (all neutral)(think of Likert scale)

Playing it safe by answering in the middle of the scale for every question in a survery or interview.

• people might also answer in the middle (say “idk”) when a question is confusing or unclear. Also weaken a surverys construct validity when middle of the road scores suggest that some responders dont have an opinion

Force them to make a choice (make it even number of choices so there wont be a neutral option)

2 ways to ensure survery quesions are answered accurately

clean wording + hiding identity

socially desirable responding (faking good)

Giving asnwers on a survery that make one look better than one really is

• respondents are embrarased, shy, or worried about giving an unpopular opinion

Faking bad

Giving answer on a survery thak makes one look worse than one really is

observational research

the process of watching people or animals and systematically recording how they behave or what they are doing

• because people cannot always report their behavior or past events accurately some scientists believe observing behavior is better than collecting self-reports through surverys

• basis for frequency claims. can also be used to perationalize variables in associating claims and causal claim

observational research is way to operationalize a conceptural variable, so when iterrogating a study we need to ask about the construct validity of any observational measure

observer bias

• when the observer see what they expect to see

a bias that occurs when observer expectations influence the interpretation of participants behaviours or the outcome of the study

observer effects (expetancy effect)

a change in behaviour of study participants in the direction of observer expectations

• Bright and dull rats (giving students rats; one side says the rats are bright, so the students that have those rats trained them more and the other side says the rats are dull, so the students that have those rats treated them as such)

• Clever Hans

masked design (blind design)

• masking which partcipant gets the conditioned and what the researcher see(a blind study; does not know what condition the participants have so it reduces the observer bias)

A study design in which the observers are unware of the experimental conditions to which partcipants have been assigned

Reactivity

A change in behavior of study participants (such as acting less spontaneously because they are aware they are being watched)’'

• they might react being on their best behavior or in some cases their worst-rather than displaying their typical behavior

• this can also occur in animal subjects

Ways to avoid observer effects (reactivity: when participants react to being watched)

1. Blend In → Unobtrusive observation= made indirectly through physical teaces of behavior or made by someone who is hidden or is posing as a bystander

2. Wait it out → a researcher plans to observe at a school might let the children get used to his/ her presence untul they forget theyre being watched

3. Measure the Behavior’s results(unobtrusive data) → instead of observing behavior directly, researchers measure the traces a particular behavior leaves behind. (Using indirect methods, researchers can measure behavior without doing any direct participant observation)

Observing people ethically

most psychologist believe it is ethical to watch people in musuems and classrooms, sporting events, and evet at the sinks of public bathrooms because in those settings people can reasonably expect their actitives to be public not private.

• one-way mirrors, covert video recoding

Question 1

a

question 2

B

question 3

A

quiz 4

A

quiz 5

b

Generalizability

Interrigating external validity

Population, Sample, and Census

Population= a larger group from which a sample is drawn; the group to which a study’s conclusions are intended to be applied(called population of interest)

Sample= the group of people, animals, or cases used in a study; a subset ofthe population of interest

Census= a set of observation that contains all members of the population of interest

population of interest-defined by research

Biased Samples

Unbiased sample( sample some members of the population are systematically left out and the result cannot generalize to the population of interest)

Biased sample( sample all members of the population are equally likely to be included → through a random method← and results can generalize to the population of interest)

Two ways when Sample could be biased

• Conveience sampling = choosing a sample based on those who are easiest to access and avaiable

• self-selection= A form of sampling bias that occurs when a sample contains only people who volunteer to participate

They both threaten external validity because people who are convienient or more willing might have a different opinion from those who are less handy and less willing

Representative Sample

Probability sample( a category name for random sampling techniques → simple random, stratified, and cluster← every member of the population of interest has an equal and known chance of being selected for the sample regardless if they are convenient or motivated to volunteer (also called random sampling) = excellent external validity and generalize the population of interest

Nonprobability sample( a category name for nonrandom sampling techniques → convenience, purposive, and quote sampling← results in a biased sample

• Systematic sampling(

Nonrandom sample

Representive sample vs Unrepresentative sam

Probability Sampling

• Simple random Sample( the most basic form of probability sampling → the sample is chosen completely at random from the population of interest (drawing names out of a hat)

• systematic sampling= a probability sampling technique in which researchers uses a randomly chosen number N, and counts off every Nth member of a population to achieve a sample

While simple random sampling and systematic sampling work well in theory they can be surprisingly difficult and time consumingpick every kth person after a random start(every 10th name)

• cluster sampling= divide population into clusters(school/neighborhoods); randomly select some clusters, survey everyone inside them

• multistage sampling= sampling in stages; cluster first then random selection within each; mixes several techniques for flexability(EX: choose 5 schools then 20 students from each)

→ Both these samples are easier and both should still produce a representative sample because they involve random selection

• stratified random rampling= researcher purposely split population into strata(categories)→(age, gender, income level); randomly sample within each group and ensure key subgroups are proportionally represented

→ Strata are meaningful categories(ethnic or religious groups) whereas clusters are more abritary(any set of highschool would do)

→Final sample sizes of the strata reflect their proportion in population. Clusters are not selected with such proportions in mind.

• oversampling= purposely select extra cases from small or underrepresented groups; allows enough data from meaningful subgroup analysis

Unrepresentative Sample: Nonprobability Sampling

• Convenience sampling(Choosing a sample based on those who are easiest to access and readily available → Biased sampling)

• purposive sampling(a biased sampling technique in which only certain kinds of people are included in a sample)

• snowball sampling( a variation on purposive sampling a biased sampling technique in which participants are asked to recommend acquaintances for the study)→ help researchers find rare individuals

• quota sampling( a biased sampling technique in which a researcher identifies subsets of the population of interest → sets a target number for each category in the sample and nonrandomly selects individuals within each category until the quotas are filled)← similar to stratifired random sampling

Combining those techniques

Population of interest

→ Probability sampling (all members of population habe an equal & known chance)

• Simple random sample

• Systematic sample

• Oversample

• Cluster sample, Multistage sample

• Stratafied random sample

→ Nonprobability (some types of people are systematically left out)

• Convenience sample

• Purposive sample

• Quota sample

• Snowball sample

Random Sampling increase external validity

Random assignment: used only in experimental designs to assign participants to groups at random → Increases internal validity

In frequency claim external validity is a priority

Cases where external validity is a lower priority

• nonprobability samples in the real world

• nonprobability samples in research studies

Q1

A and D

Q2

D; dont have external validity

Q3

B;