COGS 14A | Chapter 11: Surveys and Interviews

questionnaires

* set of questions created to learn about individuals


* not aggregated data, we just want to learn about the individual!

surveys

* questions to learn about individuals, where data collected is aggregated
* used to gather reliable and valid information

mail survey

a written, self-administered survey sent through the mail that should be __self explanatory__ and __clearly worded__

* pros
* reach! → less sampling bias
* responses more likely to be complete bc self-scheduled
* better for collecting sensitive info bc feels more anonymous
* cons
* distribution is not perfect
* not willing to respond

internet surveys

* email or website
* self-administered + can’t clarify questions
* pros
* cost
* easier to find population of interest
* responses can be gathered immediately
* more sensitive info
* cons
* less response rate
* not sure who responded + snowballing

group-administered surveys

* given to a group of respondents in one setting
* pros
* high response rate
* appropriate group not always available for surveying or topic not always appropriate

telephone surveys

* done through the phone
* pros
* higher response rate
* cons
* possible sampling bias
* interviewer bias
* dated

personal interviews

* face to face meeting
* pros
* sure of identity
* can determine if data collected from them is relevant
* high response rate
* responders will devote time
* can clarify/follow up
* cons
* harder to assure anonymity
* interviewer bias, subjective interpretation
* socially desirable responses but not true beliefs
* sampling bias
* expensive
* time-consuming

funnel structure

begins w broad, easy questions; gradually becomes more thoughtful/specific

demographic questions

descriptions on social statistics - age, income, gender, etc

branching

answers to demographic questions might determine which specific questions will come next

filter question

prevents respondents from having to answer irrelevant questions → determines next question

* e.g. “if you said no, skip to question 5”

open-ended questions

* respondent form their own response
* can provide more complete info
* but more difficult to score

closed questions

* choose an answer from provided options
* limited responses
* can use more sensitive measurements (e.g. rating scales)

leading questions

presents info w/in the question that leads the respondent to answer in a desired manner

loaded questions

emotionally laden, suggests writer’s opinion

double-barreled questions

ask more than one question at the same time, confusing and convoluted

reliability

degree to which a measurement tool provides consistent answers

test-retest reliability

measures degree to which a test generates the same responses upon retesting

alternative-forms reliability

asses how well two forms of the same test yields comparable results

construct/composite reliabilty

measure the degree to which respondents’ replies w/in the instrument are consistent

split-half reliability

determine if a person’s replies to one half of the items in the construct are related to the same person’s replies to the other half of the items

Cronbach’s alpha

assesses internal consistency when calculated; compiles correlation of every item w every other item w/in the tool

validity

degree to which a measuring tool measures what it purports to measure

face validity

degree to which a measurement tool appears to be measuring what it is supposed to be measuring

construct validity

extent to which concepts thought to be measured w/in the tool are actually being measured

criterion validity

measures how well the results of an instrument correlate w other outcomes or behaviors

pilot study

small group of people is given the survey to work out any bugs in questions/data gathering

population

all of the members of a given group to which the research is meant to generalize

sample

subset of the population

sampling rame

samples are chosen from this, a list of all the members of the population

random selection

all members of population are equally likely to be chosen for the sample

systematic sampling

chosen according to specific plan/strategy

* e.g. every 3rd person in the list is chosen
* random + systematic: randomly selecting a starting point and choosing every kth person

stratified sampling

the sampling frame is divided into subsamples and people are systematically/randomly chosen from the subsamples

* if random - stratified random sampling
* sample more likely to be representative of population

cluster sampling

clusters of potential respondents that represent the population are identified, and all of the people in those clusters are included in the sample

convenience sample

chosen from readily available situation

* risk of bias!

quota sampling

* convenience _ stratified
* convenient sources of subgroup/strata members

snowball sampling

research participants identify other potential participants

* can be helpful in finding hard-to-find populations

nonprobability sampling

each member of the population is not equally likely to be selected and outcome could be a biased sample

* convenience, quota, snowballing