HAN 251 ch 8+9

Non-experimental

does not manipulate variables

• can’t seek to identify causal relationships

• seeks to discover, describe, explain, understand, or predict relationships

typically involves one group

sampling methods

• either probability or non probability

  • excluding purposive, theoretical sampling

experiments

variables are manipulated by investigator

purpose to test the efficacy of a treatment or intervention

• experiments: identify a cause-effect relationship

• quasi: test the efficacy of intervention or treatments but can’t establish causality

must involve: control groups

sampling methods:

• experiments: probability + random assignment

• quasi: non probability sampling

Non experimental designs

developmental

• cross-sectional

• longitudinal

observational

cohort

• retrospective

• prospective

case control

correlational

survey

developmental designs

study characteristics or variables as they develop or change over time

• most frequency used by developmental researchers (ex: child psychologist)

maturations

how people change over time

cross sectional design

longitudinal design

cross sectional design

developmental design

people from different age groups are sampled, data collected once and compared

• How retired individuals in their 70s, 80s, and 90s spend leisure time = quota sampling

• What is the nature of campus involvement in U1, U2, U3, U4 students

quota sampling: comparing data analysis across the buckets to see changes

longitudinal design

developmental

a single group is followed over time (months and/or years) many data collection points

ex: take the 70s individuals and collect data for 30 years

developmental: cross sectional design

stimulates people getting older through quota sampling

simultaneously gather info from individuals about their exposure and disease status

quick and economical approach

• can get answer really quickly

• useful approach for hypothesis generating and for health service planning

• no control or comparison group

• stimulates through quota sampling

associations may reflect various biases or confounding variables

• what is the nature of campus involvement in U1, U2, U3, U4 students

  • difference might be linked to residential v commuter

• makes sure to see differences

developmental: longitudinal design

expensive, requires a team of researchers, perhaps a grant

• follow up is key on a specific time interval is essential in this research

attrition is a problem due to death, move, etc

long time before results are obtained

• long time to get to answer

• can’t get to the whole story until the end

• can take up years depending on the study

can’t quickly test new hypothesis

cross sectional

takes a snapshot of stages of development from different groups at the same time

• researcher collects data are time from different groups to simulate development over time

• allows for quick, inexpensive data collection

• low internal validity due to possibility of confounding variables

longitudinal

follows a group over a period of time to study development

• allows for the researcher to set up multiple data collection points over the life of the study and look at multiple variables

• often expensive and requires a team of researchers

• high rate of attrition (loss to follow up)

observational research design

records/measures naturally occurring behavior to better understand:

• what behavior is occurring

• the frequency of a behavior

measures visual data

observational design

Can provide QUAN alternative to QUAL approaches

Prespecified focus to observations, underlying theoretical framework

• Operational define what is being observed in order to count or evaluate observation

• Data collection is divided into small segments of time

• Codes are predetermined backed on theory (word alert)

• Rating scale (data collection) is used to evaluate the observation

  • Often more than one independent raters

• Involves in-depth training of team and raters to ensure consistency

  • Interrater reliability

observational vs. QUAL designs

all use observations, so how are they different?

• data analysis

• data collections = data collection/2 step process

QUAL

• makes observations but do not predetermine what they will observe

• Codes (data analysis)= reveal meaning units, when combined lead to themes

QUAN- observational research design

• predetermine observations, predetermined code book (variables though rating scales) from theory and hypothesis

• use numbers, rating, and statistical analysis

Cohort research design

used to determine if an exposure is linked to the development of a disease or outcome when the relationship between the two has not yet been determined

start with exposure

study if the exposure is through disease or conditions

Cohort Studies

no manipulating variable

sees association doesn’t approve it

cohort studies looks to determine if an exposure is linked to the progression of a condition

they can be either:

• prospective: looking forward

• retrospective: looking back into existing data

  • look at medical record

  • is there is cluster of disease higher rate than nationally

follow them for five years

if the exposure of the disease or condition have a higher rate than nationally

Prospective cohort study

Study begins with the identification of a population and exposure status (exposed/not exposed groups)

Population is followed over a period of time for the development of disease or condition

Retrospective cohort study

Previously collected data is reviewed to identify the population and the exposure status (exposed/not exposed groups)

Determines at present the (development) status of disease or condition

cohort study

advantages & disadvantages

Advantages
– Subjects in cohorts can be matched, which limits the influence of confounding variables
– Standardization of criteria/outcome is possible

Disadvantages
– Cohorts can be difficult to identify due to confounding variables
– No randomization, which means that imbalances in patient characteristics could exist
– Blinding/masking is difficult

case control research design

begins after the development of the outcome and look back in time to identify exposure (risk factor)

starts with disease and looks for exposure - why one group is ill and another group is welll

similar groups aren’t control group

case studies studies

Designed to estimate the odds of developing the studied disease/condition. Looks to determine if there is an associational relationship between condition and risk factor.

Case control studies

• The goal is to determine how the exposure differs between the two groups of individuals: cases and controls.

case control studies

advantages and disadvantages

Advantages
– Good for studying rare conditions or diseases
– Less time needed to conduct the study because the condition or disease has already occurred
– Simultaneously looks at multiple risk factors
– Useful as initial studies to establish an association

Disadvantages
– Retrospective studies have more problems with data quality because they rely on memory and people with a condition will be more motivated to recall risk factors (also called recall bias).

Cohort vs. case control

Cohort studies

• Begin with a group of people (a cohort) free of disease.

• The people in the cohort are grouped by whether or not they are exposed to a potential risk

• See if the development of new cases of the disease (or other outcomes) differs between the groups with and without exposure.

Case-control studies

• Begin with the selection of cases (people with a disease) and controls (people without the disease).

• The controls should represent people who would have been study cases if they had developed the disease (population at risk).

• The exposure status to a potential exposure cause of disease is determined for both cases and controls.

• Then the occurrence of the possible cause of the disease could be calculated for both the cases and controls.

statistical tests

cohort design

• Data analysis = determining relative risk (RR) as long as the outcome is not rare.

  • Relative risk is defined as the probability of an outcome of interest developing as a result of the exposure being followed

Case-control design

• Data analysis = odds ratio (OR) the measurement of association between an exposure and an outcome.

  • “The OR represents the odds that an outcome will occur given a particular exposure, compared to the odds of the outcome occurring in the absence of the exposure”

correlational research design

predicted association

Attempts to determine if the characteristics of one or more variables are associated with the characteristics of another variable.

Examines the extent (strength and direction, degree of association) of relationship between characteristics or variables within a group or between two or more groups

good common sense and look at literature reviews

correlational designd

Examines the extent (strength and direction, degree of association) of relationship between characteristics or variables within a group or between two or more groups

Correlation exists if:

• One or more variables increase or decrease in relation (strength and direction)

• The values of those variables are distributed in a consistent manner

  • Age is related to Reading level: typical as age increases so does reading
    level

• Discrete combination of variables combined can predict the
  phenomena

  • Residential status, SAT scores, and Greek membership predict XX outcome

Pearson r (linear or product moment correlation)

Correlation coefficients can range from -1.00 to +1.00
– Value of -1.00 represents a perfect negative correlation
– Value of +1.00 represents a perfect positive correlation
– Value of 0.00 represents a lack of correlation

positive correlation

points lie close to a straight line, which has a positive gradient

one variable increase the other increases

negative correlation

points lie close to a straight line, which has a negative gradient

one variable increases the other decreases

no correlation

no pattern to the points

no connections between two variables

caution about correlational research

Interpret results with caution

• For example:

  • Strong and positive correlation (r = .90) between ice cream sales and gun violence

    • Can I determine that ice cream sales is correlated with gun violence?

    • Can I determine that ice cream causes gun violence?
      

• Strong and positive correlation (r = .90) between elephant population in Thailand and size of the Florida orange crop

  • Any two things that increase at the same time annually appear to be correlated but only faulty logic sees a relationship between two unrelated events

correlational vs experimental

Correlational research usually does not influence any variables

• Looks for relations (correlations) between some set of variables

• Data analysis looks at the strength and direction of the relationship

• Cannot prove causality

Experimental research manipulates variables

• Measure the effects of this manipulation on other variables

• Data analysis also calculates "correlations" between variables, specifically, those manipulated and those affected by the manipulation

• Has the power to prove causality

Do not get confused by data analysis techniques and research design

definition of words change based on the context

Correlation(al)

• Statistical test (used during data analysis in any QUAN study)

• Research design

Code

• A step in QUAL analysis

• QUAN data collection (code book – rating tool)

Observation(al)

• QUAL data collection procedure

  • prolonged engagement/persistent observation

• QUAN data collection procedure

  • rating scale or rating tool of behavior

• Research design

• The O in scientific notation

Research Design

Survey Research

Survey research is one of the most important areas of measurement in applied social research

• Survey research: can seek to correlate variables BUT the main focus is on learning about a population

• Correlational research: main focus is determining how the variables are related or how variables can predict outcomes

Surveys can collect data about: to understand the population…
– Knowledge
– Attitudes/Feelings
– Perceptions/Beliefs
– Behaviors

survey research

• Surveys can be defined as any measurement procedures that involve asking questions of respondents. 

• Information may be collected directly (face to face interview, telephone interview) or indirectly (mail or private completion of a questionnaire)

A "survey" can have different formats

• Short paper-and-pencil feedback form

• Mailed surveys/questionnaires 

• Focus group

• Intensive one-on-one in-depth interview 

  • How is this different from QUAL?

    • Purpose of research is different 

• Automated telephone surveys that use random dialing methods 

• Computerized kiosks in public places that allow people to ask for input 

• On line surveys (i.e., survey monkey or Qualtrics).

Personal interviews

strengths and limitations

strengths:

• generally has high response rate

• allow the interviewer to elaborate on questions or ask for confusion

• responses are usually easy to analyze

limitations

• costly due to large number of interviewers needed

• data colleciton is slow and overall study requires a lot of time

• difficult to control for interviewer bias

telephone interview

strengths and limitations

strengths

• less costly because no field work is required

• random digital dialing (RDD) allows researcher to reach to large representative sample

• data collection takes less time than in person interviews

• allows interviewers to elaborate on questions or ask for clarification

• better response rate than mailed surveys

limitations

• only reach household that have telephones

• higher non-response rate than inperson interviews

• subject listens and responds without no visual data

• hard to control for question confusion when asnswering

mail survey

strengths and limitations

strengths

• cheaper than phones or inperson interviews

• need for a smaller number of interviewers/staff contributes to lower cost

• provides access to a large representative sample

• respondents can participates when it is convenient.

limitations

• easy for respondent to not participant or forget to participate

• low response rate

• incentives may increase participation but also increase cost

• longer waiting periods for responses to be returned

• increase response rate but also increase cost

online survey

strengths and limitations

strengths

• lowest cost

• provides assess to global population

• timely

• easy to collect only relevant data through online programs

• provides access to an enormous representation sample

limitations

• varying compute capabilities may not allow access to some households

• easy for respondents to ignore or delete requests, within lead to very low response rates

• may contain higher response rate from those interested in the topics, resulting in bias data

open or unstructured survey questions

allows respondent some sense of freedom to answer the question and give opportunity to elaborate on topics using their own words

less structure to the response than closed

answer to these questions usually involve some from of qual analysis

closed or structured survey questions

responses are easy to quanitfy and are turned into numerical form of analysis

all responses were turned into numbers for data analysis

Semi structured survey questions

respondents is asked predetermined questions with an occasional open-ended question when researcher is looking for clarification or elaboration on a response

Sampling

can the population be enumerated

• for some populations, you have a complete listing of the units that will be sampled

• for other populations, it will be difficult

  • if your study requires input from people who are unhoused, you will find a way to sample your population

    • may rule out mail survey or telephone interviews

will your sampling lead to unintentional subgroups which will bias your findings

sampling— response rates

low response rates are among the most difficult of problems in survey research

• some members of your sample will simply refuse to respond

• others have the best of intentions, but can’t seem to find the time to send in your questionnaire by due date

• still others misplace the instrument or forget about the appointment for an interview

data collection

interviewer distortion

• May ask a question in a way that distorts the meaning of a question.

• May not ask questions that make the respondent uncomfortable (i.e., may only ask ‘safe’ questions).

• May not listen carefully to respondents on topics for which they have strong opinions.

• May make the judgment that they already know what the respondent would say to a question based on their prior responses, even though that may not be true.

training the interviewers

• Here are some of the major topics that should be included in interviewer training:

• Explain interviewer bias 

  • This is especially a problem when the content of the survey is highly charged and people have strongly held convictions 

• Skills in questioning

  • Don’t use cues, leading questions

  • Manage group dynamics in focus groups 

focus group

• Are a way to listen to people and see how interactions between people impact their answers

  • body language

• Should not have more than 8-12 people

• Props (video clips, products, public service announcements, etc.) can be used to generate discussions

• Length varies between 90-120 minutes

  • Video or tape recorded

  • Transcribed

false respondents

With mail and phone surveys  

• Did the head of household complete the survey or someone else? 

• Did the CEO actually give the responses or pass the task off to an assistant? 

• Is the person you're speaking with on the phone actually who they say they are?

implications in using questionnaires

• Require that your respondents can read!

• What grade level will the questions be written at?

• Questions need to developed without using medical or professional jargon  

• Can you produce multiple versions of your questionnaire effectively?

  • Culturally specific?  

  • Will nuances get lost in the process of translating your questions?

IRB makes sure the survey or questionnaires is at a level that does not exceed 8th grade reading level

translators can be hired and that are qualified

survey as data collection tool

avoid pitfalls when constructing questions

wording - too many complex questions

• How often have you visited a doctor during the past year?

• How many times have you been treated as a patient in an emergency department during the past year?

• During the past year, how many times were you admitted to the hospital?

simplify and combine questions

• During the past year, how many times have you:

  1. Visited a doctor? 

  • None

  • 1-2

  • 3-4

  • More than 5

  1. Been a patient in the emergency department?

  • None

  • 1-2

  • 3-4

  • More than 5

  1. Been admitted to the hospital?

  • None

  • 1-2

  • 3-4

  • More than 5

dichotomous question format

When a question has two possible responses

• Surveys often use dichotomous questions that ask the respondent to select one of two possible answers 

  • Yes/No 

  • True/False 

  • Agree/Disagree response

• closed questions that dictates exactly how respondents must answer with no room for variation in responses

• structured, easy to quantify during data analysis

• often used filter/screening questions when researcher is looking to remove a portion of respondents whom survey doesn’t relate

demographic question format

what is your occupation? (can be iv)

• physician

• doctor of osteopathy

• nurse practitioner

• physician assistance

nominal questions

are used to label, name, group responses

doesn’t assign a value to each response

not ranked

ex: my hair is

brown black blond gray

ordinal question

assign meaning to response by ranking them in order from lowest to highest or the other way around

ranked order

interval questions

rank order response like ordinal question

distance between the response is unknown, the distance between response is measured in standard increments

interval scales allow researcher to utilize a wider range of data analysis including inferentials statisitcs and averages

rank order question format

• Make sure instructions are clear – 

Rank the following exercises in order of preference from best to worst 

____ running

____ walking

____ weight training

____ swimming

• Do you want respondent to put a 1, 2, 3, etc. next to the exercise, where 1 is the respondent's first choice? 

• The prompt was not explicit

  • Respondent needs directions to figure out we want a rank order with 1 at highest rank and 4 at the lowest rank. 

  • Otherwise respondent might check their favorite or check all their favorites

likert scale

strongly worded statement

Used to quantitatively assess abstract concepts (insubstantial phenomena) attitudes, self-reported behaviors, feelings, values, etc. 

Ask an opinion question on a bipolar scale (it's called bipolar because there is a neutral point and the two ends of the scale are at opposite positions of the opinion).

ex: I think dealth penalty should be banned in the US

1 strongly disagree 2 disagree 3 neutral

4 agree 5 strongly agree

semantic differential scale (SDS)

measure attritudes, values, and opinions by having respondents rate their opinion or belief on a scale using bipolar adjective

ask a number of diff questions in short number of time and space making it a useful option for many researcher

utilizing a neutral point between bipolar adjective serves as a zero point

uses adjective or phrases that are completely opposite of each other to classify within a relatively short amount of time and space the respondents degree of feeling positivity and negativity.

guttman scale

Cumulative or Guttman Scale

• Respondent checks each item with which they agree. 

  • The items themselves are constructed so that they are cumulative, ranked ordered in difficulty from least to most extreme or most to least extreme.

Bogardus social distance scale is an example of a Guttman scale

1. __Are you willing to permit immigrants to live in your country?

2. __Are you willing to permit immigrants to live in your community?

3. __Are you willing to permit immigrants to live in your neighborhood?

4. __Are you willing to permit immigrants to live next door to you?

5. __Would you be comfortable if your child wanted to marry an immigrant?

• Agreement with item 3 implies agreement with items 1 and 2

will filter questions be needed

filter questions

For instance, you wouldn't want to ask someone their opinions about a specific issue without first "screening" them to find out whether they have any experience.

Sometimes you have to screen on several variables (age, gender, experience).

The more complicated the screening, the less likely it is that you can rely on paper-and-pencil instruments without confusing the respondent.

• Computer surveys make screening much easier.

if they answered no they are unfilter out

filter people out for those who have no exeprience

open ended question

Not uncommon to add a qualitative question at the end of the survey.

• Is there any other information you would like to provide about how you think or feel about vaccines? 

Does not make it a mixed method study but does require the research to do qualitative data analysis on the open ended responses

unstructured

allows respondents to write response in their own words

easier to write but hard to turn into quan data

can be misunderstood by respondent

• difficult to quantify or group together

issues to be aware of when developing questions for your survey

content

question necessary/useful

double barrel

time frames

biased questions

neutrality in questions

wordings

sensitive questions

truthful in answer

issues to be aware of when developing questions for your survey

content issues

The content of your study can pose challenges. 

• Can the respondents be expected to know about the issue?

  • A study of family finances 

    • you are talking to the spouse who doesn't pay the bills on a regular basis, they may not have the information to answer your questions.

• inclusion criteria to help

issues to be aware of when developing questions for your survey

is the question necessary or useful

The question must be directly related to your research questions. 

• If you have a question about children, what do you need to know?

  • the age of each child, does the child live with you, or it is sufficient to know the number of children under a certain age?

• If you have a question about income, do you need to know the exact amount or is a range sufficient?

issues to be aware of when developing questions for your survey

double barrel questions

you can often spot these kind of problems by looking for conjunction “and” in questions

asking question that have two concepts and therefore unable to answer

• can support one concept but not other

• unable to answer question with yes or no

• conflict between the two

issues to be aware of when developing questions for your survey

time frames

Time frame must be specified in questions

“How much exercise do you usually get?”

• You are leaving that up to the person answering question – some will answer the question with very different time frames (day, week, month)? 

• Resulting in worthless data

Better question: 

“In a typical week, how many hours do you spend walking for exercise?

issues to be aware of when developing questions for your survey

biased questions

use likert scale

framing questions that doesn’t allows respondents to disagree it

or create an assumption of the respondents feelings or beliefs

ex:

• What are the benefits of a tax cut?

• What are the disadvantages of eliminating the WIC programs?

  • You're only asking about one side of the issue.

  • Already telling people how they should answer 

    • tax cuts have benefits

    • Cutting WIC has disadvantages 

issues to be aware of when developing questions for your survey

neutrality in questions

Avoid ‘loaded’ words or words that cue a certain response. 

Also avoid vague words: what’s too much? 

• During the last month, how often did you drink too much alcohol? 

vs 

• During the last month, how often did you drink 5 alcohol beverages in 2 hours or less?  

  • You could change the # of drinks and time frame depending on what information you want to capture.

issues to be aware of when developing questions for your survey

will the respondent answer truthfully

• Respondents generally want to "look good" in the eyes of others. 

  • None of us likes to look like we don't know an answer.

  • We don't want to say anything that would be embarrassing.

  • May not tell you the truth, or may "spin" the response so that it makes them look better.

When writing survey questions, consider wording the question very neutrally – will help respondents to answer the question truthfully. 

Some people are sensitive about answering questions about their exact age or income. 

issues to be aware of when developing questions for your survey

sensitive questions

Set the tone by using an introduction that allows someone to admit to an undesirable behavior.

• People sometimes forget to take the medication their doctor prescribes. 

  • How many times in a typical [insert time frame] did you forget to take your medication?

• Time frame [day – week – month] will be based on medication/disease studied or on what specific behavior you are researching.

In much of our health based research, we have to ask respondents about challenging, private, or uncomfortable subjects. 

• Before asking such questions, you should attempt to develop some trust or rapport (close relationship/bond) with the respondent. 

• Often, preceding the sensitive questions with some easier warm-up ones will help. 

  • But, you have to make sure that the sensitive material does not come up abruptly or appear unconnected with the rest of the survey. 

It is often helpful to have a transition sentence between sections:

• In this next section of the survey, we'd like to ask you about your personal relationships. Please do not answer any questions if you feel uncomfortable doing so.