Advanced Research Methods Final Exam (Class Notes)

How can you get involved in research?

1. assisting in research teams or research labs

2. contacting researchers in some way

3. thesis, dissertation, other independent projects

4. grant & IRB applications

5. collaborations

What are different types of lab activities?

1. lab meetings

2. training sessions

3. checking equipment and manipulations

4. recruiting RAs

5. recruiting and running participants

What are ways you can prevent bias in a study?

1. prevent experimenter effects/bias

2. prevent expectancy/placebo effects

3. check for demand characteristics

4. check for social interaction differences

5. manipulation checks

6. use pilot studies

What are the steps to take after you have collected the raw data?

1. have step-by-step instructions for what is to be done once the raw data is collected

2. look at the data, familiarize yourself with the variables, the type of data, deal with and recognize if data is missing

3. compute summary scores

4. relate data to hypotheses and determine the outcomes

How do you work the data?

1. cleaning the data

2. transferring to SPSS

3. dealing with missing data

4. looking for outliers and response sets

5. assessing normality and kurtosis

6. reliability and factor analyses

How can data be missing?

1. MCAR-- missing completely at random (best)

2. MAR-- missing at random (will skew results)

3. MNAR-- missing not at random (the worst!)

What are some data techniques to fill in missing data as best as possible?

1. list-wise deletion

2. pairwise deletion

3. maximum likelihood

4. multiple imputation

List ways experimenter effects/bias can creep into your study.

1. it can occur through non-consistency with the delivery of manipulations

2. the interpretation of the data could be skewed

3. leading questions could have been used

How do you deal with expectancy/placebo effects?

1. we can control for knowing that a placebo effect will happen

2. can now account for the space between the treatment and control group

What is a demand characteristic?

anything within the experiment that could lead the participants to learn what the hypothesis is

How can you prevent demand characteristics from creeping into your study?

1. pilot studies

2. make sure that all of the individuals do not know the hypotheses

What are examples of social interaction differences?

1. training vs. no training and the participants talk to one another

2. you can bring participants into a lab for one group and then after the experiment they will tell the other participants so now the second group is tarnished (participants are now different-- not good)

3. if social interaction is your manipulation, the problem we could have is that these people now have groups of talk and the groups that don't and there is so much more noise in that having more people/groups

How you check your manipulations?

1. asking yourself if you are creating enough difference between the groups; aka, they are not the same treatment

What do pilot studies allow us to do?

1. manipulation checks

2. can check all portions of the study to ensure that all of the factors are up to par with statistical, theoretical, and methodological standards

When you look over the raw data to familiarize yourself with it, what are you looking for?

1. the nature of the data

2. a threshold for throwing out participant data: more than 5 items missing out of 100

3. deciding what to do with the outliers: keep them

4. backing up the data

5. checking the response rate

6. checking assumptions of normality and kurtosis

What should you keep in mind when you are computing summary scores?

1. be SUPER familiar with the coding scheme you created

2. don't forget to recode reverse items

3. look for patterns within the responses

4. figuring out if you want to transform the data

When should you transform data?

1. if the units are different

2. when you have set a limit and defined what score threshold means

Explain the list-wise deletion approach to dealing with missing data.

1. any little item that is not answered will be thrown out

2. problems with this approach: it will kill your sample size and power-- you will loose a lot of people

3. tend not to be used a lot

4. does not help with missing data

5. will stay unbiased

Explain the pair-wise approach to dealing with missing data.

1. we have to do totals for all of our scales, in SPSS if it comes across a missing variable it is not going to give you a total for that scale-- the ends will be different

2. so what you are basically doing is losing scale for people without dropping them out

3. problems with this approach: once you go to run bigger stats that have four or more variables in them, you are losing even more people for them; small amount of data will be okay but large amounts of data will be problematic

4. does not help with missing data

5. will stay unbiased

Explain the maximum likelihood approach to dealing with missing data.

1. figuring out something to put in for the missing data and putting in statistics to do it

Explain the multiple imputation approach to dealing with missing data.

1. imputation, analysis, and pooling of data

2. finding a way to fill in seems to be the better approach than the deletion side of it

What do in-text citations do?

1. "may provide key background information, support or dispute your thesis, or offer critical definitions and data"

2. allow us to use previous information to back our own points/ideas while avoiding plagarizing

What do you cite?

1. anything that is not your own original idea

2. definitions of psychological concepts

3. findings from a research study

4. theories developed by past researchers

What do you NOT have to cite?

1. your thoughts and opinions

2. reports of your methods or data in your paper you are currently writing (not published yet)

3. your commentary on a study done by someone else

When do you use direct quotes within a paper?

1. when citing a definition word-for-word

2. when explaining the specific portion of the procedure, such as a survey item or directions given

3. when the language is particularly striking or articulate

What does "et al." mean?

1. short for "et alli"

2. meaning "and other"

What does "e.g." mean?

1. short for "exempli gratia"

2. meaning "for example"

3. used to give examples

What does "i.e." mean?

1. short for "id est"

2. meaning "that is"

3. used as "in other words" or for clarification

What are some solutions to unusual citation examples?

1. when there is no author-- use the first few words of the title and the year

2. when it is a government & organization document-- use the name of the gov. or org. and the year

3. when it is a classic work (like Aristotle)-- the author's name and the year your copy was translated

What does the IRB do/prevent?

1. prevent unethical treatment of participants

How does the IRB reinforce what they do?

1. they have a review that one has to go through in order to conduct research

Who has IRB's?

1. universities, research hospitals, certain government agencies, any kind of research institution

Who serves on IRB's?

1. one non-scientist

2. scientists who are qualified to evaluate protocol

3. someone outside of the institution/organization

What are the different types of submission responses from the IRB?

1. expedited: no more than minimal risk

2. full-review: anything that has more than minimal risk

3. exempt: chair looks at it; just to see if it fits the category

When reviewing an IRB submission, what is the IRB specifically looking for?

1. trying to see if there will be risks involved

2. they want the benefits to outweigh the costs

3. they want to make sure that the methodology is sound; otherwise it will be hard to see how benefits from this research

What are some IRB focal points for informed consent?

1. confidentiality

2. identity risks

3. who they can contact

4. voluntary

What is the difference between privacy and confidentiality?

1. privacy happens before and confidentiality happens second

2. privacy is basically the right you have to keep things that you want to be private, private; the participant chooses a little bit to give up, but we need to ensure that we do not give the rest of their privacy up

3. confidentiality is the idea that data can be linked to someone

What are some IRB focal points for selection of participants?

1. sound methodology

2. helps to keep justice maintained

3. equal opportunity to being a participant

What are some IRB focal points for incentive for participation?

1. that participants were not influenced into participation

What are some IRB focal points for methodology and data analysis?

1. that the data will be helpful and can be used for benefits

What all needs to be in the informed consent given to the research participant?

1. basic purpose of the study (minus deception)

2. what will they have to do

3. risks, benefits, compensation information

4. promises of privacy and confidentiality and how

5. contact information of the researcher

6. contact information of IRB

7. thanking the participant for their vital role in research

What is APA style?

1. a systematic way to provide information about the sources of information used in a paper

2. tells us how to put our paper into sections and what to put in each section

What is the general format of an APA style paper?

1. entirely double-spaced

2. no extra space other than the double spacing between paragraphs

3. margins set to 1" on each side

4. left-aligned (right side can be jagged)

5. 12-point font

6. body of paper uses serif font (have feet)

What is the tone of an APA styled paper?

1. scientific, not literary, writing

2. be clear, concise, and formal

3. no long-winded sentences loaded with extravagant words that are not necessary

4. avoid repetition

5. avoid wordiness

6. continuity

7. avoid colloquial and slang expressions

8. avoid jargon

9. use active voice, not passive

10. do not use contractions

11. don't give life to inanimate objects

What are the main uses for commas?

1. break up three or more things

2. after a clause, prepositional phrase, or any introductory element

3. set off parenthetical elements

What are some APA rules for manuscript titles?

1. recommends no more than 12 words

2. should be completely self-explanatory and concise

3. should not include ambiguous terms

4. can be fun interesting, but should eventually explain the study

5. running head cannot exceed 50 characters

What are some general guidelines for abstracts

1. typically 100-150 words

2. contains parts of all sections of the manuscript

3. should stand alone, and thus must explain confusing terms or abbreviations

4. have "Abstract" centered at the top of the page, and start writing flush to the left (no indent)

5. on its own page other than key words if there are any

What are the components of an introduction?

1. to start strong and avoid vague statements

2. end first paragraph with one sentence about purpose

3. cite all factual information

4. don't over-explain the study

5. include personal voice

6. be specific in your hypotheses in terms of your study

What are the components of a method's section?

1. be specific in designs and analyses

2. use your research methods words

3. if using a survey, what kind

4. be thorough but not too thorough

5. provide good rationale for the sample

6. list if there was an incentive given to the participants for participating

What are the types of responses a journal could give you when submitting your paper?

1. accepted

2. revise and resubmit (minor revisions)

3. revise and resubmit (encouraged)

4. revise and resubmit (risky)

5. rejected

6. desk rejection-- where when you send it to the editor they do not send it out for review-- usually means it does not fit the scope of the journal

What are factors?

1. independent variable is synonymous with "factors"

2. this is a variable we often have control over and call the "predictor"

3. a basic experiment will have just one, but some more complex designs can be 2 or 3-factored

What are levels?

1. levels is the breakdown of IV into groups

2. this is the part of the IV we manipulate

3. have IV has at least 2 levels

List some of the different types of studies.

1. experimental

2. quasi-experimental

3. survey/correlational

4. observational/qualitative

Explain the methodology of surveys.

1. collect data on a variety of topics

2. the data can be collected all at once (cross-sectional) and at multiple points in time (longitudinal)

3. variables are not always identifiable as IV or DV, variables don't always have "levels"

4. statistical analysis used: correlation

5. when you bring other variables into the equation that you use a regression

When do you use a bar graph to represent your data?

1. when your x-axis (IV) is nominal (&ordinal)

When do you use a line graph to represent your data?

1. when your x-axis (IV) is interval &ratio

When do you use scatterplots to represent your data?

1. when you want to graph individual data points (someone's score on both the IV & DV) in order to determine the relationship between the two variables (correlations or regressions)

What kind of graph do you use for nominal variables?

1. a bar graph

2. it would be incorrect to use a line graph because we do not have an in-between value

What kind of graph do you use for continuous variables?

1. a line graph because it makes more sense to see how the information unfolds over time

2. a bar graph is alright

What statistic do you use to compare group membership/percentages?

1. a chi square

What is a common graph deception/error?

1. magnifying the graph starting at a higher number than zero

What is the goal of experimental between-subject methodology?

1. to compare the difference in scores on DV between group A and group B

2. the statistic you use: independent samples t-test

What is the goal of non-experimental between-subject methodology?

1. to compare the scores in characteristic A vs. characteristic B

2. it is a non-equivalent design

3. instead of manipulating groups, you can only place people into groups

4. the statistic you use: independent samples t-test

What is the goal of experimental within-subject methodology?

1. to compare how people perform in condition A vs. Condition B (or at time A vs. time B)

2. often called "repeated measures" design

3. the statistic you use: paired-samples t-test

What do single-factor, multi-level designs allow researchers to do?

1. enable researcher to detect nonlinear effects

2. the statistic you use: ANOVA

What is the goal of multiple factors methodology?

1. compare the differences on test score between males and females

2. compare the differences on test score between hot and cold rooms

3. examine the interaction between these two variables on test performance

4. the statistic you use: 2-way ANOVA

What should a successful mediator analyses tell you?

1. show that Y is related to X

2. show that M is related to X

3. show that M is related to Y (even when X is in the equation)

When do you use a moderator regression?

1. when you have a third or interaction variable, but your IV and possible third variable are continuous

What is qualitative research?

1. more open-ended, more narrative, less numbers based

2. can be helpful in applied settings too

What are types of qualitative research?

1. open-ended questions

2. interviews

3. focus groups

4. observations

Explain the differences between quantitative and qualitative research.

1. quantitative research is usually represented in the form of numbers, average scores for different groups on some tasks are found, and typically use large samples

2. qualitative research involves studies that collect detailed and descriptive information, the results are presented as analytical narratives, typically uses small groups, and information is gained through observations or interviews

What are some tips for conducting an interview?

1. create a structured interview with specific questions and follow-up questions

2. mix in "quantitative" type questions especially for demographics

3. thorough training and practice all interviewers to ensure reliable results

4. take copious notes and/or record interview

What are some advantages/uses of qualitative methods?

1. help with item development

2. theory development and hypothesis generation

3. discovery of stressors, coping strategies, syndromes

4. explanations of difficult-to-interpret quantitative findings/intervention results

5. rich descriptions

What are some limitations of qualitative research?

1. participant reactivity

2. over-identify with study participants

3. Glaser-Strauss idea that hypotheses arise from data unsullied by prior expectations

4. cause-effect conclusions?

5. publish-ability

6. whether to quantify or not?

How can you code/report responses within qualitative methodology?

1. narrative vs. quantifying

2. grounded theory-- let the data do the speaking

3. inductive vs. deductive approach

4. create categories-- specific, defined border

5. multiple or singular themes per response

6. interrater reliability

What are some practical purposes behind qualitative research?

1. diagnosing problems

2. creating/choosing solutions

3. complimenting quantitative survey results

4. put interventions into motion

5. give employees a voice

What are the pros and cons of focus groups?

1. pros: more information more quickly, people are bouncing ideas off of one another

2. cons: some people may not feel comfortable, and you are also loosing confidentiality

What are pros and cons of naturalistic observation?

1. pros: can narrow what you are looking at, less reactivity, and more naturalistic setting/real world setting

2. cons: objectivity and it is time consuming

What are some ethical issues with observational research?

1. invasion of privacy

2. lack of informed consent

Informed consent is not necessary in naturalistic observations if:

1. behavior is studied in a public environment

2. people are not interfered with in any way

3. strict confidentiality and anonymity is maintained