Research Methods Test
CHAPTER 1:
Why should you care about research methods? Name at least four reasons. 1. Knowledge of research methods helps you be a better consumer of science/research, especially when research findings are reported in the media
Name four cognitive biases and give an example of each.
What are the three goals of science? Description: what happened, Explanation: why did the results happen?, and prediction: what do we predict would happen if?
Description: how many drinks do college students drink in a typical week?
Explanation: why do some college students drink more than 5 drinks in a row? (positive outcome expectancies, peer pressure, coping)
Prediction/causality: if X happens, does X lead to Y?, change in X leads to change in Y, does treatment of depression lead to a reduction in drinking?
Name at least three aspects of the scientific method, and briefly describe each.
Anchoring - a heuristic that leads individuals to use a particular value as a base for estimating an unknown quantity and adjust their estimate based on that quantity, even if the value given is entirely arbitrary.
Decision fatigue - occurs when an individual's ability to make a rational decision is negatively impacted after making many decisions.
Discounting base-rate information - a cognitive bias that leads individuals to favor anecdotal evidence over more detailed information that is available.
Framing effect - a cognitive bias caused by seemingly inconsequential differences in wording in a question or problem that lead respondents to vary their choices
Mood effect - influence on decision making that occurs because of either a positive or a negative mood state
Peer-review journals - scholarly journals whose editors send any submitted article out to be evaluated by knowledgeable researchers or scholars in the same field
Reliability - the idea that an investigation or measurement tool should yield consistent findings if researchers use the same procedures and methods repeatedly.
Replication - The process through which either the original researcher or the researchers from an independent laboratory repeat the investigation and obtain the same or highly similar results.
CHAPTER 2:
What are the two key aspects of ethical research?
Treating participants ethically (protection of human subjects)
Acting ethically as a psychological researcher (responsible conduct of research)
What does treating participants ethically involve (name three points)
Informing participants about what they will do
Not harming
Not coercing
Maintaining confidentiality
Belmont report - developed in 1976, it is a series of basic ethical principles to guide researchers as they perform studies with human subjects
Somewhat incited as a result of the Tuskegee syphilis study
A group of ethicists, physicians, scientists, and others met at the Belmont Conference Center in Maryland in 1976
Outlines 3 general principles: respect for persons, beneficence, justice
Ethics - conduct in research that reflects the social responsibility that the researcher has toward society at large.
Informed consent - agreement of a participant to take part in a study, having been made aware of the potential risks.
An agreement between the subject and the investigator based upon the subject’s full comprehension of all relevant information to voluntarily participate in a study
Consent is a process, not just a form!
Begins with recruitment and goes on to the end of the subject’s participation in the research
Institutional review board - a committee that is active in almost every U.S. hospital, university, and organization that supports research. These boards are responsible for protecting the health and wellbeing of research participants, and they are liable for the research they approve.
Their goal is to oversee research at their institution to protect the well-nbeing and health of human subjects
CHAPTER 4
Know the makeup of the sections of the scientific article and what information each section presents
Title - represents the most important message of the research for the vast majority of readers
Accurately convey the main topics of the study, be concise, attract readers
Abstract - Abstracts provide a brief summary of what the article is about
It helps you screen an article: who were the subjects, what was the goal, what did the study find
Introduction/literature review - the introduction presents the research question (background)
The introduction motivates why the research question is interesting (often in the context of past research)
The introduction will present research hypotheses and what the authors were expecting
Theory - foundation of basic research in psychology
It is used to provide a model for understanding human thoughts, emotions, and behaviors
Two components: 1. It must describe behavior 2. It must make predictions about future behaviors
Theories can lead to biases
Methods - the methods describes how the research was done
Who participated?
How did the experimenter present the stimuli
What did the participants do
What were the measures of the study variables
What was the design of the research
Results - the heart of the research article
Presents the finding w o the researchers interpretation
Presents the statistics (if any) used to analyze the data
Discussion - this section is the authors interpretation of their findings
Authors often present their data in context
Implications, limitations, and suggestions for future directions are made
References - you will find useful sources to read to further formulate your own research
This section is critical to achieve one of the goals of the scientific method, which is to build on existing principles and past knowledge
CHAPTER 5:
Give an example of a testable, falsifiable hypothesis - questions that cannot be answered by direct observation or by evidence gathered through experimental inquires are not tesable questions
Specific and clear
Example: is secure attachment better than insecure attachment
Do individuals who had a secure attachment to their mothers have higher marriage satisfaction?
Researchers can test whether the hypothesis or claim can be proven wrong
Do individuals who had a secure attachment to their mothers have higher marriage satisfaction?
People with secure attachment to their mothers would have higher marriage satisfaction
Children are more likely to display secure attachment styles when both mothers and fathers display secure attachment styles.
Give an example of an unfalsifiable hypothesis - Botticelli’s Birth of Venus is beautiful. There is no experiment that might show this statement to be false. To test a hypothesis, a researcher will conduct one or more experiments designed to eliminate one or more of the hypotheses.
What does it mean to operationalize a variable? Give an example. - Operationalization is used to convert abstract concepts into observable and measurable traits. For example, the concept of social anxiety is virtually impossible to measure directly, but you can operationalize it in different ways. Using a social anxiety scale to self-rate scores is one such way. How are you going to measure the variables
Define and give an example of a causal and correlational design - smoking causes an increase in the risk of developing lung cancer or smoking is correlated with alcoholism, but it does not cause alcoholism
Describe an experimental and a non-experimental method, and give an example of each.
An experimental design is carefully controllers studies in which scientists manipulate certain elements to evaluate outcomes of interest and non-experimental designs is A group of research approaches that do not attempt to manipulate or control the environment, but rather involve the researcher using a systematic technique to examine what is already occurring
Name one advantage and one disadvantage of an experimental and a non-experimental method
Non experimental does not allow for causal claims
However, it has higher validity - as variables are measured as they occur naturally without manipulation
Often they avoid ethical dilemmas of manipulating IVs since some manipulations cannot be done ethically
Experimental design manipulates
Define a independent variable (IV) and a dependent variable (DV), and provide an example of a study with an IV and a DV (identify which is which) - Manipulated/predicting variables are Independent variables and outcome variables are dependent variables. Whether teacher expectation impacts the changes in student GPA. IV: teacher expectations DV: change in student GPA
What is the key component that lets scientists make causal claims in their experimental research?
Construct validity - a measure of the extent to which a particular variable or measure actually captures what is meant to capture. (are we studying what we think we are studying?)
Control group - in an experimental design, the set of participants who do not receive the experimental treatment. This group is compared with the experimental group
Convergent validity - The degree to which two assessments designed to measure the same construct or behavior actually do measure the same thing. Example: is intelligence as measured by the Stanford- Binet intelligence scale highly Correlated with intelligence as measured by the Wechsler adult intelligence scale?
Dependent variable - The response that is measured by the experimenter that is thought to be related to levels of the independent variable
Discriminant validity- The degree to which two assessments designed to measure different constructs or behaviors are in fact measuring different things. Example: Does a high school achievement test measure something different than the stanford-binet Intelligence scale or is performance on both assessments highly correlated?
Experimental group - In an experimental design, the set of participants that receives a special treatment with the goal of determining whether the treatment impacts the outcome.
External validity - A measure of the degree to which the conclusions drawn from a particular set of results can be generalized to other samples or situations. Example: do the results obtained in a study of the efficacy of baby einstein videos to boost intelligence in middle class infants generalize to children from different classes and cultures?
Falsifiable- The concept that researchers can test whether the hypothesis or claim can be proven wrong
Independent variable- A factor that is systematically varied in an experiment
Internal consistency- The degree to which items within an assessment are measures of the same thing. a measure of how much the scores on items within an assessment yield the same values- to what extent are items within the assessment correlated with each other. Example: you design your infant IQ measure with multiple items that assess “numerosity” ( having a general sense of a number, even at a young age). if you have high internal consistency, then one item that reflects a greater sense of numerosity will yield similar values as a second item that also reflects a greater sense of numerosity.
Internal reliability -
Internal validity - The degree to which a particular variable is actually the cause of a particular outcome. Example: does watching Baby Einstein videos boost intelligence? or do the parents who play these videos for their children do other things that are actually the cause of any increases in intellectual performance?
Nonexperimental methods - A group of research approaches that do not attempt to manipulate or control the environment, but rather involve the researcher using a systematic technique to examine what is already occurring.
Operationalizing - The process by which a researcher strives to Define variables by putting them in measurable terms.
Predictive validity - The degree to which an assessment correlates with a future measure of a construct or behavior. Example: is IQ measured in infancy highly correlated with measures of intelligence assessed in later childhood and even adulthood?
Qualitative research - any systematic study that yields numerically measured data
Quantitative methods are usually used to examine the relationship between variables with the primary goal being to analyze and represent that relationship mathematically through statistical analysis
The data collected is numeric or can be coded numerically
Statistical analysis allows for greater objectivity when reviewing results and therefore, results are independent of the researcher.
The results of quantitative analysis are most commonly reported in the form of statistical tables or graphs.
Quantitative research - emphasizes meaning
Qualitative researchers are generally interested in participants subjective experience; they try to understand the way participants experience some event and what the experience means to them.
Does not deal w variables
Data are words, texts and stories
Does not deal with actual causal relationships, only participants' perspectives on cause and effect
Does not “prove” anything (hypothesis testing)
Random assignment - The assignment of subjects to different conditions in an experiment by methods that rely on chance and probability so that potential biases related to assignment to conditions are removed.
Reliability - This concept refers to the extent to which you can repeat your measurements and/or methods and obtain the same or highly similar results.
Split-half reliability - Within a test, this is a measure of internal consistency where the scores on half the items on an assessment are correlated with the scores on the other half of the assessment. Example: in an IQ test made up of 100 items, to what extent do the scores obtained on a random selection of 50 items correlate with those obtained on the remaining 50 items?
Testable hypothesis - A claim that makes a specific prediction that can be supported or refuted through the collection of relevant data or information
Test-retest reliability - When measures given more than once obtain the same or highly similar results. a measure of the consistency of results obtained using the same assessment tool on multiple occasions. Example: if you measure an infant's IQ today, How likely are you to obtain the same score tomorrow or at some later date using the same test? typically measured by calculating the correlation between the two different scores measured at two different times. researchers must be careful to consider effects that arise from repeated testing. taking the same test twice can lead individuals to change their responses because they have learned something or because their views have changed.
Validity - Overall, this concept refers to the idea that your measurements and methodology allow you to capture what you think you are trying to measure or study. there are a number of different types of validity. Replicating findings across different methodologies would demonstrate both the internal and external validity of a study
CHAPTER 6
Convenience sampling - the researcher draws on the most readily available group of participants
Interval scale- contains properties of ordinal scales and has equal intervals between reponses (temperature)
Can attachment style predict disordered drinking in college students? (number of drinking 0,1,2,3,4….
Temperatures, IQ scores, many scales in behavioral research, likert items
Likert-type ratings- (1) Strongly Disagree; (2) Disagree; (3) Neither Agree nor Disagree; (4)Agree; (5) Strongly Agree
Nominal scale - responses are unordered categories
May be coded by number but the number magnitude is irrelevant (male-1, female=2)
Can attachment style predict alcohol use in college students? (yes/ no)
Political party affiliation (democrat, republican etc)
Sex (male/female)
Religious affirmation ( christian, jewish etc )
Ordinal scale - responses are ordered or a ranking (the distance between rankings/ order is unknown)
Socioeconomic status (low, middle, high income)
Education level (highschool, BS, MS, PhD)
Can attachment style predict alcohol use in college students?
(no drinking, 1-4 drinks, more than 4 drinks)
Likert ranking (strongly disagree to strongly agree)
I like x most and y second
Oversampling - intentionally over-recruit underrepresented groups into your sample to ensure that you have enough representation of those groups
Population - the entire group of individuals relevant to your research
Random sampling - each member of the population has an equal chance of being selected
Ratio scale - a scale sharing all properties of an interval scale (rank order information and equal interval), but also containing a meaningful absolute zero point, where zero represents the absence of the thing that is measured. (time, speed, length, reaction time, temp in degrees K where theres an absolute zero, amount of money earned)
Representative sample - most researchers try to gather a representative sample: one that shares the essential characteristics of the population which it was drawn
Sample - a subset of individuals drawn from the population of interest
Snowball sampling - the researcher ask participants to help recruit additional participants through referral
Statistical significance - An indicator of the probability of obtaining an effect size as large as or larger than the one you obtained.
Stratified random sampling - divide a population into homogeneous groups along some dimensions and randomly sample within each of the subgroups
CHAPTER 7
Name at least three advantages and three disadvantages of surveys
Advantages
Efficient and economical
Easy to obtain large samples
Short time frame
Allure of anonymity: Sexual behavior, illicit drug or alcohol use, attitudes about race orgender, religious beliefs
Flexibility
Disadvantages
Possible biases (selection bias - who are the participants)
Online surveys - miss people with no online access (Low SES, older people)
Participation bias - who choses to complete the surveys
Non response bias - lack of time
Self selection bias etc
What are three strategies to detect threats to validity in surveys?
You can never be sure that respondents are honest and accurate
Respondents might not understand the question
Respondents might have agendas
Respondents might be careless
Collect IP addresses
Pre-screen participants
Add similar or same questions within the survey
Attrition - the loss of research participants prior to completion of the study (dropping out of study)
Careless responding - lack of careful attention to one's own responses in a survey bc of disinterest or the desire to complete the survey as fast as possible
Closed-ended response item - a survey item/question that requires respondents to choose from a number of predetermined responses
Compound question - question that attempts to ask about more than one issue in the same question (double barreled question)
Double-barreled question - a question that refers to more than one issue but requires or expects only one response. “Do you think your introductory psych class was interesting, and was the instructor engaging?” or “Do you like the president of the United States, and do you think the president is doing a good job?”• “Do you like the president of the United States?” “Do you think the president is doing a good job?
Experimenter bias - in survey/ interview research, bias introduced by the way in which experimenters ask questions (related to framing) - construction of survey questions (“it should be legal for doctors to give terminally ill patients the means to end their lives” or “It should be legal for doctors to assist terminally ill patients in committing suicide”)
Fatigue effects - for longer surveys! Negative effects on survey responses or completion resulting from subjects tiring of the survey
Internal consistency - a survey is internally consistent to the extent that items intended to measure the same variable yield similar responses
Interview - a data collection technique in which researchers ask participants questions orally, either in person or on the phone.
Loaded question - a question that presupposes information or potentially biases a response by including an assumption that may not be justified. “Have you stopped beating your wife?”
Midpoint - the center value of a scale with an odd number of response possibilities. An example of a midpoint on a scale of agreement is “neither agree nor disagree”
Motivated response bias - for some topics. Bias that occurs in situations where people are highly motivated to complete a survey for the purpose of affecting public opinion
Nonresponse bias - bias that occurs when individuals who are contacted and choose to complete a survey differ in some ways from others who are contacted but chose not to participate. Lack of time
Parallel-forms reliability - a measure of survey reliability that examines the consistency of responses of respondents across two versions of a survey, with items in both surveys having been designed to probe the same variables. (ex: SAT and ACT) not everyone takes the exact same version of tests
Participation biases - who choses to complete the surveys?
Pilot study - a “pre study” conducted before your actual study to assess your survey/interview instrument or test a methodology
Selection bias - bias resulting from a selected sample not being representative of the population of interest. This may come about because of under or oversampling of particular types of respondents, respondent self-selection, or respondent nonresponse
Social desirability bias - evaluation apprehension. Effects arise when you question participants about behaviors or attitudes that might not be viewed as acceptable in society
Split half reliability - a measure of internal consistency assessed by dividing survey items intended to measure the same variable into two sets and examining how well responses to those two sets of items agree with one another
test-retest reliability - a measure of survey reliability that examines the consistency of responses of respondents who complete the same survey on two separate occasions (asking participants to complete a survey then do it again later - respondents may remember how they answered the first time leading to overestimates of reliability)
CHAPTER 8:
Control group - participants who serve as a direct comparison for the experimental group and receive either an inert version or no treatment at all
Dependent variable - the factor of interest being measured
Experimental group - participants who receive the intervention or treatment for example, a dose of the independent variable
External validity - External validity is the degree to which conclusions drawn from a particular set of results can be generalized to other samples and situations
• This is often seen as a challenge for experimental work
• Because the experimental situation may not resemble the real-world context that it is designed to model because of its artificiality (too many factors controlled)
IV - systematically manipulated by researchers. All other factors held costar to see if the dependent variable changes in response to manipulation of the independent variable
Internal validity - a design that has high internal validity allows you to conclude that a particular variable is the direct cause of a particular outcome
A well-designed experimental method has a high level of internal validity
Quasi-independent variable - are existing characteristics of the participant that are not manipulated by the experimenter (sex,race)
Random assignment - ensures that there are no systematic difference between the participants in each group
Any differences between participants in each group are due to chance
Isolates variables of interest - the only difference across conditions is the variable of interest
Stereotype threat - is a phenomenon where activating a stereotype leads individuals to change their behavior reflecting the stereotype