Research Methods

Define three common biases based on personal experience:

Define three common biases based on personal experience:

Confirmation bias: favor observations that align with personal beliefs Availability heuristic: rely on readily available observations Present/Present bias: rely on most recent information

Define empirical.

Based on observations (direct or indirect)

How is a hypothesis different from a theory?

A hypothesis is a result a researcher predicts to observe from a study (what you think will happen). A theory is a unified explanation of observation (multiple studies that predict larger phenomena)

What are the major steps in the scientific method?

Develop research question and/or initial observations Conduct a research/literature review Develop a hypothesis Run the experiment; collect data Analyze data Report results

Contrast experimental and nonexperimental methods? What are some major features experimental methods have that nonexperimental methods do not?

Experimental: manipulate IVs, control groups, random sampling/assignment, blind/double-blind Non-Experimental: less variable control, correlation, use surveys + polls

Define quantitative and qualitative measurements and contrast their advantages.

Quantitative: measurements in numbers Advantages: easy to analyze statistically Qualitative: data collected in words, and narratives, analyzed for themes Advantages: greater depth, exploration

Define translational research.

Uses basic research results to develop and test applications.

What was the Tuskegee Syphilis Study?

A group of black men was used as test subjects to observe the natural development of syphilis. They were not told of being infected with syphilis (no consent) nor were given treatment to cure it, even though it was readily available.

What is the Belmont report and why is it important to the development of modern ethical rules for research? How was it connected to the Tuskegee Syphilis Study (hint: 1974 National Research Act and the National Commission for the Protection of Human Subjects)?

The Belmont Report summarizes ethical principles and guidelines for research. It's important to the development of modern ethical rules for research due to its highlight of 3 basic principles respect of persons, beneficence, and justice The Tuskegee study led to the national research act which highlights research conduct, regulations of consent, and studies required to be reviewed/approved to ensure they meet ethical standards.

Define the four basic ethical principles:

Beneficence: doing good, being kind, improving well-being Nonmaleficence: avoid doing harm, focus on NOT reducing wellbeing Autonomy (respect for persons, informed consent): person's right to make his/her own decisions; competence; autonomous rights of one person shouldnt infringe on rights of others Justice: obligation to be fair in the distribution of benefits and risks

what is a literature review and why is it an important step in research process?

comprehensive summary of previous research on a topic, which prepares you for making your own argument on that topic, or for conducting your own original research

What is an open access journal?

A refereed/peer reviewed journal that's freely available, digital, online information

What is a predatory journal? What led to their rise?

publications that claim to be legitimate scholarly sources but misrepresent publishing practices Growth of predatory journals include: proliferation of research, rising journal subscription costs, tenure pressure

what is journal impact factor and how is it calculated?

measure of the frequency with which the average article in a journal has been cited in a particular year

# of citations in that year/ published articles cited in Scopus

are open access journals and predatory journals the same thing?

NO! open access journals are peer reviewed while predatory journals are not.

contrast variable and constant

constant: characteristic/condition that does not change variable: inconsistent figure that changes over time, between groups, etc.

what is a dichotomous variable?

a variable that can have only one of two values (typically, 0 or 1) to designate membership in one of two possible categories, such as female versus male or Republican versus Democrat

independent variable

The experimental factor that is manipulated; the variable whose effect is being studied. * Control independent variable*

dependent variable

The outcome factor; the variable that may change in response to manipulations of the independent variable. Observe changes in the dependent variable

central tendency

a number that describes something about the "average" score of a distribution ( mean, median and mode)

When is the median a more useful indicator of central tendency than the mean?

When their are outliers in the group the median is a better representation of the entire group.

descriptive statistics

procedures used to summarize, organize and simplify data, and may be used with either a sample or a population

What type statistics includes central tendency and variability ?

Descriptive statistics

inferential statistics

techniques that allow us to study a sample and then make generalizations about the population from which the sample is drawn ( T-test, Anova and correlation)

What type of statistics compares groups?

Inferential statistics

What are the four scales of measurement?

nominal, ordinal, interval, ratio

Nominal

values that have no relationships or order. Examples would include, social security numbers, and zip codes

Ordinal

used to order a hierarchical series, such as rank order (first, second, third) or percentiles. (has identity and order) Each value is greater or less than another.

Interval

contains an ordered series with equal intervals between values but no absolute zero point. Examples are IQ and SAT scores.

Ratio

series with equal intervals between values and an absolute zero or starting point

Skewness

lack of symmetry on one side. it represents tendency towards high or low scores and may be caused by outliers.

Postive skewed distribution

Most of the scores are bunched towards the left.

negative skewed distribution

Most of the scores are bunched towards the right.

How do outliers affect skewness?

The median may be a more accurate measure of central tendency

Kurtosis

how flat or peaked a normal distribution is

Leptokurtic distribution

Distribution curve is very tall, thin and peaked.

mesokurtic distribution

Having a normal bell shaped curve; a normal distribution of scores

Platykurtic distribution

Flatter and more spread out than a normal curve.

What is the relationship between a distribution's variability and kurtosis (i.e., which curve type has the least/greatest variability?

Platykurtic has a high variability while leptokurtic has a low variability.

simple random sample

every member of the population has a known and equal chance of selection

cluster sampling

a sampling technique in which clusters of participants that represent the population are used ( group of participants, usually location-based)

convience sampling

create a sample by using data from population members that are readily available

snowball sampling

participants are asked to recommend a few acquaintances for the study

internal validity

the degree to which changes in the dependent variable are due to the manipulation of the independent variable

Why is random assignment important for internal validity?

randomly assigning to control and treatment groups ensures observed effects are due to the variable manipulated, enhancing internal validity.

external validity

extent to which we can generalize findings to real-world settings

Why is random sampling important for external validity?

When random sampling has been employed in a study, the unbiasedness of the sampling method is strong evidence for external validity; we have a much higher belief in generalizations to the larger population

What is sampling bias and how does it negatively affect external validity?

• occurs when some members of a population are systematically more likely to be selected in a sample than others

• it limits the generalizability of your findings to a broader group of people.

what is validity?

is an open-ended question, a matter of degree, rather than all-or-none interpretation

responses processes

type of validity evidence; performance or response engaged in by the examinee, including strategies and reasoning

internal structure

type of validity evidence; degree to which test components agree with construct structure (e.g., cognitive ability as unitary vs. multifaceted)

criterion validity

comparison to criteria (e.g., differences in scores of people with and without expressive language)

define reliability

consistency of measurement

Interrater reliability

consistent scores between observers

Test-retest reliability

consistency over time (e.g., 100 people take a test then take it again 6 months later)

What are the four requirements for an experimental design?

variable control, variable manipulation, random sampling, and random assignment

Is a correlation study an experimental design? Can it be used to demonstrate causality?

it's a non-experimental design study and correlations can't confirm causation

confounding variable

extra variable that causes confusion about the explanation for a given difference

operational definition of a construct

what and how you are going to measure a construct (?)

content validity

assesses whether a test is representative of all aspects of the construct ex:

What is participant observation and what is meant by observation "in context"?

direct observation in context; example: an anthropologist goes to live with a tribe in the Amazon rainforest or a sociologist moves into a housing project to learn about poverty.

Interviews vs surveys advantages & disadvantages

Interview advantage: can go into more depth and clarify Interview disadvantage: more time consuming Survey advantage: simple Survey disadvantage: reduced response rates (fatigue)

Contrast interviews and focus groups.

Interviews: structured, semi, or not, allow to observe non-verbal cues Focus Groups: includes 6-12 participants responding to questions with group discussion for ~ 1 hour, led by facilitator, highlights narratives not behaviors

What distinguishes a biographical study from other case studies?

long chronological frame, interview questions tend to be exploratory and give wide range for participant to lead direction of conversation over multiple sessions

What is ethnography and how does it differ from other qualitative designs (hint: what does it focus on)?

investigates complex and contextually sensitive aspects of social life and cultures; it focuses on exploring cultures and societies rather than individuals or specific phenomena

Define lamination in terms of qualitative design.

forming a hypothesis but it changes with more data being collected; refining/changing question as data comes in

What is the hallmark of grounded theory?

systematic coding of data into themes, categories and subcategories

Define these three types of grounded theory coding: Open coding Axial coding Selective/Core coding

open coding: identify broad theme, typically by keywords or frequency of mentions axial coding: identify relationships between themes (casual, temporal, spatial) selective/core coding: use these relationships to identify overarching categories

Define mixed-methods research. What is the requirement for a design to be considered mixed-methods?

combines quantitative and qualitative data collection and analysis in one study; has to have at least one qualitative and one quantitative research component

What is a field or pilot test?

giving test to a small group to check validity and refine for a larger group

Contrast selected-response and constructed-response items, including the advantages of each type.

Selected response items: multiple choice, true/false, ranking, matching Advantages: simple scoring, time efficient, simple administration, greater reliability of content

Constructed response items: short/long write ins, oral discussion, writing samples Advantages: more creative, expansive, authentic samples of behavior

What are double-barreled questions?

questions that ask two questions at once

What is item "keying"?

Define and identify an example of each of the following survey biases: Acquiescence/Dissent bias ("yea/nay-saying") Extreme/Neutral response bias Social desirability Demand bias

Acquiescence/Dissent bias: tendency to agree/disagree with items regardless of meaning Extreme/Neutral response bias: tendency to choose extreme/moderate options, regardless of actual opinion Social desirability: tendency to reply to sentitive questions in socially desirable ways Demand bias: occurs when participant changes their answers to match perceived demands of the research question or agenda

How is test fairness different from test bias?

test bias is inconsistency in scores due to test context, content, structure etc. test fairness is the appropriate use of test scores, which include social/political considerations and represents value judgment