Experimental Lecture Final Review

What is Experimental Research?

involves manipulating one or more independent variables and randomly assigning participants to conditions to examine cause-and-effect relationships.

(can determine cause and effect relationships, high control (randomization and control groups), lab and field experiments)

What is Non-Experimental Research?

does not involve manipulation of variables or random assignment; it observes and measures variables as they naturally occur.

(no RA, cannot establish causality directly, low control over confounds, surveys, observational studies and correlational research)

What is the Self-Selection Bias?

In surveys, when the sample is composed of only those who voluntarily choose to respond, the result can be a biased sample.

What is the Nonresponsive Bias?

Occurs in survey research when those who return surveys differ systematically (e.g., in political attitudes) from those who don't respond.

Risky to draw conclusions

Older unmarried male without a lot of edu

Attribute that males survey irrelevant to them

What is the Social Desirability Bias?

A type of response bias in survey research; occurs when people respond to a question by trying to put themselves in a favorable light.

Ensuring anonymity can help reduce, problem is persistent and hard to gauge

Return rates of 85% or higher are considered excellent (and rare), 70-

85% very good, and 60-70% more or less acceptable

What is Observer Bias?

Can occur when preconceived ideas held by the researcher affect the nature of the observations made.

What is the Response/Response acquiescence Bias?

the tendency to agree with statements) by including reverse-scored items

When discussing survey item wording, what do we mean by a question being "favorably" worded?

the wording and structure are designed to encourage responses that are positive or aligned with the desired outcome.

What is a Leading Question?

In a survey, a question asked in such a way that the answer desired by the questioner is clear.

How can the order that items (or scales) appear in the survey bias the results of the survey?

Priming effects occur when earlier questions influence responses to later ones, leading to skewed results.

respondents may show a tendency to select response options at the beginning or end of a list, a phenomenon known as primacy and recency effects

What is a Doubled-Barreled Question?

In a survey, a question or statement that asks or states two different things in a single item.

What is an Interview Survey?

A survey method in which the researcher interviews the participant face to face; allows for more in‐depth surveying (e.g., follow‐up questions and clarifications).

(expensive, comprehensive and highly detailed, reduces problem of unclear questions, representative sample issues, interviewer bias)

What are Mailed Written Surveys?

Plus -> in-person, ease of scoring

Minus -> cost, response rates (nonresponse bias, social desirability bias

What are Phone Surveys?

A survey method in which the researcher asks questions over the phone.

Plus -> cost, efficiency

Minus -> must be brief, response rate, sugging

What is Sugging?

A marketing strategy in which an attempt to sell a product is made by disguising the sales pitch with what appears to be a legitimate survey; the term is short for Selling Under the Guise of a survey.

What are Online Surveys?

Survey research conducted over the Internet; can be a survey sent via e‐mail or posted on a website or social media site.

large amount of data can be collected in a relatively short time for minimal cost.

can be completed in less time than other forms of surveys.

Sampling issues, ethical concerns

What is Participant Reactivity? Problems?

when experimenters changing the group’s behavior.

Use of unobtrusive measures (i.e., measures that participants are not aware of) helps.

Ethics

Consent and privacy issues

APA ethics code allows naturalistic observations in public (not private) places without informed consent.

Potential legal issues?

What are Open-Ended Questions?

A type of question found on surveys that requires a narrative response rather than a yes or no answer.

Elicits wide range of responses

Increase the respondents sense of control

Difficult to score and can add to the time required to complete

Should be used sparingly

Should be used after closed

When would we use Open-Ended over Closed?

Pilot study = identify alts for a subsequent questionnaire to be composed of closed items

can inform options for closed questions

When you need to gather context and understanding.

When you want to explore opinions and perspectives.

When you want to uncover nuanced details and reasoning.

qualitative

What are Close-Ended Questions?

A type of question found on surveys that can be answered yes or a no or by marking a point on a scale.

quick and easy (High response rate)

Reduced Bias and Misinterpretations

statistical analysis

When would we use Closed-Ended over Open?

When you need clear, definitive answers.

When you want to categorize respondents into distinct groups.

When you need to collect demographic information (age,

gender, etc.).

When you need to analyze data statistically.

Specific and Quantifiable Data (Can summarize the means)

Qualities of a Good Survey?

Opt for simplicity over complexity.

Use complete sentences.

Avoid negatively phrased questions and statements.

Use balanced (rather than biased/leading) questions.

Avoid abbreviations, especially ones that participants are not likely to know / may not know.

Avoid slang and colloquial expressions.

Avoid jargon.

Avoid carry-over effects (i.e., effects related to asking certain questions before others).

Start with questions that are not especially personal and both easy to answer and interesting

disguise knowledge questions by prefacing them with such statements as "Using your best guess.. ." or "Have you heard or have you read that..."

What is a DK alternative?

In survey research, when assessing levels of participant knowledge, this is an alternative that means don't know.

Not useful if a lot of DK

What are Positive Correlations?

A relationship between variables X and Y such that a high score for X is associated with a high score for Y and a low score for X is associated with a low score for Y.

What are Negative Correlations?

A relationship between variables X and Y such that a high score for X is associated with a low score for Y and a low score for X is associated with a high score for Y.

What is the Coefficient of Determination?

For two correlated factors, the proportion of variance in one factor that can be attributed to the second factor; found by squaring Pearson's r.

the coefficient will always be a positive number, regardless of whether the correlation is positive or negative.

correlation of +.70 is much stronger than a correlation of +.50.

Strength (Effect Size)? (Cohens d)

.10 for a small effect size, .30 for a medium effect size, and .50 for a large effect size.

Correlations and Regressions...

(by default) only examine linear relationships between variables.

Magnitude of Coefficients?

The magnitude of a coefficient, particularly in correlation or regression, refers to the strength of the relationship between two variables.

A larger magnitude, regardless of direction (positive or negative), indicates a stronger relationship.

What is Observational research?

involve examining something as it happens.

may still require IRB approval

What is Naturalistic Observation?

Descriptive research method in which the behavior of people or animals is studied as it occurs in its everyday natural environment.

does not interact in any substantial way with the group being observed

informed consent not required if public, not extended to private spaces (only debrief if there is deception)

What is Participant Observation?

researcher directly involved with the group being studied and might even become a member of it

Descriptive research method in which the behavior of people is studied as it occurs in its everyday natural environment and the researcher becomes a part of the group being observed.

some sort of consent (esp if interacting or private spaces) Need IRB approval. Sometimes debriefing is required

What is Archival research?

A method in which existing records are examined to test a hypothesis.

often includes independent variables, but because these variables are non‐manipulated (i.e., no random assignment), archival research can be considered non‐experimental research.

may not need informed consent if using publicly available data or de-identified records (rarely need informed consent or debriefing)

When informed consent and debriefing processes are not included in a study, why is that so

waiver of informed consent (minimal risk to participants)

Deception in research (for informed consent but debriefing is a must)

Publicly available/Anon data

Crowley (2001) Study

Naturalistic observation in a science museum

Consent obtained (unusual in observational research)

Event sampling used

Results -> parents (Dads and Moms) explain science concepts more to their sons than to their daughters

boys and girls were not significantly different in whether they initiated engagement (Controlled for child beh)

controlled for parent gender (both mothers/fathers present)

Matched topics/setting

considered SES

What is Thematic Analysis?

A method of identifying and analyzing patterns of responses (or themes) within qualitative data.

what is inductive reasoning (TA)?

researcher is discovering recurring themes emerging from the data; this approach would be an inductive thematic analysis because you are allowing the data to guide the discoveries of themes.

process of reasoning from specific events (data) to develop a theory,

What is Deductive Reasoning (TA)?

researchers doing a more theoretical thematic analysis may predict certain themes prior to data analysis, based on their deductive reasoning rooted in theories about the phenomenon they are studying.

process of deduction is reasoning from theories to predictions specific events.

When would a Thematic Analysis be used?

widely used method in qualitative research that allows researchers to identify patterns within data.

(Non verbal beh, social interactions, contextual influences)

The research question is exploratory.

The data is rich in description (e.g., field notes, video recordings).

The aim is to understand meanings, experiences, or processes rather than quantify behavior.

What is Factor Analysis?

A multivariate analysis in which a large number of variables are intercorrelated; variables that correlate highly with each other form factors.

only identifies factors; what they should be called is left to the researcher's judgment.

It is then determined whether groups of these variables cluster together to form factors.

What is a "factor" in factor analysis?

a hidden or underlying variable that we infer from a set of directly measurable variables

(like customer purchase satisfaction)

What is the benefit of using factors instead of just using the items/scales that are combined to create a factor?

simplify complex data by reducing the number of variables, allowing researchers to capture underlying relationships with fewer dimensions.

also help identify hidden patterns, understand structural relationships, and even guide targeted marketing or product developmen

What is a Factor Analysis used for?

approach to handling a large amount of data, including data from survey research and archival research

What are Factor Loadings?

are correlations between each of the measures and each of the identified factors.

What is Archival Data?

Data initially collected for a purpose not related to a current research study and then used later for a specific purpose in the current research.

Public to private information

What is Big Data?

The vast amount of data available in electronic databases that can be extracted and analyzed with the use of advanced data analytic tools.

The data extracted from archival sources sometimes stand on their own, ready for analysis.

What is Content Analysis?

A procedure used to systematically categorize the content of the behavior (often verbal behavior) being recorded.

analysis normally occurs with verbal materials, that is not always the case.

involves a degree of subjectivity, these procedures also typically include multiple coders and inter‐rater reliability estimates.

Pros and Cons of Archival Research?

amount of info available virtually unlimited

some info vital to a researcher may be missing, or may not be rep of population

experimenter bias (tending more closely to records that support one's hypothesis or interpreting the content of records in a way that is biased by one's expectations.)

potential limitations in controlling for external factors

difficulty establishing causal relationships

What is Meta-Analysis?

where a researcher statistically analyzes the effect sizes from various completed studies on a particular topic.

uses data from pre‐existing sources (that is, completed research studies), it can be considered a form of archival research.

Purpose of Meta-Analysis?

primary goal of a meta‐analysis is to systematically synthesize the scientific literature on some phenomenon and combine the results across studies to better understand the phenomenon.

What are Demographic Variables/Questions?

Data that classifies or identifies individuals (e.g., gender, age, income).

a good idea to put questions about demographic information at the end of a survey.

how can we decide what other (i.e., beyond just the basics) demographic we might want to ask for when planning a study?

should include only demographic categories that are important for the empirical questions that interest you.

When asking about age, ask for date of birth.

When asking for annual income, provide ranges to reduce privacy concerns (e.g., $50,000 or less; $50,001-$100,000).

Don't let the alternatives overlap (e.g., as in the income ranges above).

If making comparisons to other surveys, ask for identical demographic information.

Borrow items from the U.S. Census Bureau

Why is it important to collect at least some basic demographic information in all studies

helps researchers understand the characteristics of their participants, ensuring that their findings are representative of the target population and can be generalized more broadly

What is a Regression Analysis?

In correlational research, knowing the size of a correlation and a value for variable X, it is possible to predict a value for variable Y; this process occurs through a regression analysis.

What is a Regression Line?

Summarizes the points of a scatterplot and provides the means for making predictions.

What is a Criterion Variable?

In a regression analysis, the variable being predicted from the predictor variable (e.g., college grades are predicted from SAT scores)

What is a Predictor Variable?

In a regression analysis, the variable used to predict the criterion variable (e.g., SAT scores are used to predict college grades).

use regression techniques to predict behaviors, based on the correlations between variables.

What is a Bivariate?

A statistical analysis investigating the relationship between two variables.

What is a Multivariate?

A statistical analysis investigating the relationships among more than two variables.

What is a Multiple Regression?

A multivariate analysis that includes a criterion variable and two or more predictor variables; the predictors have different weights.

solves the problem of having more than one predictor of some outcome.

What is the Directionality Problem?

In correlational research, the fact that for a correlation between variables X and Y, it is possible that X is causing Y but it is also possible that Y is causing X; the correlation alone provides no basis for deciding between the two alternatives.

What is Cross-lagged Panel Correlation?

A type of correlational research designed to deal with the directionality problem; if variables X and Y are measured at two different times and if X precedes Y, then X might cause Y but Y cannot cause X.

investigates correlations between variables at several points in time. Hence, it is a type of longitudinal design, adding the causal element of A preceding B.