Advanced Research Methods Exam

Flashcards covering slopes, intercepts, ANCOVA, ANOVA, correlation, effect size, factor analysis, p-values, ethical guidelines, research design, and systematic reviews.

Regression

A linear relationship that is expressed as a straight line.

b1

Regression coefficient for the predictor; Gradient (slope) of the regression line; Direction/Strength of Relationship.

b0

Intercept; value of Y when X = 0; point at which the regression line crosses the Y-axis (ordinate).

ANCOVA

Is an ANOVA when we know that an extraneous variable affects the DV too, and adjusts for nuisance.

Reduces error variance

By explaining some of the unexplained variance (SSR), the error variance in the model can be reduced.

Greater experimental control

By controlling known extraneous variables, we gain greater insight into the effect of the predictor variable(s).

Assumptions for ANCOVA

Assumes that a covariate must be related to DV and there is linearity between the covariate and the DV.

ANOVA

Compares several means; can be used when you have manipulated two or more IVs.

Why not use multiple t-tests?

Inflates the Type I error rate.

ANOVA Null Hypothesis

Tests the null hypothesis (are the means the same?).

ANOVA Omnibus Test

It tests for an overall difference between groups but it doesn’t tell us exactly which means differ.

SST

How much variability there is between scores; Total Sum of squares.

SSM

How much of this variability can be explained by the model we fit to the data; Model Sum of Squares.

SSR

How much cannot be explained by our model; Residual Sum of Squares.

How do we conduct follow-up tests?

Planned comparisons (a priori hypothesis driven look only at differences you expected to investigate) and Post Hoc Tests (not planned compare all pairs of means).

Quantitative Methods

Testing theories using numbers.

Qualitative Methods

Testing theories using behavior/language.

Measurement error

Discrepancy between the actual value we’re trying to measure, and the number we use to represent that value.

Non-experimental research

Observing what naturally goes on in the world without directly interfering with it.

Experimental research

One or more variable is systematically manipulated to see their effect on an outcome variable so statements can be made about cause and effect.

Between-group/subjects

Different entities in experimental conditions.

Within-subjects

The same entities take part in all experimental conditions.

Variable Type

Outcome must be continuous and Predictors can be continuous.

Non-Zero Variance

Predictors must not have zero variance.

Linearity

The relationship we model is, in reality, linear.

Independence

All values of the outcome should come from a different person.

No Multicollinearity

Predictors must not be highly correlated with each other.

Homoscedasticity

For each value of the predictors the variance of the error term should be constant.

Independent Errors

For any pair of observations, the error terms should be uncorrelated.

Multicollinearity

Multicollinearity exists if predictors are highly correlated Checked with collinearity diagnostics VIF < 10 & Tolerance > 0.2.

Standardised Residuals

In an average sample, 95% of standardized residuals should lie between 2 and 99% of standardized residuals should lie between 2.5. Outliers - Any case for which the absolute value of the standardized residual is 3 or more, is likely to be an outlier.

Cook’s Distance

Measures the influence of a single case on the model as a whole, and ensure there is no case with a value greater than 1.

Screening for Multivariate Outliers

Cases that are extreme on multiple variables requiring Mahalanobis Distance needing too consult Chi Square significance table (df = number of predictors).

Introduction to Cochrane

Advocated RCTs to inform healthcare practice Cochrane Reviews (>10,000) published and registered Identify, appraise and synthesise research-based evidence and present it in accessible format.

Introduction to the Campbell Collaboration

Applies the Systematic Review approach in non-medical fields Primarily in the effects of social interventions.

Introduction to EQUATOR

Detailed reporting guidelines and frameworks for different types of studies.

How might you conduct a systematic review?

Define a specific question, search the literature, assess the studies, extract relevant data from each paper/combine the results, and put the findings in context.

Correlation

Measuring the extent to which two variables are related and measures the pattern of responses across variables.

Correlation

Assesses the linear relationship between continuous variables and requires assumptions of linearity, normality, continuous variables, and homoscedasticity).

Non-parametric tests for correlation

Non-parametric tests are procedures that don’t rely on restrictive assumptions like Spearman’s Rho and Kendall’s Tau.

CORRELATION & CAUSALITY

Correlation coefficients say nothing about which variable causes the other to change because of tertium quid.

Partial correlation

Measures the relationship between two variables, controlling for the effect that a third variable has on them both.

Semi-partial correlation

Measures the relationship between two variables controlling for the effect that a third variable has on only one of the others.

MEASURING EFFECT SIZE

Reports the magnitude and tells how much of a relationship there is.

Positive correlation

As the value of one variable increases, the value of the other variable also increases.

Negative correlation

As the value of one variable increases, the value of the other variable decreases.

Coefficient of determination (r2)

By squaring the value of r you get the amount of variance in one variable that is shared by the other.

Initial Considerations for Factor Analysis

Within one broad construct, the test variables should correlate quite well, r > .3, but avoid Multicollinearity where several variables highly correlated, r > .8.

Common variance

Variance that a variable shares with other variables.

Unique Variance

Variance that is unique to a particular variable.

Communality

The proportion of common variance in a variable.

Principal Components Analysis

Assume all variance is shared

Factor Analysis

Estimate Communality Use Squared Multiple Correlation (SMC).

Kaiser’s Extraction

Retain factors with Eigen values > 1.

Scree Plot

Use ‘point of inflexion’ of the scree plot.

Rotation

Maximise the loading of a variable on one factor while minimising its loading on all other factors known as Factor Rotation.

Orthogonal

Are unrelated, i.e., not correlated.

Oblique

Are considered to be related, which is far more common in Psychology research.

Exploratory

“What is there?

Confirmatory

“Does the data I have collected match the theory?

What are ‘factors’ and ‘models’?

An IV. Factorial = when you use 2+ IVs (i.e. “factors”). Model = the relationship between our variables of interest .

DESIGNS WITH 2+ IV

Often in real life the story is a little more complicated than X < ->Y or X ->Y and often there is more than one IV that affects the DV .

P value

Measure of statistical evidence that appears in virtually all medical research papers, but is not part of any formal system of statistical inference.

Misconception #9 - P > .05 means if you reject the null hypothesis, the probability of a type I error is only 5%

Equivalent to Misconception #1. A type I error is a “false positive,” a conclusion that there is a difference when no difference exists.

What is ethical?

Ethics is about protecting others, minimising harm and increasing the sum of good ensuring trust and integrity and value and not thinking that we have the inalienable right to conduct research involving other people.

Should we use the results of unethical research?

Remember that there is no such thing as a safe null position that doing nothing is a decision in itself and will have its own consequences.

Main effect

The effect of an IV on a DV (averaging out the levels of all other IVs).

Interaction

If two IVs interact, the relationship between each IV and the DV varies depending on the value of the other IV.

What is a Meta-Analysis?

The use of statistical methods to summarize the results of independent studies with the additional criterion is to ensure included studies can be directly compared, removing the reliance on any one paper whose effect may be an outlier.

Four different ways of “knowing”

Observation, logic, intuition, and authority.

The psychologist

Both scientist and a practitioner.

MIXED ANOVA DESIGNS

Different levels of 1+ IV experienced by different entities and Different levels of 1+ IV experienced by same entities.

Sampling

Size is important for generalisability, but it’s not the only factor!

Sampling: Size matters

All null hypotheses can be rejected given a large enough sample!

Sampling

The most important reason that size matters

Non-population based sampling

What can we tell about human behaviour from studying: Western, Educated, Industrialized, Rich, Democratic & WEIRD people!

Hierarchical Regression

Known predictors (based on past research) are entered into the regression model first, then new predictors are then entered in a separate step/block Experimenter makes the decisions.

Forced Entry Regression

All variables are entered into the model simultaneously and the results obtained depend on the variables entered into the model.

Stepwise Regression

Variables are entered into the model based on mathematical criteria.

Observational design

Researcher looks at associations between variables and does not manipulate a variable.

Experimental design

Researcher manipulates one or more variables to examine their effect on some other variable(s).

Establishing Causation

Essential factors: Co-variation, Temporal order & Ruling out alternative explanations for covariation requiring a population-level data or an experimental design!

Cochrane

Protocol should be published before the actual results of your review!

Codes of ethics

Nuremberg Code, Declaration of Helsinki, & Belmont Report Informed the current codes Australian Code for the Responsible Conduct of Research, The National Statement on Ethical Conduct in Human Research & Australian Psychological Society Code of Ethics.

What is ethical?

There are no such things as safe null positions that doing nothing is a decision in itself and will have its own consequences!

Narrative Reviews

Summaries key evidence according to them based on research agendas.

Systematic Reviews:

Agenda-agnostic with an objective methodology providing equal weighting to perspectives collating all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question.

t-tests

Tests the significance of the difference between means with for Dependent or “paired-samples” or Independent-samples.

Belmont Report: Respect

The welfare, beliefs, perceptions, customs, cultural heritage, privacy, confidentiality and cultural sensitivities of participants and the right of participants to make their own decisions is respected.

Belmont Report: Beneficence

That Researchers are responsible for the welfare of participants so Research should designs research to minimise and manage risk.

Belmont Report: Justice

Everyone has an opportunity to take part, participants are not exploited, and fair distribution of the burdens or benefits of research.

Multiple Regression

A natural extension of the simple regression model to include multiple predictors.

Intercept

The intercept is the value of the Y variable when all Xs = 0 and is the point at which the regression plane crosses the Y-axis.

Sums of Squares for our Model

Looks for “how different is our line from the Outcome Mean?

Between-groups

Each person can only experience one level of the independent variable and Dependent variable may be measured only once.

Within-groups

Each person must experience both (or all) levels of the independent variable Dependent variable must be measured at least twice.

Testing the model through ANOVA

When you run a regression you will get an ANOVA output

So did we explain more with our model?

That our model better than using the Mean to say… If the model results in better prediction than using the mean, then we expect SSM to be much greater than SSR

r2

is… The proportion of variance accounted for by the regression model.

Reliability Inter-rater

Measures degree to which different judges independently agree upon a ‘subjective’ observation Internal Measures degree to which all the specific items/observations in a multiple-item measure behave in the same way