GRE Psych - Measurement, Methodology, and Other

Reliability

The consistency of a measurement - whether it gives the same result on repeated use.

Validity

The degree to which a test measures what it claims to.

Test-retest reliability

The consistency of results when a test is administered at different points in time.

Inter-rater reliability

The degree of agreement among different raters evaluating the same phenomenon.

Construct validity

Whether a test measures what it claims to measure.

Criterion validity

Examines how well one measure predicts an outcome based on another, established measure.

External validity

Whether the findings of a test generalize to other settings, people, and times.

Type I error

Rejecting a true null hypothesis, concluding there is an effect when there is none; false positive.

Type II error

Failing to reject a false null hypothesis, missing a real effect; false negative.

ANOVA

Compares the means of three or more independent groups at once.

t-test

A test that involves looking at the difference between two means.

Wilhelm Wundt

Founded the first psychology laboratory in 1879

Cross-sectional research

Involves comparing data from multiple different groups at one specific point in time.

Confounding variable

An extraneous variable that correlates with both the dependent and independent variables, potentially leading to a false assumption of causation.

Meta-analysis

Combining data from multiple studies to derive a cumulative understanding of a particular phenomenon or research question.

Null hypothesis

The default claim that there is no effect or difference.

Internal validity

Whether a study can confidently attribute results to the independent variable.

Statistical significance

A result unlikely to be due to chance, conventionally p < 0.05.

Internal consistency

The degree to which a test’s items measure the same construct (e.g., Cronbach’s alpha).

p-value

The probability of obtaining results as extreme as observed if the null hypothesis were true.

Correlation coefficient

A value from -1 to 1 indicating the direction and strength of a relationship.

Positive skew

A distribution with a long right tail; the mean exceeds the median.

Negative skew

A distribution with a long left tail; the mean is below the median.

Social desirability bias

The tendency of survey respondents to answer questions in a manner that will be viewed favorably by others.

Ceiling effect

When a test does not have enough difficult items to adequately measure high ability in participants, leading to many participants scoring at or near the top of the scale.

Quasi-experimental design

Compares pre-existing groups when random assignment to conditions is impossible or unethical.

Floor effect

When a test has a minimum standard score that may not distinguish some test-takers who differ in their responses on the test item content.

Ecological validity

The degree to which the conditions in a study reflect the natural environment in which the behavior is expected to occur.

Convergent validity

The degree to which two different measures of the same construct produce similar results.

What reduces Type II error risk?

Larger effect sizes and larger samples, wich increase power

Interaction effect

When the effect of one independent variable on the outcome differs depending on the level of another independent variable, which is precisely what a factorial design is built to detect.

Main effect

The overall effect of a single factor averaged across the levels of the other.

Factorial design

An experimental setup that involves manipulating two or more indpendent variables simultaneously to observe their individual and interactive effects on a dependent variable.

Theory

A well-supported, organized explanation of phenomena, built from repeated testing of hypotheses and consistent observations.

How can Type I error risks be reduced?

Lowering the significance level (alpha)

Between-subjects design

Uses an experimental group and a control group to compare the effect of the independent variable.

Within-subjects design

Exposes each participant to the treatment and compares their pre-test and post-test results; uses each participant as their own control.

Demand characteristics

Clues participants discover about the intention of the study that alter their responses.

Hawthorne effect

Individuals who are being experimented on behave differently than in their everyday life.

Ex post facto studies

Start by looking at an effect and then attempt to determine the cause.

Nominal scale

Numbers have no meaning except for convenience as labels.

Ordinal scale

Numbers that are used as ranks (e.g. 1st place).

Interval scale

Numbers that have a meaningful difference between them (e.g. 10ºF vs 20ºF).

Ratio scale

Numbers that have a meaningful ratio between them on a scale with a real zero point (e.g. weight and height).

Descriptive statistics

Numbers that summarize a set of research data from a sample.

z-score

Used to summarize how far an individual case is from the mean in standard deviations; allows for comparison between different scales.

Hermann Ebbinghaus

One of the first psychologists to demonstrate that one could study psychological processes using experimental psychology.

Intelligence quotient

Describes the ratio between someone’s chronological and his/her mental age (created by William Stern).

Stratified random sampling

Divides a population into strata sharing a key characteristic, then randomly samples within each stratum.

Matched-subjects design

Participants are paired based on similar levels of a potential confounding factor to eliminate its effect.

Counterbalancing

An experimental technique in which we make sure both the experimental and control group will experience both levels of the independent variable, just at different times.

Nonequivalent group design

When existing groups of participants are compared who were not randomly assigned.

Inferential statistics

Making inferences from a data set that go beyond the actual data points.

T-score

Sets up a curve such that the mean is always 50 and each standard deviation is 10.

Factor analysis

Uses multiple sets of correlations to see which variable correlations cluster together to create a factor or group of variables which are presumed to be measuring the same value, based on their high rates of correlation.

Chi-square tests

Used for data that is categorical, not numerical.

Domain-referenced testing

Determines how much information the test-taker knows about a certain subject.

Norm-referenced testing

A test in which one’s score is compared to that of all of the other test-takers.

Content validity

Whether a test's questions genuinely and fully cover the topic they are meant to measure.

Divergent validity

The extent to which a measure does not correlate strongly with measures of different, unrelated constructs.

Aptitude tests

Predicts future ability with training and growth.

A priori hypothesis

Occurs if one has a predicted hypothesis about a relationship (and the direction of relationship) between variables prior to collecting data.

Law of large numbers

The larger the sample size, the more reliable and valid the findings, assuming there is no significant sampling error.

Effect size

Measures the strength of a relationship or finding, indicating how significant the observed effect is.

Test bias

Occurs when a test systematically disadvantages certain groups.

What does Cohen’s d measure in psychological research?

The size of the effect and the difference between two means in terms of standard deviation.