Experimental Exam 2

Variable

A condition or characteristic that can take different values or categories (ex. gender, reaction time)

Measurement

the assignment of symbols or numbers to something according to a set of rules (ex. gender- male/female)

Four Scales of Measurement

Nominal, Ordinal, Interval, Ratio

Nominal Scale

use of symbols (words, numbers) to classify or categorize (non-quanitative)

Nominal Scale Examples

gender, ethnicity, religion, major in college

Ordinal Scale

rank-order scale of measurement (determine higher/lower)

Ordinal Scale Examples

finishing order in a race, letter grades, SES

Interval Scale

Equal intervals of distance between adjacent numbers (no absolute zero point)

Interval Scale Examples

temperature on Fahrenheit or Celsius scale

Ratio Scale

rank ordering, equal intervals, and an absolute zero point

Ratio Scale Examples

weight, height, number grades, temperature on Kelvin scale, reaction time, length

Reliability

the consistency or stability of the scores

Validity

accuracy of inferences, interpretations, or actions made on the basis of test scores

A measure must be ____ to be ____

reliable, valid

Types of Reliability

Test-retest, Equvialent-forms, Internal consistency, and Interrater

Test-retest reliability

consistency of a group of individuals’ scores on a test over time

Equivalent-forms reliability

consistency of scores on two versions of test (ex. SAT, GRE, IQ)

Internal consistency reliability

consistency with which items on a test measure a single construct (ex. learning, shyness, love, extraversion)

Coefficient alpha (Cronbach’s alpha)

a common index that should be +0.70 or higher

Interrater reliability

degree of agreement between two or more observers (raters)

Construct Validity

involves the measurement of constructs (ex. intelligence, happiness, self-efficacy)

Validation

Gathering of evidence regarding the soundness of inferences made from test scores

Content Validity

judgement by experts of the degree to which items, tasks, or questions on a test adequately represent to construct

Internal Structure

how well individual items relate to the overall test scores or to other items on the test.

Factor Analysis

statistical procedure used to determine the number of dimensions present in a set of items

Homogeneity

degree to which a set of items measures a single construct

Coefficient alpha

larger value = more strongly related

Validity coefficient 

correlation coefficient used in validation research 

Criterion

the standard or benchmark used to correlate with or predict accurately on the basis of your test scores (ex. future performance, already-established test)

Criterion-related validity

degree to which scores predict or relate to a known criterion

Criterion-related validity two types

Predictive validity and Concurrent validity

Predictive validity

using scores obtained at one time to predict the scores on a criterion at a later time (ex. GRE and graduate school GPA, LSAT and law school GPA, MCAT and medical school GPA)

Concurrent validity

degree to which scores obtained at one time correctly relate to the scores on a known criterion obtained at the same time (ex. new depression scale and Beck Depression Inventory)

Convergent validity

extent to which test scores relate to other measures of the same construct

Discriminant validity

extent to which test scores do not relate to other test scores measuring different constructs (ex. happiness and depression, depression and IQ)

Sampling

process of drawing elements from population to form a sample

Equal probability method of selection method (EPSEM)

each individual element has an equal probability of selection into the sample

Statistic

a numerical characteristic of sample data (ex. sample mean, sample standard deviation)

Parameter

a numerical characteristic of population data (ex. population mean, population standard deviation)

Sampling Error

The difference between the value of sample statistic and the value of the population parameter

Two major sampling techniques

Nonrandom sampling and Random sampling

Nonrandom sampling

produce biased samples

Random sampling

preferred when the goal is to generalize, because theu produce representative samples

Types of Random Sampling

Simple, stratified, cluster, systematic

Simple random sampling

Choosing a sample in a manner in which everyone has an equal chance of being selected (EPSEM) (random number generators)

Stratified random sampling

random samples drawn from different groups or strata within the population (mutually exclusive groups)

Proportional stratified sampling

Ensures that each subgroup in the sample is proportional to the subgroups in the population (605 of pop is females; select 60% of sample to be female)

Disproportional stratified sampling

Numbers of people selected from the groups are not proportional to their sizes in the population (ex. 50% of sample is female)

Cluster random sampling

random selection of clusters, rather than indivual units

Cluster

a collective type of unit that includes multiple elements 

Systematic sampling

three step process: 1. Determine the sampling interval (k) (population size divided by desired sample size) 2. Randomly select a number between 1 and k, and include that person in the sample. 3. Select each kth elemtent in the sample

Systematic sampling problem

periodicity- cynical pattern in the sampling frame

Nonrandom Sampling Techniques

Convenience, Quota, Purposive, Snowball

Convenience sampling

using research participants that are readily available (ex. college students)

Quota Sampling

identifying quotas for individual groups and then using convience sampling to select participants within each group (gender- 25 females and 25 males)

Purposive sampling

involves identifying a group of individuals with specific characteristics (ex. college freshmen with ADHD)

Snowball Sampling

technique in which research participants identify other potential participants (parents of children with autism)

Random Selection

selection of participants using a random sampling method

Random Assignment

Placement of participants into experimental conditions on the basis of a chance process (ex. treatment or control group)

Simple Rules for Determing Sample Size

1. Less than 100, use population

2. Larger sample sizes

3. Compare to other research studies

4. Use Sample Size chart

5. Use a sample size calculator

Research Validity

truthfulness of inferences made from a research study

Statistical Conclusion Validity

the validity with which we can infer that the independent and dependent variables covary

Statistical Significane

the observed relationship is probably not due to chance 

Construct Validity

the extent to which we can infer higher-order constructs from the operations we use to represent them (operational definition)

Construct Validity

Research Participants, Independent Variables, Dependent Variables, Experimental Setting

Threats to Construct Validity Participants

Participant reactivity, Participant Effect (naive) (influenced by demand characteristics), Implication of research (positive self-presentation)

Threats to Construct Validity Experimenters

Experimenter effect, Experimenter expectancies, Experimenter attributes (biosocial, psychosocial, situational)

Internal Validity

the correctness of inferences made by researchers about cause and effect

Criteria for identifying a causal relation

cause (IV) must be related to the effect (DV), changes in IV must precede changes in DV, no other plausible explanation must exist for the effect

Threats to Internal Validity

Confounding extraneous variables, Constancy, Equating the groups, History, Maturation, Instrumentation, testing effect, regression artifact, attrition, and selection

Constancy

the influence of an extraneous variable is the same on all of the independent variable groups (eliminate differential influences)

History

any event that can produce the outcome, other than the treatment condition, that occurs during the study before posttest measurement (comparison control group)

Maturation

any physical or mental change that occurs with the passage of time and affects dependent variable scores (ex. age, learning, fatigue, boredom, and hunger) (comparison control group)

Instrumentation

changes from pretetst to posttest in the assessment or measurement of the dependent variable (multiple observers)

Testing Effect

changes in a person’s score on the second administration of a test resulting from having previously taken the test (comparison control group)

Regression artifact

effects that appear to be due to treatment, but are due to regression to the mean (comparison control group)

Regression to the mean

the tendency for extreme scores to be closer to average at posttest

Attrition

loss of participants because they don’t show up or they drop out of the research study

Selection

production of nonequivalent groups because a different selection procedure operates across the groups

Additive and interactive effects

difference between groups is produced by the combined effect of two ore more threats to internal validity

External Validity

degree to which the study results can be generalized to and across other people, settings, treatments, outcomes, and times

Failure to generalize

lack of random selection, chance variation (replication), failure to identify interactive effects of independent variables

Types of External Validity

Population, Ecological, Temporal, Treatment Variation, Outcome

Population Variation

degree to which the study results can generalized to and across the people in the target population

Ecological Validity

the degree to which the results of a study can be generalized across settings or environmental conditions 

Temporal Validity

the degree to which the results can be generalized across time (cyclical, seasonal, circadian)

Treatment Variation Validity

the degree to which the results of a study can be generalized across variations in the treatment

Outcome Validity

the degree to which the results of a study can be generalized across different, but related, dependent variables

Relationship between Internal and External Validity

increase our ability to establish cause and effect tend to decrease our ability to generalize (inverse) External validity is established through replication

Similarity between internal and external validity

emphasis of internal or external validity depends on whether or not a causal relationship has been established

Control of Extraneous Variables

eliminate differential influence and had method of difference

Control Technique at Beginning of Experiment

Radomization

Random Assignment (randomization)

ensure every member has an equal chance of being assigned to any group

Matching

Groups matched to related variables (ex. intelligence, age, gender)

Type of Matching Techniques

holding variables constant, building the extraneous variable into the research design (blocking), Statistical Control, Yoked Control, Matching by Equating Participants

Holding Variables Constant

example- using only females

Blocking

make th extraneous variable another IV in the study

Statistical Control

control of measured extraneous variables during data analysis. 

Yoked Control

matches participants on the basis of the temporal sequence of administering an event (ex. snakc breaks prof vs student choice)

Matching by equating participants

each participant is matched with another participant on selected variables