CJ Research Methods Test 2

Operationalization

When we connect concepts to empirical observations by identifying specific variables and indicators

What is measurement validity?

Are we measuring what we think we are measuring? 4 types.

Facial validity

If the measure obviously pertains to the concept of interest, it is facially valid. ex: number of drinks consumed last week as measure of alcohol use

Content validity

If the measure covers the full range of the concept’s meaning, then it is content-valid. ex:frequency is only one dimension of alcohol use. what about type of alcohol consumed, sex, body weight, etc.?

Criterion validity

If the measure can be accurately compared with a separate and more accurate/already accepted measure. ex: could compare survey responses to BAC measures during the period indicated by the response.

This is the most difficult to achieve. Another example is LSAT and law school success - predictive measure

Construct validity

If there is no measure against which to compare the current metric, researchers will instead strive for construct validity - which is when the metric is related to a variety of other measures laid out in research or theory. ex: judicial ideology and voting behavior.

Reliability

achieved when a measure yields consistent observations on different occasions. Prerequisite for validity. Various methods of assessment.

Test-Retest Reliability

same result from a second, later experiment?

Interitem Reliability (internal consistency)

use multiple items to measure single concept, i.e., alcohol use being tested by asking people how many drinks they’ve had along with how many nights they had those drinks

Alternate forms reliability

compare results from slightly different tests, such as when a survey questions answers are shuffled around or worded in different ways

Interobserver reliability

multiple observers for one test.

Intraobserver reliability

same as test-retest, but observer is providing the ratings to be assessed rather than the observed.

Nominal Measurement

qualitative and without a mathematical interpretation. ex: self-reported race

Ordinal

specify order and permit greater-than or less-than distinctions. ex: Strongly Disagree thru strongly agree measuring level or agreement.

Interval

More rare. Fixed interval between quantitative units, but the zero is not meaningful. ex: a date on a calendar

Ratio

Fixed interval quantitative units with a meaningful zero. ZERO MEANS NO AMOUNT OF WHAT THE VARIABLE REPRESENTS.

Collapse

when you take data which permits a high level of measurement and report using a lower level of measurement. For instance, age reported in years can be collapsed from a ratio level into the ordinal levels of young, middle aged, and old.

Dichotomous

can be nominal but can also indicate the presence or absence of an attribute. Two mutually exclusive options, i.e., agree or disagree.

Attributes must pass what two criteria

Mutually exclusive: attributes may not overlap

Exhaustive: all cases must be covered by the attributes

ex: is the person married or non-married? no room for both at once, and no escaping classification with an attribute.

Mode

The most frequent value in a distribution. Distributions can have no mode (equiprobable), one mode (unimodal), or two or more (bimodal/trimodal/)

theoretically available to all levels of measurement

Median

The category which holds the 50th percentile observation, or the 50th percentile observation itself. Good for everything but nominal. Locate by adding cumulative percent until it reaches past 50.

Mean

The arithmetic average of all scores in a distribution. good for interval and ratio

Range

simplest measure of variation. reports the minimum and maximum values within a distribution. good for ordinal, interval, and ratio.

Interquartile range

difference between the scores at the first and third quartiles. eliminates fluctuations generated by outliers. Locate first quartile (25th percentile) and third quartile (75th percentile) in the same way as the median. Good for everything but nominal.

variance

the average square deviation of each case from the mean. ratio and interval

standard deviation

square root of variance. often the preferred measure of variability. ratio and interval

population

what you are interested in generalizing back to using a sample

Sample

what you are using in a study to approximate a population of interest

elements

what make up the sample

sampling error

the difference between the characteristics of the sample and the characteristics of the population

representative sample

when the sample’s characteristics match the characteristics of the population. more likely when sample is large and population is homogenous.

sampling frame

list of all elements (e.g. adults) from which sample is drawn

enumeration units

if no easy sampling frame can be acquired, then use enumeration units, i.e., households instead of adults

probability sampling

researcher knows the likelihood that any element will be selected from the frame

must not be zero for any element: if so, biased.

Nonprobability sampling

unkown likelihood. i.e., standing outside of the library and asking people to fill out a survey

useful when there can be no sample frame, exploring a research question that does not concern/need to be generalized to a large population, or when conducting a preliminary or exploratory study.

How do you assess sample quality?

from what population were cases selected? What method was used in selection? do the cases represent the population well?

probability type: simple random sampling

requires procedure to generate random numbers or identify cases strictly on the basis of chance - random number table, random digit dialing, replacement sampling

systematic sampling

less time consuming than simple random sampling. first element selected randomly, then every nth element. this interval is found by grabbing the total number of cases in sampling frame divided by the number of cases required for the sample. convenient

very good for representative samples unless there is periodicity. stratification may be necessary.

periodicity

when the sequence varies in a regular, periodic pattern. for instance, a systematic sample may be biased if it ends up selected a high amount of december dates.

stratified random sampling

ensure representative sample by randomly selecting within specified characteristics of the sample frame. may sample proportionately or disproportionately

availability sampling (haphazard, accidental, convenience)

e.g., waiting outside of tate

quota sampling

designating sample population into proportions reflecting general population

purposive sampling

each element selected for a purpose

snowball

select one element, ask him or her or it to identify others

what are the two types of causal explanations

nomothetic - ceteris paribus, a variation in the independent variable will be followed by variation in the dependent variable

iodiographic - concrete cause has a particular effect on a particular individual.

while a nomothetic explanation of a car crash might be that, on average, the presence of open containers in cars is followed by a increase in the rate of wrecks, while the idiographic explanation is that bobby joe had one too many at the local bar and stole somebody’s keys.

what are the five criteria for establishing causation

empiricial association - variation in one is empirically associated with variation in another

time order - variation in independent variable must come before variation in the dependent variable

nonspuriousness - the relationship must not be due to the interference of a third variable

mechanism/intervening variable - identify the process by which the independent variable effects the dependent variable. e.g., drunk driving is associated with increased vehicle wrecks because of the loss of faculties, which is also an associated effect with other types of drugs, sleep deprivation, and mental handicaps. nonetheless, that doesn’t change that the presence of alcohol causes and increase in car wrecks.

context - specifies the conditions under which the hypothesized relationship “holds”

ecological fallacy

when a researcher uses group data to form a conclusion about an individual

reductionism

when a researcher uses individual data to form a conclusion about a group