Methods of Social Research Test 2

the measurement process

-many options for opreationalizing

-can be based on activities as diverse as: asking people questions, reading judicial opinions, observing social interactions, coding words in boks, checking census data, drawing blood samples

-goes from general topic (conceptualization) set of concepts (operationalization) variables (measurement) data

most common operation for measuring social variables

-asking people questions

-usually single questions

-public opinion polls are usually based on answers to single questions

-single questions can be designed with or without explicit response choices

closed-ended (fixed choice) questions

Survey questions providing preformulated response choices for the respondent to circle or check.

open-ended questions

questions that allow respondents to answer however they want

unobtrusive measures

collecting data about individuals/groups without them letting know that they are studies (no knowledge or participation)

-can also be created from existing bedia or archives

triangulation

-combining measures (2 or more) for the same variable

-using available data, asking questions, making observations, and using unobtrusive indicators are interrelated measurement tools

-can strengthen measurement considerably

-when archive similar results with different measures of the same variable, more confident in the validity of each measure

measurement

the process of linking abstract concepts to empirical indicants

-the process of assigning numbers or labels to units of analysis in order to represent conceptual properties

variable

any characteristic of a unit of analysis that tales on different values, categories, or attributes for different observations

-a logical grouping of attributes or characteristics

levels of measurement

-the mathematical precision with which the values of a variable can be expressed

-generated by a variable

-levels: nominal (qualitative- categorical), ordinal, interval, ratio (last 3 are quantitative)

Nominal Level

data that is classified into categories and cannot be arranged in any particular order

-attributes of nominal variable provide classification, but convey no information about order

-number the categories correspond to only acts as a label

-qualitative and categorical (no mathematical meaning)

-ex. state of residence, marital status, occupation, religious affiliation, region of the country, census categories

ordinal level

-attributes of an ordinal variable provide information on rank order

-numbers corresponding to the categories of an ordinal variable can tell "greatest" and "least"

-ex. strongly agree to strongly disagree

-quantitative- underlying order of responses can be placed on a continuum

-missing a precise statement on how different categories differ from one another

Interval Level

ranks data, and precise differences between units of measure do exist; however, there is no meaningful zero

-provides information on interval distance

-numbers can be added and subtracted but ratios are not meaningful

-quantitative

-ex. temperature (*F), SAT scores, number of church visits

ratio level

a level of measurement that has all the properties of an interval level scale and also has a true zero point

-has classification, rank order, interval distance, and meaningful (true, non arbitrary) zero point

-zero means absence of quantity or amount

-ex. illiterate persons have zero years of formal education, zero on income/salary

-number corresponding to the categories can be used to construct ratios

-ex. number of siblings, number of arrests

-quantitative

classification

(categorization)

-all variables must be able to efficiently classify or categorize observations

-attributes of the variable must be exhaustive and mutually exclusive

-nominal, ordinal, interval, ratio all have

exhaustive (classification)

every observation corresponds to a category

-one of the variable's attributes

mutually exclusive (classification)

every observation corresponds to only 1 category/variable's attributes

observed value (sources of measurement error equation)

observed value= true value+ systematic error + random (idiosyncratic) error

random (idiosyncratic) errors

affect a relatively small number of individuals in unique ways that are unlikely to be repeated in just the same way

-individuals make them when they don't understand a question or when some unique feelings are triggered by the wording of question

systematic errors

when the responses of groups of individuals are affected by factors that are not "what the instrument is intended to measure" (e.g., unbalanced response choices)

-some individuals like to please others by giving socially desirable responses (agree with statements to avoid disagreement)

validity

accuracy a measure

(a not valid measure is affected by errors in measurement)

-measure yields valid information to the extent that it adequately reflects the "real" meaning of the concept under consideration

-when measure "misses the mark" (not valid) a measure procedure affected by measurement error

4 types of validity

face, content, criterion, construct

face validity

does a measure "make sense?" (may tell us nothing about measurement validity)

-face valid if it obviously pertains to the meaning of the concept being measured more than to other concepts

-denotes the confidence gained from careful inspection of a concept to see if it is appropriate "on its face"

-weakest type of validity- not convincing evidence of measurement validity

content validity

does a measure cover the full range (as exhaustively as possible) of a concept's meaning?

-experts may be solicited to gauge the different dimensions of a concept

criterion validity

when scores from one measure can be compared to those obtained with a more direct or already validated measure of the same phenomenon (the criterion)

-shoudl not assume that a measure that is validated in 1 study is also valid in another setting or with a different population

-criterion can be measured at same time or after the variable to be validated

-concurrent validity and predictive validity

concurrent validity

when the scores of our measure of interest are compared to the scores on a criterion that is measured simultaneously as our measure of interest

predictive validity

predicting scores on a criterion that is measured in the future

construct validity

established by showing that a measure is related to a variety of other measures as specified in a theory

-used when no clear criterion exists for validation purposes

-often necessary to work with concepts that have been included in well developed theory

-two approaches: convergent and discriminant

convergent validity

when one measure of a concept is associated with different measures of the same concept (relies on a logic similar to triangualtion)

discriminant validity

when scores on the measure to be validated aree compared to scores on measures of different but related concepts

-measure to be validated is not associated strongly with the measures of different concepts

reliability

obtaining stable, consistent results every time a measure is used

-prerequisite of validity (in the absence of reliability, we cannot talk about validity)

types: test-retest, alternate forms, inter-item, split-half, inter-observer

measurement reliability

the extent to which a measure is affected by idiosyncratic or random error

test-retest reliability

obtaining the same results when used repeatedly on the same phenomenon

ex. math test two months apart yielding same results (assuming your math ability has not changed)

alternate-forms reliability

using slightly different versions of the measure (or survey question)

ex. "how happy are you?" and another says "rate your level of happiness?"

inter-item reliability

when researchers use multiple items to measure a single concept using an index, must be concerned with this

-the stringer the association, the higher reliability

-Cronbach's alpha (if the constructive items of a composite measure are associated)

split-half reliability

index items are randomly divided into two subscales and scores are compared for correspondence

-used when researchers want to measure a concept using a multiple item index

inter-observer reliability

using 2 or more observers to rate social events or phenomena (situations)

-increase by observer training, data quality checks, and higher levels of communication among research team

-most important when the rating task is complex

unit of analysis

the level of social life that a research question is focused (like individuals, groups, towns, or nations)

units of observation

individual data (individuals = unit of observation) may be aggregated and analyzed at the group level (groups= unit of analysis)

ecological fallacy

using group level data to draw conclusions about individual level processes

reductionist fallacy

using data about individuals to draw conclusions about group-level processes

-also known as reductionism or individualist fallacy

design types

cross sectional, longitudinal

cross sectional designs

-data collected at one point in time

-time order may be difficult to determine

go back and clarify

longitudinal designs

-data collected at 2 or more points in time

-identification of time order of effects can be observed

-3 major types: repeated cross sectional, fixed panel, and event based

repeated cross sectional (trend study)

data are collected at 2 or more points in time but from different samples of the same population

fixed-sample panel design (panel study)

data are collected at two or more points in time from the same individuals (the panel)

event-based (cohort study)

data are collected at two or more points in time from individuals in a cohort (different samples within a cohort)

-can be a type of repeated cross sectional or a type of panel design

causal explanations

causes and effects

-one concept somehow leads to another

-an explanation for some characteristic, attitude, or behavior of groups, individuals, or other entities (such as families, organizations, or cities) or for events

-IV is the presumed cause and the DV is the potential effect

simple causation

IV -> DV

multiple causation

(IV1, IV2, IV3)->DV

indirect causation

IV1->DV1 (also IV2); IV2->DV2

nomothetic approach to causality

identification of a few causal factors that influence a class of conditions or phenomena

idiographic approach to causality

exhaustive identification of specific or particular causes of a limited set of conditions or phenomena

nomothetic causal explanation

one involving the belief that variation in an IV will be followed by variation in the DV, when all other things are equal (ceteris paribus)

-researchers who claim a causal effect concluded that the value of cases on the DV differs from what their value would have been in the absence of variation in the IV

nomothetic causal effect

when variation in an IV leads to or results, on average, in variation in the DV

-ex. individuals arrested for domestic assault tend to commit fewer subsequent assaults than do similar individuals who are accused in the same circumstances but not arrested

idiographic causal explanation

the concrete, individual sequence of events, thoughts, or actions that resulted in a particular outcome for a particular individual or that led to a particular event

-an idiographic explanation also may be termed an individualist or a historicist explanation

-individualist, historicist

idiographic causal effect

when a series of concrete events, thoughts, or actions result in a particular event or individual outcome

criteria for causation

-5 criteria: association, time order, nonspuriousness, causal mechanism, causal context

-when one or more criteria unmet, may have important doubts about causal assertions the researcher may have made

association

if variation in one variable leads to variation in another variable

-an empirical (or observed) association between the IV and the DV

-true experiment association between IV and DV: there are two or more groups that differ in terms of their value on the iv

-non-experimental association between IV and DV: seeing whether values of cases that differ on the IV tend to differ in terms of the DV

time order

independent variable comes before the dependent variable

-determine temporal order/direction of influence

-experiment: time order determined by researcher

-non-experiment: cross-sectional conclusions about causation must be more tentative because they do not establish time order of effects

nonspuriousness

when the relationship between 2 variables is not due to variation in a third variable (correlation does not prove causation)

-Randomization and statistical control are two mechanisms to ensure that spuriousness is not present in experimental and non-experimental designs, respectively

randomization:

a technique used to reduce the risk of spuriousness

-greater number of cases assigned randomly to the groups, the more likely that the groups will be equivalent in all respects

statistical control

a method in which one variable is held constant so that the relationship between two or more other variables can be assessed without the influence of variation in the control variable

causal mechanism

the process that connects variation in the independent variable with variation in the dependent variable it is hypothesized to cause

causal context

identification of the broader context in which a causal relationship occurs (helps to understand the relationship)

3 basic research designs

experiments, surveys, qualitative methods

experiments

best to test nomothetical causal hypothesis/ also most appropriate for studying treatment effects

-laboratory setting: more control over conditions at espense of generalizability of findings

-usually quantitative

Surveys

nomothetic, usually quantitative

-most rely on random sampling- better suited for descriptive research intended to produce generalizable findings

-have measurement validity

-can establish causal effects but weaker than true experiments

qualitative methods

suited for idiographic causal assertions

our understandings of causal relationships are ____?

always partial

sampling theory

focuses on the generalizability of descriptive findings to the population from which the sample was drawn

-considers whether statements can be generalized from one pop to another

sampling

the process of selecting observations or study elements

population

the entire set of individuals or entities under study; the group about which a researcher would like to draw conclusions

target population

-may or may not be different from population

sample

a subset (segment) of a population used to study the whole population

elements

the individual members of the population whose characteristics are to be measured

sampling frame

the set of all cases from which the sample is actually selected

population parameter

aggregate characteristic of a population

sample statistic

aggregate characteristic of a sample

enumeration units

units containing one or more elements and that are listed in a sampling frame

sampling units

units listed at each stage of a multistage sample

census

representativeness

is a sample representative of a population, does it approximate the same characteristics of interest as the population from which it has been drawn? - aggregate characteristics of the sample closely approximate the same aggregate characteristics in the population

- a representative sample looks alike the population from which it was selected in all relevant aspects

-an unrepresentative same has some characteristics over- or underrepresented

generizability

extrapolating the conclusions drawn from the sample to the population from which the sample was drawn

1) can findings from a sample of the population be generalized to the population from which the sample was selected?

2) can findings from a study of one population be generalized to another, somewhat different population?

-depends on sample quality (amount of sampling error)

sample bias

distortion in the representativeness of the sample

sampling error

the difference in the characteristics of a sample and those of a population (the larger, the less representative the sample is, the less entitled to making generalizations we are)

-random error is ok- happens in nearly every sample

-systematic error- not ok

probability sampling

random selection of elements from a population of interest (the chance of selection is known, unlike non-probability sampling)

-probability varies between 0 (absence) and 1 (presence)

-non-response rate may be cause of concern in probability sampling

-larger the sample, the more representative it is; the more homogenous the population, the more representative the sample (which doesn't need to be v large)

-no known systemic bias

probability of selection

the likelihood of an element to be included in a sample

-probability varies between 0 (absence) and 1 (presence)

-as the size of the sample as a proportion to the population decreases, so does the probability of selection

4 types of probability sampling

simple random, systematic random, stratified random, multiple-stage cluster

simple random

using some procedure (random number table, computer, random digit dialing) to generate numbers identifying the vases strictly on the basis of chance

-replacement sampling

random number table

a list of random numbers used to select elements in a population

random selection

of elements using computer algorithms

random digit dialing

the use of a robot that dials random numbers within the phone prefixes corresponding to the area in which survey is to be conducted

replacement sampling

when sample elements are returned to the sampling frame to be sampled again (rarely used)

systematic random

when elements are selected from a list with every nth being selected (first element is randomly selected, then a sampling interval is used)

-sampling interval, periodicity

sampling interval

the number of cases from one sampled case to another in a systematic random sample

periodicity

may lead to biased samples, if the sequence of elements (in a list to be sampled) varies is in regular (periodic) pattern

stratified random

info about the population is used prior to sampling (such as census data)

-sampling elements separated into strata (identified prior to the actual sampling, with each element belonging into one and only one stratum)

-more efficient sampling process

-created on basis of some relevant characteristic

-weighting can be used

disproportionate stratified sampling

proportion of each stratum included in the sample is intentionally varied from what it is in the population

proportionate stratified sampling

each stratum would be represented exactly in proportion to its size in the population

multi-stage cluster

used when a sampling frame is not available

-elements selected in 2 or more stages (first stage random selection of naturally occurring clusters, last stage random selection of elements within clusters)

-more clusters with fewer individuals in each cluster= more representative sample; if clusters are homogenous, then fewer elements per cluster may be needed; sampling error is greater than in simple random sampling