SOC150 Week 4 Reading

sample

the subset of the larger population that the researcher has collected data from, and that represent the target population within the study

population

the larger group that a sample of people (or other units of analysis) are drawn from, and that the sample is intended to represent

cases

members of the sample that the researcher has gathered data on, such as the individual interviewees or organizations being studied

sampling

the process of selecting cases or observations that will be analyzed for research purposes

unit of analysis

the class of phenomena (e.g., individuals, groups, objects, societies) that researchers want to learn about through their research

units of observation

the class of phenomena (e.g., individuals, groups, objects, societies) that researchers can actually observe for their study, which may also be their unit of analysis or may be another unit that provides indirect knowledge of their unit of analysis

population of interest

the larger group (of people, organizations, objects, etc.) that a researcher is interested in learning about and that their research question applies to (aka target population)

target population

the larger group (of people, organizations, objects, etc.) that a researcher is interested in learning about, and that their research question applies to (aka population of interest)

probability sampling (aka random sampling)

a type of sampling in which the researchers know the likelihood that a person (or other unit of analysis) in the population will be selected for membership in the sample (widely used in quantitative studies)

nonprobability sampling

a type of sampling in which the researchers do not know the likelihood that a person (or other unit of analysis) in the population will be selected for membership in the sample (common in qualitative research)

generalizable

when a study's results can reasonably tell us sth about the larger population from which its sample was drawn

sampling unit

each case in the researcher's sample - for example, each individual interviewed in an interview-based study

statistics

numbers like means, medians, and standard deviations that describe the sample (or some group within our sample)

parameters

the means, medians, standard deviations, and so on for the whole population—are the numbers we really care about but that we usually cannot observe (that is the whole point of sampling)

representative sample

a sample whose characteristics are similar to the population from which it was drawn, which means that findings from that sample can be reasonably generalized to the population

WEIRD societies

Societies that fall into the categories of "Western, educated, industrialized, rich, and democratic"—which, given inequalities in where scientific research occurs, tend to be where samples for many studies are drawn

scope conditions

the conditions under which a relevant theory derived from a study's empirical research can and cannot reasonably be applied (aka boundary conditions)

sampling frame

a list of members of a population that is available to researchers, which they use to select cases for their sample, ideally, the sampling frame includes every single member of that population

sampling bias (aka selection bias or sample selection bias)

A type of bias that occurs when the elements selected for inclusion in a study do not represent the larger population from which they were drawn

bias

a systematic error that may make our research findings inaccurate in some way

sampled population

all the people (or other units of analysis) whom researchers seek to recruit from the population of interest

respondents

individuals who answer questions for a survey or in-depth interview

nonrespondents

individuals who decline to answer questions for a survey or in-depth interview or who cannot be reached by researchers

response rate

a percentage determined by dividing the number of completed survey questionnaires by the number originally distributed, or the number of individuals successfully interviewed by the number contacted for an interview

nonresponse bias

bias introduced into a study when respondents and nonrespondents differ in important ways, which means that the relevant characteristics observed in the sample differ from those in the target population

generalizability

refers to the idea that the results obtained from a given sample apply to the population from which it was drawn

random selection

a sampling process that ensures that cases from the population are picked at random

error

inaccuracy in the measurement of key variables for a given population

sampling error

the difference expected between the results obtained from a sample and the actual parameters of a population

simple random sampling

the researcher gives all members of a population (more accurately, of a sampling frame) an equal probability of being selected

systemic sampling

the researcher selects elements from a sampling frame in specified intervals—for instance, every kth element on the list (where the selection interval k is calculated by dividing the total number of population elements by the desired sample size)

selection interval

the distance between the elements of the sampling frame that researchers select for inclusion in their sample

periodicity

the tendency for a pattern to occur at regular intervals

stratified sampling

researchers divide the study population into two or more mutually exclusive subgroups (known as strata) and then draw a sample from each subgroup

proportionate stratified sampling

an approach to stratified sampling where the sizes of the subgroup samples match their relative sizes in the population (or sampling frame)

disproportionate stratified sampling

an approach to stratified sampling in which the sizes of the subgroup samples do not match their relative sizes within the population (or sampling frame)

oversampled

when a subgroup represents a greater share of a sample than the same subgroup represents in the larger population

weighting

adjusting for how much particular cases contribute to the statistics for a sample

cluster sampling

a researcher begins by sampling groups (or clusters) of population elements and then selects elements from within those groups

probability proportionate to size

a procedure to adjust the sample selection process so that each element across the differently sized clusters has an equal chance of being selected

(aka PPS)

pilot testing

any preliminary vetting of a survey questionnaire, interview guide, or other research instrument

pilot study

research that explores initial ideas or tests out data collection or analysis procedures as part of a more comprehensive project

purposive sampling

a nonprobability sampling approach where the selection of cases is guided by the researcher's theory about what concepts and processes matter (aka theoretical sampling)

convenience sampling

a researcher draws a sample from part of the population that is convenient to obtain—for instance, because potential interviewees are located near the researcher or otherwise are readily available

snowball sampling

a sampling technique where researchers ask study participants they have already recruited to help identify additional participants

quota sampling

a nonprobability sampling strategy that quantitative researchers use frequently

postsurvey

performing statistical weighting to adjust the sample after it has been collected and make its statistics better reflect the actual parameters of the target population

weighted sample

a sample that has been adjusted to better reflect the distribution of relevant characteristics in the population

field site

a setting, organization, or other social context that a researcher chooses to study in order to understand a phenomenon (aka site)

extreme cases

organizations or other sites of study that display characteristics or behaviours that are not the norm

outliers

particular cases (e.g., persons, organizations, observations) that contradict an observed pattern or existing theory, such as by having exceptionally high or low values on a noteworthy variable (aka deviant cases)

sample for range

a sampling strategy where the researcher seeks to find a sufficient number of cases that reflect the range of variation across one or more key variables

inclusion criteria

criteria that a researcher uses to decide whether to include a person (or other unit of analysis) within a sample

exclusion criteria

criteria that a researcher uses to decide whether to exclude a person (or other unit of analysis) from a sample

iterative sampling (aka recursive sampling)

a sampling strategy in which researchers move back and forth between the process of sampling and preliminary analysis of their data

self-selection bias

bias that occurs when certain types of people are more likely to volunteer for (or be selected into) a sample

attrition bias

bias that occurs when the participants who leave a study tend to come from particular subgroups, thereby undermining the representativeness of the study's sample

conceptualization

the stage of the research process at which researchers explicitly and clearly define the concepts they are using in their study

dimensions

aspects of a concept that can vary

operationalization

the stage of the research process at which the researcher specifies explicitly and clearly how a concept will be measured

indicators

another term for variables—operational definitions of concepts—that is frequently used in quantitative research, especially for variables that are routinely collected by government agencies and other organizations

measures

another term for variables (operational definitions of concepts)

reference group

the social group to which people compare themselves when judging their own social status, such as when they assess their income relative to others

attributes

characteristics of a variable representing its different possible values or categories

level of measurement

a classification system that categorizes variables by how their attributes are related to one another

nominal level

the lowest level of measurement, where a variable's attributes are different from one another, but there is no mathematical order to the attributes

mutually exclusive

when the different attributes defined for a variable (or response options provided for a survey question) do not overlap with one another

collectively exhaustive

when the list of attributes associated with a variable covers all possibilities

response categories (aka response options)

the listed options that a respondent can choose when answering a particular question (item) on a survey instrument

ordinal level

a level of measurement in which a variable's attributes can be numerically ranked from low to high values (or vice versa) using some kind of meaningful comparison

Likert scales

ordinal-level scales used to measure the intensity of people's agreement with a particular statement

scale level

a level of measurement where the variable's attributes are ordered (like for ordinal-level variables) and the distance between those attributes is meaningful (unlike for ordinal-level variables)

categorical variable

variables measured at the two lower levels of measurement, nominal and ordinal

ratio level

the highest level of measurement, where the variable's attributes are separated by equal and meaningful distances (like for interval-level variables) and the variable has a true zero point (unlike for interval-level variables)

index

a type of measure that contains multiple indicators designed to assess a more general concept

scale

a type of measure that contains multiple indicators designed to assess a more general concept

reliability

the consistency of a measure. A measure is said to be reliable if it gives the same result upon repeated applications to the same phenomenon

validity

the accuracy of a measure (also called construct validity). A measure is said to be valid if it accurately reflects the meaning of the concept under study

test-retest reliability

a method of assessing the reliability of a measure by collecting data from a sample and then retesting the same sample after a period of time

inter-rate reliability

a method of assessing the reliability of a measure by examining the degree to which two or more observers (raters) agree on the measurement of one or more cases (i.e., on the values assigned to those cases)

internal consistency

the degree to which participants' answers to items within a multiple-item measure are consistent

face validity

a method of assessing the validity of a measure in which the researcher evaluates whether it is plausible and logical that a given variable (specifically, the operational definition of a concept) actually measures what it intends to measure

content validity

a method used to assess the validity of a measure where a researcher evaluates whether the measure covers all the possible meanings, domains, and dimensions of a concept

predictive validity

a method of assessing the reliability of a measure by determining if it predicts future phenomena that it should be able to predict

convergent validity

a method used to assess the validity of a measure by comparing scores on that measure to those derived from an existing measure of the same or a similar concept

discriminant validity

a method of assessing the validity of a measure by comparing scores on that measure with those derived from an existing measure that, logically speaking, should be entirely unrelated

measurement error

the difference between the measured values of a variable and the true but unobserved values of that variable

random error

a type of measurement error—present in every measurement—where unpredictable differences occur between the measured values of a variable and the true but unobserved values (compare to systematic error)

systematic error

a type of measurement error that occurs when a measure consistently produces incorrect data (compare to random error)

leading question

when the wording of a question encourages a person to answer in a potentially biased way

social desirability bias

bias that occurs when participants in a research study answer or act in particular ways to present themselves to the researcher in a more positive light

acquiescence bias

bias that occurs when people tend to say "yes" to whatever the researcher asks, even when doing so contradicts previous answers (aka yea-saying)

positionality

how a researcher's personal identities and perspectives shape how they interpret reality, including what they observe over the course of their research and what conclusions they draw

authenticity

a criterion for assessing the rigour of qualitative research that focuses on the extent to which researchers capture the multiple perspectives of their research participants (fairness is a dimension of authenticity)

trustworthiness

refers to the overall truthfulness and usefulness of the results of the research study. We can think of trustworthiness in three dimensions (credibility, dependability, and confirmability), which resemble the criteria of validity and reliability used to evaluate quantitative studies, though with important differences in emphasis

credibility

refers to the degree to which the results are accurate and viewed as important and believable

dependability

a criterion for assessing the rigor of qualitative research that focuses on whether the researcher followed proper procedures in conducting the project (a larger criterion of trustworthiness)

confirmability

refers to the degree to which the results reported are actually grounded in the data obtained from participants

audit

a method used to evaluate the rigour of qualitative research by tracing each point made in the published study back to the original data that was its basis