Chapter 5: Measuring Variables and Sampling

Chapter 5 Measuring Variables and Sampling

Learning Objectives

  • 5.1 Explain the concept of measurement and describe the characteristics of the four scales of measurement.
  • 5.2 Differentiate between the different types of reliability and validity evidence.
  • 5.3 Analyze the strengths and weaknesses of each of the sampling methods.
  • 5.4 Distinguish between random selection and random assignment.
  • 5.5 Explain how to determine recommended sample size.
  • 5.6 Summarize the sampling methods used in qualitative research.

Chapter Outline

Variables
  • Definition: A condition or characteristic that can take on different values or categories.
Defining Measurement
Measurement
  • Definition: The assignment of symbols or numbers to something according to a set of rules.
    • Example: Measurement of the length of books.
Scales of Measurement
Stevens (1946)
  • Measurement can be categorized by the type of information communicated by the way the variables are measured.
    • Four levels or “scales” of measurement:
    1. Nominal scale
    2. Ordinal scale
    3. Interval scale
    4. Ratio scale
    • See Table 5.1 for detailed comparison.
Nominal Scale
  • Characteristics:
    • Use of symbols (words/numbers) to classify or categorize measurements.
    • Simplest and most basic level of measurement.
    • Nonquantitative scale of measurement.
    • Identifies types rather than amounts of something.
  • Example: College major classification (e.g., 1 = psychology, 2 = engineering, 3 = philosophy).
    • Other examples include personality type, country of birth, gender, and research group categories (experimental group or control group).
Ordinal Scale
  • Characteristics:
    • Rank-order measurement scale where the distance between levels is unknown.
    • Allows identification of higher or lower status on a variable of interest.
  • Examples:
    • Finish order in a marathon.
    • Social class levels (high, medium, low).
    • Rank ordering of job applicants or need for services.
Interval Scale
  • Characteristics:
    • Measurements with equal intervals or distances between adjacent numbers, along with characteristics of lower-level scales (naming and ranking).
    • Examples:
    • Temperature in Fahrenheit or Celsius (e.g., difference between 15°F and 25°F equals the difference between 505°F and 515°F).
    • Year measurement and IQ scores.
    • Absence of an absolute zero point (zero does not indicate the absence of the quantity measured).
Ratio Scale
  • Characteristics:
    • Measurements with rank order, equal intervals, and an absolute zero point.
    • Considered the highest level of measurement, conveying the most numerical information.
  • Examples:
    • Weight and height measurements.
    • Kelvin temperature (where zero equals no molecular movement).
    • Annual income, where zero indicates no income.

Psychometric Properties of Good Measurement

Reliability and Validity Overview
Reliability
  • Definition: The consistency or stability of scores in a measurement instrument.
    • Applies to psychological testing and research instruments used to measure variables.
  • Types of Reliability:
    1. Test–retest reliability:
    • Measures consistency over time. Administer the test, wait, and administer again. Use correlation between scores to measure reliability.
    1. Equivalent-Forms Reliability:
    • Compares two equivalent forms of a test. Strong positive correlation indicates consistency.
    • Example tests include SAT, GRE, ACT.
    1. Internal Consistency Reliability:
    • Measures consistency among items on a test. Affected by test length and quality of items.
    • Commonly reported index: Coefficient alpha (Cronbach’s alpha); aim for ≥ .70.
    1. Interrater Reliability:
    • Examines consistency between two or more raters or observers.
    • Example: Two judges rate the same set of student essays and their agreement is analyzed.
Validity
  • Definition: The extent to which the measurement procedure is measuring what it is supposed to measure and whether the scores are interpreted correctly.
  • Importance: Reliability is necessary for validity but not sufficient on its own.
  • Types of Validity:
    • Content Validity: Juries of experts judge how well-test items represent the intended construct.
    • Construct Validity: Involves establishing how well a test or instrument measures the construct it is supposed to measure.
    • Includes convergent validity (test correlates with similar constructs) and discriminant validity (test does not correlate with dissimilar constructs).
    • Criterion-related Validity: Assesses how well one measure predicts another. Includes predictive validity and concurrent validity.
Reliability Coefficients
  • Quantitative Index: Should ideally be greater than or equal to .70 for strong consistency.
  • Low Reliability: Coefficients below .70 indicate insufficient reliability for measurement instruments.
Sampling Methods: Strengths and Weaknesses
Overview of Sampling Methods
  • Sample: An element or set taken from a larger population.
  • Population: The complete set of elements or individuals from which samples can be drawn.
  • Sampling Techniques:
    • Random Sampling: Aim for a representative sample from the population.
    • Simple Random Sampling: Each member of population has an equal chance of selection.
    • Stratified Random Sampling: Divides population into strata and randomly samples from each.
    • Systematic Sampling: Selects every k-th individual from a list.
    • Non-random Sampling: Typically leads to biased samples but is sometimes practical.
    • Convenience Sampling: Participants selected based on availability.
    • Quota Sampling: Specific quotas set for different groups before sampling.
    • Purposive Sampling: Participants chosen based on characteristics relevant to research.
    • Snowball Sampling: Participants recruit other participants based on inclusion criteria.
Random Selection vs Random Assignment
  • Random Selection: Method for obtaining a sample representing the population; crucial for survey research.
  • Random Assignment: Method used in experiments to create equivalent groups; aims to counterbalance extraneous variables.
Determining Sample Size
Guidelines for Sample Size
  • If population is 100 or fewer, include the entire population.
  • Larger sample sizes are preferred for better reliability and to capture effects.
  • Refer to research literature to observe common sample sizes.
  • Use sample size calculators (e.g., G-Power) to help determine size based on various factors.
  • Consider the characteristics of the population and required precision for subcategory analysis.
Sampling in Qualitative Research
  • Focus: In-depth understanding of cases rather than breadth.
  • Usually pursues purposive sampling for information-rich cases.
  • Theoretical Sampling: Continual selection of relevant participants during the study.
  • Mixed Sampling: Combines qualitative and quantitative approaches for richer data.
Vocabulary Definitions
  • Biased Sample: A sample that does not accurately reflect the population.
  • Census: A count of all individuals in a population.
  • Cluster: Multiple elements associated in a group.
  • Coefficient Alpha: A statistic used to measure internal consistency reliability, aiming for ≥ 0.70.
  • Content Validity: Judged degree of test items representing the intended construct.
  • Convergent Validity Evidence: Evidence showing that scores correlate with similar constructs.
  • Criterion-related Validity: Validity assessed against an established standard or benchmark.
Essay Questions
  1. Define variable and measurement. Describe the difference between them and give an example for each.
  2. List the four scales of measurement from least complex to most complex – provide an example of each.
  3. Using an example, explain the difference between reliability and validity. Discuss what it means to say a reliable measure is not always valid, but a valid measure is always reliable.
    …
    (continue with more essay questions as noted in the original input)
Classroom Exercise Suggestions
  1. Conduct a brief anonymous survey for students to identify the four scales of measurement based on their responses.
  2. Use an internet-based test to illustrate concepts of reliability and validity.
  3. Discuss the importance of random assignment and its distinction from random selection in research validity.
  4. Reference videos on nominal variables, reliability, and validity concepts for classroom engagement.

(Continue with summarized suggestions and activities related to the content)