W6 QUANTITATIVE DATA QUALITY AND LEVELS OF MEASUREMENT

measurement as assessment of quantitative data

• measurement involves rules for assigning numeric values to qualities of people or objects etc

  • to designate the quantity of the attribute

• no attribute inherently has a numeric value

• we need rules to measure “how much” of an attribute is present

• quantification is used to communicate that amount

• assigning numbers differentiates among people or objects etc.

what is rigour

• it is an aspect of data quality

• it determined by measurement instruments that reflect the concepts being tested in a study

what is psychometric assessments

• part of data quality

• obtains evidence of the quality of instruments and includes 2 properties related to the quality of a measurement

  • assessment of the reliability and validity of measurement instruments

what is reliability

• this is the extent to which an instrument yields the same results on repeated measures

• concerned with consistency, accuracy, percision

what are the 3 aspects of reliability *

• stability

  • test-retest reliability

  • does the instrument produce the same results with repeated testing

• internal consistency (homogeneity)

  • do the items on a survey belong together and measure the same concept/characteristic

• equivalence

  • interrater reliability - would the tool produce the same results when equivalent procedures are used

componenet of reliability #1 - test-retest reliability stability

• stability of scores of na instrument when administered more than once to the same participants under similar conditions

• doesn’t work with food , pain because your not likely to get the same results

what is a major disadvantage of test-retest relaibility (stability)

• MAJOR DISADVANTAGE- many traits of interest do change overtime independently of the instruments stability

component of reliability - internal consistency

• homogeneity with which the items in a scale refelct measures the same concept

  • items in the scale are correlated with each other and the subparts measure the same characteristics

• what is the most ocmmonly used test of internal consistency

  • CRONBAHCS ALPHA

    • each item in the scale is simultaneously compared w/ other items in scale and total score used to analyze the data

• what is the most ocmmonly used test of internal consistency

• • CRONBAHCS ALPHA

    • each item in the scale is simultaneously compared w/ other items in scale and total score used to analyze the data

component of reliability - Equivalence - interrater/interobserver reliability

• equivalence is either consistency or agreement among observers who use the same measurement tool or agreement between alternative forms of a tool

• used with structured instruments that used to observe behaviours that are systematically recorded

• refers to the consistency of observations between 2 or more observers with several occasions

• has to do with the reliability of the observer not instrument

waht is reliability coefficient

• reliability assessment invovles computing a reliability coefficient (numeric index of an instrument reliability)

• reliability coefficient (resignated as r) rangess from 0-1

  • the closer the coefficient to 1 = higher the reliability of the measure

  • reliability coefficient of over0.70 or higher = acceptable

• different tests of reliability cna be used to calculate a reliability coefficient depending on the tool

what is cronbahcs alpha

• most common test for reliability

• low reliability score = items on the instrument are not well correlated and therefore the instrument is unreliable

• if the instrument is unreliable = jepordize findings

• how low is low

  • reliability coefficient of 0.70 considered acceptable but higher reliability coefficient the better

example of cronhbachs alpha

• a researcher reports a cronbachs alpha of 0.95 for an efficacy measurement scale

• which form of reliability are they describing

  • internal consistency

• are their findings adequate

  • it is adequate tool - considere pretty good - pretty stable cuz its 0.9 and clsoer to one

what is validity

• refers to whether a measurement isntrument measures what it intends to measure

what are 3 major kinds of validity

• content validity

  • subtype - face validity

• criterion-related validity

• construct validity

content validity

• assess whether the items in a test or measurement adequately represent the entire content of the construct being measured

• focus - ensures that the measure covers all relevant aspects of the concept

  • ex. it tests aims to measure “math ability” content validity would involve checking whether the test items cover all areas of math relevant to it

• evaluation- often evaluated through expert judgement and by reviewing the test items to ensure they align with the construct

• CVI- quantitative measure used to assess the content validity of a measurement instrument - ex. questionnaire or test

  • systematic way to evaluate how well items in a tool represent the construct being measured

content validity - subtype - FACE VALIDITY

• giving descriptive feedbakc

• instrument appears to be measuring the appropriate concept

• colleagues are asked to review the content of the instrument as to whether it reflects the concept that the researcher intends to measure

• based on judgement

• not considered an alternative to other types of validity

criterion-related validity

• the researcher measures the relationship between scores from the instrument withexternal criteria (gold standard)

• scores on the instrument are compared to scores on another isntrument that measuresthe same attribute

• there are 2 types withing this

  • predictive validity

  • concurrent validity

criterion related validity - predicitive validity

• degree of correlation between the measureme of the concept adn a future measure of the same concept (ex. nursing students empowerment teaching behaviour scale scores predicted thier structural empowerment scores)

criterion related validity - concurrent validity

• degree of correlation 2 measures of the same construct administered of the same time (eg. pulse oximeter readings vs. arterial blood gases; correlation between distress using 2 different distress scales)

• high correlation = high agreement

what is construct validity - 3rd types

• extent to which a test measures a theoretical construct or trait

• complex approach

• invovles empirical testing of hypothesized relationships to validate the instrument (ex. construct validity of the breastfeeding seld efficacy scale-short form, was tested based on 2 hypothesis

• other forms of construct validity - convergent, divergent, factor analysis

reliability and validity

• measure can be reliable but not valid

  • ex. a researcher measures anxiety by measuring temepraturs ; temperature is measured accurately; however temperatures would not be a valid indicator of anxiety

• a valid instrument is reliable

  • ex. an instrument that is erratic, inconsistent and inaccurate cannot validly measure the attribute

measurement error

• rarely if ever can a single measurement strategy completely measure all the aspects of an abstract concept

measurement error pt 2 !

• random or chance errors are unpredictable and difficult to control and cannot be corrected

  • examples

    • transient human condition - hunger, fatigue, mood

    • variations in measurement procedure - environmental factors [room temp], misplacement of bp cuff)

    • errors in data processing [ data entry errors]

• systematic error or constant errors

  • some examples

    • attributed to relatively stable characteristics of the study population that might bias behaviour (level education, socioeconomic status, social desirability bias)

    • improperly calibrated instrument (ex. weigh scale)

RELIABILITY AND VALIDITY ON TARGETS SCALE

• reliable not valid

  • hitting the target in teh same location (stable) but miss the bullseye

  • ex. not measuring what we want to measure

  • consistently and systematically measuring the wrong value for all respondents

• valid not reliable

  • hits the target in scattered locations (not stable) but some do hit bullseye

  • ex. measuring what we want to measure but not all of the time

• not reliable not valid

  • hits are not stable, scattered across the target and consistently miss the bullseye

  • ex. not measuring what we want to measure

• reliable , valid

  • hits the target in the same location (stable) AND htis bullseye

  • ex. measuring what we want to measure

critiquing data quality

• consider whether

  • appropriate method was used to assess the reliability and validity of the tool

  • the appropriate method used to test the reliability and validity of the tool

what is measurement

• numbers are assigned to variables or events and are based on a set of rules

what are the 4 levels of measurement

• nominal

• ordinal

• interval

• ratio

• the levels of measurement help determine the types of statistical analysis to use when analyzing data

• the higher the level fo measurement , the more flexibility in choosing statistical procedures

• use the highest level fo measurement so that the maximum information can be obtained from the data

levels of measurement - nominal level

• dichotomous (two true values) or cateogrical (more than two true values ) variables

• mutually exclusive categories used to classify data

• the researcher assigns a number to each category , however the numbers are meaningless

• measure of central tendecy can be used with this is MODE

levels of measurement - ORDINAL

• categorical, mutually exclusive categories - variables given a number - ONE GROUP

• numbers assigned are not arbitrary and have some meaning - ASSIGNED RANK

• ranks variables based on their relative standing on an attribute (ex. memebers of a higher ranked category have more of an attribute than members of a lower ranked category)

• distance between categories are not known or exact

• examples

  • stages of cancer diagnosis (stage 0,1,2,3. the cleint who has stafe 2 is higher than stage 0, but we cannot say they are twice as severe as stage 0

  • undergraduate, masters, PhD

• measures of central tendency - mode, median

levels of measurement - interval level

• variables are continous and ranked order

• equal intervals between numbers ex. temeprature on the centigrade scale : difference between 20 and 30 and between 30 adn 40 is presumed ot be equivalent

• Zero point is arbitrarily not assigned and not absolute (ex. zero on the centigrade scale does not represent the absence of temperature)

  • ITS JSUT A # CUX U CAN GO INTO NEGATIVES

• examples

  • many psychological/psychosocial varibles and tsts (ex. differences between testscores represent equal intervals but a score of zero does not indicate an absence of knowledge

  • likert scale ratings (ex. somewaht satisfied, very satisfied…) are treated numerically and responses are of the interval level of the measurement

• statisticla potential

  • can calculate teh average or mean

  • ex. the mean score on the research satisfaction survey for nurseswas 3/possible 4

• the central tendency of a variable measured of the interval level can be represented by its mode, median, mean

  • mean gives the most useful information

Levels of measurement - RATIO level

• highest level of measurement

• variables are continous and rnak order and have an absolute , meaninginful zero, therefore, provide the absolute magnitude of the attribute

• distances between the values are numerically equal

• an absolute zero means absence of any of the attribute being measured

  • this is true zero

  • ex. hemoglobin levels- there is no negative values

CHART OF THE LEVELS OF MEASUREMENT

• nominal

  • discrete/categorical exclusive adn exhaustive

  • #s assigned used as labels

  • do not have quantitative meaning

  • no order to categories

  • examples

    • marrital status, gender

  • MCT - MODE

  • bar graph/chart

• ordinal

  • discrete/categorical

    • exclusive adn exhaustive

    • #’s assigned not arbitrary

    • categories can be palced in a meaningingful numerical order (rank ordering)

      • ex. lowest to highest

    • distances between categories are not known or exact

    • exmaples

      • olympic medals

      • stages of illness

    • MCT= mode, median,

    • bargrpah/chart

  • INTERVAL

    • continous ex. infinite # values possible

    • can be palces in a meaniningful numerical order (rank ordering)

    • distances between values are numerically equal

    • zero is arbitrary (does not have true zero)

    • therefore , not possible to provide absolute amount of attribute

    • often psychosocial/psychological variables

      • ex. temperature

        • test scores

        • QOL

    • MCT = mode , mean, median

    • histogram

  • RATIO

    • continous ex. infinite #of values possible

    • can be palced in meaningful numerical order (rank)

    • distances between values are numerically equal

    • zero is absolute (true zero) - indicates absence of the attribute being measured

    • values cannot go below zero

    • therefore, possible to provide absolute amount fo attribute

    • often biologic and physical measures

    • ex.

      • height

      • weight

      • bp

      • pulse

      • age

    • MCT- mode, median, mean

    • histogram