NSE 212 WEEK 5 DATA QUALITY AND LEVEL OF MEASURMENT

___ involves rules for assigning numeric values to qualities of people or objects

Measurment

No attribute inherently has numerical value T/F

True, we have to designate the quantity of the attribute being measured

Assessing reliability and validity of instruments is important for

Critiquing nursing research

Rigour is determined by

Determined by measurment instruments that reflect concepts being tested in a study

Focused on obtaining evidence of the quality of measurement tools to establish your reliability

Psychometric assessemnts

What is reliability

Extent to which a instrument yeilds the same results on repeated measures

Concerned with consistency, accuracy and precision of measure obtained of an concept

Reliability

A thermometer that gives the same correct reading time after time is an example of

Reliability

3 aspects of reliability

Stability, internal consistency (Homogeneity), equivalence

Test retest reliability is the stability of scores of an instrument when

Stability of scores of an instrument when administered more than once to same participants under similiar coniditons, can be 2 minutes apart, 2 weeks, 2 years

Does the instrument product the same results with repeated tested, is an example of

Stability - Test retest reliability

Trying to assess fairly stable attributes

Test retest reliability

Higher value of Cronbach alpha reflect

Higher internal consistency (0.7 and above) thus they are more accurate in terms of measuring the same characteristic

A cronbach's alpha of 0.5 would mean

Lower internal consistenct

A low reliability score indicates that items on the instrument are

Not well correlated and therefore item is unreliable which jeopardizes study findings

A researcher reports a cronbach's alpha of 0.95 for efficacy measurment scale, What form of reliability?

Internal consistency - Cronbach's alpha always indicates internal consistency

Disadvantage : Test- retest reliability (2)

Some attributes change over time E.g knowledge, stress, Participants can respond differently since they have experienced testing once

Parallel or alternate form reliability

This is where you use differently worded items when you’re retesting for test retest reliability

Internal consistency/homogeneity is asking if items on a survey

Do the items on a survey belong together and measure the same concept/characteristics? Items in a scale reflect measure the same concept.

Equivalence: Interrater reliability is asking does the tool

Would the tool produce the same results when equivalent procedures are used?

Internal consistency example

Depression scale - measures characteristics of depression

Test rest reliability advantage

Some traits we don't want to see consistency over time

This is the most commonly used test of internal consitency/homogenity

Cronbach's alpha: each item in scale is comapred with oteher items in scale and a total score is used to analyze data

Cronbach's alpha measures the ability to ensure that

It measures the ability to ensure that everything in your tool and all the items are measuring the same thing

Interrater or Interobserver Reliability is also called

Equivalence

consistency or agreement among 2 or more observers who use the same measurement tool

Equivalence (interrater reliability)

consistency or agreement among 1 observers who use the same measurement tool

Intra rater reliability

Equivalence has to do with the realibility of the ____ and not ___

It has to do with the reliability of the observer and not the instrument

Study on facial expressions of infants undergoing painful procedures in hospital, how can researchers estabilish interrater reliability

measuring the facial changes the same way according to the structured instrument, must be done consistently

Equivalence is used with strucutred instruments mostly used to observe

Used to observe behaviours

Reliability coefficient are used to describe

Reliability values in a numerical value, ranges form 0-1

Reliability coefficient ranges from

0-1

Often designtaed as r

reliability coefficient

The closer the reliability coefficient to 1 indicates

higher the reliability of measure

1 interrater reliability means

Perfect reliability among 2 observers

0 interrater reliability means

no reliability

A reliability coefficient of _____ or higher is considered acceptable

0.7

Internal consistency is a type of

Reliability

Refers to whether a measurment instrument measures what it intends to measure. (accuracy)

Validity

3 major kinds of validity (3c)

Content with subtyle face validity, criterion related, construct validity,

Used in development of measurments tool

Content validity: Determines wheter measurment tool and items in measure represent content reseaerch intends to measure

How is content validity carried out

Items in the tool are given to experts on the content and experts are asked to judge whether items reflect content or not

Content validity index (CVI) is used to measure

Central validity index is used to measure the relevance of each item based on expert's judgment (1-4)

CVI of 1 would indicate

Not relevant at all to content

2 types of criterion related validity

Predictive validity and concurent validity

Criterion related validity measures relationship between

Scores from the instrument with external criteria (gold standard), we want to see them related to gold standard measures and if they measure same attribute

Construct validity

Extent to which a test measures a theoretical construct of trait

Face validity

Subtype of content validity - does instrument appear to be measuring appropriate concept

How is face validity carried out

Colleagues are used to review content of instrument as to whether it reflects the concept the researcher intends to measure - based on judgment, not as accurate

With face validity, we report a stastical numver T/F

F, With face validity, you tend not to report any statistical number, not rigorous

Gold standard measures are

Ideal way to measure something, usually not achieveable due to fesiability and time restriants

Should we use face validity as an alternative to other validity

It’s not considered an alternative to other validity

Predictive validity

Correlate measure of concept and future meausre of same concept, helping predict future performance

Does pap smear predict cancer rates, does high school grades predict post secondary GPA

Predictive validity

Concurrent validity

Correlation of 2 measures of same construct administed at same time. High correlation indicates high agreement.

Pulse oximeter compared with arterial blood gas (gold standard)

Concurrent validity

What would be the gold stadnard of postpartium depression tool

Diagnosis by psychatrist

A measurement can be reliable but not valid

Researcher measures anxiety

Systematic error or constant error

Attributed to relatively stable characteristics of the study population that bias behaviour, who is entering my study and biases researcher may have

Social desirability

Bias where participants respond to what researchers want

Construct validity is the extent to which a test measures

Theory or theoretical construct or trait by testing different hypothesis relatinoships

Construct validity: hypothesis testing approach

Researcher develops hypothesis about behaviors of individuals with varying scores of a measure, then gather data to test hypotheses

Construct validity: Convergent

2 or more tools measuring same concept. E.g 2 depression tools will both measure depression

Construct validity: Divergent

Tool on one concept differentiates form another concept e.g tool on depression is different from tool on measuring stress

Construct validity: Contrast group/known group

2 known groups expected to score either high or low on measurement tool and they both fully use the tool, research then sees if we can distinguish E.g highly athletic vs not athletic uses tool regarding self perception of the athleticism

Construct validity: Factor analysis

Assessing stastically whether itesmc cluster around main dimension or main theoretical contrcut we want to assess

A measure can be reliable but not valid E.g

E.g research measures anxiety by measuring temperatures, however temperature would not be a valid indicator of anxiety

A valid instrument is always reliable E.g

A instrument that is erractic, inconsistent an dinaccurate cannot validly measure the attribute

We can't have validity without

Reliability

Reliable but not valid example

Always getting the weight, but it is off by 2 pounds each time

Valid but not reliable

Getting the correct weight one time, but not all the time. NOTE: If something is not reliabile that it is not truly valid

Not reliable not valid example

Never getting the right weight, getting different weight each time

Reliable valid

Getting the correct weigh each time and consistently

Measurment errors

difference between what exists in reality and what is being measured by an instrument

Measurment error: Random or chance

Unpredictable and difficult to control errors, cannot be corrected

Examples of random or chance error (4)

Transient human condition (hunger, fatigue), variance in measurement procedure (room temp, misplacement of BP cuff), error in data processing, scores varying in no particular direction

Measurment error: Systematic error or constant error (3)

Erorrs due to bias or improperly calibrated instrument, or something else being measured in addition to concept

Systematic error or constant error examples (3)

Socioeconomic status, weight scale not being calibrated, measurement error is not random thus measuring something else in addition to concept

Questions to ask when critquing data quality (3)

Appropriate methods, is reliability adequate (above 0.7), is validity adequate

LEVELS OF MEASURMENT

To identify levels of measurment in a research study

Measurment

Numbers assigned to variables or events based on a set of rules

Norminal measurement

Dichotomus (2 true values) or categorical (more than 2 values). Mutually exclusive (true or false) Researcher assigns number to each category but numvers are meaningless

mutually exclusive means if you have 1 fav color,

you can't have a second

Ordinal measurment

Categorical, variables given a number that can be placed in meanigful numical order

Example of ordinal measurment

Stages of cancer diagnosis, stage II is higher than 0 but we cannot say they are twice as severe as stage 0.

Ordinal measurement: do we know the distance between variables

No

Interval measurment

Variables are continuous and rank ordered. Equal intervals between numbers. Zero is arbitarily assigned but not absolute

level of measurement commonly used for psychological and psychosocial variables and tests

Interval because there is no true 0, just cuz someone scores 0 on IQ test doesn't mean they have no IQ

Example of interval

Temperature on centigrade scale. Difference btween 20 and 30 is and 30 to 40 is equivalent. Zero does not represent absence of temperature

What differentiates ratio and interval

Zero on interval data is arbitarily

Stastical potential for using interval level

We can caculate mode, average or mean

What is the highest lvel of measurment

Ration

Ratio measurment

Variables are continour and rank ordered and have absolute meanigful zero. Distance btween values are numerically equal

The levels of measurments helps determine

Type of stastitical analysis to use when analyzing data

What graph or chart to represent nominal and ordinal

Bar graph/chart

What graph to represent interval or ration

Histogram

Mneomic to remember levels of measurment

NOIR

Examples orf ratio level

many physical measures: Weight, height, hemoglobin levels, pulse rate, BP, age in years, time

The higher the level of measurment

The more flexibility in choosing stasticial procedures (interval and ratio) therefore maximum information can be obtained from data