5. CRITICAL APPRAISAL I: HYPOTHESIS, VARIABLES, LEVELS OF MEASUREMENT & DATA DISPLAY

Steps in EBDM Process

1. Ask: Ask an answerable clinical question

2. Acquire: Search for the evidence

3. Appraise: Critically appraise the evidence

4. Apply: Implement in practice

5. Assess: Evaluate the outcome

Critical Appraisal

the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context

If results are likely to be ….

• true

• Important

• Applicable to my patients

Hypothesis

a declarative statement that identifies a predicted relationship between X and Y variables in a study population

Flows from the research question, literature review, and theoretical framework

determines how data are collected, analyzed, and interpreted

Formulated before the study is started

Two types of hypothesis

Null/Statistical hypothesis (Ho)

Research/Alternative hypothesis (H1)

Hypothesis is

Testable: variables can be observed or measured

Theory base: consistent with existing theory and research findings

The variables of the hypothesis:

The population being studied

The predicted outcome of the hypothesis

Null hypothesis/ statistical hypothesis

States there is NO relationship between X (independent) and Y (dependent) variables

Alternative hypothesis

a statement that there is a relationship between variables

May be:

A directional research hypothesis predicts the expected direction of the relationship between X and Y.

A non-directional research hypothesis does not predict the anticipated direction of the relationship between X and Y.

1. the hypothesis might not be explicitly stated; it may be inferred.

2. Even when stated, the hypothesis should be re-examined in the results or discussion section.

3. Data analysis should answer the hypothesis.

4. The hypothesis should logically follow from the literature review and theoretical framework.

5. if hypothesis is directional, determine whether it is the appropriate direction.

6. The hypothesis should be stated in such a way that it can be clearly supported or not supported.

7. The way the hypothesis is stated suggests the type of research design that should be used and the level of the evidence.

Hypotheses are NEVER

PROVEN. The findings either support or do not support the hypothesis.

Variable

A trait or characteristic that is assumed to vary

e.g., systolic blood pressure (SBP)

Data

Values of variables when they vary

e.g., readings of SBP from different individuals

Dataset

A collection of these data values

Qualitative variables

Values that are non-numeric

Quantitative variables

Values that are numeric

can be either discrete or continuous

Discrete variables

Values that are countable but that do not assume numeric value between countable categories

Continuous variables

Variables that have every possible value on a continuum

Independent variable

The variable that is either manipulated by the researcher or that affects another variable

Dependent variable:

The variable that is affected by an independent variable and that becomes the outcome

Not manipulated by the researcher

Must be measurable

Extraneous Variables

uncontrolled variables that can adversely affect the results of an investigation, but are not the focus of the investigation

Levels of Measurement: Qualitative / Discrete Data

Nominal level of measurement

Ordinal level of measurement

qualitative levels of measurement: Nominal Level of Measurement

Data are classified into mutually exclusive categories and no ranking or ordering is imposed on categories

e.g., gender, ethnicity, religious affiliation, political party membership, and hair color

Dichotomous or Categorical

Nominal Level of Measurement: Dichotomous

two mutually exclusive values

e.g., yes/no, true/false

Nominal Level of Measurement: Categorical

multiple mutually exclusive values

e.g., single married, divorced, separated, or widowed

qualitative levels of measurement: Ordinal Level of Measurement

Data are classified into mutually exclusive categories, and ranking or ordering is imposed on the categories

e.g., grouped age (18 and younger/19–30/31–49/ 50 and older), letter grade, Likert-type scale, ranking in a race (first, second, third), and histological ratings

Levels of Measurement: Quantitative / Continuous Data

Interval level of measurement

Ratio level of measurement

quantitive levels of measurement: Interval Level of Measurement

Data are classified into categories with rankings and are mutually exclusive as in ordinal level measurement.

In addition, specific meanings are applied to the distances between categories.

There is no absolute zero, and the comparison can NOT be made in ratio form.

e.g., temperature and standardized tests such as IQ, SAT, ACT, TOEFL

quantitive levels of measurement: Ratio Level of Measurement

Highest degree of measurement

Data possess characteristics of the interval level of measurement.

there is an absolute zero and the comparison can be made in ratio form.

e.g., age, height, weight, income, blood pressure, pulse..etc.

Tool or instrument

A device for measuring variables

There are two important issues when a tool or instrument is used to measure variables:

Reliability + Validity

Reliability

Is a tool consistently measuring a variable of interest?

Types of reliability

Internal consistency

Test-retest reliability

Interrater reliability

Types of reliability: Internal consistency

Asks whether items within a tool that propose to measure the same thing do

Types of reliability: Test-retest reliability

Addresses the consistency of the measurement from one time of use to another time

Types of reliability: Interrater reliability

Asks whether different raters’ scores on a variable agree

Validity

Is a tool measuring what it is supposed to measure?

Types of validity:

Content validity

Criterion-related validity

Construct validity

internal validity

external validity

Types of validity: Content validity

Whether a measurement tool measures all aspects of a construct of interest

Types of validity: Criterion-related validity

How well a tool is related to a particular criterion

Types of validity: Construct validity

The extent to which scores of a measurement tool correlate with a construct testers wish to measure

Types of validity: Internal validity

Whether there is an uncontrolled or confounding variable that may influence the end results of a study

Types of validity: External validity

Whether the results of a study can be generalized beyond the study

grouping data: frequency distribution

One of the most common ways of presenting data

a display of possible values and corresponding frequencies.

can be either ungrouped or grouped

Ungrouped frequency distribution

Suitable for categorical, nominal, ordinal, and continuous measurements with a small range of data values

Grouped frequency distribution

Suitable for continuous measurements with a large range of data values

Graphs and charts

good ways of displaying and describing the data.

The goal is to find the best chart or graph that shows the data in a meaningful, clear, and efficient manner.

an important factor to determine which graphs or charts should be used.

Level of measurement

Bar chart

Discrete or categorical data

A typical chart that has response categories on the horizontal axis and
frequencies of each category on the vertical axis

Pie chart

Discrete or categorical data

A circular chart in which pieces in the chart represent a corresponding proportion of each category

Graphing Continuous Data

When the data are measured on continuous levels of measurement, both bar charts and pie charts become inefficient at displaying the data.

Better charts are histograms and box plots.

Histogram

organizes a group of data points into a number of intervals, and the bar
represents the frequency in corresponding intervals

Box Plot

Conveys more information than histogram, including overall distribution, the center of distribution, quartile, and any potential outliers

Good for comparison across groups

Box plot anatomy: Middle line

Median (50th percentile)

Box plot anatomy: Lower edge

First quartile (25th percentile)

Box plot anatomy: Upper edge

Third quartile (75th percentile)

Box plot anatomy: Ends of the vertical lines

Lower and Upper whiskers

Any data point below lower whisker or above upper whisker can be treated as an outlier

Line chart

Another choice for categorical data

Created by connecting dots that represent the data values of each category

Scatter plot

A good choice to examine relationship between two continuous variables