week 2- quan

quantitative research

journal article

descriptions of studies published in professional journals, 10-20 pages double spaced

• manuscript reviewed by 2+ peer reviewers that make recommendations to editor

• reviews usually blind to researcher name

IMRAD format

structure of research articles

• Introduction

• Method

• Results

• and Discussion

journal title for qual vs quan

qual: includes phenomenon and group

quan: includes variables and population (PICO components)

abstract

brief description of study at beginning of article

components of introduction

• central phenomena, concepts, or variables

• purpose and research question/hypothesis

• review of related literature

• theoretical or conceptual framework

• significance of study

components of method

how the question is answered

quan:

• research design

• sampling plan

• how variables measured

• study procedure

• data analysis method

qual:

• research setting and context

• describe researchers’ effort to enhance study

purpose of results

present findings

components of results

all:

• descriptive information

• description of participants (age…)

quan

• name of statistical tests used

• value of calculated statistics

• statistical significance

meaning of statistically significant

findings are probably true and replicable

level of significance

index of how probably it is that the findings are reliable

p < 0.05 : 95/100 times the same results would be found with different samples

purpose of discussion

present conclusions about meaning and implications of findings

why research articles are hard to read

• compact

• jargon

• objectivity

  • avoid subjectivity so sounds bland

  • less inviting to read

• statistical information

critique

• peer reviewers

• evaluate strengths and weaknesses, research methods, conclusions and style

inference

conclusion drawn from study evidence using logical reasoning and taking into account methods used to generate evidence

reliability

accuracy and consistency of information

ex. not reliable thermometer if giving different readings in same time

validity

• concerns soundness

• whether methods are measuring what they are supposed to measure

• quality of evidence about relationship between independent and dependent variable

trustworthiness

• credibility, transferability, confirmability, dependability, authenticity, triangulation

credibility

achieved to extent that the research methods inspire confidence that the results are truthful and accurate

bias

• a distortion or influence that results in an error in inference

• caused by: researchers’ preconceptions, faulty methods of collecting data, participants’ lack of candor

systematic bias

• when bias is consistent or uniform

• ex. scale weighing everyone 2 pounds heavier than they actually are

research control

holding consistent influences on outcomes so that relationship between independent and dependent variables can be understood

confounding variables

• other factors/variables that lead to outcome that are not accounted for

• other determinants of the outcome

• must be controlled for and held constant

mediating variable

• a third variable that explains the process, mechanism, or "why" behind the relationship between an independent variable (the cause) and a dependent variable (the effect)

methods of bias reduction

1. randomness

2. blinding

randomness

• have features of study established by chance instead of researcher preference

blinding

• concealing information from participants, data collectors, or care providers

• enhances objectivity

reflexivity

• process of reflecting critically on the self and analyzing and noting personal values and beliefs that could affect data collection and interpretation

generalizability

in quan: the extent the findings can be applied to other groups and settings

transferability

• qual: extent findings can be transferred to other settings

• aspect of trustworthiness

• to increase: increase descriptive information

sampling plan

says how many participants will be selected

basic sampling plan includes:

• populations

• samples and sampling

• strata

population

the entire group of interest

eligibility criteria

• inclusion criteria and exclusion criteria

target population

entire population of interest

accessible population

portion of the target population that is accessible to the researcher

sampling

selecting cases to represent the population

sample

subset of population elements (usually human)

representative sample

one whose characteristics closely approximate those of the population

sampling bias

systematic overcompensation or underrepresentation of a population segment on a characteristic relevant to the research question

ex. too few or too may men in sample

strata

• subpopulations of population

• mutually exclusive

  • ex. nurses → RN group and RPN group

• can enhance representativeness

2 sampling designs in quan research

• nonprobability sampling

• probability sampling

non probability sampling

• researchers select elements by non random method

• less likely to be representative

• most nursing research is non probability

non probability sampling methods

• convenience sampling

• quota sampling

• consecutive sampling

• purposive sampling

convenience sampling

• select most conveniently available participants

• being convenient may be atypical for the research

• risk of bias

• weakest form of sampling, but most common

quota sampling

• researchers identify population strata and figure out how many people needed from each stratum

• ensures diversity

• convenience sample from each stratum (little bit better)

consecutive sampling

• recruiting all people from accessible population over specific time interval or for specified sample size

• best when there is “rolling enrollment"

purposive sampling

• using researcher’s knowledge about population to handpick sample members

• might choose who is knowledgeable about issue under study

• can lead to bias

• useful when researchers want experts

probability sampling

• random selection of elements from population

• includes random sampling and random assignment

random sampling

each element in population has equal, independent chance of being selected

random assignment

• signature of RCT

• no effect on how participants are selected

simple random sampling

• most basic probability sampling

• to create sampling frame:

  • list of population elements

    • elements in sample frame are numbered then table of random numbers or online randomizer used to draw random sample of desired size

• differences only due to chance

stratified random sampling

• population divided into 2+ strata then elements randomly selected

• aim to enhance representativeness

multistage sampling

• population first divided into 2+ strata from which elements are randomly selected

• aim to enhance representativeness

systematic sampling

• selection of every kth person on a list

• first person chosen at random

• essentially same as simple random sampling

which method to get representative samples

probability sampling

sampling error

• difference between population values and sample values

power analysis

• way for researchers to estimate how large their samples should be for testing hypotheses

• researchers must estimate how large group difference will be on a key outcome (based on prior research)

• if expected sizeable difference: large sample not needed to reveal group difference statistically

• small differences predicted: large samples necessary

response rate

number of people actually participating in a study relative to the number of people sampled

non response bias

differences between participants and those who declined to participate (also sometimes referred to as response bias)

quantitative critical appraisals

3 types of data often used by nurse researchers (quan)

• self report/patient reported outcomes (PRO)

• observation

• biomarkers

self report/patient reported outcomes

• participants responses to researchers questions

• ex. interview

• most common data collection approach

when is PRO used

when researchers know exactly what they need to know and can frame appropriate questions to obtain desired information

instrument

structured self report data are collected with a formal, written document

interview schedule

what the instrument is known as when the questions are asked orally face to face or by phone

questionnaire

what the instrument is known as when respondents complete the instrument themselves

closed-ended questions

• response options are specified

• question types

  • dichotomous

  • multiple choice

  • forced choice

  • rating question

• easier to analyze

open ended questions

• participants answer in their own words

• richer information

scale

device that assigns a numeric score to people along continuum, like a scale for measuring weight

psyhcosocial scale

used to measure attitudes, perceptions, and psychological traits such as anxiety or depression

Likert scale

• declarative statements

• respondents indicate how much they agree or disagree with statements

• positive worded statement: score 5 for strongly agree, 1 for strongly disagree

• negative worded statement: score 1 for strongly agree, 5 for strongly disagree

• total score is the summated rating scales/composite scales

response set biases

• scales are susceptible for problems:

  • social desirability response set bias

  • extreme response set bias

  • acquiescence response set bias

social desirability response set bias

tendency to misrepresent oneself by giving answers that are consistent with prevailing social views

extreme response set bias

tendency to consistently select extreme alternatives like strongly agree

acquiescence response set bias

• tendency of some people to agree with statement regardless of content

• opposite where people disagree with statements independently of question content (less common)

vignette

brief descriptions of situations to which respondents asked to react

visual analog scale

used to measure subjective experiences on continuum

q-sorts

present participants with set of cards that statements are written on. participants sort cards along specified dimension such as most helpful/least helpful

observational methods

• to gather info such as

  • condition

  • verbal communication

  • nonverbal communication

  • activities

  • environmental conditions

• human sense observations

• frequent in infants, children, impaired communication skills

structured observations

• use formal instruments and protocols that dictate what to observe, how long to observe, how to record

• document specific behaviours, actions, events

method of structured observation

category system

category system

• method of recording in systematic fashion the behaviours and events of interest that ranspire within setting

• can have systems that require all observed behaviours in specified domain be classified OR system where only particular types of behaviour are categorized

checklist

• instrument observers use to record observations

• formatted with list of behaviours from category system and space for tally

rating scale

instrument that requires observes to rate phenomena along a continuum

time sampling

time period must be selected during which observations will occur

• can be selected systematically or at random

event sampling

researchers elect integral events to observe

good for when events are infrequent and might be missed if time sampling used

types of biomarkers

• in vivo

• in vitro

• anthropomorphic

in vivo

• measurement within or on living organism

• ex. BP, temp

in vitro

• extracting biophys material from participants and then lab findings

• ex. chemical measures, microbio measures, biopsies…

anthropomorphic measures

• BMI and waist circumference

psychometrics

branch of psychology concerned with theory and methods of measurements of psychosocial phenomena like anxiety, depression, pain

2 focuses of psychometrics

reliability and validity

reliability

• extent free from measurement error OR extent scores for people who have not changed are the same for repeated measurements

• consistency

test-retest reliability

• replication takes form of administering a measure to the same people on 2 occasions

• assume traits have not changed (differences based on measurement error)

• when score differences across waves are small, reliability is high

interrater reliability assessment

• purpose: ensure consistency across multiple rates using sam einstrument

• 2+ observers independently applying the measure with same people to see if scores are consistent across raters

  • Cohen’s Kappa Score (indicates level of agreement)

  • re-train and/or revise instrument as needed

internal consistency

• In responding to a self-report item, people are influenced not only by the underlying construct but also by idiosyncratic reactions to item wording

• By combining multiple items with various wordings, item irrelevancies are expected to cancel each other out

• An instrument is said to be internally consistent to the extent that its items measure the same trait

• For internal consistency, replication involves people’s responses to multiple items during a single administration

• Whereas other reliability estimates assess a measure’s degree of consistency across time or raters, internal consistency captures consistency across items 

reliability coefficients

• .00 to 1.00 → higher values desirable

• .80 or higher considered acceptable

validity

• measurement context: degree an instrument is measuring the construct its purpose is to meausre

• measuring what its actual meant to be measuring

face validity

• if the instruments looks like its measuring the target construct from the surface

• not good evidence but helpful

content validity

• adequate coverage of topic

• extent an instrument’s content adequately captures construct

  • whether a composite instrument has appropriate sample of items for construct being measured

• assessed by having panel of experts rate scale items for relevance to construct and comment on need for revisions