ACTIVITY 3: DRUG LITERATURE EVALUATION

To remain competent, trustworthy health care professionals must:

• Keep current with new advances

• Possess drug literature retrieval and evaluation skills 

• Able to carefully review and critique the literature

  • Have skill in identifying the strengths and limitations of the biomedical literature

CONTROLLED CLINICAL TRIAL

• Premier study design to measure and quantify differences in the effect of the intervention and control

CONTROLLED CLINICAL TRIAL

consist of 

an investigational (intervention) group being directly compared to a control group

CONTROLLED CLINICAL TRIAL

Consists of an investigational (intervention) group being directly compared to a control group (e.g. standard therapy, placebo)

• Most robust method to measure and quantify differences in effects between a therapy under study and the control group

• Proper interpretation of clinical trials is vital to providing appropriate health care

While the use of clinical trials to guide patient treatment has become the “norm,” it is important to remember that

not all clinical trials are of the same quality and relying on results from poorly designed clinical trials may not be in the best interest of patients

It is the obligation of the end user (in this case, the health care practitioner) to be able to distinguish

Thus, having strong literature evaluation skills is important for all health care practitioners involved in the patient care process.

Fundamental elements should be reported in all studies:

• Appropriate qualifications of the investigators conducting the research  

• Valid investigational methods

• Proper research techniques; and

• Appropriate analysis and interpretation of the results

Peer review process

involves sending a draft version of the study or article to a group of individuals with expertise on the topic or in the field under study

Peer review process
goal 

to reduce the publication of manuscripts that have inappropriate methods/design, are poorly written, and/or do not meet the needs of the journal’s readership.

Method that to evaluate the importance of a publication:

Impact factor

• number of times that articles published in a specific journal are cited by other articles

Method that to evaluate the importance of a publication:

Assess how many article retractions or errata are published - journals with a high number of retractions or errors may indicate

a less robust peer-review process compared to those with less retractions 

Major problem is that any significant amount of writing often involves a lot of

potentially frustrating work, because few people are natural writers 

The author must practice to

become proficient at writing, which will involve false starts, numerous drafts, roadblocks, and other problems

Writing exposes a person to

criticism and possible rejection

PUBLICATION TYPES

• journals 

• meeting abstracts 

• journal supplements

MEETING ABSTRACTS

• Research presented during a professional organization’s meeting, whether as a platform or poster presentation, (published abstract). 

• Usually undergo a peer-review process.

JOURNAL SUPPLEMENTS

• To publish a collection of articles related to a specific topic in a separate journal issue

• Venue for organizations to publish disease state practice guidelines

AUTHORSHIP

Other factors to evaluate in a published study

Investigators’ credentials

• with proper training and experience of the authors

• Declare any conflicts of interest with the research and outside interests

Practice site(s)

- should be a site that has the capability to perform the study

DRUG LIT ASSESSMENT QUESTION FOR CLINICAL TRIALS
overall assessment 

• Was the article published in a reputable, peer-reviewed journal?

• Are the investigator’s training/ education/practice sites adequate for the study objective?

• Can the funding source bias the study?

PARTS OF A CLINICAL LITERATURE

• The information in each section that you need to evaluate

TITLE

• Reflective of the work, unbiased, specific, and concise (usually ≲ 10 words) but not too general or detailed.

• Declarative sentences that tend to overemphasize conclusions are not preferred.

• RCT should be identified in the title if performed.

• Should include key words that are both sensitive (easing the task of locating the appropriate articles) and specific (excluding those not being searched for)

ABSTRACT

• A concise overview of the study or a synopsis of the significant principles of the article.

• It includes information addressing the article’s objective, methods, results, conclusions.

• It should be thorough, complete, and unbiased inwording selection

DRUG LIT ASSESSMENT QUESTION FOR CLINICAL TRIALS

Title/ Abstract

• Was the title unbiased?

• Did the abstract contain information not found within the study?

• Did the abstract provide a clear overview of the purpose, methods, results, and conclusions of the study?

DRUG LIT ASSESSMENT QUESTION FOR CLINICAL TRIALS

methods

• Was an appropriate study design used to answer the question?

• Were reasonable inclusion/exclusion criteria presented to represent an appropriate patient population?

• Was a selection bias present?

• Was subject recruitment described? If so, how were subjects recruited?

• Was the method appropriate?

• Was IRB approval obtained?

• Was subject informed consent obtained?

• Were the intervention and control regimens appropriate?

• What type of blinding was used? Was this type appropriate?

• Was randomization included? If so, what type was used? Was this appropriate?

• Who generated the allocation sequence, enrolled participants, and assigned participants to groups? Was this appropriate?

• Which ancillary treatments were permitted? Would they have affected the outcome?

• Was a runin period included? How does this affect the results?

• Did the investigators measure compliance? How was compliance measured? Was compliance adequate?

• Was the primary endpoint appropriate for the study objective?

• Were secondary endpoints measured? If so, were they adequate for what was being studied?

• Were subgroup analyses specified a priori? If so, were they appropriate?

• Was the method used to measure the primary endpoint appropriate?

• What type of data best describes the primary endpoint?

• Were data collected appropriately?

• How many patients were needed for the primary endpoint to detect a difference between groups (power analysis)? Was the necessary sample size calculated? Were there enough patients enrolled to reach this endpoint?

• What were the alpha (α) and beta (β) values? Were these appropriate?

• Were the statistical tests used appropriate?

DRUG LIT ASSESSMENT QUESTION FOR CLINICAL TRIALS
results

• Were the numbers of patients screened, enrolled, administered treatment, completing, and withdrawing from the study reported?

• Were reasons for subject discontinuations reported? Were withdrawals handled appropriately?

• Was the trial adequately powered?

• Were the subject demographics between groups similar at baseline? If not, were the differences likely to have an effect on the outcome data?

• Were data presented clearly?

• Were the results adjusted to consider confounding variables?

• Was intention to treat analysis used? Was this appropriate?

• Were estimated effect size, p values, and confidence intervals reported?

• Were the results statistically significant? Clinically different?

• Was the null hypothesis accepted or rejected?

• Were the numbers of patients screened, enrolled, administered treatment, completing, and withdrawing from the study reported? Were reasons for subject

• Can the trial results be extrapolated to the population?

• Based on the results, could a Type I or Type II error have occurred?

• Are subgroup analyses presented? Are these appropriate?

• Was ancillary therapy included? Did this affect the study results?

• Were therapy adverse effects included?

DRUG LIT ASSESSMENT QUESTION FOR CLINICAL TRIALS
Conclusions/ Discussion

• Did the information appear biased, and did the trial results support the conclusions?

• Were trial limitations described?

• Did the investigators explain unexpected results?

• Are the results able to be extrapolated to the population?

• Were the study results clinically meaningful?

The introduction tackles:

• study rationale

• study purpose/objective 

Study rationale

- refers to the set of reasons a researcher uses to justify the need to conduct another research on the chosen topic.

Study purpose/objective

• refers to the statement that helps readers assess the importance of the study relative to individual values.

  • It should include immediate and more extensive, eventual purpose.

After formulating the objective, formulate:

• research hypothesis

• null hypothesis 

Research hypothesis

- states the difference in the therapy under investigation and control.

Null hypothesis

- states no difference between the two groups.

A hypothesis may be missing in the introduction section and may be considered a deficiency but

does not mean that the paper contains unreliable information.

DRUG LIT ASSESSMENT QUESTION FOR CLINICAL TRIALS
introduction

• Did the authors provide sufficient background information to demonstrate the rationale for the study? What research gap does it fill?

• Were the study objectives clearly identified?

• What were the major null hypothesis and alternate hypothesis?

METHODS

• Design of the study is essential for the results to be valid.

METHODS
includes 

• types of subjects enrolled, 

• the comparative therapy description,

• outcome measures, and 

• statistics

Internal Validity

Quality of the study design

Internal Validity
application

Strong design should translate into reliable results

External Validity

Ability to apply results to  practice

External Validity
application

Study results meaningful to  practitioners and can be used for patient care

The methods section must thoroughly describe the

process by which the study was conducted, and it should be  written with sufficient detail to allow the study to be easily replicated by another investigator. 

STUDY DESIGN

Study questions dictate the study design:

STUDY DESIGN
For Controlled Clinical Trial 

• Prospectively measures a difference in effect between two or more therapies

  • Groups are similar and treated identically except for the treatments under observation.

  • Parallel design – subjects in the study are assigned to one of the groups and monitored

• Most rigorous method of establishing a cause-and-effect relationship between treatment and outcome.

Inclusion Criteria

Lists subject demographics that must be present for the subject to be enrolled in the trial

Exclusion Criteria

Characteristics that prevent a subject from enrollment in the trial or necessitate withdrawal from the study

The results of a controlled clinical trial should be

extrapolated to the patient type enrolled in the study.

Readers must be aware of the limitations of

surrogate endpoints and subgroup analysis results.

Surrogate endpoint

• a study measurement (lab value or physical assessment) that serves as a substitute marker for an actual clinical outcome (LDL levels for cardiovascular events)

Surrogate endpoint
example 

• if subjects HPN are the target group to be enrolled in a trial, HPN needs to be defined in terms of the min and max SB and DBP.

Clinical characteristics of study participants should reflect the disease under investigation, but the existence of

complex and/or extensive comorbid conditions(e.g. diabetes) may prevent an accurate measurement of differences in effect between the groups

SELECTION BIAS

• May be present 

• Occurs after subjects meet the inclusion criteria but are disqualified from taking part in the study. 

• Investigators prevent them from being enrolled because they may alter the results either positively or negatively

• A common form in the run-in phase (Lead-in phase)

RUN-IN-PHASE (LEAD-IN-PHASE)

• 2-4 weeks before being officially enrolled

• Can identify subjects who may or may not adhere to the therapy regimen, experience side effects from the therapy, or did not meet pre-specified criteria

  • They are excluded from participating even if they met the original inclusion criteria

  • Produces a bias by selecting a group who do not entirely represent the population

Intervention Group

Consists of the therapy under investigation (medication or procedure)

Control grp

Consists of either:

1. No therapy (placebo)

2. Another therapy (active control) 

3. Existing data (historical/retrospective data)

INTERVENTION AND CONTROL GROUPS

Both are to be as similar as possible in all respects (average age, no. of male/female, medication use, disease states)

other than the treatment received

Institutional Review Board (IRB) -

committee charged with ensuring the participants are protected and not exposed to unnecessary harm or unethical medical procedures

Informed consent form

•  participant is presented with a document to notify:

1. study procedures 

2. the rights and responsibilities of the participant 

3. the risks, benefits, compensation, and voluntary participation

4. right to withdraw without penalty

BLINDING

• A technique in which subjects, investigators, or both are unaware of who is in the intervention or control group

BLINDING

PURPOSE 

• to reduce the influence of bias on measuring a difference in effect between the intervention and control

No blinding (open-label)

Investigators and subjects are aware of the assignment to the intervention or control

Single

Either investigators or subjects, but not both, are aware of the assignment

Double

Both investigators and subjects are unaware of the assignment

Triple

In addition to both investigators and subjects unaware, data interpretation personnel (statisticians) is unaware of subject assignment as well

RANDOMIZATION

• All persons in a clinical trial who have an equal chance to be in the intervention(treatment) or control group

• Results are more dependable than nonrandomized trials

Primary Endpoint

• One effect caused by the intervention and control 

• The primary endpoint should be appropriate for the study purpose and measured using correct techniques and methods.

Secondary Endpoint

Routine and useful measure; not considered to be the primary purpose of the study

Composite Endpoint

• A combination of a group of endpoint measures into one primary endpoint 

• Consists of clinical outcomes directly related to morbidity and mortality as opposed to a pharmacological action (reduction in any incidence of stroke/MI/CV-related death vs. lowering cholesterol levels) 

• Aims to measure the overall effect of therapy

Data gathering will include

actual data collection period, follow-up, and monitoring of adherence.

Insufficient or inappropriate data collection methods and nonadherence usually lead to

biased results

CONSIDERATIONS FOR DATA GATHERING:

• The study should be conducted for an appropriate duration 

• Data need to be consistently collected throughout the entire trial

• Monitoring of the trial results at predetermined intervals is essential throughout the trial.

THINGS THAT SHOULD BE ESTABLISHED BEFORE DATA GATHERING PROCEEDS:

• The protocol for discontinuing the clinical trial earlier than scheduled 

• Data collection methods 

• Competency of trial personnels 

• Availability of data collection materials

FOR CCTs, MEASURING THE ADHERENCE TO THERAPY OF PARTICIPANTS IS VERY IMPORTANT, HERE ARE SOME WAYS TO MEASURE:

• Medication dosage unit counts 

• Serum drug levels 

• Regular follow-up communications (i.e., telephone conversations)

SAMPLE SIZE

• Refers to the number of subjects randomized into a study and is of considerable importance to the validity of the study results.

• appropriate sample size is vital for the study results to have any significant meaning; 

• conducting a power analysis is vital to determine a suitable sample size.

Sample size should not be determined based on

convenience, arbitrarily, or by the number of quickly recruited subjects

sample size id Dependent on the

expected magnitude of difference in the endpoint effect between the intervention and control

Large sample size

Needed to detect a small difference in effect between the intervention and control outcome

Smaller sample size

Needed to detect large differences between the two groups

STATISTICAL ANALYSIS 

• Means to analyze sample data and apply it to the population

• To collect sufficient evidence to reject H₀ in favor of accepting the research hypothesis (H₁) — new terminology may refer to this as failure to accept H₀

• Appropriate tests are selected based on the type of data that will be collected and analyzed

Nominal

• Categorical data

• Data placed in one category, but not more than one category, mutually exclusive

Ordinal

• Ranking

• Ordered

Interval

• Data with measurable equal distances between points

• No absolute zero

Ratio

• Data with measurable equal distances between points

• Presence of absolute zero

Inferential Statistics

• Used to conclude, based on the sample, for the application of the trial results to the population

• Used to determine if a statistical difference is present between the intervention and control groups

p-value

is calculated based on trial results and statistical tests; afterward, the p-value is compared to the alpha (α)-value established before to the beginning of the trial

Inferential Statistics

Selection of the statistical test depends on the

data being parametric (i.e., normal distribution) versus nonparametric

continuous data are assessed via

parametric statistics; common tests are Student’s t-test, analysis of variance (ANOVA), and analysis of covariance (ANCOVA)

Nonparametric tests are used for

nominal and ordinal data; examples are chisquare (χ2) and Mann-Whitney U test

Descriptive statistics

• Describe the characteristics of the sample and results for some studies

• Presented as measures of central tendency or variability, or both.

Chi-square test

Comparison of nominal data for independent groups (2x2)

Fischer’s exact test

Comparison of nominal data for 2 groups when expected frequency is <5

McNemar’s test

Comparison of nominal data for 2 matched or paired groups

Contingency table analysis (R x C)

Comparison of nominal data when there are >2 groups or >2 possible outcomes

Cochran Mantel- Haenszel test

Comparison of nominal data for multiple 2x2 tables

Wilcoxon Rank Sum test Mann-Whitney U test

Comparison of continuous data taken from 2 independent groups

Wilcoxon signed rank test

Comparison of continuous data taken from 2 paired groups

Kruskal Wallis test

Comparison of continuous data taken from >3 independent groups

Friedman’s test

Comparison of continuous data taken from >3 paired groups