Lecture 4.6 External Validity

_____ refers to the extent to which research findings can be applied to and taken as a measure of the target population.

Generalizability

Generalizability is largely determined by the selection of a(n) _____ and sufficiently _____ sample, as well as the overall thoroughness of the methodology.

Representative

Large

True/False:

The terms external validity and generalizability are used interchangeably.

True

Which of the following involves the study answering the question it claims to answer in the study sample?

A) Internal validity

B) External validity

A) Internal validity

Which of the following involves the absent or minimization of random and/or systematic errors?

A) Internal validity

B) External validity

A) Internal validity

Which of the following involves the study being generalizable to the population it claims to represent?

A) Internal validity

B) External validity

B) External validity

If the sample is not typical of the target population, there might be concerns on its _____.

Generalizability

Is the following scenario generalizable?

A new RCT on a statin drug included patients older than 18 years attempts to represent a target population of 18-93 year olds.

No

* Age distribution may not be well representative of that seen in real world patients

* Treatment may be ineffective and dangerous in geriatric patients

One solution for poor generalizability is _____ in which results are reported for each level.

Subgroup analyses

Which of the following is NOT a problem with subgroup analyses?

A) Smaller sample sizes in each subgroup

B) Studies are not "powered" for subgroup analyses

C) Randomization is not maintained in subgroups

D) Higher chances for confounding and bias

E) Increased randomization of patients in subgroups

E) Increased randomization of patients in subgroups

* No randomization into subgroups

When evaluating generalizability, researchers should list the _____ between the study sample and the clinical population, then assess the _____ of each one.

Differences

Relevance

Which of the following would NOT be something to consider for when evaluating generalizability?

A) Demographics

B) Patient consent status

C) Clinical characteristics

D) Regional and temporal differences

B) Patient consent status

Is the following scenario generalizable?

A new RCT on the pharmacological treatment of benign prostate hyperplasia (BPH) has a sample of 300 males within the range of 55-83 year olds, all residing in the state of Florida

Yes

* Sample looks reasonable

* BPH ONLY affects men so sample and population would be the same (includes majority of target patients)

* Differences with other populations EXIST, but may not alter the results

Is the following scenario generalizable?

A new survey on use of contraceptives in front of UF campus.

No

* Mainly college students

* Higher education

* Age range NOT representative

* Not generalizable BEYOND college students

Is the following scenario generalizable?

A new RCT on the efficacy of leukemia treatment in Finland with a sample of 200 Finnish patients (45% male) attempting to represent a population in Florida

No

* Race, socioeconomic status, education, diet, climate, and healthcare system between Finland and FL are very different

Which of the following consists of a SMALLER number of patients with MINOR variation in demographic and clinical parameters?

A) Explanatory (narrow) studies

B) Pragmatic (broad) studies

A) Explanatory (narrow) studies

* Specialized/controlled conditions

Which of the following consists of a LARGER number of patients with MORE variation in demographic and clinical parameters?

A) Explanatory (narrow) studies

B) Pragmatic (broad) studies

B) Pragmatic (broad) studies

* Normal conditions

Which of the following has high EXTERNAL validity (good generalizability)?

A) Explanatory (narrow) studies

B) Pragmatic (broad) studies

B) Pragmatic (broad) studies

Which of the following has a simple design?

A) Explanatory (narrow) studies

B) Pragmatic (broad) studies

B) Pragmatic (broad) studies

Which of the following occurs in mainly Phase IV trials?

A) Explanatory (narrow) studies

B) Pragmatic (broad) studies

B) Pragmatic (broad) studies

Which of the following occurs in mainly Phase II or III trials?

A) Explanatory (narrow) studies

B) Pragmatic (broad) studies

A) Explanatory (narrow) studies

Which of the following occurs has high INTERNAL validity?

A) Explanatory (narrow) studies

B) Pragmatic (broad) studies

A) Explanatory (narrow) studies

Which of the following occurs has a sophisticated design?

A) Explanatory (narrow) studies

B) Pragmatic (broad) studies

A) Explanatory (narrow) studies

Which of the following is narrow?

A) Explanatory studies

B) Pragmatic studies

A) Explanatory studies

Which of the following is broad?

A) Explanatory studies

B) Pragmatic studies

B) Pragmatic studies

Which is more important?

A) Explanatory studies

B) Pragmatic studies

C) Both are equally important

C) Both are equally important

If a study is not valid _____, the question of _____ is not relevant.

Internally

Generalizability

If a study IS internally valid, but not applicable to the population, its value is _____.

Limited