Chapter 7 Healthcare Informatics

Susan White

sample

sample

subset of population or universe

universe

set of all units eligible to be sampled

sampling frame

list of all the subjects in the universe

sampling plan

includes a definition of the population, any inclusion or exclusion criteria, and the sampling methodology.

probability sampling

each member of a population has a known probability of being selected for the sample

non-probability sampling

is that which members of a sample are deliberately selected for a specified purpose. This should be utilized if the goal of an analysis is to gain understanding of a process or if EDA is conducted.

probability sampling includes…

simple random sampling, stratified random sampling, systematic sampling, and cluster sampling

non probability sampling includes….

quota sampling, convivence sampling, and judgement sampling

When is probability sampling appropriate?

If the goal of the analysis is is to generalize results of the analysis to the full population.

What is a key part of a statistically valid sample?

must be large enough to provide information about with sufficient precision to meet the goals of the analysis. typically all participants have an equally likely chance of being selected. This sample should also be reproduceable.

What do the centers of medcare and medicaid services (CMS) define as the following documentation for genearting a sample for the purposes of performing an audit

3.10,4,4.41: Documentation of universe and frame

(rev.282, Issued 1-08-09, Effective 1-26-09, implementation 1-26-09

An explicit statement of how the universe is defined and elements included shall be made and maintained in writing. Further, the form of the frame and specific details as to the period covered, definition of sampling units, identifiers of the sampling units (eg claim numbers and carrier control numbers) and dates of service and source shall be specified and recorded in your record of how the sampling was done. A record shall be kept of the random numbers acutally used i the sample and how they were selected. Sufficient documentation shall be kept so the sampling frame can be recreated, should the metholdology be challenged. (CMS 2011)

Simple random sampling

every member of the population has an equal chance of being selected from the sampling. The sample can be chosen through random drawing or by numbering the population and making the choice through random number tables and or a random number generator. If a random generator is used, than a seed should be set so the sample can be replicated.

Random seed

every random number generator has a starting point, called a random seed. If the seed is designated and recorded as part of a sampling plan, then the series of random numbers can be replicated by another analyst using the same software and seed. The steps in selecting a simple random seed are

1)Assign a sequential number as a row label to every unit in the sampling frame

2)Select a series of random integers between 1 and n, where n is the sample size or

1)assign a random number to each unit in the sampling frame

2)Order the units by the random number

3)Select the first n as the sample, where n is the sample size

(this second option works well with excel, you can use the RAND () function)

systematic random sampling

is a simple random sample that may be generated by selecting every 5th or every 10th member of the sampling frame.

stratified random sampling

the population is divided into smaller groups or strata based on a set of criteria. Each unit in the population must be assigned to UNE stratum. Therefore, the strata don’t overlap. This sampling uses unique statistics techniques.

When should stratified random sampling be used?

it’s appropriate to use when there are subsets of the population that must be included in the analysis or natural seperations of the data. Also the strata are weighted.

Cluster random sampling

the population is divided into groups before the sample is selected. As in stratified random sampling, the groups or clusters must be mutually exclusive and exhaustive. Every unit in the population is assigned to ONE cluster.

How many stages can cluster sampling be performed in and what do those comprise of?

Single-stage or two-stage. In single-stage cluster sampling, clusters are selected in random and and all of the units are included in the sample. Two-stage clustering- the clusters are selected at random and then the units within the randomly selected units within the cluster are also chosen randomly.

When should cluster sampling be used?

This is helpful when a sampling frame containing the entire population isn’t available, but natural groups of the population are available for selection. Care should be taken to ensure samples are homogenous.

Convenience sampling

The sample is chosen as the name implies-by convenience. Volunteers may be utilized but all applicants are chosen. Other types of convivence samples are ones most readily available or that are cheap. -not representative of enture population

Judgement sampling

completed when a researcher with expert knowledge of theh subject being studied, and chocies are made based on researchers knowledge base. (example, analyzing claim histories from a data report)

Quota sampling

segments the population into mutually exclusive groups, as is done in stratified random sampling. Then judgement sampling is used to select a number of individuals for the sample. This sampling is not a probability sample because judgement is used to select groups. This type of sampling is used to determines satisfaction with food in a hospital cafeteria.

What is some biases that can occur with quota sampling?

reserachers select samples at random, but are more baised to approachable and positive people.

How are sample sizes used for an audit?

The office of inspector general (OIG) created a free stats program called RAT-STATS that can be used for this purpose.

When did the OIG create RAT-STATS?

late 1970s

What does the RATS and RAT STATS mean?

The RATS stands for Regional Advance Techniques Staff (this tech was developed in SF, CA)-this technology is a package of stats software tools to help determine sample sizes and evaluate audit results.

When was the latest edition of RAT STATS made?

2019

What types of studies can RAT STATS help with?

The RAT-STATS Software supports determining an appropriate sample size for two types of studies

1)Attributes-studies where the variable of interest is a rate or proportion. Examples include MS-DRG change rates, coding accuracy rates, or complication rates

2)Variable-studies where the variable costs of interest is a quantity measured on an interval or ratio scale. Examples include payment error, length of stay in a specialty unit, or wait times in an emergency department.

What are the parameters RAT STATS has for selecting a sample for an attribute study?

1)Confidence level- The results of the study are typically observed as a confidence interval. For example, a 95% confidence interval for the complication rate is 5%. Select the desired confidence level. Click select all to observe the impact of the confidence level on the required sample size.

2)Anticipated Rate of Occurrence: Selection of a sample size requires some knolwedge of the rate to be correct. The width of the confidence interval for a rate is a function of the rate derived from the sample. Sample size-determination in any study requires some educated guess on the end results. This is sometimes derived from a smaller probe study, previous studies, or rates found in research literature. The most conservative value is to use for an attribute value study is 50%, but 50% should only be used i know additional information is available. This value will result in a larger sample size.

3)Universe size-Number of units in the universe

4)Desired precision range- This is the target width of the confidence interval to be derived from the sample. A wider confidence interval will require a smaller sample size. A narrower confidence interval will require a larger sample size.

Is sample size often a compromise?

Yes- between desired confidence level, precision, and the information available for an audit. There are no published guideliens for the correct confident levels of precision. The OIG does reccomend that confidence levels are of 90% and precision of 25%. This applies to providers who are in a corporate integrity agreement.

How does RAT STATS include sample size determinations for variable studies?

-For this demonstration, an unrestricted variable study using a probe sample with no probe file, will be used.

Selecting a sample for a variable study requires the selection of the following parameters.

1)Probe sample format- The probe sample is the smaller three-study that will be used to determine the sample size. As with attribute studies, the confidence interval is determined by the standard deviation of the sample. The probe sample will supply an estimate of the standard deviation that may be used to determine the sample size. In this example, no probe sample file is designated. The user is prompted to the mean and standard deviation from the probe sample in subsequent steps.

2)Confidence level-Desired confidence level for the reporting of the results.

3)Precision-width of the confidence interval to be reported.

4)Universe size- number of units in the population

PROBE SAMPLES ARE REQUIRED FOR VARIABLE STUDIES DETERMINATIONS WITH THE RAT STATS SOFTWARE

OIG SUGGESTS A PROBE SAMPLE OF 50 UNITS, BUT THERE IS A PROBE SAMPLE SIZE MAY BE DRIVEN MORE BY BUDGET AND TIME CONSTRAINTS.