Class 9 (unit 3)

Systemic random Sampling

A sampling method where elements are selected from a larger population at regular intervals, starting from a randomly chosen point. This technique ensures a spread across the population.

Systematic random sampling example

Why is Systematic Random Sampling not a SRS:

• Not every group of five students is equally likely to be chosen.

• For example, it is not possible to select the first five students on the list as our sample

• Note: with this sampling method, every student does have the same chance of being chosen: 1/8

How to calculate the systematic random sample

• In this sampling procedure, we start with a numbered list of all N individuals in the population.

• To select a sample of n individuals, we randomly select a number, call it p, between 1 and k, where k= N/n

• Our sample then consists of the p-th individual on the list, and every k-th individual after that

What does the value k represent?

sampling interval

What are the advantages of systematic random sampling?

• A systematic sample is easy and fast to select

• When selecting a systematic sample, we don’t always need a list of the entire population

  • E.g. a manufacturer could sample every k-th item made on a production line

  • E.g. a cashier could sample every k-th customer making a purchase at a store

What are the disadvantages of systematic random sample?

We need to be careful of patterns in the population list

• Example: If we numbered the days of the year as 1 - 365, we should not select a systematic random sample with a sampling interval of 7

  • Our entire sample would consist of only one day of the week! (e.g. maybe the sample would consist entirely of Mondays)

What is the ideal type of sample?

census

what is a census

a “sample” of the entire population

• is always the ideal type of sample. However, it’s usually impossible due to time, cost, and logistical constraints

Leading questions

are questions that suggest a particular answer or influence the respondent's reply, often leading to biased results.

Example of leading question

• “Given that it would almost certainly increase prices for everyone, should the government raise the minimum wage?”

• “Given that a person working for minimum wage lives below the poverty line, should the government raise the minimum wage?”

Given that it would almost certainly increase prices for everyone, should the government raise the minimum wage?” (Leading question)

• The wording of questions can greatly influence the respondent! The results here were obtained using a leading question

• This survey will likely underestimate the true proportion of people who think the government should raise the minimum wage.

“Given that a person working for minimum wage lives below the poverty line, should the government raise the minimum wage?” (Leading question)

• Likely, a much higher percentage of respondents would answer “yes” to this question (even though it’s asking the same thing!)

• This survey will likely overestimate the true proportion of people who think the government should raise the minimum wage

What are some other things that might influence a respondent?

• The tone or brevity of the interviewer

• The interviewers race, or gender identity, or any other visible characteristics

• The respondent may be unable to recall the information requested, making their answer a “guess”

• The respondent may lie, to avoid revealing unflattering opinions or behaviours

What is nonresponse?

occurs when selected individuals do not participate in a survey or refuse to answer questions, leading to potential bias in the results.

What is nonresponse (class)

When respondents refuse to answer the questions

When is nonresponse often the case

• This is often the case with telephone surveys - people hang up

• This is usually an unavoidable source of bias: we must try our best to make non-response, a non-issue

• E.g. if you are conducting a telephone survey asking people about their job satisfaction, it would not be wise to only make calls on weekdays between 9:00 am and 5:00 pm

What is undercoverage?

occurs when certain members of the population are inadequately represented in a survey, leading to biased results.

what is undercoverage (class)

Results when some units in the population have no chance of being included in the sample

What is a example of undercoverage?

For example, phone surveys exclude a certain percentage of the population - those without phones

How to select a good sample?

• Select the sample in an unbiased and representative manner

• Proper interviewer training

• Use good (non-leading, easy to understand) wording of questions

• Do what you can to make non-response a non-issue

How to select a sample in an unbiased and representative manner?

• Include all population units in your possible sample

• Avoid voluntary response and convenience samples

• If you can do something better than a SRS (e.g. making the sample more representative of the population), then do it!

Experimental design (what we know up to this point)

An observational study simply measures variables on an individual. An experiment deliberately imposes some treatment on an individual, in order to observe their response to the treatment

• We have seen in several examples that an observational study does not allow us to establish the cause of an observed difference in the value of some response variables for different groups

We will now dive deeper into how to design “good” experiments

Experimental design (Suppose that we find that those who drink more coffee, get less sleep. Can we say conclude that co!ee causes people to get less sleep?)

No!!!! (remember, association does not imply causation!!)

• There may be a lurking variable present - for example, maybe subjects that drink a lot of co!ee also tend to have high stress levels, which are causing them to sleep less. We cannot know for sure!!

Confounding

Confounding occurs when the effects of two or more variables are mixed together, making it difficult to determine the individual impact of each variable on a response. This can lead to incorrect conclusions about causal relationships.

Confounding (class)

We say that two variables their effects on the response cannot be separated

How is confounding an issue with?

observational studies

What do we have to do to make an accurate representation without confounding?

really want to know whether drinking coffee causes a person to sleep less, we have to perform a proper experiment

What do we have to be careful of when designing an experiment

make sure it is serving its intended purpose

What are we imposing in a experiment?

a change in a way that ensures that the results we observe are truly a response to that change, and not to any lurking variables

What is experimental units?

The individuals on which the experiment is being performed are called

What is subjects

If the individuals are people, they are called

What is treatment?

specific set of experimental conditions applied to the units is called

What is the purpose of a experiment?

Is to observe the response of one or more variables to changes in other variables. Thus the distinction between explanatory and response variables is absolutely necessary

Factors

The explanatory variables in an experiment

Factor levels

The different values of the factors

Treatment (factors)

The combination of factor levels applied to a unit

Example

Response variable:

Experimental units:

Factors:

Factor Levels:

Treatments:

Response variable (example doctor)

Cholesterol

Experimental units (example doctor)

the patients

Factors (example doctor)

Diet type and exercise frequency

Factor levels (Example doctor)

Omnivore, vegetarian, 0 times per week, 1 –3 times per week, 4+ times per week

Treatments (example doctor)

There are six treatments in total:

• Omnivore/0 times per week

• Omnivore/ 1 –3 times per week

• Omnivore/ 4+ times per week

• Vegetarian/0 times per week

• Vegetarian/ 1 – 3 times per week

• Vegetarian/ 4+ times per week

How to calculate number of treatments

The product of the numbers of factor levels for each factor

What is required to be a proper experiment?

Teatment groups need to be formed randomly.

• Groups formed by randomization don’t depend on any characteristics of the experimental unit

• When treatment groups are formed randomly, the groups should be relatively homogeneous (i.e. similar) prior to the experiment.

what is the distribution between groups?

its the treatment they receive

How can we conclude experiment

Then if there is a significant difference in cholesterol levels among the groups, we can conclude it was because of the treatment those units received (we eliminated the effect of any lurking variables!!!)

What is another advantage of experiments?

we can simultaneously examine the effect of several variables on the response variable

• This allows us to examine any interaction that might be present among the factors

Example of examining several variables

However, when taken together, these medications might interact and cause an adverse reaction!

• This is why it is very important to examine the e!ects of both factors simultaneously in one experiment

Selecting units

It is typically not possible to select the experimental units randomly

• This is especially true in the case of human subjects: people must volunteer to participate in the experiment

• Even though we may not be able to select units randomly, what is important is that we can view the units as representative of the population they come from

what do we need for an experiment to be legitimate?

we need to have an element of control: we need to control the environment to eliminate the effect of any potential lurking variables

What is one method of control?

comparison of responses to the various treatments

Comparison (method of control)

• For the comparison to be legitimate, the groups receiving each treatment must be similar with respect to all other variables

• This was achieved by forming the treatment groups randomly

Completely randomized design (CRD)

experiment where all units are randomly assigned to receive the various treatments

• The cholesterol example was an example of a CRD

Drawing a experimental design

creating a plan that outlines the structure of an experiment, including treatment groups and randomization methods

Example (placebo effect)

Unlikely!!!

• We can’t know for sure if the elimination of pain is because something helpful is being done, or because Oliver perceives something helpful is being done. (Placebo effect)

placebo effect

Dummy treatment that is known to have no physical effect. It may, however, have beneficial psychological effects.

When are placebo effects most common?

The placebo effect often occurs when we have conducted an observational study rather than a proper experiment

Example (2) - placebo effect

This may occur for many reasons:

• The pill may be working

• Perhaps other things in the patients life have become less stressful

• The placebo effect: patients may believe that the pill should be helping, and so they perceive it to be doing so

If we want to know if the pill is really working, i.e. responsible for causing the decrease in anxiety, we would need to perform a proper experiment

How could we set up an experiment to see if the pill is working?

To do so, we could:

• Randomly divide our patients into two groups

• Give one of the groups the actual medication, and the other group a placebo (a “sugar pill”)

• We compare the stress levels for the two groups after one year

• If the medication group’s stress has decreased significantly more than that of the placebo group’s, then we can conclude that the medication is working

In the pill experiment we set up are all lurking variables eliminated?

Have been eliminated, and the only systematic di!erence between the groups is the type of medication

What is the group that recieved the placebo is called

the control group

what is a control group

A group which we compare our treatment of interest. It can be a group for which no treatment (or a “fake” treatment) is received. It can also be a group to which a standard (known) treatment is assigned

Why do we use control groups?

to eliminate the possibility that an observed change is due to something other than the factors being studied.

Is a control group a way to achieve control in an experiment?

yes

When is a control group nessary?

when we have no other means of comparison, i.e. there’s only one treatment

When is a control group not necessary?

If there are multiple treatments, we can achieve control by comparing the response to the various treatments

are there any other problems with the pill example?

he doctor administering the pills does know whether each patient is in the treatment group or the control group. The doctor could unintentionally say/do something that may give clues to the patient about which group they’re in.

How do we avoid this unintentional telling to the patient

Double bind experiment

double blind experiment

This is an experiment where neither the subject nor the person administering the treatment knows which treatment is the one being applied.

How does the double blind experiment work with the pills?

The doctor gives the pill bottle to the nurse, who does not know whether it contains the actual medication or a placebo, and the nurse gives the pills to the patient

What is another form of control

Blinding

What is a biased experiment>

If the methodology used systematically favours certain outcomes

Replication

Is the administration of each treatment to more than one unit

What is another thing to keep in mind when designing experiments?

Sample size. If an experiment is performed on more units, our results will be more trustworthy

Three principles of experimental design

• Randomization

• Control

• Replication

Randomization (three principles)

Of the Experimental units to the various treatments

Control (three principles)

the effects of lurking variables on the response. Control can be achieved by:

• Comparing responses to various treatments, and/or

• including a control group, and/or

• conducting a double blind experiment

• Blocking

Replication (three principles)

the assignment of each treatment to several individuals to reduce variation in the results