HSCI 207 Quiz 2

Main Goals of Quantitative Research

1. Measurement

2. Establish Causality

3. Generalize findings

4. Replication

Measurement

Data are used to understand or quantify social phenomena, concepts, and their interrelations in general

Establishing causality (internal validity)

Researchers want to know what causes health and disease and/or social phenomena

Generalization

• Goal is to come up with law like findings that ply to large numbers of people (external validity)

• this is of particular concern for researcher using cross sectional and longitudinal design

• Experimental model research is concerned more with internal validity than external validity

• Having a presented sample is essential for generalization

Replication

• Provides a check for biases and routine errors

• If the findings are not the same as those of the original study, the comparison provides reason to re evaluate the methods and findings of the original study

• If the findings are the same, researcher have greater confidence in the original findings

Cross sectional studies: advantages

• If based on representative sample of the general population:

  • Highly generalizable

  • Provides estimates of population prevalence of disease and exposures or risk markers that can be used for program and resource planning

• Less costly, relatively quick, no follow-up required

• Can provide important directions for further research

  • Associations can evaluated further with other more rigorous study designs

Cross-sectional studies: limitations

• Usually cannot establish temporal relationship between exposure and outcome

  • Difficult to separate cause from effect

  • Exposure status at time of study may not be related to exposure at time disease began

• Series of prevalence cases will have a higher proportion of cases with disease of long duration than series of incident cases

  • Can’t tell if observed associations between exposure and disease is due to association of exposure with duration

• Rare condition cannot efficiently be studied using cross sectional studies because even in large samples there may be no one with the disease

Potential biases in cohort studies

• Biases from no response and losses to follow up

• Bias in assessment of the outcome

  • Particularly important if person who is assessing the outcome is aware of the participants exposure and hypothesis being tested

• Information bias

  • Most common in historical cohort studies where different information is obtained for exposed compared to non exposed person

When is a cohort study warranted

• When we have an idea of which exposures are suspected as possible causes of a disease

• When we can minimize attrition (losses to follow up) of the study population

When is a cohort study not warranted

• Lack of evidence to justify mounting a large and expensive study

• A cohort of exposed and nonexposed persons often cannot be identified

• Many of the diseases that are of interest today occur at very low rates

Cohort studies advantages

• Can establish population-based incidence

• Can examine rare exposures

• Temporal relationship can be inferred

• Time-to-event analysis is possible

• Magnitude of a risk factors effect can be quantified

• Selection and information biases are decreased

• Multiple outcomes can be studied

Cohort studies disadvantages

• Lengthy and expensive

• May require very large samples

• Not suitable for rare diseases

• Not suitable for diseases with long-latency

• Unexpected environmental changes may influence the association

• No response, migration and loss to follow up biases

• Sampling, ascertainment and observer biases are still possible

• Even though obtain data on exposure prior to disease diagnosis, exposure at baseline may not properly reflect a person cumulative exposure

• Sub-clinical disease may go undetected at baseline

Case-control studies: Advantages

• Relatively inexpensive

  • compared with prospective cohort studies

• Can study multiple exposures at once

  • Including investigations of interactions among exposures

• Can be conducted in relatively short time period

• Generally require relatively small numbers of cases and controls for study

Case-control studies: Limitations

• Not well suited to study weak associations

  • Hard to distinguish between a true weak associations and one due to bias

• If low participations rates

  • Often potentially differential response rates by exposure status for cases and controls leading to selection bias

• Misclassification of exposure

  • Recall bias, poor recall or other information bias

• For prevalence cases, nay be especially hard to establish temporality

• Finding appropriately representative case and control groups may be difficult

Placebos

• An inert substance that looks, tastes, and smells that the intervention agent

• Placebos play a major role in identifying both the real benefits of the agent and the its side effects

Problems with randomized studies

• Random allocation is difficult to achieve in practice

• There are ethical issues in withholding educational interventions

• it is virtually impossible to avoid contamination of a control or comparison area in a health promotion intervention

• It is ideologically unsound for health promotion to treat people as objects health promotion research requires individual and community participation

Nominal

Describes the concept in words, much like a dictionary definition

Operational

Describes how the concept is to be measured

Nominal level of measurement

• Type of scale allows a researcher to classify characteristics of the study population into categories

• Qualitative measure

• Least precise level of measurement

• Mutually exclusive

• No mathematical interpretation

Ordinal level of measurement

• The characteristics can be put into categories, AND the categories can be ordered in some meaningful way

• Rank ordered according to amount of characteristics the object possesses

• Mutually exclusive

• Distances between variables not equal across the range

Interval level of measurement

• Can be rank ordered and

• The actual value between values has come meaning

• Numbers have meaning, but no true zero point

Indicators

• Something employed to measure a concept

• Can be direct or indirect measures of the concept

• Tells us that there may be a link and indicate how strong that link may be

• Sometimes, one indicator for each concept is adequate

• Often, it is advantageous to use more than one indicator of one concept

Multiple indicators

• Reduces the likelihood of misclassifying some people because the language of a question is misunderstood

• Ensures the definition of the underlying concept is understood correctly

• Gets access to a wider range of issues related to the concept, allows the researcher to make finer distinctions

• Allows for factor analysis and cluster analysis

• Helps to weed out response sets

Coding Unstructured Data

• Derive codes: labels or titles given to the themes or categories

• Assign numbers to the codes

• Basic principles to observe

  • Categories must not overlap

  • Categories must be exhaustive

  • There must be clear rules for how codes are applied

Reliability

1. Stability over time

2. Internal reliability

3. Inter-observer consistency

Stability over time

• Whether the results of a measure fluctuate as time progresses, assuming that what is being measured is not changing

• Stability can be measured using the test retest method

• It is extremely difficult to quantify stability over time because of the number of factors that may come into play over the passage to time

Internal reliability (internal consistency)

• Whether multiple measure that are administered in one sitting are consistent

• this can be measured using Cronbach’s alpha coefficient or the split half method

  • These calculations are completed using statistical programs

  • A correlation of .8 or higher on a scale of 0-1 is generally accepted as minimum of internal reliability, although results with lower figures may still be used by some researchers

Inter-observer consistency

All observers should classify behaviour or attitudes in the same way

Measurement Validity

• Face validity

• Concurrent validity

• Construct validity

• Convergent validity

Face validity

Established if, at first glance, the measure appears to be valid

Concurrent validity

• Established if the measure correlates with some criterion thought to be relevant to the concept

• A lack of correlation brings some doubt onto the validity of the original measure

Construct validity

• Established if the concepts relate to teach other in a way that is consistent with the researchers theory

• This is confirmed by seeing that the results match what would be predicted given the theory

Convergent validity

• Establish if a measure of a concept correlates with a second measure of the concept that uses a different measurement technique

Advantages of Open Questions

• Allow for replies that the survey researcher might not have contemplated

• Make it possible to top the participants unprompted knowledge

• Salient of particular issues that respondents can be examined

• Can generate fixed choice format answers

• Enhances spontaneity

Disadvantages of Open Questions

• More time consuming

• Answers must be coded

• less convenient to compose an answer

• May require transcribing

• face interviewer variability

Advantages of Closed Questions

• Minimizes intra-interviewer variability and inter-interviewer

• May make it easier to understand question because the answers are provided

• Can be answered quickly and easily

Disadvantages of Closed Questions

• Loss of spontaneity and authenticity because relevant answers may be excluded from the choices provided

• Respondents may differ in their interpretation of the wording fixed responses

• Respondents may not find a fixed

Probability sampling

Uses random selection methods, associated with quantitative methods

Non-probability sampling

Does not use random selection methods, associated with qualitative research

Why sample

• Minimize cost

• Minimize data collection over time therefore reduce history threat

• Better access to subjects such as for the study and for future research

• Enhanced data quality such as focused efforts regarding recruitment types of interaction

Sources of Bias in Sampling

1. Not using a random method to pick a sample

2. The sampling frame (Human judgement that selects one group over the other)

3. Non-response (Some people in the sample fail to participate which skews the data

Sampling Error

• DOES NOT MEAN AN ERROR WHEN SELECTING A SAMPLE

• Errors of estimation that occur because there is a discrepancy between the sample statistic and the corresponding total population parameter are sampling errors

• Virtually impossible to eliminate sampling error tho

Four types of Probability Samples

• Simple random sample

• Systematic sample

• Stratified random sampling

• Multi-stage cluster sampling

Simple random sample

• Each element has the same probability of being selected

• To select a simple random sample

  • Devise a sampling frame

  • Number all the elements consecutively starting at 1

  • Pick a sample size (n) from the total population (N)

  • Use a random number table or computer program to generate a list of random numbers

  • The sample will be comprised of the cases whose element numbers match the randomly generated numbers

• Sampling ratio

  • n/N

  • (sample size = n, population size = N)

Systematic sample

• Selected directly from the sampling frame, without using random numbers

• i = size of sampling interval

• To being, choose a number at random from 1 to i

  • The number known as a “random start”

  • The case with a that number is the first case to be selected

• A potential problem with systematic sampling is periodicity

  • This occurs if the cases in the sampling frame are arranged in some systematic order

Stratified Random Sampling

• This type of sampling ensures that subgroups in the population are proportionally represented in the sample

• To select a stratified random sampling

  • Stratify the population

  • Select a simple random sample or a systematic sample from each stratum

• Using the procedure ensures that each stratum is proportionally represented in the total sample

Multi-stage Cluster Sampling

• Used for large populations

  • No adequate sampling frame

  • Elements are geographically dispersed

• It involves two or more stages

  • Selecting clusters

  • Then selection subunits within the clusters

• Issues with Multi-stage Cluster Sampling 

  • Technical complications

  • Cluster samples are usually stratified as well

Sample Error of the Mean

• Probability samples with sufficient sample sizes minimize the amount of sampling error, but some sampling error is bound to occur

• This sort of sampling error is measured by a statistic called the standard error of the mean

• About 95 percent of all samples means lie within 1.96 SE off the mean

Standard Error

• As sample size increased, the standard error decreases

• Standard error is the standard deviation of a sampling distribution

• Population Variance for a paramter/n

• SE = Standard Deviation/SQRT(n)

Sample Size

• The abosulte size of the sample matters

• As sample size increases, sampling error tends to decrease

• Each size increase cuts the sampling error by 1/2, then 1/3, then ¼

Issues with Sampling Size

• Non-response

  • The response rate is the percentage of the sample that participates in the study

  • If there is some particular issue common to the non-responders that brings them to differ in some important way from those who participate

• Heterogeneity of the population

  • Generally, the greater the heterogeneity of the population on the characteristic of interest, the larger the sample size should be

• Kind of analysis

  • The sample size needed may vary depending on what sort of analysis will be done

  • If small groups in the population are to be compared to larger groups, it may be necessary to oversample the smaller group in order to make meaningful comparisons

  • Certain statistical procedures, such as some multivariate analyses, require large sample sizes to work properly

Types of Non-Probability Sampling

• Convenience Sampling

• Snowball Sampling

• Quota sampling

Convenience Sampling

• Cases are included because they are readily available

• Problem: One cannot generalize the results to some larger population with any confidence

• Useful for pilot studies, for testing the reliability of measures to be used in a larger study, for developing ideas, learning how to do research

Snowball Sampling

• A form of Convenience sampling

• The researcher makes contact with some individuals, who in turn provide contacts for other participants

  • For example, students who participate in survey studies are asked to come up with the names of some non students who may be willing to participate

Quota Sampling

• Collecting a specific number of cases in particular categories to match the proportion of cases in that category in the population

• For example, there are quotas for people in certain groups such as age, gender, ethnicity, class

Strengths of Quota Sampling

• Cheaper and easier to manage compared to random sampling

• Can be conducted much more quickly than randomly sampling

• Good for pilot tests, exploratory research

Weakness of Quota sampling 

• Not likely to be represented 

• Judgement about eligibility may be incorrect

• It is not appropriate to calculate a standard error term from a Quota sample

Structured observation and sampling

• Often no sampling frame

• May involve time sampling

• May include place sampling

• May include behaviour sampling

Content Analysis Sampling

• Sampling Media

  • For example, a study of newspaper articles may involve sampling of different papers, or articles on a given topic

• Sampling Dates

  • For example, if researching media portrayals of sex workers, one could use a random method to select the years for which the media are to be analyzed

Limits to generalization

Even when a sample is selected using probability sampling, any findings can be generalized only to the population from which the sample are taken

Reducing Non-response

• For telephone interviews

  • Call backs are useful but sometimes several are needed

• For face-to-face contact

  • Dress appropriately

  • Be flexible to accommodate participants

• For mailed questionnaires:

  • Write a good covering letter explaining the reasons for the research

  • Make it personal by including the respondents name and address in the cover letter and personally signed

• How to reduce non-response in SFU course experience

  • Offer opportunity for bonus marks on final exams

Virtual Sampling Issues

• Major limitation of online surveys is that not everyone is online and has technical ability to handle these kind of questionnaires

• Many people have more than one email address

• Some households have on computer but several users

• Internet users are biased sample of the population

• Few sampling frames exist for the general online population

Assessing Sample Quality

• Is the sampling frame reported and open to scrutiny

• Are intra-class correlations and design effects provided

• Does it ensure coverage of small populations

• Is the sample size large enough tot permit estimates of the key perimeters

• Is the response rate high enough to have confidence about the representativeness of the above

• Is there information about non-responders

• How have the investigators tackled non-sampling error

• Is there information on measurement error and its estimation and control