Context, Sources, and Methods in Data Analysis

Context and Sources of Data

  • Example Data Context Analysis:

    • A representative dataset consists of a table containing two rows of data:
    • Row 1: The number of pleasure boats, measured in tens of thousands.
    • Row 2: The number of man deaths (manatee fatalities).
    • Analytical Questions Raised by the Data:
    • Given that the manatee is a slow-moving sea mammal, the primary question raised is whether manatee fatalities increase as the number of pleasure boats in an area increases.
    • Is there any relationship between the number of pleasure boats operating in a specific area and the number of manatee fatalities?

  • Evaluating Data Sources and Sampling Methods:

    • Prior to analyzing any data, it is critical to evaluate both the source of the data and the sampling method used to collect it.
    • Official Data Sources Example:
    • Florida Department of Highway Safety and Motor Vehicles
    • Florida Marine Research Institute
    • Sampling Method Evaluation:
    • Because the pleasure boat and manatee fatality data is obtained from official government records known to be reliable, the sampling method is assessed as sound.

Flawed Sampling Methods and Voluntary Response

  • Flawed Sampling Techniques:

    • Many sampling methods used in research and polling are fundamentally unsound.

  • Voluntary Response Poll Example:

    • A poll hosted on the website of a national donut chain illustrates a common flawed sampling methodology.
    • Characteristics of Voluntary Response Samples:
    • Respondents themselves decide whether or not to participate in the study or poll.
    • The resulting data is non-random because participants are entirely self-selected.
    • The sample is not representative of the broader population; for instance, a donut chain website poll will only attract people who feel strongly about coffee or donuts.
    • Methodological Conclusion: Voluntary response samples are not sound statistical tools and should not be used to draw population-level inferences.

Data Analysis Workflow and Statistical Execution

  • Sequential Steps in Data Analysis:

    1. Preparation: Consider the context, source, and sampling methods of the dataset.
    2. Data Exploration and Visualization:
    • Begin data analysis with an appropriate graph and initial data exploration.
    • Role of Graphs: Graphs allow analysts to determine the overall shape or distribution of the data and to identify outliers.
    • Outliers: Defined as specific data points that lie far away from all other data points in a set.
    1. Application of Statistical Methods: Apply formal statistical techniques to analyze patterns.

  • Requirements for Sound Statistical Analysis:

    • Computational Skills: Strong manual computational skills are not required because software tools such as Excel are used to perform the calculations.
    • Core Requirements:
    • Application of common sense.
    • Careful attention to sound statistical methods.
    • Careful and repeated reading of the analytical context and results.

  • Formulating Analytical Conclusions:

    • Every statistical analysis must culminate in a formal conclusion.
    • Conclusions must always be expressed in clear, simple terms.
    • Analysts must develop the ability to distinguish between statistical significance and practical significance.

Statistical Significance Versus Practical Significance

  • Statistical Significance:

    • Definition: Statistical significance is achieved in a study if the likelihood of an observed event occurring by chance is less than 5%5\% (i.e., P(event by chance)<0.05P(\text{event by chance}) < 0.05).

  • Practical Significance:

    • Definition: Practical significance assesses whether an outcome has meaningful, real-world usefulness, application, or value.
    • Diagnostic Assessment: To determine if a finding is practically significant, ask the practical evaluation question: "Would I buy this?" or "Could I sell this?"
    • Key Relationship: An event or finding can be statistically significant without possessing any practical significance.

Case Studies and Analytical Applications

  • Case Study 1: Probability of Birth Outcomes (Evaluating Statistical Significance):

    • Baseline Assumption: The probability of having a baby girl in any individual birth is 50%50\% (P=0.50P = 0.50).
    • Event 1: Obtaining 9898 girls in 100100 random births.
    • Analysis: This result is extremely unusual under the baseline assumption, with a very small probability of occurring purely by chance.
    • Conclusion: Event 1 is statistically significant.
    • Event 2: Obtaining 5252 girls in 100100 random births.
    • Analysis: This result is not unusual and could easily happen by chance.
    • Conclusion: Event 2 is not statistically significant.

  • Case Study 2: Diet Plan Commercial (Comparing Significance Types):

    • Claim: A commercial claims that customers in a study of a diet plan lost an average of 8 pounds8\text{ pounds} per year.
    • Statistical Probability: There is only a 2%2\% probability that customers would lose that much weight by chance without using the diet plan.
    • Financial Cost: The diet plan costs $600\$600 per year.
    • Statistical Significance Assessment: The diet plan outcome is statistically significant because the probability of achieving the results by chance (2%2\%) is less than the 5%5\% threshold (2%<5%2\% < 5\%).
    • Practical Significance Assessment: The diet plan is not practically significant. Most people would not spend $600\$600 a year with the expectation of losing only 8 pounds8\text{ pounds}.

  • Critical Evaluation of Daily Statistics:

    • Statistics are presented daily across news media and commercial advertisements.
    • Critical thinking is required when consuming statistical messages because statistics can often be misleading.