Statistical Foundations – Data, Tables, Visualisation, Central Tendency & Dispersion

Unit III – Statistics & Data Collection

4.1 Introduction

• Everyday statements (e.g. “I walk on an average 4 km/day”) reflect statistical thinking.

• Humans naturally collect observations (data) and seek patterns to survive & make decisions.

• Key issues raised:

  • How reliable are our conclusions?

  • Can we use structured, scientific methods? → Statistics provides that framework.

4.1.1 What is Statistics?

• Science of collecting, classifying, analysing data to draw conclusions.

• Tools employed are Statistical Methods.

• Term derives from German “Statistik”, Italian “Statistica”, Latin “Status” (originally data for the State).

4.1.2 Functions of Statistics

• Represent facts quantitatively → Removes vagueness of words such as “high/low”.

• Summarises complex masses of figures.

• Facilitates comparisons (e.g. country population contextualised by area, density).

• Enables forecasting (production planning, inventory, climate, elections).

• Supports policy formulation (budgets, taxation, resource allocation).

4.1.3 Scope of Statistics

• Applied to virtually all sciences & business fields; notable domains:

  • State administration (population, crime, education, military).

  • Business & global trade (data mining, forecasting consumption).

4.1.4 Limitations of Statistics

• Not for individual measurement; deals with aggregates.

• Requires quantifiable characteristics; qualitative traits (honesty, affection) excluded.

• Results are valid ‘on average’, not deterministic cause–effect.

• Only one approach among many; complements other methods.

• Susceptible to misuse (biased data collection, selective reporting). Learning stats helps detect such misuse.

4.2 Basic Statistical Concepts

• Data: plural of datum; numerical facts collectively.

• Population: complete set of objects/individuals under study (not just human beings).

• Sample: selected subset used for economical, practical study.

• Characteristic: attribute/quality investigated (e.g. rainfall).

• Variables vs Attributes:

  • Variable: measurable characteristic → quantitative values.

  • Attribute: non-measurable, qualitative (religion, nationality).

4.3 Collection of Data

Primary vs Secondary

• Primary Data: collected first-hand for specific purpose.

• Secondary Data: previously collected by others; reused.

Methods of Primary Data Collection

• Direct personal observation/interview (accurate but costly).

• Indirect oral investigation via witnesses.

• Mailed/emailed questionnaires (wide reach, low cost, poor response risk).

• Enumerators with schedules (market research; quality depends on training).

Sources of Secondary Data

• Published: Government departments (CSO, NSSO, Census), international agencies (UNO), local bodies, committee reports, journals/newspapers, research institutes.

• Unpublished: company records, academic theses, internal reports.

• Limitations: unknown procedures, relevance, bias, accuracy, adequacy.

4.4 Classification & Tabulation of Data

• Classification groups similar items; objectives: condense data, remove details, aid comparison, prepare for statistics.

• Tabulation: systematic arrangement in rows/columns. Parts of a table: number, title, headnote (units), captions, stubs, body, footnotes, data source.

• Advantages: simplification, comparison, computation, space-saving, easy graphing.

• Good table requirements: clear objective, readability, logical order, rounding rules, highlighting, totals/margins, splitting if oversized.

• Types by characteristics: Simple (one-way), Two-way, Manifold (three-way+). Numerous solved examples illustrate filling blank cells.

Unit IV – Diagrams & Graphs

Importance

• Visual, attractive, time-saving, comparison, universal language.

• Limitations: low precision, vague for experts, supplement—not substitute—for tables.

Main Diagram Types

1 . One-Dimensional (Bar)

• Simple, Multiple, Sub-divided, Percentage bars.
  2 . Two-Dimensional (Pie/Circular).

Interpretation skills: reading enrollments, trade figures, family budgets; examples given with Q&A.

Graphs for Frequency Distributions

• Histogram: rectangles on class boundaries; equal class width assumed.

• Frequency Polygon: join mid-points; easier comparison of >1 distribution.

• Frequency Curve: smooth free-hand version of polygon.

• Ogive (Cumulative Frequency Curve):

  • Less-than ogive: plot cumulative frequencies vs upper class boundaries.

  • More-than ogive: plot vs lower boundaries.

  • Intersection of both ogives gives median; also used for quartiles, percentiles.

  • Important approximate relationships: Median from N/2 line; Q1 at N/4; Q3 at 3N/4.

Illustrations: drawing histograms, polygons, curves; locating median and mode graphically via intersections and rectangle tops.

Unit IV – Measures of Central Tendency

Definition & Purpose

• Single representative value around which data cluster; ‘first-order measures’.

• An ideal average: rigid, simple, based on all items, algebraically manageable, stable in sampling, not unduly affected by extremes.

Types

1 . Arithmetic Mean (AM)
2 . Median (Md)
3 . Mode (Mo)
4 . Geometric Mean (GM)
5 . Harmonic Mean (HM)

Arithmetic Mean

• Ungrouped: $\bar{x}=\frac{\sum x}{n}$

• Discrete: $\bar{x}=\frac{\sum f x}{\sum f}$

• Continuous: use class mid-points $x_m$.

• Weighted mean: $\bar{x}_w=\frac{\sum w x}{\sum w}$.

• Combined mean for $k$ groups: $\bar{x}=\frac{\sum n<em>i \bar{x}</em>i}{\sum n_i}$.

• Merits: rigid, easy, uses all data, algebraic, least sampling fluctuation.

• Demerits: sensitive to extremes, not for open‐ended classes, impossible with missing items, may not be an actual data value.

Median (Positional)

• Arrange data; odd $n$: middle; even $n$: mean of two middles.

• For grouped data (median class where cumulative freq ≥ $N/2$):
  $M=l<em>1+\frac{\frac{N}{2}-cf}{f}(l</em>2-l_1)$

• Merits: unaffected by extremes, usable for open classes & qualitative scores.

• Demerits: ignores many values, not algebraically tractable.

Quartiles

• Divide distribution into four equal parts.

• Grouped formulas:
  $Q<em>1=l</em>1+\frac{\frac{N}{4}-cf}{f}(l<em>2-l</em>1)$
  $Q<em>3=l</em>1+\frac{\frac{3N}{4}-cf}{f}(l<em>2-l</em>1)$

• Graphical extraction from ogives.

Mode

• Most frequent value.

• For grouped data (modal class highest $f1$): $Z=l</em>1+\frac{(f<em>1-f</em>0)}{(2f<em>1-f</em>0-f<em>2)}(l</em>2-l_1)$

• Graphical via histogram by drawing diagonals atop modal rectangle.

• Merits: easy, unaffected by extremes, can use open end classes.

• Demerits: not unique, unstable in samples, ignores many values.

Unit IV – Measures of Dispersion

Need

• Averages hide variability; dispersion quantifies spread.

• Absolute measures (original units) vs Relative measures (dimension-free coefficients for comparison).

Range

• $R=L-S$; coefficient $\frac{L-S}{L+S}$.

• Simple, but uses only extremes → unstable.

Quartile Deviation (Semi-Interquartile Range)

• $Q.D.=\frac{Q<em>3-Q</em>1}{2}$; coefficient $\frac{Q<em>3-Q</em>1}{Q<em>3+Q</em>1}$.

• Ignores middle 50 % extremes; less sensitive than range.

Mean Deviation (Average Deviation)

• About mean: $MD=\frac{\sum |x-\bar{x}|}{n}$

• About median similar; coefficient $\frac{MD}{\text{central value}}$.

• Uses all observations but ignores algebraic signs → less tractable.

Standard Deviation & Variance

• Best general measure; incorporates all data & algebraic signs.

• Population variance $\sigma^2=\frac{\sum (x-\bar{x})^2}{n}$; SD $\sigma=\sqrt{\sigma^2}$.

• For frequencies: $\sigma=\sqrt{\frac{\sum f x^2}{\sum f}-\left(\frac{\sum f x}{\sum f}\right)^2}$.

• Step-deviation (change of origin) for large numbers: $\sigma=c\sqrt{\frac{\sum f u^2}{N}-\left(\frac{\sum f u}{N}\right)^2}$ with $u=\frac{x-A}{c}$.

• Coefficient of variation $CV=\frac{\sigma}{\bar{x}}\times 100\%$ → compares relative consistency; smaller CV → more uniform.

• Empirical rule (Normal approx.):

  • $\mu \pm \sigma$ covers 68.27 % observations

  • $\mu \pm 2\sigma$ covers 95.45 %

  • $\mu \pm 3\sigma$ covers 99.73 %

Properties & Limitations of SD

• Rigidly defined, algebraically tractable, least sample fluctuation.

• Unchanged by adding/subtracting constant; multiplied if all data scaled.

• Sensitive to extreme values (advantage or drawback depending on context).

Quick Reference of Key Formulas

• $\bar{x}=\frac{\sum x}{n},\qquad \bar{x}_w=\frac{\sum wx}{\sum w}$

• $M=l<em>1+\frac{\frac{N}{2}-cf}{f}(l</em>2-l_1)$

• $Q<em>D=\frac{Q</em>3-Q_1}{2}$

• $Z=l<em>1+\frac{(f</em>1-f<em>0)}{2f</em>1-f<em>0-f</em>2}(l<em>2-l</em>1)$

• $MD=\frac{\sum f|x-C|}{\sum f}$ (C = chosen centre)

• $\sigma=\sqrt{\frac{\sum f x^2}{\sum f}-\left(\frac{\sum f x}{\sum f}\right)^2}$

• $CV=\frac{\sigma}{\bar{x}}\times 100\%$

Here are some questions from the notes that could be formed for 5 marks each:

1. What is Statistics? Define Statistics and elaborate on its five main functions as discussed in the notes. Provide examples where applicable.

2. Limitations of Statistics: Discuss the key limitations of statistics. Why are statistical results often valid only on 'average' and not deterministic?

3. Data Collection Methods: Differentiate between primary and secondary data, and describe at least three methods of primary data collection. What are the limitations of secondary data?

4. Classification & Tabulation: Explain the objectives of data classification and list the essential parts of a statistical table. What are the advantages of tabulating data?

5. Measures of Central Tendency: Define 'measures of central tendency' and explain the concept of an 'ideal average'. Briefly describe Arithmetic Mean, Median, and Mode, highlighting one merit and one demerit for each.

6. Graphical Representation: Discuss the importance of diagrams and graphs in presenting data. Describe two main diagram types and explain how a Histogram and an Ogive are constructed and what information they can provide (e.g., median from Ogive intersection).

7. Measures of Dispersion: Explain why measures of dispersion are needed in addition to measures of central tendency. Briefly describe Range, Quartile Deviation, and Standard Deviation. Which is considered the best general measure and why?