NumPy · Pandas · Matplotlib – Complete Bullet-Point Reference

Python Data Analysis & Visualization – Comprehensive Notes (NumPy · Pandas · Matplotlib)

1. Library Imports & Basic Setup

• Always begin with the canonical imports:
• import numpy as np
• import pandas as pd
• import matplotlib.pyplot as plt
• Optionally add:
• import math for $\text{floor}$ & $\text{ceil}$
• np.random.randn(r,c) to generate random $N(0,1)$ arrays.
• Windows‐style CSV paths illustrated multiple times:
• 'C:\\Users\\User-pc\\Desktop\\emp.csv'

2. Pandas Series – Creation Methods

• From Python list → pd.Series([90,67,88])
• From list w/ explicit index → pd.Series([20,30,15],['first','sec','third'])
• Scalar broadcasting → pd.Series(10,index=['A','B','D','F'])
• NumPy array w/ default / custom index → pd.Series(np.arange(10,20,3),index=[14,17,20,1])
• Dictionary → pd.Series({'Corbett':'Uttarakhand','Sariska':'Rajasthan'})
• Repetition operator demo:
• seq=['-4','55']; ser=pd.Series(seq*2) multiplies list * 2 then vector ops work element-wise.

3. Series Indexing & Slicing (Label vs. Positional)

• Positional slicing $\textbf{Series}[start:end:step]$
• s[1:5:2], s[-1::-2], s[:-4:-2]
• Label slicing via loc (inclusive end)
• s.loc['b':'f'], s.loc[['rubber','scale']]
• Purely positional via iloc
• s.iloc[2:5], s.iloc[[0,2,3]]
• Single-element fast access:
• .at[label] and .iat[pos]
• Negative-step demos: s1[::-2] prints every 2nd element backwards.

4. Boolean Filtering & Assignment

• Boolean mask → s[s>12], s[s.isnull()].
• Compound logic → df[(df['Age']>27) & (df['Age']<31)].
• Masked assignment:
• s[s==s.max()]=50 → replace max.
• s[s%2==0]*=2 doubles even entries.
• .where(cond,other) example replaces values $\le 15$ with $25$.
• Input-driven update:

  item=input('Enter item')
  if item in s:
      s[item]=int(input('Enter number'))

5. Series Attributes & Meta-Data

• Size / shape / dimension:
• s.size, s.shape, s.ndim
• Data volume:
• s.nbytes (bytes), s.hasnans (bool)
• Descriptive counts:
• s.count() = non-NaN elements
• len(s)-s.count() gives NaN count.
• Sorting & heads/tails:
• s.sort_values(ascending=True).head(3) → smallest 3 values.

6. Arithmetic Between Series (Automatic Alignment)

• Different indices → unmatched labels ⇒ $NaN$.

  s1 = pd.Series([39,41,42,44],['A','B','C','D'])
  s2 = pd.Series(10,['A','BB','D','F'])
  s3 = s1 * s2  # only A & D multiply

• .mod, .add, .sub, etc. obey alignment.

7. DataFrame Creation Techniques

• From 2-D NumPy array:

  arr=np.array([[10,20,30],[15,25,35]])
  df=pd.DataFrame(arr,columns=['C1','C2','C3'],index=['r1','r2'])

• List of dict (L.o.D) automatically unions keys → $NaN$ for missing:

  lod=[{'a':10,'b':20},{'a':5,'c':30}]
  df=pd.DataFrame(lod)

• Dict of Series aligns on index intersection.
• Nested dict – outer keys become columns, inner keys rows.
• Append / concat:
• df1._append(df2,ignore_index=True) (alias of deprecated append).
• pd.concat([df1,df2],axis=1,ignore_index=True) for column-wise join.

8. Core DataFrame Accessors

• Column: df['col'] or df.col (if no space).
• Row slicing: df[0:3], df.iloc[0:3], df.loc['r1':'r3'].
• Cell selectors:
• .iat[row_i,col_i], .at[row_lbl,col_lbl].
• Example: df.at[3,'Mark2']=50.
• Mixed selection:
• df.iloc[2:4,[1,3]] – rows 2-3, columns 1 & 3.

9. Adding / Modifying Columns & Rows

• Column math:
• df['Total']=df[['Mark1','Mark2']].sum(axis=1)
• df['comm']=df['sales']*0.10
• .insert(loc,col,value) or .assign(newcol=series) for placement.
• Row addition:
• df.loc[len(df.index)]=[...] or df.at[3]=[...].
• Mass update via masks:
• df.loc[df['Dept']=='Physics','Score'] += 10
• Renaming:
• df.rename(index={'N':'North'},columns={'rating':'Rating'},inplace=True).
• Re-indexing reorder:
• df.reindex(index=[2,0,1],columns=['B','A']).

10. Deleting Data

• Rows: df.drop([1,2],axis=0,inplace=True) or by label list.
• Columns: df.drop(['Name','Mark1'],axis=1), del df['Name'], df.pop('col').
• Series pop/drop: s.pop('sec') / s.drop(['a','b'],inplace=True).

11. Aggregation & Descriptive Statistics

• Aggregate on axis:
• df.count(axis=0) → non-NaN per column.
• df.count(axis='columns') → per row.
• Multi-agg: df[['Mark1','Mark2']].aggregate(['sum','count']).
• GroupBy:

  df1=df.groupby('Name')['Total'].sum()

• Custom map to grades:

  def grade(v):
      if v>=102: return 'A'
      elif v>=100: return 'B'
      elif v>90: return 'C'
  df['grades']=df['Total'].map(grade)

• Mode / median / mean examples demonstrated for physics dept.

12. Missing-Value Handling

• Detect: .isnull(), .isna(), df[df.isnull().any(axis=1)].
• Replace:
• df.fillna(method='pad') (forward-fill) or 'bfill'.
• Series: s[s.isnull()]=100.

13. Sorting & Ranking

• df.sort_values(by=['Science'],ascending=True).tail(1) to get max.
• Multi-column sort with individual orders: df.sort_values(['Salary','Score'],ascending=[True,False]).

14. File I/O (CSV)

• Export: df.to_csv('dff1.csv') or s.to_csv('rubb.csv').
• Import with options: pd.read_csv('emp.csv',nrows=10,skiprows=1).

15. Visualization with Matplotlib / Pandas

• Line & bar directly via DataFrame: df.plot(kind='bar').
• Sub-plots grid:

  plt.subplot(2,2,1); plt.plot(names,English)
  plt.subplot(2,2,2); plt.plot(names,IP)
  plt.subplots_adjust(hspace=0.4,wspace=0.4)

• Multiple bar clusters using width offset $\Delta x$ = $0.25$.
• Histogram weight-binned: plt.hist(wtofmuffins,bins=[50,100,...]) with grid.
• Customization: labels, title, legend location 'lower right', ticks via plt.xticks().

16. NumPy Array Essentials

• Creation & properties:
• arr.shape, arr.ndim, arr.size, arr.itemsize, arr.nbytes.
• Slicing identical semantics to lists/Series.
• Concatenation: np.concatenate([Ar1,Ar2],axis=0).
• Vectorised arithmetic: arr*2, arr+4.

17. Misc. Python / Math Utilities

• math.floor() vs. math.ceil() examples inc. negatives.
• ord() / chr() for ASCII comparisons in Series filters.
• zip(a,b) pattern for simultaneous iteration.
• Repetition operator on strings & lists: 'hello'*3, [1,2]*2.

18. Mini-Case Studies Recap

• House Series: update last two elements by $+10\%$ using negative slicing House[-1:-3:-1]*=1.10.
• Vehicle dataframe (vdf) end-to-end tasks: mask update, add comm, row index rename, re-order, etc.
• Teacher pay & department analytics: median, mode, salary increment.
• Student marks marksUT: groupby total & grade mapping.
• Sales multi-year (Sales df): add 2017 col, append row, drop col/row, total per row.
• Plotting wait-time histogram & scoreboard bar charts, gender scores grouped bars.

19. Common Error Patterns Highlighted

• Scalar vs. list mismatch ⇒ ValueError: Length of passed values ….
• Index mismatch arithmetic returns $NaN$.
• .loc requires label, .iloc requires integer position.
• .Series([1,2,3,4]**2) illegal – use NumPy first then cast.
• .plot(kind='bar',x='name') fails if column missing.

20. Formulas & Code Patterns to Memorise

• Percentage calculation: $\text{percent}=\dfrac{\text{sum}}{\text{total max}}\times100$
• Even/Odd conditional assignment:

  s[s%2==0]*=2; s[s%2!=0]*=3

• Bar-offset coordinates:
$x,\;x+0.25,\;x+0.5,\;x+0.75$ for 4-bars per group.
• CSV round-trip skeleton:

  df.to_csv('file.csv'); df2=pd.read_csv('file.csv')

• Grade function w/ map or apply adapting thresholds.

These bullet points compile every demonstrated concept, syntactic variant, example output, and typical pitfalls covered in the transcript, forming a stand-alone reference for NumPy-Pandas-Matplotlib exam revision.