Week 3: Missing data Reading

Module 2

What does null data mean?

A value is missing or unavailable.

[Missing Data Reading] What are common names for missing data?

Null, NA, and NaN.

[Missing Data Reading] What are the two main missing-data strategies discussed?

Masking and sentinel values.

[Missing Data Reading] What is a mask for missing data?

A Boolean structure identifying missing values.

[Missing Data Reading] What is a sentinel value?

A special value used to represent missing data.

[Missing Data Reading] What does NaN stand for?

Not a Number.

[Missing Data Reading] What is None?

A Python object often used to represent missing data.

[Missing Data Reading] What dtype can result when NumPy contains None?

object

[Missing Data Reading] Why can object dtype be slower?

Operations occur at the Python-object level.

[Missing Data Reading] Why can None cause arithmetic errors?

Arithmetic between numbers and None is undefined.

[Missing Data Reading] What type of value is NaN?

A floating-point value.

[Missing Data Reading] What happens to 1 + np.nan?

It returns NaN.

[Missing Data Reading] What happens to 0 * np.nan?

It returns NaN.

[Missing Data Reading] What can happen when normal aggregations include NaN?

Results may become NaN.

[Missing Data Reading] What does np.nansum() do?

Calculates a sum while ignoring NaN values.

[Missing Data Reading] Does integer data have a normal NaN integer value?

No; NaN is floating-point.

[Missing Data Reading] How does Pandas treat None and NaN?

Nearly interchangeably in many situations.

[Missing Data Reading] What can happen to integer data when NaN appears?

It may be converted to float.

[Missing Data Reading] What does isnull() do?

Returns True where values are missing.

[Missing Data Reading] What does notnull() do?

Returns True where values are present.

[Missing Data Reading] What kind of result does isnull() produce?

A Boolean mask.

[Missing Data Reading] What does dropna() do to a Series?

Removes missing entries.

[Missing Data Reading] What does dropna() do to a DataFrame by default?

Drops rows containing any missing value.

[Missing Data Reading] Can dropna() remove columns instead of rows?

Yes.

[Missing Data Reading] How do you tell dropna() to work on columns?

Use axis="columns".

[Missing Data Reading] What does how="any" mean?

Drop if any value is missing.

[Missing Data Reading] What does how="all" mean?

Drop only if all values are missing.

[Missing Data Reading] What does thresh control?

Minimum number of non-missing values required to keep data.

[Missing Data Reading] What does fillna() do?

Replaces missing values.

[Missing Data Reading] What does fillna(0) do?

Replaces missing values with zero.

[Missing Data Reading] What is forward fill?

Fill missing values using the previous available value.

[Missing Data Reading] What is back fill?

Fill missing values using the next available value.

[Missing Data Reading] What does ffill stand for?

Forward fill.

[Missing Data Reading] What does bfill stand for?

Back fill.

[Missing Data Reading] Can forward fill leave a missing value unchanged?

Yes, if no previous value exists.

[Missing Data Reading] Why is filling data not automatically correct?

The replacement introduces an assumption.

[Missing Data Reading] Why might dropping data not be ideal?

It can remove useful observations.

[Missing Data Reading] What question should you ask before handling missing data?

Why is it missing, and what assumption is reasonable?