Pandas Data Handling Lecture Notes

Comprehensive flashcards covering Pandas basics, Series and DataFrame structures, manipulation, selection, merging, and file I/O based on the provided lecture transcript.

What is Pandas?

A package useful for data analysis and manipulation that provides easy ways to create, manipulate, and wrangle data.

Which two major data structures does Pandas provide?

$Series$ and $DataFrame$.

How does Pandas handle missing data?

Pandas is designed to easily handle missing data values.

What is the dimensionality of a Pandas $Series$?

One-dimensional ($1D$).

What is the dimensionality of a Pandas $DataFrame$?

Multi-dimensional (specifically two-dimensional).

What does a data structure allow for in Pandas?

It allows for arranging data so it can be accessed quickly for operations like retrieval, deletion, and modification.

Which three data structures are dealt with in Pandas?

$Series$, $DataFrame$, and $Panel$.

What makes a $Series$ special compared to a standard array?

Its index attribute, which has incredible functionality and is heavily mutable.

What are the two parts of a $Series$?

The Data part (array of actual data) and the Associated index (array of indexes or data labels).

Is the data in a $Series$ mutable?

Yes, the data of a $Series$ can be changed.

Is the size of a $Series$ mutable?

No, the size of a $Series$ is always immutable and cannot be changed.

What are the row labels in a $Series$ called?

$Index$.

What syntax is used to create a $Series$ object?

$\text{<Series Object> = pandas.Series(data, index=idx)}$, where index is optional.

Which four types of data can be used to create a $Series$?

Python sequences (Lists), $ndarray$, scalar values, or Python dictionaries.

What is the default index for a $Series$ of length $4$ if no index is specified?

$0$, $1$, $2$, and $3$.

How is a $Series$ created from a scalar value?

An index must be provided, and the scalar value will be repeated for the length of that index.

When creating a $Series$ from a dictionary, what do the keys become?

The keys of the dictionary become the indices of the $Series$.

What happens if you perform a mathematical operation like multiplying a $Series$ by $2$?

Every value in the $Series$ is multiplied by $2$. (Vectorized operation).

What is printed if non-matching indexes are found while adding two $Series$?

$NaN$ (Not a Number).

How can you handle non-matching indexes during $Series$ addition to avoid $NaN$?

The non-matching index value can be filled as $0$ before performing the operation.

What is the purpose of the $head()$ function in a $Series$?

It is used to access the first $5$ rows of the $Series$ by default.

What is the syntax to access the first $3$ rows of a $Series$ named $s$?

$s.head(3)$.

What is the purpose of the $tail()$ function in a $Series$?

It is used to access the last $5$ rows of the $Series$ by default.

What code is used to retrieve the last $4$ rows of a $Series$?

$\text{series\_name.tail(4)}$.

Which three methods does Pandas provide for row/column selection in a $Series$?

$loc$, $iloc$, and the square bracket operator $[]$.

What is the syntax for using $loc$ in a $Series$?

$\text{series\_name.loc[StartRange : StopRange]}$.

Is the $StopRange$ inclusive or exclusive in $loc$ slicing for labels?

It is inclusive.

What is the purpose of $iloc$?

It is used for selection using integer-based index labels.

What is the syntax for using $iloc$ in a $Series$?

$\text{series\_name.iloc[StartRange : StopRange]}$.

In $iloc$ slicing, is the $StopRange$ inclusive or exclusive?

It is exclusive.

How do you access a value at a specific index using the $[]$ operator?

$\text{series\_name[index]}$.

How do you get or set the index attribute of a $Series$?

Using the $.index$ attribute.

What is the general syntax for slicing a $Series$?

$\text{SERIES\_NAME[start : end : step]}$.

What do 'start', 'end', and 'step' represent in a slice object?

'start' is the first item, 'end' is the last item, and 'step' is the increment between items.

Define a Pandas $DataFrame$.

A two-dimensional object representing data in rows and columns, similar to a spreadsheet or SQL table.

What is the row index axis number in a $DataFrame$?

$axis=0$.

What is the column index axis number in a $DataFrame$?

$axis=1$.

Does a $DataFrame$ contain homogeneous or heterogeneous data?

Heterogeneous data.

Is the size of a $DataFrame$ mutable?

Yes, size is mutable.

Is the data within a $DataFrame$ mutable?

Yes, data is mutable.

What are four common sources used to create a $DataFrame$?

$Series$, Lists, Dictionaries, and $Numpy$ $2D$ arrays.

What is the default column name when creating a $DataFrame$ from a single $Series$?

The default column name is $0$.

How can you perform row-wise iteration on a $DataFrame$?

Using the $iterrows()$ function.

How can you perform column-wise iteration on a $DataFrame$?

Using the $iteritems()$ function.

What code access the column 'empid' as a subscript?

$df['empid']$ or $df.empid$.

How do you select multiple columns (e.g., 'empid' and 'ename') from a $DataFrame$?

$df[['empid', 'ename']]$.

How do you rename the columns of a $DataFrame$ named $df$ to 'List1'?

$df.columns = ['List1']$.

How do you create a new column 'List2' with all values as $20$ in $df$?

$df['List2'] = 20$.

How do you create a column 'List3' as the sum of 'List1' and 'List2'?

$df['List3'] = df['List1'] + df['List2']$.

What are three methods to delete a column from a $DataFrame$?

$del$, $pop()$, and $drop()$.

What is the syntax to delete 'List3' using the $del$ keyword?

$del\,df['List3']$.

How do you delete a column using the $pop()$ method?

$df.pop(\text{'List2'})$.

What argument must be passed to $drop()$ to delete a column?

$axis=1$.

What code deletes rows with index $2$ and $3$ using $drop()$?

$df.drop(index=[2, 3], axis=0)$.

What is the basic syntax for accessing a subset via labels using $loc$ in a $DataFrame$?

$df.loc[StartRow : EndRow, StartColumn : EndColumn]$.

In $DataFrame.loc$, what does passing a colon $:$ by itself indicate?

It indicates selection of all rows or all columns respectively.

Which method is used to access groups of rows/columns based on numeric index values?

$iloc()$.

By default, what do $df.head()$ and $df.tail()$ return?

The first $5$ rows and last $5$ rows respectively.

What syntax returns the $3rd$ to $4th$ row of a $DataFrame$ if indexing starts at $0$?

$df[2 : 4]$ (Note: transcript indicates $df[2:5]$ for $2nd$ to $4th$ in one example).

What is 'Boolean Indexing' in a $DataFrame$?

Selecting data from a $DataFrame$ using a boolean vector.

What Pandas function is used to combine series or dataframes along an axis?

$pd.concat()$.

What is the default axis for $pd.concat()$?

$axis=0$ (concatenates along rows).

In $pd.concat()$, what does $join='outer'$ represent?

A union of indexes from the objects being concatenated.

In $pd.concat()$, what does $join='inner'$ represent?

An intersection of indexes from the objects being concatenated.

What does the $ignore\_index=True$ argument do in concatenation?

It discards the original index values and labels the resulting axis as $0, \dots, n-1$.

What function is used to join two $DataFrames$ linked by a common feature/column?

$merge()$.

What does the 'how' argument in $merge()$ define?

It defines the type of join (left, right, outer, inner).

Which join combines all records from both $DataFrames$ and fills NaNs for missing matches?

Full Outer Join ($how='outer'$).

Which join produces only those records that match in both $DataFrames$?

Inner Join ($how='inner'$).

What is a Right Join?

A join that produces all records from the right $DataFrame$ and matching records from the left $DataFrame$. (Missing left matches result in $NaN$).

What is a Left Join?

A join that produces all records from the left $DataFrame$ and matching records from the right $DataFrame$. (Missing right matches result in $NaN$).

What suffixes are appended by default in a merge when column names overlap?

$x$ and $y$.

Which arguments are used to perform a join on the indexes instead of columns?

$left\_index=True$ and $right\_index=True$.

What does CSV stand for?

Comma Separated Values.

How are records and fields typically separated in a CSV file?

Fields are separated by commas and individual rows (records) are separated by newlines.

Which function is used to read data from a CSV file into a $DataFrame$?

$pd.read\_csv()$.

Which function is used to export a $DataFrame$ to a CSV file?

$df.to\_csv()$.

When merging columns with different names, which arguments specify the local keys?

$left\_on$ and $right\_on$.

Who are the creators listed for these Pandas notes?

Sachin Bhardwaj PGT(CS) KV No1 Tezpur and Vinod Verma PGT(CS) KV OEF Kanpur.

What website is referenced for more updates in the notes?

Python4csip.com.

Is the size of a $Series$ mutable?

No, it is immutable.

If you calculate the square of a $Series$ values, what notation is used?

$s \times s$ or $s^2$ (represented as $s**2$ in Python code).

What is the axis value to delete data row-wise in $drop()$?

$axis=0$.

What happens when you select a single column using $df.empid$?

It returns a $Series$ containing the data for that column indexed by the row labels.

Can a $DataFrame$ be created from a list of dictionaries?

Yes.

Which Pandas data structure is similar to a $3D$ array?

$Panel$ (though noted as not in syllabus).

What is the default value of the $axis$ parameter in $pd.concat()$?

$0$.

What is the default value of the $how$ parameter in the $merge()$ function?

By default, it performs an inner join (though specific default behavior depends on configuration, notes showcase inner/outer logic).

In the syntax $df[2:5]$, which rows are actually displayed?

The rows at index $2$, $3$, and $4$ ($2nd$ to $4th$ row labels).

How do you print values of a $Series$ greater than $2$?

$s[s > 2]$.

What is required when creating a $Series$ from a scalar value like $15$?

An explicit index must be provided (e.g., $index=[1, 2, 3]$).

Is a $DataFrame$ data structure considered two-dimensional?

Yes.

What represents the row index in a spreadsheet context for Pandas?

The $Index$.

What represents the column index in a spreadsheet context for Pandas?

The column name.

If $obj1$ and $obj2$ are dataframes, what does $pd.concat([obj1, obj2])$ do?

It appends $obj2$ below $obj1$ vertically (default $axis=0$).

What file extension is used to save comma-separated tabular data?

$.csv$.

Can you open a CSV file in Microsoft Excel?

Yes, MS Excel can open and display CSV data in a table format.

In $iloc$, if you use $[:2, 1]$, what does it select?

The first two rows and the second column (index $1$).

How do you specify a specific path for exporting a CSV in $to\_csv$?

By passing a string path, e.g., $\text{'E:\\Dataframe1.csv'}$, as the first argument.

What does a $DataFrame$ allow you to do once data is stored?

Perform various operations useful for analyzing and understanding the data.