Stats 21 Final

Type()

Used to check the types of data in base python

Base data types

“” - str, text data

2 - int, integers

2.0 - float, numbers with decimals

True/False - Boolean, true or false, must be spelled True and False

None - NoneType - indicates Null values

Math operations

X // Y = integer floor division of x by y

X % Y = integer remainder of x//y

X ** y = x to the power of y

Y += x means y = y + x, used in loops

Integers added or multiplied always results in an integer

Division always results in a float

The sum or product of an integer with a float is always a float

Floating Point Type and differences with real numbers and how to compare them

Uses 64 bits to represent decimal values

Has a max value of a little less than 2^1024

1.79+e^(308)

Big numbers are shown in scientific notation

Operations using floating point type numbers that should result in the same answer sometimes don’t, so == comparisons return FALSE

Use isclose() instead with the 2 numbers and that will return TRUE

Integer type

Uses variable amounts of memory to show large numbers with great precision

Prints out the entire huge number

Sqrt in base python

Does not exist

Use x ** 0.5 to do the same as sqrt(x)

Math package / module

To do more complex math in python you need this package

Math.Pi

Math.Exp returns e^x

Math.Sin

Math.sqrt returns a float

String Type Intro

Created with single or double quotes, normally returned as single quotes, but print(string) returns no quotes

Len() returns number of characters

String * 4 returns multiplication with strings or StringStringStringString

.format method allows you to place named variables into strings using curly braces, say name is defined as Gavin, “My name is {name}”.format(name = name) will return ‘My name is Gavin’

Assignment with =

When a variable is first assigned there is no output

Program instructions

Input - get input

Output - display output

Math - perform an operation

Conditional execution - check conditions and run code

Repetition - repeat an action, with variation

Defining a function

Def functionname(arguments) :

Code

If you want the function to return an object put

Return object

If a function does not use return to return a value, the result of the function will be None (which only shows up if you print)

Returning multiple values

Using a tuple, a function can return multiple values

Def powersof(number):

Square = number ** 2

Cube = number ** 3

Return number, square, cube

Tuple unpacking

If the function returns a tuple it can then be unpacked into separate elements (the number of elements must match the values being assigned)

X, y, z = powersof(3) would assign x to 3, y to 9, and 27 to z

The function can also just be saved as a singular tuple and assigned to one object

J = powersof(3) will store all the values

Returning a print

Print(anything) just prints the value, it doesn’t actually make it a physical output, so if a function calls print as the return value it will print the value, but assign that output to anything and it will just be None

Function default arguments

Allows you to call a function like function() and not get an error

Or if there are multiple arguments, you can just enter maybe one or 2 arguments of 5 if all 5 have default values and the function will still work

You can either directly specify, or place based on where the arguments were defined, but you cannot mix both methods

Function local env vs Global env

Variables defined inside a function only exist inside that function

Arguments / parameters are also local

if a variable does not exist in the function env but exists in the global, or is nested in another function above it, the function will use that value instead.

Magic Global Env commands

%who - prints names of all global env variables

%whos - prints the names and details of each object

%who_ls - returns a list with object names as strings

Altering global variables using a function

Def alter_global_x():

Global x

X = x + 1

Return x

This will call the global defined x and then change whatever the global defined value of x is by adding one to it

Putting global before a variable that does not exist yet will create on globally

Updating non local variables (variables defined in nested functions a level up from the current one)

Same format as global, but use nonlocal instead as the keyword

If there is no nonlocal variable defined however, this will return an error

General conditional statement execution flow for if, elif, else

No need to use parenthesis

Use a colon to end the conditional statement, if x ==0:

Any lines indented after the colon are associated with the statement

When indentation stops, the lines are no longer associated with the statement

Elif and else must be on the same level of indentation as the first if statement

Elif and else are only executed if the original if statement is false

Recursion

A function calls itself inside the function

When you write these there should always be a base case using an if statement that does not call the function to avoid it running forever.

Can be used to achieve repetition

Recursion

A function calls itself inside the function

When you write these there should always be a base case using an if statement that does not call the function to avoid it running forever.

Can be used to achieve repetition

For x in iterable:

Repeat set of commands

Iterable data structures

List

Tuple

Range

Strings

The range object

Range(start, end, step size)

Range(10) gives the sequence 0-9, Output will be range(0,10)

Produces a sequence of numbers, end value not included, does not accept floats

To actually see the values do list(range(0,10))

List(range(10,5,-1)) returns 10 9 8 7 6

You need to specify the negative step if you have start > end or the list will be empty

While loop

While conditional true:

Repeat set of commands until conditional false

While True;

(Runs forever until a break statement is hit)

The break and continue statements

Used in if statements that are nested within for / while loops

Continue skips the current iteration, but goes to the next iteration

Break stops the loop completely

Boolean Operations on strings

“A” < “B”

True

Things coming earlier in the alphabet are less than things later (think like number 1 - 26)

“A” < “a”

True

All uppercase letters are less than lowercase letters

‘0’ < ‘00’ is True, while 0 < 00 is false (data type matters?)

Digits in strings are less than uppercase letters

Logical Operators

And

Or

Not

Written like:

True and False

True and not False

True and not True

List Creation

Use square brackets and = assignment

Lists can hold any data type

You can nest lists within lists

Subletting lists

Python is ZERO INDEXED

List[0] selects the first element

List[-1] selects the last element

Subsetting nested lists

List[0][0]

Selects the first element within the big list

Then selects the first element of the nested list

This can go on for as many nests there are

List Slicing

Will not include the item in the index after the colon

List[1:3] slices at the first and third comma in the list, returning elements 1 and 2

List[1:2] is the same thing as list[1]

List[1:1] returns an empty list

Mutable lists

methods applied to the lists change the lists themselves and not print a modified copy

If a list is referenced by 2 names, any changes to one of the names will change the other name reference to the list too

How to shallow copy a list (copy the list, not its references)

List.copy()

List[:]

List(current_name_of_list)

Appending a list

List.append(x) adds x as the last element in the list

Equivalent to list[len(list):] = [x]

List = list + [x] also works

If x is a list, the append method adds the entire object as one list entry

List.insert(i, x)

Inserts x before position i

List.insert(0, x) inserts x at the front of the list

This only allows you to insert 1 element

List.insert(i, x)

Inserts x before position i

List.insert(0, x) inserts x at the front of the list

This only allows you to insert 1 element, which can be a list

List.extend(iterable)

Extends the list by appending all the items in the iterable as individual elements to the list

Slice assignment

The way to insert multiple items to a specific position

List[4:4] = [5, 10] will insert 5 at index 4 and 10 at index 5

List.remove(x)

Removes the first element in the list whose value is x, error if no item exists

List.pop(i)

Remove the item at the given position in the list and return it

If no index specified it removes and returns the last item in the list

List.clear()

Removes all items in the list

Equivalent to del list[:]

List.index(x)

Return the zero-based index in the list of the first item whose value is x, raises value error if no item exists

List.count(x)

Returns number of times x appears in the list

List.reverse()

Reverses elements of the list in place, doesn’t return anything

Modify in place vs. return a new value

Ask does this operation mutate in place? .sort, .append, .reverse, .extend all return None but directly change the list

Sorted(list), reversed(list) return a new modified object

List.sort(key = none, reversed = false)

Sort the items of the list in place, can’t sort strings with floats / ints

Strings are immutable, this means

When you use a method on a string, it does not modify the string itself

It will always return a new modified object

Creating multi line strings

Use triple quotes

‘’’

String.strip()

Removes extra white space

String.split()

Breaks a string into a list substrings based on white space

Can specify”,” in parentheses to split by commas

String.isalpha()

True if the string only has letters, no spaces or digits or special characters

String.splitlines()

Separates longer strings / multi-line strings at the line ends and returns a list of the lines as strings

String.find(‘x’) and string.index

Returns the index of the first instance of x

Find returns a -1 if the character doesn’t exist

Index returns an error

Creating a dictionary

{key:value, key:value}

Dict(key = value, key = value)

Keys must be any immutable object (string, numbers, tuple, function)

Values can be any object type

Accessing items in the dictionary

Dict[key] will return the value, or an error if the key doesn’t exist

Or Dict.get(key) does the same, but returns None if the key doesn’t exist, you can specify a default value though

NOT INDEXED BY POSITION

CANNOT BE SLICED (ONE VALUE BACK AT A TIME)

Duplicate keys

Python doesn’t throw an error but discards all duplicate key value pairs except the last instance reading left to right

Checking for a Dict entry

The in operator applies to keys when called as key in dict

To check a value use value in dict.values()

Deleting keys from a dict

Use del dict[key]

Dictionary methods

Dict.pop(key) removes an entry from the dictionary while getting the value associated with the key

Dict.update(dict2) combines 2 dictionaries, if the dictionary used to update has keys that exist in the first dictionary, the updated keys will take their place

Dictionary View Objects

dict.keys()

Dict.values()

Dict.items() contains tuples of key value pairs

The elements in these view objects change when the dictionary changes, so if they get assigned to a variable name, they dynamically update unlike how other methods create static copies in time for lists or other iterables

View objects only support Len() or in as functions to use on them, to do more convert the view objects to lists, but then you lose the dynamic update

What makes tuples different from lists

They are immutable, once their values are set, they cannot be modified

Creating Tuples

They are created using () instead of []

Or you can just set tuple_name = value1, value2, value3 and it will work (parenthesis not required)

Tuples can hold one value by using tuple_name = “a”,

Tuple() creates an empty tuple or converts an iterable to a tuple

Replacing one tuple with another

Say T is a already defined tuple

T = (“A”,) + T[1:]

Will create a new tuple that has the first element changes to “A”, and now T points to that tuple

Tuple methods

Tuple.index() returns the index of the first appearance of an element in the tuple

Tuple.count() counts the amount of times an element appears in a tuple

What functions work with tuples

Any that don’t directly affect the stored tuple

Len

Sum

Sorted

Min

Max

Tuples as function arguments

When writing a function put *args as the argument to allow for the argument to be a tuple so that the arguments hold one position, but many values are within that one argument. This is gathering

The opposite is scattering

If you have an already written function that takes multiple arguments and they are stored in a tuple, call the function with *tuple as the argument and it will unpack the elements

Zipping iterables

List(Zip(sequence, sequence)) Takes 2 or more sequences and interleaves them, if you have 2 sequences it makes a list of tuples with 2 elements in each tuple. If you have 3 it makes a list of tuples with 3 elements in each tuple and so on

If the sequences are not the same length the result has the length of the shorter one

Enumerate(iterable)

Zips a range object of the same length of the iterable to the iterable

Creates index, value pairs from an iterable

Opening a file

Open(‘file name’) opens a file

Open(file name, ‘w’) will replace or create the file and allow you to write into it

Writing into a file

File name.write(line) allows you to write strings into the open file

It returns the number of characters that were written

When done writing use file name.close to close the file

When formatting numbers into strings, what does adding g in the replacement field do?

It prints floats in general format, rounding to 6 sig figs

You can specify the amount by doing .8g =8 sig figs

When formatting numbers into strings, what does adding f in the replacement field do?

It prints in fixed point format, default is 6 places after the decimal,

.8f will allow you to specify to 8 points

What does 05d in the replacement field do?

It pads zeroes to whatever integer is on the left side of the colon until the integer is 5 digits

What does .1% in the replacement field do

Multiplies any number in the value field times 100

Left right center alignment

Left <#spaces

Right >#spaces

Center ^#spaces

try and except

Used to have python try a set of commands, it may get an error, so except works by checking if the error matches an error we would have expected to get and if so it executes some code and doesn’t throw an error. If there is no error at all the try code runs, if the error is different from the one specified in the except condition, an error will still be thrown

JSON.dump(data, file)

Writes the object into a file

Doesn’t only need strings, can be any python object

JSON.load(file)

Reads the file back into a python object

How to save many objects to the same file

Store all of the objects in a dictionary, then write the dictionary into the file. Then to read the dictionary back into memory, use json.load to load the file and then pull out the variables you want using the keys

Numpy Arrays

Np.array(list)

Where the list must be items of all the same data type

When printed, the array has no commas

Adding a list of lists creates a multi-dimensional array (matrix), but the length of the lists within the lists must be equal

Order of dimensions of a numpy array

Blocks x Sheets x rows x columns

Np.zeroes((2, 2, 3, 4))) returns 2 blocks with 2 sheets of 3 rows by 4 columns of zeros

Array sequences in numpy

Np.arange(start, stop, step)

Same idea as a range item in base python, but it is an array and doesn’t need to be wrapped in a list to show the values

Np.linspace(start, stop, num, endpoint = True)

Creates an array of linearly spaced values starting at start and ending at stop (inclusive), with a length of num

If endpoint is changed to false, the stop value is excluded

Upcasting / coercion

Bool > int > float > string

True is 1 and False is 0

Array Attributes

Array.ndim() returns the number of dimensions

Array.shape() returns the size of each dimension

Array.dtype() returns the data type

Reshaping Arrays

Np.reshape(array, [new shape]) returns a new array that is reshaped, doesn’t impact original copy

Array.reshape(new shape) also works

Using -1 for a dimension will ask python to figure out the number to use for that dimension (only for one dimension)

Array.T will transpose an array, but leaves the original unaffected

One dimensional array

Their shape = (Len,)

Their transpose is the same as their raw form in terms of shape and appearance

To make a one dimensional into 2D

Np.reshape(array, (1, Len)) will make this a 2D array, the output will have 2 square brackets to confirm

Array.ravel() vs array.flatten()

Ravel and flatten both turn multi dimensional arrays into a single dimension

However ravel creates a view, which reflects edits to the original array, while flatten creates a copy of the original array that doesn’t reflect edits

Subsetting and slicing arrays

Similar to lists but you can also slice with a second colon

Array[start:stop:step size) will return values from start to stop by step size

For higher dimensions the Subsetting is array[row index, column index]

Slices of numpy arrays are VIEW objects and update if the original array is updated and their updates change the original array

Np.concatenate(array1, array2, axis = 0)

Appends an existing array to another.

Axis = 0 specifies appending to the number of rows = np.vstack(array1, array2)

Axis = 1 specifies appending to the number of columns = np.hstack(array1, array2)

Summaries for matrices

Np.sum(array) by default sums the entire array to one number

Np.sum(array, axis = 0) will sum over rows, giving column totals

Dealing with nan

Non is the float value for something that is not a number

Np.nan will create it

Np.nansum(array) will sum an array and ignore the nan

Np.nanmean(array) averages an array and ignores nans

Almost all aggregate functions in numpy have a nan version except any and all

Testing for nan involves np.isnan()

Broadcasting

Like recycling values for vector / matrix ops in R

If A is shape (3,2) and B is shape (2,), A + B will broadcast the 2 values of B across all 3 rows of A and output the updated (3,2) matrix

However, dimensions must be compatible, in the sense of going across columns, the one dim array length must match the number of columns in the 2D array

In the sense of going across rows, the one dim array must be converted into a 2D array of one column, which then spreads the column values across the rows

When a x by 1 and 1 by x array are being operated on, the result will be an x by x matrix with the operation performed element wise

Numpy Fancy Indexing

Print(array[0,1,5]) returns elements 0, 1, and 5, which cannot be done in base python

You can also subset arrays using arrays

Np.argsort()

As opposed to sorting the actual values in the array, argsort returns the indices of the array if it were sorted. This allows you to subset arrays by the argsort to arrange them in the way you want

Pandas, series, and data frames

Pandas creates data frames where each column is an array, allowing us to have tables of data where each column can be a different data type

Pandas series are printed in table form with their index, and are type pandas series

Data.to_numpy()

Converts pandas series to numpy array

Slicing by numerical vs name index

Slicing by numerical index does not include the last value, while slicing be name index does include the last value

Converting dictionaries to series

The keys become the index and values become the array