Python for data analysis

Munge / munging / wrangling

The overall process of manipulating unstructured and/or messy data into a structured or clean form

Pseudocode

A description of an algorithm or process that takes a code-like form while likely not being actual valid source code

Syntactic sugar

Programming syntax which does not add new features, but makes something more convenient or easier to type

Python object

Every number, string, data structure, function, class, module, and so on exists in the Python interpreter in its own "box" which is referred to as a Python object

Do Python statements need to be terminated by semicolons? Y/N

No, however, semicolons can be used to separate multiple statements on a single line

Symbol preceding a comment in Python

#

Method

An attached function that has access to the object's internal contents

Binding

Assignment is also referred to as binding, as we are binding a name to an object. Variable names that have been assigned may occasionally be referred to as bound variables.

Attribute

Other Python objects stored "inside" the object. Objects typically have both these and methods.

None

The Python "null" value (only one instance of the None object exists)

str

String type

Unicode

unicode string type

float

double-precision (64-bit) floating point number

bool

A True or False value

int

Signed integer with maximum value determined by the platform

long

Arbitrary precision signed integer. Large int values are automatically converted to long

T/F: Many people use Python for its powerful and flexible built-in string processing capabilities (you can write string literal using either single quotes or double quotes

True!

For multiline strings with line breaks, how many quotes do you use?

Three (triple quotes), either ''' or """

Break

Ends a loop

pass

No-op statement. Can be used in blocks where no action is to be taken. It is only required because Python uses whitespace to delimit blocks. Common to use pass as a placeholder in code while working on a new piece of functionality.

finally

Executes code regardless of whether the code in the 'try' block succeeds or not

ternary expression

Allows you to combine an 'if-else' block which produces a value into a single line or expression. Syntax: "value = true-expr if condition else fales-expr"

Immutable

The object cannot be changed after it is created

Mutable

The object can be changed after it is created

Tuple

One-dimensional, fixed-length, immutable sequence of Python objects

Foo

placeholder for a value that can change

count

counts the number of occurrences of a value (in a tuple or a list)

List

One-dimsensional, variable-length, mutable sequence of objects. Can be defined using square brackets [] or using the "list" type function

Append

Used to add an element to the end of a list

Insert

Used to insert an element at a specific location in a list. Insert is computationally expensive compared to "append" since references to subsequent elements need to be shifted internally to make room for the new element

Pop

removes and returns an element at a particular index; the inverse operation to "insert"

Remove

Remove an element by value, which locates the first such value specified and removes it from the list

In

Keyword used to check if a list contains a value. Returns true or false. Note that checking whether a list contains a value is a lot slower than dicts and sets as Python makes a linear scan across the values of the list, whereas the others (based on hash tables) can make the check in constant time.

Extend

Can append multiple elements to an already defined list.

Extend is comparatively a more efficient / less expensive operation than concatenation to a list, especially if building up a large list since, with concatenation, a new list must be created and the objects copied over.

len

short for length

Sort

Sorts a list in-place (i.e. without creating a new object). Has options including passing a secondary sort key, i.e. a function that produces a value to use to sort the objects such as sort(key=len)

Bisect

implements binary-search and insertion into a sorted list (must be sorted first)

bisect.bisect

finds the location in a list where an element should be inserted to keep it sorted

bisect.insort

Inserts an element into the location in a list that will keep it sorted

NumPy

Numerical Python

Array

Used to store multiple values in one single variable

Slice notation

Used to select sections of list-like types (arrays, tuples, NumPy arrays). Its basic form consists of start:stop passed to the indexing operator []

dict

Python data structure aka a hash map or associative array. It is a flexibly-sized collection of key-value pairs, where key and value are Python objects

hashability

Means a key in a key-value pair is immutable. It could be an object like a scalar type (int, float, string) or a tuple (all the objects in the tuple need to be immutable too).

enumerate

function that returns a sequence of (i, value) tuples

sorted

function that returns a new sorted list from the elements of any sequence

sorted(set('some value'))

trick to return a sorted list of the unique elements (rather than having duplicates) in a sequence

zip

function that "pairs" up the elements of a number of lists, tuples, or other sequences, to create a list of tuples. number of elements produced is determined by the shortest sequence.

reversed

iterates over the elements of a sequence in reverse order

set

unordered collection of unique elements

a.add(x)

add element x to the set a

a.remove(x)

remove element x from the set a

a.union(b)

operation to return all of the unique elements in a and b. alternate syntax = a | b

a.intersection(b)

operation to return all of the elements in both a and b. Alternate syntax = a & b

a.difference(b)

operation to return all of the elements in a that are not in b. Alternate syntax = a - b

a.symmetric_difference(b)

operation to return all of the elements in a or b but not both. Alternate syntax = a ^ b

a.issubset(b)

True if the elements of a are all contained in b

a.issuperset(b)

True if the elements of b are all contained in a

a.isdisjoint(b)

True if a and b have no elements in common

list comprehension

Allow you to concisely form a new list by filtering the elements of a collection and transforming the elements passing the filter in one concise expression. Basic form = [expr for val in collection if condition]

syntax of a dict comprehension

dict_comp = {key-expr : value-expr for value in collection if condition}

syntax of a set comprehension

{expr for value in collection if condition} ... same as syntax of a list comprehension except with curly brackets

def

keyword to declare (define) a function

return

keyword to return a function

Anonymous (lambda) functions

simple functions consisting of a single statement, the result of which is the return value. It's often clearer and requires less typing to pass a lambda function than to write out a full-out function declaration

lambda

keyword meaning you're declaring an anonymous function

closure

any dynamically-generated function returned by another function. The key property is that the returned function has access to the variables in the local namespace (e.g. your Python setup) where it was created

currying

deriving new functions from existing ones by partial argument application

iterator protocol

a generic way to make objects iterable. Having a consistent way to iterate over sequences, like objects in a list or lines in a file, is an important Python feature.

generator

a simple way to construct a new iterable object. returns a sequence of values lazily, pausing after each one until the next one is requested. uses the yield keyword instead of the return keyword in a function

itertools

a standard library module that has a collection of generators for many common data algorithms