Data Managment Exam 1

Python typing: static or dynamic?

Dynamic. You don't declare variable types, and they're checked at runtime

How does Python mark code blocks?

Indentation plus a colon (Java uses { })

Python single-line vs multi-line comment

single-line. '''…''' or """…""" multi-line

Is Python compiled or interpreted?

Interpreted

Year Python was first released

1991

range(3, 10) produces what?

3, 4, 5, 6, 7, 8, 9 (stop is excluded)

range(17, 100, 2) means what?

Start at 17, stop before 100, step 2 (odd numbers)

Slice syntax

seq[start:end:step]. Start is inclusive, end is exclusive, default step is 1

s = "Python": s[-1], s[0:4], s[::2]

'n', 'Pyth', 'Pto'

How do you reverse a string or list?

[::-1]

my_list[-2::-1] on [10,20,30,40,50,60]

[50, 40, 30, 20, 10] (start at 2nd-last, go backwards)

my_list[:3] vs my_list[2:]

First 3 elements (indices 0-2) vs everything from index 2 onward

Are strings mutable?

No, immutable. Methods return a new string

"a,b,c".split(",") and "-".join(parts)

['a','b','c'] and 'a-b-c'

s.find("x") when "x" isn't found

Returns -1

"ab" * 3

'ababab'

List: mutable or immutable? Ordered?

Mutable, ordered

list.append(x) vs list.insert(i, x)

Adds to the end vs inserts at index i

list.sort() vs sorted(list)

Sorts in place (returns None) vs returns a new sorted list

b = a (a is a list): does it copy?

No. Both names point to the same list. Use a.copy()

How do you access element in row 2, column 1 of a list of lists?

matrix[2][1] (first index = sublist, second = element)

Access the red value of an RGB image pixel

image[row][column][color]

pop() vs remove(x)

pop() removes and returns the last item (or by index). remove(x) removes the first matching value

Tuple: mutable? Syntax?

Immutable, ordered. (1, 2, 3)

Why use a tuple?

Immutability. Can be used as dictionary keys

Syntax for a one-item tuple

(5,) (the trailing comma is required)

Dictionary: what is it?

Collection of key-value pairs in {}

Rules for dictionary keys

Unique and immutable (str, int, tuple OK. List not OK)

What can dictionary values be?

Anything, including lists and other dicts

d.get(key) vs d[key] for a missing key

get returns None (or a default). d[key] raises KeyError

d.keys(), d.values(), d.items()

All keys / all values / (key, value) pairs

Does "x" in d check keys or values?

Keys

Dictionary comprehension

{x: x**2 for x in range(5)}

Set: what is it?

Unordered collection of unique elements

How do you make an empty set?

set(). Not {} (that's an empty dict)

Remove duplicates from a list

set(my_list)

Set union, intersection, difference, symmetric difference symbols

Pipe |, &, -, ^

{1,2,3} & {3,4,5}

{3}

{1,2,3} ^ {3,4,5}

{1, 2, 4, 5} (in either, not both)

{1,2,3} - {3,4,5}

{1, 2}

List comprehension: squares 0 to 4

[x**2 for x in range(5)]

Why are dicts and sets fast for lookup?

Hash-based, so insert/delete/lookup are ~O(1) on average

Time to check x in list vs x in set

O(n) vs O(1)

Lists vs tuples performance

Tuples are immutable and faster for read-only use

File mode r

Read (default). Error if the file doesn't exist

File mode w

Write. Creates or overwrites (erases existing contents)

File mode a

Append to the end

File mode rb / wb

Read / write binary

Why use with open(...)?

File is automatically closed, even if an error occurs (prevents leaks)

read() vs readline() vs readlines()

Whole file as one string / one line / list of all lines

Best way to read a big file

Loop: for line in file: (one line at a time)

Does write() add a newline?

No. Add "\n" yourself

Text mode vs binary mode

Text decodes bytes to strings. Binary returns raw bytes

Modules for CSV and JSON

csv and json

Catch a missing file

try: ... except FileNotFoundError:

Why does error handling matter?

Prevents crashes, handles missing or bad files gracefully

What is journaling in a file system?

Tracks changes to reduce corruption after crashes (NTFS, ext4, APFS)

FAT32 max file size

4 GB. No journaling, very limited security. Used for USB drives and cards

Default file systems: Windows / Linux / macOS

NTFS / ext4 / APFS What is pathlib good for?

Path.cwd()

Current working directory

How do you join paths in pathlib?

The / operator: BASE / "data"

OUT.mkdir(parents=True, exist_ok=True)

Creates the folder and any missing parents. No error if it exists

DATA.glob("*.csv")

Finds all .csv files in DATA

stat.st_size

File size in bytes

st_mtime

Last modification time (content changed)

st_ctime

Windows: creation time. Unix: last metadata change

st_atime

Last access time

Convert a raw timestamp to readable

datetime.fromtimestamp(stat.st_mtime)

collections.Counter

Dict subclass for counting occurrences

Stream vs load a large file

Stream line by line: constant memory

csv.DictReader

Reads each row as a dict keyed by the header names

csv.DictWriter steps

Create writer with fieldnames, writeheader(), writerows(rows)

Sort by rating, ties broken by votes

sorted(rows, key=lambda r: (r["rating"], r["votes"]), reverse=True)

json.load() vs json.loads()

From a file vs from a string

json.dump() vs json.dumps()

To a file vs to a string

pd.read_csv(..., chunksize=N)

Reads a big CSV in chunks of N rows

usecols and dtype in read_csv

Load only needed columns / use smaller types to save memory

gzip.open(..., "rt")

Read a compressed file as text

What is JSONL?

One JSON object per line

What is NumPy?

Fundamental package for scientific computing, built around the N-dimensional array

ndarray: homogeneous?

Yes, all elements the same type

NumPy key attributes

shape, dtype, size

Ways to create arrays

np.array(), np.zeros(), np.ones(), np.arange()

What is broadcasting?

Operating on arrays of different shapes (e.g., array × scalar)

a * b for NumPy arrays

Element-wise product (not matrix multiplication)

Matrix multiplication / dot product

np.dot()

x.reshape(2,3) rule

Total number of elements must stay the same

Why is NumPy faster than lists?

Contiguous, C-style memory and typed values

Who created pandas, and when?

Wes McKinney, 2008, at AQR Capital

Where does the name "pandas" come from?

"Panel data" (an econometrics term)

What is pandas built on?

NumPy

Series vs DataFrame

1D labeled array vs 2D table (rows and columns)

axis=0 vs axis=1

Rows/index vs columns

df.sum(axis=1)

Sums across each row

df.shape vs df.size

(rows, columns) vs total number of cells

df.info(), df.head(), df.columns, df.describe()

Column types and non-nulls / first rows / column names / summary stats

df.set_index('PARTY')

Makes PARTY the row index

pd.concat([df1, df2])

Stacks rows (append)

pd.merge(df1, df2, on='key')

Joins on a common key column

Which pandas tools are for big data that doesn't fit in memory?

Dask, Vaex Inner join