Lecture 7: Testing, Debugging, Exceptions, Assertions

Defensive Programming

• Write specifications for functions

• modularize programs

• check conditions on inputs/outputs (assertions)

Testing/Validation

• Compare input/output pairs to specification

• “It’s not working!”

• “How can I break my program?”

Debugging

• Study events leading up to an error

• “ Why is it not working?”

• “How can I fix my program?”

How to set yourself for easy testing and debugging

• from the start, design code to ease this part

• break program up into modules that can be tested and debugged individually

• document constraints on modules (expectations of inputs/outputs)

• document assumptions behind code design

When are you ready to test?

• ensure code runs by removing syntax and static semantic errors

• have a set of expected results (an input set where for each input has the expected output)

Classes of Tests

• Unit Testing

• Regression Testing

• Integration Testing

Unit Testing

• validate each piece of program

• testing each function separately

Regression Testing

• add test for bugs as you find them

• catch reintroduced errors that were previously fixed

Integration Testing

• does overall program work?

• tend to rush to do this

Cycle of Testing

Make sure that interactions between all different pieces work as expected. If not go back to unit and regression testing from Integration testing

Testing Approaches

• Intuition about natural boundaries to the problem (come up with natural partitions)

• Random testing (more testing = more likely program is correct)

• Black box testing

• Glass box testing

Black box testing

• explore paths through specification (build test cases in different natural space partitions and also consider boundary conditions (empty lists, singleton list, large numbers, small numbers)

• designed without looking at the code

• can be done by someone other than the implementer to avoid some implementer biases

• testing can be reused if implementation changes

Glass box testing

• use code directly to guide design of test cases

• called path-complete if every potential path through code is tested at least once (could miss a bug)

• Drawbacks: can go through loops arbitrarily many times and missing paths (make sure to have test cases that covers all of these)

How to Debug with Print Statements

• good way to test hypothesis

• print when inside functions, loops, and to show function results

• use bisection method (print half of program to see if there’s a bug or not in that section)

Debugging by studying program code

don’t ask what is wrong. Instead, ask how did I get the unexpected result and is it part of a family?

Debugging with Scientific Method

study available data → form hypothesis → repeatable experiments → pick simplest input to test with

IndexError

trying to access beyond the limits of a list

TypeError

trying to convert an inappropriate type; mixing data types without appropriate coercion; operand doesn’t have correct type

NameError

referencing a non-existent variable; local or global name not found

SyntaxError

forgetting to close parenthesis, quotation, etc.; Python can’t parse program

Logic Errors

• think before writing new code

• draw pictures, take a break

• explain the code to someone else or a rubber ducky

Write entire program • Test entire program • Debug entire program

Introduces a lot of bugs and hard to isolate when bugs are affecting other ones; instead write, test, and debugs functions one-by-one (unit and regression testing) then do integration testing

Dont: Change code • Remember where bug was • Test code • Forget where bug was or what change you made • Panic

Instead: Backup code • Change code • Write down potential bug in a comment • Test code • Compare new version with old version

what happens when procedure execution hits an unexpected condition?

get an exception… to what was expected

AttributeError

attribute reference fails

ValueError

operand type okay, but value is illegal

IOError

IO system reports malfunction (e.g. file not found)

Dealing with Exceptions

exceptions raised by any statement in body of try are handled by the except statement and execution continues with the body of the except statement

Handling Specific Exceptions

have separate except clauses to deal with a particular type of exception

else

body of this is executed when execution of associated try body completes with no exceptions

finally

body of this is always executed after try, else and except clauses, even if they raised another error or executed a break, continue or return; useful for clean-up code that should be run no matter what else happened (e.g. close a file)

what to do when encounter an error?

stop execution, signal error condition (raise an exception); do NOT fall silently or return an “error” value

Exceptions as Control Flow

• don’t return special values when an error occurred and then check whether ‘error value’ was returned

• instead, raise an exception when unable to produce a result consistent with function’s specification

Assertions/ Assert Statements

used to make sure that the assumptions on computations are exactly what the function expects them to be or else an AssertionError exception is raised; good example of defensive programming

ASSERTIONS AS DEFENSIVE PROGRAMMING

• assertions don’t allow a programmer to control response to unexpected conditions

• ensure that execution halts whenever an expected condition is not met

• typically used to check inputs to functions, but can be used anywhere

• can be used to check outputs of a function to avoid propagating bad values

• can make it easier to locate a source of a bug

Where to use Assertions?

• goal is to spot bugs as soon as introduced and make clear where they happened

• use as a supplement to testing

• raise exceptions if users supplies bad data input

• use assertions to

• check types of arguments or values

• check that invariants on data structures are met

• check constraints on return values

• check for violations of constraints on procedure (e.g. no duplicates in a list)