Debugging and Profiling

debugging

the process of identifying, isolating, and fixing bugs or errors in software code

• fix correctness issues

• find runtime errors

• stack trace analtysis

profiling

the process of analyzing the performance of a program, focusing on resource usage such as CPU time, memory consumption, and input/output operations

• improve performance

• find bottlenecks

• execution time analysis

syntax errors

break the language’s grammatical rules and prevent program from starting

• caught by compiler or interpreter

syntax error example

writing prnt(“Hello”)

runtime errors

issues that occur while the program is actively running and cause it to crash

• caught by the OS/environment

runtime error example

division by zero

trying to open a file that doesn’t exist

logic errors

the code runs and compiles without crashing, but it produces the wrong output or behavior

• caught by programmer/user

logic error example

using Area = length + width

performance errors

slow down the system

• caught by profiling tools

performance error example

using wrong search/sort algorithm

True or false: you should read traceback errors from top to bottom

false; bottom to top

bottom line of traceback

diagnosis - states the exact exception and gives a short explanation

underline markers of traceback

the ~ and ^ symbols highlight the exact expression causing the crash

call stack of traceback

tells file name, line number, and function name

pdb

built-in source code debugger that allows you to pause program execution, step through code line-by-line and inspect stack frames

method 1 of invoking pdb

insert the breakpoint() function anywhere in the script

method 2 of invoking pdb

insert import pdb; pdb.set_trace()

method 3 of invoking pdb

run the script “python -m pdb my_script.py”

What happens when pdb triggers?

it presents an interactive prompt

print (p)

(pdb)

evaluate and print the value of a variable/expression

next (n)

(pdb)

execute the next line in the current function without stopping function calls

step (s)

(pdb)

execute the current line and step into a function if one is called on that line

continue (c)

(pdb)

continue execution until the next breakpoint is encountered or the program ends

list (l)

(pdb)

show the source code around the current line

quit (q)

(pdb)

quit the debugger and exit the program immediately

help (h)

(pdb)

show a list of commands, or help for a specific command

logging

module that replaces print() statements to track events, troubleshoot bugs, and application metrics

• must use “import logging”

logging.basicConfig()

configures the default global logger

DEBUG

10 - detailed information, typically of interest only when diagnosing problems

INFO

20 - confirmation that things are working as expected

WARNING

30 - an indication that something unexpected happened

ERROR

40 - due to a more serious problem, the software has not been able to perform some function

CRITICAL

50 - a serious error, indicating that the program itself may not be able to continue running

cProfile

bulit-in deterministic profiler designed to identify performance bottlenecks by measuring how often and how long functions exxecute

cProfile table columns

• ncalls

• tottime

• percall

• cumtime

• percall

• filename:lineno(function)

ncalls

(cProfile)

total number of times the function was called

tottime

(cProfile)

total time spent in the function including the time spent in sub-functions

percall (1)

(cProfile)

the average time spent per call (tottime/ncalls)

cumtime

(cProfile)

cumulative time spent in the function including all sub-functions

percall (2)

(cProfile)

the average cumulative time spent per call (cumtime/primitive calls)

filename:lineno(function)

the file name, line number, and name of function being analyzed

line_profiler

third party library that breaks down the execution time for every single line of code within a function

line_profiler table columns

• line#

• hits

• time

• per hit

• % time

line #

(line_profiler)

the line number in your script

hits

(line_profiler)

number of times the specific line was executed

time

(line_profiler)

total time spent on that line

per hit

(line_profiler)

average time per execution of that line

% time

(line_profiler)

percentage of the total function time was spent on that line

memory_profiler

monitors how much RAM the program is consuming line by line

• use “from memory_profiler import profile”

memory_profiler columns

• mem usage

• increment

• occurrences

• line contents

mem usage

(memory_profiler)

the total memory the Python process is using after the line executes

increment

(memory_profiler)

shows change in memory caused by the line

occurrences

(memory_profiler)

number of times the line was run

line contents

(memory_profiler)

the actual code being analyzed