Starting Out with Python - Chapter 6: Files and Exceptions

Fundamentals of File Input and Output

  • Data Persistence Between Program Runs:

    • To retain data between program executions, data must be saved to secondary storage, typically a computer disk.

    • Saved data persists across sessions and can be retrieved, loaded, and processed at any future point.

  • Input and Output Terminology:

    • Writing data to: The operation of storing and saving data onto a file.

    • Output file: Any file to which a program writes and persists data.

    • Reading data from: The operation of retrieving stored information from a file back into program memory.

    • Input file: Any file from which a program reads data.

  • The Three-Step File Processing Cycle:

    • 1. Open the file: Establishes a communication channel between the running program and the file on disk.

    • 2. Process the file: Performs write operations to add data or read operations to extract data.

    • 3. Close the file: Terminates the connection between the file and program, releasing operating system resources and ensuring all pending data buffers are flushed.

  • Data Movement Between RAM and Disk Storage:

    • During write operations, values held in RAM variables (such as employee_name holding 'Cindy Chandler', employee_id holding '7451Z', and pay_rate holding 18.6518.65) are copied from memory and serialized onto a disk file.

    • During read operations, the process is reversed: data stored in records on the disk file is read, copied into RAM, and assigned to program variables.

Data flow between variables in RAM and a disk file

File Classifications and Access Methods

  • Primary File Types:

    • Text file: Stores character data that has been encoded using character encoding schemes (such as ASCII or Unicode). Can be viewed and edited with basic text editors.

    • Binary file: Stores raw byte sequences that have not been converted to human-readable plain text. Requires specialized software to interpret properly.

  • File Access Methods:

    • Sequential access:

    • Data is processed in continuous, ordered fashion from the beginning of the file to the end.

    • To read an item located in the middle or end, the program must first read through all preceding items; skipping ahead is not permitted.

    • Direct access:

    • Also termed random access.

    • Allows a program to jump immediately to any arbitrary byte or record within the file without reading the data that comes before it.

Filenames, File Objects, and File Opening

  • Filename Extensions:

    • Short sequences of characters situated at the conclusion of a filename, preceded by a dot/period (e.g., .txt).

    • The extension denotes the file format and the nature of the data stored within.

  • File Objects:

    • An internal software entity instantiated in memory that serves as an interface between the program and a physical file located on the storage disk.

    • Programs interact with files exclusively through methods called on file objects referenced by variable names.

A variable referencing a file object associated with a disk file

  • The open() Built-in Function:

    • Instantiates a file object and links it to a specified file on disk.

    • General syntax: python file_object = open(filename, mode)     

    • Parameters:

    • filename: A string literal or variable holding the name or path of the target file.

    • mode: A string specifying the access mode and intended operations.

  • Standard File Access Modes:

    • 'r': Reading only. Opens an existing file for reading. The file must exist, or an error is raised.

    • 'w': Writing. Opens a file for writing. If the file already exists, its contents are completely overwritten and erased. If it does not exist, a new empty file is created.

    • 'a': Appending. Opens a file to append new data to the very end of existing contents. Existing contents are preserved. If the file does not exist, it is created.

  • Specifying File Paths:

    • If a filename passed to open() does not include an explicit directory path, Python assumes the file is located in the current working directory alongside the executing program.

    • Newly created files without explicit paths are generated in the current working directory.

    • When specifying alternative directory paths containing backslashes on systems such as Windows, prefix the string literal with the character r to define it as a raw string (e.g., open(r'C:\data\records.txt', 'r')), preventing unintended escape character processing.

Reading and Writing Operations

  • Writing Data to a File:

    • A method is a function bound to an object that performs operations on that object.

    • write() method: Transmits a string to the associated file object: python file_variable.write(string)     

  • close() method: Flushes unwritten buffers and closes the connection to the disk: python file_variable.close()     

  • Reading Data from a File:

    • read() method: Reads the entire contents of a file into memory in a single operation, returning it as a single string. Requires the file to be opened in read mode ('r').

    • readline() method: Reads a single line from the file, up to and including the trailing newline character (' '), returning it as a string.

    • Read position: An internal pointer managed by the runtime that indicates the location of the next byte/line to be read. Operations advance this position sequentially toward the end of the file.

  • Managing Newline Characters:

    • Concatenating newlines when writing:

    • Data written via write() does not automatically append line-break characters.

    • Multiple values must be separated by concatenating the newline escape sequence ' ' using the string concatenation operator +: python file_variable.write(data_item + ' ')       

  • Stripping newlines when reading:

    • The readline() method preserves the newline character attached to the end of each line.

    • To remove trailing newline characters for data processing or clean console output, invoke the string method rstrip(' '): python line = file_variable.readline() line = line.rstrip(' ')       

  • Appending Data:

    • Opening a file with mode 'w' destroys existing file data immediately.

    • Using mode 'a' preserves prior contents and directs any subsequent write() calls to place data at the end of the current content.

  • Reading and Writing Numeric Data:

    • Text files store data strictly as character sequences. Numbers cannot be written directly without explicit type conversion.

    • Writing numbers: Convert numeric values to strings using the str() built-in function before passing them to write(): python file_variable.write(str(numeric_value) + ' ')     

  • Reading numbers: Data retrieved via read() or readline() arrives as string objects. Strings representing numbers must be converted using int() or float() to perform arithmetic operations: python int_value = int(file_variable.readline()) float_value = float(file_variable.readline())     

Processing Files with Loops

  • Motivations for Loop Processing:

    • Disk files often store large, variable quantities of items where the exact count of lines is not known in advance.

    • Looping constructs allow programs to iterate through items until reaching the end of the file (EOF).

  • Detecting End-of-File with while Loops:

    • When the readline() method reaches the end of the file, it returns an empty string ('').

    • The empty string functions as a sentinel value.

    • General logic using a priming read and a while loop: python file_object = open('data.txt', 'r') line = file_object.readline() while line != '': # Process the line item line = file_object.readline() file_object.close()     

Logic flowchart for detecting the end of a file using readline

  • Step-by-Step Flowchart Procedure for EOF Detection:

    • 1. Open the file.

    • 2. Perform a priming read: use readline to read the first line.

    • 3. Evaluate the decision diamond: Did readline return an empty string?

    • 4. If No (False): Process the item that was just read from the file, use readline to read the next line from the file, and cycle back to the decision step.

    • 5. If Yes (True): Exit the loop branch and close the file.

  • Iterating with Python's for Loop:

    • Python file objects are iterables. A for loop automatically reads successive lines from a file and terminates at EOF without requiring explicit priming reads or sentinel comparisons.

    • General format: python for line in file_object: statements     

  • The loop iterates once for each line present in the file.

The with Statement for File Management

  • Automatic Resource Management:

    • The with statement establishes a runtime context that guarantees open files are automatically closed once execution leaves the block, even if unhandled exceptions arise.

    • General syntax: python with open(filename, mode) as file_variable: statement statement     

    • Components:

    • filename: String designating the target file.

    • mode: Access mode indicator (e.g., 'r', 'w', 'a').

    • file_variable: Variable referencing the opened file object within the scope.

    • with suite: The indented block of statements operating on the file resource.

  • Writing with a with Statement:

  with open('myfile.txt', 'w') as output_file:
      statement
      statement
  ```
  * Opens `myfile.txt` in write mode; commands inside the suite write via `output_file`; the file automatically closes upon exiting the block.

* **Reading with a `with` Statement:**

python with open('myfile.txt', 'r') as input_file: statement statement   ```

  • Opens myfile.txt in read mode; commands read data via input_file; the file automatically closes when the suite completes.

    • Opening Multiple Files in a Single with Statement:

  • Multiple file streams can be opened simultaneously by separating open() calls with commas: python with open('file1.txt', 'r') as input_file, open('file2.txt', 'w') as output_file: statement statement     

    • Reads data from file1.txt using input_file while concurrently outputting data to file2.txt using output_file.

    • When execution leaves the with suite, all opened files close automatically.

Record and Field Processing

  • Structural Concepts:

    • Record: A complete collection of related data elements describing one specific item or entity.

    • Field: A single, individual component of data residing within a record.

  • Processing Sequential Records:

    • Writing records: Each field of a record is written sequentially one after another, separated by newlines.

    • Reading records: The program reads each individual field in sequence until the full complement of fields constituting a single record has been retrieved.

  • Standard Operations on Record Files:

    • Adding records: Appending new complete entity entries to the end of an existing file.

    • Displaying records: Reading and formatting file records for user inspection.

    • Searching for a specific record: Reading sequentially through the file until an entry matching specified search criteria is identified.

    • Modifying records: Locating an existing record, altering one or more of its field values, and saving the updated data.

    • Deleting records: Removing target records from the dataset, typically by copying all non-deleted records to a temporary file and replacing the original file.

Exceptions and Error Handling

  • Exceptions and Diagnostic Output:

    • Exception: A runtime error condition encountered during program execution that halts execution abruptly if unhandled.

    • Traceback: A detailed multi-line diagnostic error message printed to the console upon an unhandled exception, displaying the file name, line numbers that produced the failure, exception type, and a short explanation.

  • Preventable vs. Unavoidable Exceptions:

    • Preventable exceptions: Errors that can be anticipated and avoided using conditional validation logic (e.g., checking user input values before processing).

    • Unavoidable exceptions: Errors caused by external states or system events that cannot be prevented by simple decision constructs alone:

    • Attempting to parse a non-numeric string into an integer (e.g., int('abc') raising a ValueError).

    • Attempting to open a non-existent file for reading using 'r' mode.

  • The try/except Statement:

    • Exception handler: A block of code designed to intercept an active exception, handle the error gracefully, and prevent the program from crashing.

    • General syntax: python try: statements except exceptionName: statements     

    • Try suite: The indented block of code under try: that contains statements capable of throwing an exception.

    • Handler block: The indented code under except: that executes when the named exception type occurs.

    • Execution mechanics:

    • If statements in the try suite run without error, all except handlers are skipped, and the program proceeds directly to the code below the construct.

    • If a statement inside the try suite raises the specified exceptionName, the try suite terminates immediately, the corresponding handler runs, and execution continues below the construct.

    • If an unhandled or mismatched exception type is raised, the program terminates immediately with a traceback.

  • Handling Multiple Specific Exceptions:

    • Try suites that can raise multiple error types can be paired with stacked except clauses: python try: # Statements that may throw IOError, ValueError, etc. except IOError: # Handles file I/O failures except ValueError: # Handles numeric conversion errors     

    • Catch-all handlers: An except clause without an explicit exception name catches all exception types raised within the try suite. This bare clause must always be placed as the final clause in the sequence.

  • Displaying Default Error Messages via Exception Objects:

    • Exception object: An object created in memory when an error occurs, encapsulating error codes and a default descriptive message.

    • The object can be assigned to a variable in the except header using the as keyword: python except ValueError as err: print(err)     

    • Passing the assigned variable (err) to the print() function outputs the runtime's default diagnostic text.

  • The else Clause:

    • An optional block placed after all except clauses, aligned with try and except.

    • Else suite: Executes only if the statements in the try suite run without raising any exceptions.

    • If an exception occurs, the else suite is completely bypassed.

  • The finally Clause:

    • An optional block positioned at the end of the construct after all except and else clauses.

    • General format: python finally: statements     

    • Finally suite: A sequence of cleanup statements guaranteed to execute under all circumstances—whether an exception was raised, handled, or avoided entirely.

    • Used primarily for critical cleanup actions, such as closing file streams and releasing hardware or network handles prior to exiting.

  • Consequences of Unhandled Exceptions:

    • An exception remains unhandled if:

    • 1. No except clause matches the specific type of exception thrown.

    • 2. An exception is raised outside the boundaries of an active try suite.

    • In either scenario, execution halts abruptly, and Python displays a traceback diagnostic.