Chapter 4 – File Processing & Programming Constructs

Files: Fundamental Concepts

  • Definition: Collection of related data stored permanently on storage devices (hard drive, SSD, cloud, …)
  • Purposes
    • Store, organise, retrieve data efficiently
    • Backbone of every OS; used for text, images, video, executables, backups, logs, …
  • Two broad file types
    • Text files – human-readable (extensions: .txt, .csv, .log, …)
    • Binary files – non-human-readable (extensions: .jpg, .exe, .mp3, …)

File Organisation Methods

  • Serial files
    • Records stored strictly in the order entered (time order)
    • No intrinsic key-based order; easiest for simple append-only workloads
  • Sequential files
    • Records stored in sorted/logical sequence (e.g. ID, name, date)
    • Still read sequentially from start → specific record; update ⇒ rewrite whole file
  • (Implied contrast) Indexed/random-access files – quicker look-up via index (not covered here)

CRUD – Core File Operations

  • Creating – generate a new physical file
  • Reading – extract data
  • Updating/Writing – modify or add data
  • Deleting – remove file from storage
  • All operations require:
    • OPENFILE/open() ⇒ obtain handle & specify mode
    • Perform action(s)
    • CLOSEFILE/close() ⇒ flush buffers, free resources, permit other apps to edit

Serial Files: Details & Use-Cases

  • Characteristics
    • Sequential storage in chronological entry order
    • Fast appends; inefficient searches (scan entire file)
    • No automatic sort
  • Typical uses
    • Logs, transaction histories, backup streams, simplistic data exchange
    • Example formats: plain .txt, simple .csv where each line = record

Sequential Files: Details & Use-Cases

  • Characteristics
    • Data kept in logical (often sorted) order – alphabetical, numerical, date, …
    • Good for batch processing; search still linear
    • Insert/update/delete usually ⇒ rewrite whole file to preserve order
  • Typical uses
    • Payroll, bank statements, billing, report generation, archival datasets
    • Example formats: JSON, XML where explicit structure defines order

Text Files: Advantages & Python Flexibility

  • Easy creation with ubiquitous editors (Notepad, VS Code, …)
  • Human-readable & editor-agnostic
  • In Python, stored string data can be type-cast into other data types for processing

A Text File as a String

  • Single-line model: Entire file treated as one long string
  • Multi-line model: File stores several lines separated by newline character \n; entire file terminated by EOF marker
  • Equivalent printing example:
    • print("JURONG PIONEER JUNIOR COLLEGE\n2023 H2 COMPUTING\n(Syllabus:9569)")

Opening & Closing Files (General Concept)

  • Inform OS you intend to interact with file – OS checks permissions & locks appropriately
  • Modes (conceptual)
    1. READ – view-only; multiple read handles allowed
    2. WRITE – create/overwrite; may require exclusive lock
  • Must close to flush write buffers & release locks; failure → data loss or locked file

Pseudocode Primitives (Language-Independent)

  • CREATE_FILE("Computing.txt")
  • Reading template:
    • OPENFILE sample.txt FOR READ
    • WHILE NOT EOF(sample.txt)
    • READFILE sample.txt INTO LineOfText
    • OUTPUT LineOfText
    • CLOSEFILE sample.txt
  • Writing template:
    • OPENFILE sample.txt FOR WRITE
    • WRITEFILE sample.txt, "hello world"
    • CLOSEFILE sample.txt

Python File Handling Essentials

  • Key function: open(filename, mode)
  • Core modes
    • "r" – read (default); error if file absent
    • "x" – create; error if already exists
    • "a" – append; create if absent
    • "w" – write (overwrite or create)
  • Always close (file.close()) unless using with context manager

Reading Patterns in Python

  1. Whole file → string
    • content = f.read()
    • Pro: simple for small files; Con: memory heavy for large
  2. Explicit readline() loop
    • line = f.readline() then while loop
  3. Implicit iteration
    • for line in f: (efficient, pythonic)
  4. readlines()
    • Returns list of lines; convenient but whole file in RAM
  5. read(n)
    • Reads exactly n characters ⇒ good for fixed-width formats

Key tips

  • Use .strip() to remove trailing \n
  • Context manager: with open(...) as f: automatically closes

Writing Patterns in Python

  • write() – write single string exactly as given
  • writelines(list_of_strings) – sequentially writes iterable
  • Common scenarios
    1. File doesn’t exist → open in "w" ⇒ create & write new content
    2. File exists and "w" ⇒ truncate then write
    3. Append "a" ⇒ add after existing data
  • Example: write 50 numbers
  with open("sample.txt", 'w') as file:
      for num in range(50):
          file.write(str(num) + "\n")
  • str() cast necessary because + concatenates strings; omitting "\n" results in all numbers stuck together with no line breaks

Output Formatting Helpers

  • Escape sequences in strings
    • \n → newline, \t → tab
  • Alignment
    • .ljust(width, fill), .rjust(width, fill), .center(width, fill)
  • str.join(iterable) – concatenate list/tuple using delimiter (ensure elements are strings)

Using with Statement (Context Manager)

  • Calls __enter__ on file open and __exit__ on block exit → auto closes
  • Example equivalences shown for read(), readline(), readlines(), and write()
  • Examination note: still comment “file closed” for Cambridge marking scheme

String Processing Utilities Demonstrated

  • .split() – default by whitespace; .split(',') for CSV
  • int() / float() casting when numeric fields read from text

Mini End-to-End Example (Data.txt)

  • Reading
  with open('data.txt','r') as file:
      for line in file:
          name, age = line.strip().split()
          age = int(age)
          print(f'Name: {name}, Age: {age}')
  • Writing subset
  with open('data.txt','r') as infile, open('names.txt','w') as outfile:
      for line in infile:
          outfile.write(line.split()[0] + '\n')

CSV Example with .split(',')

  • Read students.csv → parse name, age, gender
  • Output formatted summary per record

Practice & Assessment Exercises

Exercise 1 (Easy)

  • Read words from data1.txt, split by space, print individually

Exercise 2 (Medium)

  • Read marks.csv → display scores & average

Exercise 3 (Harder)

  • Read people.csv, skip header, count genders, write female names to females.txt

Answer keys supplied showing loops, .split(), counters, file writes, formatted output

Suggested Real-World Tasks

  • Extract emails from contacts.txt to emails.txt
  • Longest word finder in story.txt
  • Word-frequency counter in essay.txt (descending order display)

Practical 4 – File Processing Programming Tasks

Q1 (Car Makers)

  • Given list makers, write each to car_makers.txt line-by-line
  • Then read & display in tabular format with heading "Name of Maker"
  • Append additional makers from makers2.txt (maintain same format)
  • Explain risk of forgetting close(): write buffer not flushed ⇒ data loss & file lock
  • Implement all steps in Python (should use with for parts iii/iv or manual close)

Q2 (SLS User-ID Batch Program)

  • Read sample_users.txt where each line = Full Name,Identity Number
  • (i) Count total users; append count to last line of same file
  • (ii) Generate 10-character user-ID = first five letters of name (spaces removed, lower-case) + last five chars of NRIC (include letter)
  • (iii) Create output file where each line: UserID,Full Name,Identity Number
  • Must handle reading, string processing (replace(" ", ""), slicing), writing

Q3 (A-Level 2019 Practical Extract)

Task 1.1

  • Read TIDES.TXT (tab-delimited)
  • Find & print
    • Highest HIGH tide value
    • Lowest LOW tide value
      Task 1.2
  • Compute tidal ranges between successive records (absolute difference)
  • Determine largest & smallest ranges
  • Output range values and date of second tide for each
  • Requirements emphasise reading sequential file, parsing with split('\t'), tracking previous record, storing max/min

Ethical & Practical Considerations

  • Always close files to ensure integrity & permit other processes to access
  • Validate file paths & handle exceptions (missing file, permissions)
  • Protect PII when processing user data (e.g., identity numbers)
  • Use text files for interoperability; prefer binary for efficiency & structured storage (e.g., databases) when scaling

Connections & Broader Context

  • Concepts underpin higher-level data storage (databases, cloud object storage)
  • CRUD mirrors database operations; sequential vs indexed parallels linear vs random access in data structures
  • Real-world relevance: log processing, ETL pipelines, financial batch jobs, IoT data streams

Quick Reference Cheat-Sheet

  • Open file: f = open(path, mode) / with open(path, mode) as f:
  • Modes: "r", "w", "a", "x", plus optional "b" for binary & "+" for read-write
  • Read methods: read(), read(n), readline(), readlines(), iterator
  • Write methods: write(str), writelines(list)
  • String helpers: strip(), split(delim), join(), ljust()/rjust()/center()
  • Casting: str(), int(), float()
  • Newline: \n; Tab: \t