COMP3000-FS-updated

Overview of File Systems and Storage Management

  • Course Information: COMP 3000, Operating Systems by Lianying Zhao.

Memory vs. Storage

  • Differences:

    • RAM: low capacity, high speed.

    • In-RAM content is volatile and lost when powered off (Persistence issue).

  • Why can't a program run directly on the hard drive?: It involves I/O operations, which are abstracted from hardware.

Drivers in Operating Systems

  • Driver Functionality:

    • Necessary for user applications to interact with files via read/write system calls.

    • Applications view devices as files, not direct hardware.

  • Block Layer Illustration:

    • User applications → File System → Block Interface → Device Driver.

    • Some programs require direct raw access (e.g., disk repair tools).

Block Device Layer

  • Characteristics:

    • Media and its corresponding drivers (e.g., USB Mass Storage).

    • May consist of multiple layers; block sizes may differ from the filesystem.

    • File systems can exist on any block devices.

  • Performance Considerations:

    • write() does not immediately write to block device.

    • sync command needed to flush data.

File System Layer

  • Purpose:

    • Simplifies the view from blocks to files.

    • No universal file system abstraction for full portability.

  • Tight Coupling with OS Kernel:

    • Example: file-based access control.

Types of Files

  • Classification:

    • Regular File

    • Directory

    • Symbolic Link

    • FIFO (Named Pipe)

    • Socket

    • Device File (Block, Character).

File Descriptors (fd)

  • Definition:

    • Non-negative integer pointing to a kernel data structure; akin to indices in arrays.

    • Example: stdin, stdout, stderr as special file descriptors.

File Access Tracking

  • Structures:

    • File Descriptor Table: per process.

    • Open File Table: system-wide.

    • i-node Table: manages in-memory file descriptors.

Understanding inodes

  • Definition:

    • POSIX concept representing a file, identified uniquely by inode number.

  • Types of Inodes:

    • Directory, Regular file, Character device, Block device, Named pipe, Symbolic link, Socket.

Inode Structure

  • Components:

    • Metadata stored within

    • Data structures that reside in filesystem storage; both on-disk and in-memory copies exist.

Usage of stat Command

  • Functionality:

    • Displays detailed file/directory information beyond ls.

    • Utilizes system calls: stat(), fstat(), lstat().

Directory Entry (dentry)

  • Functionality:

    • Represents a directory entry.

    • System calls: getdents() (not read()), Library call: readdir().

    • Maps file to inode through the parent directory.

    • The root directory’s inode number is always 2.

Links

  • Differences Between Links:

    • Symbolic Link: links to target file name (pathname).

      • If the target file is deleted, the link breaks.

    • Hard Link: links to inode number, identical in every aspect except the name.

      • Not allowed for directories to prevent cyclical references.

    • Link Count Concept:

      • Compares concepts with Windows shortcuts and reparse points.

File Operations

  • Copy, Move, and Remove:

    • Copy: new inode created.

    • Move: creates new inode across filesystems, relinks within the same filesystem.

    • Remove: decreases link count; if count hits 1, it removes the inode.

Accessing Devices

  • Special Files:

    • Mostly found under /dev/*.

    • Special files represent physical/virtual devices related to hardware.

  • Examples: Windows' . PHYSICALDRIVE0 and Linux's /dev/sda.

Device Files/Nodes

  • Functionality:

    • Serve as a file system interface between device drivers and applications.

    • Identified by major and minor numbers.

  • Character Devices: access data byte-by-byte, stream-based.

  • Block Devices: accessed in block sizes, addressable storage.

Superblocks

  • Purpose:

    • Store metadata about the entire filesystem.

    • Includes primary/backup superblocks.

    • Command dumpe2fs to view superblock information.

Blocks on Filesystem

  • Components:

    • inode: contains metadata excluding file name.

    • directory: maps file names to inodes (denoting dentries).

    • data blocks: stores actual data, along with superblocks.

File Sizes

  • Logical Size: Actual size of the file visible to users.

  • Physical Size: Actual allocated disk space, may include holes.

Using dd Command

  • Functionality:

    • Command-line tool for copying/converting data with input/output files.

    • Can work with block devices and special files (e.g., /dev/null).

dd vs. cp

  • Differences:

    • cp: handles file granularity, multiple files/directories.

    • dd: manages byte-level control, more suited for data manipulation.

    • Considered a file-based pipeline tool.

Risks of Corrupted File Systems

  • Causes:

    • Failures during updates lead to data inconsistency (power failures, crashes).

    • Media damage can affect long-lived on-disk data.

Addressing File System Corruption

  • fsck Tool:

    • Checks superblocks, allocation, and link counts for integrity.

    • The lost+found directory aids in recovery.

Journaling File Systems

  • Functionality:

    • Helps avoid full file system scans by logging changes before applying.

    • Potential performance/storage overhead.

    • Common Examples: NTFS, ext3, ext4.

Data Recovery Considerations

  • Differentiates file system repair from storage device repair.

  • Backup Importance:

    • Must back up data for recovery options.

Special File Systems

  • procfs:

    • Provides process-related information; e.g., /proc/cpuinfo.

    • Configures kernel at runtime using /proc/sys.

  • sysfs:

    • Interfaces with kernel subsystems and hardware devices, exposing kobject structures.

User-space File Systems

  • Purpose:

    • Enhances portability, convenience, security, and stability.

    • FUSE (Filesystem in Userspace) allows converting various data into a filesystem.

Network File Systems

  • Examples:

    • SSHFS: SSH-based file system for remote file access.

    • NFS: Dedicated server file system, chosen for performance/reliability.

File System Permission Bits

  • Permission Notation:

    • Symbolic (e.g., rwx) vs. Octal (7 → 111).

    • Involves setuid/setgid configurations with corresponding masks and conditions.

Signal Handling in System Calls

  • Context:

    • What if a signal handler triggers during a system call?

    • System call can abort and return an error; or be paused for the handler to complete (SA_RESTART).