File Systems

Learning Objectives

  • By the end of this module, you should be able to:

    • List the basic functions common to modern file systems

    • Use and describe the file systems used by Windows OSs

    • Use and describe the file systems used by Linux systems, including ufs and ext

    • Use and describe the macOS file system

File System Functions

  • All information stored on a computer’s hard disk is managed, stored, and retrieved through a file system

    • The file system allocates locations on a disk for storage and keeps a record of where specific information is kepy

  • Some file systems also implement recovery procedures when a disk area is damaged or when the OS goes down

  • File systems used by operating systems perform the following general tasks:

    • Provide a convenient interface for users and applications to open and save files

    • Provide a hierarchal structure to organize files

    • Store file metadata to provide detailed information about files

    • Organize space on a storage device

User Interface

  • When a user double-clicks a file to open it, the user interface calls the file system with a request to open the file

  • the file type determines how the file is opened

    • If the file is an application, the application is loaded into memory and run by the CPU

    • If the file is a document, the application associated with the document type is loaded into memory and opens the file

Hierarchical Structure

  • The overall purpose of a file system is to create a structure for filing data

  • A file is a set of data that is grouped in some logical manner, assigned a name, and stored on the disk

    • Data contained in files can be text, images, music and sounds, video, or Web pages

  • OSs typically organize files in a hierarchy of folders or directories

    • The top of the hierarchy is called the “root” of the file system

    • The root of the file system often represents a disk drive or other mass storage device

  • Folders can be organized in a hierarchy that is similar to a tree structure

  • A default OS structure might consist of folders for the following:

    • OS files

    • Software applications

    • Individual user files

    • Public files that are available to all users

    • Other folders needed by applications and the administrator

  • General best practices when designing a storage and file structure for a server:

    • The OS files should be on a separate hard disk from user files

    • Applications and system data files should be on a separate hard disk from the OS and user files

    • Folders should be organized with access controls in mind

    • Folders should be named to reflect the purpose of the files they contain

  • The Linux folder structure is different from the Windows folder structure

File Metadata

  • Metadata is information that describes the file and its contents but is not the actual data itself

  • Information found in metadata:

    • Filename

    • Date and time the file was created

    • Date and time the file was last modified

    • Date and time the file was last accessed

    • Size of the file

    • Folder or file attributes, such as permissions, whether the file is read-only, etc.

Storage Device Space Organization

  • Hard disks arrive from the manufacturer with low-level formatting

    • A low-level format is a software process that marks the location of disk tracks and sectors

    • Tracks are like several circles around a disk; each track is divided into sections of equal size called sectors

  • Block allocation divides the disk into logical blocks called clusters, which correlate to sectors, heads, and tracks on the disk

    • Called allocation units in Windows systems

  • Block allocation is stored using one of two techniques:

    • The file allocation table (FAT) uses a fixed portion of the disk to store this data

    • The New Technology File System (NTFS) and Linux file systems use various locations on the disk to store a special type of file called the Master File Table (MFT) that is used for directory and file allocation information

    • All OSs have tools that let you check, and sometimes repair, common file system and disk problems

  • Partitioning is the process of reserving some or all of a disk to be used by a particular file system, such as FAT or NTFS

  • After partitioning, the disk might be high-level formatted in order for the OS to store files

  • A single disk can contain one partition or more

    • Each formatted partition is usually referred to as a volume in most OSs

    • You might want multiple file systems to allow for a dual-boot system where you can boot to Windows 10 or Linux

  • When a partition is created, information about that partition is stored in a special area of the disk known as the partition table (in MS-DOS, Mac OS, and Windows) or disk label (in UNIX/Linux)

  • A traditional BIOS uses the Master Boot Record (MBR) partitioning method

    • It contains a small block of program code used to locate and boot the OS

    • This code is called the bootstrap code or master boot code

  • Most MBRs contain the following elements:

    • The bootstrap code examines the partition table to determine the partition from which to boot (the active partition)

    • An optional timestamp indicates when the MBR was created

    • An optional disk signature identifies the disk

    • The partition table contains up to four entries that describe the partitions

    • A boot signature is set to hexadecimal 55AA and marks the end of the MBR

  • MBR supports two partition types: primary and extended

  • A primary partition can be formatted and assigned a drive later

    • It can also be an active partition, which can contain boot code to start an OS

  • An extended partition cannot be formatted

    • One or more logical drives must be created in the extended partition

    • A logical drive can then be formatted and assigned a drive letter, but cannot be active

  • Due to some limitations of MBR, a newer partitioning scheme called GUID Partition Table (GPT) was introduced in the late 1990s

  • GPT is more reliable than MBR because multiple copies of the partition data are stored in various places on the disk

  • GPT also makes use of a cyclic redundancy check (CRC) to ensure that no unauthorized changes have been made to the table

    • A CRC is a formula that uses the data bytes of a file or other data object to calculate a 32-bit value that verifies the integrity of the data

  • GPT disks are typically used with UEFI firmware

  • Windows supports two types of drive partitioning: basic and dynamic

  • Basic disks use either the MBR or GPT disk partitioning scheme

    • They do not support disk spanning, disk stripping, or RAID configurations

  • Dynamic disks do not use traditional fixed partitioning schemes such as MBR and GPT

    • Volumes can be extended into free space on any disk drive

    • Spanned volumes can be extended into free space on any disk drive

    • Spanned volumes can be created

    • Striped (RAID 0), mirrored (RAID 1), and striped with parity (RAID 5) volumes can be created

  • Since Windows XP, you can access a volume as a folder on another NTFS volume

  • When using the Windows Disk Management utility to format a disk:

    • You have the option to mount the disk instead of assigning a drive letter

    • The empty folder into which a volume is mounted is called a volume mount point

  • In versions of Windows starting with Windows 2000, you typically partition disks during installation of the OS

    • You can also reconfigure partitions after the OS is installed

  • Formatting is the process of placing the file system on a partition

    • When installing a Windows OS, Windows will automatically create and format the necessary partitions on the first hard disk

    • A formatted partition is referred to as a volume in Windows systems

    • You can manually format a hard disk using the format command from the Command Prompt window

    • The format command includes several switches

      • To view a list of switches, type format /?

  • A disk cluster is a group of one or more sectors used to store files

  • When a file is stored to disk:

    • Data is written to clusters on the disk

    • The filename is stored in the folder, along with the number of the first cluster the data is stored in

    • When the OS fills the first cluster, data is written to the next free cluster and the FAT entry corresponding with the first cluster points to the number of the second cluster used

    • When the second cluster is full, the OS continues with the next free cluster and the FAT entry for the second cluster points to the number of the third cluster used, and so on…

    • When a file is completely written to the disk, the FAT entry for the final cluster is filled with all 1s (which means the end of the file)

  • Unusable spots are marked in the FAT as bad clusters (never used for file storage)

  • Cluster sizes can vary between 512 bytes up to 2 MB

    • A typical cluster size is 4 KB

Windows File Systems

  • Windows OSs support the following file systems:

    • FAT16, extended FAT16, FAT32, and exFAT

    • NTFS

    • ReFS

  • All OSs that have a file system support FAT16, but it is mostly used for formatting removable media

  • This section focuses on FAT32, exFAT, NTFS, and ReFS

FAT32 and exFAT

  • Support for FAT32 started with Windows 95 Release 2

    • FAT32 is designed to accommodate larger-capacity disks

  • FAT32

    • Can use disk space more efficiently than FAT16 because it uses smaller cluster sizes

    • The largest volume that can be formatted on Windows systems is 32 GB

    • The maximum file size is 4 GB

  • The exFAT file system, also known as FAT64, is a proprietary file system is introduced by Microsoft for mobile personal storage

  • exFAT supports volumes ip tp 128 PB and files up to 16 EB

    • A good choice for high-capacity flash drives

  • Support is available for Linux from a third party

NTFS

  • NTFS is the primary Windows file system for all Windows OSs starting with Windows NT 3.1

  • NTFS uses a Master File Table (MFT) instead of FAT tables

  • When a file is created, a record of that file is added to the MFT

    • The record contains additional attributes such as security settings, ownership, and permissions

  • The MFT record reflects the sequence of clusters that a file uses

  • It is possible to have multiple filenames that refer to the same file

    • A technique known as hard linking

    • This feature is also available in UNIX/Linux file systems

  • Features incorporated into NTFS:

    • NTFS File and Folder Permissions—NTFS is equipped with security features that meet the US government’s C2 security specifications

      • Refers to high-level, “top-secret” standards for data protection, system auditing, and system access

    • NTFS File Compression and Encryption are implemented as file attributes

      • On NTFS volumes, you can enable file compression on the entire volume, a folder and its contents, or a file

      • NTFS volumes use Encrypting File System (EFS) for encryption

      • Disk quotas can be used to put a hard limit on the amount of storage a user’s files can occupy

      • Volume mount points enable you to access a volume as a folder in another volume instead of using a drive letter

      • Shadow copies are a feature where users can access previous versions of files in shared folders and restore files that have been deleted or corrupted

      • Journaling is the ability to keep a log or journal of file system activity

      • The hot fix capability allows NTFS to automatically copy information from one bad area of a disk to another disk area that is not damaged

      • Self-healing NTFS is a utility that runs in the background to correct hard disk problems, making downtime less frequent

Resilient File System (ReFS)

  • ReFS became available starting with Windows Servers 2012

    • The main use is in large file-sharing applications where volumes are managed by Storage Spaces

    • ReFS is mostly backwards-compatible with NTFS

      • Doesn’t support file compression, disk quotas, and EFS (Encrypting File System)

    • Windows can’t be booted from an ReFS volume

    • ReFS can correct some types of data corruption automatically

CDFS and UDF

  • Windows OSs since Windows 2000 recognize some additional file systems used by peripheral storage technologies

  • CD-ROM File System (CDFS) is supported so that OSs can read and write files to DVD/CD-ROM drives

  • Universal Disk Format (UDF) is used on DVD/CD-ROMs, which in turn are used for large file storage to accommodate movies and games

The Linux File System

  • There are many different file systems that can be used with Linux

    • Some file systems are more “native” to specific Linux operating systems than others

  • Most versions of Linux support the UNIX file system (ufs), which is the original native UNIX file system

    • Ufs is a hierarchical file system that is expandable, supports large storage, provides excellent security, and is reliable

  • In Linux, the native file system is called the extended file system (ext or ext fs)

  • Ext is modeled after ufs and enables the use of the full range of built-in Linux commands, file manipulation, and security

  • The first ext version had bugs

    • ext2—reliable file system that handles large disk storage

    • ex3—added journaling capabilities

    • ext4—supports file sixes up to 16 TB

  • Both ufs and ext use the same structure, which is built on the concept of information nodes (or inodes)

    • Each file has an inode and is identified by an inode number

    • An inode contains general information about the file, such as:

      • User and group ownership, permissions, size and type of file, date the file was created, and the date the file was last modified and read

  • Each disk is divided into logical blocks

    • The superblock contains information about the layout of blocks, sectors, and cylinder groups on the file system

  • The inode does not contain a filename; the filename is stored in a folder

  • Several folder entries can point to the same inode

    • Called a hard link

  • A Linux system can have many file systems

  • Linux users only mount points, in which each file system is a subfolder of the root

    • All file systems are referred to by a path

    • The path starts out with root (/)

  • The mount command has several options

    • Typing it without parameters results in a display of the disks currently mounted

  • Disks are referenced by a special inode called a device

  • There are two types of devices:

    • A raw device has no logical division in blocks

    • A block device does have logical division in blocks

  • Devices are normally kept in the /dev or /devices folder

  • The symbolic link is used to link a directory entry to a file that is on a different partition

    • A symbolic link is merely a pointer to a file

  • You must first partition a disk in order to use the Linux file system

  • The command to partition the disk differs slightly

    • Most Linux systems use either fdisk or format

    • Typing man fdisk or man format at the command prompt gives you an overview of available commands

  • Once a partition is made, a file system can be created

    • You must know the device name of the partition on which you want to create a file system

    • Type newfs, followed by the name of the device

    • The newfs command is not available in all versions of Linux

      • Use the mkfs command instead

  • When a file is saved to disk, the system stores part of the data to memory until it has time to write to disk

    • If a computer is shut down prior to data being written to disk, you can end up with a damaged file system

    • Manually force a write of all data in memory by using the sync command

  • Fsck is a utility that verifies the integrity of the superblock, the inodes, all cluster groups, and all directory entries

MacOS File Systems

  • MacOS supports the following file systems:

    • Mac OS Extended

    • Apple File System (APFS)

    • FAT and exFAT

Mac OS Extended

  • Mac OS Extended is used on macOS 10.12 and earlier versions

    • It is a journaling file system. which allows data to be recovered from a journal file if a disk or system problem occurs while data is being uploaded or modified

  • Mac OS Extended supports volume sizes of up to 16 TB

  • The first two sectors of a Mac-formatted disk are boot sectors, or boot blocks

  • Boot blocks are followed by a volume information block, which points to other important areas of information

    • The catalog-b tree is the list of all files on the volume and the extents b-tree keeps track of the location of file fragments

  • Mac OS has always supported medium-length filenames (up to 31 characters)

    • The current Mac OS Extended system supports filenames of up to 255 characters

    • Any character may be used in a filename except the colon (:)

    • Macintosh paths are written as colon-separated entities such as:

      • Hard Drive:System Folder:Preferences:Finder Prefs

  • Mac uses type codes and creator codes instead of the filename extensions used in Windows

  • Macintosh files can contain two parts, or forks:

    • A data fork contains frequently changing information (such as word-processing data)

    • A resource fork contains information that is fixed (such as a program’s icons, menu resources, and splash screens)

  • Apple’s equivalent to a Windows shortcut is the alias

    • The system-level Alias Manager keeps track of the original

    • The word “alias” is added to the filename when the alias is created, and the filename is italicized

Apple File System (APFS)

  • APFS was introduced with macOS 10.13 High Sierra

  • Its underlying partitioning structure is GPT, but APFS uses a concept of containers and volumes

    • A container is a block of reserved space on a drive that contains one or more volumes

    • The volumes share the available space in a container and are dynamically sized according to the amount of actual space required by the files stored on them (a feature Apple calls Space Sharing)

  • APFS supports the following features:

    • Compatibility across devices

    • Encryption

    • Snapshots