Chapter 3: Linux Files and Directories

Linux Directory Structure and Pathname Navigation

  • Filesystem Tree Hierarchy:

    • Operating systems organize files into a logical inverted tree structure.

    • Physical Storage vs. Logical Structure: Files are physically stored on storage device partitions formatted with specific filesystems, but presented to users as a unified directory tree.

    • The Linux Root Directory (/): Unlike Windows, Linux has a single unified tree starting at the root directory, denoted by the forward slash (/) symbol.

  • Comparison: Windows vs. Linux Filesystem Structures:

    • Windows Filesystem Structure:

    • Uses multiple root drives denoted by drive letters followed by colons and backslashes (e.g., C:\, D:\).

    • Uses backslashes (\) as directory path separators.

    • Example: C:\windows\system and D:\home\mary represent separate drive root hierarchies.

    • Linux Filesystem Structure:

    • Features a single top-level root directory (/) for the entire system.

    • All hard drives, storage partitions, and removable media are mounted as subdirectories under / rather than separate drive letters.

    • Uses forward slashes (/) as directory path separators.

    • Example: /home/user/Documents or /windows/system reside under the single root.

  • Pathname Types:

    • Absolute Pathname:

    • Represents the complete file path starting explicitly from the root directory (/).

    • Always begins with a / character.

    • Example: /home/alex/docs/report.txt or /var/log/syslog.

    • Relative Pathname:

    • Specifies a file or directory location relative to the current working directory.

    • Does NOT start with a / character.

    • Example: If currently located inside /home/alex, the relative path to report.txt is docs/report.txt.

  • Directory Types and Metacharacters:

    • Home Directory (~): A specialized directory set aside for each user account on the filesystem to store personal files, documents, and configuration settings. Referenced using the tilde (~) metacharacter.

    • Parent Directory (..): The directory located one level closer to the root directory relative to the current directory. Referenced using two consecutive dots (..).

    • Current Working Directory (.): The directory in which the user or process is currently operating. Referenced using a single dot (.).

    • Subdirectory: Any directory located inside another parent directory in the filesystem tree.

  • Shell Tab Completion:

    • A productivity feature of the BASH shell that automatically completes typed directory or file pathnames upon pressing the Tab key.

    • Saves time and prevents typing errors.

    • If multiple matching names exist (an ambiguous match), pressing Tab twice displays a list of all matching files or subdirectories.

  • Directory Navigation Commands:

    • pwd (Print Working Directory): Displays the full absolute pathname of the current working directory.

    • cd [path] (Change Directory): Switches the current working directory to the specified absolute or relative path.

    • cd ~: Changes directory to the current user's home directory.

    • cd ..: Moves up one directory level to the parent directory.

Linux File Types, Filenames, and Directory Viewing

  • Linux File Types Classification:

    • Text Files (Symbol: - | Code: -): Stores information in human-readable plain text format. Commonly used for configuration files, source code, and scripts.

    • Binary Data Files (Symbol: - | Code: -): Contains non-text machine language (binary 1s and 0s) storing compiled routines, functions, and graphics used by applications.

    • Executable Program Files (Symbol: * | Code: -): Binary programs or executable scripts that can be run in system memory as active processes.

    • Directory Files (Symbol: / | Code: d): Special structural files that serve as folders to organize other files and subdirectories.

    • Linked Files (Symbol: @ | Code: l): Files that point to or represent the exact data of another file on the filesystem.

    • Special Device Files (Symbol: None / Device-specific | Code: b for block or c for character): Files representing hardware devices such as hard disk drives, storage controllers, and serial communication ports.

    • Named Pipe Files (Symbol: | | Code: n or p): Temporary memory buffers that route output from one command or process directly into another process on the local machine.

    • Socket Files (Symbol: = | Code: s): Inter-process communication channels enabling a process on a remote computer to connect to and write to a file on the local computer while a local process reads from it.

  • Filename Rules and Standards:

    • Filenames can be up to 255 characters long (most are rarely longer than 20 characters).

    • Allowed characters include alphanumeric characters, underscores (_), dashes (-), and periods (.).

    • Extensions: Identifiers following a period (.) at the end of a filename (e.g., .txt, .csv) indicate file type, though extensions are not required by the kernel.

    • Hidden Files: Files with names starting with a period (.), such as .bashrc. They are hidden from standard directory listings.

    • Metacharacters to Avoid in Filenames: *, ?, [, ], {, }, |, ;, &, <, >, (, ), , !, `, \, ', ".

  • Directory Listing and Information Commands:

    • The ls Command & Options:

    • ls: Lists contents of a directory.

    • ls -l: Long format listing displaying detailed file properties.

    • ls -a: Lists all files, including hidden files starting with a dot (.).

    • ls -F: Classifies filenames by appending indicator symbols (/ for directory, * for executable, @ for link).

    • ls -R: Recursively lists directory contents and all subdirectories.

    • ls -r: Reverses the sort order of the file listing.

    • ls -S: Sorts file listing by file size (largest first).

    • ls -t: Sorts file listing by modification time (newest first).

    • Components of ls -l Long Listing:

    1. File type character (-, d, l, b, c, n, s)

    2. List of permissions

    3. Hard link count

    4. Owner of the file

    5. Group owner of the file

    6. File size in bytes

    7. Most recent modification time

    8. Filename

    • Additional File Information Commands:

    • file [file/directory]: Displays the file type of a specified file or directory (identifies binary types, executable types, and empty files).

    • stat [file]: Displays additional comprehensive details and metadata about a file.

    • tree [directory]: Displays contents of a directory and subdirectories formatted in a visual tree structure.

Shell Wildcards, File Viewing, and Manipulation

  • Shell Wildcards (Globbing):

    • Used by the shell to match filename patterns when issuing commands.

    • *: Matches 0 or more characters in a filename (e.g., ls project*).

    • ?: Matches exactly 1 character in a filename (e.g., ls project?).

    • [abc]: Matches 1 character from the specified set (e.g., ls project[124]).

    • [a-z]: Matches 1 character within the specified character range (e.g., ls project[a-d]).

    • [!abc]: Matches 1 character that is NOT in the specified set or range (e.g., ls project[!a-d]).

  • Displaying Text File Contents:

    • cat [file]: Concatenates and displays the full contents of a text file. Option -n displays line numbers.

    • tac [file]: Displays text file contents in reverse order, starting from the last line.

    • head [-n N] [file]: Displays the first set of lines of a text file (default is 10 lines).

    • tail [-n N] [file]: Displays the last set of lines of a text file (default is 10 lines).

    • more [file]: Displays a text file page-by-page and line-by-line.

    • less [file]: Displays a text file page-by-page with interactive navigation (arrow key scrolling up and down).

    • diff [fileA] [fileB]: Compares two text files line-by-line to identify differences.

  • Displaying Binary File Contents:

    • strings [file]: Searches for and displays human-readable text characters inside a binary file.

    • od [-opt] [file]: Displays file contents in octal format (numeric base 8). Option -x displays contents in hexadecimal format (numeric base 16).

  • Searching Text with Regular Expressions and Grep:

    • Wildcard Metacharacters vs. Regular Expressions:

    • Wildcards are interpreted by the BASH shell to match file/directory names on the filesystem.

    • Regular Expressions (regexp) are interpreted by text tool programs to match character patterns within text files.

    • Regular Expression Metacharacters:

    • * (Common): Matches 0 or more occurrences of the previous character.

    • ? (Extended): Matches 0 or 1 occurrence of the previous character.

    • + (Extended): Matches 1 or more occurrences of the previous character.

    • . (Common): Matches any single character.

    • [...] (Common): Matches one character from the specified set/range.

    • [^...] (Common): Matches one character NOT in the specified set/range.

    • { } (Extended): Matches a specific number or range of occurrences of the previous character.

    • ^ (Common): Matches pattern if located at the start of a line.

    • (Common): Matches pattern if located at the end of a line.

    • (...|...) (Extended): Matches either of two specified text patterns.

    • The Grep Family Commands & Options:

    • grep [pattern] [file]: Global Regular Expression Print; searches text files using common regular expressions.

    • egrep [pattern] [file]: Searches files using extended regular expressions (equivalent to grep -E).

    • fgrep [pattern] [file]: Fixed Grep; searches for literal strings without interpreting regular expressions for faster execution (equivalent to grep -F).

    • grep -i: Case-insensitive search.

    • grep -v: Inverts search results to display lines that do NOT match the specified pattern.

  • Editing Text Files:

    • The vi / vim Text Editor:

    • A standard bi-modal command-line text editor supporting regular expressions and over 1,000 functions.

    • Command Mode: Default mode for navigation, deletion, copying, searching, and saving (no text insertion).

      • i / a / o: Enters Insert Mode (inserts before cursor, after cursor, or on a new line).

      • dd: Deletes current line.

      • yy: Copies (yanks) current line.

      • p: Pastes copied/deleted text below current line.

      • /pattern: Searches forward for pattern.

      • :[line#] or [line#]G: Moves cursor directly to specific line number.

      • :w: Saves changes.

      • :q: Quits (if no unsaved changes).

      • :wq or :x: Saves changes and quits.

      • :q!: Quits without saving changes.

    • Insert Mode: Mode for typing and entering text into document.

      • Esc: Returns to Command Mode.

    • Alternative Text Editors:

    • nano: Easy-to-use terminal text editor using Ctrl key shortcuts.

    • emacs: Advanced, highly extensible terminal text editor using Ctrl key combinations.

    • gedit: Graphical user interface (GUI) text editor.


ls and grep Practice (Instructor Answer Key)

  • ls options

    • List the contents of the current directory: ls

    • List all files including hidden files: ls -a

    • List files in long format (permissions, owner, size): ls -l

    • List hidden files in long format: ls -la

    • Sort by modification time, newest first: ls -lt

    • Display files in reverse order: ls -r

    • Sort by file size: ls -lS

    • Human-readable sizes (KB, MB, GB): ls -lh

    • List the /home directory: ls /home

    • List recursively through subdirectories: ls -R

    • List only directories in the current folder: ls -d */

    • Show inode numbers: ls -i

    • Display file sizes in bytes: ls -l --block-size=1

    • One file per line: ls -1

    • Show file types with symbols (/, *, etc.): ls -F

    • Sort by extension: ls -X

    • Detailed info for report.txt: ls -l report.txt

    • List only hidden files: ls -d .*

    • Newest files at the bottom: ls -ltr

    • /etc in long format with human-readable sizes: ls -lh /etc

  • ls with wildcards

    • All .txt files: ls *.txt

    • All .log files: ls *.log

    • Files that begin with data: ls data*

    • Files that end with 2024: ls *2024

    • All .jpg and .png files: ls *.jpg *.png

    • Start with report, end with .txt: ls report*.txt

    • A single character followed by .txt: ls ?.txt

    • Exactly three characters before .txt: ls ???.txt

    • Start with a or b: ls [ab]*

    • Start with any letter a–f: ls [a-f]*

    • Start with any number 0–9: ls [0-9]*

    • Start with anything except a or b: ls [!ab]*

    • End with a number: ls *[0-9]

    • .txt files that begin with file: ls file*.txt

    • Names exactly four characters long: ls ????

    • test followed by one character: ls test?

    • All .c source files: ls *.c

    • .cpp files starting with lab: ls lab*.cpp

    • Start with an uppercase letter A–Z: ls [A-Z]*

    • Contain backup in the filename: ls *backup*

  • grep options

    • Search for "error" in log.txt: grep "error" log.txt

    • Search for "hello" in notes.txt: grep "hello" notes.txt

    • "warning" in system.log with line numbers: grep -n "warning" system.log

    • "admin" in users.txt ignoring case: grep -i "admin" users.txt

    • "failed" in all files in the current directory: grep "failed" *

    • "root" recursively in /etc: grep -r "root" /etc

    • Lines that do NOT contain "error": grep -v "error" log.txt

    • Count matching lines for "login" in auth.log: grep -c "login" auth.log

    • Print only the matching word: grep -o "pattern" file.txt

    • "error" OR "fail" in log.txt: grep -E "error|fail" log.txt

    • 3 lines after each match of "server": grep -A 3 "server" file.txt

    • 2 lines before each match of "warning": grep -B 2 "warning" file.txt

    • Exact word "cat" (not "category"): grep -w "cat" file.txt

    • "timeout" in all .log files: grep "timeout" *.log

    • Count how many times "localhost" appears: grep -o "localhost" hosts.txt | wc -l

    • Lines beginning with "ERROR": grep "^ERROR" file.txt

    • Lines ending with ".com": grep "\.com"domains.txt</code></p></li><li><p>Emailscontaining"gmail"incontacts.txt:<code>grep"gmail"contacts.txt</code></p></li><li><p>"database"inapp.logandserver.log:<code>grep"database"app.logserver.log</code></p></li><li><p>Highlight"permissiondenied"insystem.log:<code>grep−−color"permissiondenied"system.log</code></p></li></ul></li><li><p><strong>grepwithregularexpressions</strong></p><ul><li><p>Linesthatcontainerror:<code>grep"error"file.txt</code></p></li><li><p>LinesthatstartwithERROR:<code>grep"ERROR"file.txt</code></p></li><li><p>Linesthatendwith.log:<code>grep"l˙og" domains.txt</code></p></li><li><p>Emails containing "gmail" in contacts.txt: <code>grep "gmail" contacts.txt</code></p></li><li><p>"database" in app.log and server.log: <code>grep "database" app.log server.log</code></p></li><li><p>Highlight "permission denied" in system.log: <code>grep --color "permission denied" system.log</code></p></li></ul></li><li><p><strong>grep with regular expressions</strong></p><ul><li><p>Lines that contain error: <code>grep "error" file.txt</code></p></li><li><p>Lines that start with ERROR: <code>grep "^ERROR" file.txt</code></p></li><li><p>Lines that end with .log: <code>grep "\.log" file.txt

    • Lines containing cat or dog: grep -E "cat|dog" file.txt

    • Lines containing any digit: grep "[0-9]" file.txt

    • Three consecutive digits: grep "[0-9][0-9][0-9]" file.txt

    • A lowercase letter: grep "[a-z]" file.txt

    • An uppercase letter: grep "[A-Z]" file.txt

    • Lines starting with a lowercase letter: grep "^[a-z]" file.txt

    • Lines ending with a digit: grep "[0-9]" file.txt

    • Exactly the word cat: grep "\<cat\>" file.txt

    • apple, orange, or banana: grep -E "apple|orange|banana" file.txt

    • Any vowel: grep "[aeiou]" file.txt

    • Two identical characters in a row: grep "\(.\)\1" file.txt

    • Lines that start with #: grep "^#" file.txt

    • Lines that do not contain numbers: grep -v "[0-9]" file.txt

    • Emails ending in gmail.com: grep "gmail\.com" file.txt

    • Four consecutive digits (like a year): grep "[0-9]\{4\}" file.txt

    • Lines starting with INFO or ERROR: grep -E "^(INFO|ERROR)" file.txt

    • Lines that contain only numbers: grep "^[0-9]*" file.txt (note: this also matches blank lines; grep -E "^[0-9]+$" is stricter)