Systems Programming Overview

Overview of Systems Programming

• Definition of a System: A system is defined as a collection of various interactive components working together.

• Definition of Programming: Programming is the specific activity of designing and implementing programs.

• Definition of Systems Programming: This activity involves the designing and implementing of system programs.

• Definition of System Programs: These are programs required for the effective execution of general user programs on a computer system. They serve as the foundation upon which other software operates.

Utilities for Systems Programming

• Utilities: Small programs or tools designed to perform specific tasks that assist in managing, maintaining, or supporting the overall functionality of the system.

• Disk Cleanup (Windows): A utility used to free up disk space by removing unnecessary files from the storage medium.

• Task Manager (Windows): A utility designed to monitor and manage running processes and evaluate real-time system performance.

• Terminal (Unix/Linux) / cmd.exe (Windows): A command-line interface (CLI) that allows users and developers to interact directly with the operating system via text commands.

Security and Configuration Tools

• Antivirus Programs: Software such as Norton or McAfee designed to protect the system from malicious software (malware) and viruses.

• Firewall Software: Tools like Windows Firewall that monitor and control incoming and outgoing network traffic based on predetermined security rules.

• Control Panel (Windows): A centralized tool providing access to system settings and internal configurations.

• System Preferences (macOS): An interface that allows users to configure system-specific settings on Apple Macintosh computers.

Classification of Software: Application vs. System

• Application Software:

  • Defined as a program or set of programs used to solve specific problems using the computer as an instrument.

  • Its primary role is to provide services directly to the end-user.

  • Examples include: Word processors, web browsers, media players, and spreadsheets.

• System Software:

  • Supports the core operation of the computer.

  • Acts as an interface between the computer hardware and the application software.

  • Examples include: Compilers, interpreters, assemblers, linkers, loaders, debuggers, and the Operating System (OS).

Characteristics and Differences of System and Application Software

• Usage and Purpose:

  • System Software: Used for operating hardware; effectively manages the computer system; works in the background and runs independently to provide a platform.

  • Application Software: Used by users to perform specific tasks; installed according to user requirements; primarily concerned with problem-solving using the computer as a tool.

• Language and Interaction:

  • System Software: Developed using Low or Middle-level languages; types include device drivers and linkers; uses languages that are not necessarily readable or user-friendly.

  • Application Software: Uses High-level languages; allows direct user interaction.

• Environment:

  • System Software: Part of the OS installation package; makes machines perform work.

  • Application Software: Cannot run independently; requires the presence of system software to function; makes system programs perform specific work.

• Maintenance and Debugging:

  • System Software: Difficult to make changes to programs; debugging is difficult; error correction (fixing badly written programs) is hard but results are highly reliable.

  • Application Software: User-friendly and modifiable; debugging is not difficult; fault-fixing and error correction are easy and reliable.

Complexity and Management of Operating Systems

• Complexity Scale: Operating systems are categorized as exceptionally complex systems. They are characterized by being large, parallel, expensive, and often not fully understood in every detail.

• Historical Example: Windows NT/XP took approximately $10\,years$ and involved thousands of people in its development. Despite this, bugs still occur.

• Comparisons to Other Complex Systems: Systems programming complexity is compared to systems like the Internet, air traffic control, governments, weather patterns, and human relationships.

• Managing Complexity: OS developers deal with complexity through Abstractions and Layering.

• Primary Goal: To create systems that can be trusted with sensitive data and critical roles in society.

Definition and Structural Components of an Operating System

• The Operating System (OS): A special program that resides between the computer hardware and the application software.

• Logical OS Structure (Hierarchical Layers):

  • User Level Applications: Examples include Quake and SQL Server.

  • Intermediate Tools: System utilities, shells, and windowing/graphics.

  • OS Interface: The boundary between user-level tools and kernel services.

  • Operating System Services: Includes Naming, Windowing & Gfx, Networking, Virtual Memory, Access Control, Generic I/O, File Systems, and Process Management.

  • Device Layer: Device Drivers and Memory Management.

  • Physical Machine Interface: Interrupts, Cache, Physical Memory, Translation Lookaside Buffer (TLB), and Hardware Devices.

The Linux Operating System Environment

• Nature of Linux: A multiuser, multitasking operating system that studies systems programming effectively.

• Functionality: Allows users to run applications, manage system resources, and communicate with external systems.

• Identity: It provides a Unix-like experience but is a free and open-source operating system.

• Distributions (Distros): These consist of the Linux kernel, basic software/utilities, and a software package manager. Common distributions include:

  • Debian (and its derivative, Ubuntu)

  • Fedora

  • Gentoo

  • Kali

• Resources:

  • UNT Laboratory Access: Centralized Environment Linux Labs (CELL) machines.

  • Official site: $https://www.kernel.org/linux.html$

Linux Architecture and System Call Workflow

• Core Components:

  • User Programs: Interacting with the system via system calls.

  • Applications: Programs meant for user tasks.

  • Shell: The user interface responsible for command execution.

  • Kernel: The central part of the OS that interfaces directly with hardware.

• The Execution Lifecycle:

  1. Initiation: A user command or application is initiated to interact with the system.

  2. API Interaction: The API receives the request from the user command/application.

  3. Translation: The API translates the request into specific system calls.

  4. Execution: The system executes the necessary system calls to access hardware resources.

  5. Access: System resources or services are accessed as a result.

The GNU C Library (glibc) and System Libraries

• Function: The system libraries facilitate system calls and provide application services.

• Contents of glibc:

  • Standard C Library and C++ C Library.

  • Wrappers for System Calls.

  • Support for Threading.

• Compatibility: Supports POSIX-compatible applications and Linux-specific applications.

• Version Check: The glibc version on Linux can be checked using specific C language methods.

System Calls: Accessing Kernel Resources

• Native Form: System calls are generally available as assembly-language instructions.

• Development Transition: Languages like C were defined to replace assembly for systems programming, allowing system calls to be made directly within the code.

• Prohibited Actions: Some instructions are strictly prohibited in user mode for security and stability. For example, a program cannot modify page tables while in user mode.

• Function Call vs. System Call:

  • Function Call: Both caller and callee reside in the same process, same user, and same "domain of trust."

  • System Call: The OS is the trusted entity while the user is not; the OS possesses super-privileges. This requires measures to prevent abuse.

Mechanisms for Passing System Call Parameters

1. Use Registers: Parameters are passed directly into CPU registers. This is the fastest method of access.

2. Memory Table: A block or table in memory holds the parameters; the address of this table is then passed in a register.

3. Stack Method: Parameters are pushed onto the stack by the program and popped off by the OS. This method ensures the correct order of parameters is maintained.

System Call Operation and Context Switching

• Cost of System Calls: They are "expensive" in terms of performance due to the requirement of context-switches.

• Operations Steps:

  1. The hardware saves the current state.

  2. OS code takes control of the CPU and updates privilege levels.

  3. OS examines the call parameters for validity.

  4. OS performs the requested function.

  5. OS saves its own state and results.

  6. OS returns CPU control to the caller.

  7. The system transitions from kernel mode back to user mode.

Logical Hierarchy of User and Kernel Modes

• User Mode (Standard Libs/Users):

  • Includes applications, shells, commands, compilers, interpreters, and system libraries.

  • Interfaces with the kernel via a system-call interface.

• Kernel Mode (The Kernel):

  • Signal/Terminal Handling: Character I/O, terminal drivers, terminal controllers.

  • File System: Swapping, block I/O system.

  • Core Management: CPU scheduling, page replacement, demand paging, and virtual memory.

  • Hardware Interface: Disk/tape drivers, memory controllers.

• Hardware:

  • Physical components like terminals, disks, tapes, and physical memory.

Categories of System Calls

1. Process Control: End, Abort, Load, Execute, Create Process, Terminate Process, and Allocate/Free Memory.

2. File Manipulation: Create, Delete, Open, Close, Read, and Write File.

3. Device Manipulation: Request Device and Release Device.

4. Information Maintenance: Get Time and Set Date.

5. Communications: Send Messages and Receive Messages.

Comparing System Calls and Library Calls

Error Handling in System Calls

• Logic Flow:

  • When a system call is made, the program checks if an error occurred.

  • If an error occurs, the call typically returns $-1$.

  • The specific error number is stored in the global variable $errno$.

  • Calling $perror()$ will display a descriptive error message based on the value in $errno$.

  • If no error occurs, the execution continues normally.

Detailed System Call Functions and Syntax

• File Creation: $int\,creat(char*\,filename, mode)$; Example: $creat("newfile", 0666)$; Header: #include .

• Opening Files: $int\,open(char*\,filename, int\,access, int\,mode)$; Example: $open("filename", O\_RDONLY | O\_TRUNC, 0)$; Header: #include .

• Reading Files: $size\_t\,read(int\,fd, char *buffer, size\_t\,bytes)$; Example: $read(0, buf, 100)$; Header: #include .

• Writing Files: $size\_t\,write(int\,fd, char *buffer, size\_t\,bytes)$; Example: $write(1, buf, 100)$; Header: #include .

• Closing Files: $int\,close(int\,fd)$; Example: $close(fd)$; Header: #include .

• Seeking Positions: $long\,lseek(int\,fd, int\,offset, int\,origin)$; Examples: $lseek(fd, 0, 0)$ (start), $lseek(fd, 0, 2)$ (end), $lseek(fd, -10, 1)$ (relative).

• Directory Operations:

  • $mkdir(char*\,name, int\,mode)$ (Header: $sys/types.h, unistd.h$)

  • $rmdir(char*\,name)$ (Header: $sys/types.h, unistd.h$)

  • $opendir(char*\,dir\_name)$ (Header: $sys/dir.h$)

  • $readdir(DIR*\,dp)$ (Header: $sys/dir.h$)

• Permissions: $int\,chmod(const\,char *path, mode\_t\,mode)$; Header: #include .

• Status and Path:

  • $stat(char*\,file, struct\,stat *buf)$ and $fstat(int\,fd, struct\,stat *buf)$ (Header: $sys/stat.h$)

  • $char*\,getcwd(char*\,dirname, int)$ (Header: $unistd.h$)

Rationales for Studying Systems Programming

• Security Concerns: Necessary for understanding and mitigating digital threats.

• Business Issues: critical for informed decision-making regarding technology investments.

• Personal Computers: Provides insight into factors influencing PC performance and hardware choice.

• Building Complex Systems: Teaches how future project complexities can be managed effectively.

• Engineering Issues: Addresses web performance, system design, and the challenges of hardware diversity (different CPUs like Pentium, PowerPC, ARM, or MIPS).

Review of the C Programming Language

• Structure of a C Program:

  • Preprocessor Directive: #include is used to include the standard input/output library, primarily for the $printf()$ function.

  • Comments: Written as $/* comment */$.

  • Main Function: $int\,main()$ is the entry point for all C programs.

  • Strings: A literal C-string like "Hello World!\n" contains $14\,characters$ (including the newline and the hidden NULL terminator).

  • Return: $return\,0;$ indicates the successful termination of the program.

Similarities Between C and C++

• Core Components: They share built-in data types, primary data types (char, int, float, double, void), and secondary types (array, pointer, structure, union, enum).

• Preprocessor: Both use the same compiler preprocessor for directives like #include, #define, #if, #ifndef, and #endif.

• Operators: They use identical built-in operators (Arithmetic, Unary, Multiplicative, Additive, Shift, Relational, Equality, Logical AND/OR/XOR, Conditional, and Assignment).

• Control Structures: Both utilize `$if$, $for$, $while$, and $switch$ statements.

• Execution Point: Both require a function named $main$ to determine where program execution begins.

• Modularity: Both allow programs to be split across separate files.

Key Differences Between C and C++

• Paradigm: C is procedural; C++ is multi-paradigm (procedural and object-oriented).

• Focus: C emphasizes program steps; C++ focuses on data, making implementation and editing often easier.

• Security: In C, data is not inherently secured; in C++, data is secured and hidden through Object-Oriented Programming (OOP) features.

• Language Level: C is considered low-level; C++ is middle-level as it combines low-level hardware focus with high-level program concentration.

• Approach: C uses a Top-down approach (formulated step-by-step); C++ uses a Bottom-up approach (base elements formulated first and then linked).

• Building Blocks: Functions are the building blocks of C; Objects are the building blocks of C++.

• Overloading: C++ supports function overloading; C does not.

• Namespaces: C++ uses the $NAMESPACE$ feature to avoid name collisions; C lacks this feature.

• Input/Output: C uses $scanf()$ and $printf()$; C++ uses $cin$ and $cout$.

• Compatibility: C++ is a superset of C and can run most C code; C cannot run C++ code.

Bitwise Operators and Low-Level Memory Handling

• Smallest Accessible Unit: The smallest type is the 8-bit $char$. Individual bits cannot be accessed directly because they do not have their own unique addresses.

• The Power of Bitwise: Bitwise operators allow for direct manipulation of memory, giving C its power as a low-level language. This is crucial for high-efficiency, real-time applications.

• Integral Types: Bitwise operations only perform on integral types, and unsigned types are generally preferred.

• Common Operators:

  • $\&$ : Bitwise AND

  • $\mid$ : Bitwise OR

  • $\land$ : Bitwise XOR

  • $\sim$ : 1's complement (NOT)

  • $\ll$ : Shift left

  • $\gg$ : Shift right

• Compound Assignment: All can be suffixed with $=$ (e.g., $x \, \&= \, y;$ is equivalent to $x = x \, \& \, y;$).

Specific Bitwise Operations and Masking

• Mathematical Examples (given $x = 60, y = 13$):

  • $x = 0011\,1100$

  • $y = 0000\,1101$

  • $x \, \& \, y = 0000\,1100$

  • $x \, \mid \, y = 0011\,1101$

  • $x \, \land \, y = 0011\,0001$

  • $\sim x = 1100\,0011$

  • $x \ll 2 = 1111\,0000$

  • $x \gg 1 = 0001\,1110$

• Status Storage: Bits can be used to store on/off statuses (e.g., for 8 rooms) within a single $unsigned\,char$ using a mask.

• Masking Rules:

  • Bitwise OR of anything with 1 results in 1.

  • Bitwise AND of anything with 0 results in 0.

  • Bitwise AND of anything with 1 results in the same value.

Memory Manipulation Functions in C

• $void *memcpy (void *dest, const\,void *src, size\_t\,n)$: Copies a block of $n\,bytes$ from $src$ to $dest$. Returns a pointer to $dest$.

• $void *memmove(void *dest, const\,void *src, size\_t\,n)$: Moves a block of $n\,bytes$ from $src$ to $dest$. Returns a pointer to $dest$.

• $void *memset (void *s, int\,c, size\_t\,n)$: Sets the first $n\,bytes$ of memory area $s$ to the constant byte value $c$.

• $void *memchr (const\,void *s, int\,c, size\_t\,n)$: Searches the first $n\,bytes$ of array $s$ for the first occurrence of character $c$. Returns a pointer to the location.

• $int\,memcmp (const\,void *s1, const\,void *s2, size\_t\,n)$: Compares the first $n\,bytes$ of arrays $s1$ and $s2$. Operation is case-sensitive.

Questions & Discussion

• Interaction of Software Types: "How do System Software and Application Software work together to help us use a computer effectively? Can you give examples of both?"

• Comparison of Platforms: "Can you compare two different operating systems—like Windows and macOS or Android and iOS—and discuss how they might offer different experiences for users?"

• Structural Deliberation: Questions raised regarding system design include:

  • Should the programmer write a single program that performs many independent activities?

  • Does every program have to be altered for every piece of hardware?

  • Does a faulty program crash everything?

  • Does every program have access to all of the hardware?", "title": "Systems Programming and Operating Systems Fundamentals" }