Operating Systems & Parallel Computing

Session 1: Introduction

Operating System (OS) → A type of software that manages computer hardware resources and provides common services for computer programs

• Acts as an interface between software and hardware
• Controls and monitors the execution of programs (apps and system software)
• Ex) Windows, Linux, Mac OS, IOS

Why Do We Need an OS?

• Device Management → OS manages all devices connected to a computer
  • Printers, scanners, external hard drives
• Memory Management → OS manages the memory of a computer
  • Ensures each program gets the necessary memory to run correctly
• Process Management: OS manages all the processes running on a computer
  • Ensures each process gets the necessary resources to run correctly
  • Allows multiple programs to be in memory and run at the same time
• Security: OS provides safety features to protect from viruses/malware
• User Interface (UI) → OS provides a user interface that allows us to interact with the computer and run apps

Kernel Mode → The processor mode that enables software to have full and unrestricted access to the system and its resources

Machine Level:

1. Instruction set
2. Memory Organization → To store data
3. Input/Output
4. Bus Structure → Communication system that transfers data between components inside a computer (or between computers)

• Internal Bus → Connects computer internals to the motherboard
• May be:

1. Parallel → Multiple bits of data are transmitted simultaneously over multiple channels
2. Serial → Data is transmitted one bit at a time over a single channel

• Ex) USB (universal serial bus)

Device Driver → A group of files that enable one or more hardware devices to communicate with the computer’s OS

• Without drivers, the computer wouldn’t be able to send/receive data correctly to hardware devices

Resources Managed by OS:

1. Processors
2. Memory
3. Timers
4. Disks → RAM, SSD
5. Mice → allows for us to control the movement of computer cursor
6. Network interfaces

Bus Interactions:

Types of OS:

1. Mainframes → Designed to run on large, centralized computers that handle high volumes of data processing and storage
2. Servers → Specialized computers that operate within a client/server architecture to serve the requests of client computers on the network
3. Multiprocessors → Used to boost the performance of multiple CPUs within a single computer system

• CPUs are linked together so that a job can be divided and executed more quickly

1. Personal Computers → Provide a good interface for a single user

• Used for word processing, spreadsheets, and internet access

1. Handheld (Mobile) → Designed to run on mobile devices

• Touchscreens, mobile apps, wireless connectivity

1. Embedded → Designed to run on embedded systems such as consumer electronics, industrial machines, and automobiles

• Low power consumption and small memory footprint

1. Sensor Node → Run on small devices that can sense and transmit data wirelessly (sensor)

• Provide low power consumption, real time processing, wireless connectivity

1. Real Time → Designed to handle real time applications that require immediate response to external events
2. Smart Cards → Run on small plastic cards with embedded sensors microchips

• Provide secure storage and cryptographic functions

Session 2: Processes & Threads

Lifecycle Of A Process:

1. Edit Time → Phase in which source code of a program is written, modified, or reviewed by a programmer
2. Compile Time → Phase in which source code of a program is translated into machine-readable code by a compiler (object code)
3. Link Time → Phase in which the object code is linked with other object files and libraries to create an executable file or a shared library
4. Distribution Time → Phase in which the executable file/shared library is packaged and distributed to end-users or customers (i.e. downloads, media, etc.)
5. Installation Time → Phase in which the end-user installs the software in their computer
6. Load Time → Phase in which the OS loads the executable file/shared library into memory and prepares it for execution
7. Run Time → Phase in which the program is executed by the computer

Thread → A sequence of instructions that can be executed independently by a computer’s CPU

• Can run concurrently with other threads
• Share the same memory space as the parent process
• Improve the performance of applications that require parallel processing or multitasking

Multithreading → Allows multiple threads of execution to run concurrently on a single CPU

Threadpool → Collection of worker threads that are maintained by the OS and are waiting for tasks to be allocated for concurrent execution by the program

Locking → Mechanism that enforces limits on access to a shared resource when there are many threads

• Used to ensure that only one thread can access a shared resource at a time

Session 3: Kernels

Kernel → The main layer between an OS and the underlying computer hardware

• Processes memory management, file systems, device control, and network

Purpose Of A Kernel:

1. Provides interfaces needed for users and applications to interact with the computer
2. Launches and manages applications
3. Manages underlying system hardware devices

Scheduler → Component of an OS that manages the allocation of resources and the execution of tasks

• Decides which task to perform next
• Ensures each task gets its fair share of resources
• Ensures that the system remains responsive and efficient
• Goals:

1. Maximize throughput
2. Minimize wait time
3. Minimize latency/response time
4. Maximize fairness (equal CPU time)

Windows Scheduler/Task Scheduler → Check scheduler in terminal

Create Task → edit crontab (list of commands scheduled to run at specific times)

Vim & Nano:

• Editors on Linux
• Vim:
  • Mode based text editor → You need to enter INSERT mode before you can write text to the file
• Nano:
  • Modeless editor
  • What you see is what you get command line text editor

Linux Commands:

1. ls → List and directories in the current directory
2. cat → Joins and displays file
3. touch → Creates an empty fil or updates the timestamp of an existing file
4. mkdir → Creates a new directory
5. rmdir → Deletes an empty directory
6. grep → Searches for a pattern in a file or input
7. traceroute → Traces the route taken by packets over an IP network
8. ping → Tests the reachability of a host on an IP network
9. chown → Changes the owner of a file or directory
10. chmod → Changes the permissions of a file or directory
11. alias → Creates an alias for a command or sequence of commands
12. pwd → Prints the current working directory
13. cd → Changes the current working directory
14. cp → Copies files and directories from one location to another
15. rm → Removes files and directories
16. mv → Moves or renames files and directories
17. sudo → Executes a command with elevated privileges
18. echo → Prints text to the terminal or redirects it to a file
19. vim → A text editor that allows users to create, edit, and view files
20. nano → A text editor that allows users to create, edit, and view files
21. which → Displays the location of an executable file in the user’s PATH environment variable
22. tail → Displays the last few lines of a file or input stream
23. head → Displays the first few lines of a file or input stream
24. wc → Counts the number of lines, words, and characters in a file or input stream
25. find → Searches for files and directories that match specified criteria in a given location
26. wget → Downloads files from the internet

Session 4: Interprocess Communication

Race Conditions → When two or more processes are reading/writing some shared data and the final result depends on who runs precisely when

Critical Regions → The resource that is being shared between processes

Problems:

1. No two processes may be simultaneously inside their critical regions
2. No assumptions may be made about the speeds or the number of CPUs
3. No process running outside its critical region may block any process
4. No process should have to wait forever to enter its critical region

Semaphore → An integer variable that is shared between threads

• Used to solve the critical section problem and to achieve process synchronization
• Binary Semaphore (Mutex) → Can only have two values (0 or 1) and it’s initialized to 1
  • Used to implement the solution of critical section problems with multiple processes
• Counting Semaphore → Its value can range over an unrestricted domain
  • Used to control access to a resource that as multiple instances

Beautiful Soup Library → Python library used for pulling data out of HTML and XML files

• Navigates, searches, modifies the parse tree
• Saves hours/days of work

Session 6: Linux OS & Directory Structure

Root:

• Every single file and directory starts from the root directory
• Only root user has writing privileges under this directory
• /root is the root user’s home directory
  • Not the same as /

/bin:

• Contains binary executable files
• Ex) ps, ls, ping, cp

/sbin:

• Contains binary executable files
• Used by the system administrator for system maintenance
• Ex) iptables, reboot, fdisk, ifconfig, swapon

/etc:

• Contains configuration files required by all programs
• Contains startup/shutdown shell scripts used to start/stop individual programs

/dev:

• Contains device files
• Terminal devices, usb, or any device attached to the system

/proc:

• Contains info about system process
• Pseudo file system that contains info about running processes
• Virtual file system that contains info about system resources

/var:

• Variable files
• Content of files expected to grow can be found here

/temp:

• Directory that contains temporary files created by the system/users
• Files here are deleted when system is rebooted

/src:

• Contains binaries, libraries, documentation, and source code for second level programs

/home:

• Home directories for all users to store their personal files

/boot:

• Contains boot loader related files
• Kernel initrd

/lib:

• Contains library files that support the binaries located under /bin and /sbin
• File names are either ld* or lib*.so.*

/opt:

• Optional
• Contains add on applications from individual vendors

/mnt:

• Temporary mount directory where sysadmins can mount file systems

/media:

• Temporary mount directory for removable devices

/srv:

• Service
• Contains server specific services related data

Daemons → A program with a unique purpose

• Utility programs that run silently in the background to monitor and take care of certain subsystems
  • To ensure the OS runs properly

Types of Processes:

1. Interactive Processes → Run by a user at the command line
2. Batch Processes → Not associated with the command line and are presented from a list of processes

• “Groups of tasks”
• Bets when system usage is low

1. Daemon Processes

• rsyslogd → Used to log system messages
• udisksd → Handles querying, mounting, unmounting, formatting, detaching storage devices
• logind → A tiny daemon that manages user logins and seats in various ways
• httpd → the HTTP service manager
• sshd → Responsible for managing the SSH service which is used to securely log onto remote syste,s
• ftpd → manages the File Transfer Protocol (FTP) which transfers files between computers
• crond → Scheduler daemon for time based actions such as software updates

Sessions 8 & 9: Files

File Attributes:

1. File type
2. File permissions

• User → The owner of the file (creator by default)
• Group → Contains multiple users that have the same permissions access to a file
• Other → Any other user who has access to the file

1. Number of links
2. UID of the owner (User Identifier which assigns a number to each user)
3. GID of the group owner (Group Identifier which assigns a number to each group
4. File size in bytes
5. Date and time of last modification
6. Date and time of las access
7. Date and time of the last change of the inode
8. An array of pointers that keep track of all disk blocks used by the file

Types of Permissions:

• Can be combined

1. R → read permission
2. W → write permission
3. X → execute permission
4. - → no permission

Linux Groups:

1. Primary/Log in Group → The group that is assigned to the files that are created by the user
2. Secondary/Supplementary Group → Used to grant certain privileges to a set of users

• A user an be a member of 0 or more secondary groups

Pipe Command → Helps combine two or more commands and are used as input/output concepts in a command

• |

Redirecting to a File → When we wish the program’s output to be saved in a file instead of printed to the screen

• >

Session 10: SSH

Secure Shell Protocol (SSH) → Cryptographic network protocol for operating network services securely over an unsecured network

• Applications have a client-server architecture
  • Connect SSH client instance with an SSH server

SSH Principal Components:

1. The transport layer provides server authentication, confidentiality, and integrity
2. The user authentication protocol validates the user to the server
3. The connection protocol multiplexes the encrypted tunnel into multiple logical communication channels

SSH Keys:

• Usef to provide a secure and scalable method of authentication
  • Passwords aren’t as effective as they once were
• Come in pairs: private and public key
• Public Key → Can be shared freely with any SSH server to which the user wishes to connect
• Normally managed by an organization’s IT team
• When the public key is present on the remote end and the matching private key is present on the local end, typing the password is no longer necessary
  • For additional security it may be necessary

Cryptographic Key → A string of characters used within an encryption algorithm for altering data so that it appears random

• Locks (encrypts) data
• Only someone with the right key can unlock (decrypt) it

Session 11: Drivers

Driver → A software component that lets the OS and device communicate with each other

• Not all drivers are written by the company that designed the device
• Not all drivers communicate directly with a device
  • There are often several drivers layered in a driver stack

Function Driver → The one driver in the stack that communicates directly with the device

Filter Driver → The driver that performs auxiliary functions

Selenium → Python API used to access all functionalities of Selenium WebDriver in an intuitive way