Unix - Shell Programming

Shell Programming

Introduction to Shell Programs

Unix provides a vast array of commands, but inevitably, there will be tasks for which no existing command is perfectly suited.
In such scenarios, users can build their own solutions by writing shell programs (also known as shell scripts).
The shell itself functions as a programming language.
A shell program is fundamentally a sequential list of Unix commands that are executed in the specified order, akin to a batch file or a list of programs to run.

Comments

Comments in a shell script begin with the # symbol.
Example:
bash # This is a comment
Comments are ignored by the shell interpreter; their sole purpose is to improve human readability and understanding of the code.
It is good practice to use comments to explain the logic and purpose of different parts of your script.

Shell Executables and Execution Methods

Basic Structure of a Shell Script: A typical shell script starts with a "shebang" line, which specifies the interpreter to use for running the script.
- Example Script (testScript.sh):
  bash #!/bin/bash echo "hello world!" exit 0
- When executed, this script would print "hello world!" to the console.
Methods for Running Shell Executables:
1. source FILENAME: Runs the script in the current shell environment.
2. /bin/bash FILENAME (or sh FILENAME): Explicitly invokes the specified shell interpreter to run the script.
3. Making the file executable:
  - Change file permissions: chmod +x FILENAME
  - Then, run it directly: ./FILENAME (if in current directory) or /path/to/FILENAME.
  - Shebang Line: The first line of the script, #!/bin/bash (or #!/usr/bin/perl for Perl scripts, etc.), tells the operating system which interpreter should be used when the script is executed directly.

Return Values and Exit Codes

Every program and shell command returns an integer value upon completion.
This return value (or exit code) communicates the success or failure of the program to other programs or to the shell.
A return value of $0$ typically signifies successful execution or a True condition.
A return value of $1$ (or any other non-zero integer) generally indicates failure or a False condition.

Variable Discussion

Definition and Naming Conventions

Shell variables are used to store values within a script.
Naming Rules:
- Can consist of any combination of letters, numbers, and the underscore (_) symbol.
- Must begin with a letter or an underscore.
- Are case-sensitive (e.g., D and d are considered two distinct variables).

Setting and Accessing Variables

Setting Variables: Assignment is done using the $=$ operator.
- No prior declaration of variable type is necessary; all variables are treated as strings of characters by default.
- Example: d=date (assigns the string date to variable d).
Accessing Variables: Values are retrieved by prefixing the variable name with a symbol.
- The shell performs variable substitution, replacing the variable name with its stored value.
- Examples:
  - echo $d
  - echo ${d}

The Difference Between and `{ }` for Variables

Using $VAR is often sufficient, but VAR can cause ambiguity if the variable name is immediately followed by characters that could be part of a longer variable name.
Example Scenario: If you want to print "blueSky" and have Color=blue.
- echo "$ColorSky": The shell will attempt to find a variable named ColorSky, which likely doesn't exist, resulting in an empty output.
- echo "${Color}Sky": The curly braces { } explicitly delineate the variable name (Color), ensuring that Sky is treated as a separate literal string appended to the variable's value.
  - This would correctly output "blueSky".

Variable Usage Examples

d=date
- echo $d -> date (prints the string "date")
- echo ${d}: -> date: (prints the string "date" followed by a colon)
- $d -> Executing date as a command (since d holds the string date, which is a recognized command), outputs system date/time (e.g., Thu Feb 10 07:43:51 PST 2005).
- ${d} -> Also executes date as a command, outputs system date/time (e.g., Thu Feb 10 07:44:02 PST 2005).
- Note: If a variable contains a command name, using VAR or ${VAR} without quotes will execute that command.

Reading User Input into Variables

Usage: read var1 var2 ...
The read command reads values from standard input (typically the keyboard) and assigns them sequentially to the specified variables.
Behavior with multiple words/variables:
- If more words are typed by the user than there are variables, the excess words are all assigned to the last variable.
- If more variables are specified than words provided by the user, the excess variables remain empty.
Examples:
- read var, then user types hello -> echo $var outputs hello.
- read var var2, then user types hello world -> echo $var $var2 outputs hello world.
- read var, then user types hello again world -> echo $var outputs hello again world (entire line goes to var).
- read var var2, then user types hello -> echo $var2 outputs an empty line (because var2 received no input).

Single, Double, and Backtick Quotes

Single Quotes (''): Do not interpret any variables, commands, or special characters. The text within single quotes is treated as a literal string.
- Example: echo '$d' outputs $d.
Double Quotes (""): Interpret variables (VAR $</code>), command substitutions (<code>$ ext{(command)}), and some escape sequences, but prevent word splitting and pathname expansion.
- Example: echo "$d" outputs date (if d=date).
Backticks (`): Treat the enclosed content as a command to be executed. The output of that command is then substituted into the current command.
- Example: If d=date, then echo exttt{ extasciigrave}date exttt{ extasciigrave} executes the date command and outputs its result (e.g., Thu Feb 10 07:21:07 PST 2005). This is a form of command substitution.

Command Line Parameters

Accessing Command Line Inputs

Just like other Unix programs, shell scripts can receive parameters directly from the command line when they are executed.
This differs from user input (using read), as command-line parameters are known before the script begins execution, not typed in during execution.
Example: If you run testScript.sh testFile, testFile is a command-line parameter.

Special Variables for Command Line Parameters

The shell provides special built-in variables to access command-line arguments:
- 0 $</code>: Represents the name of the running script itself.</li><li><code>$ 1 $</code> - <code>$ 9 $</code>: Represent the first through ninth arguments passed to the script.</li><li><code>$ * $</code>: Represents all command-line arguments as a single string.</li><li><code>$ # $</code>: Represents the total number of command-line arguments passed to the script (excluding <code>$ 0).

Example of Command Line Parameters

Script (testScript.sh):
bash #!/bin/bash echo "The name of the program is: $0" echo "The first parameter is: $1" echo "There are $# parameters" echo "All of the parameters are: $*" exit 0
Execution: testScript.sh par1 par2 par3 par4
Output:
text The name of the program is: /testScript.sh The first parameter is: par1 There are 4 parameters All of the parameters are: par1 par2 par3 par4

Arithmetic Discussion

Basic Arithmetic Operations

Variables containing numeric values can be treated arithmetically (added, subtracted, etc.).

The `(( ))` Command for Arithmetic

Usage: result=$(( EXPRESSION )) evaluates the EXPRESSION and substitutes its numerical result.
- Parentheses can also be used explicitly (( EXPRESSION )) in contexts like if statements or for simple evaluation without outputting results.
Supported Operators:
- + $</code>: Addition</li><li><code>$ - $</code>: Subtraction</li><li><code>$ * $</code>: Multiplication</li><li><code>$ / $</code>: Division</li><li><code>$ %: Modulo (remainder)
Examples:
- echo $(( 1 + 6 )) outputs 7.
- echo $(( 2 * 3 )) outputs 6.
- echo $(( 4 % 3 )) outputs 1.

Other Arithmetic Options

( ) $</code>: Used for grouping expressions, influencing order of operations.</li><li><code>$ | $</code>: Bitwise/logical OR (check <code>man</code> page for full details).</li><li><code>$ & $</code>: Bitwise/logical AND (check <code>man</code> page for full details).</li><li>Many other operators are available; consult the shell's <code>man</code> page (e.g., <code>man bash</code>) for a comprehensive list.</li></ul><h4 id="31e3caa8-d629-4574-a908-1607f424db64" data-toc-id="31e3caa8-d629-4574-a908-1607f424db64" collapsed="false" seolevelmigrated="true">Control Flow Statements</h4><h5 id="abdaa56c-015e-466e-a510-1398c21d3b91" data-toc-id="abdaa56c-015e-466e-a510-1398c21d3b91" collapsed="false" seolevelmigrated="true">Overview</h5><ul><li>Shell scripts can be more sophisticated than just a linear list of commands.</li><li>Control flow statements allow scripts to make decisions and repeat actions based on conditions.</li><li>Example Use Case: Printing a file only if it is not a binary file.</li></ul><h5 id="96ee9559-baea-4f82-b522-b67b19637385" data-toc-id="96ee9559-baea-4f82-b522-b67b19637385" collapsed="false" seolevelmigrated="true">The <code>test</code> Command</h5><ul><li>Purpose: Used to check if specific conditions are true.</li><li>Usage: <code>test EXPRESSION</code></li></ul><h5 id="0f9646a1-e35a-4c23-bef2-507704917440" data-toc-id="0f9646a1-e35a-4c23-bef2-507704917440" collapsed="false" seolevelmigrated="true"><code>test</code> Command Conditions: Comparisons</h5><ul><li>Numeric Comparisons (for integers):<ul><li><code>-eq</code>: Equal to (<code>$ =) (e.g., test $VAR1 -eq $VAR2)
-ne: Not equal to ( ext{!}=) (e.g., test $VAR1 -ne $VAR2)
-lt: Less than (<) (e.g., test $VAR1 -lt $VAR2)
-le: Less than or equal to ( ext{<=}) (e.g., test $VAR1 -le $VAR2)
-gt: Greater than (>) (e.g., test $VAR1 -gt $VAR2)
-ge: Greater than or equal to ( ext{>=}) (e.g., test $VAR1 -ge $VAR2)

`test` Command Conditions: System/File Checks

-d FILE: True if FILE exists and is a directory.
-e FILE: True if FILE exists.
-f FILE: True if FILE exists and is a regular file (not a directory or special file).
-s FILE: True if FILE exists and its size is greater than zero (i.e., non-empty).
-x FILE: True if FILE exists and is executable.

`test` Command Conditions: Logical Operators

! EXPRESSION: Logical NOT. Negates the result of the following check.
- Example: test ! -x tFile (True if tFile is not executable).
EXPRESSION1 -a EXPRESSION2: Logical AND. True if both EXPRESSION1 and EXPRESSION2 are true.
- Example: test $1 -eq $2 -a $2 -gt 5 (True if first arg equals second arg AND second arg is greater than 5).
EXPRESSION1 -o EXPRESSION2: Logical OR. True if EXPRESSION1 is true OR EXPRESSION2 is true.
- Example: test $1 -eq $2 -o $2 -gt 5 (True if first arg equals second arg OR second arg is greater than 5).

Shortcut for `test`: Square Brackets `[ ]`

The test command is very frequently used, so a concise shortcut [ ] exists.
Usage: [ EXPRESSION ]. Crucially, there must be spaces around the brackets and the expression.
Example: [ -f tFile ] is equivalent to test -f tFile.

Application of `test` and `[ ]`

These commands are fundamental for controlling the flow of a script.
The result of a test or [ ] condition (success/failure, represented by the exit code) guides subsequent execution.
They are primarily utilized within conditional statements like if.

Conditional Statements

`if then fi`

Purpose: Executes a block of STATEMENTS only if a CONDITION is true.
Syntax:
bash if [ CONDITION ]; then STATEMENTS fi
Flowchart: Condition $\rightarrow$ Yes $\rightarrow$ Statements $\rightarrow$ End if
Condition $\rightarrow$ No $\rightarrow$ End if
Example Script (safeCat.sh):bash #!/bin/bash if [ -x "$1" ]; then cat "$1" fi
- Running safeCat.sh regularFile (assuming regularFile is executable, cat would print its content).
- Running safeCat.sh /bin/ls (assuming /bin/ls is executable, cat would attempt to print its binary content, leading to control characters, or in context of the example, no desired output).

`if then else fi`

Purpose: Executes one block of STATEMENTS-YES if the CONDITION is true, and a different block of STATEMENTS-NO if the CONDITION is false.
Syntax:
bash if [ CONDITION ]; then STATEMENTS-YES else STATEMENTS-NO fi
Flowchart: Condition $\rightarrow$ Yes $\rightarrow$ Statements-Yes $\rightarrow$ End if
Condition $\rightarrow$ No $\rightarrow$ Statements-No $\rightarrow$ End if
Example Script (safeCat.sh):bash #!/bin/bash if [ -x "$1" ]; then cat "$1" else echo "Executable: not printing $1" fi
- Running safeCat.sh regularFile (if executable) $\rightarrow$ Prints regularFile's content.
- Running safeCat.sh /bin/ls (if executable, but cat is not intended for it) $\rightarrow$ Outputs Executable: not printing /bin/ls.

`if then elif else fi`

Purpose: Allows for multiple conditions to be checked sequentially. If the first CONDITION1 is true, STATEMENTS-1 execute. Otherwise, CONDITION2 is checked, and STATEMENTS-2 execute if true. If neither is true, STATEMENTS-3 (the else block) execute.
Syntax:
bash if [ CONDITION1 ]; then STATEMENTS-1 elif [ CONDITION2 ]; then STATEMENTS-2 else STATEMENTS-3 fi
Flowchart: Condition1 $\rightarrow$ Yes $\rightarrow$ Statements-1 $\rightarrow$ End if
Condition1 $\rightarrow$ No $\rightarrow$ Condition2 $\rightarrow$ Yes $\rightarrow$ Statements-2 $\rightarrow$ End if
Condition1 $\rightarrow$ No $\rightarrow$ Condition2 $\rightarrow$ No $\rightarrow$ Statements-3 $\rightarrow$ End if
Example Script (safeCat.sh):bash #!/bin/bash if [ -x "$1" ]; then cat "$1" elif [ "$1" == "/bin/ls" ]; then echo "I see /bin/ls" else echo "Executable: not printing $1" fi
- Running safeCat.sh regularFile (if executable) $\rightarrow$ Prints regularFile's content.
- Running safeCat.sh /bin/ls $\rightarrow$ Outputs I see /bin/ls (matches elif condition).
- Running safeCat.sh testScript.sh (if testScript.sh is not executable in this specific test or otherwise falls through the if and elif checks) $\rightarrow$ Outputs Executable not printing testScript.sh.

`case esac`

Purpose: Provides a clean way to execute different blocks of code based on matching a STRING against several patterns, often useful for menu-driven scripts or handling different file types.
Syntax:bash case STRING in pattern1) STATEMENTS-1 ;; pattern2) STATEMENTS-2 ;; *) STATEMENTS-DEFAULT ;; esac
- Each pattern block ends with ;;.
- The * pattern acts as a wildcard, matching anything not caught by preceding patterns (default case).
Flowchart: Evaluates STRING $\rightarrow$ Checks pattern1 $\rightarrow$ If match, STATEMENTS-1 $\rightarrow$ Exit case
$\rightarrow$ Else, checks pattern2 $\rightarrow$ If match, STATEMENTS-2 $\rightarrow$ Exit case
$\rightarrow$ Else, checks patternN $\rightarrow$ If match, STATEMENTS-N $\rightarrow$ Exit case
Example Script (caseScript.sh):bash #!/bin/bash case $1 in *sh) echo "Shell Script" ;; *.txt) echo "Text File" ;; *) echo "Some Other File" ;; esac
- Running caseScript.sh testScript.sh $\rightarrow$ Outputs Shell Script.
- Running caseScript.sh file.txt $\rightarrow$ Outputs Text File.
- Running caseScript.sh blah $\rightarrow$ Outputs Some Other File.

Looping Constructs

Overview of Loops

Loops are used to perform a set of STATEMENTS repeatedly.
Shell scripting offers different types of loops:
- for loop: Iterates for a set number of items in a list.
- while loop: Continues as long as a condition remains true.
- until loop: Continues until a condition becomes true (i.e., as long as it's false).

`for` Loop

Purpose: Iterates over a LIST of items, assigning each item in turn to a VAR and executing a block of STATEMENTS for each iteration.
Syntax:
bash for VAR in LIST do STATEMENTS done
Flowchart: Initialize VAR from LIST $\rightarrow$ If LIST is not empty:
Execute STATEMENTS
Assign next item to VAR
Repeat
$\rightarrow$ If LIST is empty (or exhausted): Exit for loop.
Example Script (forScript.sh):
```
#!/bin/bash
for myVar in $* # Iterates through all command-line arguments ($* provides the list)
do
    echo "$myVar"
done
```
- Running forScript.sh arg1 arg2 $\rightarrow$ Outputs arg1 (on one line), then arg2 (on another line).
Special Form of for Loop (without explicit LIST):
- If the in LIST part is omitted, the for loop defaults to iterating through all of the command-line arguments ($$).
- Syntax:
  bash for VAR do STATEMENTS done
- Example (forScript.sh - special form): This script behaves identically to the previous example. bash #!/bin/bash for myVar do echo "$myVar" done
  - Running forScript.sh arg1 arg2 $\rightarrow$ Outputs arg1, then arg2.

`while` Loop

Purpose: Continuously executes a block of STATEMENTS as long as a specified CONDITION remains true.
Syntax:
bash while CONDITION do STATEMENTS done
Flowchart: Check CONDITION $\rightarrow$ If True:
Execute STATEMENTS
Loop back to CONDITION check
$\rightarrow$ If False: Exit while loop.
Example Script (whileScript.sh):bash #!/bin/bash counter=0 while [ "$counter" -lt 10 ] do echo "$counter" counter=$(( counter + 1 )) # Increment counter using modern arithmetic expansion done
- Running whileScript.sh $\rightarrow$ Outputs numbers 0 through 9, each on a new line.

`until` Loop

Purpose: Continuously executes a block of STATEMENTS as long as a specified CONDITION remains false (i.e., until the condition becomes true).
Syntax:
bash until CONDITION do STATEMENTS done
Flowchart: Check CONDITION $\rightarrow$ If False:
Execute STATEMENTS
Loop back to CONDITION check
$\rightarrow$ If True: Exit until loop.
Example Script (untilScript.sh):bash #!/bin/bash counter=0 until [ "$counter" -eq 10 ] do echo "$counter" counter=$(( counter + 1 )) # Increment counter using modern arithmetic expansion done
- Running untilScript.sh $\rightarrow$ Outputs numbers 0 through 9, each on a new line.

Unix - Shell Programming

Shell Programming

Introduction to Shell Programs

Comments

Shell Executables and Execution Methods

Return Values and Exit Codes

Variable Discussion

Definition and Naming Conventions

Setting and Accessing Variables

The Difference Between and { } for Variables

Variable Usage Examples

Reading User Input into Variables

Single, Double, and Backtick Quotes

Command Line Parameters

Accessing Command Line Inputs

Special Variables for Command Line Parameters

Example of Command Line Parameters

Arithmetic Discussion

Basic Arithmetic Operations

The (( )) Command for Arithmetic

Other Arithmetic Options

test Command Conditions: System/File Checks

test Command Conditions: Logical Operators

Shortcut for test: Square Brackets [ ]

Application of test and [ ]

Conditional Statements

if then fi

if then else fi

if then elif else fi

case esac

Looping Constructs

Overview of Loops

for Loop

while Loop

until Loop

The Difference Between and `{ }` for Variables

The `(( ))` Command for Arithmetic

`test` Command Conditions: System/File Checks

`test` Command Conditions: Logical Operators

Shortcut for `test`: Square Brackets `[ ]`

Application of `test` and `[ ]`

`if then fi`

`if then else fi`

`if then elif else fi`

`case esac`

`for` Loop

`while` Loop

`until` Loop