Unix - Shell Programming
Shell Programming
Introduction to Shell Programs
Unix provides a vast array of commands, but inevitably, there will be tasks for which no existing command is perfectly suited.
In such scenarios, users can build their own solutions by writing shell programs (also known as shell scripts).
The shell itself functions as a programming language.
A shell program is fundamentally a sequential list of Unix commands that are executed in the specified order, akin to a batch file or a list of programs to run.
Comments
Comments in a shell script begin with the
#symbol.Example:
bash # This is a commentComments are ignored by the shell interpreter; their sole purpose is to improve human readability and understanding of the code.
It is good practice to use comments to explain the logic and purpose of different parts of your script.
Shell Executables and Execution Methods
Basic Structure of a Shell Script: A typical shell script starts with a "shebang" line, which specifies the interpreter to use for running the script.
Example Script (
testScript.sh):bash #!/bin/bash echo "hello world!" exit 0When executed, this script would print
"hello world!"to the console.
Methods for Running Shell Executables:
source FILENAME: Runs the script in the current shell environment./bin/bash FILENAME(orsh FILENAME): Explicitly invokes the specified shell interpreter to run the script.Making the file executable:
Change file permissions:
chmod +x FILENAMEThen, run it directly:
./FILENAME(if in current directory) or/path/to/FILENAME.Shebang Line: The first line of the script,
#!/bin/bash(or#!/usr/bin/perlfor Perl scripts, etc.), tells the operating system which interpreter should be used when the script is executed directly.
Return Values and Exit Codes
Every program and shell command returns an integer value upon completion.
This return value (or exit code) communicates the success or failure of the program to other programs or to the shell.
A return value of
typically signifies successful execution or a True condition.A return value of
(or any other non-zero integer) generally indicates failure or a False condition.
Variable Discussion
Definition and Naming Conventions
Shell variables are used to store values within a script.
Naming Rules:
Can consist of any combination of letters, numbers, and the underscore (
_) symbol.Must begin with a letter or an underscore.
Are case-sensitive (e.g.,
Danddare considered two distinct variables).
Setting and Accessing Variables
Setting Variables: Assignment is done using the
operator.No prior declaration of variable type is necessary; all variables are treated as strings of characters by default.
Example:
d=date(assigns the stringdateto variabled).
Accessing Variables: Values are retrieved by prefixing the variable name with a
symbol.The shell performs variable substitution, replacing the variable name with its stored value.
Examples:
echo $decho ${d}
The Difference Between { } for Variables
Using
$VARis often sufficient, butVARColorSky": The shell will attempt to find a variable namedColorSky, which likely doesn't exist, resulting in an empty output.echo "{Color}Sky": The curly braces{ }explicitly delineate the variable name (Color), ensuring thatSkyis treated as a separate literal string appended to the variable's value.This would correctly output
"blueSky".
Variable Usage Examples
d=dateecho $d->date(prints the string "date")echo ${d}:->date:(prints the string "date" followed by a colon)d-> Executingdateas a command (sincedholds the stringdate, which is a recognized command), outputs system date/time (e.g.,Thu Feb 10 07:43:51 PST 2005).VARor${VAR}without quotes will execute that command.
Reading User Input into Variables
Usage:
read var1 var2 ...The
readcommand reads values from standard input (typically the keyboard) and assigns them sequentially to the specified variables.Behavior with multiple words/variables:
If more words are typed by the user than there are variables, the excess words are all assigned to the last variable.
If more variables are specified than words provided by the user, the excess variables remain empty.
Examples:
read var, then user typeshello->echo $varoutputshello.read var var2, then user typeshello world->echo $var $var2outputshello world.read var, then user typeshello again world->echo $varoutputshello again world(entire line goes tovar).read var var2, then user typeshello->echo $var2outputs an empty line (becausevar2received no input).
Single, Double, and Backtick Quotes
Single Quotes (
''): Do not interpret any variables, commands, or special characters. The text within single quotes is treated as a literal string.Example:
echo 'd'outputsVAR ext{(command)}), and some escape sequences, but prevent word splitting and pathname expansion.Example:
echo "$d"outputsdate(ifd=date).
Backticks (
`): Treat the enclosed content as a command to be executed. The output of that command is then substituted into the current command.Example: If
d=date, thenecho exttt{ extasciigrave}date exttt{ extasciigrave}executes thedatecommand and outputs its result (e.g.,Thu Feb 10 07:21:07 PST 2005). This is a form of command substitution.
Command Line Parameters
Accessing Command Line Inputs
Just like other Unix programs, shell scripts can receive parameters directly from the command line when they are executed.
This differs from user input (using
read), as command-line parameters are known before the script begins execution, not typed in during execution.Example: If you run
testScript.sh testFile,testFileis a command-line parameter.
Special Variables for Command Line Parameters
The shell provides special built-in variables to access command-line arguments:
019*#0).
Example of Command Line Parameters
Script (
testScript.sh):bash #!/bin/bash echo "The name of the program is: $0" echo "The first parameter is: $1" echo "There are $# parameters" echo "All of the parameters are: $*" exit 0Execution:
testScript.sh par1 par2 par3 par4Output:
text The name of the program is: /testScript.sh The first parameter is: par1 There are 4 parameters All of the parameters are: par1 par2 par3 par4
Arithmetic Discussion
Basic Arithmetic Operations
Variables containing numeric values can be treated arithmetically (added, subtracted, etc.).
The
(( ))Command for ArithmeticUsage:
result=$(( EXPRESSION ))evaluates theEXPRESSIONand substitutes its numerical result.Parentheses can also be used explicitly
(( EXPRESSION ))in contexts likeifstatements or for simple evaluation without outputting results.
Supported Operators:
+-*/%: Modulo (remainder)
Examples:
echo $(( 1 + 6 ))outputs7.echo $(( 2 * 3 ))outputs6.echo $(( 4 % 3 ))outputs1.
Other Arithmetic Options
( ) |&=) (e.g.,test $VAR1 -eq $VAR2)-ne: Not equal to (ext{!}=) (e.g.,test $VAR1 -ne $VAR2)-lt: Less than (<) (e.g.,test $VAR1 -lt $VAR2)-le: Less than or equal to (ext{<=}) (e.g.,test $VAR1 -le $VAR2)-gt: Greater than (>) (e.g.,test $VAR1 -gt $VAR2)-ge: Greater than or equal to (ext{>=}) (e.g.,test $VAR1 -ge $VAR2)
test Command Conditions: System/File Checks
-d FILE: True ifFILEexists and is a directory.-e FILE: True ifFILEexists.-f FILE: True ifFILEexists and is a regular file (not a directory or special file).-s FILE: True ifFILEexists and its size is greater than zero (i.e., non-empty).-x FILE: True ifFILEexists and is executable.
test Command Conditions: Logical Operators
! EXPRESSION: Logical NOT. Negates the result of the following check.Example:
test ! -x tFile(True iftFileis not executable).
EXPRESSION1 -a EXPRESSION2: Logical AND. True if bothEXPRESSION1andEXPRESSION2are true.Example:
test $1 -eq $2 -a $2 -gt 5(True if first arg equals second arg AND second arg is greater than5).
EXPRESSION1 -o EXPRESSION2: Logical OR. True ifEXPRESSION1is true OREXPRESSION2is true.Example:
test $1 -eq $2 -o $2 -gt 5(True if first arg equals second arg OR second arg is greater than5\rightarrow\rightarrow\rightarrow\rightarrow\rightarrow EndifExample Script (
safeCat.sh):bash #!/bin/bash if [ -x "$1" ]; then cat "$1" fiRunning
safeCat.sh regularFile(assumingregularFileis executable,catwould print its content).Running
safeCat.sh /bin/ls(assuming/bin/lsis executable,catwould attempt to print its binary content, leading to control characters, or in context of the example, no desired output).
if then else fiPurpose: Executes one block of
STATEMENTS-YESif theCONDITIONis true, and a different block ofSTATEMENTS-NOif theCONDITIONis false.Syntax:
bash if [ CONDITION ]; then STATEMENTS-YES else STATEMENTS-NO fiFlowchart: Condition \rightarrow\rightarrow\rightarrow\rightarrow\rightarrow\rightarrow End
ifExample Script (
safeCat.sh):bash #!/bin/bash if [ -x "$1" ]; then cat "$1" else echo "Executable: not printing $1" fiRunning
safeCat.sh regularFile(if executable) $\rightarrow$ PrintsregularFile's content.Running
safeCat.sh /bin/ls(if executable, butcatis not intended for it) $\rightarrow$ OutputsExecutable: not printing /bin/ls.
if then elif else fiPurpose: Allows for multiple conditions to be checked sequentially. If the first
CONDITION1is true,STATEMENTS-1execute. Otherwise,CONDITION2is checked, andSTATEMENTS-2execute if true. If neither is true,STATEMENTS-3(theelseblock) execute.Syntax:
bash if [ CONDITION1 ]; then STATEMENTS-1 elif [ CONDITION2 ]; then STATEMENTS-2 else STATEMENTS-3 fiFlowchart: Condition1 $\rightarrow$ Yes $\rightarrow$ Statements-1 $\rightarrow$ End
if
Condition1 $\rightarrow$ No $\rightarrow$ Condition2 $\rightarrow$ Yes $\rightarrow$ Statements-2 $\rightarrow$ Endif
Condition1 $\rightarrow$ No $\rightarrow$ Condition2 $\rightarrow$ No $\rightarrow$ Statements-3 $\rightarrow$ EndifExample Script (
safeCat.sh):bash #!/bin/bash if [ -x "$1" ]; then cat "$1" elif [ "$1" == "/bin/ls" ]; then echo "I see /bin/ls" else echo "Executable: not printing $1" fiRunning
safeCat.sh regularFile(if executable) $\rightarrow$ PrintsregularFile's content.Running
safeCat.sh /bin/ls$\rightarrow$ OutputsI see /bin/ls(matcheselifcondition).Running
safeCat.sh testScript.sh(iftestScript.shis not executable in this specific test or otherwise falls through theifandelifchecks) $\rightarrow$ OutputsExecutable not printing testScript.sh.
case esacPurpose: Provides a clean way to execute different blocks of code based on matching a
STRINGagainst severalpatterns, often useful for menu-driven scripts or handling different file types.Syntax:
bash case STRING in pattern1) STATEMENTS-1 ;; pattern2) STATEMENTS-2 ;; *) STATEMENTS-DEFAULT ;; esacEach pattern block ends with
;;.The
*\rightarrow\rightarrow\rightarrow\rightarrow\rightarrow\rightarrow\rightarrow\rightarrow\rightarrow ExitcaseExample Script (
caseScript.sh):bash #!/bin/bash case $1 in *sh) echo "Shell Script" ;; *.txt) echo "Text File" ;; *) echo "Some Other File" ;; esacRunning
caseScript.sh testScript.sh$\rightarrow$ OutputsShell Script.Running
caseScript.sh file.txt$\rightarrow$ OutputsText File.Running
caseScript.sh blah$\rightarrow$ OutputsSome Other File.
Looping Constructs
Overview of Loops
Loops are used to perform a set of
STATEMENTSrepeatedly.Shell scripting offers different types of loops:
forloop: Iterates for a set number of items in a list.whileloop: Continues as long as a condition remains true.untilloop: Continues until a condition becomes true (i.e., as long as it's false).
forLoopPurpose: Iterates over a
LISTof items, assigning each item in turn to aVARand executing a block ofSTATEMENTSfor each iteration.Syntax:
bash for VAR in LIST do STATEMENTS doneFlowchart: Initialize
VARfromLIST$\rightarrow$ IfLISTis not empty:
ExecuteSTATEMENTS
Assign next item toVAR
Repeat
$\rightarrow$ IfLISTis empty (or exhausted): Exitforloop.Example Script (
forScript.sh):#!/bin/bash for myVar in * # Iterates through all command-line arguments (* provides the list) do echo "$myVar" doneRunning
forScript.sh arg1 arg2\rightarrow).Syntax:
bash for VAR do STATEMENTS doneExample (
forScript.sh- special form): This script behaves identically to the previous example.bash #!/bin/bash for myVar do echo "$myVar" doneRunning
forScript.sh arg1 arg2$\rightarrow$ Outputsarg1, thenarg2.
whileLoopPurpose: Continuously executes a block of
STATEMENTSas long as a specifiedCONDITIONremains true.Syntax:
bash while CONDITION do STATEMENTS doneFlowchart: Check
CONDITION$\rightarrow$ If True:
ExecuteSTATEMENTS
Loop back toCONDITIONcheck
$\rightarrow$ If False: Exitwhileloop.Example Script (
whileScript.sh):bash #!/bin/bash counter=0 while [ "$counter" -lt 10 ] do echo "counter" counter=(( counter + 1 )) # Increment counter using modern arithmetic expansion doneRunning
whileScript.sh\rightarrow\rightarrow\rightarrow If True: Exituntilloop.Example Script (
untilScript.sh):bash #!/bin/bash counter=0 until [ "counter" -eq 10 ] do echo "$$counter" counter=$(( counter + 1 )) # Increment counter using modern arithmetic expansion doneRunning
untilScript.sh$\rightarrow$ Outputs numbers0through9, each on a new line.