Topic 3: The Power of the Command Line

compression

lessen the memory a set of data consumes

archiving

bundling files and/or directories into a single file

tar

a tool that archives files. Does not compress them by itself. (Name is short for "tape archive." Files created with tar are called "tar balls"

zip

an archive file format that supports lossless data compression; may contain one or more files or directories

Comes with windows operating systems, needs to be installed for Linux

unzip

the command that decompresses and expands a zipped file

bzip2

A lossless file compression command. Uses the .bz2 file extension.

gzip

a lossless file compression command. Uses the .gz file extension

xz

a lossless file compression command. Uses the .xz file extention

bunzip2

uncompresses .bz2 files

gunzip

uncompresses .gz files

unxz

uncompresses .xz file

compression levels

entered as options, usually as numbers from 1 (least compression) to 9 (most compression). 9 is default

gzip reading tools

Versions of common commands (cat, less, grep, etc) which work on gzip .gz files. These can be accessed by prefixing a z. For example: zcat, zless, etc

bzip2 reading tools

bzcat, bzless, etc

xz reading tools

xzcat, xzless, etc

tar options

c create

x extract

f file name

t view contents

u add or replace files (must be uncompressed)

v lists files processed

A concatenates archive files

z compresses using gzip

j compresses using bzip2

J compresses using xz

W used to verify an archive file

r update file or directories in .tar file

tar -tf name.tar

displays the contents of name.tar

tar -options filename(to edit) files/directories(if relevant)

format of a tar command. First the options, then the tarball to edit or work in, then finally all other arguments the chosen options require

tar cf newtarball.tar targetdirectory

makes a new tarball from the target directory

tar -tf tarball.tar

shows the contents of the named file

tar xf tarball.tar

extracts the entire tarball

tar xvf tarball.tar target_file

extracts target_file from the tarball and prints all the files tar interacted with

tar cjf file.tar.bz2 bigfile bigfile2 bigfile3

creates a compressed tarball using bzip2

stdin

-standard input: information inputted into terminal through keyboard or input device. AKA channel 0

stdout

standard output: is the information outputted to the screen after a process is run. AKA channel 1

stderr

a stream that contains error output, known as channel 2. Goes to the screen by default

>

redirects standard output (stdout) to a file. If it doesn't exist, it creates a new one. If it does, it is overwritten

>>

redirects standard output, appending it to a file

2>

redirects just error messages

2>>

appends error messages to a file

/dev/null

null device. sending data to it will do nothing

<

used to redirect standard input (stdin)

$ cat < file.txt

Hello!

Most commonly used with commands that don't accept file arguments

<<

a redirection operator that acts as a 'here document'

&>

writes both channel 1 and 2 to a file

&>>

appends both channel 1 and 2 to a file

|

also known as redirection or command line pipes

Output of first command used as input of second command, etc

$ cat /etc/passwd | less

uses less on the text result of cat

wc

counts the lines, words, and bytes

-w counts only words

-l counts only lines

-c counts only bytes

tail -f

view last 10 lines of a file and follow output

grep

abbreviation of "global regular expression print"

finds lines with specified pattern

-i case insensitive

-r recursive (also in subdirectories)

-c outputs number of matches instead of matches

-v outputs lines that DON'T match

-E advanced RegEx (for ?, |, etc)

and more

sun | moon

Find either of the listed strings

^ RegEx

start of a line

$ RegEx

end of a line

+ RegEx

matches when the character preceding + matches 1 or more times, and makes it mandatory

? RegEx

matches zero or one of the preceding pattern

. RegEx

represents any single character

script

a file containing commands that can run on the command line one after another

./

prefix that runs a file as a script

shebang

#!/bin/bash for bash. It specifies which interpreter

.sh

or .bash

The conventional ending of bash scripts

vi

text editor installed by default. vim is a clone which is slightly improved. Has a number of different navigational modes

nano

a simple text editor

#This is a Bash comment

Comments are defined by #

arguments

delimited inside the script as $1, $2 etc. (After 9, it gets odd).

$#

variable containing the number of arguments/paramters

format of conditional logic in Bash

if [ condition ]

then

action

else

action2

fi

-eq

checks for equality

-ne

not equal to

-gt

greater than

-ge

greater than or equal to

-lt

less than

-le

less than or equal to

$ built-in variable

this contains the exit code of the last command to run

exit 0

exits the script with an exit code of 0

$@ or $*

a built-in variable for all arguments

for loop syntax

FILES="/usr file2 file3"

for file in $FILES

do

action

done

Bash array syntax

Separated by spaces

item1 item2 item3

shift

removes the first element of our array