Shells are the interpretters that take your commands and translate them into a set of actions. There are many different shells available, bash happens to be the one we generally use for the csci student accounts. This page is essentially a quick list of commands available for use on our systems.
command interpretters
- sh (bourne shell)
- csh (command shell)
- ksh (korn shell)
- bash (bourne again shell)
- each has own syntax and built in commands
- chsh to change your login shell
- shell names to start a new shell locally
shell commands vs linux executables
- bash commands
alias, bind, break, builtin, cd, command, continue, declare, echo,
enable, eval, exec, exit, export, getopts, hash, help,
let, local, logout, printf, pwd, read, readonly, return,
set, shift, shopt, test, times, trap, type, typeset,
ulimit, umask, unalias, unset [ ] : .
- linux executables
clear, cp, ls, mv, rm, mkdir, rmdir, ps, top,
- path variable
- environment variables
- VARNAME=VALUE;export VARNAME
- useful environment variables
- path
"PATH=$PATH:new/path/1:new/path/2"
- shell
- man path
- prompt
- history
setting up your environment
- .bash_profile (contains commands run when you login, usually have it run .bashrc)
- .bashrc (contains commands run when you open a non-login shell, or when called by .bash_profile)
- .bash_aliases (contains shortcuts for commands you want to run, e.g.
alias ls='ls -aF' # when you type ls it will actually run ls -aF
You can also use certain programming constructs as part of the file, some very abbreviated notes on the options available are shown below.
writing scripts: the first line specifies where the bash interpretter is found:
#! /bin/bash
comments: everything to the right of a #
# blah blah
command separators
bash assumes one command per line, so each new line is treated as a new command
if you want a command to span multiple lines, end the line with \
it you want multiple commands on one line line, separate them with ;
note that bash keywords are restricted to one per line, e.g.
if, then, else, elif, fi, for, done, etc.
to put more than one keyword on the same line use the ; to separate them
variables
- do not need to be declared, and are dynamically typed
blah=whatever # assigns a value, no spaces around =
use $blah to evaluate a variable (i.e. use its value), e.g.
foo=$blah # copy value in blah to foo
echo "blah is $blah" # print blah
note that ${blah} also evaluates the variable value, but permits you
to more carefully delimit the variable name, e.g.
${foo}blah as opposed to $fooblah
the variable name acts like a pointer, and $ dereferences it, e.g.
a=1 # a is 1
b=$a # b is now 1
a=2 # a is 2
c=a # c is now (a ref to) a
d=c # d is now (a ref to c)
echo "$a" # displays 2
echo "$b" # displays 1
echo "$c" # displays a
echo "$[$c]" # displays 2
echo "$d $[$d] $[$[$d]]" # displays c a 2
weak quotes (double quotes)
- substitutes special meanings, e.g.
echo "$blah" # prints value of blah
strong quotes
- uses literal string contents, e.g.
echo '$blah' # really prints $blah
output can use echo or printf
echo "varname is $varname"
options
-n suppress newline
special chars, require you to use echo -e "blah blah blah"
\a bell
\b backspace
\t tab
\n newline
\\ backslash
\xHH print hex
# printf allows placeholders like in cstdio,
# but also allows the use of $varname, e.g.:
printf "blah is $blah, foo is %d", $foo;
input is carried out with read
read varname1 varname2 varname3
options
-d DELIM # to specify what ends a line
-t TIMEOUT # to specify how long to wait before timing out
-s # silent mode (don't echo to terminal)
-p PROMPT # display prompt then read
-e # read entire line
command redirection
command < file # take input from file (instead of stdin)
command > file # send output to file (instead of stdout)
command 2> file # send stderr output to file instead
command >> file # append output to file (instead of stdout)
command 2>> file # append stderr output to file instead
command &> file # send stdout AND stderr output to file
command | tee file1 file2 ... filen # send command output to stdout AND all the files
command < infile > outfile # combination of both
command1 | command2 # use output of command 1 as input to command 2
sending input to a command from a text string instead of user input
can be done with the "here-string": <<<
command <<< "text string"
you can capture the output of a command string using backtics, e.g.
contents=`ls -aF .`
equivalently, you can also capture the output using $( ), e.g.
contents=$(ls -aF)
you can capture the return status of a command using variable $?, e.g.
contents=$(ls -aF)
retval=$?
echo "the command output was $contents, and return value was $retval"
you can run commands on data consisting of multiple lines of text using <<NNNN
where NNNN is the word you want to use to mark end of input, e.g.
wc -l <<LASTWORD
here is a line of input
here is another
LASTWORD # now it's over and the results of wc -l will be displayed
you can capture the command output by wrapping the entire thing in $(), e.g.
result=$( wc -l <<LASTWORD
here is a line of input
here is another
LASTWORD
)
e.g. you can combine this with the : (no-op) operator to
effectively comment out a chunk of a script
(rather than commenting out each line with a #)
: <<ENDCOMMENT
this would be the stuff
in the bash script I wanted commented out
ENDCOMMENT
arithmetic expressions and assignments:
use $ to take value of expressions
i=0 # declare i and initialize to 0
c-style math (including +=, ++, --, etc) is supported inside (( )), e.g.
(( i = x + y / z ))
(( j++ ))
and you can get the value of a (( )) using $, e.g.
x=$(( 3 + 4 / z % 10 ))
you can also use let to evaluate math in text strings, e.g.
let "z = z + 3"
old school technnique:
(very whitespace sensitive, meant to be depracated in the future)
enclose the expression in $[ ], e.g.
i=$[$i+1] # set i to value of ((value of i) + 1)
arithmetic operators
+ - * / % **
only integer arithmetic is directly supported, floating point
can be handled by routing the formula string through bc, e.g.
answer=$(bc <<< "$x/$y*10")
logic operators, [ ], and [[ ]]
the square brackets [ ] are in fact an alias for the linux test command,
which evaluates a true/false expression, e.g.
if [ $x -lt $y ]
the double square brackets [[ ]] are bash syntax, and support a more diverse
range of expression syntax than the single brackets
comparison operators and data types
integer comparisons use -lt -gt -ne -eq -ge -le, e.g.
[ $x -le $y ]
string comparisons use < > == !=
[ $str1 < $str2 ]
unary tests (there are many many more than this)
[ -z "$string" ] true if string is null (zero length)
[ -n "$string" ] true if string is not null (non-zero length)
[ -f "$file" ] true if file exists and is regular file
[ -d "$dir" ] true if directory exists
[ -r "$file" ] true if file is readable
[ -w "$file" ] true if file is writable
[ -x "$file" ] true if file is executable
selection
if [ $x = "foo" ] ; then
......
elif [ ... ] ; then
......
else
......
fi
boolean operators
not uses !
e.g. if ! [ $x -eq $y ] ; then
and uses && or -a
e.g. if [ $x -eq $y -a $y -lt $z ] ; then
e.g. if [[ $x -eq $y && $y -lt $z ]] ; then
or uses || or -o
e.g. if [ $x -eq $y -o $y -lt $z ] ; then
e.g. if [[ $x -eq $y || $y -lt $z ]] ; then
iteration
for x in a b c ; do
......
done
for x in $mylistvar ; do
.....
done
for (( x=1; x<10; x++ )); do
.....
done
for x in $(seq m n) ; do
......
done # example:
x=1
while [ expr ] ; do while [ $x -lt 9 ]; do
....... echo $x
done x=$[$x+1]
done
# iterate through command line arguments by traditional for loop
for arg in $@ ; do
echo ${arg}
done
# iterate through command line args by number
for (( i=1; i<=$#; i++ )) ; do
echo ${!i} # !i looks up value of i
# then that is used for the ${argnum}
done
# iterate through command line arguments by shifting
while [ $# -gt 0 ] ; do
echo ${1}
shift
done
iterating across words in a line of text
text="blah blah blah"
for word in $text ; do
... do something with $word ...
done
(note: don't put double quotes around $text,
or you'll find it treats the whole line
as a single word)
iterating across lines of text, here assuming the text is already in variable "content"
IFS=$'\n' # here specifying newlines mark the end of lines
for line in $content ; do
... do something with $line ...
done
iterate through files in a directory
dirname="someplace/wherever"
for file in $dirname/* ; do
# here $file has the path/name for the file
done
arrays:
can be initialized using
arr=(10 20 4 8)
can access elements positionally,
if the position is out of range the array is resized
arr[6]=3
can access by variable
arr[$indx]=3
can look up elements by enclosing in ${ }, e.g.
x=${arr[3]}
y=${arr[$indx]}
can look up the current array size as follows
len=${#arr[@]}
can access all the array contents (e.g. to copy array) using
content=${arr[@]}
can delete an element or the array using unset
unset arr[$i]
unset arr
example of looping through the array contents
for val in "${arr[@]}"; do
echo "${val}"
done
cases (switch-like) compare a variable's value to a
set of patterns, and execute the instructions associated
with the first matching pattern
case $x in
pattern1 )
.... # if $x matches pattern1 do this stuff
;; # ;; acts like break in a switch statement
pattern2 )
....
;;
patternN )
....
;;
* )
.... # default case (since * matches everything)
;;
esac
regular expressions describe matchable patterns (e.g. for case patterns)
[a4q] # match any of the three characters
[a-z] # match lowercase a-z
[a-zA-Z0-9] # match an alphanumeric character
[^a-z] # match anything EXCEPT an a-z
[pattern]? # match 0 or 1 instances of the pattern
[pattern]* # match any number of instances of the pattern (including 0)
[pattern]+ # match one or more instances of the pattern
[pattern]{m} # match exactly m instances of the pattern
[pattern]{m,} # match m or more instances of the pattern
[pattern]{m,n} # match m to n instances of the pattern
. # match any single character
* # match anything (except whitespace)
$ # match end of string
^ # match beginning of string
example of testing if a variable holds an integer
re='^[0-9]+$' # must be 1 or more digits, nothing else
if [[ $x =~ $re ]] ; then # variable x matches the regex
echo "$x is an integer"
fi
# example of using negation with a pattern match
if ! [[ $x =~ $re ]] ; then # variable x does not match the regex
echo "$x is not an integer"
fi
example regex for a fixed-point float (optional +/-, optional .)
re='^[-+]?[0-9]+\.?[0-9]*$'
example of matching everything after the last : on each line of a file
grep -oE '[^:]+$' $Filename
example of getting a substring of length N starting at position P (0-based)
substr=${orig:P:N}
functions
function_name()
# parameters are named $1 $2 ... etc up to $#
# $# returns the number of arguments passed
# $* returns all the arguments
# $@ returns all the arguments, but with each as a quoted string
# $0 is the function/scriptname
{
# can create locals (otherwise are global)
local x = $1
local y = $2
# iterate across parameters
for arg in "$@"; do
echo "$arg"
done
# return num of params passed
return $#
}
functions can only return short (1 byte) status codes,
but you can have them use echo to print something,
then capture that at the call, e.g.
x=$(foo a b c)
or use backticks for the same idea, e.g.
x=`foo a b c`
(the actual return value can be retrieved afterward from variable $?)
calling functions from the command line:
first source the file they are in, e.g.
source mybashscript.sh
anytime after that (in your current session) you can simply
call the function, e.g.
foo x y z
note that if you source the file from your .bashrc
(after making sure it is suitably debugged)
then the command is always available to you
terminal control
clear # clears the screen and puts the cursor top-left
tput cup ROW COL # moves the cursor to specified row and column (0,0 is top left)
tput bold # turns on bold
tput dim # make half-bright
tput rev # reverse background/foreground lighting
tput reset # mostly resets defaults
tput smso # turn on reverse bold
tput rmso # turn off reverse bold
tput smul # turn on underline
tput rmul # turn off underline
tput setb N # set background colour 0-7
tput setf N # set foreground colour 0-7
# 0 black, 1 blue, 2 green, 3 light blue, 4 red, 5 purple, 6 gold, 7 grey
useful variables (there are many many others)
$USER # the username
$HOME # the user's home directory
$SECONDS # how many seconds the script has been running
$LINENO # line number in the script
$FUNCNAME # name of the current function
$PWD # the current working directory
$OLDPWD # the previous working directory
$PATH # current search path for commands/executables
$HOSTNAME # system host name
$HOSTTYPE # like machine type
$OSTYPE # operating system type
$MACHTYPE # machine type (system hardware)
$BASH_VERSION # current version of bash in use
$BASH # path to the bash binary
$PS1 # the prompt shown at the main command line
$PS2 # secondary prompt, shown when additional input requested by command
$$ # id of the current process
$PPID # id of the parent process
misc
debugging aid
set -xv # turns on command echoing and verbose mode
set +xv # turns off command echoing and verbose mode
exiting
exit n
pausing
sleep n
testing if file exists
if [ -f $filename] ; then
....
fi
capturing the first K lines of a text file
head -n K Filename
capturing the last K lines of a text file
tail -n K Filename
random number generation (integers)
x=$RANDOM
bash guides
- tldp.org/LDP/Bash-Beginners-Guide/html/
#! /bin/bash
# example of parsing through command line arguments
# -------------------------------------------------
# where two arguments, srcfile, destfile, are expected
# and additional options are supported:
# -v (meaning turn on verbose mode, default is off)
# -m maxval (setting some max value for processing, default is 100)
# where the options can appear before, between, or after srcfile destfile
# start with default values for everything
srcfile=""
destfile=""
verbose=0
max=100
# iterate through the command line arguments,
# process -m, -v if encountered,
# assuming remaining args are "fixed position" so put those in array
fixed=()
numargs=0
while [ $# -gt 0 ] ; do
# remove the next argument and see what it is
key=$1
shift
case $key in
-v|--verbose)
verbose=1
;;
-m|--max)
# the next item should be the maxval
if [ $# -lt 1 ] ; then
echo "error: -m must be followed by maxval"
else
# remove and store the maxval arg
max=$1
shift
fi
;;
*)
# anything else is assumed to be one of the positional arguments
# (presumably srcfile or destfile)
fixed+=("$key")
((numargs++))
;;
esac
done
# make sure all the required positional args are present
if [ $numargs -lt 1 ] ; then
echo "missing arguments srcfile and destfile"
elif [ $numargs -lt 2 ] ; then
echo "missing argument destfile"
else
# assign the positional arguments to the appropriate variables
# and begin the actual processing
srcfile=${fixed[0]}
destfile=${fixed[1]}
echo "processing ${srcfile} and ${destfile}, max is ${max}, verbose is ${verbose}"
if [ ${numargs} -gt 2 ] ; then
echo "warning: ignored extra args"
fi
fi