Introduction to shell scripting ____.ppt

its.unc.edu 1
UNIX Shells
 sh Bourne Shell (Original Shell) (Steven Bourne of AT&T)
 bash Bourne Again Shell (GNU Improved Bourne Shell)
 csh C-Shell (C-like Syntax)(Bill Joy of Univ. of California)
 ksh Korn-Shell (Bourne+some C-shell)(David Korn of
AT&T)
 tcsh Turbo C-Shell (More User Friendly C-Shell).
 To check shell:
• $ echo $SHELL (shell is a pre-defined variable)
 To switch shell:
• $ exec shellname (e.g., $ exec bash or simply type $ bash)
• You can switch from one shell to another by just typing the name
of the shell. exit return you back to previous shell.

its.unc.edu 2
Which Shell to Use?
 sh ( Bourne shell) was considered better for programming
 csh (C-Shell ) was considered better for interactive work.
 tcsh and korn were improvements on c-shell and bourne shell
respectively.
 bash is largely compatible with sh and also has many of the nice
features of the other shells
 On many systems such as our LINUX clusters sh is symbolically linked to
bash, /bin/sh -> /bin/bash
 We recommend that you use sh/bash for writing new shell scripts but
learn csh/tcsh to understand existing scripts.
 Many, if not all, scientific applications require csh/tcsh environment
(GUI, Graphics Utility Interface)
 All Linux versions use the Bash shell (Bourne Again Shell) as the
default shell
• Bash/Bourn/ksh/sh prompt: $
• All UNIX system include C shell and its predecessor Bourne shell.
• Csh/tcsh prompt: %

its.unc.edu 3
What is Shell Script?
 A shell script is a script written for the
shell
 Two key ingredients
•UNIX/LINUX commands
•Shell programming syntax

its.unc.edu 4
A Shell Script Example
#!/bin/sh
`ls -l *.log| awk '{print $8}' |sed 's/.log//g' > file_list`
cat file_list|while read each_file
do
babel -ig03 $each_file".log" -oxyz $each_file".xyz“
echo '# nosymmetry integral=Grid=UltraFine scf=tight rhf/6-311++g** pop=(nbo,chelpg)'>head
echo ' ' >>head
echo ''$each_file' opt pop nbo chelp aim charges ' >> head
echo ' ' >>head
echo '0 1 ' >>head
`sed '1,2d' $each_file.xyz >junk`
input=./$each_file".com"
cat head > $input
cat junk >> $input
echo ' ' >> $input
done
/bin/rm ./junk ./head ./file_list
#!/bin/sh
do
echo ' ' >>head
echo ' ' >>head
echo '0 1 ' >>head
cat head > $input
cat junk >> $input
echo ' ' >> $input
done

its.unc.edu 5
User Input During Shell
Script Execution
 As shown on the hello script input from the standard
input location is done via the read command.
 Example
echo "Please enter three filenames:”
read filea fileb filec
echo “These files are used:$filea $fileb $filec”
 Each read statement reads an entire line. In the above
example if there are less than 3 items in the response
the trailing variables will be set to blank ‘ ‘.
 Three items are separated by one space.

its.unc.edu 6
Hello script exercise
continued…
 The following script asks the user to enter his
name and displays a personalised hello.
#!/bin/sh
echo “Who am I talking to?”
read user_name
echo “Hello $user_name”
 Try replacing “ with ‘ in the last line to see
what happens.

its.unc.edu 7
Debugging your shell scripts
 Generous use of the echo command will help.
 Run script with the –x parameter.
E.g. sh –x ./myscript
or set –o xtrace before running the script.
 These options can be added to the first line of the
script where the shell is defined.
e.g. #!/bin/sh -xv

its.unc.edu 8
Shell Programming
 Programming features of the UNIX/LINUX shell:
Shell variables
Shell variables: Your scripts often need to keep values in
memory for later use. Shell variables are symbolic names
that can access values stored in memory
Operators
Operators: Shell scripts support many operators, including
those for performing mathematical operations
Logic structures
Logic structures: Shell scripts support sequential logic (for
performing a series of commands), decision logic (for
branching from one point in a script to another), looping
logic (for repeating a command several times), and case
logic (for choosing an action from several possible
alternatives)

its.unc.edu 9
Variables
 Variables are symbolic names that represent values stored in memory
 Three different types of variables
• Global Variables: Environment and configuration variables, capitalized, such as
HOME, PATH, SHELL, USERNAME, and PWD.
When you login, there will be a large number of global System variables that are
already defined. These can be freely referenced and used in your shell scripts.
• Local Variables
Within a shell script, you can create as many new variables as needed. Any variable
created in this manner remains in existence only within that shell.
• Special Variables
Reversed for OS, shell programming, etc. such as positional parameters $0, $1 …

its.unc.edu 10
A few global (environment)
variables
SHELL Current shell
DISPLAY Used by X-Windows system to identify the
display
HOME Fully qualified name of your login directory
PATH Search path for commands
MANPATH Search path for <man> pages
PS1 & PS2 Primary and Secondary prompt strings
USER Your login name
TERM terminal type
PWD Current working directory

its.unc.edu 11
Referencing Variables
Variable contents are accessed using ‘$’:
e.g. $ echo $HOME
$ echo $SHELL
To see a list of your environment variables:
$ printenv
or:
$ printenv | more

its.unc.edu 12
Defining Local Variables
 As in any other programming language, variables can be defined and
used in shell scripts.
 Unlike other programming languages, variables in Shell Scripts are
not typed.
 Examples :
a=1234 # a is NOT an integer, a string instead
b=$a+1 # will not perform arithmetic but be the string ‘1234+1’
b=`expr $a + 1 ` will perform arithmetic so b is 1235 now.
Note : +,-,/,*,**, % operators are available.
b=abcde # b is string
b=‘abcde’ # same as above but much safer.
b=abc def # will not work unless ‘quoted’
b=‘abc def’ # i.e. this will work.
IMPORTANT NOTE: DO NOT LEAVE SPACES AROUND THE =

its.unc.edu 13
Referencing variables
--curly bracket
 Having defined a variable, its contents can be referenced by
the $ symbol. E.g. ${variable} or simply $variable. When
ambiguity exists $variable will not work. Use ${ } the rigorous
form to be on the safe side.
 Example:
a=‘abc’
b=${a}def # this would not have worked without the{ } as
#it would try to access a variable named adef

its.unc.edu 14
Variable List/Arrary
 To create lists (array) – round bracket
$ set Y = (UNL 123 CS251)
 To set a list element – square bracket
$ set Y[2] = HUSKER
 To view a list element:
$ echo $Y[2]
 Example:
#!/bin/sh
a=(1 2 3)
echo ${a[*]}
echo ${a[0]}
Results: 1 2 3
1

its.unc.edu 15
Positional Parameters
 When a shell script is invoked with a set of command line parameters each
of these parameters are copied into special variables that can be accessed.
 $0 This variable that contains the name of the script
 $1, $2, ….. $n 1st
, 2nd
3rd
command line parameter
 $# Number of command line parameters
 $$ process ID of the shell
 $@ same as $* but as a list one at a time (see for loops later )
 $? Return code ‘exit code’ of the last command
 Shift command: This shell command shifts the positional parameters by
one towards the beginning and drops $1 from the list. After a shift $2
becomes $1 , and so on … It is a useful command for processing the input
parameters one at a time.
Example:
Invoke : ./myscript one two buckle my shoe
During the execution of myscript variables $1 $2 $3 $4 and $5 will contain
the values one, two, buckle, my, shoe respectively.

its.unc.edu 16
Variables
 vi myinputs.sh
#! /bin/sh
echo Total number of inputs: $#
echo First input: $1
echo Second input: $2
 chmod u+x myinputs.sh
 myinputs.sh HUSKER UNL CSE
Total number of inputs: 3
First input: HUSKER
Second input: UNL

its.unc.edu 17
Shell Programming
 programming features of the UNIX
shell:
Shell variables
Shell variables
Operators
Operators
Logic structures
Logic structures

its.unc.edu 18
Shell Operators
 The Bash/Bourne/ksh shell operators are
divided into three groups: defining and
evaluating operators, arithmetic operators,
and redirecting and piping operators

its.unc.edu 19
Defining and Evaluating
 A shell variable take on the generalized form
variable=value (except in the C shell).
$ set x=37; echo $x
37
$ unset x; echo $x
x: Undefined variable.
 You can set a pathname or a command to a
variable or substitute to set the variable.
$ set mydir=`pwd`; echo $mydir

its.unc.edu 20
Pipes & Redirecting
Linux Commands
Piping: An important early development in Unix , a way to pass the
output of one tool to the input of another.
$ who | wc −l
By combining these two tools, giving the wc command the output
of who, you can build a new command to list the number of users
currently on the system
Redirecting via angle brackets: Redirecting input and output follows
a similar principle to that of piping except that redirects work with
files, not commands.
tr '[a-z]' '[A-Z]' < $in_file > $out_file
The command must come first, the in_file is directed in by the
less_than sign (<) and the out_file is pointed at by the greater_than
sign (>).

its.unc.edu 21
Arithmetic Operators
 expr supports the following operators:
• arithmetic operators: +,-,*,/,%
• comparison operators: <, <=, ==, !=, >=, >
• boolean/logical operators: &, |
• parentheses: (, )
• precedence is the same as C, Java

its.unc.edu 22
 vi math.sh
#!/bin/sh
count=5
count=`expr $count + 1 `
echo $count
 chmod u+x math.sh
 math.sh
6

its.unc.edu 23
 vi real.sh
#!/bin/sh
a=5.48
b=10.32
c=`echo “scale=2; $a + $b” |bc`
echo $c
 chmod u+x real.sh
 ./real.sh
15.80

its.unc.edu 24
Arithmetic operations in
shell scripts
var++ ,var-- , ++var , --
var
post/pre
increment/decrement
+ , - add subtract
* , / , % multiply/divide,
remainder
** power of
! , ~ logical/bitwise negation
& , | bitwise AND, OR
&& || logical AND, OR

its.unc.edu 25
Shell Programming
 programming features of the UNIX
shell:
Shell variables
Shell variables
Operators
Operators
Logic structures
Logic structures

its.unc.edu 26
Shell Logic Structures
The four basic logic structures needed for program development
are:
Sequential logic: to execute commands in the order in
which they appear in the program
Decision logic: to execute commands only if a certain
condition is satisfied
Looping logic: to repeat a series of commands for a given
number of times
Case logic: to replace “if then/else if/else” statements when
making numerous comparisons

its.unc.edu 27
Conditional Statements
(if constructs )
The most general form of the if construct is;
if command executes successfully
then
execute command
elif this command executes successfully
then
execute this command
and execute this command
else
execute default command
fi
However- elif and/or else clause can be omitted.

its.unc.edu 28
Examples
SIMPLE EXAMPLE:
if date | grep “Fri”
then
echo “It’s Friday!”
fi
FULL EXAMPLE:
if [ “$1” == “Monday” ]
then
echo “The typed argument is Monday.”
elif [ “$1” == “Tuesday” ]
then
echo “Typed argument is Tuesday”
else
echo “Typed argument is neither Monday nor Tuesday”
fi
# Note: = or == will both work in the test but == is better for readability.

its.unc.edu 29
 string1 = string2 True if strings are identical
 String1 == string2 …ditto….
 string1 !=string2 True if strings are not identical
 string Return 0 exit status (=true) if string is not null
 -n string Return 0 exit status (=true) if string is not null
 -z string Return 0 exit status (=true) if string is null
Tests
 int1 –eq int2 Test identity
 int1 –ne int2 Test inequality
 int1 –lt int2 Less than
 int1 –gt int2 Greater than
 int1 –le int2 Less than or equal
 int1 –ge int2 Greater than or equal

its.unc.edu 30
Combining tests with logical
operators || (or) and && (and)
Syntax: if cond1 && cond2 || cond3 …
An alternative form is to use a compound statement using the –a
and –o keywords, i.e.
if cond1 –a cond22 –o cond3 …
Where cond1,2,3 .. Are either commands returning a a value or test
conditions of the form [ ] or test …
Examples:
if date | grep “Fri” && `date +’%H’` -gt 17
then
echo “It’s Friday, it’s home time!!!”
fi
if [ “$a” –lt 0 –o “$a” –gt 100 ] # note the spaces around ] and [
then
echo “ limits exceeded”
fi

its.unc.edu 31
File enquiry operations
-d file Test if file is a directory
-f file Test if file is not a directory
-s file Test if the file has non zero length
-r file Test if the file is readable
-w file Test if the file is writable
-x file Test if the file is executable
-o file Test if the file is owned by the user
-e file Test if the file exists
-z file Test if the file has zero length
All these conditions return true if satisfied and false otherwise.

its.unc.edu 32
Decision Logic
 A simple example
#!/bin/sh
if [ “$#” -ne 2 ] then
echo $0 needs two parameters!
echo You are inputting $# parameters.
else
par1=$1
par2=$2
fi
echo $par1
echo $par2

its.unc.edu 33
Decision Logic
Another example:
#! /bin/sh
# number is positive, zero or negative
echo –e "enter a number:c"
read number
if [ “$number” -lt 0 ]
then
echo "negative"
elif [ “$number” -eq 0 ]
then
echo zero
else
echo positive
fi

its.unc.edu 34
Loops
Loop is a block of code that is repeated a number
of times.
The repeating is performed either a pre-
determined number of times determined by a
list of items in the loop count ( for loops ) or
until a particular condition is satisfied ( while
and until loops)
To provide flexibility to the loop constructs there
are also two statements namely break and
continue are provided.

its.unc.edu 35
for loops
Syntax:
for arg in list
do
command(s)
...
done
Where the value of the variable arg is set to the values provided in the list one
at a time and the block of statements executed. This is repeated until the
list is exhausted.
Example:
for i in 3 2 5 7
do
echo " $i times 5 is $(( $i * 5 )) "
done

its.unc.edu 36
The while Loop
 A different pattern for looping is created using the
while statement
 The while statement best illustrates how to set up a
loop to test repeatedly for a matching condition
 The while loop tests an expression in a manner
similar to the if statement
 As long as the statement inside the brackets is true,
the statements inside the do and done statements
repeat

its.unc.edu 37
while loops
Syntax:
while this_command_execute_successfully
do
this command
and this command
done
EXAMPLE:
while test "$i" -gt 0 # can also be while [ $i > 0 ]
do
i=`expr $i - 1`
done

its.unc.edu 38
Looping Logic
 Example:
#!/bin/sh
for person in Bob Susan Joe Gerry
do
echo Hello $person
done
Output:
Hello Bob
Hello Susan
Hello Joe
Hello Gerry
 Adding integers from 1 to 10
#!/bin/sh
i=1
sum=0
while [ “$i” -le 10 ]
do
echo Adding $i into the
sum.
sum=`expr $sum + $i `
i=`expr $i + 1 `
done
echo The sum is $sum.

its.unc.edu 39
until loops
The syntax and usage is almost identical to the while-
loops.
Except that the block is executed until the test condition
is satisfied, which is the opposite of the effect of test
condition in while loops.
Note: You can think of until as equivalent to not_while
Syntax: until test
do
commands ….
done

its.unc.edu 40
Switch/Case Logic
 The switch logic structure simplifies the
selection of a match when you have a list of
choices
 It allows your program to perform one of
many actions, depending upon the value of a
variable

its.unc.edu 41
Case statements
The case structure compares a string ‘usually contained in a
variable’ to one or more patterns and executes a block of code
associated with the matching pattern. Matching-tests start with
the first pattern and the subsequent patterns are tested only if
no match is not found so far.
case argument in
pattern 1) execute this command
and this
and this;;
pattern 2) execute this command
and this
and this;;
esac

its.unc.edu 42
Functions
 Functions are a way of grouping together commands so that they can later be
executed via a single reference to their name. If the same set of instructions
have to be repeated in more than one part of the code, this will save a lot of
coding and also reduce possibility of typing errors.
SYNTAX:
functionname()
{
block of commands
}
#!/bin/sh
sum() {
x=`expr $1 + $2`
echo $x
}
sum 5 3
echo "The sum of 4 and 7 is `sum 4 7`"

its.unc.edu 43
Take-Home Message
 Shell script is a high-level language that must be
converted into a low-level (machine) language by UNIX
Shell before the computer can execute it
 UNIX shell scripts, created with the vi or other text editor,
contain two key ingredients: a selection of UNIX
commands glued together by Shell programming syntax
 UNIX/Linux shells are derived from the UNIX Bourne, Korn,
and C/TCSH shells
 UNIX keeps three types of variables:
• Configuration; environmental; local
 The shell supports numerous operators, including many
for performing arithmetic operations
 The logic structures supported by the shell are sequential,
decision, looping, and case

its.unc.edu 44
To Script or Not to Script
 Pros
• File processing
• Glue together compelling, customized testing utilities
• Create powerful, tailor-made manufacturing tools
• Cross-platform support
• Custom testing and debugging
 Cons
• Performance slowdown
• Accurate scientific computing

its.unc.edu 45
Shell Scripting Examples
 Input file preparation
 Job submission
 Job monitoring
 Results processing

its.unc.edu 46
Input file preparation
#!/bin/sh
do
echo ' ' >>head
echo ' ' >>head
echo '0 1 ' >>head
cat head > $input
cat junk >> $input
echo ' ' >> $input
done

its.unc.edu 47
LSF Job Submission
$ vi submission.sh
#!/bin/sh -f
#BSUB -q week
#BSUB -n 4
#BSUB -o output
#BSUB -J job_type
#BSUB -R “RH5 span[ptile=4]”
#BSUB -a mpichp4
mpirun.lsf ./executable.exe
exit
$chmod +x submission.sh
$bsub < submission.sh

its.unc.edu 48
Results Processing
#!/bin/sh
`ls -l *.out| awk '{print $8}'|sed 's/.out//g' > file_list`
do
file1=./$each_file".out"
Ts=`grep 'Kinetic energy =' $file1 |tail -n 1|awk '{print $4}' `
Tw=`grep 'Total Steric Energy:' $file1 |tail -n 1|awk '{print $4}' `
TsVne=`grep 'One electron energy =' $file1 |tail -n 1|awk '{print $5}' `
Vnn=`grep 'Nuclear repulsion energy' $file1 |tail -n 1|awk '{print $5}' `
J=`grep 'Coulomb energy =' $file1 |tail -n 1|awk '{print $4}' `
Ex=`grep 'Exchange energy =' $file1 |tail -n 1|awk '{print $4}' `
Ec=`grep 'Correlation energy =' $file1 |tail -n 1|awk '{print $4}' `
Etot=`grep 'Total DFT energy =' $file1 |tail -n 1|awk '{print $5}' `
HOMO=`grep 'Vector' $file1 | grep 'Occ=2.00'|tail -n 1|cut -c35-47|sed 's/D/E/g' `
orb=`grep 'Vector' $file1 | grep 'Occ=2.00'|tail -n 1|awk '{print $2}' `
orb=`expr $orb + 1 `
LUMO=`grep 'Vector' $file1 |grep 'Occ=0.00'|grep ' '$orb' ' |tail -n 1|cut -c35-47|sed 's/D/E/g'
echo $each_file $Etot $Ts $Tw $TsVne $J $Vnn $Ex $Ec $HOMO $LUMO $steric >>out
done
/bin/rm file_list

its.unc.edu 49
Reference Books
 Class Shell Scripting
http://oreilly.com/catalog/9780596005955/
 LINUX Shell Scripting With Bash
http://ebooks.ebookmall.com/title/linux-shell-scripting-with-bash-burtch-
ebooks.htm
 Shell Script in C Shell
http://www.grymoire.com/Unix/CshTop10.txt
 Linux Shell Scripting Tutorial
http://www.freeos.com/guides/lsst/
 Bash Shell Programming in Linux
http://www.arachnoid.com/linux/shell_programming.html
 Advanced Bash-Scripting Guide
http://tldp.org/LDP/abs/html/
 Unix Shell Programming
http://ebooks.ebookmall.com/title/unix-shell-programming-kochan-wood-
ebooks.htm

its.unc.edu
Questions & Comments
Please direct comments/questions about research computing to
E-mail: research@unc.edu
Please direct comments/questions pertaining to this presentation to
E-Mail: shubin@email.unc.edu
The PPT file of this presentation is available here:
http://its2.unc.edu/divisions/rc/training/scientific/short_courses/Shell_Scripting.ppt

its.unc.edu 51
Hands-on Exercises
1. The simplest Hello World shell script – Echo command
2. Summation of two integers – If block
3. Summation of two real numbers – bc (basic calculator) command
4. Script to find out the biggest number in 3 numbers – If –elif block
5. Operation (summation, subtraction, multiplication and division) of two
numbers – Switch
6. Script to reverse a given number – While block
7. A more complicated greeting shell script
8. Sort the given five numbers in ascending order (using array) – Do loop
and array
9. Calculating average of given numbers on command line arguments – Do
loop
10. Calculating factorial of a given number – While block
11. An application in research computing – Combining all above
12. Optional: Write own shell scripts for your own purposes if time permits
The PPT/WORD format of this presentation is available here:
http://its2.unc.edu/divisions/rc/training/scientific/
/afs/isis/depts/its/public_html/divisions/rc/training/scientific/short_courses/

Introduction to shell scripting ____.ppt

Recommended

More Related Content

Similar to Introduction to shell scripting ____.ppt (20)

Recently uploaded (20)

Introduction to shell scripting ____.ppt

Editor's Notes