Unix

grep command searches the given file for lines containing a match to the given strings or words.
By default, grep prints the matching lines. Use grep to search for lines of text that match one or
many regular expressions, and outputs only the matching lines.
The name, "grep", derives from the command used to perform a similar operation, using the
Unix/Linux text editor ed:
g/re/p
grep command syntax
grep 'word' filename
grep 'string1 string2' filename
cat otherfile | grep 'something'
command | grep 'something'
Use grep to search file
Search /etc/passwd for boo user:
$ grep boo /etc/passwd
You can force grep to ignore word case i.e match boo, Boo, BOO and all other combination with
-i option:
$ grep -i "boo" /etc/passwd
Use grep recursively
You can search recursively i.e. read all files under each directory for a string "192.168.1.5"
$ grep -r "192.168.1.5" /etc/
Use grep to search words only
When you search for boo, grep will match fooboo, boo123, etc. You can force grep to select only
those lines containing matches that form whole words i.e. match only boo word:
$ grep -w "boo" /path/to/file
Use grep to search 2 different words

use egrep as follows:
$ egrep -w 'word1|word2' /path/to/file
Count line when words has been matched
grep can report the number of times that the pattern has been matched for each file using -c
(count) option:
$ grep -c 'word' /path/to/file
Also note that you can use -n option, which causes grep to precede each line of output with the
number of the line in the text file from which it was obtained:
$ grep -n 'word' /path/to/file
Grep invert match
You can use -v option to print inverts the match; that is, it matches only those lines that do not
contain the given word. For example print all line that do not contain the word bar:
$ grep -v bar /path/to/file
UNIX / Linux pipes and grep command
grep command often used with pipes. For example print name of hard disk devices:
# dmesg | egrep '(s|h)d[a-z]'
Display cpu model name:
# cat /proc/cpuinfo | grep -i 'Model'
However, above command can be also used as follows without shell pipe:
# grep -i 'Model' /proc/cpuinfo
How do I list just the names of matching files?
Use the -l option to list file name whose contents mention main():
$ grep -l 'main' *.c
Finally, you can force grep to display output in colors:
$ grep --color vivek /etc/passwd
So what does it do ?
Back to top
grep basically searches. More precisely,
grep foo file returns all the lines that contain a string matching the expression "foo" in the file
"file".
For now, we will just think of an expression as a string. So grep returns all matching lines that contain
foo as a substring.

Another way of using grep is to have it accept data through STDIN. instead of having it search a
file. For example,
ls |grep blah lists all files in the current directory whose names contain the string "blah"
Compatibility Notes
Back to top
This tutorial is based on the GNU version of grep. It is recommended that you use this version.
To use it, firstly, it needs to be installed on your system. Secondly, your PATH needs to be set
so that GNU grep is used in preference to the standard version.
Wildcards For Grep
Back to top
The Basics: Wildcards for grep
The Wildcard Character
So the first question that probably comes to mind is something like "does this grep thing support
wildcards ? And the answer is better than yes. In fact saying that grep supports wildcards is a big
understatement. grep uses regular expressions which go a few steps beyond wildcards. But we
will start with wildcards. The canonical wildcard character is the dot "." Here is an example :
>cat file
big
bad bug
bag
bigger
boogy
>grep b.g file
big
bad bug
bag
bigger
notice that boogy didn't match, since the "." matches exactly one character.
The repetition character
To match repetitions of a character, we use the star, which works in the following way:

the expression consisting of a character followed by a star matches any number (possibly zero) of
repetitions of that character. In particular, the expression ".*" matches any string, and hence acts as a
"wildcard".
To illustrate, we show some examples:
Examples: Wildcards
The File for These Examples
>cat file
big
bad bug
bag
bigger
boogy
Wildcards #1
>grep "b.*g" file
big
bad bug
bag
bigger
boogy
Wildcards #2
>grep "b.*g." file
bigger
boogy
repetition
>grep "ggg*" file
bigger
Read the repetion example carefully, and pay careful attention to the fact that the "*" in grep patterns
denotes repetition. It does not behave as a wildcard in regular expression syntax (as it is in UNIX or DOS
glob patterns). Recall that the pattern ".*" behaves as a wildcard (because .* means "repeat any
character any number of times). The pattern "g*" matches the string "", "g", "gg", etc. Likewise, "gg*"
matches "g", "gg", "ggg", so "ggg*" matches "gg", "ggg", "gggg", etc.
Taking it Further - Regular Expressions
Back to top The wildcards are a start, but the idea could be taken further. For example, suppose we
want an expression that matches Frederic Smith or Fred Smith. In other words, the letters eric
are "optional".
First, we introduce the concept of an "escaped" character.
An escaped character is a character preceded by a backslash. The preceding backslash does one of the
following:
(a) removes an implied special meaning from a character (b) adds special meaning to a "non-special"
character
Examples
To search for a line containing text hello.gif, the correct command is
grep 'hello.gif' file
since grep 'hello.gif' file will match lines containing hello-gif , hello1gif ,
helloagif , etc.
Now we move on to grouping expressions, in order to find a way of making an expression to
match Fred or Frederic
First, we start with the ? operator.

an expression consisting of a character followed by an escaped question mark matches one or zero
instances of that character.
Example
bugg?y matches all of the following: bugy , buggy but not bugggy We move on to "grouping"
expressions. In our example, we want to make the string "ederic" following "Fred" optional, we don't
just want one optional character.
An expression surrounded by "escaped" parentheses is treated by a single character.
Examples
Fred(eric)? Smith matches Fred Smith or Frederic Smith
(abc)* matches abc , abcabcabc etc. (i.e. , any number of repetitions of the string abc , including
the empty string.) Note that we have to be careful when our expressions contain white spaces or stars.
When this happens, we need to enclose them in quotes so that the shell does not mis-interpret the
command, because the shell will parse whitespace-separated strings as multiple arguments, and will
expand an unquoted * to a glob pattern. So to use our example above, we would need to type
grep "Fred(eric)? Smith" file
We now mention several other useful operators.
More on Regular Expressions
Back to top
Matching a list of characters
Back to top
To match a selection of characters, use [].
Example
[Hh]ello matches lines containing hello or Hello
Ranges of characters are also permitted.
Example
[0-3] is the same as [0123]
[a-k] is the same as [abcdefghijk]
[A-C] is the same as [ABC]
[A-Ca-k] is the same as
[ABCabcdefghijk]
There are also some alternate forms :
[[:alpha:]] is the same as [a-zA-Z]
[[:upper:]] is the same as [A-Z]

[[:lower:]] is the same as [a-z]
[[:digit:]] is the same as [0-9]
[[:alnum:]] is the same as [0-9a-zA-Z]
[[:space:]] matches any white space including tabs
These alternate forms such as [[:digit:]] are preferable to the direct method [0-9]
The [] may be used to search for non-matches. This is done by putting a carat ^ as the first character
inside the square brackets.
Example
grep "([^()]*)a" file returns any line containing a pair of parentheses that are innermost and are
followed by the letter "a". So it matches these lines
(hello)a
(aksjdhaksj d ka)a
But not this
x=(y+2(x+1))a
Matching a Specific Number Of Repetitions of a Pattern
Back to top
Suppose you want to match a specific number of repetitions of a pattern. A good example is phone
numbers. You could search for a 7 digit phone number like this:
grep "[[:digit:]]{3}[ -]?[[:digit:]]{4}" file
This matches phone numbers, possibly containing a dash or whitespace in the middle.
Nailing it Down to Start of the Line and End of the Line
Back to top
Here's the deal. Suppose you want to search for lines containing a line consisting of white space, then
the word hello, then the end of the line. Let us start with an example.
>cat file
hello
hello world
hhello
>grep hello file
hello
hello world
hhello
This is not what we wanted. So what went wrong ? The problem is that grep searches for lines
containing the string "hello" , and all the lines specified contain this. To get around this problem, we
introduce the end and beginning of line characters

The $ character matches the end of the line. The ^ character matches the beginning of the line.
Examples
returning to our previous example,
grep "^[[:space:]]*hello[[:space:]]*$" file
does what we want (only returns one line) Another example:
grep "^From.*mscharmi" /var/spool/mail/elflord searches my inbox for headers from a
particular person. This kind of regular expression is extremely useful, and mail filters such as procmail
use it all the tims.
This or That: matching one of two strings
Back to top
The expression consisting of two expressions seperated by the or operator | matches lines containing
either of those two expressions.
Note that you MUST enclose this inside single or double quotes.
Example
grep "cat|dog" file matches lines containing the word "cat" or the word "dog"
grep "I am a (cat|dog)" matches lines containing the string "I am a cat" or the string "I am a
dog".
Backpedalling and Backreferences
Back to top
Suppose you want to search for a string which contains a certain substring in more than one place. An
example is the heading tag in HTML. Suppose I wanted to search for <H1>some string</H1> . This is
easy enough to do. But suppose I wanted to do the same but allow H2 H3 H4 H5 H6 in place of H1.
The expression <H[1-6]>.*</H[1-6]> is not good enough since it matches <H1>Hello
world</H3> but we want the opening tag to match the closing one. To do this, we use a backreference
The expression n where n is a number, matches the contents of the n'th set of parentheses in the
expression
Woah, this really needs an example!
Examples
<H([1-6]).*</H1> matches what we were trying to match before.
"Mr (dog|cat) came home to Mrs 1 and they went to visit Mr (dog|cat)
and Mrs 2 to discuss the meaning of life matches ... well I'm sure you can work it out.
the idea is that the cats and dogs should match up in such a way that it makes sense.
Some Crucial Details: Special Characters and Quotes

Back to top
Special Characters
Back to top
Here, we outline the special characters for grep. Note that in egrep (which uses extended regular
expressions), which actually are no more functional than standard regular expressions if you use GNU
grep ) , the list of special characters increases ( | in grep is the same as | egrep and vice versa, there are
also other differences. Check the man page for details ) The following characters are considered special
and need to be "escaped":
? . [ ] ^ $
Note that a $ sign loses its meaning if characters follow it (I think) and the carat ^ loses its meaning if
other characters precede it.
Square brackets behave a little differently. The rules for square brackets go as follows:
 A closing square bracket loses its special meaning if placed first in a list. for example []12]
matches ] , 1, or 2.
 A dash - loses it's usual meaning inside lists if it is placed last.
 A carat ^ loses it's special meaning if it is not placed first
 Most special characters lose their meaning inside square brackets
Quotes
Back to top
Firstly, single quotes are the safest to use, because they protect your regular expression from the shell.
For example, grep "!" file will often produce an error (since the shell thinks that "!" is referring to
the shell command history) while grep '!' file will not.
When should you use single quotes ? the answer is this: if you want to use shell variables, you
need double quotes. For example,
grep "$HOME" file
searches file for the name of your home directory, while
grep '$HOME' file
searches for the string $HOME
This is an example of a common grep usage:
grep apple fruitlist.txt
In this case, grep prints all lines containing apple from the file fruitlist.txt, regardless of word
boundaries; therefore lines containing pineapple or apples are also printed. The grep command is

case sensitive by default, so this example's output does not include lines containing Apple (with a
capital A) unless they also contain apple.
To search all .txt files in a directory for apple in a shell that supports globbing, use an asterisk in
place of the file name:
grep apple *.txt
Regular expressions can be used to match more complicated queries. The following prints all
lines in the file that begin with the letter a, followed by any one character, then the letters ple.
grep ^a.ple fruitlist.txt
As noted above, the term "grep" derives from a usage in ed and related text editors. Before grep
existed as a separate command, the same effect might have been achieved by doing:
ed fruitlist.txt
g/^a.ple/p
q
where the second line is the command given to ed to print the relevant lines, and the third line is
the command to exit from ed.
Like most Unix commands, grep accepts options in the form of command-line arguments, to
change many of its behaviors. For example:
grep -i apple fruitlist.txt
This prints all lines containing apple regardless of capitalization. The -i argument tells grep to be
case insensitive, or to ignore case.
To print all lines containing apple as a word (pineapple and apples will not match):
grep -w apple fruitlist.txt
But if fruitlist.txt contains apple word followed by hyphen (-) character, it will also get matched.
cat fruitlist.txt
apple
apples
pineapple
apple-
apple-fruit
fruit-apple
grep -w apple fruitlist.txt
apple
apple-
apple-fruit
fruit-apple

the -v (lower-case v) prints all lines that do NOT contain apple in this example.
grep -v apple fruitlist.txt
banana
pear
peach
orange
General Introduction
The vi editor (short for visual editor) is a screen editor which is available on almost all Unix systems.
Once you have learned vi, you will find that it is a fast and powerful editor. vi has no menus but instead
uses combinations of keystrokes in order to accomplish commands. If you are just beginning to learn
Unix, you might find the Pico editor easier to use (most command options are displayed at the bottom
of the screen). If you use the Pine email application and have composed or replied to a message you
have probably already used Pico as it is used for text entry. For more information please refer to the
Pine/Pico page.
Starting vi
To start using vi, at the Unix prompt type vi followed by a file name. If you wish to edit an existing file,
type in its name; if you are creating a new file, type in the name you wish to give to the new file.
%vi filename
Then hit Return. You will see a screen similar to the one below which shows blank lines with tildes and
the name and status of the file.
~
~
"myfile" [New file]
vi's Modes and Moods
vi has two modes: the command mode and the insert mode. It is essential that you know which mode
you are in at any given point in time. When you are in command mode, letters of the keyboard will be
interpreted as commands. When you are in insert mode the same letters of the keyboard will type or
edit text. vi always starts out in command mode. When you wish to move between the two modes, keep
these things in mind. You can type i to enter the insert mode. If you wish to leave insert mode and
return to the command mode, hit the ESC key. If you're not sure where you are, hit ESC a couple of
times and that should put you back in command mode.

General Command Information
As mentioned previously, vi uses letters as commands. It is important to note that in general vi
commands:
 are case sensitive - lowercase and uppercase command letters do different things
 are not displayed on the screen when you type them
 generally do not require a Return after you type the command.
You will see some commands which start with a colon (:). These commands are ex commands which are
used by the ex editor. ex is the true editor which lies underneath vi -- in other words, vi is the interface
for the ex editor.
Entering Text
To begin entering text in an empty file, you must first change from the command mode to the insert
mode. To do this, type the letter i. When you start typing, anything you type will be entered into the file.
Type a few short lines and hit Return at the end of each of line. Unlike word processors, vi does not use
word wrap. It will break a line at the edge of the screen. If you make a mistake, you can use the
Backspace key to remove your errors. If the Backspace key doesn't work properly on your system, try
using the Ctrl h key combination.
Cursor Movement
You must be in command mode if you wish to move the cursor to another position in your file. If you've
just finished typing text, you're still in insert mode and will need to press ESC to return to the command
mode.
Moving One Character at a Time
Try using your direction keys to move up, down, left and right in your file. Sometimes, you may find that
the direction keys don't work. If that is the case, to move the cursor one character at the time, you may
use the h, j, k, and l keys. These keys move you in the following directions:
h left one space l right one space
j down one space k up one space
If you move the cursor as far as you can in any direction, you may see a screen flash or hear a beep.
Moving among Words and Lines
While these four keys (or your direction keys) can move you just about anywhere you want to go in your
file, there are some shortcut keys that you can use to move a little more quickly through a document. To
move more quickly among words, you might use the following:
w moves the cursor forward one word
b moves the cursor backward one word (if in the middle of a
word, b will move you to the beginning of the current word).
e moves to the end of a word.

To build on this further, you can precede these commands with a number for greater movement. For
example, 5w would move you forward five words; 12b would move you backwards twelve words. [You
can also use numbers with the commands mentioned earlier. For example, 5j would move you down 5
characters.]
Command Keys and Case
You will find when using vi that lower case and upper case command keys are interpreted differently.
For example, when using the lower case w, b, and e commands, words will be defined by a space or a
punctuation mark. On the other hand, W, B, and E commands may be used to move between words
also, but these commands ignore punctuation.
Shortcuts
Two short cuts for moving quickly on a line include the $ and the 0 (zero) keys. The $ key will move you
to the end of a line, while the 0 will move you quickly to the beginning of a line.
Screen Movement
To move the cursor to a line within your current screen use the following keys:
H moves the cursor to the top line of the screen.
M moves the cursor to the middle line of the screen.
L moves the cursor to the last line of the screen.
To scroll through the file and see other screens use:
ctrl-f scrolls down one screen
ctrl-b scrolls up one screen
ctrl-u scrolls up a half a screen
ctrl-d scrolls down a half a screen
Two other useful commands for moving quickly from one end to the other of a document are G to move
to the end of the file and 1G to move to the beginning of the file. If you precede G with a number, you
can move to a specific line in the document (e.g. 15G would move you to line 15).
Moving by Searching
One method for moving quickly to a particular spot in your file is to search for specific text. When you
are in command mode, type a / followed the text you wish to search for. When you press Return, the
cursor will move to the first incidence of that string of text. You can repeat the search by typing n or
search in a backwards direction by using N.
Basic Editing
To issue editing commands, you must be in command mode. As mentioned before, commands will be
interpreted differently depending upon whether they are issued in lower or upper case. Also, many of
the editing commands can be preceded by a number to indicate a repetition of the command.

Deleting (or Cutting) Characters, Words, and Lines
To delete a character, first place your cursor on that character. Then, you may use any of the following
commands:
x deletes the character under the cursor.
X deletes the character to the left of your cursor.
dw deletes from the character selected to the end of the word.
dd deletes all the current line.
D deletes from the current character to the end of the line.
Preceding the command with a number will delete multiple characters. For example, 10x will delete the
character selected and the next 9 characters; 10X will delete the 10 characters to the left of the
currently selected character. The command 5dw will delete 5 words, while 4dd deletes four lines.
Pasting Text using Put
Often, when you delete or cut text, you may wish to reinsert it in another location of the document. The
Put command will paste in the last portion of text that was deleted since deleted text is stored in a
buffer. To use this command, place the cursor where you wish the deleted text to appear. Then use p to
reinsert the text. If you are inserting a line or paragraph use the lower case p to insert on the line below
the cursor or upper case P to place in on the line above the cursor.
Copying Text with Yank
If you wish to make a duplicate copy of existing text, you may use the yank and put commands to
accomplish this function. Yank copies the selected text into a buffer and holds it until another yank or
deletion occurs. Yank is usually used in combination with a word or line object such as the ones shown
below:
yw copies a word into a buffer (7yw copies 7 words)
yy copies a line into a buffer (3yy will copy 3 lines)
Once the desired text is yanked, place the cursor in the spot in which you wish to insert the text and
then use the put command (p for line below or P for line above) to insert the contents of the buffer.
Replacing or Changing Characters, Words, and Lines
When you are using the following commands to replace text, you will be put temporarily into insert
mode so that you can change a character, word, line, or paragraph of text.
r replaces the current character with the next character you enter/type.
Once you enter the character you are returned to command mode.
R puts you in overtype mode until you hit ESC which will then return
you to command mode.
cw changes and replaces the current word with text that you type. A
dollar
sign marks the end of the text you're changing. Pressing ESC when you
finish will return you to command mode.

Inserting Text
If you wish to insert new text in a line, first position the cursor to the right of where you wish the
inserted text to appear. Type i to get into insert mode and then type in the desired text (note that the
text is inserted before the cursor). Press ESC to return to command mode.
Inserting a Blank Line
To insert a blank line below the line your cursor is currently located on, use the o key and then hit ESC to
return to the command mode . Use O to insert a line above the line the cursor is located on.
Appending Text
You can use the append command to add text at any place in your file. Append (a) works very much like
Insert (i) except that it insert text after the cursor rather than before it. Append is probably used most
often for adding text to the end of a line. Simply place your cursor where you wish to append text and
press a. Once you've finished appending, press ESC to go back to command mode.
Joining Lines
Since vi does not use automatic word wrap, it is not unusual in editing lines to end up with lines that are
too short and that might be improved if joined together. To do this, place your cursor on the first line to
be joined and type J. As with other commands, you can precede J with a number to join multiple lines
(4J joins 4 lines).
Undoing
Be sure to remember this command. When you make a mistake you can undo it. DO NOT move the
cursor from the line where you made the change. Then try using one of the following two commands:
u undoes the last change you made anywhere in the file. Using u again
will "undo the undo".
U undoes all recent changes to the current line. You can not have moved
from the line to recover the original line.
Closing and Saving Files
When you edit a file in vi, you are actually editing a copy of the file rather than the original. The
following sections describe methods you might use when closing a file, quitting vi, or both.
Quitting and Saving a File
The command ZZ (notice that it is in uppercase) will allow you to quit vi and save the edits made to a
file. You will then return to a Unix prompt. Note that you can also use the following commands:
:w to save your file but not quit vi (this is good to do periodically in
case of machine crash!).
:q to quit if you haven't made any edits.
:wq to quit and save edits (basically the same as ZZ).

Quitting without Saving Edits
Sometimes, when you create a mess (when you first start using vi this is easy to do!) you may wish to
erase all edits made to the file and either start over or quit. To do this, you can choose from the
following two commands:
:e! reads the original file back in so that you can start over.
:q! wipes out all edits and allows you to exit from vi.
More about Combining Commands, Objects, and Numbers
Now that you've learned some basic vi commands you might wish to expand your skills by trying some
fancy combination steps. Some commands are generally used in combination with a text object. We've
already seen some examples of this. For example, when you use the command dw to delete a word, that
combines the delete (d) command with the word (w) text object. When you wish to delete multiple
words, you might add a number to this combination. If you wished to delete 2 words you might use 2dw
or d2w. Either of these combinations would work. So, as you can see, the general format for a command
can be
(number) (command) (text object) or (command) (number) (text object)
You might wish to try out some of the following combinations of commands and objects:
Command Text Object
d (delete) w (word to the left)
y (yank/copy) b (word to the right or backward)
c (change) e (end of word)
H (top of the screen)
L (bottom of the screen)
M (middle of the screen)
0 (zero - first character on a line)
$ (end of a line)
( (previous sentence)
) (next sentence)
[ (previous section)
] (next section)
Repeating a Command
If you are doing repetitive editing, you may wish to use the same command over and over. vi will allow
you to use the dot (.) to repeat the last basic command you issued. If for example, you wished to deleted
several lines, you could use dd and then . (dot) in quick succession to delete a few lines.
A Quick Word about Customizing Your vi Environment
There are several options that you can set from within vi that can affect how you use vi. For example,
one option allows you to set a right margin that will then force vi to automatically wrap your lines as you

type. To do this, you would use a variation of the :set command. The :set command can be used to
change various options in vi. In the example just described, you could, while still in vi, type :set
wrapmargin=10 to specify that you wish to have a right margin of 10. Another useful option is :set
number. This command causes vi to display line numbers in the file you are working on.
Other Options
To view a listing of other options, you could type :set all. To view only those options which are currently
in effect, you can type set: by itself. Options that you set while in a vi session will apply during that
session only. To make permanent changes to your vi environment, you could edit your .exrc file.
However, you should not edit this file unless you know what you are doing!
Useful vi Commands
Cut/Paste Commands:
x delete one character (destructive backspace)
dw delete the current word (Note: ndw deletes n numbered words)
dd delete the current line (Note: ndd deletes n numbered lines)
D delete all content to the right of the cursor
d$ same as above
:u undo last command
p,P paste line starting one line below/above current cursor
location
J combine the contents of two lines
"[a-z]nyy yank next n lines into named buffer [a-z]
"[a-z]p/P place the contents of selected buffer below/above the current
line
Extensions to the Above Commands:
:3,18d delete lines 3 through 18
16,25m30 move lines 16 through 25 to after line 30
23,29co62 copy specified lines and place after line 62
Cursor Relocation commands:
:[n] goto line [n]
shift g place cursor on last line of text
h/l/j/k move cursor left, right, down and up
^f/^b move forward, backward in text, one page
^u/^d move up, down one half page
$ move to end of line
0 move to beginning of line
Extensions to the Above:
b move backwards one word (Note: nb moves back n number of
words)
e move to end of current word
( move to beginning of curent block
) move to the end of current block
Searching and Substitution commands:
/ [string] search forward for string
? [string] search backwards for string

n repeat last search
N repeat search in opposite direction
cw change the contents of the current word, (use ESC to stop
replacement mode)
c$ Replace all content to the right of cursor (exit replacement
mode with ESC)
c0 Replace all content to the left of cursor (exit with ESC)
:1,$s/s1/s2/g (Yow!) global replacement of string1 with string2
r replace current character with next character typed
Entering the Insert Mode:
i Begin inserting text at current cursor location
I Begin inserting text at the beginning of the current line
a Begin appending text, one character to the right of current
cursor location
A Begin appending text at the end of the current line
o/O Begin entering text one line belowabove current line
ESC Exit insertion mode and return to command mode
Exiting and Entering VI
ZZ save file and exit VI
:wq same as above
:e! return to last saved version of current file
:q quit without save, (Note :q! is required if changes have been
made)
:w write without exit (:w! to force write)
Fancy Stuff:
:1,10w file write lines 1 through 10 to file newfile
:340,$w >> file write lines 340 through the end of the file and append
to file newfile
:sh escape temporarily to a shell
^d return from shell to VI
:![command] execute UNIX command without leaving VI
:r![command] read output of command into VI
:r[filename] read filename into VI
:$r newfile read in newfile and attach at the end of current
document
:r !sort file read in contents of file after it has been passed
through
the UNIX sort
:n open next file (works with wildcard filenames,
ex: vi file*)
:^g list current line number
:set number show line numbers
:set showinsert show flag ("I") at bottom of screen when in insert mode
:set all display current values of VI variables
:set ai set autoindent; after this enter the insert mode and
tab, from this point on VI will indent each line to
this location. Use ESC to stop the indentations.
^T set the autoindent tab one tab stop to the right
^D set the autoindent tab one stop to the left
:set tabstop=n sets default tab space to number n
>> shift contents of line one tab stop to the right
<< shift contents of line one tab stop to the left
man get manual page on a UNIX command

example: man uniq
cut extract columns of data
example: cut -f -3,5,7-9 -d ' ' infile1 > outfile1
-f 2,4-6 field
-c 35-44 character
-d ':' delimiter (default is a tab)
sort sort lines of a file (Warning: default delimiter is white
space/character transition)
example: sort -nr infile1 | more
-n numeric sort
-r reverse sort
-k 3,5 start key
wc count lines, words, and characters in a file
example: wc -l infile1
-l count lines
-w count words
-c count characters
paste reattach columns of data
example: paste infile1 infile2 > outfile2
cat concatenate files together
example: cat infile1 infile2 > outfile2
-n number lines
-vet show non-printing characters (good
for finding problems)
uniq remove duplicate lines (normally from a sorted file)
example: sort infile1 | uniq -c > outfile2
-c show count of lines
-d only show duplicate lines
join perform a relational join on two files
example: join -1 1 -2 3 infile1 infile2 > outfile1

-1 FIELD join field of infile1
-2 FIELD join field of infile2
cmp compare two files
example: cmp infile1 infile2
diff or diff3 compare 2 or 3 files - show differences
example: diff infile1 infile2 | more
example: diff3 infile1 infile2 infile3 > outfile1
head extract lines from a file counting from the beginning
example: head -100 infile1 > outfile1
tail extract lines from a file counting from the end
example: tail +2 infile1 > outfile1
-n count from end of file (n is an integer)
+n count from beginning of file (n is an integer)
dos2unix convert dos-based characters to UNIX format (the file is
overwritten).
example: dos2unix infile1
tr translate characters - example shows replacement of spaces
with newline character
example: tr " " "[012*]" < infile1 > outfile
grep extract lines from a file based on search strings and
regular expressions
example: grep 'Basin1' infile1 > outfile2
example: grep -E '15:20|15:01' infile1 | more
sed search and replace parts of a file based on regular
expressions
example: sed -e 's/450/45/g' infile1 > outfile3
Regular Expressions
Regular expressions can be used with many programs including ls, grep, sed,

vi, emacs, perl, etc. Be aware that each program has variations on usage.
ls examples:
ls Data*.txt
ls Data4[5-9].ps list ps files beginning with Data numbered 45-
49
sed examples: (these are the regex part of the sed command only)
s/450/45/g search for '450' replace with '45' everywhere
s/99/-9999.00/g search for all '99' replace with '-9999.00'
s/Basin[0-9]//g remove the word Basin followed by a single
digit
s/^12/12XX/ search for '12' at the beginning of a line,
insert XX
s/Basin$// remove the word Basin if it is at the end of
the line.
s/^Basin$// remove the word Basin if it is the only word on
the line.
s/[cC]/100/g search for 'c' or 'C' replace with 100
45,$s/([0-9][0-9]).([0-9][0-9])/2.1/g
on lines 45 to the end of file, search for two
digits
followed by a '.' followed by two digits.
replace
with the digit pairs reversed.
2,$s/,([^,]*),/,"1",/
on all lines except the first, search for a
comma,
followed by any text, followed by a comma.
replace
the found text surrounded by double quotes.
s/([0-9][0-9]):([0-9][0-9]):([0-9][0-9][0-9][0-9])/Year = 3,
Month = 2, Day = 1/
search for 2 digits, followed by a colon,
followed by 2 digits,
followed by a colon, followed by 4 digits.
replace with
text plus values in a different order.
Pipes, standard input, standard output:
Standard output, ">", places the results of a command into the file named
after the ">". A new file will be written (an old file with the same name
will be removed). In order to append to an existing file use ">>".
Pipes allow you to connect multiple commands together to form a data stream.
For example, to count the number of times the string "Nile" occurs in the
3rd column of a file run this:
cut -f 3 infile1 | sort | uniq -c | grep 'Nile'

or do this:
cut -f 3 infile1 | grep 'Nile' | wc -l
From a global STN Attributes data set (tab delimited):
- extract all North American basins draining into the Atlantic Ocean
- select only columns 2,3,4,5,11,12,13, and 17
- replace all missing data values (either -99 or -999) with -9999.0
- remove duplicate lines
- sort by the first column
- number all lines sequentially
- save to a new file
grep 'North America' STNAttributes.txt | grep 'Atlantic Ocean'
| cut -f 2-5,11-13,17 | sed -e 's/-99|-999/-9999.0/g'
| sort | uniq | cat -n > NewSTNAttributes.txt

Unix

More Related Content

What's hot (19)

Viewers also liked (11)

Similar to Unix (20)

Recently uploaded (20)

Unix