Welcome to the first entry of Linux in the Shell.  Before delving into specific commands, redirection will be explored as redirection will be used frequently in the examples going forward.  The Unix philosophy posits program simplicity and that a program should do one thing and do it well (Mike Gancarz, the Unix Philosophy).  Eric Raymond adds the Rule of Composition:  “Design programs to be connected to other programs.”  Redirection is the glue that achieves this design.

Redirection is applied to any of the following standard streams to achieve results beyond simply outputting some value from a single command:

• Standard Input (stdin) – 0
• Standard Output (stdout) – 1
• Standard Error (stderr) – 2

The numbers: 0,1 and 2 are file descriptors – “abstract indicators for accessing a file.” Therefore, each of the standard streams are considered as acting like a file to the OS. That is, file directors can be applied to not only files but “directories, block or character devices, sockets, FIFO’s (named pipes), and unnamed pipes.”  Delving deeper in these constructs are beyond the entry of this post; suffice to say that the three streams can be treated in many different ways to achieve different outcomes through the use of redirection.  For more information on file descriptors or a deeper understanding of the standard streams consult the WikiPedia entries linked above.

The focus of this entry is the use of the following three symbols to facilitate redirection:

• > or >>  – output redirection
• < or <<  – input redirection
• | – pipe

Output redirection is simply redirecting the output of a command from stdout to somewhere else.  More often than not redirection is to a file to be processed in another manner:

ls /home/dann/music > musiclist.txt

This examples redirects the output of the ls (list) command in the /home/dann/music directory to a file called musiclist.txt.  Instead of the output of the ls command being directed to the screen, the output is directed to the file. If the musiclist.txt file is read the contents would be the output of the ls command one would typically see directed to the screen.

A single “>” writes destructively.  That is, it will take the contents of stdout and replace whatever existed in the location it is being redirected to with stdout:

cat /dev/null > musiclist.txt

Because “>” is destructive, the contents of musiclist.txt have now been overwritten by the output of the cat (concatenate) command being redirected in to musiclist.txt.  (/dev/null, aka:  the bit bucket; is a special device file that provides no data.) Simarly:

ls /home/dann/music2 > musiclist.txt

will again redirect the output of the directory listing of music2 to the file musiclist.txt obliterating any data that was already in musiclist.txt with the output of the command.

If we wanted to preserve the contents of musiclist.txt and append the output of the ls command in the third example, then “>>” is used.  Instead of overwriting the output, the stream appends to when the stream is redirected.

ls /home/music2 >> musiclist.txt

“>” by itself redirects only stdout, it does nothing for stderr.  If you want to redirect stderr you must specify the file descriptor value of stderr along with “>”:

cat mememe 2> error.list

Supposing the file mememe does not exist, instead of showing the error:  “cat mememe:  No such file or directory” that output of stderr is directed to the file error.list.  Recall that “>” is destructive so if we ran the comand again, it would replace the contents of error.list with the same message.  If we wanted to instead append errors to the error.list file:

cat mememe 2>> error.list

Take for instance this example:

cat file1 mememe 1> error.list

Notice that instead of “>”, “1>” was used.  Recall that the file descriptor for output is 1.  Essentially this command is identical to:  cat file1 mememe > error.list.  Now if “file1” exists but “mememe” does not, the contents of file1 will be written to error.list and the error:  “cat mememe:  No such file or directory” is written to the screen.  If we wanted to redirect both stdout and stderr to the file error.list the “&” is used:

cat file1 mememe &> error.list

Now nothing is written to the screen and both the output of “file1” and the error from non-existent “mememe” are written to error.list.  But what if we wanted to write only errors to the file, if there are any?  Again, the file descriptor for stderr is used in conjunction with “>”:

cat file1 mememe 2> error.list

In this example the output of “file1” is shown on the screen and the error message from “mememe” is redirected to error.list.

Take note that if you want to redirect both stdin and stderr to a file this can only be done destructively with “>”.  There is no “&>>” functionality.

 

Input redirection, “<” redirects a stream to the input of a command:

wc < musiclist.txt

In this example the file musiclist.txt is passed to the wc command.  This example is not very practical because it essentially produces the same output as wc musiclist.txt.

The use of “<<” does not mean append in the case of stdin.  Instead it can be used to represent a break point in the input.  That is, accept input until a token is received:

cat > poem.txt << “EOF”

This example is a quick and dirty text editor that only does input.  This will accept input from stdin (keyboard in this case) and write it to the file poem.txt until “EOF” is passed through stdin. At that point the cat will exit and close poem.txt.  This type of redirect is used a lot when building Linux From Scratch.

The final redirect we will cover is the “|” or pipe redirect.  “|” is used primarily to pass the output of one command to the input of another:

ls |grep music

In this example the output of the ls command is passed to grep.  That ouput is checked to see if there is a file or directory with the word music in the listing.   Another more useful example is:

ls -lt |less

This example pass the output of ls to the less command allowing you to filter through the directory listing.

The “|” can be used with other redirection techniques to chain together a useful string of commands and produce some desired result:

grep -R “dann@thelinuxlink.net” * | grep -v svn |  \

cut -d”:” -f1  > whereIAmListed.txt

This examples greps recursively through the current directory structure looking for all instances of the email address “dann@thelinuxlink.net”, filters out anything that has svn in the result, pass the filtered list to cut setting a delimiter of “:” and outputting only the first field (everything before the first “:”) which is redirected to a file called whereIAmListed.txt.

Take note of the “\” after the second “|”.  This has no bearing on the command as a whole it is just a way to represent that this is a single string across multiple lines.  If you typed this in as shown, hitting the enter key after the “\” the string would process normally.  This syntax is merely to show that the above string is all one line even though format conventions of this media spread it out over two lines.

This entry covered redirection of standard streams:  input, output, and error.  The symbols utilized were “>”, “>>”, “<“, “<<“, and “|”.  It should be noted that redirection to and from a file makes use of the “<“and “>” symbols, whereas redirection of the output of one command to another makes use of the “|” symbol.  This is just the tip of the iceberg on redirection, but enough to give an understanding as we go forward with exploring commands in the Linux bash shell.  If you are interesting in exploring redirection more thoroughly consult these links:

• Standard Input Definition
• Standard Output Definition
• Standard Error Definition
• Bash Programming – All About Redirection
• Redirection Standard Input and Output
• Input Output Redirection Made Simple
• IO Redirection
• Advanced Bash Scripting Guide – I/O Redirection

 

 

If the video is not clear enough view it off the YouTube website and select size 2 or full screen.  Or download the video in Ogg Theora format:

• LiTS 000 – Redirection Ogg Theora Video (right click save as to download)

Thank you very much!

• Audio OGG (from HPR)
• Audio Speex (from HPR)
• Audio MP3 (from HPR)