1.1 Input Redirection

Input redirection sends text from a file to the standard input of a program as if it were being typed from the keyboard.

1.1.1 Example: wc

For example, the program wc (for word count) reads text from its input stream and produces as its output stream three numbers indicating the number of lines, words, and characters that it saw. You could invoke this directly:

wc
Hello.
How are you?
^D

in which case, you would see as output:

     2     4    20

For this to be very useful, however, we need to make it accept a file as input. We can do this using the < operator in the shell. Think of the < as an arrow indicating data flowing towards the command from a filename: If hello.c is this file:

#include <stdio.h> 
int main () 
{
  printf ("Hello from C!\n"); 
  return 0;
}

then the command

wc < hello.c

produces the output

6    13    80

You can also focus the wc command on characters, lines, or words with options -c, -w, or -l, respectively. For example,

wc -l < hello.c

produces the output

6

1.2 Output Redirection

On the output end, the shell operator > redirects the standard output into a file (again, think of this as an arrow indicating data flowing into a filename from the command):

wc < hello.c > hello.wc

produces no output on the screen, but creates a file called hello.wc. That file will contain the output

6    13    80

of the wc command.

Example 1: Try This:

cd ~/playing
echo Hello > greeting.txt
more greeting.txt
echo Goodbye > greeting.txt
more greeting.txt
echo Farewell > greeting.txt
more greeting.txt

1.2.1 sed

Another command that is often used as a “filter” is sed, the “Stream EDitor”. sed allows you to enter a variety of editing commands that will be applied to every line of a stream of text coming from the standard in (or from a file).

The most common use of sed is to scan each line of input for a pattern and to replace that pattern, wherever it occurs, by some string. The sed command to do this is

sed s/pattern/replacement/g 

where pattern describes the text to search for in each line, and replacement is the text we want to ue to replace the text matching the pattern.

(The ‘/’ characters are simply necessary to indicate the beginning and end of the pattern and replacement strings. They can be replaced by nearly any character that does not appear in either the pattern or replacement strings.)

1.3 Combining Input and Output Redirection

A command can use both input and output redirection, but it is crucial that different files be used for the input and the output.

Example 3: Try This: Dual Redirection

cd ~/playing
echo Hello > greeting.txt
sed s/el/o/ < greeting.txt > holo.txt
more greeting.txt
more holo.txt
sed s/el/o/ < greeting.txt > greeting.txt
more greeting.txt

The first sed command rewrites the contents of the input file, saving the rewritten content in the output file.

The second sed command trashes the input file contents. That’s because the first step in preparing a file for output is to empty out any prior contents, and that is done by the command shell program that interprets your keyboard command and launches the sed program. So the file gets emptied out before the sed command is actually launched. sed never gets a chance to read the input file’s contents.

1.4 Appending to Files

The output redirection operator has an important variant. Sometimes we would like to add output to the end of an existing file instead of replacing that file. This is done with the operator >>. So the code sequence

wc < hello.c > hello.wc
wc < hello.c >> hello.wc

would result in a file hello.wc with contents

6    13    80
6    13    80

regardless of whether hello.wc had existed previously.

Example 4: Try This:

cd ~/playing
echo Aloha >> greeting.txt
more greeting.txt

2 Pipes

Redirection uses files for input or output.

We can also send the output of a command to the input of another command. We envision this action as if we had connected the two commands by a pipe through which the output text of one command is flowing to the input of the other.

To pipe the output of one command into the input of another, use the shell operator |. A common example of a pipe is to take a command that may have a large amount of output and to pipe it through more to facilitate viewing.

Example 5: Try This:

Try these commands:

ls /bin
ls /bin | more

As you gain facility with a greater variety of Unix text manipulation commands, you will find that redirection and pipes can be a powerful combination. For example, suppose that you have written program myprog that emits a great deal of output, among which might be some error messages starting with the phrase “ERROR:”. If you wanted to read only the error messages, you could, of course, just view all the output, watching for the occasional error message:

myprog | more

But if the program produces a lot of output, this will quickly become tedious. However, we previously encountered the program grep, which scans its input stream, printing only those lines matching a given regular expression. By piping the myprog output through grep, we can limit the output to the part we really want to see:

myprog | grep "ERROR:" | more

Example 6: Try This:

You can use sed to reformat the output from some commands. Give the following commands:

cd ~/playing
echo Hello > greeting.txt
cat greeting.txt
cat greeting.txt | sed -e 's/^/She said, "/' -e 's/o/o"./'

The -e option to sed allows us to string together multiple substitution commands.