Basic File Manipulation

Last modified: Aug 24, 2023
Contents:

1 Getting Started: Navigating Your Directories

We’ve already looked at some basic navigation commands. Now, let’s put them together with commands for setting up our own directories.

1.1 Basic Navigation: mkdir, cd, ls, pwd

If you have not yet done so, log in now so that you can work though the following commands.

 
Example 1: Try This

Now, let’s make a place to play in. mkdir will make a new directory. Enter the command

cd ~
mkdir playing

to create a directory named “playing”.

How does the mkdir command know where to put the new directory?

Give the command

ls

and you should see playing listed. In fact, it may be the only thing listed.

Now, let’s add some directories inside playing:

mkdir playing/cards
mkdir playing/games
ls
ls playing

Give the command sequence

pwd
cd playing/cards
pwd
cd ..
pwd
cd games
pwd
cd ../cards
pwd
cd ./cards
pwd
cd ../..
pwd

to review the use of relative paths.

The mkdir command can also be used to create a whole “chain” of new directories at once with the -p option. Suppose that we wanted to create a directory named a inside a directory named b inside a directory named c, and that none of those directories exist yet.

Here are three ways to do that:

mkdir c
cd c
mkdir b
cd b
mkdir a
cd ../..

or

mkdir c
mkdir c/b
mkdir c/b/a

or, quickest of all, just

mkdir -p c/b/a

The tree command is not a standard command on all Unix systems, but can be added to most of them. tree allows you to visualize the structure of your directories and files.

Example 2: Try This

Try the following commands:

cd ~
tree ~
tree playing
tree ~/playing

2 Manipulating Files

2.1 Copying Files: cp

The cp command copies one or more files. You can give this command as

cp sourcePath destinationPath

to make a copy of file whose path is sourcePath, the copy being named or located at the path destinationPath.

For example,

cp file1 file2

would make a copy of an existing file, in the current working directory, named “file1”.

You can copy multiple files in one command, but then you must give a directory as the destination path.

cp sourcePath1 sourcePath2sourcePathN directoryPath

Example 3: Try This:

Now try the following:

ls /usr/include

You should see a large number of files, including many ending in “.h”. Copy two of these files and check to see that the copy was successful, as follows:

cp /usr/include/math.h /usr/include/stdio.h ~/playing
ls ~/playing
cd ~/playing
cp /usr/include/limits.h .
ls
cp limits.h maxmin.h
ls
ls .
ls ~/playing

The first two cp commands above give a directory as the destination for the copy, so the new files should appear in that directory (with the same names as they had in their original directory). The second one uses the abbreviation “. ” to denote your current working directory.

The third cp command gives a file name as the destination, so the copy is given that new name.

The three ls statements in a row actually all list the same directory. With no parameter, the first ls lists your current directory. The final two ls commands simply give different paths to that same directory.

Try

cd ~/playing/games
cp ~cs252/Assignments/editingAsst/constitution.txt thePeople.txt
ls

You can see that this version of the cp command created a copy of the source file in the current directory, using the name supplied for the destination.

Finally, let’s get a summary of everything we have done to our playing directory.

tree ~/playing

2.2 Looking Inside Text Files with more

The more command,

more filePath1filepathN

can be used to examine the contents of a file.

The more command is used to page through a file, one screen at a time. (It gets its name from the “more” prompt at the bottom of the screen, letting you know that there are more pages to be displayed.

(You can also type ‘/’, followed by a string of characters, to search the file for that string.)

Example 4: Try This:
  1. Give the command

    more ~/playing/math.h
    
  2. Use the space bar to move forwards through the file.

  3. Use ‘b’ to move backwards.

  4. Use ‘q’ to quit.

  5. Give the commands

    cd ~/playing
    more games/thePeople.txt
    

    Again, use the space bar and b to move around, and use q to exit.

more can also be used to tame commands that generate so much output that it scrolls by before we can read it. We just add “| more” to the end of a command to page through its output. (We’ll learn the full meaning of the “|” in a much later lesson.)

Example 5: Try This:
ls /usr/include
ls /usr/include | more

The cat command dumps the contents of one or more text files to your screen.

cd ~/playing
cat math.h games/thePeople.txt
cat math.h games/thePeople.txt | more

2.3 Making Mistakes

Before we go too much further in exploring new commands, let’s take a slight diversion. Sooner or later you are going to make a mistake when typing out a command – you may already have done so.

Don’t panic! Read the error message that you get carefully. It really will help.

Let’s make some deliberate mistakes so that you can get familiar with what some of the more common error messages look like.

Example 6: Try This: (path problems)

Each of the commands listed below contains a path.

To start with, just look at these commands:

cd ~/playing
ls ~/playing/math.h
ls math.h
ls ../playing/math.h
ls playing/math.h
cd ~
ls ~/playing/math.h
ls math.h
ls ../playing/math.h
ls playing/math.h

Which commands use absolute paths and which use relative?

Answer

Now, try typing out those commands, one by one. Not all of them will work. Some will generate error messages. Can you figure out why?

Answer
Example 7: Try This: (bad commands)

Let’s get back into our playing directory:

cd ~/playing

Now issue each of the following commands and observe the error messages you get:

noSuchCommand
list

The first response is obvious enough. Of course, it’s clear that noSuchCommand is not, in fact, the name of a valid command. You’re more likely to see that response, however, because you misspelled a command name.

The second command might be an example of such a misspelling, typing list instead of ls. The response is different, however, because list is close enough to some commands that could have been, but aren’t installed on our Linux server.

Most “commands” are, ultimately, files that contain executable programs. So what happens if we just start naming some file that aren’t executable programs? Try:

~/playing
~/playing/math.h

The response to the first one is clear enough. ~/playing is a directory name, not a command. The command shell that processes our typed input has no idea what we want to do with that directory. Do we want to list its contents (ls)? Do we want to step into it (cd)? Do we want to…? You can’t simply give a directory (or file) name and expect the operating system to guess what you want to do.

The response to the second command is a bit more cryptic. Why is it talking about “permissions”? We’ll see in a later lesson that we can set permissions on what both we ourselves and other people can do with our files. One of the things that we can give ourself or other people permission to do is to execute that file. Now, it only makes sense to execute a file if that file actually holds a program, not simply data of some kind.

How does the operating system know if a file contains a program? One important way is that it looks to see if we have given anyone permission to execute it. In this case, math.h does not contain a program, so it was not set up with “execute” permission. So when we try to execute it, the operating system refuses, telling us that we lack the (execute) permission.

Example 8: Try This: (bad parameters)

Try:

cd ~/playing
ls
cp math.hh mathcopy.h
ls

In the third line, we are deliberately misspelling the name of the file “math.h”. The “No such file” response should be obvious enough.

This is a very common error message, one that is pretty much self-explanatory, and yet one that I get a lot of email about. Now, it’s very easy to misspell a file name. It’s also easy to forget where a file is or is supposed to be.

But the appropriate response to this message is not to throw your hands up in the air and give up, nor is it to to email me the message and stop working until you hear back.

Instead, look around. Use the ls, pwd, and cd commands that we have covered to look and see what has actually occurred.

The cp command can lead to lots of interesting errors and error messages. Earlier, for example, you gave the command “cp /usr/include/math.h /usr/include/stdio.h ~/playing” to copy two files at once into a directory. Let’s try a variation on that:

cp /usr/include/math.h /usr/include/stdio.h ~/playingzzz

If we are going to copy multiple files at once, we have to give cp the name of a directory in which to put them. In this example, we have (badly) misspelled the name of that directory.

Interestingly, we don’t get an error message from this:

cp /usr/include/math.h ~/playingzzz

To see what has happened, do

ls ~
more ~/playingzzz

When we are only copying a single file, cp gives us the option of either giving a destination directory (which must already exist) or a destination file (which will be created if it does not exist.

Example 9: Try This: (trying to copy directories instead of ordinary files)

Try:

ls ~/playing
cp /usr/include/err.h /usr/include /usr/include/limits.h ~/playing
ls ~/playing

You’ll see that the first and third file have, indeed, been copied. But the second file in the list was not, and the error message tells you why.

The cp command, under normal circumstances, copies only ordinary files and not directories. It is possible to change this behavior, but because copying entire directory structures is somewhat risky (you can waste a lot of storage with some simple mistakes), it’s turned off by default.

Most of the times when I see students getting the “directory omitted” error, it’s because they use the directory name when they should have named a file or a group of files inside that directory.

It’s as if I gave you a box of cookies and, instead of opening the box and eating the cookies, you instead chewed on the cardboard box itself (and then complained about the bland taste).

2.4 Getting More Information from ls

The ls command can give you more detailed information about your files if you add the -l option. (Note, that is a lower-case letter ‘L’, not a numeric digit one.)

Example 10: Try This:

let’s add a little something extra to our playing directory:

mkdir ~/playing/catch
cp /bin/which ~/playing
ls ~/playing

You should see the files that you copied earlier, plus the newly created which and catch files. But, which of these are directories? You might remember that mkdir creates directories, and so guess that around is a directory, but there’s no clue to that in this listing, and if you have a lot of files in a listing, you might not know or remember how they were all created.

Now try:

ls -l ~/playing

Now you have some additional information. This includes the size of the files and the date (and possibly the time) at which these files were last modified. The rather cryptic “rwx-” characters at the beginning are something that will be explained in a later lesson. However, if you look at the very first character on each line, it tells you something very useful. it will be ‘d’ if the file is a directory and ‘-’ if it is an ordinary, non-directory, file.

There are easier ways to find out if a file is a directory or not. The -F option asks ls to put a ‘/’ character at the end of any directory names and a ’*’ character at the end of any files that are executable programs/commands. Try the commands

ls -F ~
ls -F ~/playing
 

You should be able to easily identify directories and ordinary files.

/bin is where most of the basic Unix commands are stored, so it’s not surprising that a file copied from there would turn out to have been an executable command. In fact, try:

ls -F /bin

Notice that the ls, pwd, and mkdir commands that you have been practicing with are all present in the directory listing for /bin. (You’ll also see some files listed with a ‘@’ character. These denote symbolic links, a special type of directory entry that we won’t deal with in this course.)

2.4.1 Hidden Files

Most operating systems have a way of hiding selected files so that they do not appear in ordinary directory listing.

In Unix, any file or directory name starting with a period (‘.’) is hidden.

Hidden files can be revealed by adding a -a option to the ls command.

Example 11: Try This
cd ~/playing
ls -F games cards
cp games/thePeople.txt cards/.secret.txt

Notice the ‘.’ in front of “secret”.

ls -F games
ls -F cards
more games/thePeople.txt
more cards/.secret.txt

The secret file really is there, even though it does not show up in the listing.

To force ls to show you secret files, add the -a option, either as a seprate option or joined with others.

ls -a -F cards
ls -aF cards

You can now see the secret file. In addition to that, you see two other items listed, even though neither of them is, technically, a file inside the cards directory:

  • .’: You might recall that this is shorthand for the current directory.
  • ..’: This is shorthand for the parent or container of current directory.

I don’t know why ls feels compelled to list these – it’s just kind of a distraction.

2.5 Moving and Renaming Files: mv

Similar to the cp command, the mv command can be used to move and/or rename a file.

Example 12: Try This:

Wait, if necessary, until at least one minute has passed since you did the prior Try This. Give the commands

ls -l ~/playing
mv ~/playing/cards ~/playing/withFire
ls -F ~/playing
cp ~/playing/which ~/playing/games/why
mv ~/playing/which ~/playing/games/what
ls -F ~/playing
ls -l ~/playing/games

Notice that we are changing both names and locations of files with the mv command.

On the other hand, look at the date on which the files which and what, and why were last changed in the various listings. The fact that the which and what dates are the same is a good indication that mv has actually moved the existing files and not made a new copy of them. By contrast, why, which was produced as a copy, should have a more recent date.

2.6 Deleting Files: rm and rmdir

The rmdir command is used to delete directories.
It will only do so if the directory is empty.

Example 13: Try This

Give the following commands. Not all of the rmdir commands will work. Can you predict which ones will?

cd ~/playing
tree .
rmdir games
rmdir catch
rmdir withFire
Answer

The rm command can be used to remove files.

Example 14: Try This

Give the following commands. Not all of the rmdir commands will work. Can you predict which ones will?

cd ~/playing
ls games
rm games/why games/what
ls games
ls -a withFire
rm withFire/.secret.txt
ls -a withFire
ls -F
rmdir withFire
ls -F

Because we have removed the only file in withFire, we can now use the rmdir command to remove it.

3 File Manipulation in Other Operating Systems

3.1 MacOs File Systems

If you do not have access to an Apple OS/X or macOs PC, skip to the next section.

We’ve already seen how to navigate basic directory structures in MacOs.

Not surprisingly, the commands we have learned for manipulating files are the same in MacOs as in other Unix-based systems.

Example 15: Try This

Open a text editor on your Mac and create a file with the following text:

The five boxing wizards jump quickly.

(This is a pangram, a sentence containing every letter of the alphabet at least once.)

Save that as a file named “foo.txt” in your home directory, `/Users/yourLoginName/’.

Open a Terminal window on your Mac. In that window, try the following commands.

cd ~
pwd
ls -F
mv foo.txt playing
ls -F
ls -F playing
cd playing
pwd
cp foo.txt bar.txt
ls -F
more bar.txt
rm foo.txt
ls -F

3.2 Windows File Systems

If you do not have access to a Windows PC, skip this section.

Windows is the only commonly used operating system today that is not a Unix variant, so it will have lots of differences from Linux. Still, many of the concepts translate over, with some renaming of commands:

Example 16: Try This

Open NotePad on your PC and create a file with the following text:

The five boxing wizards jump quickly.

(This is a pangram, a sentence containing every letter of the alphabet at least once.)

Save that as a file named “foo.txt” in your home directory, `C:\Users\yourLoginName'.

Open a CMD window on your Windows PC. (Click the Start/Windows button on the left end of the task bar and type cmd, then hit enter.)

In that window, try the following commands.

c:
cd \users\yourLoginName\
mkdir playing
dir
move foo.txt playing
dir
dir playing
cd playing
dir
copy foo.txt bar.txt
dir
more bar.txt
del foo.txt
dir

4 Commands Glossary

Command Example Explanation
cp path1 path2 ... destination cp /usr/include/math.h /usr/include/stdio.h ~/playing Copy one or more files to a destination. The paths must point to ordinary files, not directories.
If there is more than one path, the destination must be a directory. All of the files listed in the paths are copied into that directory.
If there is only one path, the destination may be an existing file or directory or it might not exist. If it is an existing file, this command overwrites it with the contents of the path file. If it is an existing directory, the path file is copied into that directory. If the destination does nto exist, a file is created with with the name indicated in the destination and the contents of the path copied into it.
cat path1 path2 ... cat ~/playing/math.h List the contents of one or more files to the screen, as if they had been joined together (or concatenated).
ls -a path ls -a . list the files at that path including all “hidden” files (anems starting with ‘.’)
ls -l path ls -l ~/playing list the files at that path with information indicating whether they are directories and indicating what permissions people have o access those files and directories.
mkdir path mkdir ~/playing/cards create a directory
more path mkdir ~/playing/math.h List the contents of a file, one “page” at a time. Use the space bar to move forward, ‘b’ to move back, and ‘q’ to quit.
mv filepath1 filepath2 mv foo.txt ~/playing/bar.txt | Move and rename the file at the first path to the second. (The second path must not be an existing directory.) | |mv path1 path2destination|mv ~/playing/math.h ~/playing/stdio.h ~/playing| Move one or more files into a destination directory. | |rm path1 path2|rm ~/playing/math.h| Delete one or more files. | |rmdir path|rmdir ~/playing/cards| Delete a directory (must be empty). | |tree path|tree ~/playing` produce a visual display of an entire “tree” of directories and files

You can find a glossary of the commands covered in this course in Commands Glossary.