[ previous ] [ Contents ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] [ 7 ] [ 8 ] [ 9 ] [ 10 ] [ 11 ] [ 12 ] [ 13 ] [ 14 ] [ 15 ] [ 16 ] [ 17 ] [ 18 ] [ A ] [ B ] [ C ] [ D ] [ next ]

Debian Tutorial (Obsolete Documentation)
Chapter 16 - Advanced topics

(Should advanced topics be here? I think it would be nice to have some of these, just to show people the possibilities and give some conceptual explanation that won't really be in a reference manual. Also it always feels nice to make it to a chapter called "advanced topics". Self-esteem booster for the newbie. :)

16.1 Introduction to shell scripting

16.1.1 What and why

Automate simple tasks.

16.1.2 A simple example


16.2 Advanced files

16.2.1 The real nature of files: hard links and inodes

Each file on your system is represented by an inode (for Information Node; pronounced "eye-node"): an inode contains all the information about the file. However, the inode is not directly visible. Instead, each inode is linked into the filesystem by one or more hard links. Hard links contain the name of the file, and the inode number. The inode contains the file itself, i.e., the location of the information being stored on disk, its access permissions, the type of the file, and so on. The system can find any inode once it has the inode number.

A single file can have more than one hard link. What this means is that multiple filenames refer to the same file (that is, they are associated with the same inode number). However, you can't make hard links across filesystems: all hard references to a particular file (inode) must be on the same filesystem. This is because each filesystem has its own set of inodes, and there can be duplicate inode numbers between filesystems.

Since all hard links to a given inode are referring to the same file, you can make changes to the file, referring to it by one name, and then see those changes when referring to it by a different name. Try this:

  1. cd; echo "hello" > firstlink

    cd to your home directory and create a file called firstlink containing the word "hello". What you've actually done is redirect the output of echo (echo just echoes back what you give to it), placing the output in firstlink. See the chapter on shells for a full explanation.

  1. cat firstlink

    Confirm the contents of firstlink.

  1. ln firstlink secondlink

    Create a hard link: secondlink now points to the same inode as firstlink.

  1. cat secondlink

    Confirm that secondlink is the same as firstlink

  1. ls -l

    Notice that the number of hard links listed for firstlink and secondlink is 2.

  1. echo "change" >> secondlink

    This is another shell redirection trick - don't worry about the details. We've appended the word "change" to secondlink. Confirm this with cat secondlink.

  1. cat firstlink

    firstlink also has the word "change" appended! It's because firstlink and secondlink refer to the same file. It doesn't matter what you call it when you change it.

  1. chmod a+rwx firstlink

    Change permissions on firstlink. Do ls -l to confirm that permissions on secondlink were also changed. This means that permissions information is stored in the inode, not in links.

  1. rm firstlink

    Delete this link. This is a subtlety of rm --- it really removes links, not files. Now type ls -l and notice that secondlink is still there. Also notice that the number of hard links for secondlink has been reduced to one.

  1. rm secondlink

    Delete the other link. When there are no more links to a file, Linux deletes the file itself, that is, its inode.

All files work like this --- even special types of files such as devices (e.g. /dev/hda).

A directory is simply a list of filenames and inode numbers, that is, a list of hard links. When you create a hard link, you're just adding a name-number pair to a directory. When you delete a file, you're just removing a hard link from a directory.

16.2.2 Types of files

One detail we've been concealing up to now is that the Linux kernel considers nearly everything to be a file. That includes directories and devices: they're just special kinds of files.

As you may remember, the first character of an ls -l display represents the type of the file. For an ordinary file, this will be simply -. Other possibilities are: Symbolic links

Symbolic links (also called symlinks or soft links) are the other kind of link besides hard links. A symlink is a special file that "points to" a hard link on any mounted filesystem. When you try to read the contents of a symlink, it gives the contents of the file it's pointing to rather than the contents of the symlink itself. Since directories, devices, and other symlinks are types of files, you can point a symlink at any of those things.

So a hard link is a filename and an inode number. A file is really an inode: a location on disk, file type, permissions mode, etc. A symlink is an inode that contains the name of a hard link. A symlink pairs one filename with a second filename, while a hard link pairs a filename with an inode number.

All hard links to the same file have equal status. That is, one is as good as the other; if you perform any operation on one it's just the same as performing that operation on any of the others. This is because the hard links all refer to the same inode. Operations on symlinks, on the other hand, sometimes affect the symlink's own inode (the one containing the name of a hard link) and sometimes affect the hard link being pointed to.

There are a number of important differences between symlinks and hard links:

Try this:

  1. cd; ln -s /tmp/me MyTmp

    cd to your home directory. ln with the -s option makes a symbolic link; in this case, one called MyTmp which points to the filename /tmp/me.

  1. ls -l MyTmp

    Output should look like this:

         lrwxrwxrwx   1 havoc    havoc           7 Dec  6 12:50 MyTmp -> /tmp/me

    The date and user/group names will be different for you, of course. Notice that the file type is l, indicating that this is a symbolic link. Also notice the permissions - symbolic links always have these permissions. If you attempt to chmod a symlink, you'll actually change the permissions on the file being pointed to.

  1. chmod 700 MyTmp

    You will get a "No such file or directory" error, because the file /tmp/me doesn't exist. Notice that you could create a symlink to it anyway.

  1. mkdir /tmp/me

    Create the directory /tmp/me.

  1. chmod 700 MyTmp

    Should work now.

  1. touch MyTmp/myfile

    Create a file in MyTmp.

  1. ls /tmp/me

    The file was actually created in /tmp/me.

  1. rm MyTmp

    Remove the symbolic link. Notice that this removes the link, not what it points to. Thus you use rm not rmdir.

  1. rm /tmp/me/myfile; rmdir /tmp/me

    Clean up after ourselves. Device files

Device files refer to physical or virtual devices on your system, such as your hard disk, video card, screen, or keyboard. An example of a virtual device is the console, represented by /dev/console.

There are two kinds of devices: character devices can be accessed one character at a time, that is, the smallest unit of data which can be written to or read from the device is a character (byte).

Block devices must be accessed in larger units called blocks, which contain a number of characters. Your hard disk is a block device.

You can read and write device files just as you can from other kinds of files, though the file may well contain some strange incomprehensible-to-humans gibberish. Writing random data to these files is probably a Bad Idea. Sometimes it's useful, though: for example, you can dump a postscript file into the printer device /dev/lp0, or send modem commands to the device file for the appropriate serial port. /dev/null

/dev/null is a special device file that discards anything you write to it. If you don't want something, throw it in /dev/null. It's essentially a bottomless pit. If you read /dev/null, you'll get an end-of-file (EOF) character immediately. /dev/zero is similar, only if you read from it you get the \0 character (not the same as the number zero). Named pipes (FIFOs)

A named pipe is a file that acts like a pipe. You put something into the file, and it comes out the other end. Thus it's called a FIFO, or First-In-First-Out: the first thing you put in the pipe is the first thing to come out the other end.

If you write to a named pipe, the process which is writing to the pipe doesn't terminate until the information being written is read from the pipe. If you read from a named pipe, the reading process waits until there's something to read before terminating. The size of the pipe is always zero --- it doesn't store data, it just links two processes like the shell |. However, since this pipe has a name, the two processes don't have to be on the same command line or even be run by the same user.

You can try it by doing the following:

  1. cd; mkfifo mypipe

    Makes the pipe.

  1. echo "hello" > mypipe &

    Puts a process in the background which tries to write "hello" to the pipe. Notice that the process doesn't return from the background; it is waiting for someone to read from the pipe.

  1. cat mypipe

    At this point the echo process should return, since cat read from the pipe, and the cat process will print hello.

  1. rm mypipe

    You can delete pipes just like any other file. Sockets

Sockets are similar to pipes, only they work over the network. This is how your computer does networking: you may have heard of "WinSock", which is sockets for Windows.

We won't go into these further, because you probably won't have occasion to use them unless you're programming. However, if you see a file marked with type s on your computer, you know what it is.

16.2.3 The proc filesystem

The Linux kernel makes a special filesystem available, which is mounted under /proc on Debian systems. This is a "pseudo-filesystem" --- it doesn't really exist on any of your physical devices.

The proc filesystem contains information about the system and running processes. Some of the "files" in /proc are reasonably understandable to humans (try typing cat /proc/meminfo or cat /proc/cpuinfo) while some others are arcane collections of numbers. Often, system utilities use these to gather information and present it to you in a more understandable way.

People frequently panic when they notice one file in particular --- /proc/kcore --- which is generally huge. This is (more or less) a copy of the contents of your computer's memory. It's used to debug the kernel. It doesn't actually exist anywhere, so don't worry about its size.

If you want to know about all the things in /proc, type man 5 proc.

16.2.4 Advanced aspects of file permissions Using numeric arguments with chmod

Earlier in this chapter, we briefly mentioned that you can set file permissions using numbers. The numeric notation is called an absolute mode, as opposed to the symbolic notation (e.g. u+rx) which is often called a relative mode. This is because the number specifies an exact mode to set, and the symbol just specifies a change to make (e.g. "add user read and execute permissions").

The numeric mode is a series of four octal digits or twelve binary digits. Each octal (base eight) digit represents three binary digits: one octal digit and three binary digits are two ways to represent the decimal digits 0 through 7.

Deriving a particular mode is pretty straightforward. You simply add up the modes you want to combine, or subtract modes you don't want. For example, user permissions, with only read permission turned on, would be 100 in binary. User permissions with write only would be 010 binary. User permissions with read and write both turned on would be 100 + 010 = 110. Alternatively, you could put it in octal: 4 + 2 = 6.

For the full mode, simply add up digits from this table:

     0001        others, execute
     0002        others, write
     0004        others, read
     0010        group, execute
     0020        group, write
     0040        group, read
     0100        user, execute
     0200        user, write
     0400        user, read
     1000        sticky bit
     2000        set group id
     4000        set user id

To use the table, first decide what permissions you want to set. Then add up the numbers for those permissions. The total is your mode. For example, to get mode 0755:

       0001   o=x
       0004   o=r
       0010   g=x 
       0040   g=r
       0100   u=x
       0200   u=w
     + 0400   u=r
       0755  u=rwx go=rw

You'd actually call this mode simply 755, without the leading 0, because chmod automatically adds zeroes at the beginning of the mode --- 7 means mode 0007.

To set a file to 755, you'd type chmod 755 myfile.

755 is a very common mode for directories, as it allows anyone to use the directory but only the owner to create and delete files in the directory. 644 is the analogous mode for files, and it is also very common. It allows anyone to use the file but only the owner can change it. For executable files, 755 is a common mode; this is just 644 plus execute permissions (644 + 111 = 755).

16.2.5 chattr

A useful tip?

16.2.6 Large-scale copying

cp -a and variants on the theme.

how to copy an old system to a new one.

FIXME whoops, I also listed this topic under Backup Tools. need to decide.

16.2.7 Other concepts not yet covered, but should be

fsck, dd, fdisk, etc.

what package is a file in?

MSDOS vs. Mac vs. Unix text files


16.3 Compiling the kernel

How, what, and why

16.4 A few words on security

The basics of security from a user standpoint. Maintaining one's privacy. What other users can see of your account.

16.5 Programming on Linux

Something about the Linux programming environment. Aimed at, say, people taking CS101. Nothing on how to program, just Emacs, gcc, gdb, ddd, etc. as programming tools.

Likely based on debug.tex

[ previous ] [ Contents ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] [ 7 ] [ 8 ] [ 9 ] [ 10 ] [ 11 ] [ 12 ] [ 13 ] [ 14 ] [ 15 ] [ 16 ] [ 17 ] [ 18 ] [ A ] [ B ] [ C ] [ D ] [ next ]

Debian Tutorial (Obsolete Documentation)

29 Dezember 2009

Havoc Pennington hp@debian.org