Chapter 9. System tips

Table of Contents

9.1. The console tips
9.1.1. Recording the shell activities cleanly
9.1.2. The screen program
9.1.3. Navigating around directories
9.1.4. Readline wrapper
9.2. Customizing vim
9.2.1. Customizing vim with internal features
9.2.2. Customizing vim with external packages
9.3. Data recording and presentation
9.3.1. The log daemon
9.3.2. Log analyzer
9.3.3. Customized display of text data
9.3.4. Customized display of time and date
9.3.5. Colorized shell echo
9.3.6. Colorized commands
9.3.7. Recording the editor activities for complex repeats
9.3.8. Recording the graphic image of an X application
9.3.9. Recording changes in configuration files
9.4. Monitoring, controlling, and starting program activities
9.4.1. Timing a process
9.4.2. The scheduling priority
9.4.3. The ps command
9.4.4. The top command
9.4.5. Listing files opened by a process
9.4.6. Tracing program activities
9.4.7. Identification of processes using files or sockets
9.4.8. Repeating a command with a constant interval
9.4.9. Repeating a command looping over files
9.4.10. Starting a program from GUI
9.4.11. Customizing program to be started
9.4.12. Killing a process
9.4.13. Scheduling tasks once
9.4.14. Scheduling tasks regularly
9.4.15. Alt-SysRq key
9.5. System maintenance tips
9.5.1. Who is on the system?
9.5.2. Warning everyone
9.5.3. Hardware identification
9.5.4. Hardware configuration
9.5.5. System and hardware time
9.5.6. The terminal configuration
9.5.7. The sound infrastructure
9.5.8. Disabling the screen saver
9.5.9. Disabling beep sounds
9.5.10. Memory usage
9.5.11. System security and integrity check
9.6. Data storage tips
9.6.1. Disk space usage
9.6.2. Disk partition configuration
9.6.3. Accessing partition using UUID
9.6.4. LVM2
9.6.5. Filesystem configuration
9.6.6. Filesystem creation and integrity check
9.6.7. Optimization of filesystem by mount options
9.6.8. Optimization of filesystem via superblock
9.6.9. Optimization of hard disk
9.6.10. Optimization of solid state drive
9.6.11. Using SMART to predict hard disk failure
9.6.12. Specify temporary storage directory via $TMPDIR
9.6.13. Expansion of usable storage space via LVM
9.6.14. Expansion of usable storage space by mounting another partition
9.6.15. Expansion of usable storage space by bind-mounting another directory
9.6.16. Expansion of usable storage space by overlay-mounting another directory
9.6.17. Expansion of usable storage space using symlink
9.7. The disk image
9.7.1. Making the disk image file
9.7.2. Writing directly to the disk
9.7.3. Mounting the disk image file
9.7.4. Cleaning a disk image file
9.7.5. Making the empty disk image file
9.7.6. Making the ISO9660 image file
9.7.7. Writing directly to the CD/DVD-R/RW
9.7.8. Mounting the ISO9660 image file
9.8. The binary data
9.8.1. Viewing and editing binary data
9.8.2. Manipulating files without mounting disk
9.8.3. Data redundancy
9.8.4. Data file recovery and forensic analysis
9.8.5. Splitting a large file into small files
9.8.6. Clearing file contents
9.8.7. Dummy files
9.8.8. Erasing an entire hard disk
9.8.9. Erasing unused area of an hard disk
9.8.10. Undeleting deleted but still open files
9.8.11. Searching all hardlinks
9.8.12. Invisible disk space consumption
9.9. Data encryption tips
9.9.1. Removable disk encryption with dm-crypt/LUKS
9.9.2. Encrypted swap partition with dm-crypt
9.9.3. Mounting encrypted disk with dm-crypt/LUKS
9.10. The kernel
9.10.1. Kernel parameters
9.10.2. Kernel headers
9.10.3. Compiling the kernel and related modules
9.10.4. Compiling the kernel source: Debian Kernel Team recommendation
9.10.5. Hardware drivers and firmware
9.11. Virtualized system
9.11.1. Virtualization and emulation tools
9.11.2. Virtualization work flow
9.11.3. Mounting the virtual disk image file
9.11.4. Chroot system
9.11.5. Multiple desktop systems

Here, I describe basic tips to configure and manage systems, mostly from the console.

There are some utility programs to help your console activities.


The simple use of script(1) (see Section 1.4.9, “Recording the shell activities”) to record shell activity produces a file with control characters. This can be avoided by using col(1) as the following.

$ script
Script started, file is typescript

Do whatever … and press Ctrl-D to exit script.

$ col -bx < typescript > cleanedfile
$ vim cleanedfile

There are alternative methods to record the shell activities:

  • Use tee (usable during the boot process in the initramfs):

    $ sh -i 2>&1 | tee typescript
  • Use gnome-terminal with the extend line buffer for scrollback.

  • Use screen with "^A H" (see Section 9.1.2, “The screen program”) to perform recording of console.

  • Use vim with ":terminal" to enter the terminal mode. Use "Ctrl-W N" to exit from terminal mode to normal mode. Use ":w typescript" to write the buffer to a file.

  • Use emacs with "M-x shell", "M-x eshell", or "M-x term" to enter recording console. Use "C-x C-w" to write the buffer to a file.

screen(1) not only allows one terminal window to work with multiple processes, but also allows remote shell process to survive interrupted connections. Here is a typical use scenario of screen(1).

  1. You login to a remote machine.

  2. You start screen on a single console.

  3. You execute multiple programs in screen windows created with ^A c ("Control-A" followed by "c").

  4. You switch among the multiple screen windows by ^A n ("Control-A" followed by "n").

  5. Suddenly you need to leave your terminal, but you don't want to lose your active work by keeping the connection.

  6. You may detach the screen session by any methods.

    • Brutally unplug your network connection

    • Type ^A d ("Control-A" followed by "d") and manually logging out from the remote connection

    • Type ^A DD ("Control-A" followed by "DD") to have screen detach and log you out

  7. You log in again to the same remote machine (even from a different terminal).

  8. You start screen as "screen -r".

  9. screen magically reattaches all previous screen windows with all actively running programs.

[Tip] Tip

You can save connection fees with screen for metered network connections such as dial-up and packet ones, because you can leave a process active while disconnected, and then re-attach it later when you connect again.

In a screen session, all keyboard inputs are sent to your current window except for the command keystroke. All screen command keystrokes are entered by typing ^A ("Control-A") plus a single key [plus any parameters]. Here are important ones to remember.


See screen(1) for details.

See tmux(1) for functionalities of the alternative command.

In Section 1.4.2, “Customizing bash”, 2 tips to allow quick navigation around directories are described: $CDPATH and mc.

If you use fuzzy text filter program, you can do without typing the exact path. For fzf, include following in ~/.bashrc.

FZF_KEYBINDINGS_PATH=/usr/share/doc/fzf/examples/key-bindings.bash
if [ -f $FZF_KEYBINDINGS_PATH ]; then
  . $FZF_KEYBINDINGS_PATH
fi
FZF_COMPLETION_PATH=/usr/share/doc/fzf/examples/completion.bash
if [ -f $FZF_COMPLETION_PATH ]; then
  . $FZF_COMPLETION_PATH
fi

For example:

  • You can jump to a very deep subdirectory with minimal efforts. You first type "cd **" and press Tab. Then you will be prompted with candidate paths. Typing in partial path strings, e.g., s/d/b foo, will narrow down candidate paths. You select the path to be used by cd with cursor and return keys.

  • You can select a command from the command history more efficiently with minimal efforts. You press Ctrl-R at the command prompt. Then you will be prompted with candidate commands. Typing in partial command strings, e.g., vim d, will narrow down candidates. You select the one to be used with cursor and return keys.

After you learn basics of vim(1) through Section 1.4.8, “Using vim”, please read Bram Moolenaar's "Seven habits of effective text editing (2000)" to understand how vim should be used.

[Caution] Caution

Don't try to change the default key bindings without very good reasons.

The behavior of vim can be changed significantly by enabling its internal features through the Ex-mode commands such as "set ..." to set vim options.

These Ex-mode commands can be included in user's vimrc file, traditional "~/.vimrc" or git-friendly "~/.vim/vimrc". Here is a very simple example [2]:

colorscheme murphy             " from /usr/share/vim/vim??/colors/*.vim
filetype plugin indent on      " filetype aware behavior
syntax enable                  " Syntax highlight
"set spelllang=en_us            " Spell check language as en_us
"set spell                      " Enable spell check
set autoindent                 " Copy indent from current line
set smartindent                " More than autoindent (Drop/Pop after {/})
set nosmarttab                 " <Tab>-key always inserts blanks
set backspace=indent,eol,start " Back space through everything
set laststatus=2               " Always show status line
set statusline=%<%f%m%r%h%w%=%y[U+%04B]%2l/%2L=%P,%2c%V

Simple customization to enable securemodelines and classical IDE can be enabled by installing vim-scripts package and appending the following to user's vimrc file.

packadd! securemodelines
packadd! winmanager
let mapleader = ' '
" Toggle paste mode with <SPACE>p
set pastetoggle=<leader>p
" IDE-like UI for files and buffers with <space>w
nnoremap <leader>w         :WMToggle<CR>
" Use safer keys <SPACE>? for moving to another window
nnoremap <leader>h         <C-W>h
nnoremap <leader>j         <C-W>j
nnoremap <leader>k         <C-W>k
nnoremap <leader>l         <C-W>l

The new native Vim package system works nicely with "git" and "git submodule". One such example configuration can be found at my git repository: dot-vim. This does essentially:

  • By using "git" and "git submodule", latest external packages, such as "name", are placed into ~/.vim/pack/*/opt/name and similar.

  • By adding :packadd! name line to user's vimrc file, these packages are placed on runtimepath.

  • Vim loads these packages on runtimepath during its initialization.

  • At the end of its initialization, tags for the installed documents are updated with "helptags ALL".

For more, please start vim with "vim --startuptime vimstart.log" to check actual execution sequence and time spent for each step.

Interesting external plugin packages can be found:

It is quite confusing to see too many ways[3] to manage and load these external packages to vim. Checking the original information is the best cure.


Although pager tools such as more(1) and less(1) (see Section 1.4.5, “The pager”) and custom tools for highlighting and formatting (see Section 11.1.8, “Highlighting and formatting plain text data”) can display text data nicely, general purpose editors (see Section 1.4.6, “The text editor”) are most versatile and customizable.

[Tip] Tip

For vim(1) and its pager mode alias view(1), ":set hls" enables highlighted search.

Shell echo to most modern terminals can be colorized using ANSI escape code (see "/usr/share/doc/xterm/ctlseqs.txt.gz").

For example, try the following

$ RED=$(printf "\x1b[31m")
$ NORMAL=$(printf "\x1b[0m")
$ REVERSE=$(printf "\x1b[7m")
$ echo "${RED}RED-TEXT${NORMAL} ${REVERSE}REVERSE-TEXT${NORMAL}"

You can record the editor activities for complex repeats.

For Vim, as follows.

  • "qa": start recording typed characters into named register "a".

  • … editor activities

  • "q": end recording typed characters.

  • "@a": execute the contents of register "a".

For Emacs, as follows.

  • "C-x (": start defining a keyboard macro.

  • … editor activities

  • "C-x )": end defining a keyboard macro.

  • "C-x e": execute a keyboard macro.

Program activities can be monitored and controlled using specialized tools.


[Tip] Tip

The procps packages provide very basics of monitoring, controlling, and starting program activities. You should learn all of them.

There are several ways to repeat a command looping over files matching some condition, e.g. matching glob pattern "*.ext".

for x in *.ext; do if [ -f "$x"]; then command "$x" ; fi; done
  • find(1) and xargs(1) combination:

find . -type f -maxdepth 1 -name '*.ext' -print0 | xargs -0 -n 1 command
  • find(1) with "-exec" option with a command:

find . -type f -maxdepth 1 -name '*.ext' -exec command '{}' \;
  • find(1) with "-exec" option with a short shell script:

find . -type f -maxdepth 1 -name '*.ext' -exec sh -c "command '{}' && echo 'successful'" \;

The above examples are written to ensure proper handling of funny file names such as ones containing spaces. See Section 10.1.5, “Idioms for the selection of files” for more advance uses of find(1).

For the command-line interface (CLI), the first program with the matching name found in the directories specified in the $PATH environment variable is executed. See Section 1.5.3, “The "$PATH" variable”.

For the graphical user interface (GUI) compliant to the freedesktop.org standards, the *.desktop files in the /usr/share/applications/ directory provide necessary attributes for the GUI menu display of each program. Each package which is compliant to Freedesktop.org's xdg menu system installs its menu data provided by "*.desktop" under "/usr/share/applications/". Modern desktop environments which are compliant to Freedesktop.org standard use these data to generate their menu using the xdg-utils package. See "/usr/share/doc/xdg-utils/README".

For example, the chromium.desktop file defines attributes for the "Chromium Web Browser" such as "Name" for the program name, "Exec" for the program execution path and arguments, "Icon" for the icon used, etc. (see the Desktop Entry Specification) as follows:

[Desktop Entry]
Version=1.0
Name=Chromium Web Browser
GenericName=Web Browser
Comment=Access the Internet
Comment[fr]=Explorer le Web
Exec=/usr/bin/chromium %U
Terminal=false
X-MultipleArgs=false
Type=Application
Icon=chromium
Categories=Network;WebBrowser;
MimeType=text/html;text/xml;application/xhtml_xml;x-scheme-handler/http;x-scheme-handler/https;
StartupWMClass=Chromium
StartupNotify=true

This is an oversimplified description. The *.desktop files are scanned as follows.

The desktop environment sets $XDG_DATA_HOME and $XDG_DATA_DIR environment variables. For example, under the GNOME 3:

  • $XDG_DATA_HOME is unset. (The default value of $HOME/.local/share is used.)

  • $XDG_DATA_DIRS is set to /usr/share/gnome:/usr/local/share/:/usr/share/.

So the base directories (see XDG Base Directory Specification) and the applications directories are as follows.

  • $HOME/.local/share/$HOME/.local/share/applications/

  • /usr/share/gnome//usr/share/gnome/applications/

  • /usr/local/share//usr/local/share/applications/

  • /usr/share//usr/share/applications/

The *.desktop files are scanned in these applications directories in this order.

[Tip] Tip

A user custom GUI menu entry can be created by adding a *.desktop file in the $HOME/.local/share/applications/ directory.

[Tip] Tip

Similarly, if a *.desktop file is created in the autostart directory under these base directories, the specified program in the *.desktop file is executed automatically when the desktop environment is started. See Desktop Application Autostart Specification.

[Tip] Tip

Similarly, if a *.desktop file is created in the $HOME/Desktop directory and the Desktop environment is configured to support the desktop icon launcher feature, the specified program in it is executed upon clicking the icon. Please note that the actual name of the $HOME/Desktop directory is locale dependent. See xdg-user-dirs-update(1).

Some programs start another program automatically. Here are check points for customizing this process.

[Tip] Tip

update-mime(8) updates the "/etc/mailcap" file using "/etc/mailcap.order" file (see mailcap.order(5)).

[Tip] Tip

The debianutils package provides sensible-browser(1), sensible-editor(1), and sensible-pager(1) which make sensible decisions on which editor, pager, and web browser to call, respectively. I recommend you to read these shell scripts.

[Tip] Tip

In order to run a console application such as mutt under X as your preferred application, you should create an X application as following and set "/usr/local/bin/mutt-term" as your preferred application to be started as described.

# cat /usr/local/bin/mutt-term <<EOF
#!/bin/sh
gnome-terminal -e "mutt \$@"
EOF
chmod 755 /usr/local/bin/mutt-term

Use cron(8) to schedule tasks regularly. See crontab(1) and crontab(5).

You can schedule to run processes as a normal user, e.g. foo by creating a crontab(5) file as "/var/spool/cron/crontabs/foo" with "crontab -e" command.

Here is an example of a crontab(5) file.

# use /bin/sh to run commands, no matter what /etc/passwd says
SHELL=/bin/sh
# mail any output to paul, no matter whose crontab this is
MAILTO=paul
# Min Hour DayOfMonth Month DayOfWeek command (Day... are OR'ed)
# run at 00:05, every day
5  0  *  * *   $HOME/bin/daily.job >> $HOME/tmp/out 2>&1
# run at 14:15 on the first of every month -- output mailed to paul
15 14 1  * *   $HOME/bin/monthly
# run at 22:00 on weekdays(1-5), annoy Joe. % for newline, last % for cc:
0 22 *   * 1-5 mail -s "It's 10pm" joe%Joe,%%Where are your kids?%.%%
23 */2 1 2 *   echo "run 23 minutes after 0am, 2am, 4am ..., on Feb 1"
5  4 *   * sun echo "run at 04:05 every Sunday"
# run at 03:40 on the first Monday of each month
40 3 1-7 * *   [ "$(date +%a)" == "Mon" ] && command -args
[Tip] Tip

For the system not running continuously, install the anacron package to schedule periodic commands at the specified intervals as closely as machine-uptime permits. See anacron(8) and anacrontab(5).

[Tip] Tip

For scheduled system maintenance scripts, you can run them periodically from root account by placing such scripts in "/etc/cron.hourly/", "/etc/cron.daily/", "/etc/cron.weekly/", or "/etc/cron.monthly/". Execution timings of these scripts can be customized by "/etc/crontab" and "/etc/anacrontab".

Systemd has low level capability to schedule programs to run without cron daemon. For example, /lib/systemd/system/apt-daily.timer and /lib/systemd/system/apt-daily.service set up daily apt download activities. See systemd.timer(5) .

Pressing Alt-SysRq (PrtScr) followed by one keys does the magic of rescuing control of the system.


See more on Linux kernel user’s and administrator’s guide » Linux Magic System Request Key Hacks

[Tip] Tip

From SSH terminal etc., you can use the Alt-SysRq feature by writing to the "/proc/sysrq-trigger". For example, "echo s > /proc/sysrq-trigger; echo u > /proc/sysrq-trigger" from the root shell prompt syncs and umounts all mounted filesystems.

The current (2021) Debian amd64 Linux kernel has /proc/sys/kernel/sysrq=438=0b110110110:

  • 2 = 0x2 - enable control of console logging level (ON)

  • 4 = 0x4 - enable control of keyboard (SAK, unraw) (ON)

  • 8 = 0x8 - enable debugging dumps of processes etc. (OFF)

  • 16 = 0x10 - enable sync command (ON)

  • 32 = 0x20 - enable remount read-only (ON)

  • 64 = 0x40 - enable signaling of processes (term, kill, oom-kill) (OFF)

  • 128 = 0x80 - allow reboot/poweroff (ON)

  • 256 = 0x100 - allow nicing of all RT tasks (ON)

The following sets system and hardware time to MM/DD hh:mm, CCYY.

# date MMDDhhmmCCYY
# hwclock --utc --systohc
# hwclock --show

Times are normally displayed in the local time on the Debian system but the hardware and system time usually use UTC(GMT).

If the hardware time is set to UTC, change the setting to "UTC=yes" in the "/etc/default/rcS".

The following reconfigure the timezone used by the Debian system.

# dpkg-reconfigure tzdata

If you wish to update system time via network, consider to use the NTP service with the packages such as ntp, ntpdate, and chrony.

[Tip] Tip

Under systemd, use systemd-timesyncd for the network time synchronization instead. See systemd-timesyncd(8).

See the following.

[Tip] Tip

ntptrace(8) in the ntp package can trace a chain of NTP servers back to the primary source.

Device drivers for sound cards for current Linux are provided by Advanced Linux Sound Architecture (ALSA). ALSA provides emulation mode for previous Open Sound System (OSS) for compatibility.

Application softwares may be configured not only to access sound devices directly but also to access them via some standardized sound server system. Currently, PulseAudio, JACK, and PipeWire are used as sound server system. See Debian wiki page on Sound for the latest situation.

There is usually a common sound engine for each popular desktop environment. Each sound engine used by the application can choose to connect to different sound servers.

[Tip] Tip

Use "cat /dev/urandom > /dev/audio" or speaker-test(1) to test speaker (^C to stop).

[Tip] Tip

If you can not get sound, your speaker may be connected to a muted output. Modern sound system has many outputs. alsamixer(1) in the alsa-utils package is useful to configure volume and mute settings.


Poor system maintenance may expose your system to external exploitation.

For system security and integrity check, you should start with the following.


Here is a simple script to check for typical world writable incorrect file permissions.

# find / -perm 777 -a \! -type s -a \! -type l -a \! \( -type d -a -perm 1777 \)
[Caution] Caution

Since the debsums package uses MD5 checksums stored locally, it can not be fully trusted as the system security audit tool against malicious attacks.

Booting your system with Linux live CDs or debian-installer CDs in rescue mode makes it easy for you to reconfigure data storage on your boot device.

For disk partition configuration, although fdisk(8) has been considered standard, parted(8) deserves some attention. "Disk partitioning data", "partition table", "partition map", and "disk label" are all synonyms.

Older PCs use the classic Master Boot Record (MBR) scheme to hold disk partitioning data in the first sector, i.e., LBA sector 0 (512 bytes).

Recent PCs with Unified Extensible Firmware Interface (UEFI), including Intel-based Macs, use GUID Partition Table (GPT) scheme to hold disk partitioning data not in the first sector.

Although fdisk(8) has been standard for the disk partitioning tool, parted(8) is replacing it.


[Caution] Caution

Although parted(8) claims to create and to resize filesystem too, it is safer to do such things using best maintained specialized tools such as mkfs(8) (mkfs.msdos(8), mkfs.ext2(8), mkfs.ext3(8), mkfs.ext4(8), …) and resize2fs(8).

[Note] Note

In order to switch between GPT and MBR, you need to erase first few blocks of disk contents directly (see Section 9.8.6, “Clearing file contents”) and use "parted /dev/sdx mklabel gpt" or "parted /dev/sdx mklabel msdos" to set it. Please note "msdos" is use here for MBR.

LVM2 is a logical volume manager for the Linux kernel. With LVM2, disk partitions can be created on logical volumes instead of the physical harddisks.

LVM requires the following.

  • device-mapper support in the Linux kernel (default for Debian kernels)

  • the userspace device-mapper support library (libdevmapper* package)

  • the userspace LVM2 tools (lvm2 package)

Please start learning LVM2 from the following manpages.

  • lvm(8): Basics of LVM2 mechanism (list of all LVM2 commands)

  • lvm.conf(5): Configuration file for LVM2

  • lvs(8): Report information about logical volumes

  • vgs(8): Report information about volume groups

  • pvs(8): Report information about physical volumes

For ext4 filesystem, the e2fsprogs package provides the following.

  • mkfs.ext4(8) to create new ext4 filesystem

  • fsck.ext4(8) to check and to repair existing ext4 filesystem

  • tune2fs(8) to configure superblock of ext4 filesystem

  • debugfs(8) to debug ext4 filesystem interactively. (It has undel command to recover deleted files.)

The mkfs(8) and fsck(8) commands are provided by the e2fsprogs package as front-ends to various filesystem dependent programs (mkfs.fstype and fsck.fstype). For ext4 filesystem, they are mkfs.ext4(8) and fsck.ext4(8) (they are symlinked to mke2fs(8) and e2fsck(8)).

Similar commands are available for each filesystem supported by Linux.


[Tip] Tip

Ext4 filesystem is the default filesystem for the Linux system and strongly recommended to use it unless you have some specific reasons not to.

Btrfs status can be found at Debian wiki on btrfs and kernel.org wiki on btrfs. It is expected to be the next default filesystem after the ext4 filesystem.

Some tools allow access to filesystem without Linux kernel support (see Section 9.8.2, “Manipulating files without mounting disk”).

Solid state drive (SSD) is auto detected now.

Reduce unnecessary disk accesses to prevent disk wear out by mounting "tmpfs" on volatile data path in /etc/fstab.

You can monitor and log your hard disk which is compliant to SMART with the smartd(8) daemon.

  1. Enable SMART feature in BIOS.

  2. Install the smartmontools package.

  3. Identify your hard disk drives by listing them with df(1).

    • Let's assume a hard disk drive to be monitored as "/dev/hda".

  4. Check the output of "smartctl -a /dev/hda" to see if SMART feature is actually enabled.

    • If not, enable it by "smartctl -s on -a /dev/hda".

  5. Enable smartd(8) daemon to run by the following.

    • uncomment "start_smartd=yes" in the "/etc/default/smartmontools" file.

    • restart the smartd(8) daemon by "sudo systemctl restart smartmontools".

[Tip] Tip

The smartd(8) daemon can be customized with the /etc/smartd.conf file including how to be notified of warnings.

For partitions created on Logical Volume Manager (LVM) (Linux feature) at install time, they can be resized easily by concatenating extents onto them or truncating extents from them over multiple storage devices without major system reconfiguration.

If you have usable space in another partition (e.g., "/path/to/empty" and "/path/to/work"), you can create a directory in it and stack that on to an old directory (e.g., "/path/to/old") where you need space using the OverlayFS for Linux kernel 3.18 or newer (Debian Stretch 9.0 or newer).

$ sudo mount -t overlay overlay \
  -olowerdir=/path/to/old-dir,upperdir=/path/to/empty,workdir=/path/to/work

Here, "/path/to/empty" and "/path/to/work" should be on the RW-enabled partition to write on "/path/to/old".

Here, we discuss manipulations of the disk image.

The disk image file, "disk.img", of an unmounted device, e.g., the second SCSI or serial ATA drive "/dev/sdb", can be made using cp(1) or dd(1) by the following.

# cp /dev/sdb disk.img
# dd if=/dev/sdb of=disk.img

The disk image of the traditional PC's master boot record (MBR) (see Section 9.6.2, “Disk partition configuration”) which reside on the first sector on the primary IDE disk can be made by using dd(1) by the following.

# dd if=/dev/hda of=mbr.img bs=512 count=1
# dd if=/dev/hda of=mbr-nopart.img bs=446 count=1
# dd if=/dev/hda of=mbr-part.img skip=446 bs=1 count=66
  • "mbr.img": The MBR with the partition table

  • "mbr-nopart.img": The MBR without the partition table

  • "mbr-part.img": The partition table of the MBR only

If you have an SCSI or serial ATA device as the boot disk, substitute "/dev/hda" with "/dev/sda".

If you are making an image of a disk partition of the original disk, substitute "/dev/hda" with "/dev/hda1" etc.

The disk image "partition.img" containing a single partition image can be mounted and unmounted by using the loop device as follows.

# losetup -v -f partition.img
Loop device is /dev/loop0
# mkdir -p /mnt/loop0
# mount -t auto /dev/loop0 /mnt/loop0
...hack...hack...hack
# umount /dev/loop0
# losetup -d /dev/loop0

This can be simplified as follows.

# mkdir -p /mnt/loop0
# mount -t auto -o loop partition.img /mnt/loop0
...hack...hack...hack
# umount partition.img

Each partition of the disk image "disk.img" containing multiple partitions can be mounted by using the loop device. Since the loop device does not manage partitions by default, we need to reset it as follows.

# modinfo -p loop # verify kernel capability
max_part:Maximum number of partitions per loop device
max_loop:Maximum number of loop devices
# losetup -a # verify nothing using the loop device
# rmmod loop
# modprobe loop max_part=16

Now, the loop device can manage up to 16 partitions.

# losetup -v -f disk.img
Loop device is /dev/loop0
# fdisk -l /dev/loop0

Disk /dev/loop0: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x452b6464

      Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1               1         600     4819468+  83  Linux
/dev/loop0p2             601         652      417690   83  Linux
# mkdir -p /mnt/loop0p1
# mount -t ext4 /dev/loop0p1 /mnt/loop0p1
# mkdir -p /mnt/loop0p2
# mount -t ext4 /dev/loop0p2 /mnt/loop0p2
...hack...hack...hack
# umount /dev/loop0p1
# umount /dev/loop0p2
# losetup -d /dev/loop0

Alternatively, similar effects can be done by using the device mapper devices created by kpartx(8) from the kpartx package as follows.

# kpartx -a -v disk.img
...
# mkdir -p /mnt/loop0p2
# mount -t ext4 /dev/mapper/loop0p2 /mnt/loop0p2
...
...hack...hack...hack
# umount /dev/mapper/loop0p2
...
# kpartx -d /mnt/loop0
[Note] Note

You can mount a single partition of such disk image with loop device using offset to skip MBR etc., too. But this is more error prone.

The empty disk image "disk.img" which can grow up to 5GiB can be made using dd(1) as follows.

$ dd bs=1 count=0 if=/dev/zero of=disk.img seek=5G

You can create an ext4 filesystem on this disk image "disk.img" using the loop device as follows.

# losetup -f -v disk.img
Loop device is /dev/loop1
# mkfs.ext4 /dev/loop1
...hack...hack...hack
# losetup -d /dev/loop1
$ du  --apparent-size -h disk.img
5.0G  disk.img
$ du -h disk.img
83M disk.img

For "disk.img", its file size is 5.0 GiB and its actual disk usage is mere 83MiB. This discrepancy is possible since ext4 can hold sparse file.

[Tip] Tip

The actual disk usage of sparse file grows with data which are written to it.

Using similar operation on devices created by the loop device or the device mapper devices as Section 9.7.3, “Mounting the disk image file”, you can partition this disk image "disk.img" using parted(8) or fdisk(8), and can create filesystem on it using mkfs.ext4(8), mkswap(8), etc.

The ISO9660 image file, "cd.iso", from the source directory tree at "source_directory" can be made using genisoimage(1) provided by cdrkit by the following.

#  genisoimage -r -J -T -V volume_id -o cd.iso source_directory

Similarly, the bootable ISO9660 image file, "cdboot.iso", can be made from debian-installer like directory tree at "source_directory" by the following.

#  genisoimage -r -o cdboot.iso -V volume_id \
   -b isolinux/isolinux.bin -c isolinux/boot.cat \
   -no-emul-boot -boot-load-size 4 -boot-info-table source_directory

Here Isolinux boot loader (see Section 3.1.2, “Stage 2: the boot loader”) is used for booting.

You can calculate the md5sum value and make the ISO9660 image directly from the CD-ROM device as follows.

$ isoinfo -d -i /dev/cdrom
CD-ROM is in ISO 9660 format
...
Logical block size is: 2048
Volume size is: 23150592
...
# dd if=/dev/cdrom bs=2048 count=23150592 conv=notrunc,noerror | md5sum
# dd if=/dev/cdrom bs=2048 count=23150592 conv=notrunc,noerror > cd.iso
[Warning] Warning

You must carefully avoid ISO9660 filesystem read ahead bug of Linux as above to get the right result.

Here, we discuss direct manipulations of the binary data on storage media.

With physical access to your PC, anyone can easily gain root privilege and access all the files on your PC (see Section 4.6.4, “Securing the root password”). This means that login password system can not secure your private and sensitive data against possible theft of your PC. You must deploy data encryption technology to do it. Although GNU privacy guard (see Section 10.3, “Data security infrastructure”) can encrypt files, it takes some user efforts.

Dm-crypt facilitates automatic data encryption via native Linux kernel modules with minimal user efforts using device-mapper.


[Caution] Caution

Data encryption costs CPU time etc. Please weigh its benefits and costs.

[Note] Note

Entire Debian system can be installed on a encrypted disk by the debian-installer (lenny or newer) using dm-crypt/LUKS and initramfs.

[Tip] Tip

See Section 10.3, “Data security infrastructure” for user space encryption utility: GNU Privacy Guard.

You can encrypt contents of removable mass devices, e.g. USB memory stick on "/dev/sdx", using dm-crypt/LUKS. You simply format it as the following.

# badblocks -c 1024 -s -w -t random -v /dev/sdx
# fdisk /dev/sdx
... "n" "p" "1" "return" "return" "w"
# cryptsetup luksFormat /dev/sdx1
...
# cryptsetup open --type luks /dev/sdx1 sdx1
...
# ls -l /dev/mapper/
total 0
crw-rw---- 1 root root  10, 60 2008-10-04 18:44 control
brw-rw---- 1 root disk 254,  0 2008-10-04 23:55 sdx1
# mkfs.vfat /dev/mapper/sdx1
...
# cryptsetup luksClose sdx1

Then, it can be mounted just like normal one on to "/media/disk_label", except for asking password (see Section 10.1.7, “Removable storage device”) under modern desktop environment, such as GNOME using gnome-mount(1). The difference is that every data written to it is encrypted. You may alternatively format media in different filesystem, e.g., ext4 with "mkfs.ext4 /dev/mapper/sdx1".

[Note] Note

If you are really paranoid for the security of data, you may need to overwrite multiple times (the "badblocks" command in the above example). This operation is very time consuming though.

Let's assume that your original "/etc/fstab" contains the following.

/dev/sda7 swap sw 0 0

You can enable encrypted swap partition using dm-crypt by as the following.

# aptitude install cryptsetup
# swapoff -a
# echo "cswap /dev/sda7 /dev/urandom swap" >> /etc/crypttab
# perl -i -p -e "s/\/dev\/sda7/\/dev\/mapper\/cswap/" /etc/fstab
# systemctl restart cryptdisks
 ...
# swapon -a

Debian distributes modularized Linux kernel as packages for supported architectures.

If you are reading this documentation, you probably don't need to compile Linux kernel by yourself.

Debian has its own method of compiling the kernel and related modules.


If you use initrd in Section 3.1.2, “Stage 2: the boot loader”, make sure to read the related information in initramfs-tools(8), update-initramfs(8), mkinitramfs(8) and initramfs.conf(5).

[Warning] Warning

Do not put symlinks to the directories in the source tree (e.g. "/usr/src/linux*") from "/usr/include/linux" and "/usr/include/asm" when compiling the Linux kernel source. (Some outdated documents suggest this.)

[Note] Note

When compiling the latest Linux kernel on the Debian stable system, the use of backported latest tools from the Debian unstable may be needed.

module-assistant(8) (or its short form m-a) helps users to build and install module package(s) easily for one or more custom kernels.

The dynamic kernel module support (DKMS) is a new distribution independent framework designed to allow individual kernel modules to be upgraded without changing the whole kernel. This is used for the maintenance of out-of-tree modules. This also makes it very easy to rebuild modules as you upgrade kernels.

The hardware driver is the code running on the main CPUs of the target system. Most hardware drivers are available as free software now and are included in the normal Debian kernel packages in the main area.

  • GPU driver

    • Intel GPU driver (main)

    • AMD/ATI GPU driver (main)

    • NVIDIA GPU driver (main for nouveau driver, and non-free for binary-only drivers supported by the vendor.)

  • Softmodem driver

    • martian-modem and sl-modem-dkms packages (non-free)

The firmware is the code or data loaded on the device attach to the target system (e.g., CPU microcode, rendering code running on GPU, or FPGA / CPLD data, …). Some firmware packages are available as free software but many firmware packages are not available as free software since they contain sourceless binary data. Installing these firmware data is essential for the device to function as expected.

  • The firmware data packages containing data loaded to the volatile memory on the target device.

    • firmware-linux-free (main)

    • firmware-linux-nonfree (non-free)

    • firmware-linux-* (non-free)

    • *-firmware (non-free)

    • intel-microcode (non-free)

    • amd64-microcode (non-free)

  • The firmware update program packages which update data on the non-volatile memory on the target device.

    • fwupd (main): Firmware update daemon which downloads firmware data from Linux Vendor Firmware Service.

    • gnome-firmware (main): GTK front end for fwupd

    • plasma-discover-backend-fwupd (main): Qt front end for fwupd

Please note that non-free and contrib packages are not part of the Debian system. The access configuration to enable and to disable the non-free and contrib areas is described in Section 2.1.4, “Debian archive basics”. You should be aware of negatives associated with the use of the non-free and contrib packages as described in Section 2.1.5, “Debian is 100% free software”.

Please also note that the firmware data downloaded by fwupd from Linux Vendor Firmware Service and loaded to the running Linux kernel may be non-free.

Use of virtualized system enables us to run multiple instances of system simultaneously on a single hardware.

There are several virtualization and emulation tool platforms.

  • Complete hardware emulation packages such as ones installed by the games-emulator metapackage

  • Mostly CPU level emulation with some I/O device emulations such as QEMU

  • Mostly CPU level virtualization with some I/O device emulations such as Kernel-based Virtual Machine (KVM)

  • OS level container virtualization with the kernel level support such as LXC (Linux Containers), Docker, ...

  • OS level filesystem access virtualization with the system library call override on the file path such as chroot

  • OS level filesystem access virtualization with the system library call override on the file ownership such as fakeroot

  • OS API emulation such as Wine

  • Interpreter level virtualization with its executable selection and run-time library overrides such as virtualenv and venv for Python

The container virtualization uses Section 4.7.4, “Linux security features” and it is the backend technology of Section 7.6, “Sandbox”.

Here are some packages to help you to setup the virtualized system.

Table 9.27. List of virtualization tools

package popcon size description
schroot V:7, I:9 2708 specialized tool for executing Debian binary packages in chroot
sbuild V:1, I:4 271 tool for building Debian binary packages from Debian sources
debootstrap V:5, I:62 298 bootstrap a basic Debian system (written in sh)
cdebootstrap V:0, I:2 116 bootstrap a Debian system (written in C)
virt-manager V:10, I:43 2298 Virtual Machine Manager: desktop application for managing virtual machines
libvirt-clients V:46, I:64 1130 programs for the libvirt library
games-emulator I:0 26 games-emulator: Debian's emulators for games
bochs V:0, I:1 7194 Bochs: IA-32 PC emulator
qemu I:31 96 QEMU: fast generic processor emulator
qemu-system I:21 97 QEMU: full system emulation binaries
qemu-user V:0, I:11 89683 QEMU: user mode emulation binaries
qemu-utils V:11, I:108 6077 QEMU: utilities
qemu-kvm V:8, I:55 107 KVM: full virtualization on x86 hardware with the hardware-assisted virtualization
virtualbox V:11, I:14 107018 VirtualBox: x86 virtualization solution on i386 and amd64
xen-tools V:0, I:4 727 tools to manage debian XEN virtual server
wine V:16, I:76 191 Wine: Windows API Implementation (standard suite)
dosbox V:2, I:18 2702 DOSBox: x86 emulator with Tandy/Herc/CGA/EGA/VGA/SVGA graphics, sound and DOS
dosemu V:0, I:2 4891 DOSEMU: The Linux DOS Emulator
lxc V:11, I:14 18971 Linux containers user space tools
python3-venv V:2, I:50 6 venv for creating virtual python environments (system library)
python3-virtualenv V:9, I:64 410 virtualenv for creating isolated virtual python environments
python3-pipx I:0 NOT_FOUND pipx for installing python applications in isolated environments

See Wikipedia article Comparison of platform virtual machines for detail comparison of different platform virtualization solutions.

[Note] Note

Default Debian kernels support KVM since lenny.

Typical work flow for virtualization involves several steps.

For the raw disk image file, see Section 9.7, “The disk image”.

For other virtual disk image files, you can use qemu-nbd(8) to export them using network block device protocol and mount them using the nbd kernel module.

qemu-nbd(8) supports disk formats supported by QEMU: QEMU supports following disk formats: raw, qcow2, qcow, vmdk, vdi, bochs, cow (user-mode Linux copy-on-write), parallels, dmg, cloop, vpc, vvfat (virtual VFAT), and host_device.

The network block device can support partitions in the same way as the loop device (see Section 9.7.3, “Mounting the disk image file”). You can mount the first partition of "disk.img" as follows.

# modprobe nbd max_part=16
# qemu-nbd -v -c /dev/nbd0 disk.img
...
# mkdir /mnt/part1
# mount /dev/nbd0p1 /mnt/part1
[Tip] Tip

You may export only the first partition of "disk.img" using "-P 1" option to qemu-nbd(8).

If you wish to try a new Debian environment from a terminal console, I recommend you to use chroot. This enables you to run console applications of Debian unstable and testing without usual risks associated and without rebooting. chroot(8) is the most basic way.

[Caution] Caution

Examples below assumes both parent system and chroot system share the same amd64 CPU architecture.

Although you can manually create a chroot(8) environment using debootstrap(1). But this requires non-trivial efforts.

The sbuild package to build Debian packages from source uses the chroot environment managed by the schroot package. It comes with helper script sbuild-createchroot(1). Let's learn how it works by running it under script(1) as follows.

$ sudo mkdir -p /srv/chroot
$ sudo sbuild-createchroot -v --include=eatmydata,ccache unstable /srv/chroot/unstable-amd64-sbuild http://deb.debian.org/debian

You see how debootstrap(8) populates system data for unstable environment under "/srv/chroot/unstable-amd64-sbuild" for a minimal build system.

You can login to this environment using schroot(1).

$ sudo schroot -v -c chroot:unstable-amd64-sbuild

You see how a system shell running under unstable environment is created.

[Note] Note

The "/usr/sbin/policy-rc.d" file which always exits with 101 prevents daemon programs to be started automatically on the Debian system. See "/usr/share/doc/sysv-rc/README.policy-rc.d.gz".

[Note] Note

Some programs under chroot may require access to more files from the parent system to function than sbuild-createchroot provides as above. For example, "/sys", "/etc/passwd", "/etc/group", "/var/run/utmp", "/var/log/wtmp", etc. may need to be bind-mounted or copied.

[Tip] Tip

The sbuild package helps to construct a chroot system and builds a package inside the chroot using schroot as its backend. It is an ideal system to check build-dependencies. See more on sbuild at Debian wiki and sbuild configuration example in "Guide for Debian Maintainers".

If you wish to try a new GUI Desktop environment of any OS, I recommend you to use QEMU, KVM, or VirtualBox on a Debian stable system to run multiple desktop systems safely using virtualization. These enable you to run any desktop applications including ones of Debian unstable and testing without usual risks associated with them and without rebooting. The configuration of these tools are relatively straight forward.

Since pure QEMU is very slow, it is recommended to accelerate it with KVM when the host system supports it.

The virtual disk image "virtdisk.qcow2" containing a Debian system for QEMU can be created using debian-installer: Small CDs as follows.

$ wget http://cdimage.debian.org/debian-cd/5.0.3/amd64/iso-cd/debian-503-amd64-netinst.iso
$ qemu-img create -f qcow2 virtdisk.qcow2 5G
$ qemu -hda virtdisk.qcow2 -cdrom debian-503-amd64-netinst.iso -boot d -m 256
...

See more tips at Debian wiki: QEMU.

VirtualBox comes with Qt GUI tools and quite intuitive. Its GUI and command line tools are explained in VirtualBox User Manual and VirtualBox User Manual (PDF).

[Tip] Tip

Running other GNU/Linux distributions such as Ubuntu and Fedora under virtualization is a great way to learn configuration tips. Other proprietary OSs may be run nicely under this GNU/Linux virtualization, too.



[2] More elaborate customization examples: "Vim Galore", "sensible.vim", "#vim Recommendations" ...

[3] vim-pathogen was popular.