Debian Reference

Osamu Aoki

Abstract

This book is free; you may redistribute it and/or modify it under the terms of the GNU General Public License of any version compliant to the Debian Free Software Guidelines (DFSG).


Table of Contents

Preface
1. Disclaimer
2. What is Debian
3. About this document
3.1. Guiding rules
3.2. Prerequisites
3.3. Conventions
3.4. The popcon
3.5. The package size
3.6. Bug reports on this document
4. Reminders for new users
5. Some quotes for new users
1. GNU/Linux tutorials
1.1. Console basics
1.1.1. The shell prompt
1.1.2. The shell prompt under GUI
1.1.3. The root account
1.1.4. The root shell prompt
1.1.5. GUI system administration tools
1.1.6. Virtual consoles
1.1.7. How to leave the command prompt
1.1.8. How to shutdown the system
1.1.9. Recovering a sane console
1.1.10. Additional package suggestions for the newbie
1.1.11. An extra user account
1.1.12. sudo configuration
1.1.13. Play time
1.2. Unix-like filesystem
1.2.1. Unix file basics
1.2.2. Filesystem internals
1.2.3. Filesystem permissions
1.2.4. Control of permissions for newly created files: umask
1.2.5. Permissions for groups of users (group)
1.2.6. Timestamps
1.2.7. Links
1.2.8. Named pipes (FIFOs)
1.2.9. Sockets
1.2.10. Device files
1.2.11. Special device files
1.2.12. procfs and sysfs
1.2.13. tmpfs
1.3. Midnight Commander (MC)
1.3.1. Customization of MC
1.3.2. Starting MC
1.3.3. File manager in MC
1.3.4. Command-line tricks in MC
1.3.5. The internal editor in MC
1.3.6. The internal viewer in MC
1.3.7. Auto-start features of MC
1.3.8. Virtual filesystem of MC
1.4. The basic Unix-like work environment
1.4.1. The login shell
1.4.2. Customizing bash
1.4.3. Special key strokes
1.4.4. Mouse operations
1.4.5. The pager
1.4.6. The text editor
1.4.7. Setting a default text editor
1.4.8. Using vim
1.4.9. Recording the shell activities
1.4.10. Basic Unix commands
1.5. The simple shell command
1.5.1. Command execution and environment variable
1.5.2. The "$LANG" variable
1.5.3. The "$PATH" variable
1.5.4. The "$HOME" variable
1.5.5. Command line options
1.5.6. Shell glob
1.5.7. Return value of the command
1.5.8. Typical command sequences and shell redirection
1.5.9. Command alias
1.6. Unix-like text processing
1.6.1. Unix text tools
1.6.2. Regular expressions
1.6.3. Replacement expressions
1.6.4. Global substitution with regular expressions
1.6.5. Extracting data from text file table
1.6.6. Script snippets for piping commands
2. Debian package management
2.1. Debian package management prerequisites
2.1.1. Debian package management system
2.1.2. Package configuration
2.1.3. Basic precautions
2.1.4. Life with eternal upgrades
2.1.5. Debian archive basics
2.1.6. Debian is 100% free software
2.1.7. Package dependencies
2.1.8. The event flow of the package management
2.1.9. First response to package management troubles
2.1.10. How to pick Debian packages
2.1.11. How to cope with conflicting requirements
2.2. Basic package management operations
2.2.1. apt vs. apt-get / apt-cache vs. aptitude
2.2.2. Basic package management operations with the commandline
2.2.3. Interactive use of aptitude
2.2.4. Key bindings of aptitude
2.2.5. Package views under aptitude
2.2.6. Search method options with aptitude
2.2.7. The aptitude regex formula
2.2.8. Dependency resolution of aptitude
2.2.9. Package activity logs
2.3. Examples of aptitude operations
2.3.1. Seeking interesting packages
2.3.2. Listing packages with regex matching on package names
2.3.3. Browsing with the regex matching
2.3.4. Purging removed packages for good
2.3.5. Tidying auto/manual install status
2.3.6. System wide upgrade
2.4. Advanced package management operations
2.4.1. Advanced package management operations with commandline
2.4.2. Verification of installed package files
2.4.3. Safeguarding for package problems
2.4.4. Searching on the package meta data
2.5. Debian package management internals
2.5.1. Archive meta data
2.5.2. Top level "Release" file and authenticity
2.5.3. Archive level "Release" files
2.5.4. Fetching of the meta data for the package
2.5.5. The package state for APT
2.5.6. The package state for aptitude
2.5.7. Local copies of the fetched packages
2.5.8. Debian package file names
2.5.9. The dpkg command
2.5.10. The update-alternatives command
2.5.11. The dpkg-statoverride command
2.5.12. The dpkg-divert command
2.6. Recovery from a broken system
2.6.1. Failed installation due to missing dependencies
2.6.2. Caching errors of the package data
2.6.3. Incompatibility with old user configuration
2.6.4. Different packages with overlapped files
2.6.5. Fixing broken package script
2.6.6. Rescue with the dpkg command
2.6.7. Recovering package selection data
2.7. Tips for the package management
2.7.1. Who uploaded the package?
2.7.2. Limiting download bandwidth for APT
2.7.3. Automatic download and upgrade of packages
2.7.4. Updates and Backports
2.7.5. External package archives
2.7.6. Packages from mixed source of archives without apt-pinning
2.7.7. Tweaking candidate version with apt-pinning
2.7.8. Blocking packages installed by "Recommends"
2.7.9. Tracking testing with some packages from unstable
2.7.10. Tracking unstable with some packages from experimental
2.7.11. Emergency downgrading
2.7.12. The equivs package
2.7.13. Porting a package to the stable system
2.7.14. Proxy server for APT
2.7.15. More readings for the package management
3. The system initialization
3.1. An overview of the boot strap process
3.1.1. Stage 1: the UEFI
3.1.2. Stage 2: the boot loader
3.1.3. Stage 3: the mini-Debian system
3.1.4. Stage 4: the normal Debian system
3.2. Systemd
3.2.1. Systemd init
3.2.2. Systemd login
3.3. The kernel message
3.4. The system message
3.5. System management
3.6. Other system monitors
3.7. System configuration
3.7.1. The hostname
3.7.2. The filesystem
3.7.3. Network interface initialization
3.7.4. Cloud system initialization
3.7.5. Customization example to tweak sshd service
3.8. The udev system
3.9. The kernel module initialization
4. Authentication and access controls
4.1. Normal Unix authentication
4.2. Managing account and password information
4.3. Good password
4.4. Creating encrypted password
4.5. PAM and NSS
4.5.1. Configuration files accessed by PAM and NSS
4.5.2. The modern centralized system management
4.5.3. "Why GNU su does not support the wheel group"
4.5.4. Stricter password rule
4.6. Security of authentication
4.6.1. Secure password on the Internet
4.6.2. Secure Shell
4.6.3. Extra security measures for the Internet
4.6.4. Securing the root password
4.7. Other access controls
4.7.1. Access control lists (ACLs)
4.7.2. sudo
4.7.3. PolicyKit
4.7.4. Restricting access to some server services
4.7.5. Linux security features
5. Network setup
5.1. The basic network infrastructure
5.1.1. The hostname resolution
5.1.2. The network interface name
5.1.3. The network address range for the LAN
5.1.4. The network device support
5.2. The modern network configuration for desktop
5.2.1. GUI network configuration tools
5.3. The modern network configuration without GUI
5.4. The modern network configuration for cloud
5.4.1. The modern network configuration for cloud with DHCP
5.4.2. The modern network configuration for cloud with static IP
5.4.3. The modern network configuration for cloud with Network Manger
5.5. The low level network configuration
5.5.1. Iproute2 commands
5.5.2. Safe low level network operations
5.6. Network optimization
5.6.1. Finding optimal MTU
5.6.2. WAN TCP optimization
5.7. Netfilter infrastructure
6. Network applications
6.1. Web browsers
6.1.1. Spoofing the User-Agent string
6.1.2. Browser extension
6.2. The mail system
6.2.1. Email basics
6.2.2. Modern mail service limitation
6.2.3. Historic mail service expectation
6.2.4. Mail transport agent (MTA)
6.2.4.1. The configuration of exim4
6.2.4.2. The configuration of postfix with SASL
6.2.4.3. The mail address configuration
6.2.4.4. Basic MTA operations
6.3. The remote access server and utilities (SSH)
6.3.1. Basics of SSH
6.3.2. User name on the remote host
6.3.3. Connecting without remote passwords
6.3.4. Dealing with alien SSH clients
6.3.5. Setting up ssh-agent
6.3.6. Sending a mail from a remote host
6.3.7. Port forwarding for SMTP/POP3 tunneling
6.3.8. How to shutdown the remote system on SSH
6.3.9. Troubleshooting SSH
6.4. The print server and utilities
6.5. Other network application servers
6.6. Other network application clients
6.7. The diagnosis of the system daemons
7. GUI System
7.1. GUI desktop environment
7.2. GUI communication protocol
7.3. GUI infrastructure
7.4. GUI applications
7.5. Fonts
7.5.1. Basic fonts
7.5.2. Font rasterization
7.6. Sandbox
7.7. Remote desktop
7.8. X server connection
7.8.1. X server local connection
7.8.2. X server remote connection
7.8.3. X server chroot connection
7.9. Clipboard
8. I18N and L10N
8.1. The locale
8.1.1. Rationale for UTF-8 locale
8.1.2. The reconfiguration of the locale
8.1.3. Filename encoding
8.1.4. Localized messages and translated documentation
8.1.5. Effects of the locale
8.2. The keyboard input
8.2.1. The keyboard input for Linux console and X Window
8.2.2. The keyboard input for Wayland
8.2.3. The input method support with IBus
8.2.4. An example for Japanese
8.3. The display output
8.4. East Asian Ambiguous Character Width Characters
9. System tips
9.1. The console tips
9.1.1. Recording the shell activities cleanly
9.1.2. The screen program
9.1.3. Navigating around directories
9.1.4. Readline wrapper
9.1.5. Scanning the source code tree
9.2. Customizing vim
9.2.1. Customizing vim with internal features
9.2.2. Customizing vim with external packages
9.3. Data recording and presentation
9.3.1. The log daemon
9.3.2. Log analyzer
9.3.3. Customized display of text data
9.3.4. Customized display of time and date
9.3.5. Colorized shell echo
9.3.6. Colorized commands
9.3.7. Recording the editor activities for complex repeats
9.3.8. Recording the graphics image of an X application
9.3.9. Recording changes in configuration files
9.4. Monitoring, controlling, and starting program activities
9.4.1. Timing a process
9.4.2. The scheduling priority
9.4.3. The ps command
9.4.4. The top command
9.4.5. Listing files opened by a process
9.4.6. Tracing program activities
9.4.7. Identification of processes using files or sockets
9.4.8. Repeating a command with a constant interval
9.4.9. Repeating a command looping over files
9.4.10. Starting a program from GUI
9.4.11. Customizing program to be started
9.4.12. Killing a process
9.4.13. Scheduling tasks once
9.4.14. Scheduling tasks regularly
9.4.15. Scheduling tasks on event
9.4.16. Alt-SysRq key
9.5. System maintenance tips
9.5.1. Who is on the system?
9.5.2. Warning everyone
9.5.3. Hardware identification
9.5.4. Hardware configuration
9.5.5. System and hardware time
9.5.6. The terminal configuration
9.5.7. The sound infrastructure
9.5.8. Disabling the screen saver
9.5.9. Disabling beep sounds
9.5.10. Memory usage
9.5.11. System security and integrity check
9.6. Data storage tips
9.6.1. Disk space usage
9.6.2. Disk partition configuration
9.6.3. Accessing partition using UUID
9.6.4. LVM2
9.6.5. Filesystem configuration
9.6.6. Filesystem creation and integrity check
9.6.7. Optimization of filesystem by mount options
9.6.8. Optimization of filesystem via superblock
9.6.9. Optimization of hard disk
9.6.10. Optimization of solid state drive
9.6.11. Using SMART to predict hard disk failure
9.6.12. Specify temporary storage directory via $TMPDIR
9.6.13. Expansion of usable storage space via LVM
9.6.14. Expansion of usable storage space by mounting another partition
9.6.15. Expansion of usable storage space by bind-mounting another directory
9.6.16. Expansion of usable storage space by overlay-mounting another directory
9.6.17. Expansion of usable storage space using symlink
9.7. The disk image
9.7.1. Making the disk image file
9.7.2. Writing directly to the disk
9.7.3. Mounting the disk image file
9.7.4. Cleaning a disk image file
9.7.5. Making the empty disk image file
9.7.6. Making the ISO9660 image file
9.7.7. Writing directly to the CD/DVD-R/RW
9.7.8. Mounting the ISO9660 image file
9.8. The binary data
9.8.1. Viewing and editing binary data
9.8.2. Manipulating files without mounting disk
9.8.3. Data redundancy
9.8.4. Data file recovery and forensic analysis
9.8.5. Splitting a large file into small files
9.8.6. Clearing file contents
9.8.7. Dummy files
9.8.8. Erasing an entire hard disk
9.8.9. Erasing unused area of an hard disk
9.8.10. Undeleting deleted but still open files
9.8.11. Searching all hardlinks
9.8.12. Invisible disk space consumption
9.9. Data encryption tips
9.9.1. Removable disk encryption with dm-crypt/LUKS
9.9.2. Mounting encrypted disk with dm-crypt/LUKS
9.10. The kernel
9.10.1. Kernel parameters
9.10.2. Kernel headers
9.10.3. Compiling the kernel and related modules
9.10.4. Compiling the kernel source: Debian Kernel Team recommendation
9.10.5. Hardware drivers and firmware
9.11. Virtualized system
9.11.1. Virtualization and emulation tools
9.11.2. Virtualization work flow
9.11.3. Mounting the virtual disk image file
9.11.4. Chroot system
9.11.5. Multiple desktop systems
10. Data management
10.1. Sharing, copying, and archiving
10.1.1. Archive and compression tools
10.1.2. Copy and synchronization tools
10.1.3. Idioms for the archive
10.1.4. Idioms for the copy
10.1.5. Idioms for the selection of files
10.1.6. Archive media
10.1.7. Removable storage device
10.1.8. Filesystem choice for sharing data
10.1.9. Sharing data via network
10.2. Backup and recovery
10.2.1. Backup and recovery policy
10.2.2. Backup utility suites
10.2.3. Backup tips
10.2.3.1. GUI backup
10.2.3.2. Mount event triggered backup
10.2.3.3. Timer event triggered backup
10.3. Data security infrastructure
10.3.1. Key management for GnuPG
10.3.2. Using GnuPG on files
10.3.3. Using GnuPG with Mutt
10.3.4. Using GnuPG with Vim
10.3.5. The MD5 sum
10.3.6. Password keyring
10.4. Source code merge tools
10.4.1. Extracting differences for source files
10.4.2. Merging updates for source files
10.4.3. Interactive merge
10.5. Git
10.5.1. Configuration of Git client
10.5.2. Basic Git commands
10.5.3. Git tips
10.5.4. Git references
10.5.5. Other version control systems
11. Data conversion
11.1. Text data conversion tools
11.1.1. Converting a text file with iconv
11.1.2. Checking file to be UTF-8 with iconv
11.1.3. Converting file names with iconv
11.1.4. EOL conversion
11.1.5. TAB conversion
11.1.6. Editors with auto-conversion
11.1.7. Plain text extraction
11.1.8. Highlighting and formatting plain text data
11.2. XML data
11.2.1. Basic hints for XML
11.2.2. XML processing
11.2.3. The XML data extraction
11.2.4. The XML data lint
11.3. Type setting
11.3.1. roff typesetting
11.3.2. TeX/LaTeX
11.3.3. Pretty print a manual page
11.3.4. Creating a manual page
11.4. Printable data
11.4.1. Ghostscript
11.4.2. Merge two PS or PDF files
11.4.3. Printable data utilities
11.4.4. Printing with CUPS
11.5. The mail data conversion
11.5.1. Mail data basics
11.6. Graphic data tools
11.6.1. Graphic data tools (metapackage)
11.6.2. Graphic data tools (GUI)
11.6.3. Graphic data tools (CLI)
11.7. Miscellaneous data conversion
12. Programming
12.1. The shell script
12.1.1. POSIX shell compatibility
12.1.2. Shell parameters
12.1.3. Shell conditionals
12.1.4. Shell loops
12.1.5. Shell environment variables
12.1.6. The shell command-line processing sequence
12.1.7. Utility programs for shell script
12.2. Scripting in interpreted languages
12.2.1. Debugging interpreted language codes
12.2.2. GUI program with the shell script
12.2.3. Custom actions for GUI filer
12.2.4. Perl short script madness
12.3. Coding in compiled languages
12.3.1. C
12.3.2. Simple C program (gcc)
12.3.3. Flex — a better Lex
12.3.4. Bison — a better Yacc
12.4. Static code analysis tools
12.5. Debug
12.5.1. Basic gdb execution
12.5.2. Debugging the Debian package
12.5.3. Obtaining backtrace
12.5.4. Advanced gdb commands
12.5.5. Check dependency on libraries
12.5.6. Dynamic call tracing tools
12.5.7. Debugging X Errors
12.5.8. Memory leak detection tools
12.5.9. Disassemble binary
12.6. Build tools
12.6.1. Make
12.6.2. Autotools
12.6.2.1. Compile and install a program
12.6.2.2. Uninstall program
12.6.3. Meson
12.7. Web
12.8. The source code translation
12.9. Making Debian package
A. Appendix
A.1. The Debian maze
A.2. Copyright history
A.3. Document format

List of Tables

1.1. List of interesting text-mode program packages
1.2. List of informative documentation packages
1.3. List of usage of key directories
1.4. List of the first character of "ls -l" output
1.5. The numeric mode for file permissions in chmod(1) commands
1.6. The umask value examples
1.7. List of notable system-provided groups for file access
1.8. List of notable system provided groups for particular command executions
1.9. List of types of timestamps
1.10. List of special device files
1.11. The key bindings of MC
1.12. The reaction to the enter key in MC
1.13. List of shell programs
1.14. List of key bindings for bash
1.15. List of mouse operations and related key actions on Debian
1.16. List of basic Vim key strokes
1.17. List of basic Unix commands
1.18. The 3 parts of locale value
1.19. List of locale recommendations
1.20. List of "$HOME" values
1.21. Shell glob patterns
1.22. Command exit codes
1.23. Shell command idioms
1.24. Predefined file descriptors
1.25. Metacharacters for BRE and ERE
1.26. The replacement expression
1.27. List of script snippets for piping commands
2.1. List of Debian package management tools
2.2. List of Debian archive sites
2.3. List of Debian archive area
2.4. The relationship between suite and codename
2.5. List of key web site to resolving problems with a specific package
2.6. Basic package management operations with the commandline using apt(8), aptitude(8) and apt-get(8) /apt-cache(8)
2.7. Notable command options for aptitude(8)
2.8. List of key bindings for aptitude
2.9. List of views for aptitude
2.10. The categorization of standard package views
2.11. List of the aptitude regex formula
2.12. The log files for package activities
2.13. List of advanced package management operations
2.14. The content of the Debian archive meta data
2.15. The name structure of Debian packages
2.16. The usable characters for each component in the Debian package names
2.17. The notable files created by dpkg
2.18. List of notable Pin-Priority values for apt-pinning technique.
2.19. List of the proxy tools specially for Debian archive
3.1. List of boot loaders
3.2. The meaning of the menu entry of the above part of /boot/grub/grub.cfg
3.3. List of boot utilities for the Debian system
3.4. List of kernel error levels
3.5. List of typical journalctl command snippets
3.6. List of typical systemctl command snippets
3.7. List of other monitoring command snippets under systemd
4.1. 3 important configuration files for pam_unix(8)
4.2. The second entry content of "/etc/passwd"
4.3. List of commands to manage account information
4.4. List of tools to generate password
4.5. List of notable PAM and NSS systems
4.6. List of configuration files accessed by PAM and NSS
4.7. List of insecure and secure services and ports
4.8. List of tools to provide extra security measures
5.1. List of network configuration tools
5.2. List of network address ranges
5.3. Translation table from obsolete net-tools commands to new iproute2 commands
5.4. List of low level network commands
5.5. List of network optimization tools
5.6. Basic guide lines of the optimal MTU value
5.7. List of firewall tools
6.1. List of web browsers
6.2. List of mail user agent (MUA)
6.3. List of basic mail transport agent related packages
6.4. List of important postfix manual pages
6.5. List of mail address related configuration files
6.6. List of basic MTA operation
6.7. List of remote access server and utilities
6.8. List of SSH configuration files
6.9. List of SSH client startup examples
6.10. List of free SSH clients for other platforms
6.11. List of print servers and utilities
6.12. List of other network application servers
6.13. List of network application clients
6.14. List of popular RFCs
7.1. List of desktop environment
7.2. List of notable GUI infrastructure packages
7.3. List of notable GUI applications
7.4. List of notable TrueType and OpenType fonts
7.5. List of notable font environment and related packages
7.6. List of notable sandbox environment and related packages
7.7. List of notable remote access server
7.8. List of connection methods to the X server
7.9. List of programs related to manipulating character clipboard
8.1. List of IBus and its engine packages
9.1. List of programs to support console activities
9.2. List of key bindings for screen
9.3. Information on the initialization of vim
9.4. List of system log analyzers
9.5. Display examples of time and date for the "ls -l" command with the time style value
9.6. List of graphics image manipulation tools
9.7. List of packages which can record configuration history
9.8. List of tools for monitoring and controlling program activities
9.9. List of nice values for the scheduling priority
9.10. List of ps command styles
9.11. List of frequently used signals for kill command
9.12. List of notable SAK command keys
9.13. List of hardware identification tools
9.14. List of hardware configuration tools
9.15. List of sound packages
9.16. List of commands for disabling the screen saver
9.17. List of memory sizes reported
9.18. List of tools for system security and integrity check
9.19. List of disk partition management packages
9.20. List of filesystem management packages
9.21. List of packages which view and edit binary data
9.22. List of packages to manipulate files without mounting disk
9.23. List of tools to add data redundancy to files
9.24. List of packages for data file recovery and forensic analysis
9.25. List of data encryption utilities
9.26. List of key packages to be installed for the kernel recompilation on the Debian system
9.27. List of virtualization tools
10.1. List of archive and compression tools
10.2. List of copy and synchronization tools
10.3. List of filesystem choices for removable storage devices with typical usage scenarios
10.4. List of the network service to chose with the typical usage scenario
10.5. List of backup suite utilities
10.6. List of data security infrastructure tools
10.7. List of GNU Privacy Guard commands for the key management
10.8. List of the meaning of the trust code
10.9. List of GNU Privacy Guard commands on files
10.10. List of source code merge tools
10.11. List of git related packages and commands
10.12. Main Git commands
10.13. Git tips
10.14. List of other version control system tools
11.1. List of text data conversion tools
11.2. List of encoding values and their usage
11.3. List of EOL styles for different platforms
11.4. List of TAB conversion commands from bsdmainutils and coreutils packages
11.5. List of tools to extract plain text data
11.6. List of tools to highlight plain text data
11.7. List of predefined entities for XML
11.8. List of XML tools
11.9. List of DSSSL tools
11.10. List of XML data extraction tools
11.11. List of XML pretty print tools
11.12. List of type setting tools
11.13. List of packages to help creating the manpage
11.14. List of Ghostscript PostScript interpreters
11.15. List of printable data utilities
11.16. List of packages to help mail data conversion
11.17. List of graphics data tools (metapackage)
11.18. List of graphics data tools (GUI)
11.19. List of graphics data tools (CLI)
11.20. List of miscellaneous data conversion tools
12.1. List of typical bashisms
12.2. List of shell parameters
12.3. List of shell parameter expansions
12.4. List of key shell parameter substitutions
12.5. List of file comparison operators in the conditional expression
12.6. List of string comparison operators in the conditional expression
12.7. List of packages containing small utility programs for shell scripts
12.8. List of interpreter related packages
12.9. List of dialog programs
12.10. List of compiler related packages
12.11. List of Yacc-compatible LALR parser generators
12.12. List of tools for static code analysis
12.13. List of debug packages
12.14. List of advanced gdb commands
12.15. List of memory leak detection tools
12.16. List of build tool packages
12.17. List of make automatic variables
12.18. List of make variable expansions
12.19. List of source code translation tools