Chapter 2. Debian package management

Table of Contents

2.1. Debian package management prerequisites
2.1.1. Package configuration
2.1.2. Basic precautions
2.1.3. Life with eternal upgrades
2.1.4. Debian archive basics
2.1.5. Debian is 100% free software
2.1.6. Package dependencies
2.1.7. The event flow of the package management
2.1.8. First response to package management troubles
2.2. Basic package management operations
2.2.1. apt vs. apt-get / apt-cache vs. aptitude
2.2.2. Basic package management operations with the commandline
2.2.3. Interactive use of aptitude
2.2.4. Key bindings of aptitude
2.2.5. Package views under aptitude
2.2.6. Search method options with aptitude
2.2.7. The aptitude regex formula
2.2.8. Dependency resolution of aptitude
2.2.9. Package activity logs
2.3. Examples of aptitude operations
2.3.1. Listing packages with regex matching on package names
2.3.2. Browsing with the regex matching
2.3.3. Purging removed packages for good
2.3.4. Tidying auto/manual install status
2.3.5. System wide upgrade
2.4. Advanced package management operations
2.4.1. Advanced package management operations with commandline
2.4.2. Verification of installed package files
2.4.3. Safeguarding for package problems
2.4.4. Searching on the package meta data
2.5. Debian package management internals
2.5.1. Archive meta data
2.5.2. Top level "Release" file and authenticity
2.5.3. Archive level "Release" files
2.5.4. Fetching of the meta data for the package
2.5.5. The package state for APT
2.5.6. The package state for aptitude
2.5.7. Local copies of the fetched packages
2.5.8. Debian package file names
2.5.9. The dpkg command
2.5.10. The update-alternatives command
2.5.11. The dpkg-statoverride command
2.5.12. The dpkg-divert command
2.6. Recovery from a broken system
2.6.1. Failed installation due to missing dependencies
2.6.2. Caching errors of the package data
2.6.3. Incompatibility with old user configuration
2.6.4. Different packages with overlapped files
2.6.5. Fixing broken package script
2.6.6. Rescue with the dpkg command
2.6.7. Recovering package selection data
2.7. Tips for the package management
2.7.1. How to pick Debian packages
2.7.2. Packages from mixed source of archives
2.7.3. Tweaking candidate version
2.7.4. Updates and Backports
2.7.5. Blocking packages installed by "Recommends"
2.7.6. Tracking testing with some packages from unstable
2.7.7. Tracking unstable with some packages from experimental
2.7.8. Automatic download and upgrade of packages
2.7.9. Limiting download bandwidth for APT
2.7.10. Emergency downgrading
2.7.11. Who uploaded the package?
2.7.12. The equivs package
2.7.13. Porting a package to the stable system
2.7.14. Proxy server for APT
2.7.15. More readings for the package management
[Note] Note

This chapter is written assuming the latest stable release is codename: bullseye.

Debian is a volunteer organization which builds consistent distributions of pre-compiled binary packages of free software and distributes them from its archive.

The Debian archive is offered by many remote mirror sites for access through HTTP and FTP methods. It is also available as CD-ROM/DVD.

The current Debian package management system which can utilize all these resources is Advanced Packaging Tool (APT).

The Debian package management system, when used properly, offers the user to install consistent sets of binary packages to the system from the archive. Currently, there are 68980 packages available for the amd64 architecture.

The Debian package management system has a rich history and many choices for the front end user program and back end archive access method to be used. Currently, we recommend the following.

Table 2.1. List of Debian package management tools

package popcon size description
dpkg V:919, I:999 5989 low level package management system for Debian (file based)
apt V:874, I:999 4211 APT front-end to manage packages with CLI: apt/apt-get/apt-cache
aptitude V:60, I:321 4268 APT front-end to interactively manage packages with full screen console: aptitude(8)
tasksel V:33, I:978 346 APT front-end to install selected tasks: tasksel(8)
unattended-upgrades V:297, I:462 301 enhancement package for APT to enable automatic installation of security upgrades
gnome-software V:131, I:232 3007 Software Center for GNOME (GUI APT front-end)
synaptic V:42, I:351 7686 graphical package manager (GTK APT front-end)
apt-utils V:351, I:998 1030 APT utility programs: apt-extracttemplates(1), apt-ftparchive(1), and apt-sortpkgs(1)
apt-listchanges V:329, I:856 423 package change history notification tool
apt-listbugs V:6, I:10 475 lists critical bugs before each APT installation
apt-file V:17, I:74 89 APT package searching utility — command-line interface
apt-rdepends V:0, I:6 39 recursively lists package dependencies

[Warning] Warning

Do not install packages from random mixture of suites. It probably breaks the package consistency which requires deep system management knowledge, such as compiler ABI, library version, interpreter features, etc.

The newbie Debian system administrator should stay with the stable release of Debian while applying only security updates. I mean that some of the following valid actions are better avoided, as a precaution, until you understand the Debian system very well. Here are some reminders.

  • Do not include testing or unstable in "/etc/apt/sources.list".

  • Do not mix standard Debian with other non-Debian archives such as Ubuntu in "/etc/apt/sources.list".

  • Do not create "/etc/apt/preferences".

  • Do not change default behavior of package management tools through configuration files without knowing their full impacts.

  • Do not install random packages by "dpkg -i random_package".

  • Do not ever install random packages by "dpkg --force-all -i random_package".

  • Do not erase or alter files in "/var/lib/dpkg/".

  • Do not overwrite system files by installing software programs directly compiled from source.

    • Install them into "/usr/local" or "/opt", if needed.

The non-compatible effects caused by above actions to the Debian package management system may leave your system unusable.

The serious Debian system administrator who runs mission critical servers, should use extra precautions.

  • Do not install any packages including security updates from Debian without thoroughly testing them with your particular configuration under safe conditions.

    • You as the system administrator are responsible for your system in the end.

    • The long stability history of the Debian system is no guarantee by itself.

[Caution] Caution

For your production server, the stable suite with the security updates is recommended. The same can be said for desktop PCs on which you can spend limited administration efforts.

Despite my warnings above, I know many readers of this document may wish to run the newer testing or unstable suites.

Enlightenment with the following saves a person from the eternal karmic struggle of upgrade hell and let him reach Debian nirvana.

This list is targeted for the self-administered Desktop environment.

  • Use the testing suite since it is practically the rolling release automatically managed by the Debian archive QA infrastructure such as the Debian continuous integration, the source only upload practices, and the library transition tracking. The packages in the testing suite are updated frequently enough to offer all the latest features.

  • Set the codename corresponding to the testing suite (currently "bookworm") in the "/etc/apt/sources.list".

  • Manually update this codename in the "/etc/apt/sources.list" to the new one only after assessing situation by yourself for about a month after the major suite release. The Debian user and developer mailing list are good sources of information for this, too.

The use of the unstable suite isn't recommended. The unstable suite is good for debugging packages as a developer but tends to expose you to unnecessary risks for the normal Desktop usage. Even though the unstable suite of the Debian system looks very stable for most of the times, there have been some package problems and a few of them were not so trivial to resolve.

Here are some basic precautionary measure ideas to ensure quick and easy recovery from bugs in Debian packages.

  • Make the system dual bootable by installing the stable suite of the Debian system to another partition

  • Make the installation CD handy for the rescue boot

  • Consider installing apt-listbugs to check the Debian Bug Tracking System (BTS) information before the upgrade

  • Learn the package system infrastructure enough to work around the problem

  • Install a corresponding sandboxed upstream binary package in case of trouble (see Section 7.6, “Sandbox”)

  • Create a chroot or similar environment and run the latest system in it in advance (see Section 9.11, “Virtualized system”)

[Caution] Caution

If you can not do any one of these precautionary actions, you are probably not ready for the testing and unstable suites.

Let's look into the Debian archive from a system user's perspective.

[Tip] Tip

Official policy of the Debian archive is defined at Debian Policy Manual, Chapter 2 - The Debian Archive.

For the typical HTTP access, the archive is specified in the "/etc/apt/sources.list" file as the following, e.g. for the current stable = bullseye system.

deb bullseye main non-free-firmware contrib non-free
deb-src bullseye main non-free-firmware contrib non-free

deb bullseye-security main non-free-firmware contrib non-free
deb-src bullseye-security main non-free-firmware contrib non-free

Here, I tend to use codename "bullseye" instead of suite name "stable" to avoid surprises when the next stable is released.

The meaning of "/etc/apt/sources.list" is described in sources.list(5) and key points are followings.

  • The "deb" line defines for the binary packages.

  • The "deb-src" line defines for the source packages.

  • The 1st argument is the root URL of the Debian archive.

  • The 2nd argument is the distribution name: either the suite name or the codename.

  • The 3rd and following arguments are the list of valid archive area names of the Debian archive.

The "deb-src" lines can safely be omitted (or commented out by placing "#" at the start of the line) if it is just for aptitude which does not access source related meta data. It speeds up the updates of the archive meta data. The URL can be "http://", "ftp://", "file://", ….

[Tip] Tip

If "sid" is used in the above example instead of "bullseye", the "deb: …" line for security updates in the "/etc/apt/sources.list" is not required. This is because there is no security update archive for "sid" (unstable).

Here is the list of URL of the Debian archive sites and suite name or codename used in the configuration file.

[Caution] Caution

Only pure stable release with security updates provides the best stability. Running mostly stable release mixed with some packages from testing or unstable release is riskier than running pure unstable release for library version mismatch etc. If you really need the latest version of some programs under stable release, please use packages from bullseye-updates and (see Section 2.7.4, “Updates and Backports”) services. These services must be used with extra care.

[Caution] Caution

You should basically list only one of stable, testing, or unstable suites in the "deb" line. If you list any combination of stable, testing, and unstable suites in the "deb" line, APT programs slow down while only the latest archive is effective. Multiple listing makes sense for these when the "/etc/apt/preferences" file is used with clear objectives (see Section 2.7.3, “Tweaking candidate version”).

[Tip] Tip

For the Debian system with the stable suite, it is a good idea to include lines with "" in the "/etc/apt/sources.list" to enable security updates as in the example above.

[Note] Note

The security bugs for the stable archive are fixed by the Debian security team. This activity has been quite rigorous and reliable. Those for the testing archive may be fixed by the Debian testing security team. For several reasons, this activity is not as rigorous as that for stable and you may need to wait for the migration of fixed unstable packages. Those for the unstable archive are fixed by the individual maintainer. Actively maintained unstable packages are usually in a fairly good shape by leveraging latest upstream security fixes. See Debian security FAQ for how Debian handles security bugs.

Here the number of packages in the above is for the amd64 architecture. The main area provides the Debian system (see Section 2.1.5, “Debian is 100% free software”).

The Debian archive organization can be studied best by pointing your browser to the each archive URL appended with dists or pool.

The distribution is referred by two ways, the suite or codename. The word distribution is alternatively used as the synonym to the suite in many documentations. The relationship between the suite and the codename can be summarized as the following.

The history of codenames are described in Debian FAQ: 6.2.1 Which other codenames have been used in the past?

In the stricter Debian archive terminology, the word "section" is specifically used for the categorization of packages by the application area. (Although, the word "main section" may sometimes be used to describe the Debian archive area named as "main".)

Every time a new upload is done by a Debian developer (DD) to the unstable archive (via incoming processing), the DD is required to ensure uploaded packages to be compatible with the latest set of packages in the latest unstable archive.

If DD breaks this compatibility intentionally for important library upgrade etc, there is usually announcement to the debian-devel mailing list etc.

Before a set of packages are moved by the Debian archive maintenance script from the unstable archive to the testing archive, the archive maintenance script not only checks the maturity (about 10 days old) and the status of the RC bug reports for the packages but also tries to ensure them to be compatible with the latest set of packages in the testing archive. This process makes the testing archive very current and usable.

Through the gradual archive freeze process led by the release team, the testing archive is matured to make it completely consistent and bug free with some manual interventions. Then the new stable release is created by assigning the codename for the old testing archive to the new stable archive and creating the new codename for the new testing archive. The initial contents of the new testing archive is exactly the same as that of the newly released stable archive.

Both the unstable and the testing archives may suffer temporary glitches due to several factors.

  • Broken package upload to the archive (mostly for unstable)

  • Delay of accepting the new packages to the archive (mostly for unstable)

  • Archive synchronization timing issue (both for testing and unstable)

  • Manual intervention to the archive such as package removal (more for testing) etc.

So if you ever decide to use these archives, you should be able to fix or work around these kinds of glitches.

[Caution] Caution

For about few months after a new stable release, most desktop users should use the stable archive with its security updates even if they usually use unstable or testing archives. For this transition period, both unstable and testing archives are not good for most people. Your system is difficult to keep in good working condition with the unstable archive since it suffers surges of major upgrades for core packages. The testing archive is not useful either since it contains mostly the same content as the stable archive without its security support (Debian testing-security-announce 2008-12). After a month or so, the unstable archive may be usable if you are careful.

[Tip] Tip

When tracking the testing archive, a problem caused by a removed package is usually worked around by installing corresponding package from the unstable archive which is uploaded for bug fix.

See Debian Policy Manual for archive definitions.

Debian is 100% free software because of the followings:

  • Debian installs only free software by default to respect user's freedoms.

  • Debian provides only free software in main.

  • Debian recommends running only free software from main.

  • No packages in main depend nor recommend packages in non-free nor contrib.

Some people wonder if the following 2 facts contradict or not.

These do not contradict, because of the followings.

  • The Debian system is 100% free and its packages are hosted by Debian servers in the main area.

  • Packages outside of the Debian system are hosted by Debian servers in the non-free and contrib areas.

These are precisely explained in the 4th and 5th terms of Debian Social Contract:

  • Our priorities are our users and free software

    • We will be guided by the needs of our users and the free software community. We will place their interests first in our priorities. We will support the needs of our users for operation in many different kinds of computing environments. We will not object to non-free works that are intended to be used on Debian systems, or attempt to charge a fee to people who create or use such works. We will allow others to create distributions containing both the Debian system and other works, without any fee from us. In furtherance of these goals, we will provide an integrated system of high-quality materials with no legal restrictions that would prevent such uses of the system.

  • Works that do not meet our free software standards

    • We acknowledge that some of our users require the use of works that do not conform to the Debian Free Software Guidelines. We have created "contrib" and "non-free" areas in our archive for these works. The packages in these areas are not part of the Debian system, although they have been configured for use with Debian. We encourage CD manufacturers to read the licenses of the packages in these areas and determine if they can distribute the packages on their CDs. Thus, although non-free works are not a part of Debian, we support their use and provide infrastructure for non-free packages (such as our bug tracking system and mailing lists).

Users should be aware of the risks of using packages in the non-free and contrib areas:

  • lack of freedom for such software packages

  • lack of support from Debian on such software packages (Debian can't support software properly without having access to its source code.)

  • contamination of your 100% free Debian system

The Debian Free Software Guidelines are the free software standards for Debian. Debian interprets "software" in the widest scope including document, firmware, logo, and artwork data in the package. This makes Debian's free software standards very strict ones.

Typical non-free and contrib packages include freely distributable packages of following types:

  • Document packages under GNU Free Documentation License with invariant sections such as ones for GCC and Make. (mostly found in the non-free/doc section.)

  • Firmware packages containing sourceless binary data such as ones listed in Section 9.10.5, “Hardware drivers and firmware” as non-free. (mostly found in the non-free/kernel section.)

  • Game and font packages with restriction on commercial use and/or content modification.

Please note that the number of non-free and contrib packages is less than 2% of that of main packages. Enabling access to the non-free and contrib areas does not obscure the source of packages. Interactive full screen use of aptitude(8) provides you with full visibility and control over what packages are installed from which area to keep your system as free as you wish.

The Debian system offers a consistent set of binary packages through its versioned binary dependency declaration mechanism in the control file fields. Here is a bit over simplified definition for them.

  • "Depends"

    • This declares an absolute dependency and all of the packages listed in this field must be installed at the same time or in advance.

  • "Pre-Depends"

    • This is like Depends, except that it requires completed installation of the listed packages in advance.

  • "Recommends"

    • This declares a strong, but not absolute, dependency. Most users would not want the package unless all of the packages listed in this field are installed.

  • "Suggests"

    • This declares a weak dependency. Many users of this package may benefit from installing packages listed in this field but can have reasonable functions without them.

  • "Enhances"

    • This declares a weak dependency like Suggests but works in the opposite direction.

  • "Breaks"

    • This declares a package incompatibility usually with some version specification. Generally the resolution is to upgrade all of the packages listed in this field.

  • "Conflicts"

    • This declares an absolute incompatibility. All of the packages listed in this field must be removed to install this package.

  • "Replaces"

    • This is declared when files installed by this package replace files in the listed packages.

  • "Provides"

    • This is declared when this package provide all of the files and functionality in the listed packages.

[Note] Note

Please note that defining "Provides", "Conflicts" and "Replaces" simultaneously to an virtual package is the sane configuration. This ensures that only one real package providing this virtual package can be installed at any one time.

The official definition including source dependency can be found in the Policy Manual: Chapter 7 - Declaring relationships between packages.

Here is a summary of the simplified event flow of the package management by APT.

Here, I intentionally skipped technical details for the sake of big picture.

Repository based package management operations on the Debian system can be performed by many APT-based package management tools available on the Debian system. Here, we explain 3 basic package management tools: apt, apt-get / apt-cache and aptitude.

For the package management operation which involves package installation or updates package metadata, you need to have root privilege.

Although aptitude is a very nice interactive tool which the author mainly uses, you should know some cautionary facts:

The apt-get and apt-cache commands are the most basic APT-based package management tools.

  • apt-get and apt-cache offer only the commandline user interface.

  • apt-get is most suitable for the major system upgrade between releases, etc.

  • apt-get offers a robust package dependency resolver.

  • apt-get is less demanding on hardware resources. It consumes less memory and runs faster.

  • apt-cache offers a standard regex based search on the package name and description.

  • apt-get and apt-cache can manage multiple versions of packages using /etc/apt/preferences but it is quite cumbersome.

The apt command is a high-level commandline interface for package management. It is basically a wrapper of apt-get, apt-cache and similar commands, originally intended as an end-user interface and enables some options better suited for interactive usage by default.

  • apt provides a friendly progress bar when installing packages using apt install.

  • apt will remove cached .deb packages by default after sucessful installation of downloaded packages.

[Tip] Tip

Users are recommended to use the new apt(8) command for interactive usage and use the apt-get(8) and apt-cache(8) commands in the shell script.

The aptitude command is the most versatile APT-based package management tool.

  • aptitude offers the fullscreen interactive text user interface.

  • aptitude offers the commandline user interface, too.

  • aptitude is most suitable for the daily interactive package management such as inspecting installed packages and searching available packages.

  • aptitude is more demanding on hardware resources. It consumes more memory and runs slower.

  • aptitude offers an enhanced regex based search on all of the package metadata.

  • aptitude can manage multiple versions of packages without using /etc/apt/preferences and it is quite intuitive.

Here are basic package management operations with the commandline using apt(8), aptitude(8) and apt-get(8) /apt-cache(8).

Table 2.6. Basic package management operations with the commandline using apt(8), aptitude(8) and apt-get(8) /apt-cache(8)

apt syntax aptitude syntax apt-get/apt-cache syntax description
apt update aptitude update apt-get update update package archive metadata
apt install foo aptitude install foo apt-get install foo install candidate version of "foo" package with its dependencies
apt upgrade aptitude safe-upgrade apt-get upgrade install candidate version of installed packages without removing any other packages
apt full-upgrade aptitude full-upgrade apt-get dist-upgrade install candidate version of installed packages while removing other packages if needed
apt remove foo aptitude remove foo apt-get remove foo remove "foo" package while leaving its configuration files
apt autoremove N/A apt-get autoremove remove auto-installed packages which are no longer required
apt purge foo aptitude purge foo apt-get purge foo purge "foo" package with its configuration files
apt clean aptitude clean apt-get clean clear out the local repository of retrieved package files completely
apt autoclean aptitude autoclean apt-get autoclean clear out the local repository of retrieved package files for outdated packages
apt show foo aptitude show foo apt-cache show foo display detailed information about "foo" package
apt search regex aptitude search regex apt-cache search regex search packages which match regex
N/A aptitude why regex N/A explain the reason why regex matching packages should be installed
N/A aptitude why-not regex N/A explain the reason why regex matching packages can not be installed
N/A aptitude search '~i!~M' apt-mark showmanual list manually installed packages

apt / apt-get and aptitude can be mixed without major troubles.

The "aptitude why regex" can list more information by "aptitude -v why regex". Similar information can be obtained by "apt rdepends package" or "apt-cache rdepends package".

When aptitude command is started in the commandline mode and faces some issues such as package conflicts, you can switch to the full screen interactive mode by pressing "e"-key later at the prompt.

[Note] Note

Although the aptitude command comes with rich features such as its enhanced package resolver, this complexity has caused (or may still causes) some regressions such as Bug #411123, Bug #514930, and Bug #570377. In case of doubt, please use the apt, apt-get and apt-cache commands over the aptitude command.

You may provide command options right after "aptitude".

See aptitude(8) and "aptitude user's manual" at "/usr/share/doc/aptitude/README" for more.

In the interactive full screen mode of aptitude(8), packages in the package list are displayed as the next example.

idA   libsmbclient                             -2220kB 3.0.25a-1  3.0.25a-2

Here, this line means from the left as the following.

  • The "current state" flag (the first letter)

  • The "planned action" flag (the second letter)

  • The "automatic" flag (the third letter)

  • The Package name

  • The change in disk space usage attributed to "planned action"

  • The current version of the package

  • The candidate version of the package

[Tip] Tip

The full list of flags are given at the bottom of Help screen shown by pressing "?".

The candidate version is chosen according to the current local preferences (see apt_preferences(5) and Section 2.7.3, “Tweaking candidate version”).

Several types of package views are available under the menu "Views".

The standard "Package View" categorizes packages somewhat like dselect with few extra features.

[Tip] Tip

Tasks view can be used to cherry pick packages for your task.

The aptitude regex formula is mutt-like extended ERE (see Section 1.6.2, “Regular expressions”) and the meanings of the aptitude specific special match rule extensions are as follows.

Table 2.11. List of the aptitude regex formula

description of the extended match rule regex formula
match on package name ~nregex_name
match on description ~dregex_description
match on task name ~tregex_task
match on debtag ~Gregex_debtag
match on maintainer ~mregex_maintainer
match on package section ~sregex_section
match on package version ~Vregex_version
match archive ~A{bullseye,bookworm,sid}
match origin ~O{debian,…}
match priority ~p{extra,important,optional,required,standard}
match essential packages ~E
match virtual packages ~v
match new packages ~N
match with pending action ~a{install,upgrade,downgrade,remove,purge,hold,keep}
match installed packages ~i
match installed packages with A-mark (auto installed packages) ~M
match installed packages without A-mark (administrator selected packages) ~i!~M
match installed and upgradable packages ~U
match removed but not purged packages ~c
match removed, purged or can-be-removed packages ~g
match packages declaring a broken dependency ~b
match packages declaring broken dependency of type ~Btype
match pattern packages declaring dependency of type ~D[type:]pattern
match pattern packages declaring broken dependency of type ~DB[type:]pattern
match packages to which the pattern matching package declares dependency type ~R[type:]pattern
match packages to which the pattern matching package declares broken dependency type ~RB[type:]pattern
match packages to which some other installed packages depend on ~R~i
match packages to which no other installed packages depend on !~R~i
match packages to which some other installed packages depend or recommend on ~R~i|~Rrecommends:~i
match pattern package with filtered version ~S filter pattern
match all packages (true) ~T
match no packages (false) ~F

  • The regex part is the same ERE as the one used in typical Unix-like text tools using "^", ".*", "$" etc. as in egrep(1), awk(1) and perl(1).

  • The dependency type is one of (depends, predepends, recommends, suggests, conflicts, replaces, provides) specifying the package interrelationship.

  • The default dependency type is "depends".

[Tip] Tip

When regex_pattern is a null string, place "~T" immediately after the command.

Here are some short cuts.

  • "~Pterm" == "~Dprovides:term"

  • "~Cterm" == "~Dconflicts:term"

  • "…~W term" == "(…|term)"

Users familiar with mutt pick up quickly, as mutt was the inspiration for the expression syntax. See "SEARCHING, LIMITING, AND EXPRESSIONS" in the "User's Manual" "/usr/share/doc/aptitude/README".

[Note] Note

With the lenny version of aptitude(8), the new long form syntax such as "?broken" may be used for regex matching in place for its old short form equivalent "~b". Now space character " " is considered as one of the regex terminating character in addition to tilde character "~". See "User's Manual" for the new long form syntax.

Here are few examples of aptitude(8) operations.

[Note] Note

When moving to a new release etc, you should consider to perform a clean installation of new system even though Debian is upgradable as described below. This provides you a chance to remove garbages collected and exposes you to the best combination of latest packages. Of course, you should make a full backup of system to a safe place (see Section 10.2, “Backup and recovery”) before doing this. I recommend to make a dual boot configuration using different partition to have the smoothest transition.

You can perform system wide upgrade to a newer release by changing contents of the "/etc/apt/sources.list" file pointing to a new release and running the "apt update; apt dist-upgrade" command.

To upgrade from stable to testing or unstable, you replace "bullseye" in the "/etc/apt/sources.list" example of Section 2.1.4, “Debian archive basics” with "bookworm" or "sid".

In reality, you may face some complications due to some package transition issues, mostly due to package dependencies. The larger the difference of the upgrade, the more likely you face larger troubles. For the transition from the old stable to the new stable after its release, you can read its new Release Notes and follow the exact procedure described in it to minimize troubles.

When you decide to move from stable to testing before its formal release, there are no Release Notes to help you. The difference between stable and testing could have grown quite large after the previous stable release and makes upgrade situation complicated.

You should make precautionary moves for the full upgrade while gathering latest information from mailing list and using common senses.

  1. Read previous "Release Notes".

  2. Backup entire system (especially data and configuration information).

  3. Have bootable media handy for broken bootloader.

  4. Inform users on the system well in advance.

  5. Record upgrade activity with script(1).

  6. Apply "unmarkauto" to required packages, e.g., "aptitude unmarkauto vim", to prevent removal.

  7. Minimize installed packages to reduce chance of package conflicts, e.g., remove desktop task packages.

  8. Remove the "/etc/apt/preferences" file (disable apt-pinning).

  9. Try to upgrade step wise: oldstablestabletestingunstable.

  10. Update the "/etc/apt/sources.list" file to point to new archive only and run "aptitude update".

  11. Install, optionally, new core packages first, e.g., "aptitude install perl".

  12. Run the "apt-get -s dist-upgrade" command to assess impact.

  13. Run the "apt-get dist-upgrade" command at last.

[Caution] Caution

It is not wise to skip major Debian release when upgrading between stable releases.

[Caution] Caution

In previous "Release Notes", GCC, Linux Kernel, initrd-tools, Glibc, Perl, APT tool chain, etc. have required some special attention for system wide upgrade.

For daily upgrade in unstable, see Section 2.4.3, “Safeguarding for package problems”.

Here are list of other package management operations for which aptitude is too high-level or lacks required functionalities.

Table 2.13. List of advanced package management operations

command action
COLUMNS=120 dpkg -l package_name_pattern list status of an installed package for the bug report
dpkg -L package_name list contents of an installed package
dpkg -L package_name | egrep '/usr/share/man/man.*/.+' list manpages for an installed package
dpkg -S file_name_pattern list installed packages which have matching file name
apt-file search file_name_pattern list packages in archive which have matching file name
apt-file list package_name_pattern list contents of matching packages in archive
dpkg-reconfigure package_name reconfigure the exact package
dpkg-reconfigure -plow package_name reconfigure the exact package with the most detailed question
configure-debian reconfigure packages from the full screen menu
dpkg --audit audit system for partially installed packages
dpkg --configure -a configure all partially installed packages
apt-cache policy binary_package_name show available version, priority, and archive information of a binary package
apt-cache madison package_name show available version, archive information of a package
apt-cache showsrc binary_package_name show source package information of a binary package
apt-get build-dep package_name install required packages to build package
aptitude build-dep package_name install required packages to build package
apt-get source package_name download a source (from standard archive)
dget URL for dsc file download a source packages (from other archive)
dpkg-source -x package_name_version-debian.revision.dsc build a source tree from a set of source packages ("*.orig.tar.gz" and "*.debian.tar.gz"/"*.diff.gz")
debuild binary build package(s) from a local source tree
make-kpkg kernel_image build a kernel package from a kernel source tree
make-kpkg --initrd kernel_image build a kernel package from a kernel source tree with initramfs enabled
dpkg -i package_name_version-debian.revision_arch.deb install a local package to the system
apt install /path/to/package_filename.deb install a local package to the system, meanwhile try to resolve dependency automatically
debi package_name_version-debian.revision_arch.dsc install local package(s) to the system
dpkg --get-selections '*' >selection.txt save dpkg level package selection state information
dpkg --set-selections <selection.txt set dpkg level package selection state information
echo package_name hold | dpkg --set-selections set dpkg level package selection state for a package to hold (equivalent to "aptitude hold package_name")

[Note] Note

For a package with the multi-arch feature, you may need to specify the architecture name for some commands. For example, use "dpkg -L libglib2.0-0:amd64" to list contents of the libglib2.0-0 package for the amd64 architecture.

[Caution] Caution

Lower level package tools such as "dpkg -i …" and "debi …" should be carefully used by the system administrator. It does not automatically take care required package dependencies. Dpkg's commandline options "--force-all" and similar (see dpkg(1)) are intended to be used by experts only. Using them without fully understanding their effects may break your whole system.

Please note the following.

The installation of debsums enables verification of installed package files against MD5sum values in the "/var/lib/dpkg/info/*.md5sums" file with debsums(1). See Section 10.3.5, “The MD5 sum” for how MD5sum works.

[Note] Note

Because MD5sum database may be tampered by the intruder, debsums(1) is of limited use as a security tool. It is only good for checking local modifications by the administrator or damage due to media errors.

Although visiting Debian site facilitates easy ways to search on the package meta data these days, let's look into more traditional ways.

The grep-dctrl(1), grep-status(1), and grep-available(1) commands can be used to search any file which has the general format of a Debian package control file.

The "dpkg -S file_name_pattern" can be used to search package names which contain files with the matching name installed by dpkg. But this overlooks files created by the maintainer scripts.

If you need to make more elaborate search on the dpkg meta data, you need to run "grep -e regex_pattern *" command in the "/var/lib/dpkg/info/" directory. This makes you search words mentioned in package scripts and installation query texts.

If you wish to look up package dependency recursively, you should use apt-rdepends(8).

Let's learn how the Debian package management system works internally. This should help you to create your own solution to some package problems.

[Tip] Tip

The top level "Release" file is used for signing the archive under the secure APT system.

Each suite of the Debian archive has a top level "Release" file, e.g., "", as follows.

Origin: Debian
Label: Debian
Suite: unstable
Codename: sid
Date: Sat, 14 May 2011 08:20:50 UTC
Valid-Until: Sat, 21 May 2011 08:20:50 UTC
Architectures: alpha amd64 armel hppa hurd-i386 i386 ia64 kfreebsd-amd64 kfreebsd-i386 mips mipsel powerpc s390 sparc
Components: main contrib non-free
Description: Debian x.y Unstable - Not Released
 bdc8fa4b3f5e4a715dd0d56d176fc789 18876880 Contents-alpha.gz
 9469a03c94b85e010d116aeeab9614c0 19441880 Contents-amd64.gz
 3d68e206d7faa3aded660dc0996054fe 19203165 Contents-armel.gz
[Note] Note

Here, you can find my rationale to use the "suite", and "codename" in Section 2.1.4, “Debian archive basics”. The "distribution" is used when referring to both "suite" and "codename". All archive "area" names offered by the archive are listed under "Components".

The integrity of the top level "Release" file is verified by cryptographic infrastructure called the secure apt.

  • The cryptographic signature file "Release.gpg" is created from the authentic top level "Release" file and the secret Debian archive key.

  • The public Debian archive key can be seeded into "/etc/apt/trusted.gpg";

  • The secure APT system verifies the integrity of the downloaded top level "Release" file cryptographically by this "Release.gpg" file and the public Debian archive key in "/etc/apt/trusted.gpg".

The integrity of all the "Packages" and "Sources" files are verified by using MD5sum values in its top level "Release" file. The integrity of all package files are verified by using MD5sum values in the "Packages" and "Sources" files. See debsums(1) and Section 2.4.2, “Verification of installed package files”.

Since the cryptographic signature verification is a much more CPU intensive process than the MD5sum value calculation, use of MD5sum value for each package while using cryptographic signature for the top level "Release" file provides the good security with the performance (see Section 10.3, “Data security infrastructure”).

When APT tools, such as aptitude, apt-get, synaptic, apt-file, auto-apt, … are used, we need to update the local copies of the meta data containing the Debian archive information. These local copies have following file names corresponding to the specified distribution, area, and architecture names in the "/etc/apt/sources.list" (see Section 2.1.4, “Debian archive basics”).

  • "/var/lib/apt/lists/deb.debian.org_debian_dists_distribution_Release"

  • "/var/lib/apt/lists/deb.debian.org_debian_dists_distribution_Release.gpg"

  • "/var/lib/apt/lists/deb.debian.org_debian_dists_distribution_area_binary-architecture_Packages"

  • "/var/lib/apt/lists/deb.debian.org_debian_dists_distribution_area_source_Sources"

  • "/var/cache/apt/apt-file/deb.debian.org_debian_dists_distribution_Contents-architecture.gz" (for apt-file)

First 4 types of files are shared by all the pertinent APT commands and updated from command line by "apt-get update" or "aptitude update". The "Packages" meta data are updated if there is the "deb" line in "/etc/apt/sources.list". The "Sources" meta data are updated if there is the "deb-src" line in "/etc/apt/sources.list".

The "Packages" and "Sources" meta data contain "Filename:" stanza pointing to the file location of the binary and source packages. Currently, these packages are located under the "pool/" directory tree for the improved transition over the releases.

Local copies of "Packages" meta data can be interactively searched with the help of aptitude. The specialized search command grep-dctrl(1) can search local copies of "Packages" and "Sources" meta data.

Local copy of "Contents-architecture" meta data can be updated by "apt-file update" and its location is different from other 4 ones. See apt-file(1). (The auto-apt uses different location for local copy of "Contents-architecture.gz" as default.)

Debian package files have particular name structures.

[Tip] Tip

Here only the basic source package formats are described. See more on dpkg-source(1).

[Note] Note

You can check package version order by dpkg(1), e.g., "dpkg --compare-versions 7.0 gt 7.~pre1 ; echo $?" .

[Note] Note

The debian-installer (d-i) uses udeb as the file extension for its binary package instead of normal deb. An udeb package is a stripped down deb package which removes few non-essential contents such as documentation to save space while relaxing the package policy requirements. Both deb and udeb packages share the same package structure. The "u" stands for micro.

dpkg(1) is the lowest level tool for the Debian package management. This is very powerful and needs to be used with care.

While installing package called "package_name", dpkg process it in the following order.

  1. Unpack the deb file ("ar -x" equivalent)

  2. Execute "package_name.preinst" using debconf(1)

  3. Install the package content to the system ("tar -x" equivalent)

  4. Execute "package_name.postinst" using debconf(1)

The debconf system provides standardized user interaction with I18N and L10N (Chapter 8, I18N and L10N) supports.

The "status" file is also used by the tools such as dpkg(1), "dselect update" and "apt-get -u dselect-upgrade".

The specialized search command grep-dctrl(1) can search the local copies of "status" and "available" meta data.

[Tip] Tip

In the debian-installer environment, the udpkg command is used to open udeb packages. The udpkg command is a stripped down version of the dpkg command.

When running testing or unstable system, the administrator is expected to recover from broken package management situation.

[Caution] Caution

Some methods described here are high risk actions. You have been warned!

Caching errors of the package data cause intriguing errors, such as "GPG error: ... invalid: BADSIG ..." with APT.

You should remove all cached data by "sudo rm -rf /var/lib/apt/* " and try again. (If apt-cacher-ng is used, you should also run "sudo rm -rf /var/cache/apt-cacher-ng/* ".)

Archive level package management systems, such as aptitude(8) or apt-get(1), do not even try to install packages with overlapped files using package dependencies (see Section 2.1.6, “Package dependencies”).

Errors by the package maintainer or deployment of inconsistently mixed source of archives (see Section 2.7.2, “Packages from mixed source of archives”) by the system administrator may create a situation with incorrectly defined package dependencies. When you install a package with overlapped files using aptitude(8) or apt-get(1) under such a situation, dpkg(1) which unpacks package ensures to return error to the calling program without overwriting existing files.

[Caution] Caution

The use of third party packages introduces significant system risks via maintainer scripts which are run with root privilege and can do anything to your system. The dpkg(1) command only protects against overwriting by the unpacking.

You can work around such broken installation by removing the old offending package, old-package, first.

$ sudo dpkg -P old-package

Since dpkg is very low level package tool, it can function under the very bad situation such as unbootable system without network connection. Let's assume foo package was broken and needs to be replaced.

You may still find cached copies of older bug free version of foo package in the package cache directory: "/var/cache/apt/archives/". (If not, you can download it from archive of or copy it from package cache of a functioning machine.)

If you can boot the system, you may install it by the following command.

# dpkg -i /path/to/foo_old_version_arch.deb
[Tip] Tip

If system breakage is minor, you may alternatively downgrade the whole system as in Section 2.7.10, “Emergency downgrading” using the higher level APT system.

If your system is unbootable from hard disk, you should seek other ways to boot it.

  1. Boot the system using the debian-installer CD in rescue mode.

  2. Mount the unbootable system on the hard disk to "/target".

  3. Install older version of foo package by the following.

# dpkg --root /target -i /path/to/foo_old_version_arch.deb

This example works even if the dpkg command on the hard disk is broken.

[Tip] Tip

Any GNU/Linux system started by another system on hard disk, live GNU/Linux CD, bootable USB-key drive, or netboot can be used similarly to rescue broken system.

If attempting to install a package this way fails due to some dependency violations and you really need to do this as the last resort, you can override dependency using dpkg's "--ignore-depends", "--force-depends" and other options. If you do this, you need to make serious effort to restore proper dependency later. See dpkg(8) for details.

[Note] Note

If your system is seriously broken, you should make a full backup of system to a safe place (see Section 10.2, “Backup and recovery”) and should perform a clean installation. This is less time consuming and produces better results in the end.

[Caution] Caution

Installing packages from mixed source of archives is not supported by the official Debian distribution except for officially supported particular combinations of archives such as stable with security updates and bullseye-updates.

Here is an example of operations to include specific newer upstream version packages found in unstable while tracking testing for single occasion.

  1. Change the "/etc/apt/sources.list" file temporarily to single "unstable" entry.

  2. Run "aptitude update".

  3. Run "aptitude install package-name".

  4. Recover the original "/etc/apt/sources.list" file for testing.

  5. Run "aptitude update".

You do not create the "/etc/apt/preferences" file nor need to worry about apt-pinning with this manual approach. But this is very cumbersome.

[Caution] Caution

When using mixed source of archives, you must ensure compatibility of packages by yourself since the Debian does not guarantee it. If package incompatibility exists, you may break system. You must be able to judge these technical requirements. The use of mixed source of random archives is completely optional operation and its use is not something I encourage you to use.

General rules for installing packages from different archives are the following.

[Note] Note

In order to make a package to be safer to install, some commercial non-free binary program packages may be provided with completely statically linked libraries. You should still check ABI compatibility issues etc. for them.

[Note] Note

Except to avoid broken package for a short term, installing binary packages from officially unsupported archives is generally bad idea. This is true even if you use apt-pinning (see Section 2.7.3, “Tweaking candidate version”). You should consider chroot or similar techniques (see Section 9.11, “Virtualized system”) to run programs from different archives.

[Warning] Warning

Use of apt-pinning by a novice user is sure call for major troubles. You must avoid using apt-pinning except when you absolutely need it.

Without the "/etc/apt/preferences" file, APT system choses the latest available version as the candidate version using the version string. This is the normal state and most recommended usage of APT system. All officially supported combinations of archives do not require the "/etc/apt/preferences" file since some archives which should not be used as the automatic source of upgrades are marked as NotAutomatic and dealt properly.

[Tip] Tip

The version string comparison rule can be verified with, e.g., "dpkg --compare-versions ver1.1 gt ver1.1~1; echo $?" (see dpkg(1)).

When you install packages from mixed source of archives (see Section 2.7.2, “Packages from mixed source of archives”) regularly, you can automate these complicated operations by creating the "/etc/apt/preferences" file with proper entries and tweaking the package selection rule for candidate version as described in apt_preferences(5). This is called apt-pinning.

[Caution] Caution

When using apt-pinning, you must ensure compatibility of packages by yourself since the Debian does not guarantee it. The apt-pinning is completely optional operation and its use is not something I encourage you to use.

[Caution] Caution

Archive level Release files (see Section 2.5.3, “Archive level "Release" files”) are used for the rule of apt_preferences(5). Thus apt-pinning works only with "suite" name for normal Debian archives and security Debian archives. (This is different from Ubuntu archives.) For example, you can do "Pin: release a=unstable" but can not do "Pin: release a=sid" in the "/etc/apt/preferences" file.

[Caution] Caution

When you use non-Debian archive as a part of apt-pinning, you should check what they are intended for and also check their credibility. For example, Ubuntu and Debian are not meant to be mixed.

[Note] Note

Even if you do not create the "/etc/apt/preferences" file, you can do fairly complex system operations (see Section 2.6.6, “Rescue with the dpkg command” and Section 2.7.2, “Packages from mixed source of archives”) without apt-pinning.

Here is a simplified explanation of apt-pinning technique.

The APT system choses the highest Pin-Priority upgrading package from available package sources defined in the "/etc/apt/sources.list" file as the candidate version package. If the Pin-Priority of the package is larger than 1000, this version restriction for upgrading is dropped to enable downgrading (see Section 2.7.10, “Emergency downgrading”).

Pin-Priority value of each package is defined by "Pin-Priority" entries in the "/etc/apt/preferences" file or uses its default value.

The target release archive can be set by several methods.

  • "/etc/apt/apt.conf" configuration file with "APT::Default-Release "stable";" line

  • command line option, e.g., "apt-get install -t testing some-package"

The NotAutomatic and ButAutomaticUpgrades archive is set by archive server having its archive level Release file (see Section 2.5.3, “Archive level "Release" files”) containing both "NotAutomatic: yes" and "ButAutomaticUpgrades: yes". The NotAutomatic archive is set by archive server having its archive level Release file containing only "NotAutomatic: yes".

The apt-pinning situation of package from multiple archive sources is displayed by "apt-cache policy package".

  • A line started with "Package pin:" lists the package version of pin if association just with package is defined, e.g., "Package pin: 0.190".

  • No line with "Package pin:" exists if no association just with package is defined.

  • The Pin-Priority value associated just with package is listed right side of all version strings, e.g., "0.181 700".

  • "0" is listed right side of all version strings if no association just with package is defined, e.g., "0.181 0".

  • The Pin-Priority values of archives (defined as "Package: *" in the "/etc/apt/preferences" file) are listed left side of all archive paths, e.g., "100 bullseye-backports/main Packages".

There are bullseye-updates and archives which provide upgrade packages for stable (bullseye).

In order to use these archives, you list all required archives in the "/etc/apt/sources.list" file as the following.

deb bullseye main non-free-firmware contrib non-free
deb bullseye-security main non-free-firmware contrib non-free
deb bullseye-updates main non-free-firmware contrib non-free
deb bullseye-backports main non-free-firmware contrib non-free

There is no need to set Pin-Priority value explicitly in the "/etc/apt/preferences" file. When newer packages become available, the default configuration provides most reasonable upgrades (see Section 2.5.3, “Archive level "Release" files”).

  • All installed older packages are upgraded to newer ones from bullseye-updates.

  • Only manually installed older packages from bullseye-backports are upgraded to newer ones from bullseye-backports.

Whenever you wish to install a package named "package-name" with its dependency from bullseye-backports archive manually, you use following command while switching target release with "-t" option.

$ sudo apt-get install -t bullseye-backports package-name
[Warning] Warning

Use of apt-pinning by a novice user is sure call for major troubles. You must avoid using apt-pinning except when you absolutely need it.

Here is an example of apt-pinning technique to include specific newer upstream version packages found in unstable regularly upgraded while tracking testing. You list all required archives in the "/etc/apt/sources.list" file as the following.

deb testing main contrib non-free
deb unstable main contrib non-free
deb testing-security main contrib

Set the "/etc/apt/preferences" file as the following.

Package: *
Pin: release a=unstable
Pin-Priority: 100

When you wish to install a package named "package-name" with its dependencies from unstable archive under this configuration, you issue the following command which switches target release with "-t" option (Pin-Priority of unstable becomes 990).

$ sudo apt-get install -t unstable package-name

With this configuration, usual execution of "apt-get upgrade" and "apt-get dist-upgrade" (or "aptitude safe-upgrade" and "aptitude full-upgrade") upgrades packages which were installed from testing archive using current testing archive and packages which were installed from unstable archive using current unstable archive.

[Caution] Caution

Be careful not to remove "testing" entry from the "/etc/apt/sources.list" file. Without "testing" entry in it, APT system upgrades packages using newer unstable archive.

[Tip] Tip

I usually edit the "/etc/apt/sources.list" file to comment out "unstable" archive entry right after above operation. This avoids slow update process of having too many entries in the "/etc/apt/sources.list" file although this prevents upgrading packages which were installed from unstable archive using current unstable archive.

[Tip] Tip

If "Pin-Priority: 1" is used instead of "Pin-Priority: 100" in the "/etc/apt/preferences" file, already installed packages having Pin-Priority value of 100 are not upgraded by unstable archive even if "testing" entry in the "/etc/apt/sources.list" file is removed.

If you wish to track particular packages in unstable automatically without initial "-t unstable" installation, you must create the "/etc/apt/preferences" file and explicitly list all those packages at the top of it as the following.

Package: package-1
Pin: release a=unstable
Pin-Priority: 700

Package: package-2
Pin: release a=unstable
Pin-Priority: 700

These set Pin-Priority value for each specific package. For example, in order to track the latest unstable version of this "Debian Reference" in English, you should have following entries in the "/etc/apt/preferences" file.

Package: debian-reference-en
Pin: release a=unstable
Pin-Priority: 700

Package: debian-reference-common
Pin: release a=unstable
Pin-Priority: 700
[Tip] Tip

This apt-pinning technique is valid even when you are tracking stable archive. Documentation packages have been always safe to install from unstable archive in my experience, so far.

[Warning] Warning

Use of apt-pinning by a novice user is sure call for major troubles. You must avoid using apt-pinning except when you absolutely need it.

The apt package comes with its own cron script "/etc/cron.daily/apt" to support the automatic download of packages. This script can be enhanced to perform the automatic upgrade of packages by installing the unattended-upgrades package. These can be customized by parameters in "/etc/apt/apt.conf.d/02backup" and "/etc/apt/apt.conf.d/50unattended-upgrades" as described in "/usr/share/doc/unattended-upgrades/README".

The unattended-upgrades package is mainly intended for the security upgrade for the stable system. If the risk of breaking an existing stable system by the automatic upgrade is smaller than that of the system broken by the intruder using its security hole which has been closed by the security update, you should consider using this automatic upgrade with configuration parameters as the following.

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::Unattended-Upgrade "1";

If you are running an unstable system, you do not want to use the automatic upgrade since it certainly breaks system some day. Even for such unstable case, you may still want to download packages in advance to save time for the interactive upgrade with configuration parameters as the following.

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::Unattended-Upgrade "0";
[Warning] Warning

Use of apt-pinning by a novice user is sure call for major troubles. You must avoid using apt-pinning except when you absolutely need it.

[Caution] Caution

Downgrading is not officially supported by the Debian by design. It should be done only as a part of emergency recovery process. Despite of this situation, it is known to work well in many incidents. For critical systems, you should backup all important data on the system after the recovery operation and re-install the new system from the scratch.

You may be lucky to downgrade from newer archive to older archive to recover from broken system upgrade by manipulating candidate version (see Section 2.7.3, “Tweaking candidate version”). This is lazy alternative to tedious actions of many "dpkg -i broken-package_old-version.deb" commands (see Section 2.6.6, “Rescue with the dpkg command”).

Search lines in the "/etc/apt/sources.list" file tracking unstable as the following.

deb sid main contrib non-free

Replace it with the following to track testing.

deb bookworm main contrib non-free

Set the "/etc/apt/preferences" file as the following.

Package: *
Pin: release a=testing
Pin-Priority: 1010

Run "apt-get update; apt-get dist-upgrade" to force downgrading of packages across the system.

Remove this special "/etc/apt/preferences" file after this emergency downgrading.

[Tip] Tip

It is a good idea to remove (not purge!) as much packages to minimize dependency problems. You may need to manually remove and install some packages to get system downgraded. Linux kernel, bootloader, udev, PAM, APT, and networking related packages and their configuration files require special attention.