Chapter 6. Network applications

Table of Contents

6.1. Web browsers
6.1.1. Spoofing the User-Agent string
6.1.2. Browser extension
6.2. The mail system
6.2.1. Email basics
6.2.2. Modern mail service limitation
6.2.3. Historic mail service expectation
6.2.4. Mail transport agent (MTA) The configuration of exim4 The configuration of postfix with SASL The mail address configuration Basic MTA operations
6.3. The remote access server and utilities (SSH)
6.3.1. Basics of SSH
6.3.2. User name on the remote host
6.3.3. Connecting without remote passwords
6.3.4. Dealing with alien SSH clients
6.3.5. Setting up ssh-agent
6.3.6. Sending a mail from a remote host
6.3.7. Port forwarding for SMTP/POP3 tunneling
6.3.8. How to shutdown the remote system on SSH
6.3.9. Troubleshooting SSH
6.4. The print server and utilities
6.5. Other network application servers
6.6. Other network application clients
6.7. The diagnosis of the system daemons

After establishing network connectivity (see Chapter 5, Network setup), you can run various network applications.

[Tip] Tip

For modern Debian specific guide to the network infrastructure, read The Debian Administrator's Handbook — Network Infrastructure.

[Tip] Tip

If you enabled "2-Step Verification" with some ISP, you need to obtain an application password to access POP and SMTP services from your program. You may need to approve your host IP in advance.

There are many web browser packages to access remote contents with Hypertext Transfer Protocol (HTTP).

This section focuses on typical mobile workstations on consumer grade Internet connections.

[Caution] Caution

If you are to set up the mail server to exchange mail directly with the Internet, you should be better than reading this elementary document.

An email message consists of three components, the message envelope, the message header, and the message body.

  • The "To" and "From" information in the message envelope is used by the SMTP to deliver the email. (The "From" information in the message envelope is also called bounce address, From_, etc.).

  • The "To" and "From" information in the message header is displayed by the email client. (While it is most common for these to be the same as ones in the message envelope, such is not always the case.)

  • The email message format covering header and body data is extended by Multipurpose Internet Mail Extensions (MIME) from the plain ASCII text to other character encodings, as well as attachments of audio, video, images, and application programs.

Full featured GUI based email clients offer all the following functions using the GUI based intuitive configuration.

  • It creates and interprets the message header and body data using Multipurpose Internet Mail Extensions (MIME) to deal the content data type and encoding.

  • It authenticates itself to the ISP's SMTP and IMAP servers using the legacy basic access authentication or modern OAuth 2.0. (For OAuth 2.0, set it via Desktop environment settings. E.g., "Settings" -> "Online Accounts".)

  • It sends the message to the ISP's smarthost SMTP server listening to the message submission port (587).

  • It receives the stored message on the ISP's server from the TLS/IMAP4 port (993).

  • It can filter mails by their attributes.

  • It may offer additional functionalities: Contacts, Calendar, Tasks, Memos.

Debian mobile workstations can be configured just with full featured GUI based email clients without mail transfer agent (MTA) program after Debian 12 Bookworm.

Debian traditionally installed some MTA program to support programs expecting the /usr/sbin/sendmail command. Such MTA on mobile workstations must cope with Section 6.2.2, “Modern mail service limitation” and Section 6.2.3, “Historic mail service expectation”.

For mobile workstations, the typical choice of MTA is either exim4-daemon-light or postfix with its installation option such as "Mail sent by smarthost; received via SMTP or fetchmail" selected. These are light weight MTAs that respect "/etc/aliases".

[Tip] Tip

Configuring exim4 to send the Internet mail via multiple corresponding smarthosts for multiple source email addresses is non-trivial. If you need such capability for some programs, set them up to use msmtp which is easy to set up for multiple source email addresses. Then leave main MTA only for a single email address.

For the Internet mail via smarthost, you (re)configure exim4-* packages as the following.

$ sudo systemctl stop exim4
$ sudo dpkg-reconfigure exim4-config

Select "mail sent by smarthost; received via SMTP or fetchmail" for "General type of mail configuration".

Set "System mail name:" to its default as the FQDN (see Section 5.1.1, “The hostname resolution”).

Set "IP-addresses to listen on for incoming SMTP connections:" to its default as " ; ::1".

Unset contents of "Other destinations for which mail is accepted:".

Unset contents of "Machines to relay mail for:".

Set "IP address or host name of the outgoing smarthost:" to "smtp.hostname.dom:587".

Select "No" for "Hide local mail name in outgoing mail?". (Use "/etc/email-addresses" as in Section, “The mail address configuration”, instead.)

Reply to "Keep number of DNS-queries minimal (Dial-on-Demand)?" as one of the following.

  • "No" if the system is connected to the Internet while booting.

  • "Yes" if the system is not connected to the Internet while booting.

Set "Delivery method for local mail:" to "mbox format in /var/mail/".

Select "Yes" for "Split configuration into small files?:".

Create password entries for the smarthost by editing "/etc/exim4/passwd.client".

$ sudo vim /etc/exim4/passwd.client
$ cat /etc/exim4/passwd.client

Configure exim4(8) with "QUEUERUNNER='queueonly'", "QUEUERUNNER='nodaemon'", etc. in "/etc/default/exim4" to minimize system resource usages. (optional)

Start exim4 by the following.

$ sudo systemctl start exim4

The host name in "/etc/exim4/passwd.client" should not be the alias. You check the real host name with the following.

$ host smtp.hostname.dom
smtp.hostname.dom is an alias for smtp99.hostname.dom.
smtp99.hostname.dom has address

I use regex in "/etc/exim4/passwd.client" to work around the alias issue. SMTP AUTH probably works even if the ISP moves host pointed by the alias.

You can manually update exim4 configuration by the following:

  • Update exim4 configuration files in "/etc/exim4/".

    • creating "/etc/exim4/exim4.conf.localmacros" to set MACROs and editing "/etc/exim4/exim4.conf.template". (non-split configuration)

    • creating new files or editing existing files in the "/etc/exim4/exim4.conf.d" subdirectories. (split configuration)

  • Run "systemctl reload exim4".

[Caution] Caution

Starting exim4 takes long time if "No" (default value) was chosen for the debconf query of "Keep number of DNS-queries minimal (Dial-on-Demand)?" and the system is not connected to the Internet while booting.

Please read the official guide at: "/usr/share/doc/exim4-base/README.Debian.gz" and update-exim4.conf(8).

[Warning] Warning

For all practical consideration, use SMTP with STARTTLS on port 587 or SMTPS (SMTP over SSL) on port 465, instead of plain SMTP on port 25.

There are a few mail address configuration files for mail transport, delivery and user agents.

The mailname in the "/etc/mailname" file is usually a fully qualified domain name (FQDN) that resolves to one of the host's IP addresses. For the mobile workstation which does not have a hostname with resolvable IP address, set this mailname to the value of "hostname -f". (This is safe choice and works for both exim4-* and postfix.)

[Tip] Tip

The contents of "/etc/mailname" is used by many non-MTA programs for their default behavior. For mutt, set "hostname" and "from" variables in ~/muttrc file to override the mailname value. For programs in the devscripts package, such as bts(1) and dch(1), export environment variables "$DEBFULLNAME" and "$DEBEMAIL" to override it.

[Tip] Tip

The popularity-contest package normally send mail from root account with FQDN. You need to set MAILFROM in /etc/popularity-contest.conf as described in the /usr/share/popularity-contest/default.conf file. Otherwise, your mail will be rejected by the smarthost SMTP server. Although this is tedious, this approach is safer than rewriting the source address for all mails from root by MTA and should be used for other daemons and cron scripts.

When setting the mailname to "hostname -f", the spoofing of the source mail address via MTA can be realized by the following.

  • "/etc/email-addresses" file for exim4(8) as explained in the exim4-config_files(5)

  • "/etc/postfix/generic" file for postfix(1) as explained in the generic(5)

For postfix, the following extra steps are needed.

# postmap hash:/etc/postfix/generic
# postconf -e 'smtp_generic_maps = hash:/etc/postfix/generic'
# postfix reload

You can test mail address configuration using the following.

  • exim(8) with -brw, -bf, -bF, -bV, … options

  • postmap(1) with -q option.

[Tip] Tip

Exim comes with several utility programs such as exiqgrep(8) and exipick(8). See "dpkg -L exim4-base|grep man8/" for available commands.

The Secure SHell (SSH) is the secure way to connect over the Internet. A free version of SSH called OpenSSH is available as openssh-client and openssh-server packages in Debian.

For the user, ssh(1) functions as a smarter and more secure telnet(1). Unlike telnet command, ssh command does not stop on the telnet escape character (initial default CTRL-]).

Although shellinabox is not a SSH program, it is listed here as an interesting alternative for the remote terminal access.

See also Section 7.9, “X server connection” for connecting to remote X client programs.

[Caution] Caution

See Section 4.6.3, “Extra security measures for the Internet” if your SSH is accessible from the Internet.

[Tip] Tip

Please use the screen(1) program to enable remote shell process to survive the interrupted connection (see Section 9.1.2, “The screen program”).

You need to protect the process doing "shutdown -h now" (see Section 1.1.8, “How to shutdown the system”) from the termination of SSH using the at(1) command (see Section 9.4.13, “Scheduling tasks once”) by the following.

# echo "shutdown -h now" | at now

Running "shutdown -h now" in screen(1) (see Section 9.1.2, “The screen program”) session is another way to do the same.

In the old Unix-like system, the BSD Line printer daemon (lpd) was the standard and the standard print out format of the classic free software was PostScript (PS). Some filter system was used along with Ghostscript to enable printing to the non-PostScript printer. See Section 11.4.1, “Ghostscript”.

In the modern Debian system, the Common UNIX Printing System (CUPS) is the de facto standard and the standard print out format of the modern free software is Portable Document Format (PDF).

The CUPS uses Internet Printing Protocol (IPP). The IPP is now supported by other OSs such as Windows XP and Mac OS X and has became new cross-platform de facto standard for remote printing with bi-directional communication capability.

Thanks to the file format dependent auto-conversion feature of the CUPS system, simply feeding any data to the lpr command should generate the expected print output. (In CUPS, lpr can be enabled by installing the cups-bsd package.)

The Debian system has some notable packages for the print servers and utilities.

[Tip] Tip

You can configure CUPS system by pointing your web browser to "http://localhost:631/" .

Here are other network application servers.

Common Internet File System Protocol (CIFS) is the same protocol as Server Message Block (SMB) and is used widely by Microsoft Windows.

[Tip] Tip

See Section 4.5.2, “The modern centralized system management” for integration of server systems.

[Tip] Tip

The hostname resolution is usually provided by the DNS server. For the host IP address dynamically assigned by DHCP, Dynamic DNS can be set up for the hostname resolution using bind9 and isc-dhcp-server as described in the DDNS page on the Debian wiki.

[Tip] Tip

Use of proxy server such as squid is much more efficient for saving bandwidth than use of local mirror server with the full Debian archive contents.

Here are other network application clients.

The telnet program enables manual connection to the system daemons and its diagnosis.

For testing plain POP3 service, try the following

$ telnet pop3

For testing the TLS/SSL enabled POP3 service by some ISPs, you need TLS/SSL enabled telnet client by the telnet-ssl or openssl packages.

$ telnet -z ssl 995
$ openssl s_client -connect

The following RFCs provide required knowledge to each system daemon.

The port usage is described in "/etc/services".