Tag Archives: Linux How to

Unix Similar multi-user operating systems based on the Linux kernel and essentially on GNU software. Like CentOS, Debian, Ubuntu Fedora.

How to fetchmail on Debian 11

fetchmail on Debian with Postfix retrieving and forwarding e-mails

The fetchmail utility can run in daemon mode to repeatedly poll one or more systems at a certain interval, here on a debian system, collecting mail from servers that support all popular mail retrieval services, such as the POP3 and IMAP.

This tutorial show you how to use fetchmail on a Debian 11 (bullseye) with Postfix. The e-mails from external mail service providers are retrieved and forwarded to the recipients to the mailbox server which receives e-mails from the smarthost. No forwarding is required for the mail accounts, but the e-mails can be scanned by the smarthost for viruses and SPAM before they are delivered to the user’s mailbox.

How to install fetchmail on Debian

fetchmail can be deployed from the Debian standard repository.

$ sudo apt install -y fetchmail

Enable daemon mode in/etc/default/fetchmail

# This file will be used to declare some vars for fetchmail
#
# Uncomment the following if you don't want localized log messages
# export LC_ALL=C

# If you want to specify any additional OPTION to the start
# scripts specify them here
# OPTIONS=...

# Declare here if we want to start fetchmail. 'yes' or 'no'
START_DAEMON=yes

Change START_DAEMON from no to yes.

Create fetchmail global recourcen configuration fetchmailrc for operation as a daemon in/etc/fetchmailrc on debian.

Set Daemon 900
set no syslog
set logfile /var/log/fetchmail
Set Postmaster "Postmaster"
set no bouncemail
set no spambounce
set properties ""

poll pop.gmx.net with proto POP3
user 'john@gmx.net' there with password 'M1HXGLKQJ9OZPCA6V34R' is john@foo.org here options fetchall nokeep ssl sslcommonname mail.gmx.net smtphost localhost

fetchmail poll is fetching emails

fetchmail poll line is created for each mail server from which emails are fetched. Every 15 minutes, the external mailbox from john@gmx.net is retrieved from the POP3 server mail.gmx.net and delivered to the user john@foo.org with smtphost via localhost using Postfix to the mailbox server. So that the logging does not end up in /var/log/mail.log, they are logged in /var/log/fetchmail instead

The Common Name (CN) from the certificate, which must be passed with sslcommonname, can be determined using the web browser, or with hit the following command in the Linux shell or from the Windows command prompt.

$ openssl s_client -connect pop.gmail.com:995 | grep "CN="
# with Windows OS
C:\> openssl s_client pop.gmail.com:995 | findstr "CN ="

  On Windows 10/11, OpenSSL must first be provided, the binaries are available at slproweb.com, or you can install the package with hit the command winget install openssl

fetchmail options

fetchmail provides a number of syntactic features to make it easier to read fetchmailrc. While it is possible to provide credentials for a server on a row, common configurations are specified over a number of different lines. fetchmail is insensitive to whitespace unless the argument is between quotation marks.

There are several options for the Poll statement (for example, nofetchall (default), fetchall, keep, or nokeep).The meanings are as follows:

nofetchall: Get only new messages (default).Unless otherwise specified (e.g. fetchall, keep), this means nofetchall.
fetchall: Fetches all messages, whether seen or not.
keep: Does not delete messages on the server.
nokeep: Deletes the read messages from the server.

Set owner fetchmail for the file fetchmailrc on debian.

$ chown fetchmail /etc/fetchmailrc
$ chmod 0600 /etc/fetchmailrc

The fetchmail daemon restarts.

$ systemctl restart fetchmail

The fetchmail conversation to the external server can be checked with the following command.

$ fetchmail -vv -N --ssl -p pop3 -P 995 -ujohn@gmx.net mail.gmx.net

Test the fetchmailrc configuration file.

$ fetchmail -v -a -k -f /etc/fetchmailrc

Check the fetchmail process on our debian server.

$ ps -ef | grep -v grep | Grep Fetchmail

The output may look something like this.

fetchma+ 23566 1 0 2022 ?        00:01:42 fetchmail -vv -d 900 -a -f /etc/fetchmailrc -L /var/log/fetchmail

Logging now takes place in the fetchmail file on the debian.

$ tail -f /var/log/fetchmail

Something like the following is logged in the fetchmail log file.

fetchmail: awakened at Sat 21 Jan 2023 08:55:45 AM CET
fetchmail: 6.3.24 querying pop.gmx.net (protocol POP3) at Sat 21 Jan 2023 08:55:45 AM CET: poll started
fetchmail: Trying to connect to 212.227.17.169/995...connected.
fetchmail: Certificate chain, from root to peer, starting at depth 2:
fetchmail: Issuer Organization: T-Systems Enterprise Services GmbH
fetchmail: Issuer CommonName: T-TeleSec GlobalRoot Class 3
fetchmail: Server certificate:
fetchmail: Subject CommonName: mail.gmx.net
fetchmail: pop.gmx.net key fingerprint: 36:6D:93:38:DE:58:A2:8B:6D:61:F7:76:1F:56:70:BF
fetchmail: SSL/TLS: using protocol TLSv1.2, cipher ECDHE-RSA-AES256-GCM-SHA384, 256/256 secret/processed bits
fetchmail: POP3< +OK POP server ready H migmx106 1MMFyQ-1p2A592gZq-00YABU
fetchmail: POP3> CAPA
fetchmail: POP3< +OK Capability list follows
fetchmail: POP3< TOP
fetchmail: POP3< UIDL
fetchmail: POP3< USER
fetchmail: POP3< SASL PLAIN
fetchmail: POP3< IMPLEMENTATION trinity
fetchmail: POP3< .
fetchmail: POP3> USER john@gmx.net
fetchmail: POP3< +OK password required for user "john@gmx.net"
fetchmail: POP3> PASS *
fetchmail: POP3< +OK mailbox "john@gmx.net" has 0 messages (0 octets) H migmx106
fetchmail: selecting or re-polling default folder
fetchmail: POP3> STAT
fetchmail: POP3< +OK 0 0
fetchmail: No mail for john@gmx.net at pop.gmx.net
fetchmail: POP3> QUIT
fetchmail: POP3< +OK POP server signing off
fetchmail: 6.3.24 querying pop.gmx.net (protocol POP3) at Sat 21 Jan 2023 08:55:45 AM CET: poll completed
fetchmail: New UID list from pop.gmx.net: <empty>
fetchmail: not swapping UID lists, no UIDs seen this query
fetchmail: Query status=1 (NOMAIL)
fetchmail: sleeping at Sat 21 Jan 2023 08:55:45 AM CET for 900 seconds

  The example shows a login sequence in a somewhat shortened form.

The fetchmail man page provides a lot of useful information.

$ man fetchmail

How to use apt with apt_auth.conf on Debian

If you are in the console shell on a Debian 10 or 11, after running the apt update command, you may get these warning:

N: Usage of apt_auth.conf(5) should be preferred over embedding login information directly in the sources.list(5) entry for 'https://download.kopano.io'

This tutorial will show you how to use Login configuration file for Debian apt sources and proxies. Debian Repositories for users with a valid subscription like Kopano can use the credentials to log in to the repository to gain access that makes able to install packages using apt.

Use apt_auth.conf to Kopano repository

If you have a purchased serial key, it can be used to login to the repository withapt_auth.confor at/etc/apt/auth.conf.d/*.conffiles, this will add the login information to the configuration file for APT sources.

Open in the editor of your choice the file:

/etc/apt/sources.list.d/kopano.list

and remove your login and password information from it.

Create a new/etc/apt/auth.conf.d/kopano.conffile and place in it:

machine download.kopano.io/supported/core:/final/Debian_11/ login serial password XXXXXXXXXXXXXXXXXXXXXXXXX

Supplying login information for a user named serial with the password of your subscription serial key using debian apt.

If you want to deploy Kopano Groupware on Debian 11 (bullseye). Create in/etc/apt/sources.list.d/kopano.listfile and add the following to your Debian apt source:

deb https://download.kopano.io/supported/core:/final/Debian_11/ ./

  as seen here it place without the serial key into the apt source.

The packages are signed so we need to add the key as well.

$ sudo curl -O https://serial:<SERIALKEY>@download.kopano.io/supported/core:/final/Debian_11/Release.key
$ apt-key add Release.key

  I prefer working with elevated privileges as root (“su -“), many people use sudo, so the commands here shown with sudo, but also on Ubuntu you can become root with the (“sudo su -“), simply choose the method that suits for you.

Up to here with Debian 10 everything works fine, but with Debian 11 a warning displayed::

# sudo apt-key add Release.key
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
OK

It’s a warning, not an error. It doesn’t stop the process. The GPG key is added to your system and you can continue adding the external repository. It doesn’t stop the installation of packages.

The system is now ready to authenticate to the repository able to install packages, simply just now run apt update

Using apt-key deprecation and trusted.gpg

This message requires two steps, apt-key is deprecated, Manage keyring files in trusted.gpg.d. With add the keys of a repository, Debian apt and Ubuntu trusts the packages (signed with that key) coming from the repository. If you don’t add the key of a repository, the system won’t allow installing packages from it. It works by adding the keys to separate files located in the /etc/apt/trusted.gpg.d directory. The apt package manager trusts the keys inside this directory.

It’s the same mechanism it uses for the sources list where external repository sources are listed in their own file under /etc/apt/sources.list.d instead of keeping everything under the /etc/apt/sources.list file. It makes managing the external repos convenient.

And this is how it works, first if the gnupg2 package is not already installed, it can be added as follows:

$ sudo apt install -y gnupg2

After that the repository key can be added as follows:

$ sudo curl -sS https://serial:XXXXXXXXXXXXXXXXXXXXXXXXX@download.kopano.io/supported/core:/final/Debian_11/Release.key | gpg --dearmor > /etc/apt/trusted.gpg.d/KopanoRelease.key

Now check the keys are stored in the keyring with run apt-key list

you’ve done it now, Debian or Ubuntu won’t complain anymore.

Debian APT Login configuration file

The APT/etc/apt/auth.conffile and .conf files inside/etc/apt/auth.conf.dcan be used to store login information in a netrc-like format with restrictive file permissions.

The format defined is similar to the format of the~/.netrcfile used by ftp and similar programs interacting with servers.

machine hostname[:port][/path]

Note that apt does not support Digest access authentication, it only does Basic access authentication. As having protocol specified is not wrong (at least not in bullseye) and actually needed if the protocol is not https, so as it doesn’t leak auth info over unencrypted channels.

The authentication to several different repositories can be used flexibly and are suitable for automated processing, here is another example:

machine simple.org/deb login USER password PASSWD
machine repo.other.gov:443 login USER password PASSWD
machine archive.ops.net/sources/ login USER password PASSWD

  Login information in auth.conf and auth.conf.d are more flexible than those in sources.list. For example, login information can be specified for parts of a repository only, or if the sources.list entry redirects elsewhere, login information for the redirect destination can be supplied.