Debian Bug report logs - #31441
[REJECTED] Require both fcntl() and dotlock mailbox locking to prevent mail loss

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: John Goerzen <jgoerzen@complete.org>

To: submit@bugs.debian.org

Subject: mutt: Mail can be lost from inbox!

Date: Sun, 3 Jan 1999 18:57:02 -0600

Package: mutt
Version: 0.95-1
Severity: critical

Here is the sequence of events:

1. I am in my inbox index view.
2. A new mail arrives, placed into the inbox.  Mutt does not yet see this
   mail -- I believe even trying to scroll past the bottom of the
   list doesn't help.
3. I run ls, on the mail server

garfield ~$ ls -l /var/spool/mail/jgoerzen 
-rw-rw----   1 jgoerzen mail       397477 Jan  3 18:52 /var/spool/mail/jgoerzen

I also confirm, by using tail, that a message is there.

4. I press $ to resync mailbox or q to quit, having NOT marked anything
   for deletion.

5. I run ls again:

garfield ~$ ls -l /var/spool/mail/jgoerzen 
-rw-rw----   1 jgoerzen mail       397194 Jan  3 18:51 /var/spool/mail/jgoerzen

6. The message has disappeared from the mailbox forever!

Relevant parts of ~/.muttrc:

set editor=joe

mailboxes /var/spool/mail/jgoerzen

mailboxes =abuse
mailboxes =aclug-announce
mailboxes =aclug-l
mailboxes =archive
mailboxes =catlists
mailboxes =debian-alpha
mailboxes =debian-announce
mailboxes =debian-bugs
mailboxes =debian-changes

etc, etc...

procmail is the delivery agent.  The mail server and spool resides on
garfield.  My workstation is erwin, which uses NFS to get to the mail:

erwin ~$ ls -l /var/spool/mail
lrwxrwxrwx   1 root     root           25 Dec 21 23:27 /var/spool/mail -> /garfield/var/spool/mail/

Similarly:

erwin ~$ ls -l Mail
lrwxrwxrwx   1 jgoerzen jgoerzen        6 Dec 21 23:28 Mail -> g/Mail
erwin ~$ ls -l g
lrwxrwxrwx   1 jgoerzen jgoerzen       24 Dec 21 23:26 g -> /garfield/home/jgoerzen/

for the folders.

-- System Information
Debian Release: 2.1
Kernel Version: Linux erwin 2.1.132 #1 Sat Dec 26 12:31:03 CST 1998 alpha unknown

Versions of the packages mutt depends on:
ii  libc6.1         2.0.7u-5       The GNU C library version 2 (run-time files)
ii  slang1          1.2.2-2        The S-Lang programming library - runtime ver
ii  slang1          1.2.2-2        The S-Lang programming library - runtime ver
ii  sendmail        8.9.1-15       A powerful mail transport agent.
	^^^ (Provides virtual package mail-transport-agent)

Message #10 received at submit@bugs.debian.org (full text, mbox, reply):

From: miquels@cistron.nl (Miquel van Smoorenburg)

To: submit@bugs.debian.org

Subject: Re: Bug#31441: mutt: Mail can be lost from inbox!

Date: 4 Jan 1999 09:43:34 +0100

In article <cistron.199901040057.SAA00491@erwin.complete.org>,
John Goerzen  <jgoerzen@complete.org> wrote:
>Package: mutt
>Version: 0.95-1
>Severity: critical
>
>Here is the sequence of events:
>
>1. I am in my inbox index view.
>2. A new mail arrives, placed into the inbox.  Mutt does not yet see this
>   mail -- I believe even trying to scroll past the bottom of the
>   list doesn't help.

Mailbox mounted over NFS, right? That's more of a kernel problem - it
has been discussed on the linux-kernel mailing list recently.

The problem is the NFS client cache, which doesn't show the mailbox
has changed.

>3. I run ls, on the mail server
>
>garfield ~$ ls -l /var/spool/mail/jgoerzen 
>-rw-rw----   1 jgoerzen mail       397477 Jan  3 18:52 /var/spool/mail/jgoerzen
>
>I also confirm, by using tail, that a message is there.

On the server, yes. If you did the same on the client you'd see that
the contents hadn't changed yet at that point, so it's not mutt's fault.

There are 2 solutions:

1. Mount the mailspool without (or minimal) client attribute timeout:
   See nfs(5). You probably want to mount with "actimeo=0" or even  "noac".

2. Use a 2.2pre kernel. I think the problem has been solved there- as
   soon as you use flock() on a file, the attribute cache is invalidated.
   Ofcourse the mail program (mutt, or mutt_dotlock) needs to lock the
   mailbox with BOTH dotlocking and fcntl.

Because of this issue we might need to change policy about mailbox
locking, requiring both fcntl() and dotlock locking. A note should
be added about the "noac" option for 2.0 kernels. Liblockfile
will need to be fixed too.

This bug should be reassigned to an appropriate package- perhaps
debian-policy or liblockfile.

Mike.
-- 
Indifference will certainly be the downfall of mankind, but who cares?

Message #15 received at 31441@bugs.debian.org (full text, mbox, reply):

From: Thomas Roessler <roessler@guug.de>

To: md@linux.it, "The mutt developers' list" <mutt-dev@mutt.org>

Cc: 31441@bugs.debian.org

Subject: Re: critical bug against mutt

Date: Mon, 4 Jan 1999 13:21:05 +0100

On Mon, Jan 04, 1999 at 12:00:12PM +0100, Marco d'Itri wrote:
> Please look at http://www.debian.org/Bugs/db/31/31441.html

The problem which was observed looks indeed very strange.  I'm ccing
this to mutt-dev, maybe someone there can comment a bit more.

>garfield ~$ ls -l /var/spool/mail/jgoerzen
>-rw-rw----   1 jgoerzen mail       397477 Jan  3 18:52 /var/spool/mail/jgoerzen

>I also confirm, by using tail, that a message is there.

Fine so far.

>4. I press $ to resync mailbox or q to quit, having NOT marked anything
>   for deletion.

resynching a mailbox will first lead to a check whether the mailbox
has been modified.  This check involves comparing the folder's mtime
and size to the values stored by mutt.  Afterwards, all changes you
made to your folder (this includes marking messages as read) are
written back to the file.  Please note that the check occurs _after_
the folder has been locked.

What you are describing here looks like an NFS client not seeing
modifications on the server.

>5. I run ls again:

>garfield ~$ ls -l /var/spool/mail/jgoerzen
>-rw-rw----   1 jgoerzen mail       397194 Jan  3 18:51 /var/spool/mail/jgoerzen

This is even funnier, the file's mtime moving backward in time.

If the debian mutt version was compiled with NFS_ATTRIBUTE_HACK
defined, it may be due to the fact that mutt didn't get correct
information from a stat call.  If NFS_ATTRIBUTE_HACK was not
defined, I don't have any idea how this backwards-modification may
have occured.

Basically, I'm almost sure that this a problem with the NFS
implementation or configuration in use.  It may be interesting to
repeatedly stat the spool file from the client while mail gets
deliverd.  If it's really an NFS bug, you should see some
differences between NFs client and NFS server.

tlr
-- 
Thomas Roessler · 74a353cc0b19 · dg1ktr · http://home.pages.de/~roessler/
     2048/CE6AC6C1 · 4E 04 F0 BC 72 FF 14 23 44 85 D1 A1 3B B0 73 C1

Message #20 received at 31441@bugs.debian.org (full text, mbox, reply):

From: md@linux.it (Marco d'Itri)

To: 31441@bugs.debian.org, control@bugs.debian.org

Cc: debian-policy@lists.debian.org, debian-devel@lists.debian.org

Subject: Re: Bug#31441: mutt: Mail can be lost from inbox!

Date: Mon, 4 Jan 1999 23:52:25 +0100

reassign 31441 debian-policy
thanks

Looks like this is not a bug in mutt, so I'll reassign it to the policy
package.

-- 
ciao,
Marco

Message #37 received at 31441-done@bugs.debian.org (full text, mbox, reply):

From: Julian Gilbey <J.D.Gilbey@qmw.ac.uk>

To: 31441-done@bugs.debian.org, 43529-done@bugs.debian.org

Subject: Closing NFS-safe policy bugs

Date: Wed, 21 Jun 2000 18:35:15 +0100

The dotlock + flock mailbox locking combination is already required
for MTAs, MUAs etc in policy 3.1.1.1.  So I'm closing these bug
reports.  If mutt still suffers from these problems, please reopen the
relevant bug and reassign it to mutt.  (Instructions in
/usr/doc/debian.)

   Julian

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

  Julian Gilbey, Dept of Maths, QMW, Univ. of London. J.D.Gilbey@qmw.ac.uk
        Debian GNU/Linux Developer,  see http://www.debian.org/~jdg
  Donate free food to the world's hungry: see http://www.thehungersite.com/

Send a report that this bug log contains spam.