Re: Courier or Cyrus
--On Thursday, March 17, 2005 07:40 -0300 Henrique de Moraes Holschuh
<hmh@debian.org> wrote:
On Tue, 15 Mar 2005, Michael Loftis wrote:
million emails/day. The only issue we've run into is an occasional
deadlock with POP3 that requires rebooting. And that happens once every
few months, if that. This is with 2.1.17.
The Debian packages should fix that POP3 deadlock possibility, I think.
Something else that can cause POP3 deadlocks is apop (enabled by default),
if your box runs out of /dev/random entropy. Just disable apop if you
don't need it.
Just had it do that this AM on our production server. 85 days 20 hrs 40
minutes.
But a deadlock that requires a reboot? Don't you mean a Cyrus restart? If
it requires a _reboot_, something is very wrong indeed and I had never
heard of anythink like this.
No, a reboot. Not cyrus restart. Cyrus restart doesn't' fix the POP3
deadlock (and yes I make sure no pop3/pop3proxy processes are lying
around). Complete, full reboot. It did this to us in RedHat 7.3 systems
too, so it's nothing specific to Debian woody+your fulldirhash package.
On a restart pop3 comes up totally deadlocked like before. Only thing that
clears it is rebooting the machine. During these issues lmtp, imap, etc,
all work perfectly normal and pop3 is the ONLY service affected.
As to what/why/how, I don't know. We've totally changed hardware since
RH7.3 but we're still in the 2.1 series (2.1.17 Debian version, from you)
I had to add in a few patches but those weren't present in our RH7.3 env
except for one, I have to disable the groupchecks stuff since it ends up
downloading nearly our entire LDAP database for every login. The other
patches are all to make the hostname reporting consistent, mostly cosmetic.
And no, we can't reproduce it reliably...sometimes go days, sometimes
months...this time we went 80 some odd days. And yes I will reiterate,
nothing but a _reboot_ will get the machine back. :) Interesting eh?
Most of our problems go back to incoherent clients, mostly Outlook, but
occasionally Thunderbird -- turn on it's junk mail filtering, file to
say
Amem to that. I think Cyrus 2.2 and Debian Cyrus 2.1 can handle most
outlooketies just fine, but I would not bet anything on it. One must
always be on the look out for any new version of OutLook and all the bugs
that it brings with it...
Debian Cyrus 2.1 seems to handle lookout just fine, or atleast we don't get
any complaints. :) (Honest, as much as I want to I'm no BOFH :))
> entire point of maildirs). For example, maildir makes backing up and
> restoring ranges of email very easy.
Cyrus does that too, since it is just MH + indexes. You just reload from
backup media and run a reconstruct in that single mailbox, and incremental
backups work just fine (the message data is never changed by Cyrus once in
the spool, just the indexes).
Yeah but what if the mailbox doesn't exist in the current mailboxes db?
Then you're screwed. You have to recreate all of the mailbox/folder
structure for that user and THEN restore and then reconstruct. It's a
pain. If reconstruct could fix/readd mailboxes it'd probably work.
Hmm, come to think about it, maildir is probably superior in that it
stores the message flags in-band which is nicer for restores -- cyrus
will mark all restored messages as unread and lose all other flags on
that message as well). OTOH, how does one manage to have annotations and
per-user flags (such as seen state) using maildir?
I think they get stored in an X-IMAP-Flags: or like header.
There is one serious problem. He'll need multiple servers since the
unix UID limit hits at 65535. So you can only get about 64k users
created per
Huh? More like 2^31 in Debian Linux 2.4/2.6 kernels. But over 64K users
per box will mean bad performance unless these servers are really
something else, or your users are not that much active to begin with, so
I don't think this counts against courrier-imap.
Alright I'm not sure where i got that number heh...i know better, it's a
31/32bit UID. *sigh* more coffee. I need to run the coffee machine 24/7
here in the office.
I've yet to find a good way to backup and restore cyrus mail...basically
it's a pain to do. Thankfully we've only ever very rarely had a need to
restore mail.
There is a lot of stuff about it on the Cyrus wiki and ML archives. I too
rarely need to restore mail, and simple amanda incremental backups work
just fine for that (the effort it takes on restores is caused by amanda,
not Cyrus).
Except, as above, when the users mailbox is totally deleted -- happens on
occasion we rid a customer and they want/need to come back.
Redundancy using murder is coming soon, I think. So far, people do the
usual HA things for redundancy (but don't even think about Cyrus on top of
NFS :P).
Yeah we're just running with our system setup so I can migrate everything
to a different piece of hardware as head end manually.
AS LONG AS you do not need cluster-wide shared mailboxes, at which point a
Cyrus murder cluster is the only thing that will work, AFAIK (and I would
love to hear otherwise!).
Yup and the MURDER needs very little care and feeding once setup. It just
works :)
That's a LOT of mail. You could easily need half a dozen very beefy
boxes to handle that much mail depending on how much spam/virus/etc
features you want.
Indeed. SPAM and AV processing for this would require a small cluster of
workers of its own, with good local IO and lots of CPU power (and a good
enough ammount of RAM that you can use for IO caching).
--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh
--
Michael Loftis
Modwest Operations Manager
GPG/PGP --> 0xE736BD7E 5144 6A2D 977A 6651 DFBE 1462 E351 88B9 E736 BD7E
Reply to: