Debian Bug report logs - #6786
Packages files should be mirrored after the .debs

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: "Stephen J. Turnbull" <turnbull@turnbull.sk.tsukuba.ac.jp>

To: submit@bugs.debian.org

Subject: dselect and mirrors with incomplete archives

Date: Thu, 23 Jan 1997 13:04:50 +0900

Package: dpkg
Version: 1.4.0.6

I am using Debian 1.2, kernel version 2.0.27, and libc 5.4.13.

I observed the following problem of inconsistency between Packages and
the actual archive with a Debian mirror and reported it to the
mirrors.  Since I think it is a bug in the package management system,
I am reporting it here.  As I don't understand dpkg yet (I haven't
successfully installed the system, so don't really have access to docs
etc), I'm going to make some suggestions, but if they seem orthogonal
to common sense, they probably are.

I looked at titles to the bugs regarding dpkg, and didn't see anything 
related.  With my current bandwidth, actually reading all the reports
would take most of this week; sorry if I missed this one.

>>>>> At 11:44 AM 23/01/97 +0900, I wrote:

    me> I have been having a lot of problems with installation from the
    me> Debian Linux mirror at ftp.kreonet.re.kr.

    me> The basic cause seems to be that the package list was mirrored
    me> _before_ getting all the packages.  Thus many of the files are
    me> older than the version mentioned in the package list.  In
    me> particular, ftp.kreonet.re.kr claims to have libc_5.4.17-X.deb,
    me> but only has libc_5.4.13-X.deb.  Since many of the packages
    me> already updated depend on having at least libc 5.4.17, dselect
    me> goes crazy, and there are many errors in installation so dpkg
    me> can't recover.

    me> I suspect that this could be a problem at many overseas
    me> mirrors, as bandwidth really stinks recently.

>>>>> "Karl" == Karl Ferguson <karl@tower.net.au> responded:

    Karl> Unfortuantely we can't do much about that.  If files
    Karl> time-out enough when mirroring and mirror decided to spit
    Karl> the dummy and exit, then you'll end up having updated files
    Karl> all over the place.  It also depends on here the mirror is
    Karl> mirroring from.

    me> This really is a defect in the package management system.  As
    me> it is not possible to guarantee timely updates of a full
    me> mirror, the package list should be generated on the fly at the
    me> mirror.

    Karl> Bugs with mirrors should be reported directly to the mirror
    Karl> admins (as you see listed in the README.mirrors file) - it's
    Karl> not really the mirrors responsability to have thir own
    Karl> package maintenance system, if we demanded this then I doubt
    Karl> we'd have any mirrors left at all - I'm a mirror myself, and
    Karl> I wouldn't want that.

I agree with this; it seems to me that developing a script for
generating Packages rests with the dpkg maintenance team.  Since you
presumably already have one, it should be possible to modify it for
use by mirrors.  I recognize that this could potentially be annoying
because of various packages and versions of mirroring software, but
since dselect is the first thing us newbies see, you could chase all
of us back to RedHat and we wouldn't want that :-)

    Karl> Packages(.gz) etc are all updated and retrieved first during
    Karl> a mirror because it's first on the list.  As I said before,
    Karl> we can't help it if mirror spits the dummy and exits due to
    Karl> errors.

Right.  At a minimum, Packages ought to be the *last* thing on the
list, and *old* packages should not be purged (the Debian naming
system guarantees they won't be overwritten, as far as I can see)
until Packages is updated.  Yes, this leaves you with an old mirror
for a potentially long period of time, but at least it's consistent.

An option would be to add a "stable-in-process" distribution for
the adventurous.  This would only appear at mirrors with relatively
low bandwidth and long transition periods.

Of course, both of these options require a lot more space of the
mirror :-(

Also, dselect should probably (optionally) recheck the dependencies
after getting the packages to make sure that the packages received
actually do satisfy the dependency tree.

dselect doesn't seem very smart about large numbers of packages; maybe 
new users should be advised to do the installations incrementally.  It 
also would be helpful to add an outline feature (hiding/showing
contents below a header) to dselect.  (Yeah, I know, I ought to put
this in a separate report, but I don't feel like wrestling with the
transmission lags to check the bug list.)

Thanks for your attention to these problems.

Stephen Turnbull

-- 
                            Stephen J. Turnbull
Institute of Policy and Planning Sciences                    Yaseppochi-Gumi
University of Tsukuba                      http://turnbull.sk.tsukuba.ac.jp/
Tel: +81 (298) 53-5091;  Fax: 55-3849              turnbull@sk.tsukuba.ac.jp

Message #10 received at 6786-maintonly@bugs.debian.org (full text, mbox, reply):

From: remco@blaakmeer.student.utwente.nl (Remco Blaakmeer)

To: 6786-maintonly@bugs.debian.org

Subject: Old bugs need to be looked at

Date: Sun, 1 Feb 1998 12:29:50 +0100 (CET)

This is an automated message sent to all bugs older than one year.

This bug is very old. Please take a look at it and see if you can fix it.
If it has already been fixed, please close it.

If you have problems fixing it or if you don't have the time to fix it,
please ask the people on debian-devel@lists.debian.org for help, so that
at least the oldest bugs can be solved before Debian 2.0 is released.

Remco Blaakmeer

Message #21 received at 85521-done@bugs.debian.org (full text, mbox, reply):

From: James Troup <james@nocrew.org>

To: 85521-done@bugs.debian.org (Debian Bug Tracking System)

Subject: Re: Processed: apt bug cleanup

Date: 12 Jan 2003 06:31:01 +0000

owner@bugs.debian.org (Debian Bug Tracking System) writes:

> > reassign 85521 ftp.debian.org
> Bug#85521: apt-get gives up too early
> Bug reassigned from package `apt' to `ftp.debian.org'.

If you're reassigning this based on "how we mirror sucks", that's a
well known problem and already report, there's no need to keep this
open just for that.

-- 
James

Message #30 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Thomas Koenig <Thomas.Koenig@online.de>

To: Debian Bug Tracking System <6786@bugs.debian.org>

Subject: ftp.debian.org: Packages still to early

Date: Mon, 14 Apr 2003 22:31:25 +0200

Package: ftp.debian.org
Version: unavailable; reported 2003-04-14
Followup-For: Bug #6786

This is still a fairly frequent, and somewhat annoying, problem.

# date -u
Mon Apr 14 20:30:38 UTC 2003
# apt-get -u dist-upgrade
Reading Package Lists... Done
Building Dependency Tree... Done
Calculating Upgrade... Done
The following packages will be upgraded
  binutils castle-combat chkrootkit flex hpijs libgnutls5 librpm4 rpm samba-common smbclient symlinks w3m
12 packages upgraded, 0 newly installed, 0 to remove and 0  not upgraded.
Need to get 8662kB of archives. After unpacking 549kB will be used.
Do you want to continue? [Y/n] y
Get:1 ftp://ftp.debian.org unstable/main binutils 2.13.90.0.18-1.7 [2026kB]
Err ftp://ftp.debian.org unstable/main binutils 2.13.90.0.18-1.7
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:2 ftp://ftp.debian.org unstable/main flex 2.5.31-3 [342kB]
Err ftp://ftp.debian.org unstable/main flex 2.5.31-3
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:3 ftp://ftp.debian.org unstable/main castle-combat 0.7.3-1 [1547kB]
Err ftp://ftp.debian.org unstable/main castle-combat 0.7.3-1
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:4 ftp://ftp.debian.org unstable/main chkrootkit 0.40-1 [184kB]
Err ftp://ftp.debian.org unstable/main chkrootkit 0.40-1
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:5 ftp://ftp.debian.org unstable/main hpijs 1.3.1-1 [136kB]
Err ftp://ftp.debian.org unstable/main hpijs 1.3.1-1
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:6 ftp://ftp.debian.org unstable/main libgnutls5 0.8.6-3 [182kB]
Err ftp://ftp.debian.org unstable/main libgnutls5 0.8.6-3
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:7 ftp://ftp.debian.org unstable/main librpm4 4.0.4-20 [354kB]
Err ftp://ftp.debian.org unstable/main librpm4 4.0.4-20
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:8 ftp://ftp.debian.org unstable/main rpm 4.0.4-20 [515kB]
Err ftp://ftp.debian.org unstable/main rpm 4.0.4-20
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:9 ftp://ftp.debian.org unstable/main smbclient 2.999+3.0.alpha23-3 [1416kB]
Err ftp://ftp.debian.org unstable/main smbclient 2.999+3.0.alpha23-3
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:10 ftp://ftp.debian.org unstable/main samba-common 2.999+3.0.alpha23-3 [1284kB]
Err ftp://ftp.debian.org unstable/main samba-common 2.999+3.0.alpha23-3
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:11 ftp://ftp.debian.org unstable/main symlinks 1.2-4.1 [8064B]
Err ftp://ftp.debian.org unstable/main symlinks 1.2-4.1
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Get:12 ftp://ftp.debian.org unstable/main w3m 0.4.1-4 [668kB]
Err ftp://ftp.debian.org unstable/main w3m 0.4.1-4
  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/b/binutils/binutils_2.13.90.0.18-1.7_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/f/flex/flex_2.5.31-3_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/c/castle-combat/castle-combat_0.7.3-1_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/c/chkrootkit/chkrootkit_0.40-1_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/h/hpijs/hpijs_1.3.1-1_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/g/gnutls5/libgnutls5_0.8.6-3_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/r/rpm/librpm4_4.0.4-20_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/r/rpm/rpm_4.0.4-20_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/s/samba/smbclient_2.999+3.0.alpha23-3_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/s/samba/samba-common_2.999+3.0.alpha23-3_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/s/symlinks/symlinks_1.2-4.1_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
Failed to fetch ftp://ftp.debian.org/debian/pool/main/w/w3m/w3m_0.4.1-4_i386.deb  Unable to fetch file, server said 'Failed to open file.  ' [IP: 208.185.25.38 21]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
-- System Information:
Debian Release: testing/unstable
Architecture: i386
Kernel: Linux meiner 2.4.20 #1 Mon Feb 3 22:42:48 CET 2003 i686
Locale: LANG=C, LC_CTYPE=de_DE@euro

Message #35 received at 6786@bugs.debian.org (full text, mbox, reply):

From: "Artur R. Czechowski" <arturcz@hell.pl>

To: 6786@bugs.debian.org

Subject: Packages files should be mirrored after the .debs - proposed solution

Date: Sun, 29 Jun 2003 01:05:30 +0200

Hello

Solution of this problem could be mirroring in two stages:
1. mirror pool without delete files
2. mirror whole area, deleting old files after fetching files
It is not hard to change mirror scripts to do it.

Pro:
 - all files provided by Packages/Sources exists on mirror

Contra:
 - it requires more space on mirror machine, for some minutes it needs
   to store both (previous and new) version of upgraded packages.

I sent a Bcc to debian-mirrors, because mirror's administrators are
people who can mostly help to solve this problem. Let moderator
decide if it is important enough to disturb them. There is
a blind copy, to not annoying moderator (and list) with replies
if not interested.

If moderator decide to pass it to the list please take a look at:
http://bugs.debian.org/6786 to more information about this problem.
If you want to submit any additional information about this bug
please address (or Cc) it to 6786@bugs.debian.org.

Regards
	Artur

Message #46 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Matt Zimmerman <mdz@debian.org>

To: Dan Jacobson <jidanni@jidanni.org>

Cc: debian-devel@lists.debian.org, 6786@bugs.debian.org

Subject: Re: breaks Debian package system worldwide several hours a day

Date: Fri, 10 Sep 2004 16:25:43 -0700

severity 6786 normal
thanks

On Sat, Sep 11, 2004 at 02:39:52AM +0800, Dan Jacobson wrote:

> severity 217957 critical
> severity 6786 critical
> merge 6786 217957
> # critical: breaks Debian package system worldwide several hours a day.
> # How to fix: Packages files should get to mirrors last.

critical
    makes unrelated software on the system (or the whole system) break, or
causes serious data loss, or introduces a security hole on systems where you
install the package.

normal
    the default value, applicable to most bugs.

You need to read:

- The code of conduct for Debian mailing lists:
  http://www.debian.org/MailingLists/#codeofconduct

- The definitions of severity levels in the Debian bug tracking system:
  http://www.debian.org/Bugs/Developer#severities

and generally stop behaving disrespectfully in your interaction with those
two systems.

-- 
 - mdz

Message #53 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Dan Jacobson <jidanni@jidanni.org>

To: debian-devel@lists.debian.org

Cc: 6786@bugs.debian.org

Subject: mirror the Packages files _after_ the packages!

Date: Wed, 09 Feb 2005 09:51:44 +0800

I know you Debian people think it's just hilarious when users try to
apt-get upgrade during the period after when the Packages files
arrive on the mirrors, but before the packages they describe have
fully arrived.

"Haw haw haw, try again later", you say, never thinking that maybe
writing the Packages files last would be the right thing to do.

"The problem can't persist more then a couple of hours, what's the big
deal?"

Well, the connection to mirror-upstream broke mid-mirror-push this
time during Chinese New Years. Perhaps 10 days before the system
administrator will be back in the office to see what happened.

"Haw haw haw, try another mirror.  Haw haw haw."

The other local mirrors have the same problem, and their system admins are
also on vacation.  Foreign mirrors are on slow links.

So not fixing bug 6786 (just write the $#@% Packages file after
writing the packages) causes users' apt-get upgrades to fail
needlessly... down for 10 days. Great.

The Debian whippersnappers don't see the danger of needlessly leaving
mirrors in a broken state "for only a couple of hours a day, what's
the big deal" ... well if the mirror mechanism breaks during this
broken state, then "couple of hours" becomes indefinitely.

I can't think of any other case in Computer Science where one updates
a descriptor before updating the thing described!!! How can you defend
that???  No smug remarks can defend that!

"Never bothered me", yeah well wait until you are giving your next
apt-get upgrade demonstration and now it's too late, you did
apt-get update && apt-get upgrade instead of
apt-get upgrade && apt-get update && apt-get upgrade
and now you have to tell the class to "wait a couple of hours" until
you can show them anything.

Yeah I know, "the more I complain, the more it's not going to get
done."
OK, can somebody send me the section of code so I can fix it then?

Err http://xxxxxx.linux.org.xx sid/main xxxxxxxxxxxx  404 Not Found
Failed to fetch http://xxxxxx.linux.org.xx/debian/pool/main/xxxxx
Fetched 26.6MB in 2m39s (167kB/s)
E: Some files failed to download
Error 100

P.S., "a couple hours" is for your fancy mirror's connections.
Some mirrors far away might be in the broken state (Packages* arrived,
not all packages arrived) for half a day each day!

Message #58 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Martin Zobel-Helas <mhelas@helas.net>

To: Dan Jacobson <jidanni@jidanni.org>

Cc: debian-devel@lists.debian.org, 6786@bugs.debian.org

Subject: Re: mirror the Packages files _after_ the packages!

Date: Sat, 12 Feb 2005 19:22:17 +0100

Hi Dan,

On Wednesday, 09 Feb 2005, Dan Jacobson <jidanni@jidanni.org> wrote:
> I know you Debian people think it's just hilarious when users try to
> apt-get upgrade during the period after when the Packages files
> arrive on the mirrors, but before the packages they describe have
> fully arrived.

see http://lists.debian.org/debian-www/2005/01/msg00199.html

Greetings
Martin
--
Love is the only game that is not called on account of darkness.
		-- M. Hirschfield

Message #63 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Steve Langasek <vorlon@debian.org>

To: Dan Jacobson <jidanni@jidanni.org>

Cc: debian-devel@lists.debian.org, 6786@bugs.debian.org

Subject: Re: mirror the Packages files _after_ the packages!

Date: Sun, 13 Feb 2005 13:48:43 -0800

[Message part 1 (text/plain, inline)]

On Wed, Feb 09, 2005 at 09:51:44AM +0800, Dan Jacobson wrote:
> I know you Debian people think it's just hilarious when users try to
> apt-get upgrade during the period after when the Packages files
> arrive on the mirrors, but before the packages they describe have
> fully arrived.

> "Haw haw haw, try again later", you say, never thinking that maybe
> writing the Packages files last would be the right thing to do.

> "The problem can't persist more then a couple of hours, what's the big
> deal?"

You've been told this before -- *debian-devel does not control the mirroring
implementation used by arbitrary Debian mirrors*.  Either talk to the mirror
team and give them enough information to track this down, or -- since you
know him well enough to be kept in the loop about his vacation schedule --
talk to your local mirror operator directly and get him to stop using broken
mirroring scripts.

> I can't think of any other case in Computer Science where one updates
> a descriptor before updating the thing described!!! How can you defend
> that???  No smug remarks can defend that!

And no amount of moronic, misdirected ranting will get it fixed.

> Err http://xxxxxx.linux.org.xx sid/main xxxxxxxxxxxx  404 Not Found
> Failed to fetch http://xxxxxx.linux.org.xx/debian/pool/main/xxxxx
> Fetched 26.6MB in 2m39s (167kB/s)
> E: Some files failed to download
> Error 100

Yeah, real helpful of you to blot out the only potentially useful bit of
information in your post...

-- 
Steve Langasek
postmodern programmer

[signature.asc (application/pgp-signature, inline)]

Message #68 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Dan Jacobson <jidanni@jidanni.org>

To: debian-devel@lists.debian.org

Cc: 6786@bugs.debian.org

Subject: Re: mirror the Packages files _after_ the packages!

Date: Tue, 15 Feb 2005 23:51:10 +0800

S> You've been told this before -- *debian-devel does not control the mirroring
S> implementation used by arbitrary Debian mirrors*.  Either talk to the mirror
S> team and give them enough information to track this down, or -- since you
S> know him well enough to be kept in the loop about his vacation schedule --
S> talk to your local mirror operator directly and get him to stop using broken
S> mirroring scripts.

I'm saying that bug 6786 has the potential to turn the current perhaps
two hour per day down time for apt-get, aptitude, etc. into a several
day long down time.

D> > Failed to fetch http://xxxxxx.linux.org.xx/debian/pool/main/xxxxx

S> Yeah, real helpful of you to blot out the only potentially useful bit of
S> information in your post...

No. The root of the problem is bug 6786.  Indeed if 6786 were fixed,
the mirror process could break at any time and users could still
apt-get upgrade with yesterdays state instead of not being to apt-get
upgrade at all (if the mirror process happen to break during the 2
hour dark period each day.) Indeed, no 2 hour dark period necessary too.

Please double check 6786 to see if it is merely a local problem.
Would you close it?

Message #73 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Jeroen van Wolffelaar <jeroen@wolffelaar.nl>

To: Dan Jacobson <jidanni@jidanni.org>

Cc: debian-devel@lists.debian.org, 6786@bugs.debian.org

Subject: Re: mirror the Packages files _after_ the packages!

Date: Tue, 15 Feb 2005 19:56:04 +0100

On Tue, Feb 15, 2005 at 11:51:10PM +0800, Dan Jacobson wrote:
> S> You've been told this before -- *debian-devel does not control the mirroring
> S> implementation used by arbitrary Debian mirrors*.  Either talk to the mirror
> S> team and give them enough information to track this down, or -- since you
> S> know him well enough to be kept in the loop about his vacation schedule --
> S> talk to your local mirror operator directly and get him to stop using broken
> S> mirroring scripts.
> 
> I'm saying that bug 6786 has the potential to turn the current perhaps
> two hour per day down time for apt-get, aptitude, etc. into a several
> day long down time.

How mirrors do their mirroring is up to the local mirror administrator,
not to the general debian developer's community. Debian could promote
this two-phase mirroring a bit more maybe, and/or provide nice scripts,
that's probably why #6786 is still open.

BUT DEBIAN CANNOT IN ANY WAY FORCE/CONTROL HOW MIRRORS OPERATE,
especially not the secundary mirrors.
 
> D> > Failed to fetch http://xxxxxx.linux.org.xx/debian/pool/main/xxxxx
> 
> S> Yeah, real helpful of you to blot out the only potentially useful bit of
> S> information in your post...
> 
> No. The root of the problem is bug 6786.  Indeed if 6786 were fixed,

#6786 doesn't need to be fixed for this, the mirror admin of
xxxxxx.linux.org.xx just needs to implement two-phase mirroring,
something that anyone with a bit shell/rsync foo can implement on
his/her own, and ttbomk, already a lot of mirrors actually _do_.

As numerous people have told you before, bugger your local mirror admin,
especially since he seems to have slow bandwidth and therefore the 'dark
period' takes long.

--Jeroen

-- 
Jeroen van Wolffelaar
Jeroen@wolffelaar.nl (also for Jabber & MSN; ICQ: 33944357)
http://Jeroen.A-Eskwadraat.nl

Message #78 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Dan Jacobson <jidanni@jidanni.org>

To: debian-devel@lists.debian.org, 6786@bugs.debian.org

Subject: Re: mirror the Packages files _after_ the packages!

Date: Wed, 16 Feb 2005 08:39:03 +0800

Jeroen> Debian could promote this two-phase mirroring a bit more
Jeroen> maybe, and/or provide nice scripts, that's probably why #6786
Jeroen> is still open.

I suppose Debian is promoting one-phase mirroring and two phase
mirroring is "roll your own".

If you want me to tell my administrator to do something, I need
something succinct, like "apt-get install two-phase-mirroring".

Jeroen> #6786 doesn't need to be fixed for this, the mirror admin of
Jeroen> xxxxxx.linux.org.xx just needs to implement two-phase mirroring,
Jeroen> something that anyone with a bit shell/rsync foo can implement on
Jeroen> his/her own, and ttbomk, already a lot of mirrors actually _do_.

If this two-phase-mirroring is the way to go, then there ought to be a
standard package to do it. How can there be 10000+ packages but no
package to make sure the 10000+ packages get mirrored properly?

Any approved recommended debugged method would certainly have its own
package, where the administrator would merely enter some configuration
parameters in some /etc file.

Does http://www.debian.org/mirror/ at least have the
two-phase-mirroring script?

Is there any official documentation?

Message #83 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Frank Küster <frank@debian.org>

To: Dan Jacobson <jidanni@jidanni.org>

Cc: debian-devel@lists.debian.org, 6786@bugs.debian.org

Subject: Re: mirror the Packages files _after_ the packages!

Date: Wed, 16 Feb 2005 18:55:20 +0100

Dan Jacobson <jidanni@jidanni.org> schrieb:

> Does http://www.debian.org/mirror/ at least have the
> two-phase-mirroring script?

Why don't you go and look? And if you find the information too sparse,
submit a bug report with a patch?

Regards, Frank
-- 
Frank Küster
Inst. f. Biochemie der Univ. Zürich
Debian Developer

Message #88 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Joey Hess <joeyh@debian.org>

To: 6786@bugs.debian.org

Cc: Anthony Towns <aj@azure.humbug.org.au>

Subject: Re: Mirror split stuff

Date: Fri, 13 Jan 2006 15:44:15 -0500

[Message part 1 (text/plain, inline)]

Anthony Towns wrote:
>     rsync -av --progress --delay-updates -a 
>         --files-from :indices/files/arch-i386.files \
>         --delete --delete-after --max-delete=1000 \
>         rsync://MIRROR/debian/ ./

The file list currently sorts dists/ near the top of the list. If it
were reordered so that dists was listed last then mirrors updating this
way would not have bug #6786. You're already half way there with the
--delete-after.

Closing such an old bug is sure to appeal, so I thought I'd mention it.
:-)

-- 
see shy jo

[signature.asc (application/pgp-signature, inline)]

Message #93 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Ryan Murray <rmurray@debian.org>

To: Joey Hess <joeyh@debian.org>, 6786@bugs.debian.org

Subject: Re: Bug#6786: Mirror split stuff

Date: Fri, 13 Jan 2006 17:31:44 -0800

On Fri, Jan 13, 2006 at 03:44:15PM -0500, Joey Hess wrote:
> Anthony Towns wrote:
> >     rsync -av --progress --delay-updates -a 
> >         --files-from :indices/files/arch-i386.files \
> >         --delete --delete-after --max-delete=1000 \
> >         rsync://MIRROR/debian/ ./
> 
> The file list currently sorts dists/ near the top of the list. If it
> were reordered so that dists was listed last then mirrors updating this
> way would not have bug #6786. You're already half way there with the
> --delete-after.

a)
	the mirror script being talked about uses two passes through the
archive to minimize this problem

b)
	with --delay-updates (new sarge option), the problem goes away
as best as it can in any case, as all files are mirrored elsewhere,
and then renamed into place when they are all there.

> Closing such an old bug is sure to appeal, so I thought I'd mention it.

The use of --delay-updates for the indices does this already, but I don't
think the script has been updated in many places.
(it has been on ftp.d.o, tho)

Message #98 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Anthony Towns <aj@azure.humbug.org.au>

To: Joey Hess <joeyh@debian.org>

Cc: 6786@bugs.debian.org

Subject: Re: Mirror split stuff

Date: Mon, 16 Jan 2006 22:36:16 +1000

[Message part 1 (text/plain, inline)]

On Fri, Jan 13, 2006 at 03:44:15PM -0500, Joey Hess wrote:
> Anthony Towns wrote:
> >     rsync -av --progress --delay-updates -a 
> >         --files-from :indices/files/arch-i386.files \
> >         --delete --delete-after --max-delete=1000 \
> >         rsync://MIRROR/debian/ ./
> The file list currently sorts dists/ near the top of the list. If it
> were reordered so that dists was listed last then mirrors updating this
> way would not have bug #6786. You're already half way there with the
> --delete-after.

As it turns out, rsync seems to just reorder the stuff alphabetically anyway.
Changed anyway.

> Closing such an old bug is sure to appeal, so I thought I'd mention it.
> :-)

--delay-updates gets pretty close to the right behaviour though.
Note that anonftpsync already handles this specially.

Cheers,
aj

[signature.asc (application/pgp-signature, inline)]

Message #103 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: status

Date: Sat, 15 Sep 2007 00:22:41 +0200

Hi,

Okay, so we know solutions for this bug. The web site documentation and the
official anonftpsync script have been updated to apply the solution for
about two years now. The only thing missing is - practical implementation :)

I think it would be fair to say that we should get all official mirror
sites to get packages before Packages files, and then close the bug.
Individual unofficial mirrors that stay unfixed should be a matter for
separate bug reports.

This is the current status of setups for official mirrors:

* syncproxy.eu.debian.org - done
* syncproxy.wna.debian.org - done
* ftp.us.debian.org:
  * saens.debian.org (ftp.d.o too) - done
  * ike.egr.msu.edu - being fixed these days - I'm talking to them
  * mirrors.kernel.org - not fixed yet - I need to update
    ries:~archvsync/ftpsync-push*
  * debian.osuosl.org - don't know yet (have to get rsyncstats fixed -
    cf. RT#172)
* ftp.uk.debian.org - done
* ftp.wa.au.debian.org - ? mirrors from ftp.au.d.o
* ftp.dk.debian.org - ? mirrors from ftp.nl.d.o
* ftp.it.debian.org - ? mirrors from ftp2.de.d.o
* ftp.cz.debian.org - done
* ftp.sk.debian.org - done
* ftp.at.debian.org - not done - will contact them
* ftp.de.debian.org - done
* ftp.jp.debian.org - ? mirrors from hp.debian.or.jp (via syncproxy.wna.d.o)
* ftp2.jp.debian.org - ? mirrors from hp.debian.or.jp (via syncproxy.wna.d.o)
* ftp.bg.debian.org - ? mirrors from ftp2.de.d.o
* ftp.ru.debian.org - ? mirrors from ftp.se.d.o
* ftp.pl.debian.org - done
* ftp2.de.debian.org - done
* ftp.pt.debian.org - done
* ftp.fr.debian.org - not done - will contact them
* ftp.tw.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
* ftp.si.debian.org - not done - will contact them
* ftp.hk.debian.org - ? should mirror from ftp.de.d.o, but is semi-dead (!)
* ftp.no.debian.org - ? mirrors from ftp.se.d.o
* ftp.ee.debian.org - ? mirrors from ftp.fi.d.o
* ftp.hu.debian.org - not done - apparently bad syncs otherwise - will contact
* ftp.nz.debian.org - ? mirrors from mirrors.kernel.org (should fix that)
* ftp2.fr.debian.org - done
* ftp.es.debian.org - done
* ftp.se.debian.org - done
* ftp.fi.debian.org - done
* ftp.au.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
* ftp.br.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
* ftp.nl.debian.org - done
* ftp.is.debian.org - ? mirrors from ftp.se.d.o
* ftp.tr.debian.org - done
* ftp.hr.debian.org - done
* ftp.mx.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
* ftp.ie.debian.org - done
* ftp.ro.debian.org - done
* ftp.kr.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
* ftp.ch.debian.org - done
* ftp.cl.debian.org - don't know yet (need rsyncstats on syncproxy.wna)

I'll get the rest of the missing information from debian-admin and other
admins.

-- 
Josip Rodin
mirrors@debian.org

Message #108 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Thu, 27 Sep 2007 17:03:01 +0200

On Sat, Sep 15, 2007 at 12:22:41AM +0200, Josip Rodin wrote:
> This is the current status of setups for official mirrors:

Updating most

> * ftp.dk.debian.org - ? mirrors from ftp.nl.d.o

not done, contacted, no reply yet

> * ftp.it.debian.org - ? mirrors from ftp2.de.d.o

done

> * ftp.at.debian.org - not done - will contact them

done

> * ftp.jp.debian.org - ? mirrors from hp.debian.or.jp (via syncproxy.wna.d.o)
> * ftp2.jp.debian.org - ? mirrors from hp.debian.or.jp (via syncproxy.wna.d.o)

contacted, no reply yet

> * ftp.bg.debian.org - ? mirrors from ftp2.de.d.o

contacted, fixed

> * ftp.ru.debian.org - ? mirrors from ftp.se.d.o

contacted, fixed

> * ftp.fr.debian.org - not done - will contact them

contacted, was partially all right, now fully done

> * ftp.si.debian.org - not done - will contact them

contacted, no reply yet

> * ftp.hk.debian.org - ? should mirror from ftp.de.d.o, but is semi-dead (!)

contacted, no reply yet

> * ftp.no.debian.org - ? mirrors from ftp.se.d.o

contacted, they switched sources, not sure yet

> * ftp.ee.debian.org - ? mirrors from ftp.fi.d.o

contacted, no reply yet

> * ftp.hu.debian.org - not done - apparently bad syncs otherwise - will contact

contacted, it had some other problems, fixed

> * ftp.nz.debian.org - ? mirrors from mirrors.kernel.org (should fix that)

contacted, no reply yet

> * ftp.is.debian.org - ? mirrors from ftp.se.d.o

contacted, should be fixed

> * ftp.tw.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> * ftp.au.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> * ftp.br.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> * ftp.mx.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> * ftp.kr.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> * ftp.cl.debian.org - don't know yet (need rsyncstats on syncproxy.wna)

Still no rsyncstats...

-- 
Josip Rodin
mirrors@debian.org

Message #113 received at 6786@bugs.debian.org (full text, mbox, reply):

From: "Ricardo Yanez" <ricardo.yanez@calel.org>

To: "Josip Rodin" <joy@debbugs.entuzijast.net>

Cc: 6786@bugs.debian.org, debian-mirrors@lists.debian.org

Subject: Re: status

Date: Thu, 27 Sep 2007 11:22:12 -0400 (EDT)

Josip,

I may have missed some of this discussion, or perhaps it was held
somewhere else. In any case I wasn't aware of it.

Is what you are looking for the summary given by rsync right after the
pool is updated? If so, I can compile stats for ftp.cl.d.o from April 29th
to the present.

Ricardo

> On Sat, Sep 15, 2007 at 12:22:41AM +0200, Josip Rodin wrote:
>> This is the current status of setups for official mirrors:
>
> Updating most
>
>> * ftp.dk.debian.org - ? mirrors from ftp.nl.d.o
>
> not done, contacted, no reply yet
>
>> * ftp.it.debian.org - ? mirrors from ftp2.de.d.o
>
> done
>
>> * ftp.at.debian.org - not done - will contact them
>
> done
>
>> * ftp.jp.debian.org - ? mirrors from hp.debian.or.jp (via
>> syncproxy.wna.d.o)
>> * ftp2.jp.debian.org - ? mirrors from hp.debian.or.jp (via
>> syncproxy.wna.d.o)
>
> contacted, no reply yet
>
>> * ftp.bg.debian.org - ? mirrors from ftp2.de.d.o
>
> contacted, fixed
>
>> * ftp.ru.debian.org - ? mirrors from ftp.se.d.o
>
> contacted, fixed
>
>> * ftp.fr.debian.org - not done - will contact them
>
> contacted, was partially all right, now fully done
>
>> * ftp.si.debian.org - not done - will contact them
>
> contacted, no reply yet
>
>> * ftp.hk.debian.org - ? should mirror from ftp.de.d.o, but is semi-dead
>> (!)
>
> contacted, no reply yet
>
>> * ftp.no.debian.org - ? mirrors from ftp.se.d.o
>
> contacted, they switched sources, not sure yet
>
>> * ftp.ee.debian.org - ? mirrors from ftp.fi.d.o
>
> contacted, no reply yet
>
>> * ftp.hu.debian.org - not done - apparently bad syncs otherwise - will
>> contact
>
> contacted, it had some other problems, fixed
>
>> * ftp.nz.debian.org - ? mirrors from mirrors.kernel.org (should fix
>> that)
>
> contacted, no reply yet
>
>> * ftp.is.debian.org - ? mirrors from ftp.se.d.o
>
> contacted, should be fixed
>
>> * ftp.tw.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
>> * ftp.au.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
>> * ftp.br.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
>> * ftp.mx.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
>> * ftp.kr.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
>> * ftp.cl.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
>
> Still no rsyncstats...
>
> --
> Josip Rodin
> mirrors@debian.org
>
>
> --
> To UNSUBSCRIBE, email to debian-mirrors-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact
> listmaster@lists.debian.org
>
>

Message #118 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: Ricardo Yanez <ricardo.yanez@calel.org>

Cc: 6786@bugs.debian.org, debian-mirrors@lists.debian.org

Subject: Re: status

Date: Thu, 27 Sep 2007 17:45:17 +0200

On Thu, Sep 27, 2007 at 11:22:12AM -0400, Ricardo Yanez wrote:
> Josip,
> 
> I may have missed some of this discussion, or perhaps it was held
> somewhere else. In any case I wasn't aware of it.
> 
> Is what you are looking for the summary given by rsync right after the
> pool is updated? If so, I can compile stats for ftp.cl.d.o from April 29th
> to the present.

No, I'm just looking for the information on whether each of the official
mirrors (yours included :) uses a recent version of anonftpsync, and so
mirrors in two stages.

I wasn't planning to bother you and others who mirror from
syncproxy.wna.d.o, because obviously we (Debian) control that machine and
have the information available in the rsyncd log. We just have a local issue
of not having the logs readably by anyone but a few admins who are AWOL :)

-- 
     2. That which causes joy or happiness.

Message #123 received at 6786@bugs.debian.org (full text, mbox, reply):

From: "Ricardo Yanez" <ricardo.yanez@calel.org>

To: "Josip Rodin" <joy@debbugs.entuzijast.net>

Cc: 6786@bugs.debian.org, debian-mirrors@lists.debian.org

Subject: Re: status

Date: Thu, 27 Sep 2007 11:52:32 -0400 (EDT)

> On Thu, Sep 27, 2007 at 11:22:12AM -0400, Ricardo Yanez wrote:
>> Josip,
>>
>> I may have missed some of this discussion, or perhaps it was held
>> somewhere else. In any case I wasn't aware of it.
>>
>> Is what you are looking for the summary given by rsync right after the
>> pool is updated? If so, I can compile stats for ftp.cl.d.o from April
>> 29th
>> to the present.
>
> No, I'm just looking for the information on whether each of the official
> mirrors (yours included :) uses a recent version of anonftpsync, and so
> mirrors in two stages.

The answer is then YES, I use a fairly recent anonftpsync that mirrors in
two passes (but you should be able to see this in the rsyncd logs as
well.)

Ricardo

>
> I wasn't planning to bother you and others who mirror from
> syncproxy.wna.d.o, because obviously we (Debian) control that machine and
> have the information available in the rsyncd log. We just have a local
> issue
> of not having the logs readably by anyone but a few admins who are AWOL :)
>
> --
>      2. That which causes joy or happiness.
>
>
> --
> To UNSUBSCRIBE, email to debian-mirrors-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact
> listmaster@lists.debian.org
>
>

Message #128 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Sat, 6 Oct 2007 10:44:53 +0200

On Thu, Sep 27, 2007 at 05:03:01PM +0200, Josip Rodin wrote:
> > This is the current status of setups for official mirrors:
> 
> Updating most

...remaining ones :)

> > * ftp.dk.debian.org - ? mirrors from ftp.nl.d.o
> 
> not done, contacted, no reply yet

fixed

> > * ftp.jp.debian.org - ? mirrors from hp.debian.or.jp (via syncproxy.wna.d.o)
> > * ftp2.jp.debian.org - ? mirrors from hp.debian.or.jp (via syncproxy.wna.d.o)
> 
> contacted, no reply yet

done

> > * ftp.no.debian.org - ? mirrors from ftp.se.d.o
> 
> contacted, they switched sources, not sure yet

fixed (though the mirror is even more partial now :()

> > * ftp.nz.debian.org - ? mirrors from mirrors.kernel.org (should fix that)
> 
> contacted, no reply yet

done (also fixed the other issue :)

> > * ftp.si.debian.org - not done - will contact them
> 
> contacted, no reply yet
> 
> > * ftp.hk.debian.org - ? should mirror from ftp.de.d.o, but is semi-dead (!)
> 
> contacted, no reply yet

No reply whatsoever :( I'm resending.

> > * ftp.ee.debian.org - ? mirrors from ftp.fi.d.o
> 
> contacted, no reply yet

I actually found some old ignored queries from a replacement site for this
one, and sent more mail to different addresses, but there has been no reply yet.

> > * ftp.tw.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > * ftp.au.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > * ftp.br.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > * ftp.mx.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > * ftp.kr.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > * ftp.cl.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> 
> Still no rsyncstats...

And still no rsyncstats...

-- 
Josip Rodin
mirrors@debian.org

Message #133 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Wed, 10 Oct 2007 21:44:49 +0200

On Sat, Oct 06, 2007 at 10:44:53AM +0200, Josip Rodin wrote:
> > > This is the current status of setups for official mirrors:
> > Updating most
> ...remaining ones :)

And yet some more.

> > > * ftp.si.debian.org - not done - will contact them
> > 
> > contacted, no reply yet

Found the new admin, they fixed it.

> > > * ftp.hk.debian.org - ? should mirror from ftp.de.d.o, but is semi-dead (!)
> > 
> > contacted, no reply yet

That mirror actually isn't dead, it's working, but its project/trace
directory is broken. Still no reply from admins.

> > > * ftp.ee.debian.org - ? mirrors from ftp.fi.d.o
> > 
> > contacted, no reply yet
> 
> I actually found some old ignored queries from a replacement site for this
> one, and sent more mail to different addresses, but there has been no reply yet.

Still nothing.

> > > * ftp.tw.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > > * ftp.au.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > > * ftp.br.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > > * ftp.mx.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > > * ftp.kr.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > > * ftp.cl.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > 
> > Still no rsyncstats...
> 
> And still no rsyncstats...

Phil Hands added this, so now I verified that .tw, .br, .cl are fine.
After the stats are completed, I'll know for sure about the remaining three.

-- 
Josip Rodin
mirrors@debian.org

Message #138 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Sat, 13 Oct 2007 11:07:45 +0200

On Wed, Oct 10, 2007 at 09:44:49PM +0200, Josip Rodin wrote:
> > > > This is the current status of setups for official mirrors:
> > > Updating most
> > ...remaining ones :)
> 
> And yet some more.

And then some.

> > > > * ftp.ee.debian.org - ? mirrors from ftp.fi.d.o
> > > 
> > > contacted, no reply yet
> > 
> > I actually found some old ignored queries from a replacement site for this
> > one, and sent more mail to different addresses, but there has been no reply yet.
> 
> Still nothing.

With the help of their upstream which is actually ftp.se, we found that the
relay site (!) for ftp.ee uses a daily cron job. Still no reply.

> > > > * ftp.au.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > > > * ftp.mx.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> > > > * ftp.kr.debian.org - don't know yet (need rsyncstats on syncproxy.wna)
> 
> After the stats are completed, I'll know for sure about the remaining three.

.kr is fine, .mx is bad, .au is doing something screwy. Will contact them.

-- 
Josip Rodin
mirrors@debian.org

Message #143 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Thu, 1 Nov 2007 22:59:30 +0100

On Sat, Oct 13, 2007 at 11:07:45AM +0200, Josip Rodin wrote:
> > > > > This is the current status of setups for official mirrors:
> > > > Updating most
> > > ...remaining ones :)
> > 
> > And yet some more.
> 
> And then some.

This is another update. ftp.mx just got fixed a few days ago.

The remaining mirrors that are definitely doing something screwy are:
* ftp.ee - mirrors once, cron job
* ftp.au - mirrors once, triggered
* ftp.hk - mirrors once, cron job

All have been contacted, some several times, and there has been no reply. :(

-- 
Josip Rodin
mirrors@debian.org

Message #148 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Thu, 8 Nov 2007 02:39:18 +0100

On Thu, Nov 01, 2007 at 10:59:30PM +0100, Josip Rodin wrote:
> On Sat, Oct 13, 2007 at 11:07:45AM +0200, Josip Rodin wrote:
> > > > > > This is the current status of setups for official mirrors:
> > > > > Updating most
> > > > ...remaining ones :)
> > > 
> > > And yet some more.
> > 
> > And then some.
> 
> This is another update.

And another one. ftp.ee is fixed now.

Still no reply for ftp.au or ftp.hk.

-- 
Josip Rodin
mirrors@debian.org

Message #153 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Sun, 9 Dec 2007 15:12:18 +0100

On Thu, Nov 08, 2007 at 02:39:18AM +0100, Josip Rodin wrote:
> > > > > > > This is the current status of setups for official mirrors:
> > > > > > Updating most
> > > > > ...remaining ones :)
> > > > 
> > > > And yet some more.
> > > 
> > > And then some.
> > 
> > This is another update.
> 
> And another one.

ftp.au has been replaced.

ftp.hk is having issues not only with this, but also with disk space,
Roger So replied to us saying this is known and that we can remove the site.

If anyone has suggestions for a replacement, let us know, otherwise
we're going to remove it. On related note, I wonder if Internet connections
from Hong Kong to China are by and large good - if so, then we might
work around this problem by assigning ftp.cn.d.o (and we have a decent
candidate for that).

-- 
Josip Rodin
mirrors@debian.org

Message #158 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Tue, 26 Feb 2008 20:59:27 +0100

On Sun, Dec 09, 2007 at 03:12:18PM +0100, Josip Rodin wrote:
> ftp.hk is having issues not only with this, but also with disk space,
> Roger So replied to us saying this is known and that we can remove the site.

I've tried contacting people about this, to no avail... the site looks much
better now, but I still haven't had any confirmation about the status of
two-stage rsyncing. I've sent and re-sent more reminders :|

-- 
Josip Rodin
mirrors@debian.org

Message #163 received at 6786@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Sun, 2 Mar 2008 15:10:35 +0100

On Tue, Feb 26, 2008 at 08:59:27PM +0100, Josip Rodin wrote:
> On Sun, Dec 09, 2007 at 03:12:18PM +0100, Josip Rodin wrote:
> > ftp.hk is having issues not only with this, but also with disk space,
> > Roger So replied to us saying this is known and that we can remove the site.
> 
> I've tried contacting people about this, to no avail... the site looks much
> better now, but I still haven't had any confirmation about the status of
> two-stage rsyncing. I've sent and re-sent more reminders :|

OK, I found a mail saying ftp.hk rsyncs once a day, once per session.
Unless this is fixed, it'll have to be removed, this is sub-par...

-- 
Josip Rodin
mirrors@debian.org

Message #168 received at 6786-done@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786-done@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Sat, 15 Mar 2008 15:32:33 +0100

On Sun, Mar 02, 2008 at 03:10:35PM +0100, Josip Rodin wrote:
> > > ftp.hk is having issues not only with this, but also with disk space,
> > > Roger So replied to us saying this is known and that we can remove the site.
> > 
> > I've tried contacting people about this, to no avail... the site looks much
> > better now, but I still haven't had any confirmation about the status of
> > two-stage rsyncing. I've sent and re-sent more reminders :|
> 
> OK, I found a mail saying ftp.hk rsyncs once a day, once per session.
> Unless this is fixed, it'll have to be removed, this is sub-par...

After an ultimatum, Anthony Wong intervened and ftp.hk is finally fixed.

So, there we go. According to our records, all official Debian mirrors are
using a two-stage rsync strategy to avoid mirroring Packages files before
the debs. As for the unofficial mirrors - we can't really fix that
altogether, but we've had documentation, scripts and support mails updated
to help them for several years now, that's the best we can do.

If someone notices a broken mirror, please feel free to report it
individually, but the general issue is now practically fixed.

Therefore, I'm closing this 11 year old bug! :)

-- 
Josip Rodin
mirrors@debian.org

Message #174 received at 6786-done@bugs.debian.org (full text, mbox, reply):

From: Josip Rodin <joy@debbugs.entuzijast.net>

To: 6786-done@bugs.debian.org

Cc: debian-mirrors@lists.debian.org

Subject: Re: status

Date: Sat, 15 Mar 2008 15:49:01 +0100

On Sat, Mar 15, 2008 at 03:32:33PM +0100, Josip Rodin wrote:
> So, there we go. According to our records, all official Debian mirrors are
> using a two-stage rsync strategy to avoid mirroring Packages files before
> the debs. As for the unofficial mirrors - we can't really fix that
> altogether, but we've had documentation, scripts and support mails updated
> to help them for several years now, that's the best we can do.
> 
> If someone notices a broken mirror, please feel free to report it
> individually, but the general issue is now practically fixed.
> 
> Therefore, I'm closing this 11 year old bug! :)

Grh, just as I did that, I noticed I wasn't seeing the statistics for
one particular mirror which is part of (the process of updating) ftp.us.d.o.
But they are fast to update so the race condition window is very small,
and they have responsive maintainers, so I'm sure we'll get that last bit
fixed in no time.

I should also mention that I updated the scripts that update the kernel.org
mirrors (I said I would do it in an earlier message, but never ACK'ed that).

-- 
Josip Rodin
mirrors@debian.org

Send a report that this bug log contains spam.