Re: Bits from the CD team: plans for debian-cd v3.0

[Goswin von Brederlow <brederlo@informatik.uni-tuebingen.de>]

Steve McIntyre <steve@einval.com> writes:

> On Wed, Jul 13, 2005 at 03:28:08AM +0200, Goswin von Brederlow wrote:
>>Steve McIntyre <steve@einval.com> writes:
>>
>>>  8. Running without a local mirror
>>>
>>>     A common complaint is that debian-cd basically requires a local
>>>     mirror to be useful. This is not ideal for many end users, who may
>>>     be trying to create small custom CDs and don't have the disk space
>>>     for all the Debian packages needed. It should be feasible to make
>>>     debian-cd work with remote mirrors, grabbing just the pieces it
>>>     needs. This _may_ need custom work, or it may be possible to use
>>>     one of several packages suggested to create a partial mirror. I'm
>>>     actually leaning more towards custom code, perhaps even some
>>>     tweaks to pipe the data into mkisofs as it is needed to save local
>>>     disk.
>>
>>Given a list of package needed for the CD it is trivial to create a
>>partial mirror using one of various methods. If that mirror is then
>>used to hardlink or symlink the files into the CD dirs no space is
>>wasted.
>>
>>To save more space you would have to pipe the files and live with the
>>problem that you can't reread a file, e.g. to run apt-ftparchive or
>>md5sum over it. You have to get each file as it comes and pipe it to
>>all the things that need to read it at the same time. That might be
>>quite complicated.
>
> We're planning on avoiding most of that - all the packages and source
> files have MD5 entries we can read. And apt-ftparchive is too slow, so
> we're going to avoid it.

I thought you (someone said anyway) wanted to stick with
apt-ftparchive instead of mangling the Packages, Sources and Contents
files but enable the caching to speed up the process. That way only
the first build would be slow.

>>What would be real great though would be to combine this with the
>>realy realy fast jigdo file creation. If you don't create iso images
>>you don't even need the debs. Just get the Packages/Sources files,
>>create jidgo, done. Would that be possible? Creating isos without
>>local mirror could then go this way and run jigdo to create images.
>
> You're psychic - that's exactly what we've been thinking about. There
> are some issues to think about in this scheme, for example the fact
> that some files _have_ to be downloaded as they will go directly into
> the template file as raw data. But it might be possible to generate
> skeleton jigdo files in a few seconds per CD if we get this
> right... :-)

You probably need to get the indices/md5sum.gz file to fill in the
md5sums for tools and docs too. But all that is peanuts compared to
5GB for dvd1. :)

I say get this working and ignore the middle way of downloading
packages directly onto the isos without a local mirror. This is far
better than any space saving idea I heard before.

MfG
        Goswin