Re: To Merge or not to Merge.:: Merging partitions, for cd building.
Petter Reinholdtsen wrote:
> [Ronny Aasen]
>> Greetings
>
> Hi. Good to see work on the CD build. It has stayed the same for
> quite a long time, and could probably do with an review. :)
>
>> For historical reasons, the partition layout on administrator.s.n is
>> a bit overly complex.
>
> Most of it is not for historical reasons, most of it is to make sure
> some parts of the system can not destroy other parts by filling up
> partitions.
we should be vigilant. and have nagios help us with that. but with the
size of the ataboy, the problem should be far in the future.
>> The result of this complexity is that we can not use hardlinking
>> when building cd's something that waste a lot of space, and uses a
>> lot of extra time for building, to copy files around.
>
> We used to use hardlinks, but dropped it. I believe we dropped it to
> avoid requiring a complete mirror of every APT source we use during
> building. Not sure if I remember it correctly, thought. It might
> also have been because we ran out of disk space when we started
> building more architectures and variants.
I belive the latter is the case. if we hardlink or copy the file. the
source must be the same.
>> With the recent deployment of the ataboy2, donated by usit. space is
>> no longer a issue.
>
> Note that speed is an issue, though. The ataboy is slower than
> internal disks, and we had problems earlier with IO before I was able
> to move everything used during building to internal disks, and
> everything else to the ataboy.
If all the building and mirror stuff could be on the ataboy we should
have enough space for interactive stuff. like svn and homedirs on the
internal storage.
>> we can also consider merge the 4 home area's into a single one.
>
> I would strongly recommend against this one, as it would allow users
> to break the automatic build by filling up the partition, and make it
> harder to track how much space the cinelerra people are using compared
> to the debian edu people.
If the home areas are divided in a sane fashion like edu vs
administrative vs cinelerra then it's fine. But if they are divided
becouse of earlier space restrictions. then i'd like to atlest review
the idea.
>> And using hardlinking should mean faster builds.
>
> How much faster? I believe we have quite quick builds already, and
> most of the time is spent doing MD5 sums of the packages and final
> images, so I doubt hardlinking will have that much effect.
Not certain. only testing would tell. But i assume one would not need to
copy the 1.5GB of data for a cd and 8GB of data for a dvd around so much.
With FASTSUMS
Steve McIntyre (sledge) on #debian-cd indicated that it was
significant. COPYLINK was actualy ment to be used when you had no local
debmirror.
#quote from #debian-cd
<sepski> building takes a while. we have to use COPYLINK=1 is it very
much faster to use hard/symlinking ?
<Sledge> oh christ, yes!
<Sledge> actually, it will depend
<Sledge> if you're on a machine with enough RAM to cache everything then
you'll be fine
<Sledge> if you're doing a lot of build work, then using the hard links
is a very big win
Kind regards
Ronny Aasen
Reply to: