GSI is a German non-profit research lab that runs a heavy ion accelerator
and is part of the Helmholtz association. The HPC department of GSI is
running a Linux
farm with more than a 1000 nodes. We primarily use
Debian for our Linux installation since the last millennium (around 1997).
We have recently activated a 10,000 core compute cluster and 1.5 PB Lustre storage with Infiniband-only networking running Debian Squeeze and managed by GridEngine - on top of our existing ~5000 CPU core cluster with more than 2 PB Lustre storage. Apart from our compute cluster we also provide some hundred desktops, servers for data acquisition, central IT services and collaboration web services with Debian.
We profit a lot from the more than 2000 packages that we can simply pull from the standard Debian repositories for our default desktop installation.