Re: minimizing buffer size (i.e. page cache) for bulk copy/rsync -- Re: Swappiness in Buster
On Thu, Jul 09, 2020 at 12:02:39AM +1000, Zenaan Harkness wrote:
> What I imagine, and surely hope is that this softlimit program works as
> advertised.
It's hardly unique. You could do the same thing with a shell script
wrapper that calls ulimit and then exec's the target program.
> [stuff about buffers]
> Since if it's not written in a totally insane way, cp (and to a lesser
> extent rsync) should work like this already, what we really need to
> limit is that pig with the strange "Linux" name
rsync uses a shitload of memory not because it's copying a file with
an enormous buffer size, but because it's tracking *all* of the file
metadata in the entire hierarchy that you feed to it.
> What has been happening since the dawn of time is that this kernel
> caches _every_ disk page read by a program such as cp [...]
Well, prematurely killing rsync is not going to affect the kernel's
retention or discarding of specific disk cache pages that are more
precious to you.
Reply to: