Re: Delaying writes to disk when there's no need
Helge Hafting (email@example.com)
Sun, 30 Mar 2003 19:38:02 +0200
On Fri, Mar 28, 2003 at 09:18:59PM +1100, Tim Connors wrote:
> In linux.kernel, you wrote:
> > Helge Hafting (firstname.lastname@example.org) wrote:
> >> Erik Hensema wrote:
> >>> In all kernels I've tested writes to disk are delayed a long time even when
> >>> there's no need to do so.
> >> Short answer - it is supposed to do that!
> >>> A very simple test shows this: on an otherwise idle system, create a tar of
> >>> a NFS-mounted filesystem to a local disk. The kernel starts writing out the
> >>> data after 30 seconds, while a slow and steady stream would be much nicer
> >>> to the system, I think.
> Agreed. We have a cluster which is writing on average something like
> 20 Megs/sec/node. We had to lower the write threshold from 30% to 0%,
> because with the constant writing, linux will buffer it for 30 secs,
> fill up RAM, try to empty the write-cache, stall, wash, rinse,
> repeat. Because it was being filled up at roughly the rate it was
> being emptied, once it got 30% behind, there was no catching up, so
> the realtime system would lose data. Ouch.
Nothing can help you if you're getting input at the rate you
can write it. You need ability to write at least somewhat faster
no matter how buffering is done.
> >> You're wrong then. There's no need for a slow steady stream, why do
> >> you want that. Of course you can set up cron to run sync at
> >> regular (short) intervals to achieve this.
> Last time I checked, cron had 1 minute resolution.
If you need to sync more often than once per minute, consider
for ((;;)) ; do sleep 1 ; sync ; done
> Helge's comment about /tmp files and rewriting files multiple times:
> in real life, how often does this happen? How often do you overwrite
> one file many times in 30 seconds?
_Every_ time you write a file in chunks not perfectly aligned
on block boundaries.
> The occasional 20 kilobyte /tmp
> file perhaps, but I doubt it matters in real life. In real life, when
> writing to disk constantly (not just scientific applications - I
> believe this happens in the real world too!), waiting for 30 seconds
> is a liability!
Only if that 30-second wait makes linux buffer more than it can handle.
That may indeed be a problem, but buffering up _some_ data
before writing is still a good idea.
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to email@example.com
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/