Re: Possible Idea with filesystem buffering.

[Martin Knoblauch]

Helge Hafting wrote:
> 
> Martin Knoblauch wrote:
> >
> > > Re: Possible Idea with filesystem buffering.
> > >
> > > From: Richard B. Johnson ([e-mail])
> > > Date: Tue Jan 22 2002 - 17:10:27 EST
> > >
> > >
> > > We need a free-RAM target, possibly based upon a percentage of
> > > available RAM. The lack of such a target is what causes the
> > > out-of-RAM condition we have been experiencing. Somebody thought
> > > that "free RAM is wasted RAM" and the VM has been based upon
> > > that theory. That theory has been proven incorrect. You need
> >
> As far as I know, there is a free target.  The kernel will try to get
> rid of old pages (swapout program memory, toss cache pages)
> when there's too little free memory around.  This keeps memory
> around so future allocations and IO request may start
> immediately.  Maybe the current target is too small, but it is there.
> Without it, _every_ allocation or file operation would block
> waiting for a swapout/cache flush in order to get free pages.  Linux
> isn't nearly _that_ bad.
>

 Nobody said it is  _that_ bad. There are just some [maybe rare]
situations where it falls over and does not recover gracefully.

> >  Now, I think the theory itself is OK. The problem is that the stuff in
> > buffer/caches is to sticky. It does not go away when "more important"
> > uses for memory come up. Or at least it does not go away fast enough.
> >
> Then we need a larger free target to cope with the slow cache freeing.
>

 And as Rik said, we need to make freeing cache faster. All of this will
help the 98+% cases  that the VM can be optimized for. But I doubt that
you can make it 100% and keep it simple at the same time.

> > > free RAM, just like you need "excess horsepower" to make
> > > automobiles drivable. That free RAM is the needed "rubber-band"
> > > to absorb the dynamics of real-world systems.
> >
> >  Question: what about just setting a maximum limit to the cache/buffer
> > size. Either absolute, or as a fraction of total available memory? Sure,
> > it maybe a waste of memory in most situations, but sometimes the
> > administrator/user of a system simply "knows better" than the FVM (F ==
> > Fine ? :-)
> [...]
> >  I know, the tuning-knob approach is frowned upon. But sometimes there
> > are workloads where even the best VM may not know how to react
> > correctly.
> 
> Wasting memory "in most situations" isn't really an option.  But I
> see nothing wrong with "knobs" as long as they are automatic by
> default.  Those who want to optimize for a corner case can
> go and turn off the autopilot.
> 

 Definitely. The defaults need to be set for the general case. 

Martin
-- 
------------------------------------------------------------------
Martin Knoblauch         |    email:  [e-mail]
TeraPort GmbH            |    Phone:  [phone]
C+ITS                    |    Fax:    [phone]
http://www.teraport.de   |    Mobile: [phone]