Re: [reiserfs-list] Re: [reiserfs-dev] Re: Note describing poor dcache utilization under high memory pressure

[Andreas Dilger]

On Jan 29, 2002  09:28 -0800, Josh MacDonald wrote:
> Quoting Alexander Viro ([e-mail]):
> > On Tue, 29 Jan 2002, Hans Reiser wrote:
> > > This fails to recover an object (e.g. dcache entry) which is used once, 
> > > and then spends a year in cache on the same page as an object which is 
> > > hot all the time.  This means that the hot set of objects becomes 
> > > diffused over an order of magnitude more pages than if garbage 
> > > collection squeezes them all together.  That makes for very poor caching.
> > 
> > Any GC that is going to move active dentries around is out of question.
> > It would need a locking of such strength that you would be the first
> > to cry bloody murder - about 5 seconds after you look at the scalability
> > benchmarks.
> 
> We're not talking about actively referenced entries, we're talking about
> entries on the d_lru list with zero references.  Relocating those objects
> should not require any more locking than currently required to remove and
> re-insert the dcache entry.  Right?

But if it is unused and not recently referenced, there is little benefit
in keeping it around, is there?  I suppose there might be some benefit in
moving hot negative dentries to be on the same page as other hot dentries,
and that should be possible.  Negative dentries are a special case, though,
in that they are not actually in use, but their presence is a performance
improvement (i.e. not having to look up /usr/bin/ls, /usr/sbin/ls, etc).

Being able to move active entries between slabs is just too hard in
most cases, and the overhead (locking, back references, etc) in doing so
would probably outweigh the benefits of more tightly packed slab caches.
This could be up to the per-page (or per-slab) "try_to_free_page"
callback function to handle though.

I do agree with the assertion that you shouldn't judge the utility of
all objects on the page in which it lives by just the page use, otherwise
you will never free any other entries on a hot page.  There should still
be per-item reference bits that give hints on which entries in the slab
should be freed.

For example we can use the following simple algorithm to free pages/entries:
1) Walk slab pages (maybe page LRU order), and if all entries in the page
   can be freed, free them and then remove the page from the slab.
   This gives the VM a free page to work with, and any entries that
   are later reloaded will go to another page.
   
   As a special case for dcache (which would be handled in the dcache
   try_to_free_page callback) we could potentially move referenced
   negative dentries to another free entry in that slab page in order to
   free a page but not take the hit to go out to disk again to recreate
   the negative dentry.

2) For slab pages with in-use entries, free some fraction of entries based
   on whether the entries have been referenced.  There is no point in
   freeing all but one of the entries on a page if we can't free the page
   in the end, and having a 10% cache page utilization does nobody any good.
   Freeing some of the entries gives us free space to add new entries, but
   does not discard too many entries when we can't free the pages anyways.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/