Re: [resent PATCH] Re: very slow parallel read performance

[Daniel Phillips]

On August 27, 2001 09:43 pm, Oliver Neukum wrote:
> Am Montag, 27. August 2001 21:04 schrieb Daniel Phillips:
> > On August 27, 2001 08:37 pm, Oliver Neukum wrote:
> > > Hi,
> > >
> > > >   - Readahead cache is naturally a fifo - new chunks of readahead
> > > >     are added at the head and unused readahead is (eventually)
> > > >     culled from the tail.
> > >
> > > do you really want to do this based on pages ? Should you not drop all
> > > pages associated with the inode that wasn't touched for the longest
> > > time ?
> >
> > Isn't that very much the same as dropping pages from the end of the
> > readahead queue?
> 
> No, the end of the readahead queue will usually have pages from many 
> inodes(or perhaps it should be attached to open files as two tasks may read 
> different parts of one file).
> 
> If we are optimising for streaming (which readahead is made for) dropping 
> only one page will buy you almost nothing in seek time. You might just as 
> well drop them all and correct your error in one larger read if necessary.
> Dropping the oldest page is possibly the worst you can do, as you will need 
> it soonest.

Yes, good point.  OK, I'll re-examine the dropping logic.  Bear in mind, 
dropping readahead pages is not supposed to happen frequently under 
steady-state operation, so it's not that critical what we do here, it's going 
to be hard to create a load that shows the impact.  The really big benefit 
comes from not overdoing the readahead in the first place, and not underdoing 
it either.

> > > If you are streaming dropping all should be no great loss.
> >
> > The quesion is, how do you know you're streaming?  Some files are
> > read/written many times and some files are accessed randomly.  I'm trying
> > to avoid penalizing these admittedly rarer, but still important cases.
> 
> For those other case we have pages on the active list which are properly 
> aged, haven't we ?

Not if they never get a change to be moved to the active list.  We have to be 
careful to provide that opportunity.

> And readahead won't help you for random access unless you can cache it all.
> You might throw in a few extra blocks if you have free memory, but then you 
> have free memory anyway.

Readahead definitely can help in some types of random access loads, e.g., 
when access size is larger than a page, or when the majority of pages of a 
file are eventually accessed, but in random order.  On the other hand, it 
will only hurt for short, sparse, random reads.  Such a pattern needs to be 
detected in generic_file_readahead.

--
Daniel