Re: Streaming disk I/O kills file buffering and makes Linux unusable

[Miquel van Smoorenburg]

In article <[e-mail]>,
Peter Monta  <[e-mail]> wrote:
>> > I agree; the raw-io system will make this much more pleasant.
>>
>> Why do you think so?
>>
>> IO is IO, whether it is raw or not. And it will eat up resources from
>> everybody else trying to do IO.
>
>True; raw I/O won't magically increase resources.  But it offers
>a valuable way to tell the kernel something it can't know/infer: a good
>buffering policy for streaming data.  The existing cache mechanism
>for normal I/O works fine but is inappropriate for this case; indeed
>perfectly good working sets are messed up by buffering the streaming data,
>as Benno points out.

Indeed. One should be able to tell the kernel "as soon as I have read
this data I'm _never_ going to need it again". Not only a good idea
for streaming servers, how about backups? I'm forced to do backups
at night on our servers because during a backup it becomes way
to slow. Not because of the I/O going on, but because the backup process
keeps on filling the buffer and page cache with data nobody will never
need anymore while useful data is being thrown out.

You actually don't want to use raw I/O for this since you do want
things like readahead. You do want to be able to say that, once read,
you don't need the data ever again. Simply put the page at the start
of the LRU chain. Easy for read(), hard for mmap(), but for mmap you
could use madvise()

Mike.
-- 
... somehow I have a feeling the hurting hasn't even begun yet
	-- Bill, "The Terrible Thunderlizards"