Re: O_DIRECT performance impact on 2.4.18 (was: Re: [PATCH]

[Lincoln Dale]

as the person who started this whole thread and made the assertion that 
copying from A to B is common:

At 11:35 AM 11/05/2002 -0700, Linus Torvalds wrote:
>And I personally believe that "generate the data yourself" is actually a
>very common case. A pure pipe between two places is not what a computer is
>good at, or what a computer should be used for.

i think you'd be surprised.  if we include "pipe from disk to network" then 
a large number of 'server' applications do exactly this.
webservers do.  fileservers do. http caches do.  streaming-media servers do.

sure, they may add additional headers on the front and still generate 
dynamic content in some cases, but the "common case" is 'pipe from disk to 
network' or 'pipe from network to disk'.
'network' is typically TCP but can be UDP (with rate-limiting) in some cases.

its very good to see this being discussed.  thats a large step forward from 
many people believing the problem was nonexistent.

i'm skeptical that continuing to use the page-cache is the correct way to 
go -- many of these kinds of applications are doing their own form of 
memory-management and hot-content 'caching' so are happy to manage a 
few-to-several hundred megabytes of "page cache equivalent" data themselves.
at least on many of the 2.3.xx linux releases, that was one of the big 
attractions of 'raw' devices -- they didn't get the box into an OOM situation.
if 2.5.xx and recent 2.4.xx has the issues of 
page-cache-doesn't-shrink-fast-enough solved, then its forseeable it will fly.

cheers,

lincoln.