Linux-kernel mailing list archive 1998-50,: Re: PATCH: Raw device IO for 2.1.131

# Re: PATCH: Raw device IO for 2.1.131

David S. Miller (davem@dm.cobaltmicro.com)
Sat, 12 Dec 1998 21:12:08 -0800

- Messages sorted by: [ date ][ thread ][ subject ][ author ]
- Next message: Ed Hall: "Re: [A BIT OFF TOPIC] Very slow posting to linux-kernel"
- Previous message: MOLNAR Ingo: "Re: PATCH: Raw device IO for 2.1.131"

Date: Sun, 13 Dec 1998 05:36:59 +0100 (CET)
From: MOLNAR Ingo <mingo@chiara.csoma.elte.hu>

we _do_ have a zero-copy mechanizm in there, it's just not really
doing zero-copy currently :)

Very true...

With mmap()+copyfd() we dont ever copy the data, we just start off
DMA requests straight from kernel-space. (lets assume we have fixed
the page-cache stupidity of copying on writeout, and lets assume we
have a smart copyfd())

And now that I think about it more, it can work out for anonymous user
pages if such pages are treated as true page cache citizens which I
believe they are not right now?

while raw-IO sounds good, it's simply the wrong solution:

...

it's _much_ harder to fix the real issues, but we have to do it :
( I think raw-IO will help in the short term only.

I agree with the API issues. But the problem I have with all of this
is that I know of too many applications where the file model simply
does not fit. These people should have a back door to just slam data
to the disk if that is what they want.

I think for these situations it is not prudent to continually play
catchup with our VFS technology to what the latest block devices can
push.

framegrabbing, zero-copy disk IO is much harder through the
filesystem, but there is no conceptual problem with it, is there?
And if done right, people will suddenly have not only the
performance, but all the features of the VFS too, plus a clean
core/API. (those features are simply not there with raw-IO)

But Raw-IO will always be optimal. Do you see the difference? We can
keep making the VFS push data faster and faster, and at _best_ we'll
reach raw-IO speeds. Raw-IO is the theoretical maximum, it gives you
all of the performance. Whether VFS can hit this hardware limit is an
unknown at this time (I am not saying it's impossible) but we _know_
raw I/O gives it to us.

(i think it's entirely possible to create a 'blobfs', which is
basically a 1:1 mapping of the underlying device with only the
default VFS features, but which already operates through the
page-cache and inode space properly. Another solution would be to
embedd block devices into the page-cache via special inode
numbers.)

Anything which creates a direct path to an I/O destination via an
"abstraction layer" (here in the form of a funny filesystem) is flawed
I think. The abstraction layer costs "somthing", raw I/O has no
middle man, no middle man is necessary if all you want to do is push
data to a device from userspace, so why have one?

For file IO the same thing is harder to achieve but if we give up
our framework for shortterm gains, who will develop zero-copy for
the VFS, who will develop 2G+ file support and extent-based
allocation for ext2fs?

I simply do not buy the argument that if a back door exists here
nobody will advance the VFS technology. We do have 2G+ file support
on 64-bit architectures in ext2 already...

agreed, but really, we already have the mechanizm there. Most of
our sucky mmap()+write() (or rather mmap()+sendfile()) performance
is due to the extra copy we do when we write the page-cache.

Which is why people are going to want raw-IO until we fix things up.
You're talking major architecture redesign to remove this overhead,
Steven's raw-IO facility is a much smaller task in comparison.

the new raw-IO API is equivalent to an mmap()-ed 'temporary' file,
plus a copyfd() done from this temporary file to whatever other
(new or overwritten) file.

Sounds like the VFS equivalent has a lot of plumbing, and a couple
middle men are involved, why have these middle men at all?

I know there are downfalls to the raw-IO model, and I recognize them.
But I will continually contest that there are situations where the
plain old "pin page and go" model is simply the best.

Later,
David S. Miller
davem@dm.cobaltmicro.com

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

- Next message: Ed Hall: "Re: [A BIT OFF TOPIC] Very slow posting to linux-kernel"
- Previous message: MOLNAR Ingo: "Re: PATCH: Raw device IO for 2.1.131"