Asynch I/O broken in 2.2.15

[Jeff V. Merkey]

This is a multi-part message in MIME format.
--------------5DE8FA9DB32890CC2FE2D96C
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I have tried all sorts of stuff to make the swapper Ooops go away.  I
have also completely reviewed all of the mm code, fs code, etc, etc. 
And here's what I think I've determined to be invalid in my assumptions
about what's there:

1.  Linux 2.2.15 and 2.3.99 do not support true Asynch IO.  Asynch IO
allows a caller to post a very large number of requests, is not **NOT**
polled, and should be interrupt driven.  The first request should "kick"
the queue head into the driver, then the driver should feed the next
buffer on the IO completion interrupt until the queue head is empty. 
The way it's implemented in Linux requires that the bottom half ISR be
"polled" within wait_on_buffer() or some other function to initiate the
I/O and block until the IO returns -- how is this asynchronous I/O.  

2.  While the semantics appear to be Asynch, the interface is actually
synchronous in behavior -- you have to poll the drives after posting IO
to start the disks.  The dependency on the lock_kernel() call on 2.2.15
guarantees that most callers will block when attempting multiple calls
to ll_rw_blk(), so there went the parallelism.  I have noticed that if I
hold the kernel lock over the I/O (which is what is happening in
filemap.c and buffer.c) then the swapper does not crash the system. 
This means that all I/O requests performed are really synchronous since
only one caller seems to be able to post at one time with crashing.

3.  Feeding more than a handful of requests at one time to THE LINUX
BUFFER CACHE (forget NWFS code - I tried this with just Linux) will
cause the swapper process to Ooops and die.  I implemented some very
simple code that just calls the buffer cache and tries to do asynch I/O
and "**KAPOW!!!*** , the system crashes.  The code is attached.

Jeff Merkey
CEO, TRG
--------------5DE8FA9DB32890CC2FE2D96C
Content-Type: application/x-unknown-content-type-cfile;
 name="aio.c"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
 filename="aio.c"

DQp2b2lkIGVuZF9hc3luY2hfaW9fYnJjKHN0cnVjdCBidWZmZXJfaGVhZCAqYmgsIGludCB1
cHRvZGF0ZSkNCnsNCiAgICByZWdpc3RlciBpbnQgaTsNCiAgICBBU1lOQ0hfSU8gKmlvID0g
KEFTWU5DSF9JTyAqKWJoLT5iX25leHRfZnJlZTsNCg0KICAgIG1hcmtfYnVmZmVyX3VwdG9k
YXRlKGJoLCB1cHRvZGF0ZSk7DQogICAgdW5sb2NrX2J1ZmZlcihiaCk7DQoNCiAgICBpZiAo
IXRlc3RfYml0KEJIX1VwdG9kYXRlLCAmYmgtPmJfc3RhdGUpKQ0KICAgICAgIGlvLT5jY29k
ZSA9IEFTSU9fSU9fRVJST1I7DQoNCiAgICBpby0+Y29tcGxldGUrKzsNCiAgICBpZiAoaW8t
PmNvbXBsZXRlID49IGlvLT5jb3VudCkNCiAgICB7DQogICAgICAgZm9yIChpPTA7IGkgPCBp
by0+Y291bnQ7IGkrKykNCiAgICAgICB7DQogICAgICAgICAgTldGU0NvcHkoJmlvLT5idWZm
ZXJbaSAqIDUxMl0sIGlvLT5iaFtpXS0+Yl9kYXRhLCA1MTIpOw0KICAgICAgICAgIGJyZWxz
ZShpby0+YmhbaV0pOw0KICAgICAgIH0NCg0KICAgICAgIGlmIChpby0+Y2FsbF9iYWNrX3Jv
dXRpbmUpDQogICAgICAgICAgKGlvLT5jYWxsX2JhY2tfcm91dGluZSkoaW8pOw0KICAgICAg
IGlvLT5zaWduYXR1cmUgPSAwOw0KDQojaWYgKFBST0ZJTEVfQUlPKQ0KICAgICAgIHByb2Zp
bGVfY29tcGxldGUoKTsNCiNlbmRpZg0KICAgIH0NCiAgICByZXR1cm47DQp9DQoNCnZvaWQg
ZW5kX2FzeW5jaF9pb19id2Moc3RydWN0IGJ1ZmZlcl9oZWFkICpiaCwgaW50IHVwdG9kYXRl
KQ0Kew0KICAgIHJlZ2lzdGVyIGludCBpOw0KICAgIEFTWU5DSF9JTyAqaW8gPSAoQVNZTkNI
X0lPICopYmgtPmJfbmV4dF9mcmVlOw0KDQogICAgbWFya19idWZmZXJfdXB0b2RhdGUoYmgs
IHVwdG9kYXRlKTsNCiAgICB1bmxvY2tfYnVmZmVyKGJoKTsNCg0KICAgIGlmICghdGVzdF9i
aXQoQkhfVXB0b2RhdGUsICZiaC0+Yl9zdGF0ZSkpDQogICAgICAgaW8tPmNjb2RlID0gQVNJ
T19JT19FUlJPUjsNCg0KICAgIGlvLT5jb21wbGV0ZSsrOw0KICAgIGlmIChpby0+Y29tcGxl
dGUgPj0gaW8tPmNvdW50KQ0KICAgIHsNCiAgICAgICBmb3IgKGk9MDsgaSA8IGlvLT5jb3Vu
dDsgaSsrKQ0KICAgICAgICAgIGJyZWxzZShpby0+YmhbaV0pOw0KDQogICAgICAgaWYgKGlv
LT5jYWxsX2JhY2tfcm91dGluZSkNCiAgICAgICAgICAoaW8tPmNhbGxfYmFja19yb3V0aW5l
KShpbyk7DQogICAgICAgaW8tPnNpZ25hdHVyZSA9IDA7DQoNCiNpZiAoUFJPRklMRV9BSU8p
DQogICAgICAgcHJvZmlsZV9jb21wbGV0ZSgpOw0KI2VuZGlmDQogICAgfQ0KICAgIHJldHVy
bjsNCn0NCg0KDQpVTE9ORyBhUmVhZERpc2tTZWN0b3JzKFVMT05HIGRpc2ssIFVMT05HIFN0
YXJ0aW5nTEJBLCBCWVRFICpTZWN0b3IsDQoJCSAgICAgIFVMT05HIHNlY3RvcnMsIFVMT05H
IHJlYWRBaGVhZCwgQVNZTkNIX0lPICppbykNCnsNCiAgICByZWdpc3RlciBVTE9ORyBpLCBi
eXRlc1JlYWQgPSAwOw0KICAgIHJlZ2lzdGVyIFVMT05HIGJwczsNCiAgICByZWdpc3RlciBO
V0RJU0sgKk5XRGlzazsNCg0KICAgIE5XRGlzayA9IFN5c3RlbURpc2tbZGlza107DQogICAg
YnBzID0gTldEaXNrLT5CeXRlc1BlclNlY3RvcjsNCg0KICAgIGlvLT5jY29kZSA9IDA7DQog
ICAgaW8tPmNvdW50ID0gc2VjdG9yczsNCiAgICBpby0+Y29tcGxldGUgPSAwOw0KDQogICAg
bG9ja19rZXJuZWwoKTsNCiAgICBmb3IgKGk9MDsgaSA8IHNlY3RvcnM7IGkrKykNCiAgICB7
DQogICAgICAgaWYgKCEoaW8tPmJoW2ldID0gZ2V0YmxrKChVTE9ORylOV0Rpc2stPlBoeXNp
Y2FsRGlza0hhbmRsZSwNCgkJCSAgICAoU3RhcnRpbmdMQkEgKyBpKSwgYnBzKSkpDQogICAg
ICAgew0KCSAgZm9yIChpPTA7IGkgPCBzZWN0b3JzOyBpKyspDQoJICAgICBpZiAoaW8tPmJo
W2ldKQ0KCQlicmVsc2UoaW8tPmJoW2ldKTsNCiAgICAgICAgICB1bmxvY2tfa2VybmVsKCk7
DQoJICByZXR1cm4gMDsNCiAgICAgICB9DQogICAgICAgaW8tPmJoW2ldLT5iX2VuZF9pbyA9
IGVuZF9hc3luY2hfaW9fYnJjOw0KICAgICAgIGlvLT5iaFtpXS0+Yl9kZXZfaWQgPSBpbzsN
CiAgICB9DQogICAgbGxfcndfYmxvY2soUkVBRCwgc2VjdG9ycywgJmlvLT5iaFswXSk7DQog
ICAgdW5sb2NrX2tlcm5lbCgpOw0KDQogICAgcnVuX3Rhc2tfcXVldWUoJnRxX2Rpc2spOw0K
DQojaWYgKFBST0ZJTEVfQUlPKQ0KICAgIHByb2ZpbGVfY29tcGxldGUoKTsNCiNlbmRpZg0K
DQogICAgYnl0ZXNSZWFkID0gc2VjdG9ycyAqIGJwczsNCiAgICByZXR1cm4gKGJ5dGVzUmVh
ZCk7DQoNCn0NCg0KVUxPTkcgYVdyaXRlRGlza1NlY3RvcnMoVUxPTkcgZGlzaywgVUxPTkcg
U3RhcnRpbmdMQkEsIEJZVEUgKlNlY3RvciwNCgkJICAgICAgIFVMT05HIHNlY3RvcnMsIFVM
T05HIHJlYWRBaGVhZCwgQVNZTkNIX0lPICppbykNCnsNCiAgICByZWdpc3RlciBVTE9ORyBp
LCBieXRlc1dyaXR0ZW4gPSAwOw0KICAgIHJlZ2lzdGVyIFVMT05HIGJwczsNCiAgICByZWdp
c3RlciBOV0RJU0sgKk5XRGlzazsNCg0KICAgIE5XRGlzayA9IFN5c3RlbURpc2tbZGlza107
DQogICAgYnBzID0gTldEaXNrLT5CeXRlc1BlclNlY3RvcjsNCg0KICAgIGlvLT5jY29kZSA9
IDA7DQogICAgaW8tPmNvdW50ID0gc2VjdG9yczsNCiAgICBpby0+Y29tcGxldGUgPSAwOw0K
DQogICAgbG9ja19rZXJuZWwoKTsNCiAgICBmb3IgKGk9MDsgaSA8IHNlY3RvcnM7IGkrKykN
CiAgICB7DQogICAgICAgaWYgKCEoaW8tPmJoW2ldID0gZ2V0YmxrKChVTE9ORylOV0Rpc2st
PlBoeXNpY2FsRGlza0hhbmRsZSwNCgkJCSAgICAoU3RhcnRpbmdMQkEgKyBpKSwgYnBzKSkp
DQogICAgICAgew0KCSAgZm9yIChpPTA7IGkgPCBzZWN0b3JzOyBpKyspDQoJICAgICBpZiAo
aW8tPmJoW2ldKQ0KCQlicmVsc2UoaW8tPmJoW2ldKTsNCiAgICAgICAgICB1bmxvY2tfa2Vy
bmVsKCk7DQoJICByZXR1cm4gMDsNCiAgICAgICB9DQogICAgICAgaW8tPmJoW2ldLT5iX2Vu
ZF9pbyA9IGVuZF9hc3luY2hfaW9fYndjOw0KICAgICAgIGlvLT5iaFtpXS0+Yl9kZXZfaWQg
PSBpbzsNCg0KICAgICAgIE5XRlNDb3B5KGlvLT5iaFtpXS0+Yl9kYXRhLCAmU2VjdG9yW2kg
KiBicHNdLCBicHMpOw0KICAgICAgIG1hcmtfYnVmZmVyX3VwdG9kYXRlKGlvLT5iaFtpXSwg
MSk7DQogICAgICAgbWFya19idWZmZXJfZGlydHkoaW8tPmJoW2ldLCAwKTsNCiAgICAgICBi
eXRlc1dyaXR0ZW4gKz0gYnBzOw0KICAgIH0NCiAgICBsbF9yd19ibG9jayhXUklURSwgc2Vj
dG9ycywgJmlvLT5iaFswXSk7DQogICAgdW5sb2NrX2tlcm5lbCgpOw0KDQogICAgcnVuX3Rh
c2tfcXVldWUoJnRxX2Rpc2spOw0KDQojaWYgKFBST0ZJTEVfQUlPKQ0KICAgIHByb2ZpbGVf
Y29tcGxldGUoKTsNCiNlbmRpZg0KDQogICAgYnl0ZXNXcml0dGVuID0gc2VjdG9ycyAqIGJw
czsNCiAgICByZXR1cm4gKGJ5dGVzV3JpdHRlbik7DQoNCn0NCg0KVUxPTkcgYVplcm9GaWxs
RGlza1NlY3RvcnMoVUxPTkcgZGlzaywgVUxPTkcgU3RhcnRpbmdMQkEsIFVMT05HIHNlY3Rv
cnMsDQoJCQkgIFVMT05HIHJlYWRBaGVhZCwgQVNZTkNIX0lPICppbykNCnsNCiAgICByZWdp
c3RlciBVTE9ORyBpLCBieXRlc1dyaXR0ZW4gPSAwOw0KICAgIHJlZ2lzdGVyIFVMT05HIGJw
czsNCiAgICByZWdpc3RlciBOV0RJU0sgKk5XRGlzazsNCg0KICAgIGlvLT5jY29kZSA9IDA7
DQogICAgaW8tPmNvdW50ID0gc2VjdG9yczsNCiAgICBpby0+Y29tcGxldGUgPSAwOw0KDQog
ICAgTldEaXNrID0gU3lzdGVtRGlza1tkaXNrXTsNCiAgICBicHMgPSBOV0Rpc2stPkJ5dGVz
UGVyU2VjdG9yOw0KDQogICAgbG9ja19rZXJuZWwoKTsNCiAgICBmb3IgKGk9MDsgaSA8IHNl
Y3RvcnM7IGkrKykNCiAgICB7DQogICAgICAgaWYgKCEoaW8tPmJoW2ldID0gZ2V0YmxrKChV
TE9ORylOV0Rpc2stPlBoeXNpY2FsRGlza0hhbmRsZSwNCgkJCSAgKFN0YXJ0aW5nTEJBICsg
aSksIGJwcykpKQ0KICAgICAgIHsNCgkgIGZvciAoaT0wOyBpIDwgc2VjdG9yczsgaSsrKQ0K
CSAgICAgaWYgKGlvLT5iaFtpXSkNCgkJYnJlbHNlKGlvLT5iaFtpXSk7DQogICAgICAgICAg
dW5sb2NrX2tlcm5lbCgpOw0KCSAgcmV0dXJuIDA7DQogICAgICAgfQ0KICAgICAgIGlvLT5i
aFtpXS0+Yl9lbmRfaW8gPSBlbmRfYXN5bmNoX2lvX2J3YzsNCiAgICAgICBpby0+YmhbaV0t
PmJfZGV2X2lkID0gaW87DQoNCiAgICAgICBOV0ZTU2V0KGlvLT5iaFtpXS0+Yl9kYXRhLCAw
LCBicHMpOw0KICAgICAgIG1hcmtfYnVmZmVyX3VwdG9kYXRlKGlvLT5iaFtpXSwgMSk7DQog
ICAgICAgbWFya19idWZmZXJfZGlydHkoaW8tPmJoW2ldLCAwKTsNCiAgICAgICBieXRlc1dy
aXR0ZW4gKz0gYnBzOw0KICAgIH0NCiAgICBsbF9yd19ibG9jayhXUklURSwgc2VjdG9ycywg
JmlvLT5iaFswXSk7DQogICAgdW5sb2NrX2tlcm5lbCgpOw0KDQogICAgcnVuX3Rhc2tfcXVl
dWUoJnRxX2Rpc2spOw0KDQojaWYgKFBST0ZJTEVfQUlPKQ0KICAgIHByb2ZpbGVfY29tcGxl
dGUoKTsNCiNlbmRpZg0KDQogICAgYnl0ZXNXcml0dGVuID0gKHNlY3RvcnMgKiBicHMpOw0K
ICAgIHJldHVybiAoYnl0ZXNXcml0dGVuKTsNCn0NCg0KI2VuZGlmDQoNCg==
--------------5DE8FA9DB32890CC2FE2D96C--