RE: disk performance still wrecked in 2.4!

[Daniel Summer]

Hello kernel hackers. I have been concerned about a serious performance 
issue affecting bonnie disk benchmark results that has persisted throughout 
2.3 and 2.4 development. This issue has been brought up many times on this 
list and there doesn't seem to be a consensus as to what the origin of the 
problem is. Some have pointed to the elevator code, some to the VM 
subsystem, some to the block interfaces.

Joel Jacobsen of 3ware pointed out that these poor benchmark results may 
arise from problems with the buffer cache (see his comments below). I went 
ahead and ran some benchmarks with different quantities of RAM across 2.2 
and 2.4 to see if I could shed some light on this.

I found that bumping the RAM from 512megs to a gig gives a huge performance 
boost under 2.4 and makes little to no impact under 2.2.16. Even with a gig 
of RAM though, 2.4 still performs much worse than 2.2 at sequential reads.

So what does this mean? does anyone else agree that the buffer cache may be 
where this problem is coming from?

Kernel 2.2.16, 512 megs RAM
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
9274 99.3 24344 22.8 15534 33.9  8935 93.1 45774 38.8 379.0  4.4

Kernel 2.2.16, 1024 megs RAM
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
9318 99.6 26659 24.9 15988 33.9  8731 91.2 45424 39.1 555.8 11.5

Kernel 2.4.0 Test2 AC2, 512 megs RAM
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
8913 93.3 22391 17.9  6140  8.4  6076 63.9  9334  8.0 366.3  7.2

Kernel 2.4.0 Test2 AC2, 1024 megs RAM
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
9254 96.6 27400 23.9 12104 13.4  7627 78.6 27194 16.6 480.7  7.8

>From: Joel Jacobson <[e-mail]>
>Subject: RE: block device performance still wrecked in 2.4!
>
>the next time you run this, keep an eye on the drive light that corresponds
>to your swap file - the buffer cache was very greedy in the 2.3.* tree, and
>you might find that you're actually *paging* as a result.  there has been a
>*tremendous* amount of work on the buffer cache lately, so maybe this is
>being addressed...
>
>if you find the above is not the case, i have also done a bit of poking
>around in the buffer cache code and have found the *easiest* way to bump up
>the speed was to change the definition of NBUF in fs/block_dev.c to 512 (it
>was 64 the last time i looked in 2.3.99-pre5).  in effect, this tweaks the
>read-ahead code for the file system.  it's a mean thing to do for dumb (ie
>slow) controllers, but ours seems to handle it just fine.  if you're doing 
>a
>lot of streaming reads, this should improve things considerably. if you're
>doing a lot of random seeking, it might help some but it'll depend a lot of
>other variables.
>
>the other possible cause here is the elevator code.  i've tried turning it
>off, but either i wasnt successfull in the attempt, or that's not the
>problem.  in any event, there has been considerable amount of work on that
>stuff as well lately, as a tangenital discussion to the aforementioned
>buffer cache work (and i suspect they're pretty closely related).
>
>- j
>

> > -----Original Message-----
> > From: DV LABS [mailto:[e-mail]]
> > Subject: block device performance still wrecked in 2.4!
> >
> >
> > Calling all kernel hackers,
> >
> > The block device interface is still choking for some reason, I see
> > an 80% performance hit on disk read throughput going from 2.2 to 2.4.
> >

________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com