Re: Performance 2.4.8 is worse than 2.4.x<8 (SPEC NFS results show this)

Hans Reiser (reiser@namesys.com)
Tue, 14 Aug 2001 01:12:53 +0400


We are looking into this. Elena and Chris, please advise as to whether the
slowdown is ReiserFS code added recently or is due to layers not ReiserFS.

Hans

"HABBINGA,ERIK (HP-Loveland,ex1)" wrote:
>
> Here are some SPEC SFS NFS testing (http://www.spec.org/osg/sfs97) results
> I've been doing over the past few weeks that shows NFS performance degrading
> since the 2.4.5pre1 kernel. I've kept the hardware constant, only changing
> the kernel. I'm prevented by management from releasing our top numbers, but
> have given our results normalized to the 2.4.5pre1 kernel. I've also shown
> the results from the first three SPEC runs to show the response time trend.
>
> Normally, response time should start out very low, increasing slowly until
> the maximum load of the system under test is reached. Starting with
> 2.4.8pre8, the response time starts very high, and then decreases. Very
> bizarre behaviour.
>
> The spec results consist of the following data (only the first three numbers
> are significant for this discussion)
> - load. The load the SPEC prime client will try to get out of the system
> under test. Measured in I/O's per second (IOPS).
> - throughput. The load seen from the system under test. Measured in IOPS
> - response time. Measured in milliseconds
> - total operations
> - elapsed time. Measured in seconds
> - NFS version. 2 or 3
> - Protocol. UDP (U) or TCP (T)
> - file set size in megabytes
> - number of clients
> - number of SPEC SFS processes
> - biod reads
> - biod writes
> - SPEC SFS version
>
> The 2.4.8pre4 and 2.4.8 tests were invalid. Too many (> 1%) of the RPC
> calls between the SPEC prime client and the system under test failed. This
> is not a good thing.
>
> I'm willing to try out any ideas on this system to help find and fix the
> performance degradation.
>
> Erik Habbinga
> Hewlett Packard
>
> Hardware:
> 4 processors, 4GB ram
> 45 fibre channel drives, set up in hardware RAID 0/1
> 2 direct Gigabit Ethernet connections between SPEC SFS prime client and
> system under test
> reiserfs
> all NFS filesystems exported with sync,no_wdelay to insure O_SYNC writes to
> storage
> NFS v3 UDP
>
> Results:
> 2.4.5pre1
> 500 497 0.8 149116 300 3 U 5070624 1 48 2 2
> 2.0
> 1000 1004 1.0 300240 299 3 U 10141248 1 48 2 2
> 2.0
> 1500 1501 1.0 448807 299 3 U 15210624 1 48 2 2
> 2.0
> peak IOPS: 100% of 2.4.5pre1
>
> 2.4.5pre2
> 500 497 1.0 149195 300 3 U 5070624 1 48 2 2
> 2.0
> 1000 1005 1.2 300449 299 3 U 10141248 1 48 2 2
> 2.0
> 1500 1502 1.2 449057 299 3 U 15210624 1 48 2 2
> 2.0
> peak IOPS: 91% of 2.4.5pre1
>
> 2.4.5pre3
> 500 497 1.0 149095 300 3 U 5070624 1 48 2 2
> 2.0
> 1000 1004 1.1 300135 299 3 U 10141248 1 48 2 2
> 2.0
> 1500 1502 1.2 449069 299 3 U 15210624 1 48 2 2
> 2.0
> peak IOPS: 91% of 2.4.5pre1
>
> 2.4.5pre4
> wouldn't run (stale NFS file handle error)
>
> 2.4.5pre5
> wouldn't run (stale NFS file handle error)
>
> 2.4.5pre6
> wouldn't run (stale NFS file handle error)
>
> 2.4.7
> 500 497 1.2 149206 300 3 U 5070624 1 48 2 2
> 2.0
> 1000 1005 1.5 300503 299 3 U 10141248 1 48 2 2
> 2.0
> 1500 1502 1.3 449232 299 3 U 15210624 1 48 2 2
> 2.0
> peak IOPS: 65% of 2.4.5pre1
>
> 2.4.8pre1
> wouldn't run
>
> 2.4.8pre4
> 500 497 1.1 149180 300 3 U 5070624 1 48 2 2
> 2.0
> 1000 1002 1.2 299465 299 3 U 10141248 1 48 2 2
> 2.0
> 1500 1502 1.3 449190 299 3 U 15210624 1 48 2 2
> 2.0
> INVALID
> peak IOPS: 54% of 2.4.5pre1
>
> 2.4.8pre6
> 500 497 1.1 149168 300 3 U 5070624 1 48 2 2
> 2.0
> 1000 1004 1.3 300246 299 3 U 10141248 1 48 2 2
> 2.0
> 1500 1502 1.3 449135 299 3 U 15210624 1 48 2 2
> 2.0
> peak IOPS 55% of 2.4.5pre1
>
> 2.4.8pre7
> 500 498 1.5 149367 300 3 U 5070624 1 48 2 2
> 2.0
> 1000 1006 2.2 301829 300 3 U 10141248 1 48 2 2
> 2.0
> 1500 1502 2.2 449244 299 3 U 15210624 1 48 2 2
> 2.0
> peak IOPS: 58% of 2.4.5pre1
>
> 2.4.8pre8
> 500 597 8.3 179030 300 3 U 5070624 1 48 2 2
> 2.0
> 1000 1019 6.5 304614 299 3 U 10141248 1 48 2 2
> 2.0
> 1500 1538 4.5 461335 300 3 U 15210624 1 48 2 2
> 2.0
> peak IOPS: 48% of 2.4.5pre1
>
> 2.4.8
> 500 607 7.1 181981 300 3 U 5070624 1 48 2 2
> 2.0
> 1000 997 7.0 299243 300 3 U 10141248 1 48 2 2
> 2.0
> 1500 1497 2.9 447475 299 3 U 15210624 1 48 2 2
> 2.0
> INVALID
> peak IOPS: 45% of 2.4.5pre1
>
> 2.4.9pre2
> wouldn't run (NFS readdir errors)
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/