Re: Writeout in recent kernels/VMs poor compared to last -ac

[Andrea Arcangeli]

On Sat, Jan 12, 2002 at 10:17:39AM -0500, Adam Kropelin wrote:
> I recently began regularly transferring large (600 MB+) files to my
> Linux-based fileserver and have noticed what I would characterize as poor
> writeout behavior under this load. I've done a bit of comparison testing
> which may help reveal the problem better.
> 
> Disclaimer: I did not choose this test because it is scientific or stresses
> the system in any particular way. I use it because it's an operation which I
> perform frequently and so the faster it goes, the broader I smile. This may
> be an entirely inappropriate load to optimize for, and if so I understand.
> 
> Test consisted of FTPing in (Linux was the destination) a 650 MB file and
> timing the transfer (from the client's perspective) which capturing "vmstat
> 1" output on the server. Server hardware is Dual PPro 200, 160 MB RAM, 512
> MB swap, destination filesystem is VFAT[0] on cpqarray partition on a RAID5
> logical drive. Root and swap are not on the cpqarray.
> 
> Here are my results (average of 2 runs, each from a clean reboot with all
> unneeded services disabled):
> 
> 2.4.17: 6:52
> 2.4.17-rmap11a: 6:53
> 2.4.18-pre2aa2: 7:10
> 2.4.13-ac7: 5:30 (!)
> 2.4.17 with cpqarray driver update from -ac: 6:30
> 
> The last test was just to see if -ac's better performance had anything to do
> with the driver update. Apparently it had little or nothing to do with it.
> 
> The vmstat output is very revealing. All tests except for -ac show a strong
> oscillation on the blocks out (this is a representative sample from stock
> 2.4.17 but the other recent kernels show essentially the same behavior):
> 
>    procs                      memory    swap          io     system
> cpu
>  r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy
> id
>  0  1  1      0   4408   2712 118756   0   0     0     0  109    16   0   0
> 100
>  0  1  1      0   4436   2716 118724   0   0     1  3913  287    13   0   8
> 92
>  0  1  0      0   4436   2716 118724   0   0     0     0  118    11   0   0
> 100
>  0  0  0      0   4352   2724 118816   0   0     8     0 3639   203   2  28
> 70
>  0  0  0      0   4420   2736 118752   0   0    11     0 4530   259   0  33
> 67
>  0  0  0      0   4348   2744 118816   0   0    10     0 4273   245   0  42
> 58
>  1  0  0      0   4376   2756 118772   0   0    11     0 4551   246   1  39
> 60
>  0  1  1      0   4364   2760 118724   0   0     4  6730 1710    93   1  19
> 80
>  0  1  1      0   4364   2760 118724   0   0     0     0  108     9   0   0
> 100
>  0  1  1      0   4364   2760 118724   0   0     0  3667  117     9   0   2
> 98
>  0  1  1      0   4364   2760 118724   0   0     0     0  125     8   0   0
> 100
>  1  0  1      0   4364   2760 118724   0   0     0  3819  124    10   0   2
> 98
>  0  1  1      0   4372   2760 118704   0   0     0  2055  195    10   0   5
> 95
>  0  1  1      0   4372   2760 118704   0   0     0     0  120     6   0   0
> 100
>  0  1  1      0   4408   2760 118668   0   0     0  2415  203    16   0   4
> 96
>  0  1  1      0   4472   2760 118608   0   0     0  4321  280    18   1   6
> 93
>  0  2  1      0   4468   2760 118608   0   0     0     0  112     8   0   0
> 100
>  0  1  1      0   4352   2760 118724   0   0     1  3232  227     9   0  10
> 90
>  0  1  1      0   4352   2760 118724   0   0     0     0  108     8   0   0
> 100
>  0  0  0      0   4440   2768 118696   0   0     7     0 3233   179   1  25
> 74
>  1  0  0      0   4448   2776 118680   0   0    11     0 4417   253   0  36

I think you simply want to trigger the soft-bdflush event earlier, with
-aa something like this may do the trick:

	echo 5 500 64 256 500 3000 60 2 0 > /proc/sys/vm/bdflush

this way you'll wakeup as soon as 5% of the 118mbytes (+ free memory,
none in this case) are dirty, and bdflush will stop as soon as the level
is back to 2% (then kupdate will take care of the 2%). Those suggested
values may be too strict but this way you should get the idea if it
helps somehow or not :)

Andrea