Re: [PATCH][IO_APIC] 2.5.63bk7 irq_balance improvments / bug-fixes

[Kamble, Nitin A]

Hi Andrew, Kai,

  The bouncing is seen because of the round robin IRQ distribution in
some 
particular cases. In some cases, (such as single heavy interrupt source
in 
a 2way SMP system) binding heavy interrupt sources to different cpus is
not 
going to remove the complete imbalance. In that case we fall back to
Ingo's 
round robin approach. We have studied the previous round robin interrupt

distribution implemented in the kernel, and we found that, at very high 
interrupt rate, the performance of the system increased with the
increasing 
period of the round robin distribution. Please see the original LKML
posting 
for more details. 
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0212.2/1122.html 

So when if there is significant imbalance left after binding the IRQs to
cpus, 
there are two options now,

  1. Do not move around. Let the significant imbalance stick on a
particular 
     cpu.

  2. Or move the heavy imbalance around all the cpus in the round robin 
     fashion at high rate.

Also we can have either of the option configurable in the kernel.

Both the solutions will eliminate the bouncing behavior. The current 
implementation is based on the option 2, with the only difference of 
lower rate of distribution (5 sec).  The optimal option is workload 
dependant. With static and heavy interrupt load, the option 2 looks 
better, while with random interrupt load the option 1 is good enough.

Thanks & Regards,
Nitin