Well you have a gigabit router :-)
How is your routing setup? Packet size?
Also you'll never get increased performance of a single flow with SMP.
Aggregated performance possible at best. I've been fighting with for some
time too.
You have some important data in /proc/net/softnet_stat which are per cpu
packets received and "cpu collisions" should interest you.
As far as I understand there no serialization in forwarding path except where
it has to be -- when we add softirq's from different cpu into a single device.
This seen in "cpu collisions"
Also here we get into inherent SMP cache bouncing problem with TX interrupts
When TX has skb's which are processed/created in different CPU's. Which CPU
gonna take the interrupt? No matter how we do we run kfree we gona see a lot
of cache bouncing. For systems that have same in/out interface smp_affinity
can be used. In practice this impossible for forwarding.
And this bouncing hurts especially for small pakets....
A litte TX test illustrates. Sender on cpu0.
UP 186 kpps
SMP Aff to cpu0 160 kpps
SMP Aff to cpu0, cpu1 124 kpps
SMP Aff to cpu1 106 kpps
We are playing some code that might decrease this problem.
Cheers.
--ro
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/