Using a single processor on the receive side makes the problem go away.
I see no problems on the receiver with one cpu removed.
rcc2 29% netstat -i myri0 
Kernel Interface table
Iface   MTU Met    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0   1500   0    37857      0      0      0    36404      0      0      0 BRU
lo    16192   0       46      0      0      0       46      0      0      0 LRU
myri0  9000   0 20564644      0      0      0      312      0      0      0 BRU
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/