Well, I might be analyzing it wrong but, the same driver that I'm gonna use for this shared disk
stuff can also be enabled for ethernet emulation. The reason I'm saying that the latency of waking
up a kernel thread increased somewhere between RedHat's 2.4.3 and 2.4.9 is that with the packet
receive handler running in a tasklet I get these ping-pong numbers (measured with /bin/ping) :
[root@damd1 root]# ping sci4
PING sci4 (192.168.4.4) from 192.168.4.3 : 56(84) bytes of data.
64 bytes from sci4 (192.168.4.4): icmp_seq=0 ttl=255 time=238 usec
64 bytes from sci4 (192.168.4.4): icmp_seq=1 ttl=255 time=176 usec
64 bytes from sci4 (192.168.4.4): icmp_seq=2 ttl=255 time=200 usec
64 bytes from sci4 (192.168.4.4): icmp_seq=3 ttl=255 time=177 usec
64 bytes from sci4 (192.168.4.4): icmp_seq=4 ttl=255 time=172 usec
64 bytes from sci4 (192.168.4.4): icmp_seq=5 ttl=255 time=156 usec
64 bytes from sci4 (192.168.4.4): icmp_seq=6 ttl=255 time=160 usec
64 bytes from sci4 (192.168.4.4): icmp_seq=7 ttl=255 time=177 usec
64 bytes from sci4 (192.168.4.4): icmp_seq=8 ttl=255 time=173 usec
For simplicity I'll say ~200usec. When I change the receive handler from a tasklet to a kernel
thread, the numbers look like this :
[root@damd1 root]# ping sci4
PING sci4 (192.168.4.4) from 192.168.4.3 : 56(84) bytes of data.
64 bytes from sci4 (192.168.4.4): icmp_seq=0 ttl=255 time=4.215 msec
64 bytes from sci4 (192.168.4.4): icmp_seq=1 ttl=255 time=5.728 msec
64 bytes from sci4 (192.168.4.4): icmp_seq=2 ttl=255 time=3.825 msec
64 bytes from sci4 (192.168.4.4): icmp_seq=3 ttl=255 time=6.521 msec
64 bytes from sci4 (192.168.4.4): icmp_seq=4 ttl=255 time=5.700 msec
64 bytes from sci4 (192.168.4.4): icmp_seq=5 ttl=255 time=5.666 msec
64 bytes from sci4 (192.168.4.4): icmp_seq=6 ttl=255 time=6.495 msec
64 bytes from sci4 (192.168.4.4): icmp_seq=7 ttl=255 time=5.631 msec
64 bytes from sci4 (192.168.4.4): icmp_seq=8 ttl=255 time=6.480 msec
A bit up and down, but all of them in the msec range wich is means that the ping-pong latency has
increased atleast 100 times.
This might of course be related to the network stack (i.e it doesn't like to netif_rx() from user
context, or just a high turnaround time when trying to send the response) but on 2.4.3-12 it didn't
behave like this. The strange thing is that when I run a network benchmark such as netperf, I get
nice numbers on UDP bandwith (one-way traffic) :
(with kernel thread):
UDP UNIDIRECTIONAL SEND TEST to sci4
Socket Message Elapsed Messages
Size Size Time Okay Errors Throughput
bytes bytes secs # # MBytes/sec
262144 32768 10.00 55958 0 174.92
262144 10.00 55925 174.82
(with tasklet):
UDP UNIDIRECTIONAL SEND TEST to sci4
Socket Message Elapsed Messages
Size Size Time Okay Errors Throughput
bytes bytes secs # # MBytes/sec
262144 32768 10.01 53648 0 167.55
262144 10.01 53648 167.55
The TCP stack seems to like it better with tasklets :
(with kernel thread):
TCP STREAM TEST to sci4
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. MBytes/sec
262144 262144 262144 10.02 16.24
(with tasklet):
TCP STREAM TEST to sci4
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. MBytes/sec
262144 262144 262144 10.00 116.00
Regards,
-- Steffen Persvold | Scalable Linux Systems | Try out the world's best mailto:sp@scali.com | http://www.scali.com | performing MPI implementation: Tel: (+47) 2262 8950 | Olaf Helsets vei 6 | - ScaMPI 1.13.8 - Fax: (+47) 2262 8951 | N0621 Oslo, NORWAY | >320MBytes/s and <4uS latency - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/