Re: Gigabit Linux Server Bottlenecks

[Zachary Amsden]

> On Thu, 10 Feb 2000, Zachary Amsden wrote:
> > > > Well in order to use it and really gain on it for transmit, you want
> > > > scatter/gather skbuffs, something which I really want to look at
> > > > (provided I can find the time). If you are doing the data copy anyway,
> > > > there checksum is almost free.
> 
> Is the idea that the skb data pointer instead will be a kiovec ? So that
> the kiovec can point to the locked down COW-marked pages in memory where
> the data came from, released when TCP finally acks it ? 

Yes, that is correct.

> But the tail and head of the skb, that will still be in skb-local memory
> right ?

Not sure what you mean by skb-local memory, but I think you are saying that 
any packet headers/trailers will be in kernel space.  If so that is correct.

> 
> A scatter gather ethernet driver can then setup a DMA chain to send skb
> head, the kiovec's, and the skb tail, for each skb.
> 
> I'd love to see that work - we run linux on embedded HW and every less
> copying of data can be a pretty performance win. And our ethernet HW is
> already scatter-gather.

Actually, it will be a performance loss for the common case.  It is only for 
high throughput apps that are aware of not to touch the buffers while packets 
are in flight that it will be a win.  So it would be best controlled by having 
a socket option that defaults to off.

> Yeah, it'd be pretty stupid if an application forces the COW to trigger
> all the time, then you'd still be copying data :)

My point exactly.

> But like others said, sendfile is probably the most excellent first
> subject to try this stuff on!

Definitely, but sendfile won't be as clear a win when you have a high 
throughput daemon that does any more than throw files down the pipe.  In that
case, direct I/O from user space (possibly mmapped) may be faster.

It will be interesting to see whether it is worth the effort to do direct 
userspace I/O, or whether sendfile wins.

-- 
Zachary Amsden  [e-mail]  [phone]  09U-510  Core Protocols