Re: > 15,000 Simultaneous Connections

[Mike Jagdis]

On Tue, 7 Sep 1999, Rogier Wolff wrote:

> If you have fd-returning calls
> (accept, socket, open, pipe. any more?) mark the new fds as
> interesting, you can catch those too without having to touch the
> fdset.

How can you know in advance whether the user side will consider
a new fd interesting? How can you know in advance which set to
add the new fd to?

> If we indeed optimize select only to check every 100ms wether the
> cached, remembered fdset is the same as the given fdset, we end up
> with the situation where select may only notice a change in the fdset
> a bit later. Applications shouldn't really care: The result is the
> same as when data would've arrived a bit later. 

I'm pretty certain that having select return fds that weren't
expected, block because it checked fds that had been removed,
or fail to block because it didn't see fds that were added
would break programs. That is a change to the semantics of
select and therefore should no longer be called select. Once
you're committed to a new API you might as well design it
right rather than make it look sort-of-like an existing API
which it subtly different to.

> Linus will probably reject this on a complexity argument. 

I would hope so :-). There are plenty of complex things you
could do but there are also some surprisingly simple ways.

> I scanned the article taht someone mentioned. Implementing that API
> sounds like a good idea too. (you register interest in an fd, and then
> read the "event queue")

How about a "device like" API where you write "pollfd like" structs,
the "device" listens for events and delivers "pollfd like" structs
representing them when you read it? To do that all you need is a
way to have your wait_queue_ts queued on some sort of list when
a wake_up happens. I've already done that to optimize the blocked
select/poll cases and it's pretty simple.

  You could also use that sort of interface on other systems by
using pipes to one or more processes that do select/poll on
smallish fd sets. It would be a little more overhead than kernel
side support but I have a feeling that it would be better than
using a single in-line select for moderate set sizes - and certainly
for 15000!

  Sigh... I have the feeling I'm talking myself into another
benchmark here :-).

				Mike

-- 
    A train stops at a train station, a bus stops at a bus station.
    On my desk I have a work station...
.----------------------------------------------------------------------.
|  Mike Jagdis                  |  Internet:  mailto:[e-mail]   |
|  Roan Technology Ltd.         |                                      |
|  2 Markham Mews, Broad Street |  Telephone:  [phone]        |
|  Wokingham ENGLAND            |  Fax:        [phone]        |
`----------------------------------------------------------------------'