Linux-kernel mailing list archive 1999-36,: Re: > 15,000 Simultaneous Connections

# Re: > 15,000 Simultaneous Connections

Rogier Wolff (R.E.Wolff@BitWizard.nl)
Tue, 7 Sep 1999 15:31:00 +0200 (MEST)

- Messages sorted by: [ date ][ thread ][ subject ][ author ]
- Next message: Juan Carlos Castro y Castro: "Re: numerical username"
- Previous message: Thorsten Pastoors: "TCP retransmission in 2.2.5"
- Next in thread: Mike Jagdis: "Re: > 15,000 Simultaneous Connections"
- Reply: Mike Jagdis: "Re: > 15,000 Simultaneous Connections"
- Reply: Chuck Lever: "Re: > 15,000 Simultaneous Connections"

Mike Jagdis wrote:
> On Tue, 7 Sep 1999, Rogier Wolff wrote:
>
> > If you have fd-returning calls
> > (accept, socket, open, pipe. any more?) mark the new fds as
> > interesting, you can catch those too without having to touch the
> > fdset.
>
> How can you know in advance whether the user side will consider
> a new fd interesting? How can you know in advance which set to
> add the new fd to?

You can mark it as NEW, and then check that bit-position on the
three bitmaps.

> > If we indeed optimize select only to check every 100ms wether the
> > cached, remembered fdset is the same as the given fdset, we end up
> > with the situation where select may only notice a change in the fdset
> > a bit later. Applications shouldn't really care: The result is the
> > same as when data would've arrived a bit later.
>
> I'm pretty certain that having select return fds that weren't
> expected, block because it checked fds that had been removed,
> or fail to block because it didn't see fds that were added
> would break programs.

Ho stop, of course if the kernel notices that fd 1234 has read data
available, this fd isn't returned to the user without checking that
the user has marked fd 1234 in the read-availability.

> That is a change to the semantics of
> select and therefore should no longer be called select. Once
> you're committed to a new API you might as well design it
> right rather than make it look sort-of-like an existing API
> which it subtly different to.

Agreed.

> > Linus will probably reject this on a complexity argument.
>
> I would hope so :-). There are plenty of complex things you
> could do but there are also some surprisingly simple ways.
>
> > I scanned the article taht someone mentioned. Implementing that API
> > sounds like a good idea too. (you register interest in an fd, and then
> > read the "event queue")
>
> How about a "device like" API where you write "pollfd like" structs,
> the "device" listens for events and delivers "pollfd like" structs
> representing them when you read it? To do that all you need is a
> way to have your wait_queue_ts queued on some sort of list when
> a wake_up happens. I've already done that to optimize the blocked
> select/poll cases and it's pretty simple.

I would then prefer to go with the API that the digital guys suggested
in: http://www.cs.rice.edu/~gaurav/papers/usenix99.ps

#define EVENT_READ 0x01
#define EVENT_WRITE 0x02
#define EVENT_EXCEPT 0x03

typedef struct {
int fd;
unsigned mask;
} event_descr_t;

int declare_interest (int fd, int interestmask, int *statemask);
int get_next_event (int array_max, event_descr_t *ev_array, struct timeval *timeout);

If you want, you can then simulate select in the library.

int select(int n, fd_set *readfds, fd_set *writefds,
fd_set *exceptfds, struct timeval *timeout)
{

It should just pass on the call to the system call for small nmax.

if (n < 256)
return sys_select (n, readfds, writefds, exceptfds, timeout);

When nmax goes above a certain threshold, it will become viable to
compare an old cached version of the arrays to the current one,
register the differences with declare_interest, and then empty the
event queue. (For the example, I'll work bit_wise, but the actual
implementation will work with a long at a time.)

for (i=0;i<n;i++) {
if ((FD_ISSET(i, stored_readfds) != FD_ISSET(i, readfds)) ||
(FD_ISSET(i, stored_writefds) != FD_ISSET(i, writefds)) ||
(FD_ISSET(i, stored_exceptfds) != FD_ISSET(i, exceptfds))) {
declare_interest (i, FD_ISSET(i, readfds)?EVENT_READ:0 |
FD_ISSET(i, writefds)?EVENT_WRITE:0 |
FD_ISSET(i, exceptfds)?EVENT_EXCEPT:0);
}
}
memcpy (stored_readfds, readfds, n/8);
memcpy (stored_writefds, writefds, n/8);
memcpy (stored_exceptfds, exceptfds, n/8);

memset (readfds, 0, n/8);
memset (writefds, 0, n/8);
memset (exceptfds, 0, n/8);

rv = get_next_event (ARR_SIZE, ev_arr, timeout);
if (rv <= 0) return rv;
n = 0;
do {
for (i=0;i<rv;i++) {
if (ev_arr[i].mask & EVENT_READ) {n++;FD_SET (ev_arr[i].fd, read_fds);}
if (ev_arr[i].mask & EVENT_WRITE) {n++;FD_SET (ev_arr[i].fd, write_fds);}
if (ev_arr[i].mask & EVENT_EXCEPT) {n++;FD_SET (ev_arr[i].fd, except_fds);}
}
rv = get_next_event (ARR_SIZE, ev_arr, timeout);
} while rv > 0;
return n;

This completes the "userlevel" emulation of "select". This will
perform quite well for many applications (ones that don't change half
the "large" set between each select call).

Roger.

    --
    ** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
    *-- BitWizard writes Linux device drivers for any device you may have! --*
    ------ Microsoft SELLS you Windows, Linux GIVES you the whole house ------

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.rutgers.edu
    Please read the FAQ at http://www.tux.org/lkml/

- Next message: Juan Carlos Castro y Castro: "Re: numerical username"
- Previous message: Thorsten Pastoors: "TCP retransmission in 2.2.5"
- Next in thread: Mike Jagdis: "Re: > 15,000 Simultaneous Connections"
- Reply: Mike Jagdis: "Re: > 15,000 Simultaneous Connections"
- Reply: Chuck Lever: "Re: > 15,000 Simultaneous Connections"