Re: send_sigio() scalability

[Richard Gooch]

Alexander Kjeldaas writes:
> On Mon, May 17, 1999 at 01:17:59PM +1000, Richard Gooch wrote:
> >   Hi, all. I just noticed that send_sigio() walks the task list,
> > looking for the process(es) to send a signal to. This appears to be a
> > potential scalability problem, as a large number of tasks is going to
> > slow this down.
> > 
> > Has anyone done any benchmarking to evaluate the effect of this? In
> > the absence of numbers, how about some convincing handwaving? Is it
> > worth exploring options to fix this?
> > 
> > I can think of one quick and simple hack to fix this for 90% (maybe
> > 99%) of cases: record the task pointer at fcntl() time. Then at
> > send_sigio() time, if the recorded pid and task match, skip the
> > task list walk.
> 
> In the case where send_sigio is sending a signal to a specific
> process, why isn't it using find_task_by_pid()?.

Good question. I think the single-process case is the general
case. Certainly, for a WWW server which wants to use Linux completion
ports, it doesn't make sense to have multiple processes signalled on
an event, since then we get back to the thundering hurd(sic) problem.

I'll send in a patch to use find_task_by_pid(). Thanks for pointing it
out.

Larry: I know this isn't the Grand Unified Abstraction[tm] you were
hoping for, but how does this solution grab you? It's certainly less
hackish than the simple fix I suggested, and should work reasonably
well provided the underlying hash function distributes well. I quite
like it.

> For the other case, I'm working on a more general solution to most
> of the for_each_task() uses in the kernel.  I made a patch last year
> that adds fast for_each_task_in_pgrp(), and
> for_each_task_in_session() macros.  I have measurements that prove
> that the patch helps a lot for the fork()+exit() case.  Please look
> at
>
> http://www.guardian.no/~astor/pidhash/pidhash.gif

Looks impressive. I'm glad someone is looking at this problem.

> for a graph of this that I made for the patch for Linux 2.1.90.  It
> shows that when you have a lot of processes running, doing a
> fork()+exit() takes a long time.  This is due to for_each_task()'ing
> in exit to send signals.  I'd guess that send_sigio() will perform
> similar to the above graph.

That feels right.

> I've started to port the patch to 2.2.8.

How about doing it for 2.3.3 instead? My guess is that 2.2.x is
off-limits for this kind of development.

				Regards,

					Richard....