Re: ps performance sucks (was Re: dcache_rcu [performance results])

[Rusty Russell]

On Tue, 5 Nov 2002 19:34:47 -0500 (EST)
Alexander Viro <[e-mail]> wrote:
> On Tue, 5 Nov 2002, Martin J. Bligh wrote:
> 
> > It's not a few files if you have large numbers of tasks. It's an 
> > interface that fundamentally wasn't designed to scale, and futzing
> > around tweaking the thing isn't going to cut it, it needs a different
> > design. I'm not proposing throwing out any of the old simple interfaces,
> > just providing something efficient as a data gathering interface for
> > those people who wish to use it.
> 
> That's odd, to say the least.  Userland side is at least linear by
> number of tasks, regardless of the way you gather information.  So
> I really wonder how opening O(number of tasks) files can show up
> when you scale the things up - pure userland parts will grow at
> least as fast as that.

And I look forward to your "du" interface the kernel, too.  It uses
this terrible method of statting every file!

Now, according to wli, there's a real problem with starvation by saturating
the read side of the tasklist_lock so eg. the write_lock_irq(&tasklist_lock)
in exit.c's release_task causes a CPU to spin for ages (forever?) with
interrupts off.  This needs fixing, be it RCU or making that particular
lock give way to writers, or some other effect.

But the "ps takes too much time from my benchmarks" is a "don't do that".
Or you may really hate the /dev/kmem hack, but it's simple and effective.
The "hundreds of users doing ps/top" is easily solved with a "ps" daemon.

Fixing top is probably a worthwhile thing to do, and you'll note that after
the first iteration it should be rarely neccessary to read every "stat"
proc file.

Optimize userspace before putting a hack into the kernel, *please*!

Rusty.
-- 
   there are those who do and those who hang on and you don't see too
   many doers quoting their contemporaries.  -- Larry McVoy