Re: top/ps incorrectly reporting process execution times

[Karim Yaghmour]

I guess the real issue here is: "How much precision does the user want?"

I'm afraid that the answer to that questions is: it depends.

You are right, exact accounting might be overkill for most people, but
I've met a lot of people who would really like to know what is exactly
happening in their system (and this has been requested on the kernel mailing
list). The tracing infrastructure provides this possibility, amongst other
possibilities. I think that giving the choice to the user would be welcomed
by most. By choice, I mean a compile-time option that would enable the user
to choose between statistical profiling and exact accounting.

You are also right about the fact that the tracing overhead depends on the
quantity of events generated, this is discussed in my Usenix paper. This
is an inherent side-effect to tracing. I'm open to any suggestions on
how to circumvent this.

All in all, I do think that it would be useful if it were possible to choose
the accuracy of the results you get. As I pointed out in my last mail, choice
is better than no choice. This goes for the conventional way of accounting,
for the McCanne/Torek technique and for LTT.

John Heidemann wrote:
> 
> You're right...if you define the problem as exact accounting,
> statisical approaches won't cut it.
> 
> However, for the purposes of top and ps (the subject of the original
> message :-), exact accounting seems like overkill.  After all, top
> only updates the screen every 5 seconds (!) and ps has process startup
> overhead, so what's 100ms error?  The requirement for top/ps is that
> there not be awful degenerate cases of consistent error.  The McCanne
> and Torek paper shows a common class of awful degenerate cases and how
> to fix them (by avoiding aliasing that comes from doing profiling
> stats and other things off the same timer).
> 
> Of course, if we can get exact accounting for free, then it's good for
> top/ps.  But LTT and even TSC introduces overead proportional to the
> number of "events" (context switches, irqs, whatever depending on how
> exact you want to be), rather than a fixed background overhead.  It's
> not obvious to me why that overhead is warrented for top/ps.  Even if
> it's <2%, look at how carefully optimized the rest of the linux
> context switch is.
> 
> Don't get me wrong... tracing infrastructure is *really useful* when
> doing performance debugging.  I used kitrace a lot for this (see the
> Software Pratices and Experience paper---you can put tracepoints
> anywhere at runtime, but you do so in assembly :-().  LTT seems like a
> nice complementary piece of work and it would be useful infrastructure
> to have, but it looks to me like a debugging tool and not what should
> required to make top/ps work.
> 
>    -John Heidemann
> 

===================================================
                 Karim Yaghmour
               [e-mail]
          Operating System Consultant
 (Linux kernel, real-time and distributed systems)
===================================================