HT and idle = poll

[Andrew Theurer]

The test:  kernbench (average of  kernel compiles5) with -j2 on a 2 physical/4 
logical P4 system.  This is on 2.5.64-HTschedB3:

idle != poll: Elapsed: 136.692s User: 249.846s System: 30.596s CPU: 204.8%
idle  = poll: Elapsed: 161.868s User: 295.738s System: 32.966s CPU: 202.6%

A 15.5% increase in compile times.

So, don't use idle=poll with HT when you know your workload has idle time!  I 
have not tried oprofile, but it stands to reason that this would be a 
problem.  There's no point in using idle=poll with oprofile and HT anyway, as 
the cpu utilization is totally wrong with HT to begin with (more on that 
later).

Presumably a logical cpu polling while idle uses too many cpu resources 
unnecessarily and significantly affects the performance of its sibling. 

-Andrew Theurer