Re: [PATCH][2.5][RFC] Using xAPIC apic address space on !Summit

James Cleverdon (jamesclv@us.ibm.com)
Fri, 13 Dec 2002 11:32:06 -0800


On Thursday 12 December 2002 09:53 pm, Steffen Persvold wrote:
> On Thu, 12 Dec 2002, James Cleverdon wrote:
> > On Thursday 12 December 2002 07:26 pm, Zwane Mwaikambo wrote:
> > > On Thu, 12 Dec 2002, Nakajima, Jun wrote:
> > > > BTW, we are working on a xAPIC patch that supports more than 8 CPUs
> > > > in a generic fashion (don't use hardcode OEM checking). We already
> > > > tested it on two OEM systems with 16 CPUs.
> > > > - It uses clustered mode. We don't want to use physical mode because
> > > > it does not support lowest priority delivery mode.
> > >
> > > Wouldn't that only be for all including self? Or is the documentation
> > > incorrect?
> > >
> > > Thanks,
> > > Zwane
> >
> > I'm not sure I understand your question. Lowest Priority delivery mode
> > only works with logical interrupts. (I've tried it with physical intrs.
> > It fails miserably.) The "all including self" and "all excluding self"
> > destination shorthands don't do lowest priority arbitration. They always
> > deliver the interrupt to the CPUs mentioned in the shortand.
> >
> > Lowest priority delivery mode isn't _too_ useful in Linux yet. It would
> > be nice to preferentially target idle CPUs with interrupts in real time.
> > That means changing each CPU's Task Priority Register (TPR) to represent
> > how busy it is. I've got some patches to do that, but haven't posted
> > them as anything more than a RFC.
>
> Hmm, I though the APIC routing patch found in the LSE project
> (http://sourceforge.net/projects/lse/) did this already. Atleast I've
> tested this patch on a couple of Dual E7500 Xeon boxes (kernel 2.4.20) and
> it distributes interrupts nicely.
>
> However with the patch enabled, the interrupt latency on for example the
> Intel GbE 82544GC devices increased a fraction with this patch (a
> microsecond or two).
>
> Regards,

Sure, Dave Olien's patch adjusted the TPR. However, he wrote that for the
classic APIC; it does most of the priority adjustments in the lower nibble of
the TPR's value. xAPIC routing is done via HW in the PCI-to-host bridge
chips. There they keep a copy of each CPU's TPR value in eight XTPR
registers for lowest priority interrupt routing -- but only the TPR's upper
nibble. So, Dave's patch is less useful on xAPIC systems.

I came up with something simpler. Just 2 lines added to idle_cpu() and
do_IRQ respectively. It's a hack but it seemed useful.

Interesting that it would be a microsecond slower. Maybe that's the time it
takes to adjust the TPR. One more reason to keep those adjustments as simple
as possible.

-- 
James Cleverdon
IBM xSeries Linux Solutions
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot com

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/