Re: [PATCH][IO_APIC] 2.5.63bk7 irq_balance improvments / bug-fixes

Jeff Garzik (jgarzik@pobox.com)
Tue, 04 Mar 2003 23:38:11 -0500


Kamble, Nitin A wrote:
> There are few issues we found with the user level daemon approach.

Thanks much for the response!

> Static binding compatibility: With the user level daemon, users can
> not
> use the /proc/irq/i/smp_affinity interface for the static binding of
> interrupts.

Not terribly accurate: in "one-shot" mode, where the daemon balances
irqs once at startup, users can change smp_affinity all they want.

In the normal continuous-balance mode, it is quite easy to have the
daemon either (a) notice changes users make or (b) configure the daemon.
The daemon does not do (a) or (b) currently, but it is a simple change.

> There is some information which is only available in the kernel today,
> Also the future implementation might need more kernel data. This is
> important for interfaces such as NAPI, where interrupts handling changes
> on the fly.

This depends on the information :) Some information that is useful for
balancing is only [easily] available from userspace. In-kernel
information may be easily exported through "sysfs", which is designed to
export in-kernel information.

Further, for NAPI and networking in general, it is recommended to bind
each NIC to a single interrupt, and never change that binding.
Delivering a single NIC's interrupts to multiple CPUs leads to a
noticeable performance loss. This is why some people complain that
their specific network setups are faster on a uniprocessor kernel than
an SMP kernel.

I have not examined interrupt delivery for other peripherals, such at
ATA or SCSI hosts, but for networking you definitely want to statically
bind each NIC's irqs to a separate CPU, and then not touch that binding.

Best regards, and thanks again for your valuable feedback,

Jeff

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/