Re: [patch] kmalloc_percpu -- 2 of 2

[Andrew Morton]

Dipankar Sarma wrote:
> 
> Hi Andrew,
> 
> On Wed, Dec 04, 2002 at 08:32:58PM -0800, Andrew Morton wrote:
> > Where in the kernel is such a large number of 4-, 8- or 16-byte
> > objects being used?
> 
> Well, kernel objects may not be that small, but one would expect
> the per-cpu parts of the kernel objects to be sometimes small, often down to
> a couple of counters counting statistics.

Sorry, "one would expect" is not sufficient grounds for incorporation of
a new allocator.  As far as I can tell, all the proposed users are in
fact allocating decent-sized aggregates, and that will remain the usual
case.

The code exists, great.  We can pull it in when there is a demonstrated
need for it.  But until that need is shown, this is overdesign.

> >
> > The slab allocator will support caches right down to 1024 x 4-byte
> > objects per page.  Why is that not appropriate?
> 
> Well, if you allocated 4-byte objects directly from the slab allocator,
> you aren't guranteed to *not* share a cache line with another object
> modified by a different cpu.

If that's a problem it can be addressed in the slab head arrays - make
sure that they are always filled and emptied in multiple-of-cacheline-sized
units for objects which are smaller than a cacheline.  That benefits all
slab users.
 
> >
> > Sorry, but you have what is basically a brand new allocator in
> > there, and we need a very good reason for including it.  I'd like
> > to know what that reason is, please.
> 
> The reason is concern about per-cpu allocation for small per-CPU
> parts (typically counters) of objects. If a driver has two counters
> counting reads and writes, you don't want to eat up a whole cacheline
> for them for each CPU per instance of the device.
> 

I don't buy it.

- If the driver has two counters per device then the storage is
  infinitesimal.

- If it has multiple counters per device (always the case) then
  the driver will aggregate them anyway.

I am not aware of any situations in which a driver has a large
(or even medium) number of small, discrete counters of this nature.
Sufficiently large to justify a new allocator.

I'd suggest that you drop the new allocator until a compelling
need for it (in real, live 2.5/2.6 code) has been demonstrated.