Re: memory allocation problems

[Mark Hahn]

> > note, though, that you *CAN* actually malloc a lot more than 1G: you
> > just have to avoid causing mmaps that chop your VM at
> > TASK_UNMAPPED_BASE:
> 
> Neat trick.  I didn't realize that you could avoid allocating the mmap()
> buffers for stdin and stdout.

noone ever said you had to use stdio.  or even use libc, for that matter!

> As was pointed out to me in January, another solution for i386 would be to
> fix a maximum stack size and have the mmap() allocations grow downward
> from the "top" of the stack (3GB - max stack size).  I'm not sure why that
> is not currently done.

problems get fixed when there's some pain involved: people bumping 
into a limit, or painfully bad code, etc.  not enough people are 
feeling any pain about the current design.

this (and the "move TASK_UNMAPPED_BASE" workaround) have been known
for years; I think someone even coded up a "grow vmareas down" patch
the last time we all discussed this.

> I once wrote a tiny patch to do this, and ran it successfully for a couple
> days, but knowing so little about the kernel I probably did it in a
> completely wrong, inefficient way.  For example, some of the vma
> structures are sorted in increasing address order, and so perhaps to do
> this properly one should change them to decreasing address order.

oh, I guess you did the patch ;)
seriously, resubmit it when 2.5 opens up.  the fact is that we currently
have two things that grow up, and one that grows down.  so obviously,
one up-grower must have an arbitrary limit.  switching vma's to down-growing
is a good solution, since it's actually *good* to limit stack growth.  
I wonder whether fortraners still put all their data on the stack;
they wouldn't be happy ;)

a simple workaround would be to turn TASK_UNMAPPED_AREA into a variable,
either system-wide or thread-specific (like ia64 already has!).  that's 
compatible with the improved vmas-down approach, too.

regards, mark hahn.