Re: system call fails with ENOMEM after long operation

[Mikulas Patocka]

> So why is the kernel having difficulty allocating memory for its
> internal operations? We are suspecting memory fragmentation issues 
> (at the application level which might have adverse sideefects for the
> kernel). Is it something that we can do so that the kernel will be
> able to allocate the needed memory for carrying out the system call?

Try this patch.

Mikulas

--- linux/mm/page_alloc.c__	Thu Mar  9 15:12:31 2000
+++ linux/mm/page_alloc.c	Tue Mar 21 09:50:12 2000
@@ -237,6 +237,15 @@
 	RMQUEUE_TYPE(order, 1);
 	spin_unlock_irqrestore(&page_alloc_lock, flags);
 
+	if (!(current->flags & PF_MEMALLOC) && (gfp_mask & __GFP_WAIT)) {
+		int freed;
+		current->trashing_mem = 1;
+		current->flags |= PF_MEMALLOC;
+		freed = try_to_free_pages(gfp_mask);
+		current->flags &= ~PF_MEMALLOC;
+		if (freed) goto ok_to_allocate;
+	}
+
 nopage:
 	return 0;
 }