Linux-kernel mailing list archive 1997-48,: patch for 2.1.69 buffer.c

# patch for 2.1.69 buffer.c

Bill Hawes (whawes@star.net)
Tue, 02 Dec 1997 14:51:02 -0500

- Messages sorted by: [ date ][ thread ][ subject ][ author ]
- Next message: Bill Hawes: "Re: Linux 2.1.69.."
- Previous message: Chris Adams: "Re: memory leak detector, 2.0.32, patch"
- Next in thread: Dr. Werner Fink: "Re: patch for 2.1.69 buffer.c"
- Reply: Dr. Werner Fink: "Re: patch for 2.1.69 buffer.c"
- Reply: Bill Hawes: "Re: patch for 2.1.69 buffer.c"
- Reply: Bill Hawes: "Re: patch for 2.1.69 buffer.c"
- Reply: Gadi Oxman: "Re: patch for 2.1.69 buffer.c"

This is a multi-part message in MIME format.
--------------24F0CEE6545D3CCF8E46F22D
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I've been refining a refill_freelist patch for quite a while (since
2.1.50 or so), and have it ready for wider testing. The changes here are
executed only after the initial allocation and buffer list search have
failed, so will affect operation only under low memory and/or high load
(many dirty/locked buffers.)

The changes are as follows:
(1) Do a non-blocking wakeup if there are any dirty buffers. If we
reached this section of code without finding any available buffers, we
want to make sure any dirty buffers are flushed.

(2) Allocate buffers to reach half the goal, but don't bother repeating
the buffer list search. This is a minor change, but since the GFP_BUFFER
allocations don't block, it just wastes CPU time to search the lists
again.

(3) Return now if we got any buffers, so that the current task can
proceed.

(4) If no buffers were found or allocated, do a blocking wakeup of
bdflush, and then find a locked buffer to wait on. This is the most
important change from prior practice, as it forces an explicit wait on a
locked buffer before proceeding with atomic allocations.

In the majority of cases that this code is reached, the system will have
plenty of buffers, but the buffers will all locked. Performing atomic
allocations will drain the system's reserved pages, and just calling
schedule() wastes CPU time by repeatedly searching the lists instead of
waiting for an unlock.

(5) In the unlikely event that no locked buffers are found, do an atomic
allocation, but only we still have at least half of the reserved pages.
Allowing this limited access covers the case of using a device with
non-standard sized buffers.

This code should cover all of the cases pretty well, but it isn't a
panacea for other memory management problems. In particular, if you're
having a problem with fragmentation and allocation of network buffers,
this patch probably won't help much, if at all. But otherwise I would
expect it to allow arbitrarily extensive i/o without ever running into
buffer problems.

Happy testing!

Regards,
Bill
--------------24F0CEE6545D3CCF8E46F22D
Content-Type: text/plain; charset=us-ascii; name="buffer_ref69-patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; filename="buffer_ref69-patch"

--- fs/buffer.c.old Mon Dec 1 11:14:12 1997
+++ fs/buffer.c Tue Dec 2 09:59:03 1997
@@ -790,50 +793,85 @@
}
/*
- * If we achieved at least half of our goal, return now.
+ * If there are dirty buffers, do a non-blocking wake-up.
+ * This increases the chances of having buffers available
+ * for the next call ...
*/
- if (obtained >= (needed >> 1))
+ if (nr_buffers_type[BUF_DIRTY])
+ wakeup_bdflush(0);
+
+ /*
+ * Allocate buffers to reach half our goal, if possible.
+ * Since the allocation doesn't block, there's no reason
+ * to search the buffer lists again. Then return if there
+ * are _any_ free buffers.
+ */
+ while (obtained < (needed >> 1) &&
+ nr_free_pages > min_free_pages + 5 &&
+ grow_buffers(GFP_BUFFER, size))
+ obtained += PAGE_SIZE;
+
+ if (free_list[BUFSIZE_INDEX(size)])
return;
-
- /* Too bad, that was not enough. Try a little harder to grow some. */
- if (nr_free_pages > min_free_pages + 5) {
- if (grow_buffers(GFP_BUFFER, size)) {
- obtained += PAGE_SIZE;
- goto repeat;
- }
- }
/*
- * Make one more attempt to allocate some buffers.
+ * If there are dirty buffers, wait while bdflush writes
+ * them out. The buffers become locked, but we can just
+ * wait for one to unlock ...
*/
- if (grow_buffers(GFP_ATOMIC, size))
- obtained += PAGE_SIZE;
+ if (nr_buffers_type[BUF_DIRTY])
+ wakeup_bdflush(1);
/*
- * If we got any buffers, or another task freed some,
- * return now to let this task proceed.
+ * In order to prevent a buffer shortage from exhausting
+ * the system's reserved pages, we force tasks to wait
+ * before using reserved pages for buffers. This is easily
+ * accomplished by waiting on an unused locked buffer.
+ */
+ if ((bh = lru_list[BUF_LOCKED]) != NULL) {
+ for (i = nr_buffers_type[BUF_LOCKED]; i--; bh = bh->b_next_free)
+ {
+ if (bh->b_size != size)
+ continue;
+ if (bh->b_count)
+ continue;
+ if (!buffer_locked(bh))
+ continue;
+ if (buffer_dirty(bh) || buffer_protected(bh))
+ continue;
+ /*
+ * We've found an unused, locked, non-dirty buffer of
+ * the correct size. Claim it so no one else can,
+ * then wait for it to unlock.
+ */
+ bh->b_count++;
+ wait_on_buffer(bh);
+ bh->b_count--;
+ /*
+ * Loop back to harvest this (and maybe other) buffers.
+ */
+ goto repeat;
+ }
+ }
+
+ /*
+ * Convert a reserved page into buffers ... should happen only rarely.
*/
- if (obtained || free_list[BUFSIZE_INDEX(size)]) {
+ if (nr_free_pages > (min_free_pages >> 1) &&
+ grow_buffers(GFP_ATOMIC, size)) {
#ifdef BUFFER_DEBUG
-printk("refill_freelist: obtained %d of %d, free list=%d\n",
-obtained, needed, free_list[BUFSIZE_INDEX(size)] != NULL);
+printk("refill_freelist: used reserve page\n");
#endif
return;
}
/*
- * System is _very_ low on memory ... wake up bdflush and wait.
+ * System is _very_ low on memory ... sleep and try later.
*/
#ifdef BUFFER_DEBUG
-printk("refill_freelist: waking bdflush\n");
+printk("refill_freelist: task %s waiting for buffers\n", current->comm);
#endif
- wakeup_bdflush(1);
- /*
- * While we slept, other tasks may have needed buffers and entered
- * refill_freelist. This could be a big problem ... reset the
- * needed amount to the absolute minimum.
- */
- needed = size;
+ schedule();
goto repeat;
}

--------------24F0CEE6545D3CCF8E46F22D--

- Next message: Bill Hawes: "Re: Linux 2.1.69.."
- Previous message: Chris Adams: "Re: memory leak detector, 2.0.32, patch"
- Next in thread: Dr. Werner Fink: "Re: patch for 2.1.69 buffer.c"
- Reply: Dr. Werner Fink: "Re: patch for 2.1.69 buffer.c"
- Reply: Bill Hawes: "Re: patch for 2.1.69 buffer.c"
- Reply: Bill Hawes: "Re: patch for 2.1.69 buffer.c"
- Reply: Gadi Oxman: "Re: patch for 2.1.69 buffer.c"