Re: very slow RAID-1 resync

[Neil Brown]

On Monday October 15, [e-mail] wrote:
> On Tue, 16 Oct 2001, Neil Brown wrote:
> 
> > On Monday October 15, [e-mail] wrote:
> > > I just plugged in a new RAID-1(+0, 2 2-disk stripe sets mirrored) to a
> > > 2.4.12-ac3 machine.  The md code decided it was going to resync the mirror
> > > at between 100KB/sec and 100000KB/sec.  The actual rate was 100KB/sec,
> > > while the device was otherwise idle.  By increasing
> > > /proc/.../speed_limit_min, I was able to crank the resync rate up to
> > > 20MB/sec, which is slightly more reasonable but still short of the
> > > ~60MB/sec this RAID is capable of.
> > >
> > > So, two things: there is something wrong with the resync code that makes
> > > it run at the minimum rate even when the device is idle, and why is the
> > > resync proceeding so slowly?
> >
> > The way that it works out where there is other activity on the drives
> 
> There wasn't any activity at all.

See how fragile it is?  
It only really works if the underlying devices are real drives with
major number less than 16, and that are among the first 16 real
devices with that major number (not counting different partitions on
the device).

> 
> > is a bit fragile.  It works particularly badly when the underlying
> > devices are md devices.
> 
> Bummer.
> 
> > I would recommend that instead of mirroring 2 stipe sets, you stripe
> > two mirrored pairs.  The resync should be faster and the resilience to
> > failure is much better.
> 
> I did eventually do it that way, but the sync speed was the same.  I'm
> very curious to know why you think striping mirrors is more reliable than
> mirroring stripes.  Either way, you can lose any one drive and some
> combinations of two drives.  Either way you can hot-swap the bad
> disk.

With striped mirrors there are more combinations of two drives that
can fail without data loss, and less data needs to be copied during a
resync.  Also, hotadd is easier  - you don't have to build a raid0 and
then hot-add that, you just hot-add a drive.

It is odd that you still aren't getting good rebuild speed.  What
drives to you have and how are they connected to what controllers?

NeilBrown

> 
> -jwb