Re: Scheduled Transfer Protocol on Linux

[Zachary Amsden]

> OK, so tell me: how many locks does it take to scale up the following to
> 16 CPUs:
> 	local disks
> 	local file system
> 	remote file system
> 	processes
> 	networking interfaces and stack
>
> What do the locks cover?  At 16 CPUs, can you keep all the locks straight
> in your head?  Nope.  So what happens when you go into the kernel and add
> a feature?  You add a lock.  What does that do?  Increases the number of
> locks.  What effect does that have?  Makes it more likely that you'll add
> more locks, because now it is even less obvious what the lock protects.

Good design can avoid these problems.  If it isn't obvious what a lock 
protects, you should rethink your locking structure.

> And why do we care?  Because the hardware can't implement the abstraction
> you are using.  Go talk to the Origin 2000 designers (if any of them
> are still there) and ask them how well the coherency directories worked.
> They sucked.  Not because the hardware was bad, but because the OS design
> was wrong, wrong, wrong.

> Anyone who says that 16 way SMP is OK doesn't know squat about how hardware
> coherency works.  If you did, you'd be screaming for people to find a
> different way to run on your 8 processor and larger boxes.

I wasn't arguing that 16 way SMP is OK.  Everyone knows it isn't.

Are you saying that clusters of small SMP machines are better?  So the locking 
moves from the kernels to the application layer.  You still have the same 
synchronization concerns, it's just a matter of what layer they are 
implemented at.

-- 
Zachary Amsden  [e-mail]  [phone]  09U-510  Core Protocols