Re: Scheduled Transfer Protocol on Linux

[Larry McVoy]

: 
: > : How you can have an operating system that 
: > : runs well on single cpu workstations, scales up to 16-64 cpus
: > 
: > Perhaps SGI thinks it is a good idea to scale Linux to 64 processors.
: > that point of view would indicate that noone at SGI has learned a damn
: > thing from the unadulterated mess you call IRIX.  Well, I worked there, 
: > and I learned that it is a really stupid idea to try and scale a single
: > OS past about 4-8 processors.
: 
: I would differ with that opinion.  I think it is perfectly reasonable to have 
: an OS scale up to 16 processors.  

Why am I not surprised?

OK, so tell me: how many locks does it take to scale up the following to
16 CPUs:
	local disks
	local file system
	remote file system
	processes
	networking interfaces and stack

What do the locks cover?  At 16 CPUs, can you keep all the locks straight
in your head?  Nope.  So what happens when you go into the kernel and add
a feature?  You add a lock.  What does that do?  Increases the number of
locks.  What effect does that have?  Makes it more likely that you'll add
more locks, because now it is even less obvious what the lock protects.

And why do we care?  Because the hardware can't implement the abstraction
you are using.  Go talk to the Origin 2000 designers (if any of them
are still there) and ask them how well the coherency directories worked.
They sucked.  Not because the hardware was bad, but because the OS design
was wrong, wrong, wrong.

Anyone who says that 16 way SMP is OK doesn't know squat about how hardware
coherency works.  If you did, you'd be screaming for people to find a
different way to run on your 8 processor and larger boxes.

But don't trust me, go talk to your hardware designers.  I know there
are some left.  Go ask.  If you argue with me, you'll just get into the
argument and never learn a damn thing.  If you go ask the hardware guys,
you might actually believe them.