I can’t really seek to your SSO question, but as far as the number of message boxes it depends on your message volume. When we were running tests, we got 1 SQL 8 way to process about 300 to 400 messages per second (small messages in pure messaging). It then took 3 SQL 8 ways to get up to 900 per second and adding more SQL after that didn’t really help.
So, all in all it really depends on your load, server specs, and what you are doing. But, in general I’d avoid multiple message boxes unless you really needed them. Also note that multiple message boxes is not a highly available solution. Each message box would need to be on its’ own SQL cluster.
As far a multiple instances of SQL, I have heard of people getting some performance gain from that. But, again, I’d not go down that route unless you really had to. And if you really needed extra power, you should be using multiple SQL clusters and not just instances.
Hope this helps.