Ox,
How long is a piece of string?
There are many things that could be causing your SQL Server to have issues. I would probably start by determining whether it is actually the SQL Server process that is max-ing the CPU out. If it is, can you confirm whether you have disk IO issues or processing/RAM issues. I listed two good articles from the SQL Server PSS team over on my blog at Diagnosing Performance Problems in SQL Server that should help give you a start in diagnosing your performance issues.
If you believe that it is the BizTalk databases that are causing the SQL Server bottleneck, you’re into classic scale-up or scale-out territory. You can either add additional resources to your exisitng SQL box (scale-up) – additional CPU/cores would probably be a good starting point given that you have reached the upper RAM limit on your server depending on the OS and assuming its a 32-bit server; or add additional SQL Server boxes and create multiple MessageBox databases (scale-up).
Some good links on scale-up and scale-out are: http://msdn2.microsoft.com/en-us/library/aa559821.aspx and http://msdn2.microsoft.com/en-us/library/aa578625.aspx
If you’re running 32-bit hardware, now might also be a good opportunity to move to 64-bit servers which have much greater expandabilty. You may also be interested in my blog post Optimising System Memory for SQL Server which details optimisation techniques for SQL Server.
Hope this helps, any further questions, please ask.
Regards, Nick.