Hello.
Well, Microsoft will not officially support files of that size in BizTalk. The “limit” they posted on a blog was 10 MB. There is not any easy ways around the OOM exceptions.
The best way is to break your file up (usually pre-biztalk) and process it in smaller pieces. I have seen clients use a windows service for this type of task.
You might be able to change some setting and get the file to process but is anything else happens to hit the server at the same time you will run into OOM problems. And when that happens, BizTalk doesn’t fail gracefully.
Hope this helps.