thank you very much for your reply. The actual orchestration is miminal. It calls a sproc to update a table based upon the file's name. It calls a simple custom assembly to kick off another small process. We've verifed that once the .tif files are in the orchestration, everything runs beautifully. The bottleneck is definately loading multiple larger files (200+mb) .pgp/.tar files in at the same time and processing them through custom pipeline components. I can't give you specifics, but my gut instinct tells me there's a memory leak in our untar component.
If we decrypt and untar manually and just throw a ton of .tifs at it, there is no problem. I think the problem is coming in by reading several 200mb+ files into the message box at the same time? We've considered breaking this process out, however, that is what we're currently doing w/ a windows service, so why migrate the process to biztalk at all?
My main question is: I'm wondering if anyone else is using biztalk for the purpose of really just waiting for large files to be FTPed to us then moving them from one place to another. We've found great uses for biztalks on more system integration tasks and it almost seems to me that in this one case, we're trying to make BizTalk a file mover, when it really wasn't designed for that.
I will tell you that today we've found a way to "work around" the issue. I will post that later, but it is really just a work around and I'd like to get some other people's thoughts first, w/o "tainting the jury 😉