I’ve been out of commission for personal reasons for the last week so I apologize for not aknowledging the excellent advice everyone has had to offer.
It’s good to see that you are all on the same page. In fact everyone was right on when you figured out that I’m not using Biztalk as it is meant to be used. The original application was developed before my time using Biztalk 2002. Biztalk 2002 had problems with handling large files. To accomodate this a method was created whereby a “control” file is created by a web service for each incoming file type. The control file specifies which orchestration to use. Each orchestration/control file combination differ in the steps used to process the incomfing file which is simply streamed from the file system. The usual steps are i) validate the incoming file, ii) update a database with values from the incoming file, iii) create output files and reports, iv) send the resulting files to the appropriate destination.
In the above scenario Biztalk is used for only for 2 reasons, first as an existing service to listen for incoming files and second as a method of being able to handle new file types without rebuilding a stand alone application.
I am aware that there are alternatives such as writing our own file listening service and perhaps using web services to provide extensibility to new file types. Nevertheless, the existing solution is up and running so we’ll likely stick with it for now.
Thanks to all for your advice.
Dave