execute application windows and call biztalk rules engine

You’re right .. you can only use the Microsoft.RulesEngine.dll on a machine with the RulesEngineUpdateService installed. Technically there can be ways around that, but from a licensing standpoint, it’s not practical to pay for a BizTalk license for each user of your Windows application! The most common way to expose the Rules Engine to smart clients is via web service wrapper. Wrap up your rule(s) as a service(s) and simply call those services, leaving the rules processing for the box with the rules engine installed.