Forum Replies Created
-
Posts
-
Thanx NND for reply,
I am already writting the data to the queue as a Stream, the same stream that I use to write to the file, by setting the BodyStream of a MSMQ message. But I dont know How I can force a pipeline component to read the data from the queue (or receive location) by using my method?, rather then its own default.
Regards,
Sajid.
The usual way of doing this is a custom pipeline component, before the Flat File disassembler in the receive pipeline:
http://codebetter.com/blogs/jeff.lynch/archive/2006/04/08/142537.aspx
Hello Sajid,
It seems like a formatting problem to me when the data is pulled from MSMQ by your decryptor. See Microsoft KB Article ID : 310683
I hope that will help.
Regards,
NNdM
You are right, SQL Server does play well with others. It is reluctant to release memory once allocated.
There are options in SQL Server where you can limit the maximum memory it will use. By default this is all the memory in the system.
You could try to limit SQL to 1GB of memory. This may slow SQL somewhat (smaller cache = more disk hits) but may speed up the machine as a whole by allowing other processes more memory and reducing page faults.
Or you can use the scripting functoid with Inline XSLT
<outputNode>
<xsl:for-each select="repeatingNode">
<xsl:value-of select="."/>
</xsl:for-each>
<outputNode>Here is the Schema:-<?xml version="1.0" encoding="utf-16" ?>– <xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://DummyFlatFile.FlatFileSchema1" targetNamespace="http://DummyFlatFile.FlatFileSchema1" xmlns:xs="http://www.w3.org/2001/XMLSchema"><b:schemaInfo standard="Flat File" root_reference="Root1" default_pad_char="" pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" /><schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" /></xs:appinfo></xs:annotation><b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="1" child_delimiter_type="char" child_order="infix" child_delimiter="," /></xs:appinfo></xs:annotation><b:recordInfo sequence_number="1" structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" child_delimiter_type="char" child_delimiter="," child_order="infix" /></xs:appinfo></xs:annotation></xs:sequence></xs:complexType></xs:element></xs:sequence></xs:complexType></xs:element></xs:schema>For this input:
1C,334058,1C,W22192,1D,769475
How did you create this xml file.Can you post the xsd(schema) here
<Repeated>
<PIDNumbers LID="1C" PValue="334058" />
</Repeated><Repeated>
<PIDNumbers LID="1C" PValue="W22192" />
</Repeated><Repeated>
<PIDNumbers LID="1D" PValue="769475" />
</Repeated>Depends on the web service.
BTS 2004 comes with a standard SOAP adapter which works with most WS-I Basic Profile web services. There are some limitations particularly with arrays.
If you need and WS-* functionality then you might have to upgrade to BTS2006, There is the WSE3 Adapter from AdapterWorx http://adapterworx.com
The next version BTS2006 R2 (currently in beta) has a built in WCF adapter
I resolved this finally. If anyone faces similar issue pls let me know; will be more than happy to share the solution.
Thank you for the reply, yes it turns out this is the very problem. With further research it turns out in 2004 there was a large number of people who had this problem. While the Orchestration does not directly read from the MQSeries queue, I assumed the performance hit would not be significant so used a 12 second delay functoid to determine when there were no more messages.
Apparently a message is read from the queue by the MQSeries adapter (possible transformation), sent through the Receive Location (perhaps more transformations), written to the BizTalkMsgBoxDb, read by the Pub/Sub service and routed to my orchestration. Seems 12 seconds was way too low resulting in many zombie processes and discarding 33,303 messages out of 65,861. WOW!
I changed the delay to 20 minutes and the very first test had a single Orchestration go zombie on me due to a 23:16 minute gap in processing one message and the next. At least that time I only discarded 67 out of 65,861.
Final result is I changed the delay to 60 minutes. This will process all message, batching them into groups of 500 but the final batch of messages will be delayed that 60 minutes while the Orchestration takes that time to figure out there are no messsages. For our process this will not be a real issue.
We did look at some sample of going to a database instead but we have a very limited time to fix this and could not consider an alternative unless we could not get this working and 60 minute delay does seem to resolve the issue. For the next release I may be able to revisit this design.
Thank you for the help.
