XML debatching (long)

Home Page Forums BizTalk 2004 – BizTalk 2010 XML debatching (long)

Viewing 2 reply threads
  • Author
    Posts
    • #20929

      Hello Biztalk Gurus,

      I’ve looked over many of the posts on message debatching.  I’m very happy (as a new biztalk user) to have such excellent resources. So thank you for doing that.
      I apologize for starting yet another thread on message splitting, but if anyone has a moment, and would mind looking at my example, i would really appreciate any guidance you can give.  Right now i’m having a bit of difficulty processing a document given to us from a customer.  Their schema looks like this:
      <root>
      <entry>
      <element></element>
      <element>
      <moreElements></moreElements>
      <moreElements></moreElements>
      <moreElements></moreElements>
      <element>
      </entry>
      <entry>
      <element></element>
      <element>
      <moreElements></moreElements>
      <moreElements></moreElements>
      <moreElements></moreElements>
      </element>
      </entry>
      </root>
      You can see that there are multiple <entry> nodes in each message they send, sometimes hundreds of these complex messages.  Because i need to hit a webservice with one file at a time, I need to be able to take each entry and create a seperate file for each entry, but still contain the root so it will still match the customer’s schema.  But instead of having many <entry> nodes, each file would only contain one.  So…
      Doc1.xml
      <root>
      <entry>
      <element>
      </element>
      <element>
      <moreElements></moreElements>
      <moreElements></moreElements>
      </element>
      </Entry>
      </root>
      Doc2.xml
      <root>
      <Entry>
      <element>
      </element>
      <element>
      <moreElements></moreElements>
      <moreElements></moreElements>
      </element>
      </Entry>
      </root>
      The concept seems like I should be extracting the message <entry> (or debatching as i understand it) and putting it in it’s own envelope to continue through the orchestration.
      A couple things i have tried:
      A.  create my own document schema with the root node being <entry> and an envelope schema with <root><entry>.  Then i set the xpath to <entry>.  Next, i created my own receive pipeline that has the document type as documentSchema, and envelope type as envelopeSchema.  As soon as i put a message into the folder for processing, i get an error: “Document type ‘entry’ does not match any of the given schemas.”
      B. In the custom receive pipeline:  use the customer given schema as the document schema and an envelope schema with xpath to the repeating node <entry>.  Message says it is not recognized.
      I feel like i’m getting closer with option A, but it seems like i’m missing something (probably something very simple).  Also, i have seen people using the orchestration to split the document.  Although i would rather handle this in the pipeline, if you believe it is better practice to handle within the orchestration, i would be open to that option.
      Thank you again.

    • #20935

      When you create a schema in Visual studio, it will automatically assign a namespace but if your incoming xml doesn’t have a namespace associated with <entry> node, you have to update the namespace of <entry> node to empty.

      • #20936

        Hello Sidhu,

        Thanks for the quick reply.

        I tried changing all the schemas (4 in all)* to have nothing in target namespace.  When i try to change the namespace on the on the envelopeSchema, it has an error that says:  “Namespace attribute of an import must not match the real value of the enclosing targetNamespace of the <schema>.”

        The four schemas, in case that would help:

        customerSchema

        ourSchema (we must eventually map to this)

        documentSchema (this schema i created from the customer schema without the root node)

        envelopeSchema (has essentially the header of the customerSchema)

        Thank you,

    • #20938

      Hello Anon,

      I think you’re almost there with option A – which I would recommend as the best approach: its much better to do your debatching in the pipeline than start an orchestration to do the debatching with the additional overhead that incurs!

      You need to create two schemas, an envelope and a document/body schema; the envelope schema will define the message you will receive from your customer and the document schema the single <entry> schema.

      1. Create a new (ordinary) schema as your envelope schema as follows:

      Envelope Schema

      2. Click on the Schema node (at the top of your schema tree) and set the schema property Envelope to Yes

      3. Click on your Root node and click the elipsis button on the Body XPath property (note, this will be greyed out until you sent the Envelope property to Yes), navigate to your root node, not the entry node, and click Ok. This will update the Body XPath property.

      4. Create a separate schema that represents your single entry message so that once the message is debatched, the XmlDisassembler will recognise the individual messages.

      Deploy and test – there is no need to configure an XmlDisassembler,just use the standard XmlReceive pipeline and BizTalk will work its magic and figure out the necessary schemas. Be aware that the validate schema test in Visual Studio won’t work with the envelope schema,but it will work with the document message.

      It can be a bit fiddly getting these things to work – I’ve just spent today doing just that and it took me a few attempts – but when it works, it is beautiful. If you get any errors, post them here and we will help get this sorted. it would also be helpful if you could post an example of the actual message you are receiving.

      Cheers, Nick.

      • #20940

        Nick,

        Thank you very much for posting such a detailed suggestion.  I think we’re almost there.

        So what i did is take the customerSchema file and mark it as an envelope and mark the root node as the body xpath.  Then i created a new schema that holds the body.  Build works great, but i see the message is suspended with the following error:

        The Messaging engine failed to process a message submitted by
        adapter:FILE Source URL:C:\Mapping\In\*.xml. Details:The published
        message could not be routed because no subscribers were found. This error occurs
        if the subscribing orchestration or send port has not been enlisted, or if some
        of the message properties necessary for subscription evaluation have not been
        promoted. Please use the Biztalk Administration console to troubleshoot this
        failure. 

        So I right clicked on the suspended message and clicked on message flow.  I found 4 parts to the message:

        Unparsed Interchange

        entry

        entry

        entry

        It so happens that my test document had just 3 peices so it looks like it is splitting the message! 

        I looked over your post a few more times to see if i was missing anything.  I see that you have suggested to create 2 entirely new schemas.  At that point i would have 4 total schemas (customer, customerEnvelope, customerDocument, and ourSchema) and the envelope schema and the customer schema would have the same root name (an error i ran into when trying that way).  I can get away from the error by changing the namespace on the envelope and document schema, but then the message just totally skips the debatching.

        Thanks again.  I can’t help but think we’re very close to having a working solution.

        Bart

        • #20941

          GOT IT!!!

          Thank you very much Nick, I really appreciate your help.  It feels damn good when it works!

          My problem after getting the messages to debatch was my receive port.  It was expecting the customerSchema type, not the debatched message type (customerDocument).  So I changed my receive port to expect the customerDocument type and then I used a construct message shape to map the customerDocument message type to the customerSchema (basically the same except a different root node).

          Well, again, i really appreciate the guidance.  I think it’s time for a beer now!

          • #20942

            NO!!! I spoke too soon!

            I have the 3 files now, but they are void of any data.  Looks like i’m switching from beer to whiskey.

            Edit: I’m hypothesizing that the reason there is no data is because i’m using the customerSchema as the envelope.  When it hits that Body XPath, it’s stripping all the data off.  So now I think I understand your original post about making your own envelope.  I’ll do some experimenting here.

            Edit:  I think that hypothesis is wrong.  Because the envelope schema and the customerSchema have the same root node name, i was forced to change the target namespace.  I’m now thinking that the loss of data comes from the messages having a different target namespace.

            Edit:  FINALLY KILLED IT! problem was in the orchestration.  Now only using 3 schemas:  customerSchema(which is now the envelope), documentSchema (customerSchema without the root node), and ourSchema.  All in the same namespace.  Final product is exactly what i needed.  <entry> node separated into it’s own file and able to be processed by the map.  I had to change the map in the orchestration to use the incoming schema: documentSchema rather than what it used to be: customerSchema.  Not sure if any of that makes sense.  It’s getting late here and i’m getting a little delirious.  But thought i would write it out in case anyone could find it useful.  Thanks again for the guidance!

            • #20945

              You’re exactly right in your last edit Bart: the envelope schema (your customerSchema) will debatch at the entry level, creating multiple document schemas (your customerSchema without the root node); you then need to map your document schema to a final schema which includes the root node, but one which only allows one entry element (your outSchema).

              If you are using an orchestration, you will need to accept the debatched document schema (your customerSchema without the root node) in the Receive Shape and map from this schema to your final schema (your outSchema). The when debatching, XmlDisassembler component in your pipeline will locate your envelope schema with details on how to debatch the message, but the actual output from the component will be several individual (debatched) messages.

              Hope this all makes sense, and congrats for getting the final solution – I think I’ll have a whiskey tonight to celebrate too!

              Cheers, Nick.

              ps. If you could make this question as answered (if it is), that would be appreciated.

Viewing 2 reply threads
  • The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.