Detecting duplicate nodes in BizTalk

Home Page Forums BizTalk 2004 – BizTalk 2010 Detecting duplicate nodes in BizTalk

Viewing 1 reply thread
  • Author
    Posts
    • #19714

      I have an xml incoming to my BizTalk like in the below ex:
      <Items>
           <Item>It_1</Item>
           <Item>It_2</Item>
           <Item>It_3</Item>
      <Items>
      Can any one please help me, In this xml how can I detect to know if two “Item” Nodes have the same value?

    • #19723

      There are a number of ways, including a map with custom XSLT or a .NET class.

      The best way will be determined by what you wish to do when you detect duplicates.

      Do you wish to raise some error or do you wish to eliminate duplicates and continue processing?

       

       

      • #19734

         Greg,

        Thanks for your reply. I want to through an error if there are duplicates in it.

        Thanks

        Damodar

        • #19738

           Here is a sample XSL that can be used in a map:

          It will detect duplicate <item> records. If there are duplicate items it will output an <Errors> document listing all the nodes in error, otherwise it will output the original input file.

          This map can be used in the receive pipeline or an orchestration. If you are using in a pipeline, then you will need some process to subscribe to the Errors document and take appropriate action. If used in an orchestration then you can use a decide shape to test the output and take action on error. 

          <

          xsl:stylesheet xmlns:xsl=http://www.w3.org/1999/XSL/Transform xmlns:msxsl=urn:schemas-microsoft-com:xslt xmlns:var=http://schemas.microsoft.com/BizTalk/2003/var exclude-result-prefixes=msxsl var version=1.0>
          <
          xsl:output omit-xml-declaration=yes method=xml version=1.0 />
          <
          xsl:key name=duplicateKey match=Item use=./>
          <
          xsl:template match=/>
          <
          xsl:apply-templates select=/Items />
          </
          xsl:template>
          <
          xsl:template match=/Items>
          <
          xsl:variable name=Errors>
          <
          xsl:for-each select=Item>
          <
          xsl:variable name=group select=key(‘duplicateKey’,.)/>
          <
          xsl:if test=generate-id($group[1]) = generate-id()>
          <
          xsl:if test=count($group) &gt; 1>
          <
          xsl:element name=error>
          <
          xsl:value-of select=$group[1]/>
          </
          xsl:element>
          </
          xsl:if>
          </
          xsl:if>
          </
          xsl:for-each>
          </
          xsl:variable>
          <
          xsl:choose>
          <
          xsl:when test=count(msxsl:node-set($Errors)/error) &gt; 0>
          <
          xsl:element name=Errors>
          <
          xsl:copy-of select=$Errors/>
          </
          xsl:element>
          </
          xsl:when>
          <
          xsl:otherwise>
          <
          xsl:copy-of select=./>
          </
          xsl:otherwise>
          </
          xsl:choose>
          </
          xsl:template>
          </
          xsl:stylesheet>

          • #19793

             Greg,

            Thanks for your reply.

            My complete scenario is like this. I receive an xml document from a sharepoint doc. library Recieve_xml_docs. I have to process that in biztalk and a truncated version of it goes to another sharepoint library Correct_xml_docs. While processing it I have to check if there are duplicate items as mentioned above, and if there are any duplicate items, I have to move such xml document to a Incorrect_Xml_docs.

            Can you please tell me what would be the best way to implement this?

             

            • #19799

               Depends on what the processing requirements are:

              The simplest approach is to create a validation map based on the Xslt, producing either a Correct_xml_doc or an Incorrect_xml_doc. Put this map into the inbound maps on the receive port from Sharepoint. Have 2 send ports, one subscribing to Correct_xml_doc and sending to correct Sharepoint library and the other subscribing to Incorrect_xml_docs and sending to a different  Sharepoint Library.

              If you require more functionality you can add an one or two orchestrations to process these messages.

              • #19864

                 Thanks greg

                I am using the above xsl in an orchestration. I selected my Items schema as input schema for map. I created a schema with only one element of type xs:anyType and used it as output schema for the map and in the map properties, I selected this file as the custom xsl path. However, it is generating a empty file.

                Am I doing it the right way?

                • #19865

                   The map above will only work with the example Xml provided above. If your actual Xml is different (e.g. uses namespaces, different structure, node names then it will not work.

                  Are you able to provide an example of an actual input message that you wixh to map?

                  • #21739

                    Hi, I’m new to BizTalk. I have a similar problem below.

                    I need to convert a flat file (csv file) with multiple records in it into a single XML file. The problem is that duplicate item codes may exist in the flat file but I only need one set of them in the XML file. How can I eliminate the former ones and only convert those latest ones into the XML file?

                    Could someone please help me?  Thanks!

                    • #21745

                      The flat file disassembler will not filter records. You will need to construct a flat file schema and parse the entire  flat file and then use a map (either in the receive pipeline or in an orchestration)  to filter out unwanted records.

                      We will need a description of the incoming file (e.g. example of the parsed Xml file) and specifics about which codes are duplicates and need to be filtered out.

                      • #21749

                        Thanks.  But what should I do with the map?

                        I already created a map to link up between the flat file schema and the XML schema.  Should I use functoid to filter out unwanted records?  Could you please give an example?

                      • #21750

                        You can sometimes achieve filtering using logical comparison functoids but other times custom Xslt is required.

                        What is the record structure. What are you filtering on: fixed codes that are known at design time or Id values that are dynamic. What are the rules around filtering, map the first record and delete the rest, delete all except the last. Select one of the records based on some contained value? Find the record with the biggest value?

                      • #21751

                        The case is like this: A flat file of item master will be extracted daily with item code, description, and price respectively.  The flat file may consists of duplicate item records which the first one is the creation of item master and the second is the updated one.  I need to convert these records into XML file and only the last updated one is needed.  So I need to filter the duplicate ones and get the latest ones into my XML file.

                        I have created a flat file schema and an XML file schema respectively on these fields.  I have also created a map to link them up.  An orchestration is created to receive the flat file and transform through the map and send to XML file.

                        Any samples and hints?

          • #23123

            Hi

            Can you use the same logic without replacing the Items and Item with Errors and Error?  I would like to keep the same structure and add an attribute on the duplicates like delete=”true”.

             

Viewing 1 reply thread
  • The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.