Inserting entire XML message into database

Home Page Forums BizTalk 2004 – BizTalk 2010 Inserting entire XML message into database

Viewing 1 reply thread
  • Author
    Posts
    • #21341

      Hi all,

       

      I’m working on a project that requires the entire incoming XML message be saved to a SQL 05 database.  I brought in the message using a receive shape set for a XMLDocument type message, tried converting it to a string, adding it to the generated item set up for SQL,  then sending it to a stored procedure for insert.  But I receive a number of errors–attribute value is more than 256 characters, root element missing (which appears to happen when the mapping fails for some reason), stored procedure expected parameter ‘foo’ but it wasn’t supplied, etc.  I have working orchestrations that interact with the database currently in production and a large number of them utilize stored procedures in this fashion, but I’ve never had to insert the entire XML message into a table.  It seems like it should be rather simple so I’m convinced I’m just missing something.  Does anyone know of any good examples or resources for this scenario?  Any thoughts would be greatly appreciated!  Please let me know if more details would be helpful as well.

       

      Thanks!

    • #21346

      I once had a similar problem, what we did was create a column of the type ‘xml’ and that way we were able to insert the whole xml message unattached.  I think you need MS-SQL 2005 to use that datatype.

      Would this work for you ?

      • #21352

        Actually I tried that first but was unsuccessful in getting the XML to insert into the DB.  Can you tell me more about how you handled the message and the send port?  I implemented a workaround by changing the column type to ‘text’ and using an updategram, but I’d rather do something with more flexibility.  Any info you can provide about your solution would be a great help!  Thanks!

        • #21353

          Sure no problem.

          I created a Stored Procedure to insert the xml message into a table like this:

          CREATE PROCEDURE [dbo].[CreateLogEntry]
              @XmlMessage xml = null    

          I have more parameters there but you get the picture.  Then I generated a schema from this procedure(the xml column comes out a string element).  Promote that element !!  Important step.

          Then In my orchestration I have a message of type System.Xml.XmlDocument.  And the first thing I do after I receive my incoming message I do this in an expression shape:

          MsgXmlLogMessage = MsgSalesOrderRcvd;  // Initialize the XmlDocument message

          Then you need to Construct a message of the type CreateLogEntry (the schema we generated from the stored procedure).  Make sure in this map that you put some value into the Xml element because it is promoted and we set it in an Assignment shape but not in the map.  It will fail if you don’t set some dummy value in the map to assignit in an Assignment shape.

          Then after the Transform shape put an Assignment shape and type this:

          MsgRcvLogRequest.parameters.CreateLogEntry.XmlMessage = MsgXmlLogMessage.OuterXml;

          Now you can just send this MsgRcvLogRequest to the SQL Send Adapter to execute the procedure.

           

          Sorry for my english hope you understand if not I’ll try screenshots next 🙂

          • #21355

            BaldurKn,
            Thanks for the quick response. A couple of questions for you:
            When you say you ‘generated a schema from this procedure’, I’m assuming you mean you generated it in BizTalk through the “Add Generated Items” option. when I generate the SQLService.xsd, it creates the parameter as an attribute, not an element. Is this correct for your solution as well?
            It appears from your description that the incoming message is MsgSalesOrderRcvd and the blank XMLDocument message is MsgXmlLogMessage. But I’m unclear as to what you’re mapping in the transform shape you refered to. I would think you wanted to map the incoming message to the msg of type CreateLogEntry (created from the stored procedure), but if I’m receiving an untyped message (of type System.Xml.Document already because multiple other orchestrations need to send various messages of different formats to this one for logging in the DB) then I don’t have the option of selecting it as a source for the map. Also, if the incoming message in your example is typed (MsgSalesOrderRcvd), then the expression editor throws an error when it’s assigned to a different type message (MsgXmlLogMessage). I think I may be misunderstanding you on this part because it’s not working when I try it. I tried to create a new message manually in a message assignment so I could map something in the transform shape, but it comes out with an xpath failure when I submit a message.
            Any further details you can give?
            Thanks again!

            • #21361

              Hi, sure I’ll try to explain better.  I’ll just use our example and then you could perhaps convert that to your situation.  First to answer your questions:“I’m assuming you mean you generated it in BizTalk through the “Add Generated Items” option” – Yes.  And yes SQLService.XSD can use attributes just as well 🙂  so now on I’ll talk about attributes not elements to keep things simple.

              OK, my situation: I have a Receive shape at the top that activates this orchestration(In this case it is the ProcessSalesOrdersOrchestration).  The very first thing that I do after the Receive shape is an Expression shape with this code:

              MsgXmlLogMessage = MsgSalesOrderRcvd;  // Initialize the XmlDocument message

              Now I have two messages in the Orchestration: MsgXmlLogMessage(type System.Xml.XmlDocument) and typed MsgSalesOrderRcvd.

              Now I need to create a typed message from schema CreateLogEntry(your case SQLService) called MsgRcvLogRequest.  To do that I have a Construct shape and inside it I have a Transform and an Assignment shape.  In the Transform shape I have a map from schema SalesOrder to schema CreateLogEntry where I copy the OrderId, Date, etc to some of my columns in the Log table(in this map I for example map the OrderId to the attribute XmlMessage so that I initialize that attribute before the assignment shape).  Then in the Assignment shape I copy the message MsgXmlLogMessage into this attribute XmlMessage like this:

              MsgRcvLogRequest.parameters.CreateLogEntry.XmlMessage = MsgXmlLogMessage.OuterXml;  // I promoted the property XmlMessage

              Now I have three messages the original SalesOrder, the typed CreateLogEntry with some attributes like OrderId, Date, Origin, etc and also a attribute called XmlMessage which contains a copy of the SalesOrder xml message and the untyped XmlDocument MsgXmlLogMessage.  To do this we needed the untyped XmlDocument message just to copy the data for us through this fine OuterXml property 🙂

              OK, now I have explained better I hope.

              I just read your post over again and saw a few things:

              “Also, if the incoming message in your example is typed
              (MsgSalesOrderRcvd), then the expression editor throws an error when
              it’s assigned to a different type message (MsgXmlLogMessage)” 
              – I assign the other way around, I assign the typed MsgSalesOrderRcvd into the untyped MsgXmlLogMessage: MsgXmlLogMessage = MsgSalesOrderRcvd;

              “but if I’m receiving an untyped message (of type System.Xml.Document
              already because multiple other orchestrations need to send various
              messages of different formats to this one for logging in the DB) then I
              don’t have the option of selecting it as a source for the map” –
              I see this as an problem.  In order to assign the promoted property through Assignment shape we need to initialize the original message with a transform shape.  Perhaps you could create a very general schema with nothing but the type <any> element and then create a map from that schema to your SQLService schema and just put some hard coded value into your XmlMessage attribute (and other attributes you need) and then right after assign the correct Xml document to the XmlMessage attribute… ??  Maybe some xpath could help you here I no very little about xpath so sorry about that.

              I’ll monitor this thead until you get this to work because I know we are close 🙂

              • #21362

                Make sure you click on the “Click to show quoted text” in my earlier thead, because there is more there, I don’t know why it did that.

                • #21369

                  Thank you for the details!  Yes, I’m experimenting with something like that now although it’s not very elegant.  I’m now receiving XLANGS errors that the message doesn’t match the xpath it’s looking for, but I think I can work that one out.  Ultimately, I’d like this orchestration to be something easily implemented in the other orchestrations directly.  However, I’m running into a lot of little incompatibilities such as the message type issue we just covered.  I appreciate your patience!  Thank you for the help!

                  • #21373

                    One last thing, while playing around with this I tried your solution directly and still get an implicit conversion error with the

                    MsgXmlLogMessage = MsgSalesOrderRcvd;  // Initialize the XmlDocument message

                    statement in an expression shape immediately after the receive.  I tried it both a message and with a variable ot type System.Xml.Document.  Just out of curiosity, ow did you get that to work?

                    • #21383

                      That is very strange.  I just receive a message of type SalesOrder then I have a message of type System.Xml.XmlDocument in the orchestration and in the Expression shape I do this:

                      MsgXmlLogMessage(XmlDocument) = MsgSalesOrderRcvd(SalesOrder schema);  // Initialize the XmlDocument message

                      I don’t understand why this works for me but no you.

                      • #21394

                        I ended up making a multipart message of type System.Xml.XmlDocument and assigning the message parts instead of the messages as you did in your example, but I get a routing problem that I think has to do with promoted fields.  What pipelines are you using for your send and receive ports?

                      • #21395

                        Hi,

                        I am just using the default XML Transmit and Receive pipelines.  Are you doing something like this :

                        MsgRcvLogRequest.parameters.CreateLogEntry.XmlMessage = MsgXmlLogMessage.OuterXml;  // The promoted property XmlMessage

                      • #21396

                        Here is my Orchestration:

                      • #21409

                        Did you get this to work ?

Viewing 1 reply thread
  • The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.