My company developed ebXML Accelerator for Biztalk 2004 and I’m working on a project using the Accelerator for building eTrade system of one of the national banks in Korea. I wonder if any of you familiar with this ebXML. Any personal experience or any opinion would be appreciated. 🙂
A few script usages that I’ve used last project.
It covers,
—
BizTalk Assembly Deploy
BizTalk Assembly Start
GAC Un/Install
BizTalk Assembly Unenlist, Stop
BizTalk Assembly Undeploy
A recent question in Stephen
W. Thomas‘ excellent BizTalkGurus
forums asked about how to control the value the HTTP send-side adapter in BizTalk
Server 2006 sends on HTTP requests. After a little investigation through the adapter
code (which lives in the Microsoft.Biztalk.HttpTransport.dll assembly), here’s my
guess of what goes on:
When the adapter is going to send the message, it gets the value for the Content-Type
header from one of theses two locations:
However, that’s not all that is considered. As it turns out, the adapter explicitly
considers text content types (i.e. “text/*” MIME types). If the content type
is a text type, then the adapter will append the encoding of the message to the content
type, like this:
Content-Type: text/xml; charset=utf-8
If the content type is not a text type, then it leaves it alone.
If you don’t want the charset to be appended to the Content-Type header for any reason,
it appears there might be a way around it: The value for the charset is extracted
from the Charset
property of the body part of the message, but the Content-Type header is
only modified if this property contains a non-null, non-empty value.
It would appear possible to work around this then by simply ensuring the Charset property
is empty by the time the adapter gets hold of the message; using a custom pipeline
component in the send pipeline would seem a good way of accomplishing this.
It’s all happening folks in November – 30th there is an SOA/Business Process happening
in Sydney.
There’s some folks from Corp. coming over (BizTalk group) so this is going to be pretty
special.
I’m also co-presenting with David McGhee on the BizTalk RFID space.
Here’s the registration details – love to see you there…..
————————————————————————-
Microsoft SOA and Business Process Conference – Thursday 30th November
2006 – SYDNEY
Register now – places are limited
On 30th November we are holding the Sydney leg of the Global Roadshow – Microsoft
SOA and Business Process Conference. This is a scaled down version of the
recent conference held in Redmond in October. The morning is built for Partners
and the afternoon is built for Customers and Partners.
The afternoon starts with Keynotes by David Chappell and Microsoft’s Steve Sloan discussing
the Industry and Microsoft perspectives on SOA and BPM followed by additional sessions
discussing RFID, Software as a Service and Demystifying Workflow, plus SOA and BPM
sessions for industry segments: FSI, Public Sector and the Communications Sector.
Agenda: http://download.microsoft.com/documents/australia/Biztalk/CustomerAgenda.pdf
To Register: http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032318088&Culture=en-AU
Katie Macintosh | Product Marketing Manager, Application Platform
Servers
This blog entry has been literally a year in the making!
While working at a client, the requirement to seperate claims, decide which system it went to (QNXT or the exising mainframe system).
I decided that using the multiple 837 schema was the best approach. This means that the HIPAA accelerator is going to take the HIPAA file and submit claims (in XML format) individually to the message box. With those messages, I created a singleton orchestration process to pick up each of the messages. It would pick up each of the messages, and individually go through some calls to find out which system it went to.
Once the decision was made, I would concatonate the message to the rest of the messages that have already come in for this HIPAA transaction.
What I saw happening was that during the concatonation process it was taking longer and longer to append the current message to the rest of the messages.
Directions changed, and we moved from having BizTalk being the routing application for various reasons; speed of processingbeing one of the many reasons.
Iworked at another client, and the same issue came up. We started off working with eligibility files (834) and I started with the same approach, and immediately saw the concatonation process increasing in time to complete as the more messages it processed. This time we were able to test with some significantly large files so I could get some real numbers to look at.
It started out taking 1 second to process the first subscriber in the 834, and then by the time it was down to subscriber 1000, it was taking 10 seconds to finish the process. I then needed to come up with a different approach, becausethese were relitivley small files, and we were looking at getting files that had 200,000 subscribers.
I thought, I need a way to store the data in a manner that will not continually increase in time as the dataset grew, what could I use? Well, a database table came to mind, I could place the data in a table, process them and then when it has all been completed, I then could just extract the data out of the database table and send it off.
I implemented sending the data to a database table and not concatonating the messages together. Once I started testing I immediately saw an improvement in performance! In looking at the details, I was seeing that the first subscriber took 1 second to process, so also did subscriber 1000!
I was not satisfied though: if each subscriber was going to take 1 second, then the graph below shows the time to process the file.
Subscribers Minutes Hours 1,000 16.67 0.28 2,000 33.33 0.56 3,000 50.00 0.83 4,000 66.67 1.11 5,000 83.33 1.39 6,000 100.00 1.67 7,000 116.67 1.94 8,000 133.33 2.22 9,000 150.00 2.50 100,000 1,666.67 27.78
The question then was, how do I process them faster? How could I send them to the database faster than what improvements I have already done. I might be able to optimize the extraction and sending to the database a little, but even if I were to cut it in half, I would still be looking at 13 plus hours to complete a single file.
What if I ran multiple occurrences of the extraction process at the same time? I would break my singleton orchestration, but I would essentially open up the flood gates, and it could process as many messagesas possible at the same time. The next question then came to mind, what about the very distinct possibility that there would be table locking issues as I would be doing multiple inserts into the same table at once? I need a highly optimized process to insert data into the database that can handle the possiblity of many inserts happening at once. I am also not a database guru, so I needed something that someone else has developed that I can implement.
BAM – it hit me. BAM (Business Activity Monitoring) is optimized to accept many messags and insert them into a table and it definately has to be designed to capture many messages at the same time. There are two flavors of BAM that can be invoked from 2004, DirectEventStream and BufferedEventStream. I decided that because using DirectEventStream would cause performance issues, going to the BufferedEventStream route would be possibly the best approach. So I have many messages being processed, and then BAM datais sent to the MessageBox to be inserted into the BAMPrimaryImport database when BizTalk got around to it.
I implemented this approach, and increased the processing speed from 1subscriber persecond to 10 per second!
The next issue was,how do I know when it is complete and when can I extract the data fromthe BAM database? I needed a monitoringservice to watch and see when inserts were done for this file and once it has completed, extract the data and create the output.
What if I had each of the processes that sent data to BAM send a message to another orchestration andconsumes those messages, as soon as the messages quit coming, go and check the database to make sure that the rows are there, as soon as all of the rows are there, then extract the data.
This is where I thought would be a very simple process, it ended up being yes (kinda), but I normally have to do things the hard way before finally getting it working successully, and this did not stray too far from my past experiences.
This is the design that I had, many orchestrations would be running, I would have an orchestration that would be picking up all of the messages created by the HIPAA to BAM orchestrations, as soon as I quit receiving the messages, I would make sure that the same number of rows in BAM matched the same number that I picked up. Once everything matched, I would extract the data. I have tocheck the number of rows against what I picked up because with BufferedEventStream, messages are sent to the MessageBox and inserted when resources are availble, not directly likeDirectEventStream.So I could get the last message from the HIPAA to BAM orchestration before the last row is inserted. Below the vision I had:
This is where it got fun!
After using Keith Lam’s blogas a guide, I implemented forward partner direct binding.
I have created a simple prototype on implementing the forward partner direct binding approach. The first orchestration consumes all messages from a particular port. The sample message looks like this:
<ns0:Root SenderId=”123456” xmlns:ns0=”http://PartnerPortExample.Input“>
<Information>Information</Information>
</Data>
</ns0:Root>
It would then create create a message that just had the SenderId to be sent to the Singleton Orchestration that it would correlate on and pick up all messages for that SenderId.
<ns0:Root SenderId=”123456” xmlns:ns0=”http://PartnerPortExample.Status“ />
I promoted the SenderId in the http://PartnerPortExample.Status message. One key thing to take away is that the property item needs to be a MessageDataPropertyBase. If it is a MessageContextBase it will not work. If the promoted field is a context property, the the subscription engine cannot match the message the the HIPAA to BAM orchestration to the Singleton Orchestration, it will state, that no matching subscription could be found.
I then setoutgoing port on the HIPAA to BAM orchestration to Direct, and chose the Singleton orchestration as the partner port.In the Singleton orchestration,I set the partner port to itself.
I also set up the correlation set todrive off of the SenderId.
Below are some screen shots of the prototype:
Here is the Process orchestration that takes the original file, andextracts theSenderId into the Status message, some things to notice is that the binding on the InternalPort is set to Direct and the Partner Orchestration Port is set to the Singleton orchestration.
The code in the Assign Promotion Message Assignment shape is the following:
StatusMessage(PartnerPortExample.id)=ExternalMessage.SenderId;
Here is the Singleton Orchestration that loops thru capturing all of the messages that have the PartnerPortExample.id correlation set.
It then creates an message recording the number of files it processed an sends the following message with the SenderId as the filename:
<ns0:Root Count=”95” xmlns:ns0=”http://PartnerPortExample.Result“ />
Here is the code in the message assignment shape:
TempXML=new System.Xml.XmlDocument();
TempXML.LoadXml(“<ns0:Root Count=\””+System.Convert.ToString(Count)+”\” xmlns:ns0=\”http://PartnerPortExample.Result\” />”);
ResultMessage=TempXML;
ResultMessage(FILE.ReceivedFileName)=StatusMessage(PartnerPortExample.id);
I want to thank Jeff Davis, Keith Lim, Kevin Lam, and Adrian Hamzaon helping me determine that you cannot havecontext properties be the correlation set on partner ports.
Email me if you would like to get the solution.
When developing schemas, it helps to identify all of the necessary distinguished fields before deploying the solution into production. Making changes to a deployed schema can introduce un-needed complexity and pain if used across multiple projects.
Unlike promoted fields* however, it can be difficult to identify all of the necessary distinguished fields before deploying the solution […]
On Nov 30, here in Australia, we are holding a mini version of the SOA and Business Process Conference that was held in Redmond for our local audiences!! This is going to be awesome. The team here have been able to secure David Chappell to present in Australia, so we are looking forward to him reaching our shores and presenting at the conference! So, here is the email from our local BizTalk Product Marketing Manager – Katie McIntosh –
“On 30th November we are holding the Sydney leg of the Global Roadshow – Microsoft SOA and Business Process Conference. This is a scaled down version of the recent conference held in Redmond in October. The morning is built for Partners and the afternoon is built for Customers and Partners.
The morning session is designed for Partners to learn about Service Oriented Architecture (SOA) and Business Process Management (BPM) and how to improve their ability to sell BizTalk based engagements. It is being delivered by International Keynote Speaker David Chappell, Principal, Chappell & Associates, who delivered the same sessions in the US conference. (www.davidchappell.com)
The afternoon sessions are designed for Customers and Partners. The afternoon starts with Keynotes by David Chappell and Microsoft’s Steve Sloan discussing the Industry and Microsoft perspectives on SOA and BPM followed by additional sessions discussing RFID, Software as a Service and Demystifying Workflow, plus SOA and BPM sessions for industry segments: FSI, Public Sector and the Communications Sector”
PARTNERS Register HERE: http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032318079&Culture=en-AU
CUSTOMERS Register HERE: http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032318088&Culture=en-AU
|
Microsoft SOA and Business Process Conference SYDNEY, AUSTRALIA Microsoft Offices, 1 Epping Road, North Ryde Thursday 30th November 2006 |
|||
|
9.30am |
Registration |
||
|
10.00am – 12.00pm |
Selling BizTalk Based Engagements David Chappell, Principal, Chappell & Associates (PARTNER SESSION ONLY) |
||
|
12.00pm – 12.30pm |
Light Lunch and Customer Registration |
||
|
12.30pm – 1.30pm |
SOA, BPM, and Microsoft: An Industry Perspective David Chappell, Principal, Chappell & Associates |
||
|
1.30pm – 2.30pm |
SOA, BPM, and Microsoft: A Microsoft Perspective Steve Sloan, Senior Product Manager, Microsoft Corporation |
||
|
2.30pm – 3.00pm |
Afternoon Tea |
||
|
3.00pm – 3.55pm |
Demystifying Workflow Speakers: Chris Vidotto, Process Platform Specialist & Graham Elliott, Technology Specialist, Microsoft Australia |
The Ecosystem for BizTalk RFID Speakers: David McGhee, Senior Consultant, Microsoft Australia & Mick Badran, Technical Director, Breeze |
Delivering Software as a Service with BizTalk Server Speaker: David Lemphers, Developer Evangelist, Microsoft Australia |
|
3.55pm – 4.00pm |
Break – Change Rooms |
||
|
4.00pm – 5.00pm |
The Application of SOA and BizTalk in Public Sector Solutions Speaker: Christine Axton, Solution Sales Professional, Microsoft Australia |
Applying SOA to Service Delivery in Communications and Media Speaker: Jaron Cohen, Service Enablement Solution Specialist, Microsoft Asia-Pacific
|
BPM, SOA and Multi-Channel Integration in Banking Speaker: Neal Cross, Solution Specialist, Application Platform, Microsoft Australia |
|
5.00pm – 6.00pm |
Cocktails and Networking |
||
Yesterday a friend asked me about the Windows Workflow (WF) Persistence Service in SharePoint 2007. As you might know SharePoint persists the state of workflows when it hits a persistent point. However the SharePoint WF host does not use the default Persistence Service provided by WF. In other words, you won’t see the Persistence Service database as part of the SharePoint 2007 installation. In that case how does SharePoint 2007 persists the state of long running workflows?
The WF persistence service of SharePoint 2007 is implemented as part of the SPWinOePersistenceService class of the Microsoft.SharePoint.Workflows namespace. This class implements the SaveWorkflowInstanceState and the LoadWorkflowInstanceState of the WorkflowPersistenceService class. The implementation of those two operations calls the SaveInstanceData and LoadInstanceData methods of the SPWinOeHostServices class which in turns calls the methods with the same signature in the SPWorkflowManager class. The following diagram illustrates the calls cycle.
Where is the data stored? The data is stored as a compressed binary representation in the Workflows table of the SharePoint content database. The column InstanceData represents the current workflow instance state.
I just got back from an extended week in Barcelona, so I don’t know how long this thing is already available but you can download the Office 2007 Client Applications from the MSDN Subscribers Downloads!
Welcome to Microsoft%u00ae Office Professional 2007, the suite of Microsoft Office system products designed with business professionals in mind. Office Professional 2007 helps business professionals save time and stay organized with powerful and easy-to-use tools for managing customer information and marketing activities, analyzing and reporting business information, and producing professional-quality communications.
Let’s wait for the server components!