Modern enterprise IT is hybrid and relies on data from on-premises and the cloud. At the core of Microsoft’s Hybrid Integration Platform, we have BizTalk Server which handles all your on-premises integration scenarios and Logic Apps, which takes care of all the cloud integrations leveraging Azure Services.
BizTalk 360 already provides a comprehensive monitoring solution for your Hybrid Integrations from version 8.3 which introduced Logic Apps State monitoring.
We are excited to announce the addition of Logic Apps Metrics monitoring capabilities to our forthcoming 8.4 release (expected: Apr 2017).
With this release, you will be able to monitor your logic apps using all the available metrics that are provided in the Azure Portal.
Setting Data Monitoring for Logic Apps metrics
The new Logic Apps monitoring capability is just an extension to our already existing data monitoring functionality. You can find the newly added tab for the Logic Apps under the Data Monitoring module.
Click the Add New button to add a new monitor for the Logic Apps. A new Logic Apps Monitor setup window will appear. We can see that all the Logic Apps related settings are grouped under “SET DATA FILTER” section in the below image.
Select the subscription from the list of subscriptions that you have configured in BizTalk 360. Please check this Assist article if you want to know how to add Azure subscriptions in BizTalk360.
Once the subscription is selected BizTalk360 will display all the Logic Apps that are available in that selected subscription. Finally select the metric that you want to monitor.
Once you set the data filter you can specify the notification condition which BizTalk360 will use to send alerts for each violation as shown below.
Note: Azure metrics can have different units based on the metric type so metrics like Action Latency will have milliseconds as the unit type. In those cases, the threshold condition must be set based on the selected metric.
Save the Logic Apps monitoring schedule. BizTalk360 Monitor service will pick up the new monitor configuration and customers will start receiving the email notifications or any other notification channels based on the Alarm configuration.
You can also view the monitoring results in the Data Monitoring Dashboard as shown in the below image.
Click an alert item to view the details.
We hope this new capability will enhance the monitoring use case for all our customers having Hybrid Integration scenarios. If you have any feedback, please write to email@example.com.
Windows Azure Logic Apps now have support for variables inside a logic app!
In order to use them, just search for Variables inside the Add Action dialog box.
You have two options: one to initialize a variable and one to increment a variable.
Currently, Variables support Integer and Float variable types as shown in the image below. But with all things Logic Apps, this could change later on.
You can also use Math Functions when assigning and incrementing variables.
An example of this is using the Add internal function to add 5 to another existing variable. This would look like this:
You access a variable inside JSON like
One interesting point to note if that you can not use the output of a variable as the increment for the same variable.
You will get this error when you try to save the Logic App:
Failed to save logic app TestVariables. The inputs of workflow run action ‘Increment_variable’ of type ‘IncrementVariable’ are not valid. Self reference is not supported when updating the value of variable ‘Increment_variable’.
What would you like to see next for variables in Azure Logic Apps?
- More data types?
- Cross Logic App variable support?
- Ability to create more than one variable at a time?
- More options than just increment and create?
Finally, BizTalk Server Databases: Disaster Recovery, Troubleshooting and Best Practices whitepaper is published! I think this was my crazy project ever because I started during an MVP Summit, probably in 2013 with a very basic whitepaper of 7 pages about Disaster Recovery and my initial reviewer was Tord Glad Nordahl… for some reason I never finished or published the content and last year again during the MVP Summit I revived this idea and basically annoyed all the MVPs that were next to me that day to review the initial document again… bad idea for me because, probably to get revenge on me, they were the most demanding reviewers that I ever had, always asking for more content making, therefore, the whitepaper more complete. So, what was a 7-pages whitepaper becomes a 34-pages whitepaper.
What to expect about BizTalk Server Databases: Disaster Recovery, Troubleshooting and Best Practices whitepaper
Microsoft BizTalk Server databases and the health of the databases are very important for a successful BizTalk Server messaging environment. BizTalk Server is an extremely database-intensive platform, persisting data to disk with high frequency, and one of the main reasons for that is because one of the primary design goals of BizTalk Server is to ensure that no messages are lost. Therefore, database performance is paramount to the overall performance of any BizTalk Server solution.
There are many factors that you need to take into consideration towards troubleshooting, maintaining, monitoring or recovering from disasters. This paper will provide you some important aspects to consider when working with BizTalk Server databases and addressing the most common and important aspects:
- Size of databases and tables: performance degrades on High Size of BizTalk databases
- Important consideration to avoid large BizTalk Databases
- Separation of data files and log files (SQL Server disk I/O contention)
- Important consideration BizTalk SQL Settings
- Available tools for monitoring and troubleshooting
- Recovering from disasters situations (Clean up your BizTalk databases)
However, the content is very valuable with regards to even preventing a disaster or limit the probability of it. After reading this paper you should be prepared for any disaster but also to preventing for happening because be able to prevent is better than resolving.
Where I can download it
You can download the whitepaper here:
BizTalk Server Databases: Disaster Recovery, Troubleshooting and Best Practices (1.14 MB)
I would like to take this opportunity also to say thanks to my amazing reviewers: Steef-Jan Wiggers, Nino Crudele, Kent Weare, Mikael Hakansson and Salvatore Pellitteri for taking the time to review this whitepaper. And other people that were involved in making this “project” came true like Tord Glad Nordahl, Lex Hegt, Saravana Kumar and Sriram Hariharan.
I hope you enjoy reading this paper and any comments or suggestions are welcome.
March, the last month of Q1 in 2017, a month filled with exciting integration focussed events. After my trip down under I continued my talks at the Integration Monday, SDN Event, my colleagues at Macaw, and the long awaited Global Integration Bootcamp. The latter was a result of months of preparations, and work on labs, meetings and conversations on slack.
In this month, I wrote a few guest blogs for BizTalk360 blog, which will be published in the next months along with an eBook containing all the labs, we (Eldert, Rob, Tomasso and myself) have created for the Global Integration Bootcamp. The eBook contains the hands on labs for API Management, On premise data gateway, Service Bus, Logic Apps, IoT Hub, Stream Analytics and Event Hub. Basically with this labs, you will touch all the services and products that deal with integration.
Any reading this month, to be frank not so much. I read a lot of new papers, magazines and blogs. However, I do like to mention a new book that has been released on the market about integration that is worth to buy and read: Robust Cloud Integration with Azure.
My favorite albums that were released in March were:
- Obituary – Obituary
- Memoriam – For the Fallen
- Pallbearer – Heartless
- Junius – Eternal Rituals for The Accretion Of Light
- Sleepmakeswaves – Made of Breath Only
- Moonloop – Devocean
- Mastodon – Emperor of Sand
In March I did a couple of runs, including a few half marathons, one being the CPC in The Hague wearing a BizTalk360 running shirt.
There you have it Stef’s third Monthly Update and I can look back again with great satisfaction. The Global Integration Bootcamp was a success, which you can read in my previous blog post of this month. I’m looking forward to my trip to Sweden in April and running the full Marathon in Rotterdam on the 9th of April.
The Global Integration Bootcamp was held for the first time this last week, events spanning 12 countries, 16 locations, with over 650 attendees. If you went to either the Seattle, WA location (here at QuickLearn Training’s headquarters), or the New York location, then you may have even ran into one of our instructors!
In the weeks leading up to the day of the bootcamp, Tom Canter with Phidiax arranged a speaker line-up, refreshments, and got the word-out about the event; while over here at QuickLearn Training, we prepared to transform our classrooms into an event space. When the day arrived, all were in good spirits and ready to share knowledge, and get deep into real-world possibilities for hybrid cloud integrations using BizTalk Server and Logic Apps.
Tom kicked off the event with a keynote and introductions, and got everyone primed and excited for the day. Next up was Tord showing off some of the latest greatest features in BizTalk Server 2016 when used in concert with API Management along with a few surprises . I’m not sure what I’m allowed to share and what I’m not, so I will just leave that short, sweet, and to the point.
Gyanendra Gautam teamed up with Ashish Bhambhani (co-authors of the freshly published Robust Cloud Integration with Azure) to show some really slick B2B scenarios with Logic Apps and the Enterprise Integration Pack. Trading Partner and Agreement configuration were shown, along with a special surprise that no one had ever seen before – the world’s smallest X12 834 interchange! It was both a fun and informative session, and if you haven’t at the very least experimented with EDI in Logic Apps – do it. You’ll find your BizTalk Server experience in the same will serve you quite well.
I was up next, wearing a contraption to be explained at a later date. The focus of my talk was to demystify machine learning – and to demonstrate that it’s not just for the sexy applications (e.g., self-driving cars, HoloLens, whatever it is that I’m wearing, etc…). I spent the bulk of my session walking through a simple Hello Azure ML world demo that showed how one could train, operationalize, and then call Azure ML models from within Azure Logic Apps. It is my intention to further refine the models used in this talk and share the full talk, sample code, hardware diagrams, etc. in the summer of 2017.
After I was carted away in a straight jacket, Richard Seroter gave a really cool talk on the intersection between microservices and messaging – and how when using both, one can realize seamless multi-cloud scenarios. It was a very well executed talk with fairly complex demos involving node.js services, java services (built using Spring Boot), and Logic Apps.
Undeterred by a ruthless cold that had claimed his voice, Jeff Hollan gave an excellent talk on the concept of serverless applications. He opened with an analogy comparing owning/renting/hiring a car with the equivalent on the server-side. He then looked to where serverless would lead the development of applications (i.e., API composition).
Kevin Lam wrapped up the day by going through a list of Enterprise Integration Patterns and the implementation required to make it happen on the Logic Apps side. He also addressed how to increase throughput for Service Bus connections, how to control parallelism, advanced scheduling and other fun goodies that I will likely put to quick use (and maybe follow-up with some blog posts on later). One thing did come as quite the surprise though – Sequential Convoys!
It was a great time, and I hope to be able to share more when I can. Thanks to everyone who attended, and I really hope you all had as great of a time as we did.
QuickLearn Training’s offices were just one of many locations for the event. Below is a short gallery of photos gathered from Twitter of other venues.
(I haven’t been able to find a picture with the camera pointed the other way, but I get it, @wearsy is a model now after all).
On March 25th, BizTalk360 along with Microsoft hosted and sponsored the first ever Global Integration Bootcamp event in India at the venue Microsoft GTSC campus in Bangalore. It was a bright Saturday morning. We reached the event venue at 8 AM and surprised to see few participants already present on the venue even though the event was supposed to start only on 9.00 AM. We started the preparations for the event – setting up the registration desk, preparing the stage for the speakers and so on. By the time we were done attendees started to come in for the event.
By 09.00 AM, we had close to 70% percent of the attendance at the event venue. We had a diverse audience ranging from Students to Integration Experts from various organizations. The Welcome Note was given by Sriram Hariharan announcing the spot awards for attendees for the maximum number of Tweets and Retweets.
After the welcome note, Deepak Rajendran started off the proceedings through an excellent keynote, stressing on the importance of community to any platform. The whole keynote was filled with passion and energy and it was an excellent way to kick start Global Integration Bootcamp.
Session 1 – Cognitive Services
The keynote was followed by Harikharan Krishnaraju (Hari) with a presentation on Cognitive services and how we can provide our apps a human side. Hari started of the session with a video which was inspirational and showed the power of the Azure Cognitive Services offering. He then proceeded to explain various API’s that are available for the developers to incorporate those capabilities in their application. He also showed us a demo on how to use Cognitive Services in the Logic Apps and use the power of Emotion API to detect the sentiment using the Twitter feed as your data.
Session 2 – Azure API Management
After a short coffee break, Sunny Sharma from BizTalk360 took the stage to present on the topic Secure and Optimize APIs using Azure API Management. He showed us various Authentication mechanisms that we can employ to protect our APIs and the various options that are available in the developer portal and the publisher portal. The session was received well and followed with lot of questions from the participants.
Session 3 – On-premises data gateway
As the last session before lunch, we had Sajith C P explaining the On-Premises Data Gateway to leverage on-premise data for hybrid scenarios. He also explained the confusion around different gateways available in Azure and provided a great demo using Azure Logic Apps with BizTalk Server connector to connect to a on-premise SQL Server.
After the morning sessions, it was time for the attendees to enjoy the lunch offered by Microsoft. Even though it was the lunch break, most of participants utilized the time to interact with the speakers and clarifying their doubts which is the whole point of any community events.
Session 4 – Enterprise Integration Pack
After the lunch, Shree Divya and Shailesh Agre took the stage to commence their talk on Enterprise Integration Pack with Logic Apps. They explained how we can use Azure Logic Apps for business-to-business (B2B) workflows and seamless communication. They also showed us a demo on how organizations can exchange messages through industry standard protocols like AS2, X12 and EDIFACT.
Session 5 – Automated Workflows
It was time for Lohith G Nagaraj to take stage and talk on his topic Automated Workflows between Apps & Services using Microsoft Flow. Lohith started his session with great enthusiasm which he made sure to spread this to the participants as well. He started off explaining how easy it is to use Microsoft Flow and the difference between the Flow and Logic Apps. He also showed couple of demos on how to monitor your Twitter mentions and how to automate simple workflows within your organization. Finally he showed us various templates that are available in Microsoft Flow.
(Final) Session 6 – Hybrid Integration
It was time for the final session of the day by Roy Joseph and Rekha Kodali on Hybrid Integration. Rekha and Roy made a case for why Hybrid Integration is the future and what the available offerings in the Azure Platform to satisfy all the hybrid integration scenarios.
Pictures from the event
Vote of Thanks
It was time to wrap up the Integration Bootcamp. Arunkumar Kumaresan delivered the vote of thanks and thanked everyone involved for making the event a successful one. Arun also gave out mementos to the speakers for their active participation in the event. He gave out spot awards to one of the attendees, Prakash Nimmala, for the maximum number of Tweets and Retweets.
It was almost time to say goodbye to all the attendees and with this the Global Integration Bootcamp event came to close. We had sense of satisfaction after not just being able to successfully execute the event but the event hosted in India is the biggest in number of participants among all the countries that conducted the Global Integration Bootcamp. Our sincere thanks to all the speakers, Microsoft GTSC Bangalore and specially to the attendees for making this event a grand success.
BizTalk Server 2016 was released recently and one of the biggest questions a lot of existing BizTalk Server customers have is whether to upgrade/migrate to 2016 or stay in the current version. The answer to this question will depend on various factors, let’s try to understand the scenarios that can potentially require or force you to do the upgrade.
Before going into the details let’s take a look at 2 important pieces of information, the history of last few versions of BizTalk Server and what’s new in BizTalk Server 2016. Because these two factors are very important before we make the decision of whether to upgrade to latest version
BizTalk Server history
BizTalk Server is one of the most matured Enterprise Integration Product in the market with over 16 years continuous development and innovation.
- BizTalk Server 2000
- BizTalk Server 2002
- BizTalk Server 2004
- BizTalk Server 2006
- BizTalk Server 2006 R2
- BizTalk Server 2009
- BizTalk Server 2010
- BizTalk Server 2013
- BizTalk Server 2013 R2
- BizTalk Server 2016
What’s new in BizTalk Server 2016
At a very high level, these are the key features that are added in this release.
- Support for Windows Server 2016, SQL Server 2016, Visual Studio 2015
- Support for Host Integration Server 2016
- Support for SQL Server AlwaysOn
- Support for hosting production workloads in Azure IaaS VM’s
- SHA2 Native Support
- New Adapters (Azure Logic Apps)
- Adapter Improvements (SAP Connector, File, FTP, SFTP, Ordered Delivery for Dynamic Send Port)
- Shared Access Signature support for WCF adapters
- Import/Exports – Parties, Tracking Settings
- Simultaneous Multiple artifacts configuration in BizTalk Admin Console
For the complete information please refer to Microsoft release notes. Technically you can migrate to BizTalk Server 2016 seamlessly if you are in the last 2 versions of BizTalk (2013 or 2013 R2), but the decision to upgrade is not always made purely based on technical reasons. Let’s see some scenarios.
Scenario #1: Upgrade based on your platform requirements
This scenario is something you cannot avoid and will force you to upgrade to BizTalk Server 2016. Every enterprise will have an internal platform upgrade story both for Windows Server and SQL Server. The organisation may be currently running on Windows Server 2012 and every 5 years once they might have a platform upgrade across the company. When such things happen they will create a dependency matrix and if they identify any product that can be moved to latest platform, then they will ask the relevant teams to start the initiative. If such scenario happens then you’ll be forced to migrate to BizTalk Server 2016. A similar story can happen for SQL server upgrade across the company.
Scenario #2: Upgrade based on your current version
Based on the current version of BizTalk server you are running in your organisation, you may be forced or it may be a good idea to upgrade to BizTalk Server 2016. The below table gives the BizTalk Server product support lifecycle information with dates for the end of both mainstream and extended support. As you can see 2006, 2006 R2, 2009 and 2010 versions are already out of mainstream support
||Support End (Mainstream)
||Support End (Extended)
|BizTalk Server 2006
|BizTalk Server 2006 R2
|BizTalk Server 2009
|BizTalk Server 2010
|BizTalk Server 2013
|BizTalk Server 2013 R2
|BizTalk Server 2016
There are lot of disadvantages being in extended support,
- Non-Security hotfix support will require extended hotfix agreement, purchased within 90 days of mainstream support ending
- You incident support will be chargeable
- You cannot make warranty claims
- You cannot request design changes and feature requests
You probably do not want to run your business on limited support, so it’s better to upgrade if your current version is already in extended support or your mainstream support is coming.
Scenario #3: Upgrade based on new features
There may be some features released in BizTalk Server 2016 that might demand an upgrade. Example: SQL AlwaysOn, traditionally companies struggle to set up log shipping for DR and in many instances, they never had any DR setup that could be resolved by using SQL Always on. In a similar way, there may be other features like SAP connector, Ordered Delivery on dynamic send ports etc can all be interesting for certain scenarios.
Scenario #4: Upgrade based on your current projects
Most of the time the cost of migrating from one version of BizTalk server to another is not the BizTalk server software cost, because typically you’ll be either covered with annual support assurance (SA) or you’ll have enterprise agreement package (EAP), which will allow you to upgrade and do a true-up cost at end of the period.
The biggest cost factor is your functional testing efforts. Even though Microsoft (or any software vendor for that matter) will claim the upgrade will be seamless and will not affect anything. Chances are, something will get broken. If you look at the stack, you are upgrading a lot of things Windows, SQL, .NET, BizTalk etc. at one go and you need to be super careful.
The best situation for you to plan for your upgrade and reduce the cost is to do it along with your project plans. If you have a big release coming, and if you are already planning for a lot of functional testing, use that as an opportunity to migrate to the new version.
If somebody claims their software/tool will magically help you on the upgrade process be cautious. There is no magic bullet here, no one will understand the complexities of the legacy systems you are connecting and their underlying limitations. Only your QA team can warrant that after verifying their test cases.
Scenario #5: What if we are in the process of buying BizTalk Server
In this case, it’s kind of a no-brainer decision, you just buy the latest version of the product i.e BizTalk Server 2016. Having said that we have seen companies using the policy of always using “Latest Version – 1” formula. They don’t want to risk with some unknown potential problems and wanted to stay with the version that’s in the market for a while. If you are in that situation, it’s perfectly fine to go with BizTalk Server 2013 R2. But keep in mind BizTalk Server 2013 R2 will not run (or supported) in Windows Server 2016 and SQL Server 2016.
Please leave your comments if you have any other reasons.
The post Top reasons to upgrade to BizTalk Server 2016 appeared first on BizTalk360.
Being an MVP, and a community leader, one of my roles is to motivate others members to start contributing to the community and at DevScope we have great new technicians that sometimes only need to be pushed and motivated… this time I challenged my coworker Rui Silva to collaborate with me on this small project and the result is here: BizTalk MapperExtensions Functoid Wizard for BizTalk Server 2016.
BizTalk MapperExtensions Functoid Wizard is a Custom Functoid Project Wizard for Visual Studio 2015. It allows you to create a new Functoid project for BizTalk Server 2016 without having to create manually the project, in other words, having to manually create:
- A new class library project in Visual Studio 2015;
- Add a reference to the Microsoft.BizTalk.BaseFunctoids assembly;
- Add New Class and having to code the entire class;
This Wizard will do this process automatically, and used in conjunction with BizTalk Server: Custom BizTalk Functoid item template for Visual Studio 2015 will facilitate and significantly expedite the development process of our projects. The only thing you will need to do is create a new Custom Functoid Project and a wizard will pop-up, fill in the required fields and when you finish the process a new Custom Functoid project is built. Just change the execution method with your own algorithms, build your project and it’s done. You’ll see your new Functoid in BizTalk Server 2015 Mapper once you import it to Visual Studio Toolbox.
BizTalk MapperExtensions Functoid Wizard Installation
To use this project template download the executable files available on “Installation Files” folder and run the setup file.
Once you run the setup file follow these steps:
- On the “Welcome to the BizTalk MapperExtensions Functoid Template Setup Wizard” screen, click “Next”.
- On the “Select Installation Folder” screen, select the folder where you want to install the Wizard and select the option “Everyone”. Click “Next” to continue.
- On the “Confirm Installation” screen, confirm your intent to install by clicking “Next”
- On the “Installation Complete” screen, click close and the installation is complete
Now you can see a new BizTalk project template option under “BizTalk projects” in your Visual Studio 2015.
Create New Functoid Project
To create a new functoid using this project template choose the option “BizTalk Server Functoid Project” in Visual Studio 2015 and a Wizard will pop-up. Follow the requirements of this wizard to create the Functoid.
- In the “Start” screen, click “Next” to continue
- In the “General Project Properties” screen, fill the Functoid class name, namespace and create or select a new Strong Key (you can use an existing one). Click “Next” to continue.
- In the “Functoid Properties” screen, define the Functoid ID (must be greater than 10000), the functoid name, the tooltip (short description) and description (full description). Click “Next” to continue.
- In the second “Functoid Properties” screen, choose the functoid category and the implementation language (the language in which you want to implement your functoid behavior code): C# or VB.NET. Click “Next” to continue.
- In the “Functoid Parameters and Connection Types” screen, define the functoid function name, function inputs and types and output and types. Click “Next” to continue.
- On the final screen, click “Finish” to create the Visual Studio project.
Visual Studio will create a new Project based on your implementation language and definitions provided in the Wizard.
- Open the generated class and implement your code
Once again, special thanks to Rui Silva who accepted the challenge and collaborated with me in order to finalize this project.
You can download the Source Code or contribute to the project here:
You can download the installation files (only executables) here:
BizTalk MapperExtensions Functoid Wizard: BizTalk Server 2016 Installation files (1.1 MB)
Microsoft | TechNet Gallery
Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?
Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.
If you want to receive these updates weekly, then don’t forget to Subscribe!
Cloud and Hybrid Integration:
Hope this would be helpful. Please feel free to let me know your feedback on the Integration weekly series.
On Saturday 25 March, 2017 the Auckland Connected Systems User group held the NZ version of this global event. Saravana and Eldert have already summarised the global event. This blog will summarise the Auckland event only.
This is the third integration Saturday that ACUSG have run. The first event called “BizTalk Saturday” was organised by Nikolai Blackie and Bill Chesnut on 6th November 2010. The second event called “Integration Saturday” was held on July 18th 2015 once again had Bill Chesnut as the main speaker.
The organisers and presenters for the GIB 2017 event where Mark Brimble, Craig Haiden, Wagner Silveira, Nikolai Blackie, Mike Howell, Mahindra Morar, Morten Vellig, Abhishek Kumar and James Corbould.
Wagner said after the event “As the first team to go through the Global Integration Bootcamp 2017, I just wanted to say that it was an amazing experience. We ran from 8:30 am to 5:30 pm with an attendance of 30+ people and not a single dropout throughout the day. Everyone was quite interested in the presentations and tried the labs with real enthusiasm either individually or in groups. Actually was interesting to see a groups of people congregating around the same computer discussing, bouncing ideas, and thinking how to apply the knowledge on their own environment while doing the labs.A big shout out to the main organizers! I’m pretty sure I didn’t captured all of them here, but Eldert, Steef-Jan, Rob, please send my big thanks to them.”
Craig opened the event and then talked about API management.
Next up was Mahindra and Mike talked Service Bus, Enterprise Integration pack and On-Prem gateway.After Lunch Abhishek walked us through Hybrid Integration.He was followed by Morten who gave a compeliing demo of Power Apps and Flow.
Wagner bought the house down with a slick presentation about Logic Apps.
James his partner in crime did an even slicker run through of Azure functions and Azure storage.
Finally Nikolai blew ever one away with IOT.
I had a really enjoyable day catching up with some old friends and just soaking up all the good integration vibrations. Thanks to all and I am looking forward to next years event.