Well, Empowering Hybrid Integration segment is online since January 30, 2017, but only now I realize that… During the MVP Global Summit, and taking the opportunity of having most of the Integration MVPs gathered at the event, Microsoft Pro Integration team tends to invite the MVPs to create short videos – 8 to 10 minutes videos – about Integration (on-premise or in the cloud). It happened in previous editions of the Summit and it happened in the last edition in which they invite us to talk about Hybrid Integration that includes BizTalk Server, Azure Logic Apps, Azure API Management and Azure Service Bus.
For several reasons, I never had the chance to accept the challenge in the past, but last year I was able to accept the invitation to speak and record at the “famous” Microsoft Building 20, building that is the home of Channel 9 studio …
… and make a short talk about Hybrid Integration using Service Bus Queues, BizTalk Server, Dynamic CRM, SQL Server, Logic Apps and File.
Has I told in a previous post of my blog, that was my first experience, and I have to confess, I was a little nervous to be recording in a small professional studio, so as expected after seeing the record in its raw state, the end result could have been better, but, fortunately for you guys I was able to edit the video and remove some breaks/delays between switching from my presentation slides to my screen. Nevertheless, for a first experience, it went pretty well and I am happy with the result. The important for me is to face new challenges, different situations, leave my comfort zone and improve to be better next time.
The result is this 8 minutes video on Empowering Hybrid Integration where you can learn how you can create powerful hybrid integration solutions using existing on-premises and cloud based solutions.
Fortunately, I had the company of my good friend Kent Weare in this small “adventure”:
You can also watch his session about “Protecting Azure Logic Apps using Azure API Management”. In this video, you can learn how to use Azure API Management to protect your Logic Apps.
You can watch more sessions regarding Microsoft’s Hybrid Integration Platform on Channel 9 here.
In the last period, I invested time in something that I think is one of the most important aspect in the integration space, the perception.
During my last events I start introducing this topic and I’m going to add a lot more in the next future.
In Belgium, during the BTUG.be, I shown my point of view about open patterns and in London, during the GIB, I presented how I approach to the technology and how I like to use it, I’ve been surprised by how many people enjoyed my sharing.
The Microsoft stack is full of interesting options, in the on premise and Azure cloud space, people like to understand more about how to combine them and how to use them in the most profitable way.
In my opinion, the perception we have about a technology is the key and not the technology itself and this is not just about the technology but this is something related about everything in our life.
We can learn how to develop using any specific technology stack quite easily but it is more complicate to get the best perception of it.
I like to use my great passion for skateboarding to better explain this concept, a skateboard is one of the most simple object we can find, just a board with 4 wheels and a couple of trucks.
It is amazing seeing how people use the same object in so many different ways, like vert ramps, freestyle, downhill or street and how many different combinations of styles in each discipline and each skater has an own style as well.
Same thing needs to be done for the technology, I normally consider 4 main areas, BizTalk Server, Microsoft Azure, Microsoft .Net or SQL and the main open source stack.
I don’t like to be focused about using a single technology stack and I don’t think is correct using a single technology stack to solve a problem.
For instance, BizTalk Server is an amazing platform full of features and able to cover any type of integration requirement and looking to the BizTalk architecture we can find a lot interesting considerations.
The slide below is a very famous slide used in millions of presentations.
Most of the people look at BizTalk Server as a single box with receive ports, hosts, orchestrations, adapters, pipelines and so on.
When I look to BizTalk Server I see a lot of different technology stacks to use, to combine together and to use with other stacks as well.
I can change any BizTalk Server behaviour and I can completely reinvent the platform as I want, I don’t see real limit on that and same thing with the Microsoft Azure stack.
Microsoft Azure offers thousands of options, the complicate aspect is the perception we have for any of these.
Many times our perception is influenced by the messages we received from the community or the companies, I normally like to approach without considering any specific message, I like to approach to any new technology stack like a kid approach to a new toy, I don’t have any preconception.
Today we face a lot of different challenges and one of the most interesting is the on premise integration.
The internal BizTalk architecture itself is a good guidance to use, we have any main concept like mediation, adaptation, transformation, resilience, tracking and so on.
If we split the architecture in two different on premise areas we open many points of discussion, how to solve a scenario like that today?
Below is the same architecture but using any technology stack available, during the GIB I shown some real cases and sample about that.
I also like to consider event driven integration and in that case GrabCaster is a fantastic option.
My next closets event will be TUGAIT in Portugal, TUGAIT is the most important community event organized in Portugal, last year I had the privilege to be there and it was an amazing experience.
3 days of technical sessions and covering any technology stack, integration, development, database, IT, CRM and more.
Many people attend TUGAIT from any part of the Europe, I strongly recommend to be there, nice event and great experience.
Today I’m going to write about a different topic, we can say that is related to technology but not exactly a technical blog post. And no, I’m not feeling special (I’m always feeling special but in different contexts), proud and honored today because it’s my birthday.
January 1st, 2011, will always be a special day for me, it was the day I had the honor of being awarded Microsoft Most Value Professional (MVP) for the first time which was an unexpected good surprise. This is my 7th straight year on the MVP Program and since the first day, it has been a delighted journey and experience. I was the first and for many years the only MVP at my company DevScope but, contrary to what many people might think, is was not for that reason also that I could feel special. I’m feeling special for somehow, directly or indirectly, being able to help to grow the spirit of community collaboration, a little more (because it always existed), internally at the company and sometimes, maybe, by serving as a mentor, a role model or an inspiration to others. But especially by encouraging and helping everyone to start contributing and share their knowledge with the communities, often challenged them to collaborate with me on small projects.
I’m feeling special, proud and honored today to be part of a company who after all these years still keeps their main core identities, with a focus on cutting-edge technology, early adopter in most cases, focus on innovation and helping with the needs of our clients, but without losing the focus on the people, helping them to grow personal and professional. I’m feeling special, proud and honored today because I can say that I work with a team of amazing professionals and special to be able to realize that we already have 3 MVPs at DevScope, one former MVP and one Microsoft Regional Director (there are not many companies worldwide that can boast on these facts).
I’m feeling special, proud and honored today, not also for working with such hard working and talented team, but also, for being able to realize that new talents are being formed.
… but let’s also be fair, I way also feeling special, proud and honored today because it is my birthday and I have an amazing family, an amazing job and the best friends in the world!
Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?
Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.
If you want to receive these updates weekly, then don’t forget to Subscribe!
Cloud and Hybrid Integration:
Hope this would be helpful. Please feel free to let me know your feedback on the Integration weekly series.
NZ daylight saving time ends at 3a.m today (2/3/2017) ends. To celebrate this I am going to share a string datetime functoid that I have used for many years.
A common task when mapping one message to another is transforming a date e.g. 1/3/17 15:30 to 2017-03-01T15:30+13:00. Many years ago I was reviewing some maps in a BizTalk solution and discovered that just about every map was using different C# code in a C# scripting functoid to transform between date formats. I decided to create a functoid that would convert any string that was a valid date to any other date format. The two objectives were to make sure that all date format changes were consistent and it was easy to use.
Here is are some maps that show cases the StringDateTime functoid. In this case the UTC date 2017-02-25T08:18:48.000Z is transformed to 25/02/2017 21:18
Note in the example above that the time is incremented by 13 hours because on the 25/2 NZ is on daylight saving and is UTC+13.
The code that does the work is shown below.
You can download the class file, bit map and resource file from here
if you want to create the functoid yourself. Please see one of Sandro Pereira’s posts
if you need any help to create it.
In summary this BizTalk functoid gives consistency to the way dates are formatted and is easy to use. All you need to enter is the either a standard date format or a custom date format.
Modern enterprise IT is hybrid and relies on data from on-premises and the cloud. At the core of Microsoft’s Hybrid Integration Platform, we have BizTalk Server which handles all your on-premises integration scenarios and Logic Apps, which takes care of all the cloud integrations leveraging Azure Services.
BizTalk 360 already provides a comprehensive monitoring solution for your Hybrid Integrations from version 8.3 which introduced Logic Apps State monitoring.
We are excited to announce the addition of Logic Apps Metrics monitoring capabilities to our forthcoming 8.4 release (expected: Apr 2017).
With this release, you will be able to monitor your logic apps using all the available metrics that are provided in the Azure Portal.
Setting Data Monitoring for Logic Apps metrics
The new Logic Apps monitoring capability is just an extension to our already existing data monitoring functionality. You can find the newly added tab for the Logic Apps under the Data Monitoring module.
Click the Add New button to add a new monitor for the Logic Apps. A new Logic Apps Monitor setup window will appear. We can see that all the Logic Apps related settings are grouped under “SET DATA FILTER” section in the below image.
Select the subscription from the list of subscriptions that you have configured in BizTalk 360. Please check this Assist article if you want to know how to add Azure subscriptions in BizTalk360.
Once the subscription is selected BizTalk360 will display all the Logic Apps that are available in that selected subscription. Finally select the metric that you want to monitor.
Once you set the data filter you can specify the notification condition which BizTalk360 will use to send alerts for each violation as shown below.
Note: Azure metrics can have different units based on the metric type so metrics like Action Latency will have milliseconds as the unit type. In those cases, the threshold condition must be set based on the selected metric.
Save the Logic Apps monitoring schedule. BizTalk360 Monitor service will pick up the new monitor configuration and customers will start receiving the email notifications or any other notification channels based on the Alarm configuration.
You can also view the monitoring results in the Data Monitoring Dashboard as shown in the below image.
Click an alert item to view the details.
We hope this new capability will enhance the monitoring use case for all our customers having Hybrid Integration scenarios. If you have any feedback, please write to firstname.lastname@example.org.
Finally, BizTalk Server Databases: Disaster Recovery, Troubleshooting and Best Practices whitepaper is published! I think this was my crazy project ever because I started during an MVP Summit, probably in 2013 with a very basic whitepaper of 7 pages about Disaster Recovery and my initial reviewer was Tord Glad Nordahl… for some reason I never finished or published the content and last year again during the MVP Summit I revived this idea and basically annoyed all the MVPs that were next to me that day to review the initial document again… bad idea for me because, probably to get revenge on me, they were the most demanding reviewers that I ever had, always asking for more content making, therefore, the whitepaper more complete. So, what was a 7-pages whitepaper becomes a 34-pages whitepaper.
What to expect about BizTalk Server Databases: Disaster Recovery, Troubleshooting and Best Practices whitepaper
Microsoft BizTalk Server databases and the health of the databases are very important for a successful BizTalk Server messaging environment. BizTalk Server is an extremely database-intensive platform, persisting data to disk with high frequency, and one of the main reasons for that is because one of the primary design goals of BizTalk Server is to ensure that no messages are lost. Therefore, database performance is paramount to the overall performance of any BizTalk Server solution.
There are many factors that you need to take into consideration towards troubleshooting, maintaining, monitoring or recovering from disasters. This paper will provide you some important aspects to consider when working with BizTalk Server databases and addressing the most common and important aspects:
- Size of databases and tables: performance degrades on High Size of BizTalk databases
- Important consideration to avoid large BizTalk Databases
- Separation of data files and log files (SQL Server disk I/O contention)
- Important consideration BizTalk SQL Settings
- Available tools for monitoring and troubleshooting
- Recovering from disasters situations (Clean up your BizTalk databases)
However, the content is very valuable with regards to even preventing a disaster or limit the probability of it. After reading this paper you should be prepared for any disaster but also to preventing for happening because be able to prevent is better than resolving.
Where I can download it
You can download the whitepaper here:
BizTalk Server Databases: Disaster Recovery, Troubleshooting and Best Practices (1.14 MB)
I would like to take this opportunity also to say thanks to my amazing reviewers: Steef-Jan Wiggers, Nino Crudele, Kent Weare, Mikael Hakansson and Salvatore Pellitteri for taking the time to review this whitepaper. And other people that were involved in making this “project” came true like Tord Glad Nordahl, Lex Hegt, Saravana Kumar and Sriram Hariharan.
I hope you enjoy reading this paper and any comments or suggestions are welcome.
March, the last month of Q1 in 2017, a month filled with exciting integration focussed events. After my trip down under I continued my talks at the Integration Monday, SDN Event, my colleagues at Macaw, and the long awaited Global Integration Bootcamp. The latter was a result of months of preparations, and work on labs, meetings and conversations on slack.
In this month, I wrote a few guest blogs for BizTalk360 blog, which will be published in the next months along with an eBook containing all the labs, we (Eldert, Rob, Tomasso and myself) have created for the Global Integration Bootcamp. The eBook contains the hands on labs for API Management, On premise data gateway, Service Bus, Logic Apps, IoT Hub, Stream Analytics and Event Hub. Basically with this labs, you will touch all the services and products that deal with integration.
Any reading this month, to be frank not so much. I read a lot of new papers, magazines and blogs. However, I do like to mention a new book that has been released on the market about integration that is worth to buy and read: Robust Cloud Integration with Azure.
My favorite albums that were released in March were:
- Obituary – Obituary
- Memoriam – For the Fallen
- Pallbearer – Heartless
- Junius – Eternal Rituals for The Accretion Of Light
- Sleepmakeswaves – Made of Breath Only
- Moonloop – Devocean
- Mastodon – Emperor of Sand
In March I did a couple of runs, including a few half marathons, one being the CPC in The Hague wearing a BizTalk360 running shirt.
There you have it Stef’s third Monthly Update and I can look back again with great satisfaction. The Global Integration Bootcamp was a success, which you can read in my previous blog post of this month. I’m looking forward to my trip to Sweden in April and running the full Marathon in Rotterdam on the 9th of April.
The Global Integration Bootcamp was held for the first time this last week, events spanning 12 countries, 16 locations, with over 650 attendees. If you went to either the Seattle, WA location (here at QuickLearn Training’s headquarters), or the New York location, then you may have even ran into one of our instructors!
In the weeks leading up to the day of the bootcamp, Tom Canter with Phidiax arranged a speaker line-up, refreshments, and got the word-out about the event; while over here at QuickLearn Training, we prepared to transform our classrooms into an event space. When the day arrived, all were in good spirits and ready to share knowledge, and get deep into real-world possibilities for hybrid cloud integrations using BizTalk Server and Logic Apps.
Tom kicked off the event with a keynote and introductions, and got everyone primed and excited for the day. Next up was Tord showing off some of the latest greatest features in BizTalk Server 2016 when used in concert with API Management along with a few surprises . I’m not sure what I’m allowed to share and what I’m not, so I will just leave that short, sweet, and to the point.
Gyanendra Gautam teamed up with Ashish Bhambhani (co-authors of the freshly published Robust Cloud Integration with Azure) to show some really slick B2B scenarios with Logic Apps and the Enterprise Integration Pack. Trading Partner and Agreement configuration were shown, along with a special surprise that no one had ever seen before – the world’s smallest X12 834 interchange! It was both a fun and informative session, and if you haven’t at the very least experimented with EDI in Logic Apps – do it. You’ll find your BizTalk Server experience in the same will serve you quite well.
I was up next, wearing a contraption to be explained at a later date. The focus of my talk was to demystify machine learning – and to demonstrate that it’s not just for the sexy applications (e.g., self-driving cars, HoloLens, whatever it is that I’m wearing, etc…). I spent the bulk of my session walking through a simple Hello Azure ML world demo that showed how one could train, operationalize, and then call Azure ML models from within Azure Logic Apps. It is my intention to further refine the models used in this talk and share the full talk, sample code, hardware diagrams, etc. in the summer of 2017.
After I was carted away in a straight jacket, Richard Seroter gave a really cool talk on the intersection between microservices and messaging – and how when using both, one can realize seamless multi-cloud scenarios. It was a very well executed talk with fairly complex demos involving node.js services, java services (built using Spring Boot), and Logic Apps.
Undeterred by a ruthless cold that had claimed his voice, Jeff Hollan gave an excellent talk on the concept of serverless applications. He opened with an analogy comparing owning/renting/hiring a car with the equivalent on the server-side. He then looked to where serverless would lead the development of applications (i.e., API composition).
Kevin Lam wrapped up the day by going through a list of Enterprise Integration Patterns and the implementation required to make it happen on the Logic Apps side. He also addressed how to increase throughput for Service Bus connections, how to control parallelism, advanced scheduling and other fun goodies that I will likely put to quick use (and maybe follow-up with some blog posts on later). One thing did come as quite the surprise though – Sequential Convoys!
It was a great time, and I hope to be able to share more when I can. Thanks to everyone who attended, and I really hope you all had as great of a time as we did.
QuickLearn Training’s offices were just one of many locations for the event. Below is a short gallery of photos gathered from Twitter of other venues.
(I haven’t been able to find a picture with the camera pointed the other way, but I get it, @wearsy is a model now after all).
On March 25th, BizTalk360 along with Microsoft hosted and sponsored the first ever Global Integration Bootcamp event in India at the venue Microsoft GTSC campus in Bangalore. It was a bright Saturday morning. We reached the event venue at 8 AM and surprised to see few participants already present on the venue even though the event was supposed to start only on 9.00 AM. We started the preparations for the event – setting up the registration desk, preparing the stage for the speakers and so on. By the time we were done attendees started to come in for the event.
By 09.00 AM, we had close to 70% percent of the attendance at the event venue. We had a diverse audience ranging from Students to Integration Experts from various organizations. The Welcome Note was given by Sriram Hariharan announcing the spot awards for attendees for the maximum number of Tweets and Retweets.
After the welcome note, Deepak Rajendran started off the proceedings through an excellent keynote, stressing on the importance of community to any platform. The whole keynote was filled with passion and energy and it was an excellent way to kick start Global Integration Bootcamp.
Session 1 – Cognitive Services
The keynote was followed by Harikharan Krishnaraju (Hari) with a presentation on Cognitive services and how we can provide our apps a human side. Hari started of the session with a video which was inspirational and showed the power of the Azure Cognitive Services offering. He then proceeded to explain various API’s that are available for the developers to incorporate those capabilities in their application. He also showed us a demo on how to use Cognitive Services in the Logic Apps and use the power of Emotion API to detect the sentiment using the Twitter feed as your data.
Session 2 – Azure API Management
After a short coffee break, Sunny Sharma from BizTalk360 took the stage to present on the topic Secure and Optimize APIs using Azure API Management. He showed us various Authentication mechanisms that we can employ to protect our APIs and the various options that are available in the developer portal and the publisher portal. The session was received well and followed with lot of questions from the participants.
Session 3 – On-premises data gateway
As the last session before lunch, we had Sajith C P explaining the On-Premises Data Gateway to leverage on-premise data for hybrid scenarios. He also explained the confusion around different gateways available in Azure and provided a great demo using Azure Logic Apps with BizTalk Server connector to connect to a on-premise SQL Server.
After the morning sessions, it was time for the attendees to enjoy the lunch offered by Microsoft. Even though it was the lunch break, most of participants utilized the time to interact with the speakers and clarifying their doubts which is the whole point of any community events.
Session 4 – Enterprise Integration Pack
After the lunch, Shree Divya and Shailesh Agre took the stage to commence their talk on Enterprise Integration Pack with Logic Apps. They explained how we can use Azure Logic Apps for business-to-business (B2B) workflows and seamless communication. They also showed us a demo on how organizations can exchange messages through industry standard protocols like AS2, X12 and EDIFACT.
Session 5 – Automated Workflows
It was time for Lohith G Nagaraj to take stage and talk on his topic Automated Workflows between Apps & Services using Microsoft Flow. Lohith started his session with great enthusiasm which he made sure to spread this to the participants as well. He started off explaining how easy it is to use Microsoft Flow and the difference between the Flow and Logic Apps. He also showed couple of demos on how to monitor your Twitter mentions and how to automate simple workflows within your organization. Finally he showed us various templates that are available in Microsoft Flow.
(Final) Session 6 – Hybrid Integration
It was time for the final session of the day by Roy Joseph and Rekha Kodali on Hybrid Integration. Rekha and Roy made a case for why Hybrid Integration is the future and what the available offerings in the Azure Platform to satisfy all the hybrid integration scenarios.
Pictures from the event
Vote of Thanks
It was time to wrap up the Integration Bootcamp. Arunkumar Kumaresan delivered the vote of thanks and thanked everyone involved for making the event a successful one. Arun also gave out mementos to the speakers for their active participation in the event. He gave out spot awards to one of the attendees, Prakash Nimmala, for the maximum number of Tweets and Retweets.
It was almost time to say goodbye to all the attendees and with this the Global Integration Bootcamp event came to close. We had sense of satisfaction after not just being able to successfully execute the event but the event hosted in India is the biggest in number of participants among all the countries that conducted the Global Integration Bootcamp. Our sincere thanks to all the speakers, Microsoft GTSC Bangalore and specially to the attendees for making this event a grand success.