DevOps – Multi environment variables/ groups

DevOps – Multi environment variables/ groups

This is a topic that has been asked to me a few times, making me wonder how hard it actually was. Working with this nearly every day makes us assume some things are very easy, but not everyone has this insight.

So, exactly do we set variables for different environments and how does it work when we want to replace tokens?

Variables for different environments

Having multiple environments creates the need to have different values assigned to your variables, because, for example, that Test Webservice won’t work in PROD and you definitely don’t want to use that PROD file share and delete files in your DEV/Test environment.

Using Pipeline Variables helps you to set different values to different Stages.

This is extremely helpful because, even though you have to duplicate/triplicate variables, you won’t need to worry about the incorrect value going to the wrong stage. Also, having the Scope set to Release, it will affect all stages.

So, it’s a win-win situation.

But! It’s only valid for this Release Pipeline in specific. If you have another Release and some variables are common, you have to re-do everything… all, over, again.

Send in the Variable Groups!

Variable Groups

The Variable Groups are containers for variables that can be used in multiple Releases and Pipelines. Think of it as a common class in your project that you can reference anywhere.

You can define the Groups and their variables in the Library. Inside the group, you can set all the variables you need, and add to it any time as well, and assign the values right away.

Keep in mind that this is thought of as a static group, it’s not supposed to change often.

If you change a variable value or add a new one, it will not be considered in the already created releases. If anything changes in here, you will need to create new releases (not the pipelines) and redeploy them. When you create the release, it takes a snapshot of the values and uses them as they are. Thus the need to create a new one to get those new values.

After linking the group to the Release, you will see that you can also set a Scope. This works exactly like the pipeline variables, they will only be used in that specific Stage and nowhere else.

Also, when expanded, you can see the values that are set for that group.

Now, how does the Token Replacement task works with this?

Replace Tokens

This task, our savior (yes, I like it very very much), comes to our rescue once again.

I’ve explained before how to use it and how it works.

But for this post, I’ll explain again. The task searches in the folders/files you’ve defined and tries to match the token that you’re setting in the definition with the one in the file(s). As the token is found, it uses a string.Replace function to inject the values in the files.

It will scour the Variables for a match and take the value to insert in the file.

But how does this link with the Variable Groups?

Well, at runtime, DevOps does a magical thing and sees the groups you’ve defined for a Stage as variables. So technically, it’s as if you’ve defined all the variables in one place and not in groups.

Pretty sweet, right?

So, the Replace Tokens will use all those variables and will try to replace them in your files. You don’t have to define the group or anything, it will just see the whole picture.

Hope this helps you with your automations and deployments.

Happy coding!

The post DevOps – Multi environment variables/ groups appeared first on SANDRO PEREIRA BIZTALK BLOG.

Merry Christmas & Happy New Year 2022

Merry Christmas & Happy New Year 2022

We wish you a Merry Christmas
We wish you a Merry Christmas
We wish you a Merry Christmas and a Happy New Year!

We are entering the 3rt pandemic year. We are in a better position, the vaccination is helping minimize the impact of this pandemic, but there is still a long way to go. This makes his holiday season even more important because we never know what tomorrow will be, so on this Christmas, I hope you all can spend time with your close family and friends in the safest way you can. And if you are like me, with three young kids, then it’s all about creating happy memories that will last a lifetime, and we need it more than ever!

And once again, my sincere wishes for a Merry Christmas and a Happy New Year to all my readers, friends, customers, partners, coworkers, my amazing Integration Team at DevScope (Pedro Almeida and Diogo Formosinho – you guys rock!), to all Microsoft Integration and Azure Community (BizTalk Server, Logic Apps, API Management, Service Bus, and so on), MSFT Product Groups, all the Portuguese Communities, my MVP “family” and of course to my beautiful family.

Thanks in advance for all the support and encouragement given throughout another year – 11 years as MVP. I couldn’t make it without you guys!. It was, once again, an incredible year for me, on both a personal and professional level, and I hope that the next will be a year filled with new challenges. I promise that I will continue to share knowledge, hoping to help someone.

May this festive season sparkle and shine, may all of your wishes and dreams come true, and may the new year be made of great happiness. Merry Christmas to you and your entire family, and a happy new year!

The post Merry Christmas & Happy New Year 2022 appeared first on SANDRO PEREIRA BIZTALK BLOG.

December 20, 2021 Weekly Update on Microsoft Integration Platform & Azure iPaaS

December 20, 2021 Weekly Update on Microsoft Integration Platform & Azure iPaaS

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform and Azure iPaaS?

Integration weekly updates can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

Microsoft Announcements and Updates

 

Community Blog Posts

 

Videos

Podcasts

How to get started with iPaaS design & development in Azure?

  • Robust Cloud Integration with Azure
  • Microsoft Azure for Developers: What to Use When
  • Serverless Computing: The Big Picture
  • Azure Logic Apps: Getting Started
  • Azure Logic Apps: Fundamentals
  • Microsoft Azure Developer: Creating Enterprise Logic Apps
  • Microsoft Azure API Management Essentials
  • Azure Functions Fundamentals
  • Cloud Design Patterns for Azure: Availability and Resilience
  • Architecting for High Availability in Microsoft Azure

Feedback

Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

The post December 20, 2021 Weekly Update on Microsoft Integration Platform & Azure iPaaS appeared first on Hooking Stuff Together.

Logic Apps: Sending Files Dynamically – A little Christmas gift

Logic Apps: Sending Files Dynamically – A little Christmas gift

A few months ago I was involved in an integration project that required sending files to a local file system, that resided in an Azure VM.

This was a bit of an issue, because we had to use the ISE connector and it was in Preview at the time, we had to do a Proof-of-concept and make sure it worked properly.

It all went well, the integration went live and the client was happy.

A couple weeks ago, Michael Stephenson asked for my help, to solve an issue that he was having. The problem was on how to send files to another system.

Now, in an operative system, you’d just move the file over by code or using the UI. But in the integration with Logic Apps, what came out was a JSON message.

Of course you can copy files over to another Azure storage, but in this particular case, we were pulling files with a webservice and dropping them into a local VM. The OOTB connectors help, but they don’t create the file object on their own.

In my example, I’m using an Azure File Storage connector to get the file content. It mimics a Webservice where you’d get the same content. In this case, we’re also getting the content-type in the response.

If we set the File content to a variable, it will just send, well, the content itself. In my example, it’s a XML log file, so it’s actually translating the Base64 into a XML string, which is interesting. It might not be properly formatted, but it’s a string at the end of the day, instead of its Base64 representation.

So, we pick up the file, assign the string to a String object and set it to create in the end system.

This works, but creates a string file but this is not what we wanted. We also need to keep the MIME type of the file.

I’ve searched for help in how to achieve this, but there’s no information about it. This is quite shocking, because it almost sounds like Microsoft doesn’t want you to know how to do this or they didn’t took the time to document this.

The good thing, is that it’s quite simple to do this. The JSON object we need to create is extremely simple and very easy to understand how it works.

Let’s get another example. In this case, we have a SOAP web service that returns an array of fields, one of them being the Base64 for the file and the MIME type. This is the perfect example of our issue. As I’ve said before, if I just set the FileData field to a variable, it will end up in sending the string to a file, not caring about what type of file it is.

So, grab those fields into a JSON Parse action, we’re about to jingle some files into your system.

Create a Compose action and…

TA-Daaa! You just need to add a $content and $content-type fields in a JSON object and it will create your file.

It’s this easy!

It’s interesting that the GetFile connector provides the answer we needed, but as I’ve said, there’s no actual guide lines for this. There might be a few other metadata fields you can set for a file, I haven’t explored this yet, but it makes sense to be able to do it.

I hope this helps you and takes away one headache before Christmas.

The post Logic Apps: Sending Files Dynamically – A little Christmas gift appeared first on SANDRO PEREIRA BIZTALK BLOG.

December 13, 2021 Weekly Update on Microsoft Integration Platform & Azure iPaaS

December 13, 2021 Weekly Update on Microsoft Integration Platform & Azure iPaaS

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform and Azure iPaaS?

Integration weekly updates can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

Microsoft Announcements and Updates

 

Community Blog Posts

 

Videos

Podcasts

  • Building a User-Friendly Product with Aparna Sinha

How to get started with iPaaS design & development in Azure?

  • Robust Cloud Integration with Azure
  • Microsoft Azure for Developers: What to Use When
  • Serverless Computing: The Big Picture
  • Azure Logic Apps: Getting Started
  • Azure Logic Apps: Fundamentals
  • Microsoft Azure Developer: Creating Enterprise Logic Apps
  • Microsoft Azure API Management Essentials
  • Azure Functions Fundamentals
  • Cloud Design Patterns for Azure: Availability and Resilience
  • Architecting for High Availability in Microsoft Azure

Feedback

Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

The post December 13, 2021 Weekly Update on Microsoft Integration Platform & Azure iPaaS appeared first on Hooking Stuff Together.

FILE-RADITZ Adapter: File adapter that ignores empty (zero bytes) files

FILE-RADITZ Adapter: File adapter that ignores empty (zero bytes) files

Recently I developed a custom file adapter called FILE-Z adapter that I blogged on the BizTalk360 blog. This is an adapter created from the original idea of my dear friend Nino Crudele. It is a custom File adapter capable of reading empty files.

You can see his original blog post here: BizTalk and zero byte file. And you can download the FILE-Z adapter here: FILE-Z adapter GitHub page.

But while I was searching about this adapter, I found an interesting blog post from Yan (Pamela) Yang where she described a different scenario. Some systems can create empty files without locking them. Then, they grab the files again and write some data to them. However, this behavior may cause problems with the default BizTalk Server FILE adapter. For example, the adapter may grab the file AFTER the system creates the empty file, but BEFORE the system attempts to write to it, causing integration issues. Or, in other cases, we may not want to pick up that empty files.

You may also know that the native FILE adapter released with BizTalk grabs and deletes all zero-byte (empty) files without triggering any associated processes. Instead, it logs a warning or error saying that:

The FILE receive adapter deleted the empty file “dir:<folder><file-name>” without performing any processing.

This behavior, according to Microsoft, is by design. Though you can argue that it is not consistent how different adapters treat empty files, e.g., FTP adapter can transfer empty files with no problem.

The File-RADITZ adapter is, kind of, the arch-enemy of the File-Z Adapter. This adapter doesn’t pick up or process empty (zero-byte) files. Instead, it leaves the empty files in the original folder.

I decided to create or recreate this adapter based on Yan Yang scenario just for fun and for mentoring how you can create your custom File adapter by making use of:

  • DotNet File Adapter sample source code that was available on previous versions of the BizTalk Server SDK

This adapter can be used as a Receive Adapter or as a Send Adapter.

Why the name File-RADITZ?

Well, I named the other adapter FILE-Z because it meant to mean File-Zero, but in reality, because I was a massive fan of Dragon Ball Z :). As I mentioned before, this adapter is, kind of, the arch-enemy of the File-Z Adapter. So, making the idea for a better name, I named it based on a memorable Dragon Ball villain: Raditz, that was one of the first enemy encounters in Dragon Ball Z.

How to install it?

If you want to install the BizTalk Server FILE-RADITZ Adapter, you need to:

  • Access the FILE-RADITZ Adapter GitHub page and download the content of the folder executables.
  • Place the folder BizTalkFileRADITZAdapter on any location on your BizTalk Server box. I will recommend to put it in:
    • C:Program Files (x86)
    • Note: if you put in any other path you need to modify the StaticAdapterManagement.reg file
  • Access the folder and double-click on the StaticAdapterManagement.reg file

This will register the adapter to run under 32-bit host instances. To make it available also for 64-bits, you need to:

  • Click Start
  • Type %windir%SysWoW64cmd.exe in the search box and press enter
  • Run the same adapter registry (.reg) file from this command prompt

Now you just need to add this adapter to the BizTalk Server Administration Console:

  • Open BizTalk Administration Console by pressing the Windows key to switch to the Start menu, type BizTalk Server Administration or BizTalk, click the BizTalk Server Administration option from the Search window.
  • In the console left tree, expand BizTalk Server Administration à BizTalk Group à Platform Settings and then Adapters.
  • In the Adapter Properties
    • In the Name box, type a descriptive name for this adapter.
      • FILE-RADITZ
    • In the Adapter combo box, select the adapter from the drop-down that you want to add.
      • FILE-RADITZ
    • In the Description box, type a description for the adapter (this is optional).
      • BizTalk Server File Adapter that is able to process all files except empty (zero byte) files.
  • Click OK to complete the process of adding the adapter.

Where can you use it?

This version available on GitHub is currently compiled on .NET 4.7.2 and optimized for BizTalk Server 2020. Nevertheless, you can take this code and compile it in other versions of .NET and BizTalk Server. It will be 100% compatible.

Download

THIS ADAPTER IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download the BizTalk Server FILE-RADITZ Adapter from GitHub here:

The post FILE-RADITZ Adapter: File adapter that ignores empty (zero bytes) files appeared first on SANDRO PEREIRA BIZTALK BLOG.

December 6, 2021 Weekly Update on Microsoft Integration Platform & Azure iPaaS

December 6, 2021 Weekly Update on Microsoft Integration Platform & Azure iPaaS

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform and Azure iPaaS?

Integration weekly updates can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

Microsoft Announcements and Updates

 

Community Blog Posts

 

Videos

Podcasts

How to get started with iPaaS design & development in Azure?

  • Robust Cloud Integration with Azure
  • Microsoft Azure for Developers: What to Use When
  • Serverless Computing: The Big Picture
  • Azure Logic Apps: Getting Started
  • Azure Logic Apps: Fundamentals
  • Microsoft Azure Developer: Creating Enterprise Logic Apps
  • Microsoft Azure API Management Essentials
  • Azure Functions Fundamentals
  • Cloud Design Patterns for Azure: Availability and Resilience
  • Architecting for High Availability in Microsoft Azure

Feedback

Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

The post December 6, 2021 Weekly Update on Microsoft Integration Platform & Azure iPaaS appeared first on Hooking Stuff Together.

BizTalk Server WCF-Loopback Adapter

BizTalk Server WCF-Loopback Adapter

Unfortunately, some good BizTalk Server resources have disappeared over these years, and they are no longer available. A lot of these resources are custom adapters. One of these adapters was the BizTalk WCF Loopback Adapter initially developed by synthesis consulting (I think), providing back then the source code.

What is the WCF-Loopback Adapter?

The Loopback adapter is simply a two-way send adapter that, in its essence, returns a copy of the outbound message. This capability can be used in several situations or scenarios:

  • One of them is to replace the need to call pipelines inside an orchestration. Calling pipelines inside orchestrations is a good option, but it complicates the logic of the orchestrations with loops and expressions. You’ll also end up repeating the “code” for each message type/orchestration.
  • Atin Agarwal described another example in his blog: Simulating LOB Application using WCF LoopBack Adapter Binding in BizTalk.

The Loopback adapter gives you two opportunities:

  • To execute pipeline and maps on the way out
  • And when it comes back in without the overhead of some repository.

I was recently doing a proof-of-concept at a client, and I needed to invoke a receive pipeline inside the orchestration. That approach worked fine, but it feels like a static black hole that you don’t know is there, and if there is a need to apply any changes, you need to redeploy the orchestration. You cannot simply modify a setting on the pipeline. So, I decide to use the Loopback adapter to recreate the same experience implementing a different approach. Unfortunately, this adapter was no longer available for download, and all the references were wiped out from the face of the Earth. Luckily, I had some old print screens on my vast list of internal resources, and I decided to create a new version of this WCF adapter (king of re-creating) and make this WCF adapter available again.

It was also a good training experience to create a WCF Adapter.

How to install it?

This is a WCF adapter, so installing this adapter is the same as installing any other BizTalk Server WCF extension. And you have two options:

  • Using the machine.config files – This is the most complicated option, but it is the one I prefer to use;;
  • Or using the availability of registring WCF extensions on the receive and send handlers will not explain this approach.

You can choose to register this WCF adapter to run under 32-bit host instances, 64-bit host instances, or of course, both.

To install it, you need to:

  • Download the runtime folder from the WCF-Loopback Adapter GitHub page.
  • Install/register the Microsoft.BizTalk.WCFLoopback.Adapter.Runtime.dll in GAC
  • And add these settings of the machine.config files:
<system.serviceModel>
	<extensions>
		<bindingElementExtensions>
			<add name="WCFLoopbackAdapter" type="Microsoft.BizTalk.WCFLoopback.WCFLoopbackAdapterBindingElementExtension, Microsoft.BizTalk.WCFLoopback.Adapter.Runtime, Version=1.0.0.0, Culture=neutral, PublicKeyToken=5c92120f39ef9d11" />
		</bindingElementExtensions>
		<bindingExtensions>
			<add name="WCFLoopbackAdapterBinding" type="Microsoft.BizTalk.WCFLoopback.WCFLoopbackAdapterBindingCollectionElement, Microsoft.BizTalk.WCFLoopback.Adapter.Runtime, Version=1.0.0.0, Culture=neutral, PublicKeyToken=5c92120f39ef9d11" />
		</bindingExtensions>
	</extensions>
</system.serviceModel>
  • To run under 32-bit host instances
    • Open the machine.config present on the folder: C:WindowsMicrosoft.NETFrameworkv4.0.30319Config
    • I will also recomend, not mandatory, to change also the machine.config present on the folder: C:WindowsMicrosoft.NETFrameworkv2.0.50727CONFIG
  • To run under 64-bit host instances
    • Open the machine.config present on the folder: C:WindowsMicrosoft.NETFramework64v4.0.30319Config
    • I will also recomend, not mandatory, to change also the machine.config present on the folder: C:WindowsMicrosoft.NETFramework64v2.0.50727CONFIG

Restart the services.

Where can you use it?

This version available on GitHub is currently compiled on .NET 4.7.2 and optimized for BizTalk Server 2020. Nevertheless, you can take this code and compile it in other versions of .NET and BizTalk Server. It will be 100% compatible.

Download

THIS ADAPTER IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download the BizTalk Server WCF-Loopback Adapter from GitHub here:

The post BizTalk Server WCF-Loopback Adapter appeared first on SANDRO PEREIRA BIZTALK BLOG.