PowerShell script to download a specific version of WinSCP

PowerShell script to download a specific version of WinSCP

Today I was helping a BizTalk Server customer migrate their process from using the FTP adapter to the SFTP adapter. And if you are familiar with the BizTalk SFTP adapter, you will be aware of the painful process of choosing the correct version of WinSCP and how that hell we need to do to work correctly with the BizTalk Server, which is quite simple in general:

  • Download WinSCP and the .net Library, ensuring you get the correct version!
  • Copy the .exe and .dll to the BizTalk installation folder
  • DO NOT gac anything. If you GAC the .net library, it will not work because it expects WinSCP.exe to be in the same path, so that’s why they both go into the BizTalk installation folder.

However, the biggest issue is: what is the correct WinSCP version I need for my version of BizTalk Server 2016 or 2020?

And for that reason, Thomas E. Canter, on his day as a Phidiax consultant, decided to create this fantastic PowerShell script BizTalk WinSCP Installer, which during the years, has evolved and was improved by several people like Michael Stephenson, Nicolas Blatter, Niclas Öberg and myself.

If your environment has access to the internet, then I will recommend you use that script to install WinSCP! However, my client didn’t have access to the internet from the production server, and to complicate it a little bit, it didn’t have the same Build version. Let’s say in the test environment, it did have BizTalk Server 2016 with Feature Pack 3 and Cumulative Update 9, but in production, we would find BizTalk Server 2016 with Feature Pack 3 and Cumulative Update 5. That means we couldn’t copy the files from other environments to production. We had to use a different WinSCP version.

To address this scenario, I created a simple PowerShell version that you could choose the version you want to download and that you can run on any machine with access to the internet without checking the Build version you have in your environment.

This is a simple abstract of the PowerShell script:

$checkExeExists = Test-Path $targetNugetExe
if(-not $checkExeExists)
{
    if ($PSCmdlet.ShouldProcess("$sourceNugetExe -OutFile $targetNugetExe", "Run Invoke-WebRequest ")) {
        Invoke-WebRequest $sourceNugetExe -OutFile $targetNugetExe
        $targetNugetExeExists = Test-Path $targetNugetExe
        if (-not $targetNugetExeExists) {
            $Continue = $false
            Write-Error "`n$bangString";
            Write-Error "The download of the Nuget EXE from";
            Write-Error $sourceNugetExe;
            Write-Error "did not succeed";
            Write-Error "$bangString";
        }
        else{
            Write-Success "nuget.exe download successfully."
        }
    }
}

if ($PSCmdlet.ShouldProcess("$getWinSCP", "Run Command")) {
    Invoke-Expression "& $getWinSCP";
    $WinSCPEXEExists = Test-Path $WinSCPEXEDownload
    $WinSCPDLLExists = Test-Path $WinSCPDllDownload
    if (-not $WinSCPDLLExists) {
        $Continue = $false
        Write-Error "`n$bangString";
        Write-Error "WinSCP $winSCPVersion was not properly downloaded.";
        Write-Error "Check the folder and error messages above:";
        Write-Error "$nugetDownloadFolder";
        Write-Error "And determine what files did download or did not download.";
        Write-Error "$bangString";
    }
    else{
        Write-Success "WinSCP $winSCPVersion was properly downloaded."
    }
}

THESE POWERSHELL SCRIPTS ARE PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

Where can I download it?

You can download the complete Azure Function source code here:

Hope you find this helpful! So, if you liked the content or found it helpful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego! 

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.

He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira

Join me at INTEGRATE 2023 London | JUNE 5-7, 2023 | BizTalk Server to Azure Integration Services migration

Join me at INTEGRATE 2023 London | JUNE 5-7, 2023 | BizTalk Server to Azure Integration Services migration

INTEGRATE (formerly known as BizTalk Summit) is the primary conference for professionals working in the Microsoft Integration space and once again is back in London. From June 5 to 7, 26 speakers – 12 MVPs, 1 former MVP, and 12 Microsoft PM – will address the entire Microsoft Integration Platform landscape on topics like BizTalk Server, Logic Apps, Service Bus, Event Grid, Event Hubs, Microsoft Flow, API Management, Azure Functions and many more.

And once again, I will be there delivering a session! Since the first BizTalk Summit event, I’ve been a constant presence as a speaker at these events! However, this will be the first time I’m not going to deliver a session 100% dedicated to BizTalk Server. This time I’ll be talking about how you can migrate your BizTalk Server solution to Azure Integration Services.

You could also get an exclusive 15% discount on top of the existing early bird offer using the coupon code “INT2023-SPEAKER-SANDRO”.

About my session

Session Name: BizTalk Server to Azure Integration Services migration

Abstract: If you are embracing the journey to move your current BizTalk Server environment to the cloud, we will discuss what you need to be aware of in this session. From the preparation phase – the assessment phase to the tools and technology you can use, some best practices you may implement, what to do and not to do, and finally, presenting some samples.

Content is not all you get from this event…

Content is hugely important, and as you read previously, this event will address the entire Microsoft Integration Platform landscape… but that’s not all!

This will also be a fantastic opportunity to Network, Connect, and Reconnect with Colleagues. Meet some of the people you have been following on Twitter and blogs and network with them and others who are interested in the same things you are. To hang out with the most brilliant people you know – and I’m not talking about the speakers or Microsoft PMs! I’m talking about you guys! – last year, we had more than 200 attendees from several companies across many countries attending the event, and it was the first personal event after COVID-19, this year will be bigger! Imagine the experience that all of us have combined!

The knowledge and experience of all the attendees, speakers, and product group members at these events are unreal!!! You will not find an opportunity like this every day.

So, this event is a massive opportunity for you to:

  • Get insight and answers to your questions from these real-world experts: attendees, MVPs, and/or Microsoft PMs;
  • Know and become friends with people you are interested in or that you follow on social media and participate in post-event activities like coffee breaks and dinner with speakers and/or other attendees.
  • Build your personal Business Networking, and it is also a good opportunity for Partnerships… and new business opportunities.

And finally: Refresh and recharge and have some fun!

Fill free to contact DevScope at info@devscope.net if you want to know more about us, what we do, and/or how we can help you and your business.

INTEGRATE 2023 Registration

You are still on time to register for the conference here.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.

He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira

BizTalk Backup BizTalk Server Job Error: Could not find server ‘server name’ in sys.servers

BizTalk Backup BizTalk Server Job Error: Could not find server ‘server name’ in sys.servers

Yesterday while troubleshooting a BizTalk Server developer environment at a client, I encountered an unusual error while trying to configure the BizTalk Server Backup job:

Could not find server ‘server name’ in sys.servers. 

Based on the error description, and by the fact that I was trying to run a SQL Server job, I knew that the error should be on the SQL Server side, some incorrect configurations,

Cause

Why this problem start to happen is unclear to me, and at the time I didn’t have all the information available to understand that. However, after investigating a little this error type I realize that we can execute a SQL query to check out what is your linked server:

select * from sys.servers

Or

Select @@SERVERNAME

The problem was that once I run these scripts, I realize the server name was not the expected one, it was incorrect! For example:

  • I was expecting BTS2020LAB01.
  • But instead, I was seeing VMIMAGE01.

And that was causing the failure on the Backup job and other BizTalk server jobs.

Solution

To solve this issue we need to fix that server name and for that, we can apply the following SQL script:

sp_dropserver 'VMIMAGE01'
GO
sp_addserver  'BTS2020LAB01',local
GO

After applying this script, make sure you restart the SQL Server service.

Once you have done these steps, you can successfully start your BizTalk Backup job.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.

He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira

BizTalk Health Monitor Dashboards Customization: Monitoring BizTalk Host Instances Status

BizTalk Health Monitor Dashboards Customization: Monitoring BizTalk Host Instances Status

Have you noticed that the default BizTalk Health Monitor Dashboard doesn’t monitor/report the status of the BizTalk Server Host Instances?

A few weeks ago, while delivering a BizTalk Server training course, I was presenting and playing around with the BizTalk Health Monitor with my “students” and explaining the importance of actively monitoring your BizTalk Server platform. Because my developer environment is a machine that doesn’t have many errors and is properly configured, like a production environment, it is difficult to show the BizTalk Health Monitor presenting problems, so I decided to stop some of the Host Instances! It is an easy way to “emulate” a problem:

However, when I ran the BizTalk Health Monitor, I realized the Host Instance tile was green!

Notice also that the information provided states that I have 10 Host Instances, and only 8 were running. I was surprised by that behavior. I confirmed with the Microsoft Engineering team, and this is not a bug. Actually, by default, the Host Instances dashboard tile is NOT supposed to report a warning if some Host Instances are stopped. this tile reports, in fact, what MSFT has in the category “BizTalk Host Instance” of the KPI view:

Each default Dashboard tile normally reports the content of one or more categories of different views (Warnings, Summary, Key indicators, Queries Output…); However, the choice of content of these tiles cannot be changed by us.

Now the main question is: Can we put the BizTalk Server Health Monitor “watching” the status of the Host Instances and raising some alerts?

Luckily for us, the answer is yes, and it is not that difficult. The tool allows us to:

  • Add custom rules that will allow us to query the environment.
  • And add our own custom tiles in our profile Dashboard view.

That also means that each profile can have there own monitoring customizations.

To create some custom rules to monitor the status of the Host Instances, assuming that you have the BizTalk Health monitor tool open, we need to:

  • Right-click over the Default profile, and select the Profile settings… option from the context menu.
  • From the Default profile – Profile settings window, select the Rules tab and then click on New rule.
  • From the New Rule – Select Query window, expand the Important target category, then select the BizTalk Host Instances sub-category, and click Ok.
  • On the New Rule (Query: BizTalk Host Instances) window, select the My Rule option from the left tree and:
    • On the Caption property, give a name to the rule: Stopped Host Instances.
    • On the Comment property, provide a small description: Monitoring BizTalk Server Host Instances status.
    • On the Trigger Actions panel, select the Each time a row validated all the Rule conditions option.
    • And click Commit changes.
  • On the New Rule (Query: BizTalk Host Instances) window, select the Condition 1 option from the left tree and:
    • On the Column to Check property, leave the default value: %GLOBALPROP_REPORTVALUE:Running%.
    • On the Operator property, from dropbox, select the option: IS DIFFERENT OF.
    • On the Comparison value property, type Yes.
    • And click Commit changes.
  • On the New Rule (Query: BizTalk Host Instances) window, on the Condition option from the left tree, right-click and select the option New Condition:
  • On the New Rule (Query: BizTalk Host Instances) window, select the Condition 2 option from the left tree and:
    • On the Column to Check property, leave the default value: %GLOBALPROP_REPORTVALUE:Running%.
    • On the Operator property, from dropbox, select the option: IS DIFFERENT OF.
    • On the Comparison value property, type Not Applicable.
    • And click Commit changes.
  • On the New Rule (Query: BizTalk Host Instances) window, select the Add Summary or Warning Entry option from the left tree under the Actions option and:
    • On the Category property, type: Host Instances.
    • On the Severity property dropbox, select the Red Warning option.
    • On the Caption property, type: Host Instances Status.
    • On the Value property, type: Is %GLOBALPROP_REPORTVALUE:Name% running: %GLOBALPROP_REPORTVALUE:Running%
    • And click Commit changes.
  • Finally, click Ok.

You can always try the custom rule by clicking Test.

This rule will gather information about Host Instances that are not running. Now we are going to create another rule to gather information about Host Instances that are running. To do that, we need to:

  • From the Default profile – Profile settings window, select the Rules tab and then click on New rule.
  • From the New Rule – Select Query window, expand the Important target category, then select the BizTalk Host Instances sub-category, and click Ok.
  • On the New Rule (Query: BizTalk Host Instances) window, select the My Rule option from the left tree and:
    • On the Caption property, give a name to the rule: Running Host Instances.
    • On the Comment property, provide a small description: Monitor Running Host Instances status.
    • On the Trigger Actions panel, select the Each time a row validated all the Rule conditions option.
    • And click Commit changes.
  • On the New Rule (Query: BizTalk Host Instances) window, select the Condition 1 option from the left tree and:
    • On the Column to Check property, leave the default value: %GLOBALPROP_REPORTVALUE:Running%.
    • On the Operator property, from dropbox, select the option: IS EQUAL TO.
    • On the Comparison value property, type Yes.
    • And click Commit changes.
  • On the New Rule (Query: BizTalk Host Instances) window, select the Add Summary or Warning Entry option from the left tree under the Actions option and:
    • On the Category property, type: Host Instances.
    • On the Severity property dropbox, select the Information option.
    • On the Caption property, type: Host Instances Status.
    • On the Value property, type: Is %GLOBALPROP_REPORTVALUE:Name% running: %GLOBALPROP_REPORTVALUE:Running%
    • And click Commit changes.
  • Finally, click Ok.

Make sure that the two custom rules are selected, and then perform another analysis of your platform.

Now, what we need to do is to create a custom tile to pin to our dashboard. A custom tile can be indeed created easily from any entry or category of the Warning view, Summary view, Key indicators view, or query output view. And to do that, we need to:

  • After analyzing our BizTalk Server environment, expand the report and then select the Summary option.
  • On the Summary report page, scroll down until you find the Host Instances summary, right-click on Host Instances and select the option Pin to the dashboard and then .
  • A new window will appear, saying that a new item was added to the dashboard. Click Ok.
  • If we now click on the Default profile, we will see that the Favorite tile was added to the dashboard.
  • We can customize the name of that tile by right-clicking and selecting the Edit option.
  • On the Favorite Tile – Favorite window:
    • On the Caption property, type: Host Instances Status.
    • On the Comment property, type: Host Instances Status.
    • And click Ok.

And finally, test it by doing another analysis of the environment.

How amazing is this!

Thanks to all that helped me document this feature. You know how you are!

Hope you find this useful! So, if you liked the content or found it useful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego! 

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.

He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira

Microsoft Integration Trends 2023 Webinar | March 2, 9 and 10 | Online

Microsoft Integration Trends 2023 Webinar | March 2, 9 and 10 | Online

Just less than one day to go for the webinar series on accelerating BizTalk Server to Azure Integration Services migration!

The first session will occur on March 2 (tomorrow) from 10 – 11AM GMT, hosted by Kent Weare, about Microsoft offerings to accelerate BizTalk to Azure migration.

  • Do you wonder what Microsoft offers to advance BizTalk to Azure migration? Kent Weare, Principal Product Manager at Microsoft, will highlight the key resources in this session.

The second session will be with me, which will occur on March 9 from 10 – 11AM GMT, about migration from BizTalk Server to Azure Integration Services.

  • How do Enterprises deal with BizTalk to Azure migration? Sandro Pereira, Head of Integration at DevScope, will share the migration strategies and best practices acquired from his experience with multiple Global Enterprises!

And finally, the third and last session will occur on March 10 from 10 – 11AM GMT, hosted by Michael Stephenson and Lex Hegt, about Addressing the Operational challenges in BizTalk Server to Azure Integration Services migration.

  • Michael Stephenson, Coach and Consultant, Microsoft Azure Adoption, Connected Systems Consulting Ltd, and Lex Hegt, Lead Product Consultant at BizTalk360 & Serverless360, will educate us on the Operations perspective of BizTalk Server to Azure migration highlighting the importance of defining a support strategy using reliable solutions available.

Hurry Up & Save your spot now, the webinar is free, and if you are in the Enterprise Integration space, you don’t want to miss it.

Key takeaways from the webinar series

  • Microsoft offerings to advance the BizTalk to Azure migration.
  • Strategy options and planning considerations for migrations.
  • Best practices to accelerate BizTalk to Azure migration.
  • Operations aspects of the migration from BizTalk to Azure.

Join us in this webinar series for exclusive expert advice on BizTalk to Azure migration!

Link for the registration

You can register for this webinar series here:

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.

He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira

BizTalk Server to Azure Integration Services – Ask the Experts episode at Kent Weare Youtube Channel

BizTalk Server to Azure Integration Services – Ask the Experts episode at Kent Weare Youtube Channel

Recently I announced that I had the honor to be invited by the Microsoft Logic App product group to contribute to the BizTalk Server Migration to Azure Integration Services Architecture Guidance documentation alongside my fellow MVPs: Michael Stephenson and Stephen W Thomas. 

This documentation is intended for clients considering or having already decided to move their entire BizTalk Server on-premises integration solution to Azure or parts of the solution into Azure, making some hybrid solutions and helping them with this process.

Now I had, once again, the honor of being invited a few weeks ago by my friend – and now Microsoft Principal Product Manager – Azure Logic Apps – Kent Weare to record a special episode on BizTalk Server to Azure Integration Services – Ask the Experts on his YouTube channel.

In this episode, we are going to discuss some important questions and concerns customers may have on this journey to migrate their BizTalk Server Solutions to Azure, like:

  • What are some examples of BizTalk migrations that I have been involved in? What is the biggest driver for these customers?
  • If I’m helping a customer migrate, what advice would I provide to the customer?
  • When considering BizTalk architectures, what is an anti-pattern or something that customers should avoid/re-think as they move to Azure?
  • And many other questions.

You can see the full episode here: https://www.youtube.com/watch?v=iYLFsUK5AmY

I hope you enjoy and find this topic interesting. Let me know what you think.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.

He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira

BizTalk Server Migration to Azure Integration Services Architecture Guidance

BizTalk Server Migration to Azure Integration Services Architecture Guidance

I was honored to be invited by the Microsoft Logic App product group to contribute to the BizTalk Server Migration to Azure Integration Services Architecture Guidance documentation alongside my fellow MVPs: Michael Stephenson and Stephen W Thomas. 

This guideline is focused on clients considering or having already decided to move their entire BizTalk Server on-premises integration solution to Azure or parts of the solution into Azure, therefore, making some hybrid solutions. Of course, in this process, many questions about best practices to use, what services to use, and others will be raised.

This guide provides an overview of the reasons and benefits, product comparisons, capabilities, and other information to help you start migrating from on-premises BizTalk Server solutions to cloud-based Azure Integration Services. Following this guide, you’ll find more guides that cover how to choose the services that best fit your scenario, along with migration strategies, planning considerations, and best practices to help you deliver successful results:

This is a simple feature matchup diagram that you will find in the guide:

If you want to know more about it, the architecture guide is available here: BizTalk Server Migration to Azure Integration Services Architecture Guidance.

See the original Microsoft announcement here: Announcement: BizTalk Server Migration to Azure Integration Services Architecture Guidance.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.

He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira

CSV Structure Validation Pipeline Component

CSV Structure Validation Pipeline Component

I’m back to another blog post about BizTalk Server! I know that my latest post has been about Azure Logic Apps, and you may count on seeing many more in the future. And the reason is that I work both on-premise with BizTalk Server and in the cloud with Azure Integration Services… but relax, I will continue to post many things about BizTalk Server. BizTalk Server is not dead. It is well alive, contrary to what many think!

Today I’m returning to one of my pet projects: the BizTalk Pipeline Components Extensions Utility Pack project that is available on GitHub!

For those who aren’t familiar with it, the BizTalk Pipeline Components Extensions Utility Pack project is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in receive and sent pipelines. Those pipeline components provide extensions of BizTalk’s out-of-the-box pipeline capabilities.

CSV Structure Validation Pipeline Component

The CSV Structure Validation Pipeline Component is a pipeline component that can be used to validate the structure of a basic CSV or flat file component before being converted to an XML message. Of course, this same strategy can be used in more complex scenarios.

This is the list of properties that you can set up on the CSV Structure Validation pipeline component:

Property Name Description Sample Values
DelimiterChar Define what is the delimiter char inside a line. ;
NumberFieldsPerLine Number of fields expected per line 3

Note that this component takes as granted that the line delimiter is the CRLF (Carriage ReturnLine Feed).

If we take this example:

one;two;t
1;2;2

Then we need to configure the port in the following way:

If we receive an invalid file, then the component will raise an error suspending the message in the BizTalk Server Administration Console. For example, with the following error message:

  • Invalid format data in the document. Line number 3 has 2 fields, and it should be expected 3 fields

If you are wondering why create a Pipeline component to validate the structure of the CSV or flat-file document? Can we use instead the Flat-File Schema to do this structure validation?

And the answer is Yes and No! In many cases and with many requirements, we don’t need to create a custom pipeline component. Using a Flat-File schema can be used to address the goals but in other scenarios doing a CSV Validation with the Flat-File schema is not enough. However, I will leave that “discussion” to my next BizTalk Server Best practices, tips and tricks.

How to install it

As always, you just need to add these DLLs on the Pipeline Components folder that in BizTalk Server 2020 is by default:

  • C:Program Files (x86)Microsoft BizTalk ServerPipeline Components

In this particular component, we need to have this  DLL:

  • BizTalk.PipelineComponent.CSVFlatFileStructureValidation.dll

How to use it

Like all previous, to use the pipeline component, I recommend you to create generic or several generic pipelines that can be reused by all your applications and add the Message Archive Pipeline Component in the stage you desire. The component can be used in a stage of the receive and send pipelines.

Download

THIS COMPONENT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download CSV Structure Validation Pipeline Component from GitHub here:

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.

He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira

What makes BizTalk360 the best BizTalk Server Monitoring tool?

What makes BizTalk360 the best BizTalk Server Monitoring tool?

Key takeaways from this blog

  • Introduction
  • BizTalk Server Monitoring tools
  • Azure Serverless Monitoring
  • Pricing Structure of BizTalk360
  • If everything is a peaceful and relaxing environment, then something is wrong

Introduction to BizTalk360 

BizTalk360 has been around for more than a decade in the field of BizTalk Server Administration and Monitoring. This blog shows users how they can streamline their day-to-day monitoring. Besides providing full coverage of your BizTalk Server Monitoring, BizTalk360 also fulfils many other functions. Let us see this in detail.

BizTalk Server Monitoring tools

In this section, you can get a deeper understanding of what you can monitor with BizTalk360.

  • Application and artifacts monitoring
  • Host Instance Monitoring
  • Host Throttling
  • EDI (Electronic Data Interchange) Agreement Monitoring
  • Database size
  • Event Log Monitoring
  • Event Log Data Monitoring
  • File Location Monitoring
  • Queue Monitoring
  • BizTalk & SQL Server Monitoring
  • SQL Job Monitoring

Application and artifacts monitoring

BizTalk360 automatically shows the applications that are deployed in the BizTalk Server environment at hand. All state-based applications artifacts can be monitored, including:

  • Orchestrations
  • Receive locations
  • Send ports
  • Send port groups

Users can set the expected state of those artifacts. When there is a mismatch between the actual and the expected state, BizTalk360 sends you an email, or a notification to any of the systems such as:

  • Microsoft Teams
  • Slack
  • ServiceNow
  • PowerShell
  • Webhook
  • Twilio
  • PagerDuty

Host Instance Monitoring

In BizTalk, messages can only be processed when the host instance is running. BizTalk360 allows users to manage and monitor BizTalk Host Instances. Additionally, BizTalk360 can also monitor clustered host instances. To learn more about clustered host instances, follow the link.

Host Throttling

Microsoft does not provide any tool to understand whether your BizTalk environment is working efficiently or under throttling conditions. It is typical for administrators to use Windows Perfmon to add all the performance counters associated with throttling and then analyse throttling conditions. This requires extensive knowledge of how the BizTalk Server works, various throttling counters & conditions, whether it is running at an optimum level, etc.

Setting up Host Throttling monitoring in BizTalk360 is just a 5-minute process, where users can monitor the following:

  • Rate throttling
  • Process memory
  • System memory
  • Database size
  • Database session
  • Thread count
  • User override

EDI agreement monitoring

Any business trading partners commonly exchange data electronically through EDI. For successful transactions, it is important to maintain the agreement state. Via BizTalk360, users can monitor the states of EDI Agreements.


Database size

For a healthy BizTalk environment, it is essential to monitor the growth of the BizTalk databases. With the Database size monitoring feature, users can monitor the database and log the file size of the BizTalk and BizTalk360 databases. Follow the link to learn more!

Event log monitoring

What happens when your Windows server gets booted without your knowledge? It may be a good idea for users to check if everything is still in good shape. Without a monitoring product, it requires manual intervention to be aware of the reboot. Wouldn’t it be more convenient if there was a tool to monitor your Event log entries?

BizTalk360 comes with a feature named Event log monitoring that enables users to be alerted about specific events being written to event logs in your BizTalk or SQL servers.

There is also another type of monitoring, which is Event Log Data Monitoring where users can configure a specific value based on conditions. Follow the link to know more.

Before moving on to the next topic, it is helpful to know the difference between Event Log and Event Log Data Monitoring. Let us say you have multiple BizTalk servers and want to correlate and monitor event IDs. In such a scenario, Event Log Data Monitoring will be helpful whereas, in Event Log Monitoring, you need to create two event logs to monitor different event IDs.

Event Log Data Monitoring

BizTalk administrators look first for the Event Logs when something unexpected happens since the Event Viewer contains the exact error information. Logging in each time to the BizTalk and SQL servers to check the event logs would be cumbersome. When BizTalk360 event log data monitoring is enabled, it is easy to check for event logs. You can get the log message count as well as event log message information in the alert email.

Wouldn’t it be helpful if we see all this information in one place? Yes, BizTalk360 provides you with this option where data can be filtered with more options and conditions. An additional filtering option is that the event logs can be filtered based on multiple event ids.

File Location monitoring

BizTalk360 lists all the locations configured in the BizTalk artifacts (Send Ports and Receive Locations) for the Transport types (File, FTP, SFTP) respectively, which helps users to monitor all the File Locations mapped with Receive Locations/Send Ports.

Queue Monitoring

BizTalk360 monitors the health of the IBMMQ and MSMQ along with other parameters, including Queue Size, Active Messages, Dead Letter messages, and Journal Messages. Monitor your local and remote queues and set threshold conditions to get notified of any violations.

SQL Jobs Monitoring

BizTalk Server heavily relies on SQL Server databases. The health of these databases is crucial for the health of the BizTalk Server. Out of the box, BizTalk Server comes with several SQL Server jobs that perform all kinds of maintenance tasks, thereby contributing to the health of those databases. BizTalk360 allows you to monitor those SQL Jobs.

BizTalk & SQL Server monitoring

This feature helps the users to monitor the availability of

  • Disk space
  • CPU and Memory usage
  • Event Log entries
  • Windows NT Services

Azure Serverless Monitoring

To support hybrid integrations, BizTalk360 becomes aligned by supporting Azure Serverless technologies. Hereby, the product prevents you from switching between BizTalk360 and the Azure portal. Let’s see what Azure Services can be monitored by BizTalk360:

  • Azure Logic Apps
  • Azure API (Application Programming Interface) Apps

Azure Logic Apps

With BizTalk360, Logic Apps can be monitored and ensure they are running correctly. You will gain deeper insight into when the expected number of runs is met and the number of failures that occur.

Azure API apps

Users can monitor the health of their API apps and the configured endpoints. Monitoring API apps includes endpoints and provides a seamless monitoring experience.

Azure Service Bus Queue monitoring

Azure Service Bus allows applications and services to communicate with other applications in the cloud, on-premises, or both and ensures message exchange occurs smoothly. With BizTalk360, configure alarms for various parameters like

  • Queue Size
  • Status
  • Messages
  • Active Messages
  • Scheduled Messages
  • Transfer Messages
  • Dead letter Messages

Auto Correct

Administrators can set up monitoring on “State-based” artifacts and let the monitoring service automatically heal the artifact at any time. For example, administrators can monitor the receive locations of applications and set up auto correct functionality for the Expected state of the artifact. Whenever the receive location goes down or gets disabled, there will be a mismatch and the auto correct will try to bring the artifact back to the Expected state within the next monitoring service cycle. The feature applies to the following artifacts:

  • Send ports
  • Send port groups
  • Receive locations
  • Orchestrations
  • Host Instances
  • NT Services
  • SQL jobs
  • Logic Apps
  • EDI Agreements

Pricing Structure of BizTalk360

There are one-year and three-year subscription options available for BizTalk360. Depending on the number of BizTalk servers in the BizTalk group, the BizTalk edition (Standard or Enterprise), and the BizTalk360 feature-based license tier (Silver, Gold, Platinum), users will pay a different license fee. A license comes with unlimited users and unlimited customer support, also users can customize their license according to business requirements. A free trial is also available.

If everything is a peaceful and relaxing environment, then something is wrong

The BizTalk360 team is always there for customers when they need Product support. They frequently reach out to each customer, to update them on the latest release, and to make sure they are using the product to the fullest.

Ask for Support

When setting up BizTalk360 monitoring, you are not alone. Whenever you have a problem, the BizTalk360 technical support team is there to help. They are knowledgeable about the product and can share their experiences with other customers as well. Directly ask them to review your setup before it fails, and you find out it was set up incorrectly.

Documentation Help

It is crucial to document exactly how users have set up their business requirements and assist team members in the future. In BizTalk360, users can easily locate any information they need through the documentation portal.

Demo & Free Trial

Currently the product contains over 80 features, but Product Consultant team of BizTalk360 provides a fully customized product demo based on business needs, so please do contact the support team without any obligation. Users can either request a demo or take a free trial.

  • Introduction
  • BizTalk Server Monitoring tools
  • Azure Serverless Monitoring
  • Pricing Structure of BizTalk360
  • If everything is a peaceful and relaxing environment, then something is wrong
Building BizTalk Solution Error: Couldn’t process file  due to its being in the Internet or Restricted zone or having the mark of the web on the file.

Building BizTalk Solution Error: Couldn’t process file due to its being in the Internet or Restricted zone or having the mark of the web on the file.

One error never comes alone! Following the error reported in my last blog post while working last week with one of my clients, I was able to catch an error that I never saw during these long years working with BizTalk Server while trying to build a BizTalk Server Visual Studio solution in this specific case a custom pipeline component:

Couldn’t process file XMLAttributesStripper.resx due to its being in the Internet or Restricted zone or having the mark of the web on the file. Remove the mark of the web if you want to process these files.??????????????????

Indeed I download that resource from the Internet, from my GitHub page, like I do thousands of times for many clients and projects!

Cause

This issue happens due to the fact that you downloaded from the web these files/resources from a machine with security restrictions configured. So, when Visual Studio attempts to build the project, this error occurs because The .NET Framework resource compiler honors this marker and refuses to compile those resource files for security reasons.

The underlying cause is that the respective resource file has the so-called mark of the web applied to it. This is a marker that browsers place on downloaded files so that other applications can make informed decisions on whether to trust that file or not.

Solution

To fix this issue, the solution is quite simple. Nevertheless, there are many ways to solve or avoid this issue.

Solution 1: Fix the issue

To solve this issue, we need to remove the mark of the web, to do that, we need to:

  • Right-click on the file in windows explorer and select Properties.
  • On the General tab, at the bottom under Security, there is a check box to remove mark of the web.
  • Unselect the Unblock check box and click OK.

Note: This needs to be done with Visual Studio closed.

Solution 2: Fix the issue with PowerShell script

We can also do the same functionality as Solution 1 using the following PowerShell script:

  • On the folder for the project, run the following script
dir -Path . -Recurse | Unblock-File
  • or
Get-ChildItem -Path . -Recurse | Unblock-File

Solution 3: Fix the issue from Visual Studio

Didn’t try this approach, but apparently, we can also fix this issue directly from Visual Studio by:

  • Select the menu option Tools > Options
  • From the Options windows, select the option Trust Settings under Environment and add the project path as a trusted path.

Credits to Cody J. Mathis

Solution 4: Avoid this issue from happening again

Didn’t try this approach, but apparently, we can also fix this issue directly from Visual Studio by:

  • Open Internet Explorer and click on the gear in the top right corner.
  • Click Internet Options. and select the Security tab.
  • Click Trusted Sites and then on Sites button.
  • Enter the URL of the site you want to trust, in this case, GitHub, and click Add.
  • Click OK.