by Gautam | Oct 14, 2018 | BizTalk Community Blogs via Syndication
Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?
Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.
If you want to receive these updates weekly, then don’t forget to Subscribe!
Feedback
Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.
by Raja Sreenivasan | Oct 11, 2018 | BizTalk Community Blogs via Syndication
Introduction
SFTP (SSH File Transfer Protocol also known as Secure File Transfer Protocol) is a secure file transferring protocol between two remote systems, which runs over Secure Shell protocol (SSH). It provides strong authentication and secure encrypted data communication between two computers, which are connecting over an insecure network. It was designed by the Internet Engineering Task Force (IETF) as a secure extension. SSH provides the secure file transfer capabilities.
In this article, we will explain how to configure SFTP, how to use it with BizTalk Server and how you can set up monitoring SFTP using BizTalk360.
Contents
- How to Configure SFTP
- Types of authentication available in SFTP
- Using SFTP in BizTalk Server
- Monitoring SFTP using BizTalk360 Application
How to Configure SFTP
SFTP has replaced legacy FTP (File Transfer Protocol) and FTP/S and it provides all the functionality offered by these protocols, but the protocol is more secure and reliable. Also, configuration is easier.
Following are the steps to configure SFTP:
- Download the OpenSSH for server using OpenSSH for Windows binaries (Packages OpenSSH-Win64.zip or OpenSSH-Win32.zip)
Link: https://github.com/PowerShell/Win32-OpenSSH/releases
- Extract the package in folder location ‘C:Program Files’ as an administrator and install the SSH and SSHD services using the following command:
exe -Execution Policy Bypass -File install-sshd.psl
- Once you have run the above command, the SSH server and server agent will install the system and start the service in the services.msc
The SFTP port number for the SSH port is 22, basically just an SSH Server. Once the user has logged in to the server using SSH, the SFTP protocol can be initiated. There is no separate SFTP port exposed on the servers. There is also no need to configure another rule into the firewalls.
Once the command is executed in PowerShell, the rule is created in the firewall section.
Using Public Keys for SSH Authentication
One effective way of securing SSH access to the server, is to use a Public/Private Key pair. This means that the generated key pair, consists of a public key (allowed to know) and a private key (keep secret and don’t give to anybody). The private key can generate a Signature and cannot be forged for anybody who doesn’t have that key. But using the public key, anybody can verify that a signature is genuine. The public key is placed on the server and a private key is placed on local workstation. Using a key pair, it becomes impossible for someone to log in by using just a password, in case you have set up SSH to deny password-based authentication.
Create the .ssh directory in a local folder and create a file named as “authorized_keys”, where we store the public key for authentication.
Generating Keys
PuTTYgen is a key generator. It generates pairs of public and private keys. When you run the PuTTYgen, you will see a window where you have two choices:
- Generate – to generate a new Public/Private key pair
- Load – to load an existing private key
Before generating a key pair, using PuTTYgen, you need to select which type of key you need.
PuTTYgen currently supports the following type of keys:
- An RSA key for use with the SSH-1 protocol
- An RSA key for use with the SSH-2 protocol
- A DSA key for use with the SSH-2 protocol
- An ECDSA (Elliptic Curve DSA) key for use with the SSH-2 protocol
- An Ed25519 key (another elliptic curve algorithm) for use with the SSH-2 protocol
Here, we will generate a RSA key, for use with the SSH-1 Protocol.
- Download the PuTTYgen from the web site
- Launch the program and click “Generate” button. The program generates the keys for you
- Once you click the Generate button, you must generate some randomness, by moving the mouse over the blank area
- Enter the unique Key passphrase and Confirm passphrase fields
- Save the public and private keys, by clicking Save Public Key and Save Private Key buttons
- From the Public Key, for pasting it into the OpenSSH authorized_keys file field at the top of the window, copy all the text (starting with ssh-rsa). The copied key must be pasted either into the public key tool, in the Control Panel, or directly into the authorized keys file on your server.
Using SFTP Adapter in BizTalk Server
BizTalk Server provides the SFTP adapter to send and receive a file from a secure FTP server using the SSH file transfer protocol. Let’s see how can configure the SFTP adapter for receiving and sending a file from the secure server.
- In the BizTalk Admin Console, create a SFTP Receive Port in the BizTalk application where you want to have it
- Create a Receive Location within that Receive Port
- Select the Transport Type as SFTP from the drop-down list
In the Properties section, configure the following steps:
Others
- Connection Limit – Specify the maximum number of concurrent connections that can be opened to the server
Polling
- Polling Interval – Specify the interval at which the adapter polls the server. To poll continuously, set this value to zero
Default Value: 5
- Unit – Specifies the unit in which the polling interval is specified. For example: Seconds, Minutes, Hours or Days
Security
- Accept Any SSH Server Host key – When the option is set as True, SSH will accept the connection from the host server; when it is set as False, the Receive Location uses the fingerprint of the server for authentication. For the authentication, you need to provide the finger print in the SSHServerHostKeyFingerPrint field.
There are three client authentication methods:
- Password
- PublicKeyAuthentication
- MultiFactorAuthentication
Password authentication mode is simply providing the password in the console for authenticating the client. For the PublicKeyAuthentication, you must provide the private key file in the PrivateKey field and provide the passphrase in the PrivateKeyPassword for authenticating.
For MultiFactorAuthentication, the user must provide the user name, password and Privatekey. If the private key is protected by a passphrase, you also need to provide that in the privatekeyPassword field.
- Password – Specify the password, if you have set the ClientAuthentication mode to password
- Private Key – Specify the private key for the SFTP user, if you have set the ClientAuthenticationMode to Publickeyauthentication
- Private Key Password – Specify the passphrase key to validate the private key
- SSH Server Host Key Fingerprint – It specifies the fingerprint of the public host key for the SSH server
- Username – Specifies a username to log on to the SFTP server
SSH Server
- File Mask – Specifies the file mask to use when retrieving files from a secure SFTP server
- Folder path – Specifies the folder path on the secure SFTP server from where the Receive Location can retrieve files
- Port – Specifies the port address for the secure SFTP server on which the file transfer takes place
- Server Address – Specifies the server name or IP address of the secure SFTP server
Configuring the Send Port
For Configuring the Send Port, create a Send Port or double click an existing send port to modify it in an application in the BizTalk Administration Console.
- On the General Tab, Choose the type of SFTP artifact in the transport section and click the configuration button.
- In the SFTP Transport Properties window configure the following options based on requirement
Others
- Connection Limit – Maximum number of concurrent connections that can be opened to the server
- Log – Creating client-side log file to trouble shoot any errors. Enter the full path for creating the log file and its available from BizTalk Server 2016
- Temporary Folder – A temporary folder on the SFTP server to upload large files before automatically moved to the required location on the same server and its available from BizTalk server 2013 R2
Proxy
- Address –Specifies either DNS or IP Address of the Proxy server
- Password –Specifies the Password of the proxy server
- Port – Specifies the port of the Proxy Server
- Type – Specifies the protocol used by the proxy server
- User Name – Specifies the User Name of the Proxy server
Security
- Access Any SSH Server Host Key – When True, the send port accept any SSH Public host key from the server and if set as false, the port matches the host key with the key specified in the SSHServerHostKey
- Client Authentication Mode – Specifies the authentication method that the send port uses for authenticating the client to the SSH Server.
Three modes of authentication
- Password – If set as Password , you must provide the password in the Password Property
- PublicKeyAuthentication – if set as PublicKeyAuthentication, you must provide the private key of the user in the PrivateKey
- MultiFactorAuthentication – if set as MultiFactorAuthentication, you must provide UserName with its Password. If the private key is protected by a password, provide the password in the PrivateKeyPassword as well
- EncryptionCipher – Provide the kind of encryption cipher and available from BizTalk Server 2013 R2. Options are Auto, AES and TripleDES in the BizTalk Server 2013 R2 and for the BizTalk Server 2016 Auto, AES, Arcfour, Blowfish, TripleDES, and
- Password – Specify the SFTP user password if you set the ClientAuthenticationMode to Password
- Private Key – Specify the private key for the SFTP user if you set the ClientAuthenticationMode to PublicKeyAuthentication
- Private Key Password – Specify a private key password, if required for the key specified in the PrivateKey
- SSH Server Host Key Finger Print – Specifies the fingerprint of the server used by the adapter to authenticate the server if the AccessAnySSHServerHostKey property is set to False. If the fingerprints do not match, the connection fails.
- User Name – Specifies the username for the secure FTP Server
SSH Server
- Append If Exist – if the file being transferred to the secure FTP server already exists at the destination, this property specifies whether the data from the file being transferred should be appended to the existing file. If set to True, the data is appended. If set to False, the file at the destination server is overwritten
- Folder Path – Specifies the folder path on the secure FTP server where the file is copied
- Port – Specifies the port address for the secure FTP server on which the file transfer takes place
- Server Address – Specifies the server name or IP address of the secure FTP server
- Target File Name – Specifies the name with which the file is transferred to the secure FTP server. You can also use macros for the target file name
- Click Apply and OK again to save settings
Monitor the SFTP Location using BizTalk360
From the v8.4, under File Location in the Monitoring section, BizTalk360 has the capability to monitor SFTP servers. File Location Monitoring will list all the locations configured in the BizTalk artifacts (Send Ports and Receive Locations) for the SFTP Transport type. This helps users to easily monitor all the SFTP locations mapped within the Receive Locations/Send Ports.
It contains four sections:
- SSH Server Section has the details about the SFTP Location
- The Proxy Details Section is optional to connect to a SFTP Server behind a firewall
Note: In BizTalk, Proxy details are available from BizTalk 2013 R2
- Security Details Section has the authentication details
- In the SFTP Monitoring Config Section, you can configure the monitor with threshold conditions for the metric File Count
Based on the need, you can monitor the location with threshold conditions. If the specific condition is met, the user gets notified through an email, a SMS or another communication channel.
For monitoring the SFTP server, BizTalk360 uses the third-party tool nSoftware. Using the nSoftware IPWorks SSH product, BizTalk360 connects to the secure server with Private Keys and password for monitoring the location.
For monitoring the SFTP in BizTalk360 you can refer the knowledge base in this link.
See below, some code snippets for connecting to the secure server using nSoftware.
Password Authentication
sftp.SSHUser = “test”;
sftp.SSHPassword = “password”;
sftp.SSHPort = 22;
sftp.SSHHost = “SSHHost”;
sftp.Config(“SSHAcceptServerHostKeyFingerPrint=6a:d3:65:96:d1:9f:9d:f9:57:4e:6b:3b:11:57:5a:15”);
sftp.SSHLogon(sftp.SSHHost, sftp.SSHPort);
Console.WriteLine(“Authenticated”);
sftp.SSHLogoff();
Public key Authentication
sftp.SSHUser = "test";
sftp.SSHCert = new Certificate(CertStoreTypes.cstPPKKeyFile, "....filesserver_cert.pem", "test", "*");
sftp.SSHAuthMode = SftpSSHAuthModes.amPublicKey;
sftp.SSHPort = 22;
sftp.SSHHost = "SSHHost";
sftp.Config("SSHAcceptServerHostKeyFingerPrint=6a:d3:65:96:d1:9f:9d:f9:57:4e:6b:3b:11:57:5a:15");
sftp.SSHLogon(sftp.SSHHost, sftp.SSHPort);
Console.WriteLine("Authenticated");
sftp.SSHLogoff()
Conclusion
This article demonstrates the creation of a SFTP Server. Using the SFTP server in BizTalk Receive Locations and Send Ports, you can send files securely and monitor the SFTP server using BizTalk360.
If you have any feedback or suggestions, please write to us at [email protected].
by Sandro Pereira | Oct 9, 2018 | BizTalk Community Blogs via Syndication
And yes, this is just another “Login failed for user” SQL Server WCF-Adapter related error. In the past I wrote about a similar topic BizTalk WCF-SQL Error:
This time the error message, the cause, and the solution are slightly different. This time while trying to communicate to a brand-new SQL Server server/database to insert date on a table thru BizTalk WCF-SQL adapter I got the following error:
Microsoft.ServiceModel.Channels.Common.ConnectionException: Login failed for user ‘BTSHostSrvc’. —> System.Data.SqlClient.SqlException: Login failed for user ‘BTSHostSrvc’.
at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, DbConnectionPool pool, String accessToken, Boolean applyTransientFaultHandling, SqlAuthenticationProviderManager sqlAuthProviderManager)
at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection.
In the event viewer the message is pretty much the same:
A message sent to adapter “WCF-Custom” on send port “STAGING_BULK_SQL_WCf_SEND” with URI “mssql://SQLSRV/ /ESBAsync” is suspended.
Error details: Microsoft.ServiceModel.Channels.Common.ConnectionException: Login failed for user ‘DOMAIN BTSHostSrvc’. —> System.Data.SqlClient.SqlException: Login failed for user DOMAIN BTSHostSrvc’.
at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, DbConnectionPool pool, String accessToken, Boolean applyTransientFaultHandling, SqlAuthenticationProviderManager sqlAuthProviderManager)
at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry)
at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
at System.Data.SqlClient.SqlConnection.Open()
at Microsoft.Adapters.Sql.SqlAdapterConnection.OpenConnection()
— End of inner exception stack trace —
Server stack trace:
at Microsoft.Adapters.Sql.SqlAdapterConnection.OpenConnection()
at Microsoft.Adapters.Sql.ASDKConnection.Open(TimeSpan timeout)
at Microsoft.ServiceModel.Channels.Common.Design.ConnectionPool.GetConnection(Guid clientId, TimeSpan timeout)
at Microsoft.ServiceModel.Channels.Common.Design.ConnectionPool.GetConnectionHandler[TConnectionHandler](Guid clientId, TimeSpan timeout, MetadataLookup metadataLookup, String& connectionId)
at Microsoft.ServiceModel.Channels.Common.Channels.AdapterRequestChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.OneWayOperationSendPortRequestChannel`1.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open()
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at System.ServiceModel.ICommunicationObject.Open()
at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.GetChannel[TChannel](IBaseMessage bizTalkMessage, ChannelFactory`1& cachedFactory)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.SendMessage(IBaseMessage bizTalkMessage)
MessageId: {84B22A22-13F7-47C7-91B5-A863E64E268E}
Cause
Once again, sometimes is not quite true, the cause of the problem is simple to diagnose, and the error message gives a very good intel in the cause of the problem.
This problem occurs because the user account that you used to access the database, in my case the BizTalk Host Instance Account, don’t have permissions to connect… the SQL Server or SQL Server instance.
Just to be clear, this is not having permission to insert, read or event full permission to do operation on a specific database, that is completely different – I check all of that and the user have the correct access/permission. What I forget was to give access to connect to the SQL Server/SQL Server Instance.
Solution
To solve this issue, you must give access to the user, in my case BizTalk Host Instance Account to connect to the SQL Server and for that, you must:
- Open SQL Server Management Studio and connect to your server.
- In the Object Explorer, expand the “Security” folder under the server.
- Right click on the “Logins” folder and choose “New Login…”
- Add the username or group in the format “DomainUserNameOrGroup”
- Choose the “Securables” tab and make sure that you grant “Connect SQL” permission to the SQL Server/SQL Server instance
- Click “OK” and your user will be created and have access to connect to your SQL Server.
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira
by Praveena Jayanarayanan | Oct 9, 2018 | BizTalk Community Blogs via Syndication
Hi there! It’s time to upgrade your BizTalk360 installation! We are here with our next release of BizTalk360, v8.9. As promised, this release also comes up with a bunch of exciting new features, enhancements and of course some bug fixes.
The series of blogs, explaining the different features coming up in v8.9, have already been released. But, to make it easy for our customers, we thought it would be nice to give a brief description of all these features in a single place. This way it would be easy to get the real big picture of this new release.
As per the below quotes,
“The key is to set realistic customer expectations, and then not to just meet them, but to exceed them- preferably in unexpected and helpful ways”
– Richard Branson
The features are added to the product based on the customer feedback and suggestions. We understand the customer needs and add them to the product to make the product as suitable as possible for the user.
Come on, let’s jump in to get the list!
User Access Policy enhancements
In the User Access Policy section, the Application access section has new capabilities. Initially, it was a list of applications which needed to be checked for providing access to the Normal Users/Groups. But then, what about the newly deployed applications? Every time when a new application was deployed, the Admin would need to scroll down the entire list to check for the new applications and then provide access. This was very time-consuming.
Now, to ease the process, we have provided different rules for configuring the access. But then, only one rule can be applied at a time. The different rules include:
- Grant Access by Applications
- Grant Access to All Applications
- Wildcard Search
- Grant Access to Application groups
Grant Access to All Applications
As the name denotes, enabling this rule will provide access to all the available applications for the Normal Users/Groups. The user will automatically be granted access to all the newly deployed applications.
Wildcard Search
This enables users to select the options from the wildcard operator drop down. Once this rule is configured, the user will have access to all the applications matching this wildcard. The user will automatically be given access to the newly created applications that match the wildcard.
Grant Access to Application Groups
With this new capability, you can create Application Groups and map the applications to that group. Once the user is given access to the Application group, he/she can access all the applications which are mapped to that group.
Grant Access by Applications
For persisting the existing configuration data, we have another rule available: ‘Grant Access by Application’. Once the upgrade is completed, this will be the default rule which is selected for existing users. The only difference between this configuration and the other new rules, is that when Grant Access by Applications is configured, newly applications created will not automatically be given access as in the other rules.
Stop Alerts for Maintenance during business holidays
If a user sets up multiple maintenance windows, they need to configure the business holidays individually. It will take much of your time, to configure them for every single environment in BizTalk360. To reduce the time and ease the maintenance configuration for the users, the capability to add business holiday calendars has been introduced.
These business holiday calendars can be mapped during maintenance window setup. This new configuration section is introduced in the Monitoring Notification settings section as “Configure Business Holidays”.
In the Stop Alerts for Maintenance settings page, a new section is introduced to configure the business holiday calendars. All the configured calendars with Status enabled will be displayed in the “Select Business Holiday Calendar” drop down list. A user can select the desired calendar and use it for a maintenance window. During the business holiday, a maintenance window will be active.
The users can also exclude certain alarms during the maintenance. This means, that, except the selected alarms, other alarms will undergo maintenance. This capability is very useful in situations where administrators don’t want to receive alerts during the weekends except for few specific alarms.
Web Endpoint monitoring improvements
From 8.9 version on, BizTalk360 Web Endpoint authentication is extended to support:
Let’s have a look at the improvements in these areas.
Basic Access Authentication
This is a method for an HTTP user agent to provide a user name and password when making a request. To unauthenticated requests, the server should return a response whose header contains a HTTP 401 Unauthorized status and a WWW-Authenticate field. In the BizTalk Admin Console, an HTTP endpoint can provision Basic authentication with a username and password
Certificate Authentication
In BizTalk360, the authentication type of Basic or Windows, along with the client certificate thumbprint, is configured in the Authorization section of Web Endpoint monitoring.
Azure Services Authentication
To be able to use Azure Services Authentication, a Service Principal must be configured in Azure. A Service Principal is an application within Azure Active Directory whose authentication tokens can be used as the client Id, client secret, and tenant fields (the subscription can be independently recovered from your Azure account details).
Additional content types
BizTalk360 8.9 extends the support to additional content types in request and response objects:
- SOAP (1.2) Content Type – “application/soap+XML” is a SOAP 1.2 content type which is added to the list. With this additional content type, SOAP V1.2 protocol is supported in web endpoint monitoring. The user can configure the XPath conditions to monitor the SOAP 1.2 endpoints, based on the results of the execution.
- Custom Content Type – When Endpoint Request/Response content types are not supported by BizTalk360, the Web Endpoint throws a HTTP 415 Unsupported Media Type. To prevent this from happening, you can configure Custom Content types.
Extended Import/Export Configuration
In version 8.9, we added support for import and export of the following sections:
- Knowledge Base
- Service Instances
- ESB Exceptions
- Event Logs
- Throttling Data
- BizTalk Reports
- Dashboards
- Operation (Default & Custom Dashboards)
- Analytics (Default & Custom Dashboards)
- EDI Dashboards
- ESB Dashboards
- Custom Widgets
The details of this feature can be found here.
Additional columns filter capability
Grid columns in BizTalk360 are getting a fresh look. You can customize the column headers which are most important to your business scenario.
Grid columns can be dynamically removed or added based on the user preference. As per the settings in the configuration section, columns will be aligned and displayed in the grid view. These customized column settings can be saved for future reference as well. We are sure, this capability will add more value when the administrator is looking for the instances/messages based on various conditions.
As an initial phase, this implementation has been done in the following areas in BizTalk360:
- Message Box Queries
- Graphical Flow (Tracking)
- Electronic Data Interchange
BizTalk360 allows saving as many patterns as the user wants. To search the messages based on different scenarios, admins prefer different filter conditions to validate. In those situations, BizTalk360 allows to save different query filters and keeps them for future use. You can also download the customized column data using the Export to Excel capability.
Centralized Advanced Event Log viewer performance improvement
In our previous versions, up to v8.8, Event Log collection logic is not segregated based per server. To enable the Event Log collection, it is bound to the BizTalk environment and not on an individual server level. However, the user has control on configuring the sources based on the need.
In the new version of BizTalk360, users can control the Event Log collection according to individual servers. As an administrator, you know the value of each source which needs most considerations. So, there is an option as well in BizTalk360 to configure the BizTalk & SQL server sources separately.
Using these settings, you can customize and narrow down your Event Log search.
PowerShell Notification Channel
In our earlier versions, the users were already able to send notifications to specific notification channels (E.g.: Slack, ServiceNow, Webhook, Teams). Now the user can configure PowerShell scripts in the Notification Channel while configuring an alarm.
More Enhancements in BizTalk360 v8.9
Besides the above mentioned features, we have also brought a number of enhancements to existing features.
Monitor queues for message age – We have enhanced the option to monitor the queues with message ages (time of message till it exists in the queue) for IBM MQ and Service Bus Queues.
Notifications grouped by Error Description – Previously, Service instances were grouped by Error Code in the alert emails. Now, there is a new setting “Enable Group by Description” introduced to group the service instances based on the Error Description to get full insight about your errored service instances.
New filter option in ESB Exception portal – There is a new filter option “Service Name” introduced in the ESB Exception Data query builder. This will enable users to perform extensive search and get the desired results.
Restore XSLT templates – Whenever we make any improvements to the default email template there is no option for the users to restore the changes from the GUI. This makes them to manually copy/paste the XSLT from the database to utilize the new changes. To avoid the manual intervention, a new option has been provided “Restore System XSLT” to restore the changes from GUI.
PDF download available in more areas – PDF download capability is not new in BizTalk360. We have provided this option in few other areas of the application, to download the reports, dashboards and message flows from the GUI. This option is provided in Operation, Monitoring & Analytics Dashboards, Graphical Flow (Tracking) and Messaging Patterns.
Group your Logic Apps by and monitor on Resource Groups – In the earlier versions, Logic Apps are not grouped by Resource Group name. There was an issue to display the Logic Apps which are created in different Resource Groups with the same name. With the introduction of “Resource Group” column, Logic Apps are grouped by Resource Group in every single configured Azure subscription.
Multiple installer improvements – We have enhanced the BizTalk360 installer in v8.9 in few areas:
- Single credentials during upgrade – Only one set of credentials (User Name, Password) will be asked during the upgrade process, in case the same credentials have been used for all the installed components
- SQL Authentication – BizTalk360 will now support SQL Authentication for upgrade
Finally, of course, they are a number of bug fixes as well. Kindly refer the Release Notes for the complete details.
Conclusion
We always monitor the feedback portal and take up the suggestions and feedback. Now we would like to request you, our customers, to please take the time to fill this questionnaire to help us prioritize the next upcoming feature tasks, to let us know what are your main pain points and help us to further improve the product.
Why not give BizTalk360 a try! It takes about 10 minutes to install on your BizTalk environments and you can witness and check the security and productivity of your own BizTalk Environments. Get started with the free 30 days trial. Happy monitoring with BizTalk360!
Author: Praveena Jayanarayanan
I am working as Senior Support Engineer at BizTalk360. I always believe in team work leading to success because “We all cannot do everything or solve every issue. ‘It’s impossible’. However, if we each simply do our part, make our own contribution, regardless of how small we may think it is…. together it adds up and great things get accomplished.” View all posts by Praveena Jayanarayanan
by Gautam | Oct 7, 2018 | BizTalk Community Blogs via Syndication
Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?
Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.
If you want to receive these updates weekly, then don’t forget to Subscribe!
Feedback
Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.
by Sandro Pereira | Oct 4, 2018 | BizTalk Community Blogs via Syndication
To end these week contributions, tomorrow is national holiday in Portugal, BizTalk Pipeline Components Extensions Utility Pack community project for BizTalk Server 2016 got a new update. A new component that you can use in your custom BizTalk Server Send pipelines is now available: Remove XML Namespace Pipeline Component.
Remove XML Namespace Pipeline Component
The XML Remove Namespace pipeline component was developed in the old days by Johan Hedberg for BizTalk Server 2006 – you can check the original post here – that I now migrate to BizTalk Server 2016 with his permission and it can be used to remove XML namespaces from XML documents inside custom pipelines.
This has the availability to transform the root of the message from this:
<ns0:Blah xmlns:ns0="http://RemoveXmlNamespace.BTS.BlahMessage">
Into this:
<Blah>
As Johan mention in his original post, you shouldn’t remove the namespace from the messages, however, we don’t leave in a perfect world and we need to deal with legacy systems and sometimes they required, for unknown reasons for me, that we deliver messages without namespace.
To use this pipeline component in your projects you just copy the “RemoveXmlNamespace.PipelineComponents.dll” file into “Pipeline Components“ folder that exists in BizTalk Server Installation directory: “..Program Files (x86)Microsoft BizTalk Server <version>Pipeline Components” on every server.
You do not need to add a custom pipeline component to be used by the BizTalk Runtime to the Global Assembly Cache (GAC).
What is BizTalk Pipeline Components Extensions Utility Pack?
BizTalk Pipeline Components Extensions Utility Pack is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in received and sent pipelines, which will provide an extension of BizTalk out-of-the-box pipeline capabilities.
The project is available on BizTalk Server Open Source Community repository on GitHub (https://github.com/BizTalkCommunity) and everyone can contribute with new pipeline components that can be extended or improve the existing BizTalk Server capabilities.
At the moment it is only available for BizTalk Server 2016 but it will soon be compiled and available for previous versions of the product.
Where to download it?
You can download BizTalk Pipeline Components Extensions Utility Pack from GitHub here:
BizTalk Pipeline Components Extensions Utility Pack
GitHub
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira
by Lattetapriyanka Vishu | Oct 4, 2018 | BizTalk Community Blogs via Syndication
Recently, we released BizTalk360 v8.9. As always, each new release of BizTalk360 has completely new features in it, according to the customer feedback and suggestions we received. Keeping up with the business needs of our customers, version 8.9 has many beneficial features.
Along with the new features, we have enhanced quite a few features in BizTalk360. In our previous blog, we have covered few of these enhancements. In this blog post, I would like to provide a brief description about the following enhancements:
- ESB Service Name as a new filter option
- A new column “Resource Group” is introduced in Logic Apps (Operations & Monitoring)
- Data Monitoring Service Instances grouped by Error Description in Email Alerts
- Increased number of characters allowed in Alarm descriptions
- Ability to set Threshold Alert’s Start Time and End Time between 23.30 to 23.59 hours
- BizTalk360 v8.9 installer changes
ESB Service Name as a new filter option
In the ESB Exception Data feature, a query builder is available to fetch the precise data on a particular condition.
In version 8.9, we have added another filter property, namely Service Name. Choosing the Service Name in the filter properties will list the available service names as an option and the user can select the required option from the list. Thus, execution of the query will list the precise data in the grid. For even more detailed filtering, multi-combination of filter properties can be set.
We have implemented the same for ESB Data Monitoring; under the section Set Data Filter, the Service Name Property has been added. Selecting the Service Name in the filter will list the related options, so the Service Name can now be used for your ESB Exceptions Data Monitoring
Resource Group in Logic Apps
The Resource Group column has been implemented in the Logic Apps section, to have a clear picture over the Logic Apps being created under different Resource Groups in the same Azure Subscription.
In the previous version, all the Logic Apps under a subscription were listed. Quite often, a subscription will have different Resource Groups and each Resource Group can have their own set of Logic Apps.
There was an issue while display Logic Apps which are created in different Resource Groups, but with the same name. With the introduction of the Resource Group column, Logic Apps are now grouped by Resource Group in each configured Azure subscription.
Resource Group implementation is done in three areas, Operations, Monitoring and Data Monitoring.
Resource Group implementation in Operations and Monitoring
In the Operations section the Logic Apps are listed in the grid, based on their Azure subscription. This may lead to Logic App Name duplication, and to confusion in finding out to which Resource Group the Logic App is associated.
This is sorted in BizTalk360 v8.9, by adding the Resource Group column in the grid. Thus, once the Azure subscription is configured, the Logic Apps will be listed in the grid, along with the Resource Group to which they are associated. So, it will be easier to manage the Logic Apps by knowing their Resource Groups.
The Resource Group column is also implemented in the Monitoring Section, under Manage Mapping. When an alarm is set to monitor Logic Apps, it avoids name duplication, since Logic Apps are associated to the Resource Group.
Resource Group implementation in Data Monitoring
The implementation of Resource Groups in Data Monitoring helps in segregating the Logic Apps present under the associated Resource Groups.
In Logic App Data Monitoring, the Resource Group field is added under Set Data Filter. By filtering on Resource Group, only the associated Logic Apps will be listed and monitored.
Group By Error Description
In the Message Box Data monitoring section, the instance details were sent in the notification email, by grouping them based on the Error Code. In BizTalk360 v8.9, we have added the capability to group the instances by Error Description. This is done by implementing as a small feature under Settings / System Settings / Monitoring & Notification section, where you can find an ‘Enable Group By Description’ toggle button.
By default, Email notification errors are grouped by Error Code. Enabling this toggle button will result in grouping by the Error Description.
The Data Monitoring alert gets populated, grouping the instance details by the Error Description and list the count of it, along with its description.
As shown in the sample mail alert, having the grouping of description helps the user to have a very clear and precise clarity in the email alert for the suspended instances.
Increased the number of characters allowed in alarm descriptions
As you will probably know, the Alarm screen, in the Monitoring section, holds a Description field in which the user can feed up to 300 characters for providing a short description related to the alarm. This helps the customer to have a clarity over the alarm popped.
We now increased this limitation to 1000 characters, that in turn helps the customer to give a detailed description on the alarm. Exceeding the 1000 character will lead to a validation warning.
Hyperlink in Alarm Description
As an added advantage along with the 1000 characters, the Description field now accepts a Hyperlink and validates the HTML hyperlink tag. The field not only validates the hyperlink, it also validates basic HTML tags too.
Any syntax errors found in the hyperlink, will lead to a validation warning. By including the hyperlink tag, the customers can provide any necessary link which is related to the alarm.
We mainly have provided the hyperlink tag to comfort the user by allowing them to give more detail via a web page over the alarm created.
Threshold Alert’s Start Time and End Time increased to 23.59 pm
BizTalk360 has the capability to configure a threshold to trigger within certain time limits. The alarm configuration has a check field, “Set Alerts on Set day(s) and Time(s) only”. Once this check box is enabled, it will allow the user to access the Days Check and Time Selection, to precisely set the threshold time to trigger the alert emails during the particular days and times.
Any weekday can be selected, meanwhile timing for the day can be set by selecting the values in the Start Time and End Time dropdown boxes. The Start and End Time’s time value increases by 30 min, so any time can be set according to the need, under the condition “*Must be 60 mins later than start time”.
The lag was that, the Start Time and End time can be set only up to 23.30 pm. So, we were not able to fetch persisting violation details which occurred between 23.30 pm-23.59 pm.
As a solution to it, in the version 8.9, the End Time duration has been increased from 23.30 pm to 23.59 pm. By exceeding the End Time to 23.59 pm, the user will not find any lag in getting the violation alert. This way, the mapped artifacts of the Threshold alarm(s) can be monitored for the entire day.
BizTalk360 v8.9 Installer changes
The Installer is a facet of most software products. It is important to every Product organization to improve the installer from time to time, to provide a seamless installation experience. In our latest version, we made improvements in various aspects of the installer.
Rethinking the upgrade credential screen
The BizTalk360 installation package consists of three components:
- Web services
- Windows NT services (Monitoring, Analytics)
- Database
Users have the liberty to install the preferred components as per the business need. For an instance, the web component can be installed in a separate machine and the same counts for the other components.
Here is an interesting change we made in the Credentials update screen, for the upgrade scenario. What is the change? Previously, during the upgrade scenario, if all the components are installed in the same server, the user must provide passwords for all the components, even though all the components are using the same set of credentials. We have revisited this logic and made our installer to determine by itself whether all the installed components are using same credentials or different ones.
Let’s consider all the components are using the same credentials (User Name and Password). In that case, the installer will show only one section with the username and password. This reduces the user activity to enter the same password for the all the components.
Support for SQL authentication upgrade
Until the previous release, BizTalk360 did not support the upgrade, for the SQL authentication mode. In v8.9, the installer has been enhanced to support SQL authentication upgrade.
3.5 .NET dependency removal
For smooth functioning of the Azure services in BizTalk360, at a minimum .NET version 4.5 is recommended for BizTalk360 version v8.1 and later. The Installer blocks the installation if .NET 3.5 has not been installed in the machine. This restriction made users to install the .NET version 3.5. Now, this limitation has been removed.
Persisting config files during the update
To deploy BizTalk360 on HTTPS, your IIS must be enabled for HTTPS/SSL. To access BizTalk360 through HTTPS, there are few changes that need to be done in the BizTalk360 web.config file. Until our previous version, during the upgrade process, those changes were not persisted. After every upgrade, users had to do the changes again and again in the web.config files. As of the v8.9 upgrade, the changes made in the web.config files will be persisted.
Conclusion
We provided these improvements, hoping they will be helpful to you and make you feel easy while using BizTalk360. Happy migrating and try BizTalk360!!!
by Sandro Pereira | Oct 3, 2018 | BizTalk Community Blogs via Syndication
And once again, BizTalk Pipeline Components Extensions Utility Pack community project for BizTalk Server 2016 got a new update. A new component that you can use in your custom BizTalk Server Send pipelines is now available: Zip Pipeline Component.
Zip Pipeline Component
The Zip Pipeline Component is a pipeline component for BizTalk Server which can be used in a send pipeline (encode stage) and is intended to compress (zip/gzip) outgoing messages.
- The capabilities are similar to those available in compression software such as WinZip or 7-zip:
- This component requires two configurations that are:
- the “FileExtension” where you can specify if you want for example a .zip or .gz file.
- and “Enabled” that is a true or false value to activate the compression.
No compression/decompression software needs to be installed in the BizTalk Server machines.
This was an old component that I had that I now updated and improved to BizTalk Server 2016 with the help with my internal team at DevScope, in special, Pedro Almeida.
To use this pipeline component in your projects you just copy the “BizTalk.PipelineComponents.ZipFile.dll” file into “Pipeline Components“ folder that exists in BizTalk Server Installation directory: “..Program Files (x86)Microsoft BizTalk Server <version>Pipeline Components” on every server.
You do not need to add a custom pipeline component to be used by the BizTalk Runtime to the Global Assembly Cache (GAC).
What is BizTalk Pipeline Components Extensions Utility Pack?
BizTalk Pipeline Components Extensions Utility Pack is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in received and sent pipelines, which will provide an extension of BizTalk out-of-the-box pipeline capabilities.
The project is available on BizTalk Server Open Source Community repository on GitHub (https://github.com/BizTalkCommunity) and everyone can contribute with new pipeline components that can be extended or improve the existing BizTalk Server capabilities.
At the moment it is only available for BizTalk Server 2016 but it will soon be compiled and available for previous versions of the product.
Where to download it?
You can download BizTalk Pipeline Components Extensions Utility Pack from GitHub here:
BizTalk Pipeline Components Extensions Utility Pack
GitHub
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira
by Sandro Pereira | Oct 2, 2018 | BizTalk Community Blogs via Syndication
Welcome back to my blog post series “BizTalk Server Tips and Tricks” for developers, administrators or business users. Not all tips and tricks are sophisticated or quite robust and ingenious, some of them are quite simple. Some of them are in front of us all the time without us noticing, this is one of these cases.
Today there is a lot more information than in the past, but normally when we start BizTalk Server tutorials about schemas, they tend to say that if you want to specify how many times a record or an element will occur, you should configure the below values:
- The Max Occurs property to specify the maximum occurrences of this node (record or element). The default value is ‘1’ and cannot be less than the Min Occurs property
- The Min Occurs property to specify the minimum occurrences of this node. The default value is ‘1’ and cannot be greater than the Max Occurs property
What this type of tutorial also tells you, is that if you want to specify that a specific node can appear an unlimited number of times, at the Max Occurs property, you should type the value: “unbounded”
So, we tend to manually write the word “unbounded”, each time we want to set a node to appear an unlimited number of times… I think, I can write this word better than my personal name, so many are the times I’ve written it over the years. But to be honest, this can be a little time consuming. Especially if we type it wrongly, newbies need to remember this word – there are other words with the same meaning as unlimited – and sometimes is just a little boring.
Well, guess what, if you are at this point of the post wondering what is the alternative, in the future you should spend a little time reading the description of property inside Visual Studio; the alternative has been there in front of you all the time:
- Maximum Occurrences of this node. Its value should always be greater than or equal to minOccurs of this node. Use ‘unbounded’ or ‘*’ (asterisk) to indicate unlimited occurrences. The default value is ‘1’.
Which means that if we type * (asterisk), this will be automatically translated to “unbounded”.
Quick, simple and practical
Stay tuned for new tips and tricks!
Author: Sandro Pereira
Sandro Pereira is an Azure MVP and works as an Integration consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. View all posts by Sandro Pereira
by Sandro Pereira | Oct 2, 2018 | BizTalk Community Blogs via Syndication
BizTalk Pipeline Components Extensions Utility Pack community project for BizTalk Server 2016 got a new update and now it has another new component that you can use in your custom BizTalk Server pipelines: Carry SOAPHeader To WCF-BasicHttp Pipeline Component.
Carry SOAPHeader To WCF-BasicHttp Pipeline Component
The Carry SOAPHeader To WCF-BasicHttp is a pipeline component for BizTalk Server which can be used in a send pipeline and is intended to carry forward the received custom SOAP Header to the outgoing message, in other words:
- This component will read the Custom SOAPHeader from the Message Context Properties
- The custom header name will be defined on the component configuration and we can read it from the target “http://schemas.microsoft.com/BizTalk/2003/SOAPHeader”
- and will configure this custom header in the OutboundCustomHeaders property used by the WCF-BasicHTTP Adapter.
The OutboundCustomHeaders property is used if you want to specify the custom SOAP headers for outgoing messages. When this property is used, the property must have the <headers> element as the root element. All of the custom SOAP headers must be placed inside the <headers> element
- If the custom SOAP header value is an empty string, you must assign <headers></headers> or <headers/> to this property
This component is useful if you are implementing CBR operations on legacy service (SOAP – .asmx) using the new WCF-Adapters in BizTalk Server. Notice that the SOAP adapters, still present in BizTalk Server 2016, are obsolete and discontinued.
To use this pipeline component in your projects you just copy the “CarrySOAPHeader.dll” file into “Pipeline Components“ folder that exists in BizTalk Server Installation directory: “..Program Files (x86)Microsoft BizTalk Server <version>Pipeline Components” on every server.
You do not need to add a custom pipeline component to be used by the BizTalk Runtime to the Global Assembly Cache (GAC).
What is BizTalk Pipeline Components Extensions Utility Pack?
BizTalk Pipeline Components Extensions Utility Pack is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in received and sent pipelines, which will provide an extension of BizTalk out-of-the-box pipeline capabilities.
The project is available on BizTalk Server Open Source Community repository on GitHub (https://github.com/BizTalkCommunity) and everyone can contribute with new pipeline components that can be extended or improve the existing BizTalk Server capabilities.
At the moment it is only available for BizTalk Server 2016 but it will soon be compiled and available for previous versions of the product.
Where to download it?
You can download BizTalk Pipeline Components Extensions Utility Pack from GitHub here:
BizTalk Pipeline Components Extensions Utility Pack
GitHub
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira