You may face the same challenges that I experienced when integrating Microsoft Dynamics CRM and BizTalk. The objective is to capture events in real-time and transmit them to an outside system without losing the order of events. There are two web methods exposed by CRM web services, “Retrieve” and “RetrieveMultiple”. The CRM web services provide […]
Blog Post by: Rakesh Gunaseelan
I’m pleased to announce that the White Paper titled “The A-Y of running BizTalk Server in Microsoft Azure” which I have been working on for the last two months is now available to download on the BizTalk 360 White Paper collection. Writing this White Paper has been a momentous task, especially given that Microsoft Azure […]
Blog Post by: Johann
In this post I’ll show you how to correctly configure diagnostics in an Azure Worker Role to push custom log files (NLog, Log4Net etc.) to Azure Storage using the in-built Azure Diagnostics Agent. Configuring our Custom Logger – NLog I’m not a massive fan of the recommended Azure Worker Role logging process, namely using […]
Blog Post by: modhul
As you may know, Azure Machine Learning can execute R scripts. You can interactively see the output console. But what about retrieving the result as part of a production call to the API generated by Azure ML?
Let’s test with a word cloud example in R. Mollie Taylor has posted one here (https://gist.github.com/mollietaylor/3671518) that we can reuse in Azure Machine Learning:
The details on how to create an Azure ML workspace, insert a dataset and an R script can be found here:
for R, just use that module:
The input of the Web API is set to the input dataset of the R Script and the output is set to the R Device port. As a reminder, here is how the inputs and outputs are positioned in an R Script module:
the detail is available in the help documentation.
In our case the interesting ports to publish are the following:
and
After running the experiment, we can see the result in Azure ML Studio:
So, how could we retrieve the pictures from an API that is published that way:
Here is some sample script in Python that shows how to do it. The script is a modified version of the sample given in the API Help page for Batch Execution. The idea is to get the base64 encoded pictures from the output file and decode them out to local disk.
# -*- coding: utf-8 -*- # How this works: # # 1. Assume the input is present in a local file # 2. Upload the file to an Azure blob - you'd need an Azure storage account # 3. Call BES to process the data in the blob. # 4. The results get written to another Azure blob. # 5. Download the output blob to a local file # # Note: You may need to download/install the Azure SDK for Python. # See: http://azure.microsoft.com/en-us/documentation/articles/python-how-to-install/ import urllib2 import json import time from azure.storage import * import sys import base64 import json storage_account_name = 'a****obfuscated***4' storage_account_key = '/aV****obfuscated***vXA76w==' storage_container_name = 'benjguin' input_file = ur"C:\be****obfuscated***os\WordCloud\conventions.csv" output_file = ur'C:\be****obfuscated***os\WordCloud\myresults.csv' input_blob_name = 'conventions.csv' api_key = r'Cczx****obfuscated***WemQ==' url = 'https://ussouthcentral.services.azureml.net/workspaces/a7c****obfuscated***756/services/d328e03****obfuscated***5c2/jobs' uploadfile=True executeBES=True blob_service = BlobService(account_name=storage_account_name, account_key=storage_account_key) if uploadfile: print("Uploading the input to blob storage...") data_to_upload = open(input_file, 'r').read() blob_service.put_blob(storage_container_name, input_blob_name, data_to_upload, x_ms_blob_type='BlockBlob') input_blob_path = '/' + storage_container_name + '/' + input_blob_name debug_blob = blob_service.get_blob(storage_container_name, input_blob_name) if executeBES: print("Submitting the BES job...") connection_string = "DefaultEndpointsProtocol=https;AccountName=" + storage_account_name + ";AccountKey=" + storage_account_key payload = { "Input": { "ConnectionString": connection_string, "RelativeLocation": input_blob_path } } body = str.encode(json.dumps(payload)) headers = { 'Content-Type':'application/json', 'Authorization':('Bearer ' + api_key)} req = urllib2.Request(url, body, headers) response = urllib2.urlopen(req) result = response.read() job_id = result[1:-1] # remove the enclosing double-quotes url2 = url + '/' + job_id while True: time.sleep(1) # wait a second authHeader = { 'Authorization':('Bearer ' + api_key)} request = urllib2.Request(url2, headers=authHeader) response = urllib2.urlopen(request) result = json.loads(response.read()) status = result['StatusCode'] if (status == 0): print("Not started...") elif (status == 1): print("Running...") elif (status == 2): print("Failed...") break elif (status == 3): print("Cancelled...") break elif (status == 4): print("Finished!") result_blob_location = result['Result'] sas_token = result_blob_location['SasBlobToken'] base_url = result_blob_location['BaseLocation'] relative_url = result_blob_location['RelativeLocation'] url3 = base_url + relative_url + sas_token response = urllib2.urlopen(url3) with open(output_file, 'w') as f: f.write(response.read()) break outputdata=open(output_file) outputtxt=outputdata.read() outputdata.close() s=outputtxt.index('\"{') e=len(outputtxt) o1=outputtxt[s+1:e-3] jsonresult = json.loads(o1) i=1 for gd in jsonresult['Graphics Device']: fname = output_file + "." + str(i) + ".png" print 'writing png #' + str(i) + ' to ' + fname f = open(fname, 'wb') f.write(base64.b64decode(gd)) f.close() i += 1 print("Done!")
Here is a sample execution output:
Uploading the input to blob storage...
Submitting the BES job...
Running...
Running...
Running...
Running...
Running...
Running...
Running...
Finished!
writing png #1 to C:\be***obfuscated***os\WordCloud\myresults.csv.1.png
writing png #2 to C:\be***obfuscated***os\WordCloud\myresults.csv.2.png
Done!
The output sent back by Azure ML looks like this:
R Output JSON
"{"Standard Output":"RWorker pushed \"port1\" to R workspace.\r\nBeginning R Execute Script\n\n[1] 56000\r\nLoading objects:\r\n port1\r\n[1] \"Loading variable port1...\"\r\npng \r\n 2 \r\nnull device \r\n 1 \r\n","Standard Error":"R reported no errors.","visualizationType":"rOutput","Graphics Device":["iVBORw0K***(...)***RvX/wFzB5s8eym6ZgAAAABJRU5ErkJggg==","iVBORw0KGgo***(...)***dVorBuiQAAAABJRU5ErkJggg=="]}"
You can see the pictures
well, Python does:
The resulting files are:
and
R has tons of great data visualisation. Have a look at those blogs for instance:
Benjamin (@benjguin)
Blog Post by: Benjamin GUINEBERTIERE
Today I came across something I was keeping an eye on recently and that is the BizTalk
Flow Visualiser on Codeplex.
It basically looks at your tracking data and gives you back call details.
– Errors/Succes
– # of times something is called
– amount of time taken to process each stage.
It’s pretty easy to setup, there’s a site and a serviceand the rest you can guess.
http://biztalkflowvisualizer.codeplex.com/
Check it out and feell free to contribute!!
Enjoy!!!
Blog Post by: Mick Badran
BizTalk360 is a full featured BizTalk Server Monitoring & Operations product. The product has matured from a small web based monitoring tool to a full blown feature rich product for monitoring BizTalk Server. The product includes features for BizTalk Operation support people like the Throttling analyzer, Topology Viewer, and Backup DR Monitor. Recently a new version 7.8 has been released with
Exciting news Version 1.7.0.1 of BizTalk Mapper Extensions UtilityPack for BizTalk Server 2013 is now available! Project Description BizTalk Mapper Extensions UtilityPack is a set of libraries with several useful Functoids to include and use it in a map, which will provide an extension of BizTalk Mapper capabilities. Here’s the change-log for this release: Fixes […]
Blog Post by: Sandro Pereira
BizTalk360 just released their latest version 7.8 with interesting new features. With this new release comes some very interesting new features that make it even easier to actively monitor and maintain your BizTalk environment healthy. Be sure to check out this latest release and download the setup or request for a free trial version. Some […]
Blog Post by: mitchke
Introduction
When the Windows 10 Technical Preview came out earlier this month, I wanted to see kick the tires a bit and see what was new. However, I need my laptop to work reliably, so I couldn’t take the risk of installing Windows 10 over my Windows 8.1 installation.
So, I decided to install it to a […]
Blog Post by: jgiessner
In the first part of the 7.8 release article, we covered in detail about BizTalk Server “Advanced Process Monitoring” capabilities we added in 7.8 release. In this part 2 article let’s take a deep look at the other new features we bundled in 7.8 EDI reporting capabilities inline with BizTalk Admin Console System Profile : […]
The post BizTalk360 version 7.8 released – EDI reporting, System Profile and User Profile appeared first on BizTalk360 Blog.
Blog Post by: Saravana Kumar