Working with CloudEvents in Azure Event Grid

by | May 7, 2018 | BizTalk Community Blogs via Syndication | 0 comments

This post was originally published here

Recently Microsoft announced Azure Event Grid, a highly scalable serverless event driven offering allowing us to implement publish and subscribe patterns. Event driven scenarios are becoming more common by the day, which means that we see these type of integrations increasing a lot as well. A lot of times applications will define their own message formats for their events, however, with the recent announcement of native support in Azure Event Grid for CloudEvents our lives should be made a lot easier. CloudEvents is a standard for working with events accross platforms, and gives us a specification for describing event data in a common way. This will allow any platform or application which is working with events, to implement a common format, allowing easy integration and interoperability, for example between Azure, AWS and Oracle. The specification is still under active development, and Microsoft is one of the big contributors, especially Clemens Vasters, Lead Architect on Azure Messaging Services.

CloudEvents logo

In this blog post we will be looking into Event Grid’s support for CloudEvents, and how to set this up. The specifications for the CloudEvents message format can be found on GitHub, and how this maps to Event Grid’s own schema can be found on Microsoft Docs. For this post we will use the application created in this bogpost, which will generate events when an order has been placed, as well as when a repair has been requested. These events will be handled by a Logic App, which will send out an email. In a real life scenario we could, for example, use this Logic App to create place the order at the ship’s supplier. And because we are using the CloudEvents format, the application can easily integrate with any system which supports this new specification, so they are not just bound to Azure.

Send event from custom application to Logic Apps

Send event from custom application to Logic Apps

Currently support for Cloud Events in Event Grid is still in preview only available in a select group of regions (West Central US, Central US and North Europe), and to use it we need to enable an extension in Azure CLI by giving the following command.

az extension add --name eventgrid
Enable Event Grid extension

Enable Event Grid extension

We can now create our Event Grid topic, where we will receive the events. Currently this is not yet supported in the portal, so we will stay in our Azure CLI, and give the following commands.

az group create -l northeurope -n cloudEventsResourceGroup
az eventgrid topic create --name cloudevents -l northeurope -g cloudEventsResourceGroup --input-schema cloudeventv01schema

The first command creates the resource group, while the second command creates the Event Grid topic. Note the input-schema switch, which allows us to set the CloudEvents format.

Create Event Grid topic

When the topic has been created, go to the Event Grid Topics blade in the portal, open the topic we just created, and grab the Topic Endpoint, we will need this later on.

Save the topic endpoint for later use

Save the topic endpoint for later use

Switch to the Access keys for the topic, and grab one of the keys, we will need this later as well.

Also save on of the keys for later use

Next we will create the application which will send the events to our custom topic which we just created. For ease of this demo, this will just be a simple console application, but in a real life solution this could be any type of system. Start by creating a new solution in Visual Studio for our application.

Create console app solution

Create console app solution

Data Classes

Add the following data classes, which describe the orders and repairs, as explained in this blog post.

/// <summary>
/// Event sent for a specific ship.
/// </summary>
public class ShipEvent
{
    /// <summary>
    /// Name of the ship.
    /// </summary>
    public string Ship { get; set; }
 
    /// <summary>
    /// Type of event.
    /// </summary>
    public string Type { get; set; }
}

/// <summary>
/// Used to place an order.
/// </summary>
public class Order : ShipEvent
{
    /// <summary>
    /// Name of the product.
    /// </summary>
    public string Product { get; set; }
 
    /// <summary>
    /// Number of items to be ordered.
    /// </summary>
    public int Amount { get; set; }
 
    /// <summary>
    /// Constructor.
    /// </summary>
    public Order()
    {
        Type = "Order";
    }
}

/// <summary>
/// Used to request a repair.
/// </summary>
public class Repair : ShipEvent
{
    /// <summary>
    /// Device which needs to be repaired.
    /// </summary>
    public string Device { get; set; }
 
    /// <summary>
    /// Description of the defect.
    /// </summary>
    public string Description { get; set; }
 
    /// <summary>
    /// Constructor.
    /// </summary>
    public Repair()
    {
        Type = "Repair";
    }
}

CloudEvents class

Add the CloudEvents class, which will be used to create a CloudEvents message which we will send to our Azure Event Grid. The schema for a CloudEvents message can be found here.

/// <summary>
/// Representation of the CloudEvents specification, to be sent to Event Grid Topic.
/// </summary>
class CloudEvents
{
        /// <summary>
        /// This will be used to update the Source and Data properties.
        /// </summary>
        public ShipEvent UpdateProperties
        {
                set
                {
                        Source = $"{Program.TOPIC}#{value.Ship}/{value.Type}";
                        Data = value;
                }
        }
 
        /// <summary>
        /// Gets the version number of the CloudEvents specification which has been used.
        /// </summary>
        public string CloudEventsVersion { get; }
 
        /// <summary>
        /// Gets the registered event type for this event source.
        /// </summary>
        public string EventType { get; }
 
        /// <summary>
        /// Gets the The version of the eventType.
        /// </summary>
        public string EventTypeVersion { get; }
 
        /// <summary>
        /// Gets the event producer properties.
        /// </summary>
        public string Source { get; set; }
 
        /// <summary>
        /// Gets the unique identifier for the event.
        /// </summary>
        public string EventID { get; }
 
        /// <summary>
        /// Gets the time the event is generated based on the provider's UTC time.
        /// </summary>
        public string EventTime { get; }
 
        /// <summary>
        /// Gets or sets the event data specific to the resource provider.
        /// </summary>
        public ShipEvent Data { get; set; }
 
        /// <summary>
        /// Constructor.
        /// </summary>
        public CloudEvents()
        {
                CloudEventsVersion = "0.1";
                EventID = Guid.NewGuid().ToString();
                EventType = "shipevent";
                EventTime = DateTime.UtcNow.ToString("o");
        }
}

.Program Class

And finally we will update the Program class. Here we will get the input from the user, and create a CloudEvents message which will be sent to Event Grid. Make sure to update the topic endpoint and access key with the entries we retrieved from the portal in the previous step. Also update the topic property with your subscription id, and the resource group and topic name you used when creating the topic. One more thing to notice, is how we only send a single message, instead of a List of messages as we did in this blog post. Currently CloudEvents does not support batching of events, which is why we can only send a single event.

/// <summary>
/// Send CloudEvents messages to an Event Grid Topic.
/// </summary>
class Program
{
        /// <summary>
        /// Endpoint of the Event Grid Topic.
        /// Update this with your own endpoint from the Azure Portal.
        /// </summary>
        private const string TOPIC_ENDPOINT = "<your-topic-endpoint>";
 
        /// <summary>
        /// Key of the Event Grid Topic.
        /// Update this with your own key from the Azure Portal.
        /// </summary>
        private const string KEY = "<your-access-key>";
 
        /// <summary>
        /// Topic to which we will be publishing.
        /// Update the subscription id, resource group and topic name here.
        /// </summary>
        public const string TOPIC = "/subscriptions/<your-subscription-id>/resourceGroups/<your-resource-group>/providers/Microsoft.EventGrid/topics/<your-topic-name>";
 
        /// <summary>
        /// Main method.
        /// </summary>
        public static void Main(string[] args)
        {
                // Set default values
                var entry = string.Empty;
 
                // Loop until user exits
                while (entry != "e" && entry != "exit")
                {
                        // Get entry from user
                        Console.WriteLine("Do you want to send an (o)rder, request a (r)epair or (e)xit the application?");
                        entry = Console.ReadLine()?.ToLowerInvariant();
 
                        // Get name of the ship
                        Console.WriteLine("What is the name of the ship?");
                        var shipName = Console.ReadLine();
 
                        CloudEvents cloudEvents;
                        switch (entry)
                        {
                                case "e":
                                case "exit":
                                        continue;
                                case "o":
                                case "order":
                                        // Get user input
                                        Console.WriteLine("What would you like to order?");
                                        var product = Console.ReadLine();
                                        Console.WriteLine("How many would you like to order?");
                                        var amount = Convert.ToInt32(Console.ReadLine());
 
                                        // Create order event
                                        // Event Grid expects a list of events, even when only one event is sent
                                        cloudEvents = new CloudEvents { UpdateProperties = new Order { Ship = shipName, Product = product, Amount = amount } };
                                        break;
                                case "r":
                                case "repair":
                                        // Get user input
                                        Console.WriteLine("Which device would you like to get repaired?");
                                        var device = Console.ReadLine();
                                        Console.WriteLine("Please provide a description of the issue.");
                                        var description = Console.ReadLine();
 
                                        // Create repair event
                                        // Event Grid expects a list of events, even when only one event is sent
                                        cloudEvents = new CloudEvents { UpdateProperties = new Repair { Ship = shipName, Device = device, Description = description } };
                                        break;
                                default:
                                        Console.Error.WriteLine("Invalid entry received.");
                                        continue;
                        }
 
                        // Send to Event Grid Topic
                        SendEventsToTopic(cloudEvents).Wait();
                }
        }
 
        /// <summary>
        /// Send events to Event Grid Topic.
        /// </summary>
        private static async Task SendEventsToTopic(CloudEvents cloudEvents)
        {
                // Create a HTTP client which we will use to post to the Event Grid Topic
                var httpClient = new HttpClient();
 
                // Add key in the request headers
                httpClient.DefaultRequestHeaders.Add("aeg-sas-key", KEY);
 
                // Event grid expects event data as JSON
                var json = JsonConvert.SerializeObject(cloudEvents);
 
                // Create request which will be sent to the topic
                var content = new StringContent(json, Encoding.UTF8, "application/json");
 
                // Send request
                Console.WriteLine("Sending event to Event Grid...");
                var result = await httpClient.PostAsync(TOPIC_ENDPOINT, content);
 
                // Show result
                Console.WriteLine($"Event sent with result: {result.ReasonPhrase}");
                Console.WriteLine();
        }
}

The complete code for the Event Publisher application can also be found here on GitHub.

Our next step is to create the Logic App which will handle the events sent by our events publisher application.

Create Logic App for processing events

Create Logic App for processing events

Once the Logic App has been created, open the designer and create a HTTP Request trigger template.

Use HTTP Request trigger

Use HTTP Request trigger

Set the Request JSON Schema to the following, which is a representation of the CloudEvents schema including the ship events.

{
    "type": "object",
    "properties": {
        "CloudEventsVersion": {
            "type": "string"
        },
        "EventType": {
            "type": "string"
        },
        "EventTypeVersion": {},
        "Source": {
            "type": "string"
        },
        "EventID": {
            "type": "string"
        },
        "EventTime": {
            "type": "string"
        },
        "Data": {
            "type": "object",
            "properties": {
                "Ship": {
                    "type": "string"
                },
                "Type": {
                    "type": "string"
                }
            }
        }
    }
}
Set JSON schema for parsing the request

Set JSON schema for parsing the request

Add a step to send out an email, and authenticate using your Office365 account. If you don’t have an Office365 account you can also use one of the other connectors to send out an email.

Add the Office365 Outlook connector

Add the Office365 Outlook connector

Set the options for the email and save the Logic App. When you save the Logic App make sure to grab the HTTP POST URL of the HTTP Request trigger, as we will need this in the next step to set up the subscription.

Set email properties with body to the data element

Set email properties with body to the data element

We are now going to create the Event Grid subscription, which will catch the events from our events publisher, and route them to our Logic App. We will have to do this once again from the Azure CLI, as the portal UI does not yet support the use of the CloudEvents schema. Give the following command in the Azure CLI to create the subscription which will route messages to our Logic Apps HTTP endpoint. Remember the Event Grid extension should be enabled for this.

az eventgrid event-subscription create --name shipEventsToProcessingLogicApp --topic-name cloudevents -g cloudEventsResouceGroup --endpoint '"<endpoint-for-logic-app-http-trigger>"' --event-delivery-schema cloudeventv01schema
Run Azure CLI command to create subscription

Run Azure CLI command to create subscription

Now go to the Event Grid Subscriptions blade, make sure the filters are set right, and you will find your newly created subscription.

Subscription has been created

Testing

Open the event publisher application, and send in some events.

Send events from event publisher application

Send events from event publisher application

These events will now be received in the Event Grid topic, and routed to the subscription, which will then deliver it at the Logic App.

Logic App run shows we receive CloudEvents message

Logic App run shows we receive CloudEvents message

An email will then be sent, indicating the type of event.

Receive email with event information

Receive email with event information

Submit a Comment