TL;DR
Connect Subscription Activity Log with Azure Event Hubs for sending logs to third party SIEM using Terraform. Reduced scope for RBAC permission on Auth rule.

Jump to recipe

I have been doing some baking with sourdough 🍞 during the summer, and as such visited many blogs for guidance. On every occasion I have appreciated the “jump to recipe” link, so I can ignore the life story of whoever has blogged about it. This will be included in my posts going forward, along with the usual TL;DR on top.

Lots of things happen in the cloud. Therefore it is great to have the Azure Monitor Activity Log to keep track of everything that goes on in your Azure Subscriptions! You can find your Azure Monitor Activity log in a few different ways, which is actually just different filtered queries. Below is my preferred place for viewing these settings -  the Azure Monitor Activity Log page.

As always you need to make sure your subscriptions are included in your global subscriptions filter (Directories + Subscriptions). Otherwise they will not show in any of the below blades.

Azure Monitor Activity Log Page

Open Azure Monitor, and choose “Activity log” in the left menu. If you navigate from the subscriptions blade Activity Log, that is just a shortcut here anyway.

You can also fetch entries with Azure CLI or PowerShell.

Export the Activity Log events

Many organizations have a central SIEM (Security Incident and Event Management) solution, and want to analyze Azure Activity there. For this to happen, you need to send the Activity log to an Azure Event Hub, before you can consume them from a SIEM solution. This is what I want to show today in Terraform.

Azure Event Hubs is a service for ingesting events and streaming large amounts of data. I won’t go into details on how it works, but if you know Apache Kafka this is a similar service based on Event Hubs/topics and consumers. Events are sent to the Event Hubs and stored there for a set amount of time. From there it provides the third party SIEM access to the events on-demand.

First let’s see how it looks in the portal. We will navigate to Azure Monitor Activity Log like above, and you can see the “Export Activity Logs” button on the top. Click it.

This brings you to the Activity Log Diagnostic Settings for multiple subscriptions from the same page. Choose the subscription in the dropdown menu. 

The Event Hub export settings look like this. This example is my diagnostic setting after deploying with Terraform.

I will not go into the details on performing this in the portal. Just wanted to let you know about the Activity Log page. The manual create diagnostics settings steps are described here. If you are still using the log profile way of collecting logs, you should consider transitioning to diagnostic settings.

Creating with Terraform

I prefer doing this with IaC (Terraform/Bicep) over ClickOps. It is entirely possible to do ClickOps, but then you miss out on all the benefits of Infrastructure as Code. Bicep (or ARM in a pinch) would also be a good way of doing this, but not in scope for my post. I will show you an example of how it can be configured, and also explain how it works. If you only want to see the code, all Terraform code can be found here.

Some required resources and config

Providers are mentioned with version constraints, random string for generating a unique name each time, current client config for fetching information about client config. The AzureRM block with features { } seems to be redundant, but for some reason it needs to be there. I guess they aren’t doing anything with it, and it has been an issue for a long time.

terraform { required_providers { azurerm = { source = “hashicorp/azurerm” version = “>= 3.18.0” } azuread = { source = “hashicorp/azuread” version = “>= 2.27.0” } } }

provider “azurerm” { features {} }

data “azurerm_client_config” “current” {}

resource “azurerm_resource_group” “rg” { name = “siem-eventhub-rg” location = “norwayeast” }

resource “random_string” “unique” { length = 6 numeric = true special = false upper = false }

Event Hub, Namespace, and Authorization Rule

Any Event Hub resources needed are created here. It creates one Event Hub Namespace, which you can view as an Event Hub container. Then a single Event Hub called “insights-activity-logs” is created inside the Namespace. A default Authorization Rule called RootManageSharedAccess is created automatically, but I also want a rule with only listen/send permissions. Hence one is created with Terraform.

resource “azurerm_eventhub_namespace” “eventhub-ns” { name = “siem-eventhub-${random_string.unique.result}-ns” location = azurerm_resource_group.rg.location resource_group_name = azurerm_resource_group.rg.name sku = “Standard” capacity = 2 }

resource “azurerm_eventhub” “activity-logs” { name = “insights-activity-logs” namespace_name = azurerm_eventhub_namespace.eventhub-ns.name resource_group_name = azurerm_resource_group.rg.name partition_count = 4 message_retention = 7 }

resource “azurerm_eventhub_namespace_authorization_rule” “listen-send” { name = “RootListenSendSharedAccessKey” namespace_name = azurerm_eventhub_namespace.eventhub-ns.name resource_group_name = azurerm_resource_group.rg.name

listen = true send = true manage = false }

Subscription Diagnostic Settings

This is the part where subscription diagnostic settings are configured. All log categories you want to send must be defined here. It is up to you what you want to send, but I would recommend at least the categories I have configured. See the Terraform code in GitHub for more.

resource “azurerm_monitor_diagnostic_setting” “diag-setting” { name = “toEventHub” target_resource_id = “/subscriptions/${data.azurerm_client_config.current.subscription_id}” eventhub_name = azurerm_eventhub.activity-logs.name eventhub_authorization_rule_id = azurerm_eventhub_namespace_authorization_rule.listen-send.id

log { category = “Administrative” enabled = true

retention\_policy {
  days    = 0
  enabled = false
}

log { … } #Redacted for readability }

Role Assignment

Terraform needs access to the Authorization Rule in your Event Hub Namespace, to facilitate a connection from Azure Monitor Activity Log. This is provided with an Azure Role Assignment, which allows the current Terraform service principal to list the relevant Authorization Rule. More information on this here.

The Azure AD group is created (requires that your service principal has permissions to create groups in Azure AD), and current Terraform object id is added as a member. Then the Role Assignment is performed at Authorization Rule scope.

Tip: You can scope a role assignment to the actual Authorization Rule resource, and avoid granting access to all authorization rules in the namespace. This is possible with Terraform/PowerShell/AzCLI, but not in the portal, as far as I know.

resource “azurerm_role_assignment” “eventhub-owner-authrule-scope” { principal_id = azuread_group.aad-evh-owner.object_id scope = azurerm_eventhub_namespace_authorization_rule.listen-send.id role_definition_name = “Azure Event Hubs Data Owner” }

resource “azuread_group” “aad-evh-owner” {
display_name = “azure-sub-eventhub-owner”
security_enabled = true
members = [
data.azurerm_client_config.current.object_id
]
}

Assuming you have a Terraform service principal for each subscription (not everyone does), you will need to add these service principals to the Event Hub owners Azure AD group. Without it you may experience “Insufficient permissions” when trying to configure the Event Hub Diagnostics setting in Terraform.

The next, and last, step in collecting logs is to consume the Event Hub from your SIEM, but this is different for all products. Check your SIEM’s manual for specific instructions.

Result

The Event Hub Namespace should look something like this when events are sent here, with incoming requests for each event.

Authorization Rule, Shared Access Policy or Shared Access Signature?

Microsoft has this to say about these terms:

A shared access signature (SAS) provides you with a way to grant limited access to resources in your Event Hubs namespace. SAS guards access to Event Hubs resources based on authorization rules. These rules are configured either on a namespace, or an entity (event hub or topic).

An authorization rule is assigned a primary key and a secondary key. You can find some best practices here. As far as I can gather, the Authorization Rules live inside a Shared Access Policy.

A namespace or entity policy can hold up to 12 shared access authorization rules, providing room for the three sets of rules, each covering the basic rights, and the combination of Send and Listen.

Cleanup

If you have been following along and created this in your lab, please destroy any unnecessary resources after you are done. These resources do incur some cost, and cleanup of unused resources are always a good practice.

In summary

You have now configured Log Shipping to an Azure Event Hub from Azure Monitor Activity Log! The first step in doing Activity Log SIEM events analysis 💪

If you know how to authenticate with Azure AD for this functionality, I would appreciate a comment. Also comment if there is any room for improvement or actual errors 🗨️

Please don’t hesitate to leave a comment here, or on LinkedIn. I appreciate constructive criticism, and welcome an opportunity to learn from others 😊

Connect with me on Twitter, if you haven’t already, as I am trying to gain more followers 😊