Getting Data from the Intune Data Warehouse with PowerShell

The Intune Data Warehouse is a great addition to the Microsoft Intune service allowing visibility of historical data for reporting, data and trend analysis for your Microsoft MDM environment. It comes with an OData feed that allows you to connect to the data with PowerBI, Microsoft’s reporting and data visualization service.

The Data Warehouse RESTful API (currently in Beta) can be used to get data from the warehouse using a REST client. I decided to explore how to do this with PowerShell so I can run some ad-hoc queries and analyse trends in the data.

To get data from the Intune Data Warehouse we need to do three main things:

1. Create a native App in Azure and give it access to the Intune Data Warehouse
2. Authenticate with Azure using OAuth 2.0 and get an access token
3. Invoke the RESTful web service using http

Create a Native App in Azure

In your Azure portal, go to Azure Active Directory > App registrations. Click New application registration.

Give it a name, make sure it is a Native app (do not use Web app / API) and use the redirect URI https://login.live.com/oauth20_desktop.srf .

Click Create.

Once created, make a note of the Application ID as we will need this later.

Now, in the App in the Settings blade, click Required permissions > Add > Select an API and select Microsoft Intune API.

In the Add API access blade click Select permissions and grant the delegated permission Get data warehouse information from Microsoft Intune.

Save your changes.

Authenticate with Azure

To authenticate with Azure I wrote the following function in PowerShell:

What the code does…

To authenticate with Azure we need to use the Active Directory Authentication Library from Microsoft. This library is actually included in resources like the ConfigMgr client and the AzureRM PowerShell module, but these do not have the latest versions of the library and the methods it contains have changed over time. So I prefer to use the latest version of the library (3.17.2 at the time of writing) which is available as a NuGet package.

The function will download NuGet and use it to download the latest version of the ADAL library to your user profile area. Once we have the library we will add it in PowerShell. Then we will acquire an access token.

The access token expires after an hour so once created, the token will probably be good for your current session. If it expires, simply run the function again. The access token will be saved to a variable in the script scope and will be used by the function that queries the data warehouse.

How to use it…

Make sure the following parameters contain the required values. I recommend that you store those in the function directly so you don’t have to add them every time.

• RedirectURL: Must be the same as the RedirectURI you added to the native app in Azure, ie https://login.live.com/oauth20_desktop.srf
• ClientID: this is the Application ID from your Azure native app

Simply run the function:

New-IntuneDataWarehouseAccessToken

If you have not previously authenticated with Azure in your current session you will be prompted to sign in to your Azure account:

The first time you use the native app you created, you will also be prompted for permission:

Invoke the Web Service using the OData feed

Now that we have an access token, we can invoke the web service using http. I wrote the following PowerShell function to do that:

What the code does…

The function uses the custom OData feed URL for your tenant, creates an http client to invoke the web service and gets data for the data entity (ie collection, or table) that you specify. The results are then returned to the PowerShell console.

You can read more about the data model for the warehouse and get a reference for the various data entities and their relationships on the Microsoft Docs site.

How to use it…

Make sure the following parameter is set in the function:

• WarehouseUrl

This Url is the custom feed URL for your tenant and you can find it from the Intune blade in Azure. On the Overview blade, on the right you find Other tasks and underneath Set up Intune Data Warehouse.

To list the data entities that are available to query use the ListDataEntities switch:

Get-IntuneDataWarehouseData -ListDataEntities

To return the data from a specific data entity, use the DataEntity parameter. This example returns data from the devices table.

Get-IntuneDataWarehouseData -DataEntity devices

Working with the Data

The API supports a few query parameters in the OData protocol v4.0, so rather than returning all the results in the data entity, you can narrow them down. However, I have noticed that the query parameters do not always work as expected when they are combined, at least in the beta version of the API.

For example, you can use the Filter parameter to return only matching results. This query finds a specific device:

Get-IntuneDataWarehouseData -DataEntity devices -Filter "deviceName eq 'SW-IT-LT-AZURE1'"

You can select only specific properties to be returned in the results using the Select parameter:

Get-IntuneDataWarehouseData -DataEntity devices -Select 'deviceName,serialNumber,lastContact'

Select the top 5 results:

Get-IntuneDataWarehouseData -DataEntity devices -Top 5

Skip 10 results and return the rest:

Get-IntuneDataWarehouseData -DataEntity devices -Skip 10

Sort results by a particular property:

Get-IntuneDataWarehouseData -DataEntity devices -OrderBy 'lastContact'

You can learn more about how to use query parameters in the OData protocol from the Microsoft Developer site for the Graph API.

These query parameters are certainly helpful, but for relational queries where you want to reference more than one table or entity, the API comes short and more complex Powershell code is required.

For example, here I am searching for the ethernet MAC address from the most recent hardware inventory for a device, and even with query parameters applied I still need to do some filtering and sorting in Powershell. Remember that the data warehouse only contains snapshots of historic data, so for current data you can use the Graph API instead, but this is just for an example.

$DeviceName = "SW-IT-LT-AZURE1"
Get-IntuneDataWarehouseData -DataEntity 'mdmDeviceInventoryHistories' -Select 'deviceKey,datekey,ethernetMac' |
    Where {$_.devicekey -eq ((Get-IntuneDataWarehouseData -DataEntity 'devices' -Filter "deviceName eq '$DeviceName'").devicekey)} |
    Sort datekey -Descending |
    Select -First 1 |
    Select -ExpandProperty ethernetMac 

This code takes a while to run however because it’s querying the data warehouse more than once to get the data. Another way to do this would be to first load the required device entities into memory, then I can query them more quickly and run other queries from this cached data.

The following code will load just those two entities into a hash table:

$DataEntities = "mdmDeviceInventoryHistories","devices"
$DataHash = @{}
foreach ($DataEntity in $DataEntities)
{
    Write-host "Loading $DataEntity"
    [void]$DataHash.Add($DataEntity,(Get-IntuneDataWarehouseData -DataEntity $DataEntity))
}

Then I can run the following code to get the ethernet MAC address and it returns the result instantly:

$DeviceName = "SW-IT-LT-AZURE1"
$DataHash['mdmDeviceInventoryHistories'] |
    Where {$_.devicekey -eq (($DataHash['devices'] | where {$_.deviceName -eq $DeviceName}).devicekey)} |
    Sort datekey -Descending |
    Select -First 1 |
    Select -ExpandProperty ethernetMac 

You could load the entire data warehouse into memory using the following code, then you can simply work with the resultant hashtable:

$DataEntities = Get-IntuneDataWarehouseData -ListDataEntities
$DataHash = @{}
foreach ($DataEntity in $DataEntities)
{
    Write-host "Loading $DataEntity"
    [void]$DataHash.Add($DataEntity,(Get-IntuneDataWarehouseData -DataEntity $DataEntity))
}

The benefit of a data warehouse of course is that you can review snapshots of data over a period of time and analyse the data for trends or identify when things changed. The following example is using the data hashtable and is reporting the device inventory history of a specific device over time. The Intune data warehouse keeps up to 90 days of historic data. In particular, I want to see how much the free space on disk is changing over time.

$DeviceName = "SW-IT-LT-158"
$Results = $DataHash['mdmDeviceInventoryHistories'] |
    Where {$_.devicekey -eq (($DataHash['devices'] | where {$_.deviceName -eq $DeviceName}).devicekey)} |
    Sort datekey -Descending |
    Select dateKey, deviceName,
        softwareVersion,
        @{e={$([math]::Round(($_.storageFree / 1GB),2))};l="storageFree (GB)"},
        @{e={$([math]::Round(($_.storageTotal / 1GB),2))};l="storageTotal (GB)"}
foreach ($Result in $results){
    $Result | Add-Member -MemberType NoteProperty -Name date -Value (($DataHash['dates'] | Where {$_.dateKey -eq $Result.dateKey}).fullDate | Get-Date -Format "dd MMM yyyy")
}
$Results | Select deviceName,date,softwareVersion,'storageFree (GB)','storageTotal (GB)' | ft

You can readily see that it’s necessary to manipulate the data quite a bit to get the results I want to see, for example in order to do something equivalent to a ‘join’ in SQL I am using Where-Object, and in order to add the data from another table to my results I am using Add-Member. I am also converting the values of the storage data into GB and formatting the date using the UK short date code.

The results are returned in an array object, but for data like this it can also be useful to use a datatable as you would for SQL data for example.  Then you can add / remove columns, change column order, set the datatype for a column, change headers etc.

This code does exactly the same thing as the last example, but using a datatable for the results.

$DeviceName = "sw-it-lt-158"
$Datatable = New-Object System.Data.DataTable
[void]$Datatable.Columns.AddRange(@('deviceName','date','softwareVersion','storageFree (GB)','storageTotal (GB)'))
$Results = $DataHash['mdmDeviceInventoryHistories'] |
    Where {$_.devicekey -eq (($DataHash['devices'] | where {$_.deviceName -eq $DeviceName}).devicekey)} |
    Sort datekey -Descending |
    Select dateKey,
        deviceName,
        softwareVersion,
        @{e={$([math]::Round(($_.storageFree / 1GB),2))};l="storageFree (GB)"},
        @{e={$([math]::Round(($_.storageTotal / 1GB),2))};l="storageTotal (GB)"}
foreach ($Result in $results){
    [datetime]$Date = ($DataHash['dates'] | Where {$_.dateKey -eq $Result.dateKey}).fullDate
    [void]$DataTable.Rows.Add($Result.deviceName,$Date.ToShortDateString(),$Result.softwareVersion,$Result.'storageFree (GB)', $Result.'storageTotal (GB)')
}
$Datatable | ft

Reviewing the results I can see that the available disk space is decreasing slightly over time. It would be nice to see that data represented graphically, and of course this is where the integration with PowerBI will shine, but we can also generate graphical charts in Powershell, so let’s give that a go.

Here is a function I wrote that will generate a spline chart using the .Net chart controls and display it in a WPF window. It takes a single series of data and you need to provide a title, a data object as an array, the X and Y axis names (which must match the header names in the data object).

To generate the chart, I will use the results from my previous example (not the datatable but the array), sort them by date, select the last 20 data snapshots, select the X and Y axis data into a new object and provide this to the chart function:

$Data = $Results | Sort dateKey | Select date,'storageFree (GB)' | Select -Last 20
New-SingleSeriesSplineChart -Title "Trend of Available Free Storage on SW-IT-LT-158" -Data $Data -AxisX "date" -AxisY "storageFree (GB)"

Now I have a nice graphical view 🙂

I have focused just on devices in this blog, but there is lots of data available in the Intune Data Warehouse including users, policies, compliance, configurations, MAM data etc, all of which can provide valuable insights into your MDM estate and whether you use PowerShell, PowerBI, Excel or whichever tool, the ability to view and analyse historic data is a welcome improvement to the ever-evolving Intune service.