Microsoft Endpoint Manager aka Intune has been around for a while now and has evolved quite significantly since its early days and the old Silverlight portal (remember that?). Historically Intune hasn’t been particularly good with its reporting capability, but since end 2019 and the announcement of the new reporting framework, things are starting to improve with more built-in reports appearing all the time and now the ability to create your own Azure monitor workbooks from log analytics – welcome additions for sure.
That being said, if you use Power BI as a corporate reporting service there’s no better place to create custom reports according to your needs and share those reports with interested parties without having to give them access to Intune and the MEM portal. Intune data can be queried using Microsoft Graph yet there is currently no native way to use Microsoft Graph as data source in Power BI short of developing a custom connector. With the advent of log analytics data for Intune, we will be able to export log analytics queries to Power BI using M query language which looks promising.
In the meantime, we need to use a little creativity to get data out of Intune and into Power BI to furnish a custom report. I’ve seen examples of using Logic Apps and Function app in Azure as an intermediary process, which is cool. I decided to take a slightly different approach to this and use an Azure automation account to simply export data from Microsoft Graph on a schedule and dump it into an Azure storage account. Power BI supports the use of blob storage as a data source so this works quite nicely.
An Azure automation runbook queries Microsoft Graph, organizes the data a bit then exports it into CSV files
The CSV files are then uploaded as blobs in a container in a storage account
Power BI then uses those CSV files as its datasource allowing us to create custom reports from the data
There are a few advantages to this approach:
You can use Power BI to create your own custom reports from any Microsoft Graph data across Microsoft 365, and you can combine data from different sources in a single report
Reports support using a scheduled refresh in the Power BI service for keeping data up-to-date
Everything is in the cloud – there is no requirement for on-prem resources or a data gateway
You can use a managed identity or a Run as account for simplified, secure no-credential authentication
Graph data can be manipulated using PowerShell before export allowing customization of the final data set
On the last point, some data in Microsoft Graph contains nested data, eg a field may contain its own set of key-value pairs and you would want to expand these out into their own fields to report on them. Or you may wish to create your own calculated fields from the data, for example how many days since the last sync of an enrolled device.
Additionally Microsoft recently announced support for enabling a managed identity for an automation account, and this is excellent for simplifying and improving the security of access to Microsoft Graph.
The only slight disadvantage to this approach perhaps is that data in the report won’t be live – it will only be as good as your refresh schedule.
Just to give an example, here are a couple of screenshots of a report I created from Intune data for enrolled devices, including Android, iOS and Windows.
I’ve created a new docs site where you can download this report as a template, the runbook used to export the data as well as access detailed instructions on how to set up a data export from MS Graph into Power BI.
Having a password policy is a best practice for security of accounts, whether domain, local or wherever passwords are used. In the Windows world, domain accounts have a default domain password policy. Azure AD accounts have the Azure AD password policy. Accounts local to Windows can have a password policy too, and you can use Intune to set this if you want. In fact, if you deploy the Windows 10 Security baseline in Intune you will be deploying a password policy to your local accounts.
However, deploying a password policy on Windows with Intune can have an unexpected side effect: it can force a local account to change the password at next logon:
If you regular rotate the password for the local administrator account using a LAPS solution, for example, this becomes a right royal pain because password rotation will fail due to the requirement to change the password at next logon.
I first came across this issue when doing Windows Autopilot deployments. Our password rotation solution would fail to rotate the password on the local admin account on some machines. After some lengthy investigation with Microsoft, we discovered there was a compliance policy created in the old Silverlight Intune portal (which we could no longer access) and targeted to these users that was setting a password policy. Compliance policies created in the new portal in Azure override the policies in the old portal, but since no new compliance policy was created for Windows 10, the old policy was still in effect and was causing this issue.
Later I came across this again when rolling out the Windows 10 security baseline in Intune, which by default has a password policy. I noticed that our password rotation solution was failing on recently deployed Azure AD-joined devices, after we enabled the baseline.
What’s going on?!
So why does this happen? Well, when Intune sets a password policy it uses the DeviceLock policies in the Policy CSP. Under the hood, this is using the Exchange Active Sync policy engine to set the password policies, which was created back in the Windows 8 era to enforce some security policies on devices that sync with Exchange. Reading through the documentation, you come across this little nugget:
When password length and complexity rules are applied, all the control user and administrator accounts are marked to change the password at the next sign in to ensure complexity requirements are met.
All you have to do is enable a password policy and some default values will get set for password length and complexity, and these polices will require that a local administrator account change its password at next logon. The reason is that the policy doesn’t know if the currently set password meets the requirements of the policy. The only way it can be sure it complies is to force you to change it, and the new password must meet the policy requirements. In this way it can truthfully report whether the device is compliant to the policy.
Sounds reasonable, but forcing the password change breaks your LAPS solution as you cannot programmatically change the password, or even remove the requirement to change the password at next logon – you’ll get some nice errors if you try.
Logon Failure: EAS policy requires that the user change their password before this operation can be performed
When a password policy is set by Intune, you’ll see some registry keys set under HKLM:\Software\Microsoft\PolicyManager\current\device\DeviceLock.
You’ll also see some Exchange Active Sync keys set under HKLM:\System\CurrentControlSet\EAS\Policies
Once the EAS policies have applied, I could not find any way to override them. Removing the Intune password policy, deleting the EAS registry keys – for me, nothing worked. I still could not programmatically change the local admin password or remove the requirement to change the password at next logon.
The only thing that worked is to satisfy the policy – ie actually log in as the local administrator and change the password when prompted, or log in as another administrator account and use Local Users and Groups to change the password.
I should emphasise that this only affects local accounts, not domain, and the issue was not seen consistently on all machines where the security baseline was applied – it seemed to affect new or recently deployed machines – even those where password rotation had previously occurred successfully.
Since having to manually and interactively change a local admin password is not a feasible option at scale, I simply cannot recommend to use Intune to set a password policy if you are using a password rotation solution. Remember that a password policy can be set in different places in Intune – as part of a Windows 10 security baseline, as a configuration profile, using OMA-URIs directly, or with a compliance policy.
Is there a fix?
A workaround that has been successful for me is to use secedit to set a password policy for local accounts. This ageing utility still works nicely to deploy a local security configuration and it doesn’t use EAS policies so it doesn’t force local accounts to change their passwords to comply with the policy.
You can open secpol.msc on a test machine and set the password policy as you desire:
Then from an admin cmd, run secedit to export the configuration:
Deploy this as a remediation script using Proactive remediations in Endpoint Analytics, or an MEMCM compliance item using a dummy detection method to ensure it runs regularly.
Just be careful to avoid policy conflict by only including security configurations (like password policy) that are not being set elsewhere, as for example the Windows 10 Security baseline in Intune will also set local account security policies.
I was working on some updates to our unified reporting solution for Windows Updates (ie WUfB + MEMCM) and I wanted to figure out simply from the OS build number whether a Windows 10 workstation has the latest cumulative update installed. The only reliable and useable static list I could find for Windows 10 build numbers is Microsoft’s Windows 10 Update History web page, so I decided to build a PowerShell script that parses the page to get current patch info.
The script below can be used to report which OS build a Windows 10 workstation is currently on as well as which update is the latest update available to the device. It can also report on all Windows updates published for the version of Windows 10 a workstation is currently on.
Run the script as is to show you:
Current OS version
Current OS Edition
Current OS Build number
The installed update that corresponds to that build number, as well as the KB number and a link to the info page
The latest available update for the OS version
Compare the latest available update with the currently installed one to know if the OS is up-to-date.
If there are Preview or Out-of-band updates available that are more recent than the one you have installed, you can exclude those from being reported as a latest available update, so you can just focus on the cumulative updates.
Not so long ago HP published a customer advisory listing a number of their models that need to be on the latest BIOS release to be upgraded to Windows 10 2004. Since we were getting ready to rollout 20H2 we encountered some affected models in piloting, which prompted me to find that advisory and then get the BIOS updated on affected devices.
To be honest, until now we’ve never pushed out BIOS updates to anyone, but to get these devices updated to 20H2 we now had no choice. In this post I’m just going to share how we did that. For us, good user experience is critical but finding the balance between keeping devices secure and up-to-date without being too disruptive to the user can be a challenge!
First off was to create a script that could update the BIOS on any supported HP workstation, without needing to package and distribute BIOS update content. I know other equally handsome community members have shared some great scripts and solutions for doing BIOS updates, but I decided to create my own in this instance to meet our particular requirements and afford a bit more control over the update process. I considered using the HP Client Management Script Library which seems purpose-built for this kind of task and is a great resource, but I preferred not to have the dependency of an external PowerShell module and its requirements.
I published a version of this script in Github here. The script does the following:
Creates a working directory under ProgramData
Disables the IE first run wizard which causes a problem for the Invoke-WebRequest cmdlet when running in a context where IE hasn’t been initialised
Checks the HP Image Assistant (HPIA) web page for the latest version and downloads it
Extracts and runs the HPIA to identify the latest applicable BIOS update (if any)
If a BIOS update is available, downloads and extracts the softpaq
Checks if a BIOS password has been set, if so creates an encrypted password file as required by the firmware update utility
Runs the firmware update utility to stage the update
Everything is logged to the working directory and the logs are uploaded to Azure blob storage upon completion, or if something fails, so we can review them without requiring remote access to the user’s computer
It all runs silently without any user interaction and it can be run on any HP model that the HPIA supports.
In Production however, we used a slightly modified version of this script. Since there was the possibility that there could be unknown BIOS passwords in use out there, we decided not to try to flash the BIOS using an encrypted password file, but instead try to remove the BIOS password altogether (temporarily!) When the BIOS update is staged it simply copies the password file to the staging volume – it doesn’t check whether the password is correct or not. If it is not correct, the user would then be asked for the correct password when the BIOS is flashed and that is not cool! Removing the password meant that the user could never be unexpectedly prompted for the password in the event that the provided password file was incorrect. Of course, to remove the password you also have to know the password, so we tried the ones we knew and if they worked, great, if they didn’t, the script would simply exit out as a failsafe.
We have a compliance baseline deployed with MEMCM that sets the BIOS password on any managed workstation that does not have one set, so after the machine rebooted, the BIOS flashed and machine starts up again, before long the CB would run and set the password again.
Doing this also meant that we needed to ensure the computer was restarted asap after the update was staged – and for another reason as well – Bitlocker encryption is suspended until the update is applied and the machine restarted.
Because we didn’t want to force the update on users and force a restart on them, we decided to package the script as an application in MEMCM. This meant a couple of things:
We could put a nice corporate logo on the app and make it available for install in the Software Center
We could handle the return codes with custom actions. In this case, we are expecting the PowerShell script to exit successfully with code 0, and when it does we’ve set that code to be a soft reboot so that MEMCM restart notifications are then displayed to the user.
As a detection method, the following PowerShell code was used. This simply detects if the last write time of the log file has changed within the last hour, if it has, it’s installed. Longer than an hour and it’s available for install again if needed.
We then deployed the application with an available deployment. We communicated with the users directly to inform them a BIOS update needed to be installed on their device in order to remain secure and up-to-date, and directed them to the Software Center to install it themselves.
We also prepared collections in SCCM using query-based membership rules to identify the machines that were affected by the HP advisory, and an SQL query to find the same information and pull the full user name and email address from inventoried data.
The script does contain the BIOS password in clear text which, or course, may not meet your security requirements, although for us this password is not really that critical – it’s just there to help prevent the user from making unauthorized changes in the BIOS. In our Production script though, we simply converted these to base64 before adding them to the script to at least provide some masking. But for greater security you could consider storing the password in Azure key vault and fetch it at run time with a web request, for example.
If you wish to use the script in your own environment, you’ll need to change the location of the working directory as you desire. Additionally if you wish to upload the log files to an Azure storage container, you’ll need to have or create a container and add the URL and the SAS token string to the script, or else just comment out the Upload-LogFilesToAzure function where it’s used. I’m a big fan of sending log files to Azure storage especially during this season where many are working from home and may not be corporate connected. You can just use Azure Storage Explorer to download the log files which will open up in CMTrace if that’s your default log viewer.
Hope this is helpful to someone! The PS script is below.
For my Windows 10 feature update installation process, I like to gather lots of statistics around the upgrade itself as well as the devices they are running on so we can later report on these. These stats can be useful for identifying areas of potential improvement in the upgrade process. One stat I gather is the offline time for the upgrade, ie the time between when the downlevel (online) phase is completed and the computer is restarted and the time when the offline phases have completed and the OS is brought back to the logon screen again. Knowing this value across the estate helps to gauge the user experience and how much time is being spent waiting for the offline phases to complete.
To calculate this value is actually straightforward – you can do it by searching the SYSTEM event log for the last time the computer was restarted and comparing it with the installation time of the OS which gets recorded in WMI after the offline phases have completed successfully. The only thing is, after the offline phase is complete the event logs are refreshed and previous log entries are removed, so you have to search the event log in the Windows.old folder instead. You have to do this before the Windows.old folder gets automatically removed (depending on your policy) and manual rollback is no longer possible.
The PowerShell code below searches for the most recent event ID 1074, compares the date with the OS install date value in WMI (use the *CIM* cmdlets to get an automatic conversion to [DateTime]) and outputs as a TimeSpan which you can log however you want.
The good news is that for a 20H2 upgrade from media – at least in my various tests – the offline time has been impressively low.
I saw a tweet recently from Madhu Sanke where he had deployed updated Conexant ISST drivers to his environment to release devices from the Safeguard ID 25178825 which at the time of writing still prevents devices trying to upgrade to 2004 or 20H2.
You can read more about the issue here, but around September/October 2020 timeframe, newer drivers became available that are not affected by this Safeguard and updating the drivers will release a device from this hold.
Madhu took the approach of downloading the drivers from the Microsoft Update Catalog and packaging them with a script wrapper for the install. In researching this myself, I found that there are more than one driver available and different models will take different drivers, so I decided to write a little C# program that will update a device using Windows Update directly instead.
The program simply connects to Windows Update online, checks if a newer driver version is available for the Conexant ISST audio driver (listed under ‘Conexant – MEDIA – ‘ in the MS update catalog), and downloads and installs it.
The executable can then be deployed as is using a product like Microsoft Endpoint Configuration Manager.
For environments where software updates are deployed with Configuration Manager / WSUS, the program will check if registry keys have been set preventing access to Windows Update online and temporarily open them. It will restore the previous settings after updating.
The program also logs to the Temp directory.
On my HP laptops, the driver in question is this guy:
This one has been updated and is no longer affected by the Safeguard.
You can download the C# executable from my GitHub repo, or you can clone the solution in Visual Studio and compile your own executable. Just remember to run the program in SYSTEM context or with administrative privilege on the client as it’s installing a driver.
A reboot is usually required after installation and the driver can display a small one-time toast notification like this:
If your devices are managed by Windows Update for Business, you may see a notification from there as well, depending on your configuration of course.
After deploying the driver update to affected devices, expect to see them being released from the Safeguard after telemetry has run. You can use my PowerBI reports to help with reporting on Safeguards in your environment.
This is the first blog post in what I hope will be a series of posts demonstrating several custom solutions I created for things such as feature update deployments, managing local admin password rotation, provisioning Windows 10 devices, managing drive mappings and more. My reasons for creating these solutions was to overcome some of the current limitations in existing products or processes, make things more cloud-first and independent of existing on-prem infrastructure where possible, and to more exactly meet the requirements of the business.
Although I will try to provide a generalised version of the source code where possible, I am not providing complete solutions that you can go ahead and use as is. Rather my intention is to inspire your own creativity, to give working examples of what could be done if you have the time and resource, and to provide source code as a reference or starting point for your own solutions should you wish to create them!
Someone asked me recently how we deploy feature updates and it was a difficult question to answer other than to say we use a custom-built process. Having used some of the existing methods available (ConfigMgr Software Updates, ConfigMgr custom WaaS process, Windows Update for Business) we concluded there were shortcomings in each of them, and this provided inspiration to create our own, customized process to give us the control, reliability, user experience and reporting capability that we desired. Don’t get me wrong – I am not saying these methods aren’t good – they just couldn’t do things exactly the way we wanted.
So I set out to create a bespoke process – one that we could customize according to our needs, that was largely independent of our existing Configuration Manager infrastructure and that could run on any device with internet access. This required making use of cloud services in Azure as well as a lot of custom scripting! In this blog, I’ll try to cover what I did and how it works.
First, let’s look at the end user experience of the feature update installation process – this was something key for us, to improve the user experience keeping it simple yet informative, and able to respond appropriately to any upgrade issues.
Once the update is available to a device, a toast notification is displayed notifying the user that an update is available. Initially, this displays once a day and automatically dismisses after 25 seconds. (I’ve blanked out our corporate branding in all these images)
We use a soft deadline – ie the update is never forced on the user. Enforcing compliance is handled by user communications and involvement from our local technicians. With one week left before the deadline, we increase the frequency of the notifications to twice per day.
If the deadline has passed, we take a more aggressive approach with the notifications, modifying the image and text, displaying it every 30 minutes and it doesn’t leave the screen unless the user actions or dismisses it.
The update can be installed via a shortcut on the desktop, or in the last notification it can be initiated from the notification itself.
Once triggered, a custom UI is displayed introducing the user to the update and what to expect.
When the user clicks Begin, we check that a power adapter is connected and no removable USB devices are attached – if they are, we prompt to the user to remove them first.
The update runs in three phases or stages – these correspond to the PreDownload, Install and Finalize commands on the update (more on that later). The progress of each stage is polled from the registry, as is the Setup Phase and Setup SubPhase.
Note that the user cannot cancel the update once it starts and this window will remain on the screen and on top of all other windows until the update is complete. The user can click the Hide me button, and this will shrink the window like so:
This little window also cannot be removed from the screen, but it can be moved around and is small enough to be unobtrusive. When the update has finished installing, or when the user clicks Restore, the main window will automatically display again and report the result of the update.
The colour scheme is based on Google’s material design, by the way.
If the update failed during the online phase, the user can still initiate the update from the desktop shortcut but toast notifications will no longer display as reminders. The idea is that IT can attempt to remediate the device and run the update again after.
If successful, the user can click Restart to restart the computer immediately. Then the offline phase of the upgrade runs, where you see the usual light blue screen and white Windows update text reporting that updates are being installed.
Once complete, the user will be brought back to the login screen, and we won’t bother them anymore.
If the update rolled back during the offline phase, we will detect this next time they log in and notify them one time:
Logging and Reporting
The entire update process is logged right from the start to a log file on the local machine. We also send ‘status messages’ at key points during the process and these find their way to an Azure SQL database which becomes the source for reporting on update progress across the estate (more on this later).
A PowerBI report gives visual indicators of update progress as well as a good amount of detail from each machine including update status, whether it passed or failed readiness checks and if failed, why, whether it passed the compatibility assessment, if it failed the assessment or the install we give the error code, whether any hard blocks were found, setup diag results (2004 onward), how long the update took to install and a bunch of other stuff we find useful.
Since 2004 though, we have starting inventorying certain registry keys using ConfigMgr to give us visibility of devices that won’t upgrade because of a Safeguard hold or other reason, so we can target the upgrade only to devices that aren’t reporting any known compatibility issues.
If a device performs a rollback, we can get it to upload key logs and registry key dumps to an Azure storage account where an administrator can remotely diagnose the issue.
How does it work?
Now lets dive into the process in more technical detail.
The update starts life with a simple PowerShell script that does the following:
Creates a local directory to use to cache content, scripts and logs etc
Temporarily stores some domain credentials in the registry of the local SYSTEM account as encrypted secure strings for accessing content from a ConfigMgr distribution point if necessary (more on this later)
Downloads a manifest file that contains a list of all files and file versions that need to be downloaded to run the update. These include scripts, dlls (for the UI), xml definition files for scheduled tasks etc
Each file is then downloaded to the cache directory from an Azure CDN
3 scheduled tasks are then registered on the client:
A ‘preparer’ task which runs prerequisite actions
A ‘file updater’ task which keeps local files up-to-date in case we wish to change something
A ‘content cleanup’ task which is responsible for cleaning up in the event the device gets upgraded through any means
A ‘status message’ is then sent as an http request, creating a new record for the device in the Azure SQL database
This script can be deployed through any method you wish, including Configuration Manager, Intune or just manually, however it should be run in SYSTEM context.
All content needed for the update process to run is put into a container in a storage account in Azure. This storage account is exposed via an Azure Content Delivery Network (CDN). This means that clients can get all the content they need directly from an internet location with minimal latency no matter where they are in the world.
Feature Update Files
The files for the feature update itself are the ESD file and WindowsUpdateBox.exe that Windows Update uses. You can get these files from Windows Update, WSUS, or as in our case, from Configuration Manager via WSUS. We simply download the feature updates to a deployment package in ConfigMgr and grab the content from there.
You could of course use an ISO image and run setup.exe, but the ESD files are somewhat smaller in size and are sufficient for purpose.
The ESD files are put into the Azure CDN so the client can download them from there, but we also allow the client the option to get the FU content from a local ConfigMgr distribution point if they are connected to the corporate network locally. Having this option allows considerably quicker content download. Since IIS on the distribution points is not open to anonymous authentication, we use the domain credentials stamped to the registry to access the DP and download the content directly from IIS (credentials are cleaned from the registry after use).
Similar to how ConfigMgr sends status message to a management point, this solution also send status messages at key points during the process. This works by using Azure Event Grid to receive the message sent from the client as an http request. The Event Grid sends the message to an Azure Function, and the Azure Function is responsible to update the Azure SQL database created for this purpose with the current upgrade status of the device. The reason for doing it this way is that sending an http request to Event Grid is very quick and doesn’t hold up the process. Event Grid forwards the message to the Azure Function and can retry the message in the case it can’t get through immediately (although I’ve never experienced any failures or dead-lettering in practice). The Azure Function uses a Managed Identity to access the SQL database, which means the SQL database never needs to be exposed outside of its sandbox in Azure, and no credentials are needed to update the database.
We then use PowerBI to report on the data in the database to give visibility of where in the process every device is, if there are any issues that need addressing and all the stats that are useful for understanding whether devices get content from Azure or a local DP, what their approximate bandwidth is, how long downloads took, whether they were wired or wireless, make and model, upgrade time etc.
After the initial deployment script has run, the entire upgrade process is driven by scheduled tasks on the client. The first task to run is the Preparation script and this attempts to run every hour until successful completion. This script does the following things:
Create the registry keys for the upgrade. These keys are stamped with the update progress and the results of the various actions such as pre-req checks, downloads etc. When we send a ‘status message’ we simply read these keys and send them on. Having progress stamped in the local registry is useful if we need to troubleshoot on the device directly.
Run readiness checks, such as
Checking for client OS
Checking disk space
Check for internet connectivity
Determine the approximate bandwidth to the Azure CDN and measure latency. This is done by downloading a 100MB file from the CDN and timing the download and using ‘psping.exe’ to measure latency. From this, we can calculate an approximate download time for the main ESD file.
Determine if the device is connected by wire or wireless
Determine if the device is connected to the corporate network
If the device is on the corporate network, we check latency to all the ConfigMgr distribution points to determine which one will be the best DP to get content from
Determine whether OS is business or consumer and which language. This helps us figure out which ESD file to use.
Download WindowsUpdateBox.exe and verify the hash
Download the feature update ESD file and verify the hash
Downloads of FU content is done using BITS transfers as this proved the most reliable method. Code is added to handle BITS transfer errors to add resilience.
Assuming all the above is done successfully, the Preparation task will be disabled and the PreDownload task created.
The purpose of this script is to run the equivalent of a compatibility assessment. When using the ESD file, this is done with the /PreDownload switch on WindowsUpdateBox.exe. Should the PreDownload fail, the error code will be logged to the registry. Since 2004, we also read the SetupDiag results and stamp these to the registry. We also check the Compat*.xml files to look for any hard blocks and if found, we log the details to the registry.
If the PreDownload failed, we change the schedule of the task to run twice a week. This allows for remediation to be performed on the device before attempting the PreDownload assessment again.
If the PreDownload succeeds, we disable the PreDownload task and create two new ones – a Notification task and an Upgrade task.
We also create a desktop shortcut that the user can use to initiate the upgrade.
The Notification script runs in the user context and displays toast notifications to notify the user that the upgrade is available, what the deadline is and how to upgrade, as already mentioned.
When the user clicks the desktop shortcut or the ‘Install now’ button on the toast notification, the upgrade is initiated. Because the upgrade needs to run with administrative privilege, the only thing the desktop shortcut and toast notification button does is to create an entry in the Application event log. The upgrade scheduled task is triggered when this event is created and the task runs in SYSTEM context. The UI is displayed in the user session with the help of the handy ServiceUI.exe from the MDT toolkit.
The user interface part of the upgrade is essentially a WPF application coded in PowerShell. The UI displays some basic upgrade information for the user and once they click ‘Begin’ we run the upgrade in 3 stages:
PreDownload. Even though we ran this already, we run again before installing just to make sure nothing has changed since, and it doesn’t take long to run.
Install. This uses the /Install switch on WindowsUpdateBox.exe and runs the main part of the online phase of the upgrade.
Finalize. This uses the /Finalize switch and finalizes the update in preparation for a computer restart.
The progress of each of these phases is tracked in the registry and displayed in the UI using progress bars. If there is an issue, we notify the user and IT can get involved to remediate.
If successful, the user can restart the computer immediately or a later point (though we discourage this!). We don’t stop the user from working while the upgrade is running in the online phase and we allow them to partially hide the upgrade window so the upgrade does not hinder user productivity (similar to how WUfB installs an update in the background.)
After the user restarts the computer, the usual Windows Update screens take over until the update has installed and the user is brought to the login screen again.
Drivers and Stuff
We had considered upgrading drivers and even apps with this process, as we did for the 1903 upgrade, however user experience was important for us and we didn’t want the upgrade to take any longer than necessary, so we decided not to chain anything onto the upgrade process itself but handle other things separately. That being said, because this is a custom solution it is perfectly possible to incorporate additional activities into it if desired.
In the event the that OS was rolled back during the offline phase, a scheduled task will run that will detect this and raise a toast notification to inform the user. We have a script that will gather logs and data from the device and upload it to a storage account in Azure where an administrator can remotely diagnose the issue. I plan to incorporate that as an automatic part of the process in a future version.
The solution creates an Updater scheduled task which runs once per day. The purpose of this task is to keep the solution up to date. If we want to change something in the process, add some code to a file or whatever is necessary, the Updater will take care of this.
It works by downloading a manifest file from the Azure CDN. This file contains all the files used by the solution with their current versions. If we update something, we upload the new files to the Azure storage account, purge them from the CDN and update the manifest file.
The Updater script will download the current manifest, detect that something has changed and download the required files to the device.
A Cleanup task is also created. When this task detects that the OS has been upgraded to the required version, it will remove all the scheduled tasks and cached content to leave no footprint on the device other than the log file and the registry keys.
You can find a generalised version of the code used in this solution in my Github repo as a reference. As mentioned before though, there are many working parts to the solution including the Azure services and I haven’t documented their configuration here.
The main benefit of this solution for us is that it is completely customised to our needs. Although it is relatively complex to create, it is also relatively easy to maintain as well as adapt the solution for new W10 versions. We do still take advantage of products like ConfigMgr to allow devices to get content from a local DP if they are corporate connected, and ConfigMgr / Update Compliance / Desktop Analytics for helping us determine device compatibility and ConfigMgr or Intune to actually get the deployment script to the device. We also make good use of Azure services for the status messages and the cloud database, as well as PowerBI for reporting. So the solution still utilizes existing Microsoft products while giving us the control and customisations that we need to provide a better upgrade experience for our users.
Following on from my previous post where I shared a PowerBI report that provides information on Windows 10 feature update blocks using Update Compliance and Desktop Analytics, in this post I will share another report that exposes similar data, but this time built from custom hardware inventory data in MEMCM.
Outside of the Windows setup process, feature update compatibility is assessed on a Windows 10 device by the ‘Microsoft Compatibility Appraiser’ – a scheduled task that runs daily and the results of which form part of the telemetry data that gets uploaded to Microsoft from the device. The specifics of this process are still somewhat shrouded in mystery, but thanks to the dedication of people like Adam Gross we can lift the lid a little bit on understanding compatibility assessment results. I highly recommend reading his blog if your interested in a more in-depth understanding.
Some compatibility data is stored in the registry under HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags, and in particular the subkey TargetVersionUpgradeExperienceIndicators contains useful compatibility information for different W10 releases such as the GatedBlockIds, otherwise known as the Safeguard Hold Ids, some of which are published by Microsoft on their Windows 10 Known Issues pages. Since the release of Windows 10 2004, many devices have been prevented from receiving a W10 feature update by these Safeguard holds, so reporting on them can be useful to identify which devices are affected by which blocks, and which devices are not affected and are candidates for upgrade.
By inventorying these registry keys with MEMCM I built the PowerBI report shown above where you can view data such as:
Which Safeguard holds are affecting which devices
Rank of model count
Rank of OS version count
The upgrade experience indicators (UE)
UE red block reasons
Note that this data alone doesn’t replace a solution like Desktop Analytics which can help identify devices with potential app or driver compatibility issues, but it’s certainly helpful with the Safeguard holds.
You can also use this data to build collections in MEMCM containing devices that are affected by a Safeguard hold. Because this is based on inventory data, when a Safeguard hold is released by Microsoft those devices will move naturally out those collections.
Understanding the data
Because of the lack of any public documentation around the compatibility appraiser process, we have to take (hopefully!) intelligent guesses as to what the data means.
Under the TargetVersionUpgradeExperienceIndicators registry key for example, you may find subkeys for 19H1, 20H1, 21H1 or even older Windows 10 versions. I haven’t found any keys for *H2 releases though, and I can only assume it’s because the Safeguard holds for a H1 release are the same for the H2 release. From the Windows 10 Known Issues documentation this seems to be the case.
There is also a UNV subkey – I assume that means Universal and contains data that applies across any feature update.
Under the *H1 keys (I suppose I should call it a branch, really) we can try to understand some of the main keys such as:
FailedPrereqs – I haven’t seen any devices yet that actually failed the appraiser’s prerequisites, but I assume the details would be logged here if they were.
AppraiserVersion, SdbVer, Version, DateVer* – I assume these indicate the version of the compatibility appraiser database used for the assessment
DataExp*, DataRel* – these seem to indicate the release and expiry dates for the Appraiser database so my assumption is a new one will be downloaded at or before expiry
GatedBlock* – the Id key in particular gives the Safeguard Hold Id/s that are blocking the device from upgrade
Perf – this appears to be a general assessment of the performance of the device. A low performing device will likely take longer to upgrade
UpgEx* – these seem to be a traffic-light rating for the ‘upgrade experience’. The UpgExU seems to stand for Upgrade Experience Usage – I don’t know what the difference between the two is. Green is good, right, so a green device is going to be a good upgrade experience, yellow or orange not so great, red is a blocker. I don’t know exactly what defines each colour other than that…
RedReason – if you’ve got a red device, it’s blocked from upgrade by something – but this isn’t related to Safeguard holds as far as I can tell. It seems to be related to the keys under HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\CompatMarkers\*H1, such as BlockedByBios, BlockedByCpu for example. The only one I’ve seen in practice is the SystemDriveTooFull block.
Configure Custom Hardware Inventory
Alright, so first we need to configure hardware inventory in MEMCM to include the registry keys we want to use. You can use the RegKey2Mof utility, or you can download the files below to update your Configuration.mof file and your Client Settings / hardware inventory classes. I’ll assume you are familiar with that process.
Open opening, you’ll need to add the SQL server and database name for your MEMCM database:
You won’t see any data until devices have started sending hardware inventory that includes the custom classes.
Note that I have included pages for both 20H1 and 21H1, but the latter is just a placeholder for now as no actual compatibility data will be available until that version is released, or close to.
You can also build collections like those shown above by adding a query rule and using the names created by the custom classes – in this case TwentyHOne and TwentyOneHOne. Use the value option to find which GatedBlockIds are present in your environment.
This morning I saw an interesting tweet from Sandy Zeng with a Log Analytics workbook she’d created for W10 feature updates based on Update Compliance data. I’d been meaning to create a similar report for that myself in PowerBI for some time, so I took inspiration from her tweet and got to work on something!
Microsoft’s Update Compliance solution can be used to report on software updates and feature updates status across your estate from Windows telemetry data. If you use Desktop Analytics, you can even combine the data for richer reporting.
Here’s a screenshot of what I ended up with. You can filter the data to view devices with Safeguard holds, for example, which since Windows 10 2004 has been a show stopper for many wanting to upgrade…
There are pages for 2004 and 20H2, as well as a page listing some of the known Safeguard hold IDs that have been publicly disclosed by Microsoft.
You can download this report for your own use with the links below. Note I have created two reports – the first assumes you have both Update Compliance and Desktop Analytics using the same Log Analytics workspace. If this is not the case for you, download the second report which doesn’t link to DA data and just uses Update Compliance. The only data I’ve included from DA is the make and model since these can be helpful in analysing devices affected by Safeguard holds.
I was working on a PowerShell project recently where I needed to reference a large number of items – more than I wanted to add as an array or hash table in script, or even an external file like an XML document. So I searched for a portable, standalone database that I could distribute with the script and happened across LiteDB, a lightweight, single-file NoSQL database. It seemed the ideal solution as there’s a .Net code library for it, and so far I’m pretty impressed.
In this blog I’ll cover some of the basics for working with a LiteDB database using PowerShell.
First you need to download the package from NuGet, then extract it. Grab the LiteDB.dll file from ..\lib\net45 folder and add it to your project.
Add the library to your PowerShell session:
Add-Type -Path "C:\Temp\LiteDB.dll"
Create a database (this can also be used to open the same database):