Open Source – Scheduled Task and PS Scheduled Job Management Pack for SCOM 2012 / 2016 / 2019

While I have retired from SCOM work, I have found a little bit of time to clean up the Visual Studio project and place the source code of the Scheduled Task MP on GitHub.

In case you’re using the Scheduled Task and PS Scheduled Job Management Pack, go to GitHub and get the source code as a VSAE project here: https://github.com/rafabu/SCOM-WinTaskSchedulerMP

The sealed version along with the MP guide is now also hosted on GitHub, neatly placed as release. The current one is still the unchanged version 1.2.0.500 you are probably already using if you’re reading this post.

Should you be having issues with the MP, please log them on GitHub or even better: Get involved yourself by contributing 💘 Thankfully Bob Cornelissen of TopQore has volunteered to help looking after the MP as a contributor. I am sure that it could not be in better hands.

GitHub_WindowsTaskScheduler

Terraform: Azure Desired State onboarding for Azure IaaS Windows VM

I was struggling a bit when enabling Azure Automation Desired State agent by means of Terraform (0.12). While the official documentation was sufficient to enable Azure Automation, configuring the actual LCM on Azure VMs (Windows based) needed a bit of fine tuning.

The goal was to be using the free tier (extension >= 2.70) and the updated default configuration properties of the extension as documented here: Default configuration script properties

Here’s the TF code for azurerm_virtual_machine_extension resource, making use of the Microsoft.Powershell.DSC Azure VM extension that worked for me (note to myself). It will put a Windows IaaS VM into Pull Mode, pointing at Azure Automation’s internal DSC pull server.

locals {
  dsc_mode             = "ApplyAndAutoCorrect"
  vm-name              = "myDSCedVM"
}

variable "dscaa-resource-group-name" {
  default = ""
  description = "Azure Automation azurerm_resource_group name"
}
variable "dscaa-account-name" {
  default = ""
  description = "Azure Automation azurerm_automation_account name"
}
variable "dscaa-server-endpoint" {
  default = ""
  description = "Azure Automation azurerm_automation_account endpoint URL"
}
variable "dscaa-access-key" {
  default = ""
 description = "Azure Automation azurerm_automation_account access key"
}

#NOTE: Node data must already exist - otherwise the extension will fail with 'No NodeConfiguration was found for the agent.'
resource "azurerm_virtual_machine_extension" "dsc_extension" {
  name                 = "Microsoft.Powershell.DSC"
  location             = "${var.location}"
  resource_group_name  = "${azurerm_resource_group.rg.name}"
  virtual_machine_name = "${local.vm-name}"
  publisher            = "Microsoft.Powershell"
  type                 = "DSC"
  type_handler_version = "2.77"
  auto_upgrade_minor_version = true
  depends_on = ["module.azurerm_virtual_machine_winrmenabled"]
  #use default extension properties as mentioned here:
  #https://docs.microsoft.com/en-us/azure/virtual-machines/extensions/dsc-template
  settings = <<SETTINGS_JSON
        {
            "configurationArguments": {
                "RegistrationUrl" : "${var.dscaa-server-endpoint}",
                "NodeConfigurationName" : "rootConfig.${local.vm-name}",
                "ConfigurationMode": "${local.dsc_mode}",
                "RefreshFrequencyMins": 30,
                "ConfigurationModeFrequencyMins": 15,
                "RebootNodeIfNeeded": false,
                "ActionAfterReboot": "continueConfiguration",
                "AllowModuleOverwrite": true

            }
        }
  SETTINGS_JSON
  protected_settings = <<PROTECTED_SETTINGS_JSON
    {
        "configurationArguments": {
                "RegistrationKey": {
                    "userName": "NOT_USED",
                    "Password": "${var.dscaa-access-key}"
                }
        }
    }
  PROTECTED_SETTINGS_JSON
}


 

Note: A corresponding Azure Automation Account configuration must exist. This involves the following terraform resources:

Regarding the compilation of the DSC configuration. This may either be triggered on the Azure Automation platform but the solution becomes much more flexible if an offline MOF creation pipeline is in place. In that case, the resource azurerm_automation_dsc_nodeconfiguration can be useful.

Credits go to Sage McEnry’s post on the same topic which helped a lot with getting started.
 

PKI Certificate MP for SCOM – Open Source

While I have mostly retired from SCOM work, I have decided to finally make the most used of my SCOM Management Packs open source.

In case you’re still using the PKI Certificate Management Pack, head over to our beloved GitHub and get the VSAE project here: https://github.com/rafabu/SCOM-PKICertificateMP

Please get involved and get in touch with me in case you feel like contributing or maybe even taking over.

Error sealing Management Pack using VSAE – The “PackageToBundle” task failed unexpectedly.

I cannot count the hours I have lost initially – later, every time that tiny little error appeared, I would remember that the solution had been easy. Time to finally write a note to myself!

When sealing a SCOM Management Pack Solution using Visual Studio Authoring Extensions, the following error would appear in the output window:

The "PackageToBundle" task failed unexpectedly.
Microsoft.Deployment.WindowsInstaller.BadQuerySyntaxException: SQL query syntax invalid or unsupported.
   at Microsoft.Deployment.WindowsInstaller.Database.OpenView(String sqlFormat, Object[] args)
   at Microsoft.EnterpriseManagement.Packaging.DataAccess.Insert(String query, Object[] args)
      at Microsoft.Build.BackEnd.TaskBuilder.d__26.MoveNext()
Done building project "my.ConnectedMG.Library.mpproj" -- FAILED.

The solution in my case: Check each Management Packs’ LanguagePack. I would get above error when using certain special characters in the Name or Description element (e.g. single quotes). After removing those from the LanguagePack, the MP seals just nicely.

packagetobundletask_failed

Which PowerShell version is my SCOM agent using (or why I love SCOM 2012 R2 Update Rollup 7)

I have no idea how often I have used PowerShell scripts to build or extend SCOM Management Packs. Almost daily it must have been. Occasionally I stumbled over PowerShell version issues because the SCOM agent had its own logic of choosing a PowerShell version. To circumvent these, I basically limited myself to write (and test!!!) my scripts on PowerShell 2.0 and .NET 2.0 environments.

However; the world changed recently with the release of SCOM 2012 R2 UR7 and SCOM 2012 SP1 UR10. In the KB article the fixes are listed. Among many it says:

CLR load order change

The current behavior for agents is to choose a CLR version based on the operating system version. For Windows Server 2012 and newer, the .NET Framework 4.0 is loaded. For operating systems older than Windows Server 2012, the .NET Framework 2.0 family is loaded. On management servers, the .NET Framework 2.0 family is loaded. This essentially maps the .NET Framework version used to the version available out-of-box on the server. The problem with the current behavior is that even if the Management Pack author knows that .NET Framework 4.0 is present on the system, it cannot be used.

In the new behavior, the agent loads the .NET Framework 4.0 if it is available else it falls back to the .NET Framework 2.0.

What this do not easily reveal: Since .NET Framework 4.0 is now chosen, the SCOM agent and SCOM management server can use PowerShell 4.0 or 5.0. That is fantastic news. It means the SCOM agent (Microsoft Monitoring Agent respectively HealthService) is now making use of the most recent PowerShell version installed on an OS.

 

SCOM PowerShell module behaviour

When using the Windows Library’s PowerShell modules (e.g. Microsoft.Windows.PowerShellProbe), the following logic applies:

SCOM 2012 R2

Machine Type

<= 7.1.10218.0 (UR 6)

>= 7.1.10229.0  (UR 7)

Management Server / Gateway

PS 3.0 if installed; otherwise PS 2.0

PS 4.0 or 5.0 (if .NET 4.0 and PS >= 4.0 are installed)

PS 2.0 (if .NET 4.0 or PS >= 4.0 are not installed)

Agent

PS 2.0 / .NET 2.0

Note on legacy Windows versions: PowerShell 2.0 is the minimum requirement to use the agent’s PowerShell modules. And PowerShell is always installed or upgraded as part of the “Windows Management Framework”.

To verify that newer PowerShell versions are used after upgrading to SOM 2012 R2 UR 7, I wrote a simple MP with agent tasks. The following table shows the sample output of a Windows 2008 R2 system, having WMF 5.0 preview installed:

before UR7

with UR 7

 

Operating
System: Microsoft Windows Server 2008 R2 Enterprise

PowerShell
Host: OpsMgr PowerShell Host

Powershell Version: 2.0

Common
Language Runtime .NET: 2.0.50727.5485

 

 

Operating
System: Microsoft Windows Server 2008 R2 Enterprise

PowerShell
Host: OpsMgr PowerShell Host

Powershell Version: 5.0.10105.0

Common
Language Runtime .NET: 4.0.30319.34209

 

 

So at last I can start writing PowerShell workflows for SCOM in 4.0 or 5.0 manner – provided I can guarantee that every agent has UR7 or later installed!

 

Using PowerShell.exe as an alternative (System.CommandExecuterProbe)

When you look closely at the output of the table above, you can see that the PowerShell host is “OpsMgr PowerShell Host”. SCOM PowerShell modules are not running on Windows’ native PowerShell host (ConsoleHost). Instead a SCOM internal host is used, that is optimized for agent use. Occasionally this can lead to issues with very advanced PowerShell workflows.

If this happens, and only then (!) the following approach may be used. Instead of using the native SCOM PowerShell modules, the System.CommandExecuterProbe can call PowerShell.exe. However; you have to make sure, the execution policy for our workflow is set no stricter than “RemoteSigned”. Otherwise the script will not be run.

Examples for this are found in SharePoint MPs or were used back in SCOM 2007 days to run PowerShell scripts. Or have a look at my own sample MP linked at the end of this post.

 

PowerShell Environment Test Tasks (Management Pack)

The management pack linked below contains three diagnostic tasks, targeted at the HealthService object.

  • _Diagnose PowerShell Environment (PowerShellProbe)
  • _Diagnose PowerShell Environment (CommandExecuterProbe)
  • _Diagnose PowerShell Environment (CommandExecuterProbe with -version override)

They do provide output on the PowerShell environment used by the SCOM workflows. Essentially the objects $PSVersionTable and $Host are being reported.

PowerShell_TaskI am including both the unsealed pack and the VSAE project as a reference. Happy MP authoring.

Links

List of all update rollup packages for the SCOM 2012 family

How to apply SCOM 2012 R2 UR 7 (by Kevin Holman)

Download: Check PowerShell Version Management Pack (XML)

Download: Check PowerShell Version Management Pack (VSAE Project archive)

 

 

PKI Certificate Verification MP – Update 1.3.0.0

In the last 10 months I have added several minor features to the certificate verification management pack for SCOM 2012. Some of them rose from needs of clients but most were suggested by people using the MP. Many thanks for everyone providing feedback and especially for testing the pre-release builds.

You can head over to the SystemCenterCentral.com management pack calalog and download the updated MP version 1.3.0.0.

Changes in this update

The quick bullet list of improvements as also listed in the MP guide looks like this:

  • Added Tasks: Archive Certificate, List Certificate Properties, Disable/Enable Monitoring, Rediscover
  • Added Recoveries: Archive Certificate, Disable Monitoring
  • Added Discovery: Web Hosting certificate store (Server 2012 / 2012 R2)
  • Additional certificate property: Certificate Template. It is also listed on reports.
  • Discovery filter expanded to certificate template.
  • Alert description: Additional details on the certificate chain and SCOM action account used.
  • CRL Lifetime Monitor: Threshold is exposed as an overridable parameter.
  • CRL health roll up monitor added.
  • Expiring certificate view & report: Default threshold of 1 month may be overridden.
  • Views: Changed criteria on views to make them more reliable when using user scopes.
  • Reporting bug: Certificate inventory did not list all certificates.
  • Additional MP: Rediscovery Tasks. Immediate trigger of store content discovery after archive, disable/enable or rediscover tasks

And a few extra words on some of them

While some changes are pretty much self explanatory, I am going to give you some background information on others below. Plus two little PowerShell bits to make everyone’s lives a bit easier.

Added tasks

To make checking on the validity of certificates easier for SCOM operators, I have added agent tasks that will

  • list certificate properties (and check validity)
  • archive a certificate
  • disable respectively re-enable monitoring for a single certificate (append friendly name tag)
  • attempt immediate rediscovery

Archiving a certificate may obviously have a disastrous impact on payloads working with a certificate. However; it may be undone locally on an agent computer using some PowerShell code as in the following example:

#sample script to remove the ARCHIVED flag from certificates
#open personal computer store (read/write & include archived)
$store = New-Object  System.Security.Cryptography.X509Certificates.X509Store "My","LocalMachine"
$storeOpenFlags = [System.Security.Cryptography.X509Certificates.OpenFlags]"ReadWrite, IncludeArchived"

$unarchive = New-Object System.Management.Automation.Host.ChoiceDescription "&Yes", "Set Archive Flag to 'false'"
$skip = New-Object System.Management.Automation.Host.ChoiceDescription "&No", "Keep archive flag"
$confOpt = [System.Management.Automation.Host.ChoiceDescription[]]($unarchive, $skip)

$store.Open($storeOpenFlags)
if ($store.StoreHandle) {
    Write-Output ("Listing ARCHIVED certificates in store " + $store.Name)
    $store.Certificates | where {$_.Archived -eq $true} | % {
        $cert = $_
        $cert | ft Thumbprint, Subject, Issuer
        switch ($host.ui.PromptForChoice("Unarchive", "Remove 'ARCHIVED' flag from certificate?", $confOpt, 1))
        { 
            0 {    $cert.set_Archived($false)
                if ($cert.Archived -eq $false) {Write-Output ("SUCCESS: Removal of ARCHIVED flag on certificate with thumbprint " + $cert.Thumbprint + " succeeded")}
                else { Write-Output ("ERROR: Removing ARCHIVED flag on certificate with thumbprint " + $cert.Thumbprint + " failed"); continue }
            }
        }        
    }
}
else { Write-Output ("Failed to open store " + $store.Name) }

#don't forget to close store after operation
$store.Close()

Since there’s no undo option for deleted certificates, the MP does not contain a task to delete certificates.

 

Disabling monitoring for a single certificate is achieved by appending the tag “_DoNotMonitor” to the end of a certificate’s friendly name. Such certificates are then becoming members of a group and monitoring will stop by means of predefined overrides. Instead of using the SCOM task, a system administrator might simply use the certmgr snap-in to alter the friendly name.

Tag friendly name

Tag friendly name to disable monitoring

 

The rediscovery task will only be available, if the additional MP ReDiscoveryTasks has been imported. It will run a management server side PowerShell script, triggering the SCOM task “Trigger On Demand Discovery Task“. Overrides will be accounted for. Such, changes to an agent’s certificate store will be reflected much more quickly in SCOM. The progress can easily be verified by checking the “Task Status” view.  Note that the ReDiscoveryTasks MP will also attempt to run an OnDemand discovery after Archive and Enable/Disable tasks have been performed.

Added Recoveries

Two disabled recoveries have been added to the certificate monitors:

  • Archive Certificate
  • Disable Monitoring

They do behave exactly the same as the related tasks mentioned above. They will show their effect on the SCOM repository within minutes if the optional ReDiscoveryTasks MP is imported. Otherwise monitoring will not stop until the next discovery cycle has passed.

 

Added discovery of the Web Hosting store

Windows Server 2012 and 2012 R2 feature a new certificate store called “Web Hosting”. Behind the scenes it provides more load efficient certificate handling and is thought to be used e.g. with IIS in web hosting scenarios (what a surprise). When using the QuickStart override MP, those stores and all signed certificates in them will automatically be discovered.

Monitoring wise it is important to understand that this store is thought to hold hundreds of certificates. Hence thinking about a good filtering strategy for monitoring may crucial. Failing to do so could lead to way too many objects being discovered, causing performance issues on SCOM. Consider making use of the Subject, Issuer and Template filtering capabilities.

 

Certificate Template information

Certificates issued by Enterprise CAs can now be filtered based on their template name (and OID). Consequently, the template property has been added as an additional property to the certificate and will be listed on inventory reports.

It is important to understand that the names of certificate templates are specific to an AD forest. They cannot be resolved if such a certificates is located on a computer outside of the forest. In that case, the MP will display the template OID value only. To check what the template property of a certificate is, run the following PowerShell commands on an agent computer:

#gets template names (and/or OIDs) of local certificates
get-childitem cert:\LocalMachine\My | ft -Wrap Thumbprint, Subject, Issuer, @{Label="Template"; Expression={
	$templateName = ""
	#TemplateName (Version 1)
	$_.Extensions | where { $_.OID.Value -match "^1\.3\.6\.1\.4\.1\.311\.20\.2$"} | % {
		$templateName = $_.Format($false).trim()}
	#Template (Version 2)
	$_.Extensions | where { $_.OID.Value -match "^1\.3\.6\.1\.4\.1\.311\.21\.7$"} | % {
		#sometimes no actual name but only the OID is contained
		($_.Format($false)) -match 'Template=((?.+)\((?1\.3\.6\.1\.4\.1\.311\.[0-9.]+)\)|(?1\.3\.6\.1\.4\.1\.311\.[0-9.]+))' | Out-Null
		if ($matches.templateName) {$templateName = ($matches.templateName.trim() + "(" + $matches.templateOID.trim() + ")")}
		else {$templateName = $matches.templateOID.trim()}}
	$templateName
}}

 

Updated Alert Description

The alert description of monitors has been overhauled to be much more verbose. Specifically it is now showing both the subject and the issuer property. It also contains information about the RunAs account used for monitoring and gives details on the validity of the certificate chain.

Alert Description V1.3.0.0

Alert description example: intermediate CA certificate has expired (Level 1)

 

Such it has become much easier to pinpoint issues when not the end certificate itself but a chain certificate has become invalid; possibly just in the context of the RunAs account of the agent.

An extra hint regarding the latter in case the agent is running under the local system account: Occasionally it can be observed that a certificate chain is reported invalid by the MP but is valid if a certificate is checked with an interactive user account. One possible reason for this can be the SYSTEM account not being able to automatically update CTLs (e.g. due to internet access restriction), while an interactive account can. If the issue cannot be resolved, consider using a tool like SysInternals PSExec.exe to launch the CertMgr.msc in the SYSTEM context to investigate the cause.

Download MP version 1.3.0.0 from SystemCenterCentral.com

Scheduled Task and PS Scheduled Job Management Pack – 1.2.0.500

And now for the task scheduler: Today’s update has been on my to-do list for a very long time. Same story as with the certificate MP: I have completely re-written the scheduled task MP based on PowerShell, aiming to fully support SCOM 2012 – 2019 and Windows 2012 – 2019.

MP change history
  • SCOM >= 2012
  • Windows Server >= 2008, 2008 R2, 2012 and 2012 R2 support
  • PowerShell Scheduled Job support added (PoSh 3.0 and later).
  • Timing is mostly event-driven: significantly reduce the number of script calls required on the average agent system.
  • Verbose filters to adjust task discovery.
  • Disabled root discovery out-of-the-box (quick start MP included).
  • Disabled advanced monitors and rules out-of-the-box.
Configuration

Please read the included MP guide and the release notes carefully. Especially if you are upgrading from the previous (SCOM 2007) MP version.

Please make sure that PowerShell >=2.0 is installed on all agents that require task scheduler monitoring.

Previous versions of this MP had the root discoveries enabled. As this was changed to disabled, you will need to configure re-enable overrides to the required discoveries plus advanced monitors if you were relying on them.

 

Download (from GitHub)

Download: Scheduled Task MP 1.2.0.500 (SCOM 2012 – 2019)

  • SHA-1: 181f245b04b31eeb4afa8594eac5dec84d8e2730

Download: Scheduled Task MP 1.1.1.1 (legacy SCOM 2007). Note that this version is no longer being developed.

  • SHA-1: dcba000dff5115fe89508c7b21b90bf39c0fe9bf

Task Scheduler

PKI Certificate Verification MP – Update 1.2.1.3

The 1st update to the rewritten certificate management pack is ready. The update to 1.2.1.3 is mostly about more powerful filtering options for the certificate discoveries. It is now possible to use regular expressions to:

  • Include / Exclude based on “Subject”
  • Include / Exclude based on “Issuer”
  • Exclude based on “Enhanced Key Usage” OIDs

Note that the filters will have to be based on the exact string output of the certificate objects as presented in PowerShell. Hence check those before attempting to write RegEx filters using:

 ls cert:\LocalMachine\My | fl Subject, Issuer

All characters (including blanks) are being taken into account. The discovery filters are using .NET RegEx expression syntax. Please test your expressions using a suitabe tool before using them for your overrides (I am often using Regex Hero but there are plenty of other options out there).

Once store discovery is enabled, the default filter settings of the MP will discover any certificates with the exception of self-signed and MS NAP ones. Refer to the MP guide and the release notes if you plan to make use of the advanced filter options. And remember to override the store discovery, not the certificate one.

Download

Find the Management Pack at its home on System Center Central:

PKI Certificate Verification MP at SystemCenterCentral.com MP Catalog

PKI Certificate Verification Management Pack Update – 1.2.0.210

Many years have passed since I first published the certificate MP back in summer 2009. Almost 5 years(!) later this management pack still fills a gap by keeping an eye on PKI certificates installed locally in servers’ certificate stores. Certainly about time for an update.

Today I am able to release a major update – a complete re-write rather – of the PKI Certificate Verification MP. It is hosted over at SystemCenterCentral.com in the MP Catalog.

MP change history
  • SCOM 2012 / 2012 R2 support only (the legacy MP 1.0.1.20 is still available for use on SCOM 2007).
  • main monitoring script now uses PowerShell instead of VB Script, making it compatible with any system locale and easier to maintain.
  • new, advanced certificate verification flag overrides
  • dashboard view
Some extra words on the effort

The main aim with this update is to make the MP’s code easier to maintain. Hence I first recreated the entire MP as a Visual Studio project with the Authoring Extensions. This involves taking apart the MP’s elements, adding each one as a separate item to a VS project structure. Next I started writing a new discovery and monitoring script based on PowerShell. This script does most of the work by essentially enumerating certificates and certificate revocation lists in local certificate stores. Due to limitations in PowerShell regarding CRLs and alternate certificate stores, this script got rather complex. No chance of getting away with something easy and straight forward as ‘ls cert:\LocalMachine’. With the first CRLs getting discovered, tests, more tests, some extra testing plus updating the documentation were left.

While I did not clock the hours, the update kept me busy in much of my spare and commuting time during the last 4 months. And I must mention everybody helping with code samples, advise, by testing and reviewing.  Pete, Vadim, Marc, Joel, Bob, Dan, Marnix, Stan, Tao and Dirk – this wouldn’t be here today without your help!

Certificate MP in VSAE

MP Solution opened in Visual Studio

SQL Server Mirroring MP Update V 1.0.2.5

Today I was able to release an update to the SQL Server DB Mirroring Management Pack.

The update doesn’t bring any new features but fixes a potential issue on larger management groups which would lead to discoveries failing. In order to work around this I have altered the Powershell discovery scripts such that they do no longer make use of any SCOM console cmdlets. Instead direct SDK .NET calls are being used. I recommend to update existing installations specifically if you are using management group connectors, the Exchange 2010 correlation engine or other custom MPs which make use of SDK workflows.

Download of the management pack is available trough the SystemCenterCentral.com MP Catalog. You need to register but registration is free.

MP Change History

Version 1.0.2.5 – April, 28 2011 (no functional changes)

  • Corrected spelling in language pack.
  • Replaced all SCOM shell cmdlets with .NET calls to overcome a potential issue when several connectors and other MPs share a single SDK connection.
  • Improved discovery script timing behaviour when using desired configuration for a large number of mirrors.

Version 1.0.1.72 – September, 10 2010

  • Support for SQL Server 2005 has been added and compatibility with SQL Server 2008 R2 has been verified. Starting with version 1.0.1.72, Microsoft SQL Server 2005, 2008 and 2008 R2 are supported.
  • Default Display Names of discovered DB Mirror Groups have been shortened and may optionally be changed to a customized string. See the guide for details.

Version 1.0.1.35 – June, 04 2010

  • Original release – SQL Server 2008 support only