Sick of explaining to end users why they didn’t get a SCOM disk monitor alert?

Are you sick and tired of explaining to end users why they didn’t get a disk space alert? The built in SCOM disk space monitors are way too complicated. There are thresholds for warning, critical, system drive, and non-system drive.  Then you must hit both percentage and megabyte thresholds. You can’t alert on warning because you won’t get a critical alert. What a mess!

Want to simplify your life? I have created two simple disk space monitors.

First: How to install them.

Download the MP here: https://www.scom2k7.com//downloads/Custom.Windows.Disk.Monitoring.zip

Import the Management Pack; Custom.Windows.Disk.Monitoring.mp  **Note: these require PowerShell to be installed on all your agent managed systems.  By default SCOM 2016 requires PowerShell on all systems.**

 

Next: Disable the built in monitors: Create overrides to disable the built in monitors.

 

 

That’s it.  Now end users will get an warning alert at 10% disk free space and a critical alert at 5% disk free space.  Simplicity Is Genius

 

Want to make your life even easier?   Check out the SCOM Maintenance Mode Scheduler

 

The Deets (Details)

There are two new disk space monitors.  They work by collecting performance counters using the native modules.  When it detects 3 consecutive perfmon samples  over the threshold of 95% for the critical monitor (90% for the warning monitor) at a 20 minute interval.

If the drive is over the threshold for 60 minutes it will then launch a PowerShell script to clean up the percentage provided from perfmon from to a more readable number.  It will also query the drive for remaining GB free and the total size of the disk.  It displays this in the details of the alert.  This is much nicer then the built in ones that don’t tell you the size of the drive.

You can modify how these work by creating overrides.  You will need to create an overrides for each of the different classes.  Windows Server 2008 Logical disk, Windows Server 2012 Logical disk etc..

If you want to still have a separate threshold for your system drives.  You can simply create a dynamic group containing your c: drives.

 

Group Members

 

 

Then create overrides for the group.

Theses monitors were built using Kevin Holman’s excellent SCOM Management Pack VSAE Fragment Library. Monitor.Performance.ConsecSamples.ThenScript.TwoState.mpx

 

Please let me know if you find any bugs.  timmcfadden@scom2k7.com.com

 

Continue Reading

Monitoring Windows Server 2003 with SCOM 2016

First let me say this isn’t supported. But then again Windows Server 2003 is no longer supported anyways. Even if you have a Custom Support agreement, that just gives you the latest security hotfixes. Not support to monitor it with SCOM.

I have been to many customers recently that want to upgrade to SCOM 2016 but can’t because they have a significant number of systems still running Server 2003.

One option is to setup a new environment running SCOM 2012 R2 and move the old 2003 servers to the new environment. Then upgrade the existing environment with the majority of their servers to SCOM 2016. This creates more overhead keeping the two systems running and up-to-date.

Getting the Agents Working

You cannot install the SCOM 2016 agent on SCOM 2003 as the installer will fail.  But you likely already have a SCOM 2012 R2 Agent installed.  The differences in the SCOM Agent from SCOM 2007 to SCOM 2012 were major as we changed from XML 1.1 to XML 2.0 and included MP Binaries.  The changes from SCOM 2012 R2 to SCOM 2016 are relatively minor.

So, all we need to do is point the SCOM 2012 R2 Agent at SCOM 2016, right?  Maybe.

In my testing a Server 2003 system with a SCOM 2012 Agent, pointing at a SCOM 2016 management server went green but then went grey.

Why is this happening?  Many of the new management packs in SCOM 2016 require PowerShell.  By default, PowerShell is not part of the Windows Server 2003 OS.  To fix this follow these steps.

 

First we need to get WMI working.   Apply these Server 2003 hotfixes from Kevin’s list of recommended hotfixes.

932370 The number of physical hyperthreading-enabled processors or the number of physical multicore processors is incorrectly reported in Windows Server 2003
933061 WMI Stability in Server 2003
955360 Cscript 5.7 update for Server 2003

 

Then we first need .net 2.0 SP1 as it’s a prerequisite for PowerShell.

Download x64 Here:  https://www.microsoft.com/en-us/download/details.aspx?id=6041

Download x86 Here” https://www.microsoft.com/en-us/download/details.aspx?id=16614

 

Lastly we need to install the Windows Management Framework Core for Windows Server 2003

Download x64 Here: http://www.microsoft.com/downloads/details.aspx?familyid=909bbcf1-bd78-4e03-8c83-69434717e551

Download x86 Here: http://www.microsoft.com/downloads/details.aspx?familyid=f002462b-c8f2-417a-92a3-287f5f81407e

 

 

Now I can see the Windows Server 2003 Server is being monitored.

You may also want to disable the “Microsoft System Center Advisor Monitoring Server Opt-In/Out Rule” on these 2003 Systems as I have seen this fail.

Another helpful tool is to install Kevin Holman’s Agent Management Pack. https://blogs.technet.microsoft.com/kevinholman/2017/05/09/agent-management-pack-making-a-scom-admins-life-a-little-easier/

This management pack will help you identify which of your 2003 Systems are missing PowerShell.

 

Continue Reading

SQL Dashboards not populating with Event 26319

Custom with a large environment (7000+) agents was having issues with the SQL Dashboards not populating.  We checked the event log and came across this error.

 

Log Name: Operations Manager
Source: OpsMgr SDK Service
Event ID: 26319
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Description:
An exception was thrown while processing GetDataWarehouseStoredProcedureResult for session ID
Exception message: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
Timed out stored procedure: sdk.Microsoft_SQLServer_Visualization_Library_GetDataCenterDashboardData
Full Exception: Microsoft.EnterpriseManagement.DataWarehouse.DataAccess.SqlTimeoutException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
Timed out stored procedure: sdk.Microsoft_SQLServer_Visualization_Library_GetDataCenterDashboardData —> System.Data.SqlClient.SqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. —> System.ComponentModel.Win32Exception: The wait operation timed out

 

The fix:

When used rarely or after a long break, the dashboards may work rather slowly due to large amounts of the collected data to be processed; especially, it is related to large environments (2000+ objects).

Run this script in SQL against your DataWarehouse

USE [OperationsManagerDW]
EXECUTE [sdk].[Microsoft_SQLServer_Visualization_Library_UpdateLastValues]
EXECUTE [sdk].[Microsoft_SQLServer_Visualization_Library_UpdateHierarchy]

 

Continue Reading

SCOM 2016 Maintenance Mode Scheduler

The SCOM 2016 Maintenance Mode Scheduler application makes enabling and scheduling maintenance mode simple for all users. Your IT staff will no longer have a reason why a server wasn’t put into maintenance mode.

The application is an intuitive HTML5 interface written specifically for SCOM 2016 and SCOM 1801 from the ground up. It deeply integrates into SCOM 2016 using all the new native maintenance mode scheduling capabilities. All your IT staff can quickly schedule maintenance mode without having to install and use the slow console.

Features:

  • Easily access the new web based maintenance mode scheduler from any browser (Chrome, FireFox, Safari and IE)
  • Schedule Windows/Unix Computers, Groups, and Objects like SQL Databases for maintenance in a few seconds.
  • Import a CSV containing a list of computers or groups for scheduled maintenance.
  • Instant MM: Server and application admins can instantly place servers into maintenance during reboots and maintenance windows directly from the server without requiring admin privileges to the server.
  • Instant MM can be called by a VB, PowerShell, or code from anywhere. This can be used to start and stop maintenance on any server for a specified amount of time. This makes it easy to integrate into your current change or software management process.

Download

Additional Features:

  • Integrated Dashboard works in the SCOM Console.
  • Instant Maintenance Mode with URL Parameters.  Specify URL parameters ComputerName, ID, Min, Action to control maintenance mode.
  • Manage page will make it easy to see and manage any upcoming maintenance windows and identify any gaps
  • Uses the same role based permissions that SCOM 2016 uses. Users will only be able to schedule maintenance mode for servers, groups, or objects they already have access to in SCOM.

Schedule Windows Computers

Instant MM

This solution makes it easy for IT staff to put a server into maintenance mode without having to go to the SCOM console. On any server, the administrator can visit the SCOM 2016 Maintenance Mode Scheduler Instant MM website at http://yourMSserver/MMWeb/InstantMM.aspx

Create a shortcut on the desktop of the servers to make it even easier. This can be accomplished manually or by use SCCM, AD Group Policy or some other software deployment software

Integrated Dashboard

Instant Maintenance Mode with URL Parameters

New in SCOM Maintenance Mode Scheduler 2016 is the ability to add parameters to the URL.

  • ComputerName – Specify the Computer Name
    http://yourMMServer/MMweb/Instant.aspx?ComputerName=yourComputer.yourdomain.com
  • ID: – Specify the ID of any object in SCOM
    http://om01/MMweb/InstantMM.aspx?ID=f4ea533c-16e4-76fa-8f5a-98fc4b1cb492
  • Min – Number of Minutes for Maintenance Mode
    http://om01/MMweb/InstantMM.aspx?Min=120
  • Action – Start or Stop Maintenance Mode
    http://om01/MMweb/InstantMM.aspx?Action=Start
    http://om01/MMweb/InstantMM.aspx?Action=Stop
  • Combine multiple URL Parameters
    http://om01/MMweb/InstantMM.aspx?ComputerName=DB02.scom2k16.com&Min=120&Action=Start

Instant Maintenance Mode using PowerShell, VBScript or Code.

Using the new URL Parameters, you can now put servers into Maintenance Mode from any computer using a script or code.  Typical use case would be using SCCM when updates or software is installed.  SCCM would execute the VB or PowerShell script before the install process to start maintenance mode.  After the updates or software is installed SCCM would call the script to stop maintenance mode.

VB Script Example    Download: https://www.scom2k7.com/downloads/computerMM.renameTOvbs

Dim o, mmServer, computerName, min, action, fullURL

if WScript.Arguments.Count < 4 then
    WScript.Echo "Missing parameters"
end If

mmServer = WScript.Arguments(0)
computerName = WScript.Arguments(1)
min = WScript.Arguments(2)
action = WScript.Arguments(3)

Set o = CreateObject("MSXML2.XMLHTTP")
'Example http://om01/MMweb/InstantMM.aspx?ComputerName=DB02.scom2k16.com&Min=120&Action=Start
fullURL = "http://" & mmServer & "/MMweb/instantMM.aspx?ComputerName=" & computerName & "&Min=" & min & "&Action=" & action
WScript.Echo fullURL

o.open "GET", fullURL, False
o.sen

 

PowerShell Example     Download: https://www.scom2k7.com/downloads/computerMM.renameTOps1

param (
    [Parameter(Mandatory=$true)][string]$mmServer,
    [Parameter(Mandatory=$true)][string]$computerName,
    [Parameter(Mandatory=$true)][string]$min,
    [Parameter(Mandatory=$true)][string]$action
 )

$FullURL = "http://" + $mmServer + "/MMweb/InstantMM.aspx?ComputerName=" + $computerName + "&Min=" + $min + "&Action=" + $action
$FullURL

Invoke-Webrequest -uri $FullURL -UseDefaultCredential

Any problems, bugs, or issues please e-mail:  support@scom2k7.com.com

Continue Reading

SCOM 2016 Scheduled Maintenance Mode won’t run if you picked a different database name during install

There is a bug in SCOM 2016 where if you choose a different Operations Manager Database Name during install, schedule maintenance mode won’t work!

The Jobs will run but nothing is put into Maintenance Mode. You won’t get an error in SCOM or in the event log.

As you can see below I named my “OperationsManager” Database “SCOMDB”

The stored procedure has ‘OperationsManager’ hard coded as the Database name so it fails!

The workaround is to change the stored procedure that creates these Schedules.

The stored procedure name is p_MaintenanceScheduleCreate

Right click Modify.

Find the line

	SET @SQLCommand = CONCAT('EXEC OperationsManager.dbo.p_MaintenanceScheduleJobStep ''', @ScheduleId, '''');

 

and change OperationsManager to your database name and click Execute

You will have to delete all your Schedules in the console.

Now they run without issue.

This is a known bug and will be fixed in SCOM 2016 UR3.

 

 

 

 

Continue Reading

Identify SCOM Agent Version from the file system on SCOM 2012 R2 and SCOM 2016 Agents

When writing my SCOM Agent Update MP  I needed a way to get the SCOM Agent version from the file system.  There is one file that gets updated in every Update Rollup.  OMAgentTraceTMFVer.Dll that is located where in the \Tools\TMF directory where you installed the scom agent.  In my case C:\Program Files\Microsoft Monitoring Agent\Agent\Tools\TMF\OMAgentTraceTMFVer.Dll

I wrote this script to read the file version of this file to identify agent installed version.

Here is a similar VBscript that will pull that data. It works with SCOM 2012 R2 and SCOM 2016

Set objShell = WScript.CreateObject("WScript.Shell")
sngVersion = objShell.RegRead("HKLM\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Setup\InstallDirectory")

Set objFSO = CreateObject("Scripting.FileSystemObject")
fileVer = objFSO.GetFileVersion(sngVersion & "Tools\TMF\" & "OMAgentTraceTMFVer.Dll")


select case fileVer
   case "7.1.10184.0"
      urVersion = "RTM"
      agentVersion = "2012 R2"
   case "7.1.10195.0"
      urVersion = "UR2"
      agentVersion = "2012 R2"
   case "7.1.10204.0"
      urVersion = "UR3"
      agentVersion = "2012 R2"
   case "7.1.10211.0"
      urVersion = "UR4"
      agentVersion = "2012 R2"
   case "7.1.10213.0"
      urVersion = "UR5"
      agentVersion = "2012 R2"
   case "7.1.10218.0"
      urVersion = "UR6"
      agentVersion = "2012 R2"
   case "7.1.10229.0"
      urVersion = "UR7"
      agentVersion = "2012 R2"
   case "7.1.10241.0"
      urVersion = "UR8"
      agentVersion = "2012 R2"
   case "7.1.10268.0"
      urVersion = "UR9"
      agentVersion = "2012 R2"
   case "7.1.10285.0"
      urVersion = "UR11"
      agentVersion = "2012 R2"
   case "8.0.10918.0"
      urVersion = "RTM"
      agentVersion = "2016"
   case "8.0.10931.0"
      urVersion = "UR1"
      agentVersion = "2016"   
case Else
      urVersion = "Unknown"
end select 

wscript.echo "Agent Version: " & agentVersion
wscript.echo "UR: " & urVersion

 

Here is the script in Powershell.   Modified from Kevin’s Agent Management Pack.  https://blogs.technet.microsoft.com/kevinholman/2017/05/09/agent-management-pack-making-a-scom-admins-life-a-little-easier/

function URVersion($Version)
{
  switch($Version)
  {
    # SCOM 2012
    "7.1.10184.0" {"2012 R2 RTM"} 
    "7.1.10195.0" {"2012 R2 UR2"} 
    "7.1.10204.0" {"2012 R2 UR3"} 
    "7.1.10211.0" {"2012 R2 UR4"} 
    "7.1.10213.0" {"2012 R2 UR5"} 
    "7.1.10218.0" {"2012 R2 UR6"} 
    "7.1.10229.0" {"2012 R2 UR7"} 
    "7.1.10241.0" {"2012 R2 UR8"} 
    "7.1.10268.0" {"2012 R2 UR9"} 
    "7.1.10285.0" {"2012 R2 UR11"} 
    "7.1.10292.0" {"2012 R2 UR12"} 
    "7.1.10302.0" {"2012 R2 UR13"} 
    "7.1.10305.0" {"2012 R2 UR14"} 				

    # SCOM 2016
    "8.0.10918.0" {"2016 RTM"} 
    "8.0.10931.0" {"2016 UR1"} 
    "8.0.10949.0" {"2016 UR2"} 
    "8.0.10970.0" {"2016 UR3"} 
    "8.0.10977.0" {"2016 UR4"} 				
  }
}
                        			
$SCOMRegKey = "HKLM:\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Setup"
$SCOMPath = (Get-ItemProperty $SCOMRegKey).InstallDirectory
$SCOMPath = $SCOMPath.TrimEnd("\")

$AgentURFile = Get-Item $SCOMPath\Tools\TMF\OMAgentTraceTMFVer.Dll
$AgentURFileVersion = $AgentURFile.VersionInfo.FileVersion
$AgentURLevel = URVersion $AgentURFileVersion
$AgentURLevel

 

Continue Reading

Install and Uninstall SCOM 2012 R2 UR11 agent updates from a command line

A customer was building a SCCM package to update SCOM Agents.  They needed the command lines to install and uninstall UR11.

Patch Install command

msiexec.exe /p "C:\temp\KB3183990-AMD64-Agent.msp" /qn

Patch Uninstall Command

msiexec  /I {786970C5-E6F6-4A41-B238-AE25D4B91EEA} MSIPATCHREMOVE={1BEA7876-9751-4F7D-B0F3-AA920CF39FE8} /qn

The tricky part was getting the Uninstall GUID. You can get the GUID by using ORCA.

In ORCA load the msp file. Then go to View Summary Information.

patch

 

They also needed a way to get the version information to know if it was installed or needed to be installed.

Here is a quick vbscript to get that information.

 

Set objShell = WScript.CreateObject("WScript.Shell")
sngVersion = objShell.RegRead("HKLM\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Setup\InstallDirectory")

Set objFSO = CreateObject("Scripting.FileSystemObject")
fileVer = objFSO.GetFileVersion(sngVersion & "Tools\TMF\" & "OMAgentTraceTMFVer.Dll")

If fileVer = "7.1.10285.0" Then

 Wscript.echo "UR11 Installed"

Else

Wscript.echo "Not Installed"

End If
Continue Reading

Stop the madness of having outdated SCOM Agents in your environment

I have seen many SCOM environments where it takes multiple teams to update a single SCOM Agent to the latest Update Rollup.   Often it takes months or even years to update all Agents to the latest Update Rollup.  I created two management packs to make it easy to keep all your agents up to date.  With these Management Packs your Application Admins, Server Admins or SCOM Admins can update their servers when they feel comfortable or have an open change window without requiring admin permissions.

Management Pack 1: Contoso Agent Update Rollup

Download: Link

  • Discovers all Agents and their Update Rollup version
  • Monitors and alerts if agents don’t have the latest Update Rollup
  • Task to install the Update Rollup.  Only permissions required is Operator with permission to run the task. 

Management Pack 2: (Optional) Contoso Agent Update Rollup FileDeploy

  • Deploys out latest Update Rollup to Servers.

Download: Link

 

Getting Started

Management Pack 1  (Contoso Agent Update Rollup)

1.) Install Contoso Agent Update Rollup Management Pack

importmp1

 

2.) Download UR11 from here, extract and copy the msp file to a folder on one of your management servers or on a file server that you can set permissions on.

msp1

3.) Share out the file to all Domain Computers, or you can individually add each server you want to give access to the patch.

createshare

share2

 

4.) Open up the SCOM console, a new folder called Contoso Update Rollup will now show up

Expand that folder and click on the State View.

You can see I have two servers in my environment.  One has UR11 installed and one that is still at RTM

state1

5.) To install the update on one or more servers I select the servers and click the Task called “Install Update Rollup”

install

You can see the default is my file location.  You need this to be the location of the file you shared out.  Click override and enter the location of your update rollup patch file.  Make sure you add double quotes around the location.filelocation

newlocation

 

 

Side Note: If you don’t want to do this every time.  You can modify the xml and re-import the mp.  Just search and replace for the line (Make sure you include double quotes) <Arguments>”\\xOM01\UR11\KB3183990-AMD64-Agent.msp”</Arguments>

arguments

6.) Verify everything is correct and click run.

click-run

 

The task will run and output the results.

taskstatus

7.) After about 10 minutes.  Go back and check the state view.  The server should now show the updated UR Version and File Version

sucess

Management Pack 2 (Optional) Contoso Agent Update Rollup FileDeploy

For those who are not able to or don’t want to create a share for all domain computers for security reasons.  I have another option.  This management pack will embed the update rollup file in a management pack and deploy it to the servers.  Then you can use the task in management pack 1 to update the agent with a file on the local system.

For legal reasons I have not included the management pack containing the update rollup, but I will include the project so you can quickly build your own.

How to build the Management Pack File

1.) Open the Solution by double clicking on the .sln file.  This will open the file in visual studio.

visualstudio

 

2.) Select the Resources folder.  Right click Add Existing Item.

expand

 

3.) Pick the drop down to show all files.  Then browse to to where you have the Agent UR file located.

clickonfile

4.) Click on the file now and for Build Action choose Embedded Resource

embed

5.) Check that the Contoso.Agent.UpdateRollup.FileDeploy.Resource.mpx file has the correct file name.  This may need to be changed if you are using a newer or older update rollup then UR11.

filedeploy

6.) Build the solution under Build, Build Solution

build

 

7.) Browse out to the build location.  You should see a file called Contoso.Agent.UpdateRollup.FileDeploy.mpb

location

 

8.) Import the MPB Management Pack into SCOM.

After 10 minutes check your agents to see if the file was deployed.

filedeployedsucess

The default location is C:\Windows\Temp  *Note this can be changed directly in the MP’s XML or with an override to the rule called “Contoso Agent UpdateRollup File Deploy”

Now you can run the task in Management Pack 1 with the local file location.

override2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Continue Reading