Archive | SCOM

SCOM Monitor Reset App

SCOMMonitorResetApp

Users closing monitor based alerts creates chaos in a SCOM environment.  The SCOM Monitor Reset app keeps your alerts in sync.  Now users can close any alert and know that if the problem still exists they will get re-alerted automatically.

 

The SCOM Monitor Reset App is a scalable windows service that runs and resets any monitor based alerts closed by an end user.  You just install it and forget it.   The service takes care of all the work.

Service

Now as an SCOM administrator you no longer have to explain the difference between monitors and rules.  End users can close any and all alerts.  Closed alerts that are still a problem will re-open a new alert.

Download

 

Usage:

A disk space alert comes in.

Screenshot10

 

The user closes the alert without fixing the issue.

Normally the alert would stay closed and the monitor would remain in a critical state.

Bad

But with the monitor reset app.  After 5 minutes the alert is detected closed.  The monitor state is reset.

Screenshot11

 

Good

If disk space issue still exists a new alert will be created.

 

Fixed

 

Any problems, bugs, or issues please e-mail:  support@scom2k7.com.com

Continue Reading

Crazy DB Performance Collection rules in the SQL MPs

One of my customers was experiencing lots of growth in their OperationsManager DB.

They monitor hundreds of SQL servers. I had a look into their Top Tables using Kevin Holman’s Large table query. http://bit.ly/1REx9Os

Things looked pretty normal where Performance tables are the top tables

Table2

 

I drilled down a little further in the performance data and see this.

Tables

 

Focusing on the two top counters that are 4x larger then the next few.

I took a look at the counters for one of my SQL servers and realize that we are collecting the same counter over and over for each database instance on a SQL server.

Perfmon

 

This is crazy.  It might make some sense if a customer ran every database on a individual drive, but that is not the norm.  Most SQL servers have one or a few drives for their DB files and Logs.

To double check I looked at the OperationsManager database performance tables.

dbData

Sure enough we are collecting the same data 13 times in my case.

So how do we fix this?  Disable the rule that collects this data using an override for SQL 2005, 2008, 2012, and 2014.

Rule for SQL 2012 is called

  • MSSQL 2012: Collect DB Disk Write Latency (ms)
  • MSSQL 2012: Collect DB Disk Read Latency (ms)

Latench

 

 

 

disable

 

I then recommend collecting this data if you need it once per disk.

To enable it, create an override for Windows Server 2008 and 2012 Logical Disk

  • Collection Rule for Average Disk Seconds Per Write Windows Server 2012
  • Collection Rule for Average Disk Seconds Per Read Windows Server 2012

LogicalDisk

 

LogicalDiskTrue

Override True

 

 

Continue Reading

SCOM 2012 Maintenance Mode Scheduler Version 7

New Features in Version 7

  • CSV Server Import – Now you can schedule a list of Windows or Unix computers for Maintenance Mode using a CSV (Comma Separated Values) file.
  • Intelligent Maintenance Mode – Before a computer or object is put into maintenance mode it is checked to see if it’s already in maintenance mode. If the existing maintenance window is longer then the new maintenance window, the existing window is kept.
  • Unix/Linux Support for Immediate Maintenance Mode – Unix Admins can now use the MMNow page to immediately put servers into Maintenance Mode using a web browser or wget from a shell script.
  • Management Servers in a Group are prevented from going into Maintenance Mode – If a group is scheduled for maintenance mode that contains Management Servers. The Management Servers are now skipped from going into Maintenance Mode.
  • New Jobs Report Fixed – Some users were experiencing an issue where the MMScheduler box was grayed out. This has now been resolved.
  • Improved permissions tool – It is now easier to grant new permissions as existing users and groups show up when the tool is opened.
  • Improved Logging – Events are created when a Schedule Maintenance Job is run or when a user puts a server into maintenance mode immediately using MM Now

 

Download Now

CSV Server Import

CSV File
csv

csvimport

 

Unix/Linux Support for MMNow Page

mmNow

Calling One Click MM using wget from a Unix/Linux Shell Script

wget http://xom01/MMweb/OneClick.aspx

shell

Calling One Click MM using Invoke-WebRequest from PowerShell script on a Windows Computer

Invoke-WebRequest http://xOM01/MMweb/OneClick.aspx -UseBasicParsing

powershell

New Jobs Report Fixed

report

Improved permissions tool

secure

Improved Logging

MMLog1
MMLog2

Continue Reading

Alert on SQL Job Failure when SQL Agent Job Discovery is Enabled

Customer was asking why they were not getting alerts for SQL Agent Job failures.  They had enabled individual SQL Agent Job Discoveries that is talked about in Kevin’s post.

By default the Last Run Status monitor does not alert.  So they created an override to enable Generates Alert.

SQL1

 

Problem was no alerts came through.  Looking at Alert severity it says “Critical” and Alert on State it says “The monitor is in a critical health state”.  Looks good to me.

But still no alerts.

Lets take a look at the monitor.  The monitor is in a warning state.  That is weird I thought Alert severity is “Critical” from the above screenshot.
SQL2

 

 

Lets take a look at the monitor.  I right click and go to monitor properties.  There is no Critical Health State, only Warning and Healthy for this monitor.  Looks like a bug to me or bad MP practices.

SQL3

If I look back at my Overrides I can see that this monitor will only Alert On State if “Alert if it is in a Critical Health State”

SQL4

 

Well it will never go into a “Critical Health state” because it doesn’t exist.   There is no way to override the monitors health.  This is not usually an issue because most monitors are either two state Health and Critical, or three state, Healthy, Warning, and Critical.

So what to do?

The only way to get the monitor to alert is to create an override so that Alert on State is changed to Warning.

sql5

Now when the monitor changes to a “Warning state” I get an Alert.

SQL6

 

But why is it a “Warning”?  Not really sure.  The default behavior is Alert severity is “Critical”

If I create another override for “Alert severity” Critical

SQL7

 

Now I get a critical alert.

SQL8

 

 

But the monitor is still in a warning state and could be confusing to the end user.

sql9

 

I will file a bug to get this changed.

 

 

 

 

 

 

Continue Reading

Microsoft acquires BlueStripe

Today it was officially announced that Microsoft has acquired BlueStripe software, which has been a key partner for Operations Manager and Azure for monitoring application performance and availability of distributed business applications seamlessly and comprehensively.

bluestripe

You can read more of the announcement here

http://blogs.microsoft.com/blog/2015/06/10/microsoft-acquires-bluestripe-to-help-customers-improve-application-visibility-and-management-across-the-datacenter-and-cloud/

Continue Reading

Error upgrading SCOM 2012 RTM to SCOM 2012 SP1

Ran into this issue today trying to upgrade from SCOM RTM to SCOM SP1

I got the standard 1603 error in the OpsMgrSetupWizard.log

[15:44:20]: Error: :LaunchMSI: MSI D:\Setup\AMD64\Server\OMServer.msi returned error 1603
[15:44:20]: Error: :ProcessInstalls: Install Item Management Server failed to install. We did not launch the post process delegate.
[15:44:20]: Always: :SetErrorType: Setting VitalFailure. currentInstallItem: Management Server

Opening up the OMServer.log one of my colleagues Stefan Wuchenauer identified the issue.

 

MSI (s) (18:18) [15:43:33:134]: Executing op: CustomActionSchedule(Action=_Rollback_UninstallBIDMOF.DB577B04_77CF_4F29_BA24_844EA640D17E,ActionType=3393,Source=BinaryData,Target=UninstallMOF,CustomActionData=C:\Program Files\System Center 2012\Operations Manager\Server\mom_tracing.mof)
MSI (s) (18:18) [15:43:33:136]: Executing op: ActionStart(Name=_InstallBIDMOF.DB577B04_77CF_4F29_BA24_844EA640D17E,,)
MSI (s) (18:18) [15:43:33:138]: Executing op: CustomActionSchedule(Action=_InstallBIDMOF.DB577B04_77CF_4F29_BA24_844EA640D17E,ActionType=3073,Source=BinaryData,Target=InstallMOF,CustomActionData=C:\Program Files\System Center 2012\Operations Manager\Server\mom_tracing.mof)
MSI (s) (18:E8) [15:43:33:172]: Invoking remote custom action. DLL: C:\Windows\Installer\MSIDE96.tmp, Entrypoint: InstallMOF
InstallMOF: Custom Action Data. C:\Program Files\System Center 2012\Operations Manager\Server\mom_tracing.mof
CompileMOF: CompileFile() failed. Error Code: 0x80041002. C:\Program Files\System Center 2012\Operations Manager\Server\mom_tracing.mof
MSI (s) (18:78) [15:43:33:399]: I/O on thread 1084 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 2704 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 3316 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 4716 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 2840 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 3720 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 2924 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 1324 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 4736 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 4496 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 4048 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 3100 could not be cancelled. Error: 1168
MSI (s) (18:78) [15:43:33:399]: I/O on thread 1768 could not be cancelled. Error: 1168
MSI (s) (18!4C) [15:43:33:400]: Product: System Center Operations Manager 2012 Server — Error 25218.Failed to uninstall SDK MOF. Error Code: -2147217406 (IDispatch error #3586).

Error 25218.Failed to uninstall SDK MOF. Error Code: -2147217406 (IDispatch error #3586).
CustomAction _InstallBIDMOF.DB577B04_77CF_4F29_BA24_844EA640D17E returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)

Here is the fix. (*Note* The blog post says SCOM 2007 but it works for SCOM 2012 as well.)

http://blogs.technet.com/b/operationsmgr/archive/2009/05/05/opsmgr-2007-error-25218-failed-to-uninstall-sdk-mof-when-installing-sp1.aspx

 

Resolution: To resolve this issue follow the steps below:

1. Open a CMD prompt and navigate to the “C:\Windows\System32\Wbem” folder.

2. Run the following command:

for /f %s in (‘dir /b *.mof *.mfl’) do mofcomp %s
for /f %s in (‘dir /b /s *.dll’) do regsvr32 /s %s

3. Try installing the SP1 upgrade again.

 

Continue Reading

Error Installing SCOM 2012 “The target principal name is incorrect. Cannot generate SSPI context”

I ran into this error today trying to install SCOM 2012.  “The target principal name is incorrect. Cannot generate SSPI context.”

The Error from the Setup Wizard was “This SQL server could not be found. Please check that this SQL server is running and that the current user has login permissions.”

Then looking at the OpsMgrSetupWizard.log located at %LocalAppData%\SCOM\LOGS\

I found this error

[17:50:31]: Always: :OpenAndTestSqlConnection: connection string – Data source=vDB01,1433;Integrated Security=true;Connection Timeout=5
[17:50:31]: Debug: :SqlConnectionReady returned False.
[17:50:31]: Info: :Info:DatabaseConfigurationPage: DB connection attempt completed.
[17:50:31]: Info: :Info:DatabaseConfigurationPage: DB connection attempt completed.
[17:50:32]: Always: :DatabaseConfigurationPage: Attempting to connect to database using server\instance vDB01. If we need it, the port is 1433
[17:50:32]: Debug: :MSSQLSERVER on server vDB01 is in a running state
[17:50:32]: Info: :Info:Opening/Testing Sql Connection on vDB01, port:
[17:50:32]: Debug: :Connection was not open. We will try to open it.
[17:50:32]: Error: :Error opening SQL connection.: Threw Exception.Type: System.Data.SqlClient.SqlException, Exception Error Code: 0x80131904, Exception.Message: The target principal name is incorrect. Cannot generate SSPI context.
[17:50:32]: Error: :StackTrace: at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry)
at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
at System.Data.SqlClient.SqlConnection.Open()
at Microsoft.EnterpriseManagement.OperationsManager.Setup.Common.SetupValidationHelpers.OpenAndTestSqlConnection(DbConnection sqlConnection, Int32 retries)

 

After some research I concluded that is must be an SPN issue.

I am using a service account for SQL.  So I attempted to manually set the SPN for the service account.

But I ran into this issue.  “The operation failed because the SPN value provided for addition/modification is not unique forest-wide.”

SPNIssue

Turns out the computer account already had this SPN set.

So I removed the SPN for the computer account and added it to the SQL Service account.

Then the wizard let me continue.

 

 

Continue Reading

SCOM 2012 Maintenance Mode Scheduler 6

Here are the new features in V6:

  • Dedicated Linux / Unix Page –  Linux and Unix admins now have a dedicated page to schedule maintenance mode.  They can search and select multiple *NIX computers for maintenance mode.
  • Improved Classes Page – The new classes page is much faster and uses the display name of objects.  It also has search and the ability to schedule multiple objects of a class.
  • Classes Page now accepts a parameter  – When opening the classes page you can now specify an ObjectID as a parameter and it will find the object automatically for easy scheduling or integration in to other systems.

 

Download Free Trial

Dedicated Linux / Unix PageUnixLinux

 

Improved Classes Page

Class

 

Classes Page now accepts a parameter

Class2

Continue Reading

SCOM 2012 R2 Command Line Manual Agent Install

In SCOM 2012 R2 the parameters for a manual command line agent install have been updated.  One new required parameter in SCOM 2012 R2 is AcceptEndUserLicenseAgreement=1

To deploy 32-Bit and 64-Bit SCOM 2012 R2 Agents using SCCM in one package here is batch script updated for SCOM 2012 R2

@echo off

Set RegQry=HKLM\Hardware\Description\System\CentralProcessor\0

REG.exe Query %RegQry% > checkOS.txt

Find /i "x86" < CheckOS.txt > StringCheck.txt

If %ERRORLEVEL% == 0 (
               Echo "This is 32 Bit Operating System"
               msiexec.exe /i \\YOURFILESERVER\AgentManagement\x86\MOMAgent.msi /qn /l*v c:\MOMAgentinstall.log USE_SETTINGS_FROM_AD=0 USE_MANUALLY_SPECIFIED_SETTINGS=1 MANAGEMENT_GROUP=YOURMANAGEMENTGROUPNAME MANAGEMENT_SERVER_DNS=YOURSCOMMSSERVER.YOURDOMAIN.COM ACTIONS_USE_COMPUTER_ACCOUNT=1 AcceptEndUserLicenseAgreement=1
) ELSE (
               Echo "This is 64 Bit Operating System"
               msiexec.exe /i \\YOURFILESERVER\AgentManagement\amd64\MOMAgent.msi /qn /l*v c:\MOMAgentinstall.log USE_SETTINGS_FROM_AD=0 USE_MANUALLY_SPECIFIED_SETTINGS=1 MANAGEMENT_GROUP=YOURMANAGEMENTGROUPNAME MANAGEMENT_SERVER_DNS=YOURSCOMMSSERVER.YOURDOMAIN.COM  ACTIONS_USE_COMPUTER_ACCOUNT=1 AcceptEndUserLicenseAgreement=1
)

Download: https://www.scom2k7.com/downloads/SCOM2012Agent.bat.txt

 

Here is a list of all the parameters for a SCOM 2012 Agent install from https://technet.microsoft.com/en-us/library/hh230736.aspx

USE_SETTINGS_FROM_AD={0|1} Indicates whether the management group settings properties will be set on the command line. Use 0 if you want to set the properties at the command line. Use 1 to use the management group settings from Active Directory.
USE_MANUALLY_SPECIFIED_SETTINGS=={0|1} If USE_SETTINGS_FROM_AD=1, then USE_MANUALLY_SPECIFIED_SETTINGS must equal 0.
MANAGEMENT_GROUP=MGname Specifies the management group that will manage the computer.
MANAGEMENT_SERVER_DNS=MSname Specifies the fully qualified domain name for the management server. To use a gateway server, enter the gateway server FQDN as MANAGEMENT_SERVER_DNS.
MANAGEMENT_SERVER_AD_NAME=ADname Use this parameter if the computer’s DNS and Active Directory names differ to set to the fully qualified Active Directory Domain Services name.
SECURE_PORT=PortNumber Sets the health service port number.
ENABLE_ERROR_REPORTING={0|1} Optional parameter. Use this parameter with “1” to opt in to error report forwarding to Microsoft. If you do not include this parameter, the agent installation defaults to “0”, which opts out of error report forwarding.
QUEUE_ERROR_REPORTS={0|1} Optional parameter. Use this parameter with “1” to queue error reports or with “0” to send reports immediately. If you do not include this parameter, the agent installation defaults to “0”.
INSTALLDIR=path Optional parameter. Use this parameter if you want to install the agent to a folder other than the default installation path. Note that \Agent will be appended to this value.
ACTIONS_USE_COMPUTER_ACCOUNT={0|1} Indicates whether to use a specified user account (0) or the Local System account (1).
ACTIONSUSER=UserName Sets the Agent Action account to UserName. This parameter is required if you specified ACTIONS_USE_COMPUTER_ACCOUNT=0.
ACTIONSDOMAIN= DomainName Sets the domain for the Agent Action account identified with the ACTIONSUSER parameter.
ACTIONSPASSWORD= Password The password for the user identified with the ACTIONSUSER parameter.
NOAPM=1 Optional parameter. Installs the Operations Manager agent without .NET Application Performance Monitoring. If you are using AVIcode 5.7, NOAPM=1 leaves the AVIcode agent in place. If you are using AVIcode 5.7 and install the Operations Manager agent by using momagent.msi without NOAPM=1, the AVIcode agent will not work correctly and an alert will be generated.
AcceptEndUserLicenseAgreement=1 Used to specify that you accept the End User License Agreement (EULA).

 

 

Continue Reading