Archive | SCOM

Part 1 – Hardware and Software Setup || Part 2 – Azure Setup || Part 3 – Dashboarding and Workbooks || Part 4 – Alerting || Part 5 – Tips and Tricks

Are you having trouble keeping your Hot Tub or Pool water balanced? Are you wasting time checking to see if their is enough chlorine/bromine in your hot tub or pool everyday? Do you want to know the temperature of your pool or hot tub anytime of the day from your phone?

With Azure Log Analytics, a Raspberry PI Zero and some IOT parts you can built an awesome Hot Tub or Pool Monitoring Solution.

Here is the equipment you need:
Raspberry Pi Zero WH (Zero W with Headers) – $14
microSD Memory Card– $7.50
5V 2.5A Power Supply – $7.50
Wifi Pool Kit – $349.99
Jumper Wires – $7.49
Azure Subscription

Hardware

Remove the Arduino board that comes with the Wifi Pool Kit. Attach the micro USB cable to the power port of the Raspberry PI. Attach each of the jumper cables as specified in the drawing. To attach them to the blue board you will first have to loosen the small screws then attach the jumper cable and tighten the screws.

Move the Temperature Chip from the Temperature Slot to the AUX slot. It doesn’t work correctly in the Temperature slot with Raspberry PI

Screw in the PH, ORP and Temperature sensors into their respective ports.

When you plug it in the lights should light up blue if everything is working correctly

Software

First we need to get the Raspberry Pi Zero installed and communicating on your WiFi with SSH.

• Plug in the microSD Memory card in your computer. (If you don’t have a slot in your computer you will need a microsSD usb reader or another computer that has one)
  
• Download the Raspberry PI Imager Link
  
• Run the Raspberry PI Imager.
  • For Operating System choose Raspberry Pi OS (other)

• Then choose Raspberry Pi OS Lite (32-bit)

• Choose your MicroSD card for Storage and Click Write

• REMOVE AND REINSERT THE MICROSD CARD – This step is only necessary because Raspberry PI Imager automatically unmounts the SD card.

Before we put the microSD card into the Pi, we’ll need to edit some files.

• In the root of the microSD drive, create a file called ssh (make sure there is no file extension). This file will enable SSH on the Pi. You can create a empty txt file called ssh.txt and remove the txt extension. You might need to click view and select File Name Extensions to remove the file extension.

Create a file called wpa_supplicant.conf and copy the code below changing the ssid and psk to your wifi name and password. Save this file to the root of the microSD drive as well. This is a configuration file that will allow you to pre-configure the WiFi credentials. On boot, the Pi will copy and use this as the default configuration file.

country=US
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
ssid="YOUR_WIFI_SSID"
scan_ssid=1
psk="YOUR_WIFI_PASSWORD"
key_mgmt=WPA-PSK
}

Please double check and verify the WiFi credentials. If not, the next step will be hard.

Remove the MicroSD card from your computer and place it in the RaspberryPI Zero.

We need the IP address of the Raspberry Pi. To find out the IP address you can either log into your router and look for a computer connected to your network called raspberrypi. You an also use a IP scanning tool like Advanced IP Scanner

Now that we have the IP address we can SSH into the Raspberry PI. The default login is

UserName: pi
Password: raspberry

To login to the device using windows 10 open up Windows PowerShell (Right Click Run as administrator)

Type in ssh pi@yourIpAddress

Run the following command in the terminal.

sudo apt-get install python-smbus
sudo apt-get install i2c-tools

Once those have finished installing run

sudo raspi-config

You should see a blue screen with a grey box with options. Select 3 Interface Options. Click Enter

Next choose P5 I2C Enable/disable automatic loading of I2C kernel module. Click Enter

Choose Yes. Click Enter

Click Ok on the next screen then go down to the bottom of the screen and click Finish.

Reboot the Raspberry Pi

sudo reboot

Now with the raspberry pi is setup to communicate with the sensors lets test it out.

Log back into the raspberry pi with SSH using PowerShell.

Install git

sudo apt-get install git

Download the sample code

git clone https://github.com/AtlasScientific/Raspberry-Pi-sample-code.git

CD into the Raspberry-Pi-sample-code directory

Run the following code to make sure your devices are showing up. If you don’t see them then check the connection to the Raspberry PI and the Blue Board.

sudo i2cdetect -y 1

Run the following command to bring up a test command interface

sudo python i2c.py

Then in the enter the following command. This will start polling all three interfaces.

Poll,2.0

Control-c to exit the polling and Control-c to exit the script.

Now lets hook it up to Azure Monitor and Log Analytics.

Monitoring a Hot Tub or Pool with Azure Monitor and Azure Log Analytics – Part 2 – Azure Setup

Features:

• Easy to Use – Select the SCOM Monitor or Rule you want to send to ServiceNow. All future alerts from that Monitor or Rule will be sent to ServiceNow.
• Send alerts based upon SCOM Groups to ServiceNow
  • Send all logical disk free space alerts created from computers in the SQL Team’s SCOM Group to the SQL Teams Incident Assignment Group in ServiceNow.
  •  Send all logical disk free space alerts created from computers in the Exchange SCOM Group to the Exchange Teams Incident Assignment Group in ServiceNow.
• SCOM Console Tasks
  • Select any alert in SCOM and instantly send it to ServiceNow.
  • Quickly identify the rule or monitor that created the SCOM alert to send future alerts to ServiceNow.
• Save Money – No extra ServiceNow tables required to filter SCOM alerts

Download

Select the SCOM Monitor or Rule you want to send to ServiceNow. All future alerts from that Monitor or Rule will be sent to ServiceNow.

When a logical disk free space alert is triggered in SCOM. The alert is created in ServiceNow.

Send alerts based upon SCOM Groups to ServiceNow

SCOM Console Tasks – Select any alert in SCOM and instantly send it to ServiceNow

SCOM Alerts sent to ServiceNow automatically get the ServiceNow Ticket ID and the Owner is set to the Assignment Group.

Alerts in ServiceNow have the Web Console Link in the description if the SCOM Web Console is installed.

How to Use the SCOM To ServiceNow Connector Videos

Have you ever wanted to copy a monitor inside of a sealed management pack? For instance copy the Microsoft Logical Disk Space Monitor and make it work the way you want to.

On the surface this seems extremely difficult because there is no copy button inside the console. But it is actually is not too difficult.

Normally you would need to extract the XML from the sealed management pack and find all the dependencies for that monitor.

With MP author free and pro you are able to make a copy of the monitor.

Once you have MP Author downloaded then you can open up the sealed management pack

Select the monitor you want to copy.

Right click on the monitor and create fragment.

Click Save As and save to your local computer.

Before you close MP Author you will want to copy the Target as MP Author will automatically change the target to a variable.

Now let’s open up the mpx file or management pack fragment in Notepad++

Find and replace the text “##ClassID##” with the target ID that we copied earlier.   In my case it was “Microsoft.Windows.Server.10.0.LogicalDisk”

Now you have a complete copy of the monitor with all the dependencies.  To use this management pack fragment we will need to create a new management pack either in visual studio or in MP author pro.

One thing I wasn’t aware of until recently was that you don’t need to change the class names.  All ids in a management pack need to be unique inside the management pack, but they do not need to be unique inside of SCOM.  The management pack id acts as a namespace if you are familiar with programming.

For Information on how to build a management pack with Visual Studio and the management pack fragment we just created click the links below.

Visual Studio: Authoring Management Packs – the fast and easy way, using Visual Studio??? – Kevin Holman’s Blog

MP Author Pro:   https://www.youtube.com/watch?v=IGFoh2qcUJ4

SCOM Alert Update Connector Pro is the ultimate solution that will solve all of your alerting and ticketing problems.

Avoid all of the alert noise and save time by creating personalized alerts for each team. Easily choose which alerts should be ticketed from our fast web interface. Enhance the alerts with team specific data to inform the ticketing system how to handle the alert.

With an easy to use interface, SCOM will quickly become the essential monitoring and alerting tool for your company.

Download

SCOM Alert Update Connector Pro Solves for:

• Hundreds of alerts sitting in a new state
• Important alerts being missed because your organization won’t tune out the noise
• Complex rules in your ticketing system that need constant updating

Quickly create your workflow

Here is an example of a few workflows:

Alerts matching the criteria will be updated with the custom information you have specified.

Your downstream ticketing system, alert management system, or e-mail system can subscribe to the alerts with the SendToConnector resolution state. Now you are only sending alerts that are important to each team. Our custom fields feature allows you to send alerts to the proper ticketing queues.

SCOM Alert Flow Diagram with Alert Update Connector Pro

Additional Information

• Creating A Basic Alert Workflow
• Creating An Advanced Alert Workflow

If you are using the SCOM Maintenance Mode Scheduler with SCOM 2016 or above and you had a Daylight Savings Time change.

We recommend you running Kevin Holman’s PowerhShell script to fix any schedules that were created with a Start Date time before the change. If you look at an existing schedule, you can see that the intended run time for Maintenance Mode is 6pm, however, since the DST event, the next run time is calculated at 7pm:

PowerShell Script:

Download

More information can be found on Kevin’s blog post here.
https://kevinholman.com/2020/03/09/fix-maintenance-mode-schedules-after-a-dst-time-change/

Do you find it a huge pain to run the outdated and buggy command line tool DWDATARP to adjust SCOM DataWarehouse settings?  This little GUI based tool makes it super easy to adjust the settings.

This new GUI tool can be ran from the SCOM servers or your desktop if you have access to the SCOM DataWarehouse Server.

How to use

Before you start make sure you have DB Owner to the SCOM DataWarehouse Database.

Run the Tool.

Check you have the correct SQL server that has your SCOM DataWarehouse.  Then confirm the name of the Database.

Click Get Current Settings

This will show all of the DataWarehouse Datasets ordered by Percent of Database.

Then select the Dataset Name, Aggregation Type.  Then adjust Max Age (Day) of how long you want the data to stay in the SCOM DataWarehouse.

Then click Set

You should then get a confirmation that the settings have been changed.

It typically takes 24 hours for all the DataWarehouse grooming jobs to run.  After 24 hours run the tool again to confirm than your SCOM DataWarehouse Datasets have gotten smaller.

Kevin Holman has an excellent blog post on explaining DW Grooming in detail located here.  https://kevinholman.com/2010/01/05/understanding-and-modifying-data-warehouse-retention-and-grooming/

With Azure Monitor Workbooks you can now quickly and easily make intelligent decisions about your infrastructure.

One of the things SCOM has never been good at is reporting or dashboards. I recently on-boarded a few VMs in my lab that have been running for months. If I look at SCOM everything looks great.  No performance issues or problems.

Now if I have a look at the same VMs in Azure Monitor Workbooks I see a totally different story.

I can quickly see that my DB07 server has CPU pegged.

I log into my server.  Yep CPU is pegged at 99%.  Even worse its not even SQL that is consuming the CPU.   Where were you at on this one SCOM?

Let me give another example.   If I scroll down in my workbook I can see another chart for available memory.

I can quickly see that one server DB01 has a ton of available memory.

I log into the SQL server and verify that I am wasting memory (relative to my lab size) on this server.

You might say well its because SQL Server is limited right?  Nope not the case.  SQL is set to consume as much as it needs.  I am just wasting memory.

Can’t I get this same data in SCOM? Not easily as most of the time with medium to large SCOM environments, the reports and dashboards take forever to show up or timeout. Azure Monitor can scale to hundreds of thousands of servers and still return this data in seconds.

How to use Azure Monitor Workbooks

First you will need a Log Analytics workspace.  https://docs.microsoft.com/en-us/azure/azure-monitor/learn/quick-create-workspace

Once you have a workspace you will need to point your SCOM agents at the workspace.  I recommend this method.  https://kevinholman.com/2017/08/16/adding-direct-agent-oms-workspace-and-proxy-via-powershell/

Then go to Azure Monitor, Workbooks, and select Performance.

Select your Subscription, Workspace, and Counter

SCOM Maintenance Mode Scheduler makes enabling and scheduling maintenance mode simple for all users. Your IT staff will no longer have a reason why a server wasn’t put into maintenance mode.

The HTML5 web application now works with SCOM 2012 R2, 2016 and 2019. It deeply integrates into SCOM using all the new native maintenance mode scheduling capabilities. All your IT staff can quickly schedule maintenance mode without having to install and use the slow console.

Additional Features:

Instant MM

This solution makes it easy for IT staff to put a server into maintenance mode without having to go to the SCOM console. On any server, the administrator can visit the SCOM Maintenance Mode Scheduler Instant MM website at http://yourMSserver/Home/MM/InstantMM

Create a shortcut on the desktop of the servers to make it even easier. This can be accomplished manually or by use SCCM, AD Group Policy or some other software deployment software

Integrated Dashboard

Instant Maintenance Mode with URL Parameters

SCOM Maintenance Mode Scheduler has the ability to add parameters to the URL.

• ComputerName – Specify the Computer Name
  http://yourMSServer/MM/Home/InstantMM/?ComputerName=yourComputer.yourdomain.com
• Min – Number of Minutes for Maintenance Mode
  http://yourMSServer/MM/Home/InstantMM/?Min=120
• Action – Start or Stop Maintenance Mode
  http://yourMSServer/MM/Home/InstantMM/?MMAction=Start
  http://yourMSServer/MM/Home/InstantMM/?MMAction=Stop
• Combine multiple URL Parameters
  http://yourMSServer/MM/Home/InstantMM/?ComputerName=DB02.scom2k16.com&Min=120&MMAction=Start

Instant Maintenance Mode using PowerShell, VBScript or Code.

Using the new URL Parameters, you can now put servers into Maintenance Mode from any computer using a script or code.  Typical use case would be using SCCM when updates or software is installed.  SCCM would execute the VB or PowerShell script before the install process to start maintenance mode.  After the updates or software is installed SCCM would call the script to stop maintenance mode.

VB Script Example    Download: https://www.scom2k7.com//wp-content/uploads/2018/10/computerMM.renameTOvbs.txt

Dim o, mmServer, computerName, min, action, fullURL

if WScript.Arguments.Count < 0 then
    WScript.Echo "Missing parameters"
end If

mmServer = WScript.Arguments(0)
computerName = WScript.Arguments(1)
min = WScript.Arguments(2)
action = WScript.Arguments(3)

Set o = CreateObject("MSXML2.XMLHTTP")
fullURL = "http://" & mmServer & "/MM/Home/InstantMM?ComputerName=" & computerName & "&Min=" & min & "&Action=" & action
WScript.Echo fullURL

o.open "GET", fullURL, False
o.send

PowerShell Example     Download: https://www.scom2k7.com//wp-content/uploads/2018/10/computerMM.renameTOps1.txt

param (
    [Parameter(Mandatory=$true)][string]$mmServer,
    [Parameter(Mandatory=$true)][string]$computerName,
    [Parameter(Mandatory=$true)][string]$min,
    [Parameter(Mandatory=$true)][string]$action
 )

$FullURL = "http://" + $mmServer + "/MM/Home/InstantMM?ComputerName=" + $computerName + "&Min=" + $min + "&Action=" + $action
$FullURL

Invoke-Webrequest -uri $FullURL -UseDefaultCredential

Unix Example using Wget

wget -q http://vom/MM/Home/InstantMM?ComputerName=vDB.scom2k16.com&MMAction=start

Any problems, bugs, or issues please e-mail:  support@scom2k7.com.com

Quick Download: https://gist.github.com/timmcfadden/0e2912b6ba1e524898c81f921aefcb02/archive/022c9351b99eb9c247601c4f537e825297f803be.zip

Do you want to stop collecting all the worthless SCOM events in your environment?  By default SCOM collects thousands of events out of the box with a few Management Packs installed. 99.999% of the time no one is looking at these events.  The events bloat your database and often times provide zero value.

A customer was recently was decommissioning two SQL servers that created 35 Million SCOM events in 2 days.  The events took up 60GB of DB space and crashed their production SCOM.

First we need to create a Management Pack to hold all the overrides.

I called mine “SCOM2K16 – Disable All Events”

Copy the script to one of your management servers and run it locally.

Now we simply run this powershell script on one of your management servers

The magic in the script is that it only disables event collection rules that don’t alert.  Rules that do alert we want to keep on, so it won’t disable them.

 

The script will take a long time to run as there may be thousands of rules that need to be turned off.

Thanks to Brad Watts for help with the script.

First I need to create a Azure Virtual Network and Subnet.  I go to All services and find Virtual networks

I add a Virtual network called EastAzureVnet with a Subnet called EastServerSubnet and leave the defaults.  (Make sure this address space doesn’t overlap with your on-prem network)

Now I open my Virtual network I just created called EastAzureVnet and click Subnets and click Gateway subnet

Leave the defaults and click OK

Virtual Network Gateway

I need to create a Virtual Network Gateway.  I go to All services and find Virtual network gateways

I set the following values and click Create.  (Note: This will take about 15 minutes, so go have a or a )

Name: EastAzureVngVPN

SKU: Basic  (If this were for production I would choose VpnGw1 or higher)

Virtual network: EastAzureVnet

Public IP address: EastAzureIpVPN

Local network Gateway

Next we need to create a Local Network Gateway. I go to All services and find Local network gateways 

I use my on-prem network information. 

My lab uses IP address range 192.168.2.1 – 192.168.2.255 (aka 192.168.2.0/24)

My Example External IP is 67.37.217.79

After it is created click on the Local Network Gateway called EastAzureLngVPN and click Connections,  Then click Add

I use the following information and Click OK.  Create and save your shared key as you will need this when setting up the PFSense side. (You will want your Shared key to be more complex then the example)

PFSense

In Azure go back to Virtual Network Gateways and get your public IP Address for your Azure VPN

Next I go over to my On-Prem PFSense Firewall and click VPN, IPSec

Click Add P1, I changed the following settings

For Remote Gateway use your Public IP Address from your Azure Virtual Network Gateway

For Pre-Shared Key use your Pre-Shared Key

Click Save

Then Apply Changes

Now Click Show Phase 2 Entries, and click Add P2

For P2 (Edit Phase 2).  I go back to Azure to get the address space.

Set the Remote network address to the address space in Azure.   (Not the Subnet)

Click Save, and Apply Changes.

Now if we go to Status, IPsec

I can see we have Established a connection

Lastly I need to create a firewall rule.  I go to Firewall, Rules.  The select IPsec and click Add

Change Protocol to Any.  You can lock this rule down to suit your needs.

Click Save and Apply

To get DNS working correctly. (So you can Add VMs to your Domain) 

I set you DNS server for your Virtual Network to my local On-Prem DNS server.

One note.  Azure blocks much of ICMP traffic.  So to test, create a VM with a Public IP Address of None.  Then connect with Remote desktop.

For general information on Azure Site-to-Site VPNs see https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-howto-site-to-site-resource-manager-portal