High Health Service Handle Count on RMS

If you experience a high handle count on a RMS,  apply the steps in http://support.microsoft.com/kb/938626/en-us   and apply KB951979.

If you still experience the problem, create an override to up to 50,000 handles, or create a recovery action to restart the health service.

Microsoft is aware and working on fix.

To measure health service handle count, open perfmon, and add the counter for Process>Handle Count>HealthService.exe

clip_image002

Batch Script to Force Windows Update Downloads

I recently had an issue where a client needed to quickly download and install all windows updates.  They originally had all clients pointed to a non-existent SUS server, so most of their servers and workstations were out of date.

We removed the line from the GPO that pointed the clients to the nonexistent SUS server, meaning that the agents will look on the internet for updates; however, they still didn’t get the updates.

Windows Update logs are located here:  %systemdrive%\windows\SoftwareDistribution

When I opened the log file, I saw several errors that looked like this: Windows Update Client failed to detect with error 0x80072efd.

There errors are due to the fact that windows update uses WINhttp, and the client had a proxy server configured, but did not have a proxy server set using proxcfg.

You can check to see if a proxy server is configured for WINhttp by typing proxycfg at the command prompt.

I wrote a batch script to run on all agents that updates the proxy for WINhttp, stops the windows update service, removes the registry keys that indicate when windows update last ran, restarts the service, then forces an update download.

Here is the script, save as a .bat file, and run at will!:

@echo off
REM ======================================================================
REM
REM Batch File
REM
REM NAME: Force Auto Updates Download
REM
REM AUTHOR: Ron Williams ,
REM DATE  : 4/8/2009
REM COMMENT  : Replace [proxyserverIP]:[proxyserverPort] with the ip and port of
REM              your proxy server in the format 192.168.1.2:8080  (no brackets)
REM ======================================================================

proxycfg -p [proxyserverIP]:[proxyserverPort]
net stop wuauserv
REG DELETE "HKLM\Software\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update" /v LastWaitTimeout /f
REG DELETE "HKLM\Software\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update" /v DetectionStartTime /f
Reg Delete "HKLM\Software\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update" /v NextDetectionTime /f
net start wuauserv
wuauclt /detectnow

Installing Additional OpsMgr Component Fails After Applying Hotfixes

Environment Details: OpsMgr SP1 on Server 2008 Std 64-bit with the following OpsMgr hotfixes applied: KB951256, KB954049, and 954903.  These KB’s are request only, and cant be uninstalled using Programs and Features like a normal OS hotfix.

I had a MonitoringHost.exe application hang error on the RMS which was causing console crashes, high CPU usage, WerFault.exe high CPU usage, and general system instability.  I determined the error might be related to the installation of .NET 1.1 which was installed on the RMS and has known compatibility issues with Server 2008.
Here is the Application Hang error i was getting:
Log Name:      Application
Source:        Application Error
Date:          1/21/2009 9:21:41 AM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      RMS
Description:
Faulting application MonitoringHost.exe, version 6.0.6278.0, time stamp 0x47b71488, faulting module HealthServiceRuntime.dll, version 6.0.6278.0, time stamp 0x47b71484, exception code 0x40000015, fault offset 0x0000000000004321, process id 0xf24, application start time 0x01c97bdb36ec44ea.

My Original Plan For Troubleshooting:

  1. Uninstall OpsMgr Console/Command Shell/Web Console (all of the .NET dependent components) using the Programs and Features control panel
  2. Uninstall .NET 1.1 using Programs and Features control panel (NEVER TO BE INSTALLED AGAIN)
  3. Remove the .NET 3.0 Feature using REMOVE FEATURE
  4. Reboot
  5. Reinstall only the .NET 3.0 feature using ADD FEATURE
  6. Reinstall the OpsMgr Console/Command Shell/Web Console

So I uninstalled .NET 1.1, the OpsMgr console, command shell, and web console.  Then removed the .NET 3.0 feature, rebooted, and added the .NET 3.0 feature.

Then to reinstall the OpsMgr components, I double clicked the mom.msi from the OpsMgrSP1 media, chose Console, Shell, and Web Console and hit OK.  The got a huge glaring error that the installation failed because “The file F_Microsoft.MOM.UI.Console.exe.E6A9F744_14F8_46BE_9DA9_B6BAB981D36E cannot be installed because the file cannot be found in cabinet file Data.Cab.

The following event was logged in the application log:

Log Name:      Application
Source:        MsiInstaller
Date:          1/21/2009 2:44:32 PM
Event ID:      11334
Task Category: None
Level:         Error
Keywords:      Classic
User:          FS\catapult
Computer:      RMS
Description:
Product: System Center Operations Manager 2007 — Error 1334.The file F_Microsoft.MOM.UI.Console.exe.E6A9F744_14F8_46BE_9DA9_B6BAB981D36E cannot be installed because the file cannot be found in cabinet file Data.Cab. This could indicate a network error, an error reading from the CD-ROM, or a problem with this package.

The components would not reinstall, because the product version was different due to the hotfixes that were installed.  We figured out a way to trick the installer into thinking that this was a base SP1 install by deleting contents the following registry key, after backing up the registry key

HKEY_CLASSES_ROOT\Installer\Products\DF6E5EFF035E66C49971553D96AA0E4D\Patches\
(The long number starting with DF6E5 in the path would be different on another system).  I identified the right key location, because under the long number key name,  there is a value “ProductName” which is equal to “System Center Operations Manager”. 

I copied the current value of the “Patches” value to notepad then deleted the data in the value (E525AFA…), leaving the REG_MULTI_SZ value, but just emptying it.
image

After clearing this value, I was able to reinstall the console, command shell, and web console.

Then I restored the key’s original value, and have not seen any more application hang errors.

WHEW!!!

Disabling POP3 and IMAP Monitoring for Exchange 2007

The best way to disable POP3 and IMAP monitoring in OpsMgr, is to disable the following monitors using overrides:

  1. IMAPConnectivity
  2. POPConnectivity

These are the only two monitors that you need to disable.

I accidentally disabled the Aggregate Rollup Monitor, instead of the actual monitor, which was causing me to continually get an unhealthy state on the rollup monitor, even though all the children monitors are healthy:

 image

Here is the procedure for properly disabling POP3 and IMAP Monitoring in Operations Manager 2007

  1. Open the OpsMgr console and navigate to Authoring>Management Pack Objects>Monitors
  2. Click on Change Scope, and choose “Ex. Client Access”
  3. Type “POPConnectivity” in the find box
  4. Find the monitor labeled “POPConnectivity” which is located under MOM 2005  Computer Role Health>POP Connectivity>POPConnectivity
  5. Make sure you choose the individual state monitor, NOT the aggregate rollup monitor.  The individual monitor is located under the aggregate rollup monitor
  6. Right click on the individual state monitor labeled POPConnectivity and choose Overrides>Override the Monitor>For All Objects of type: Ex. Client Access
  7. Set the Enabled parameter to FALSE
  8. Choose a custom management pack for your override and click save
  9. Do the same steps for the monitor labeled IMAPConnectivity

Rollup Monitors Showing an Incorrect State

I had a persistent problem on one of my OpsMgr managed agents, who had an incorrect state rollup.  In other words, the parent monitor showed a warning state, even though all child monitors we in a healthy state.

I tried the accepted procedure of putting the health service and health service watcher in maintenance mode, waiting for the agent to show NOT MONITORED, then stopping maintenance mode, but this didnt work.

I resolved the issue by restarting the OpsMgr health service on the agent WHILE it was in maintenance mode, and the issue resolved.

image

Avoiding Spam on Internet Email Sent to a Blackberry

I had email forwarding to my Blackberry from three sources:  My corporate email using a BES, and two online accounts hosted at yahoo and mail.com. 

I was getting an excess of 50 v!4gra and pr0mn spam messages on my blackberry per day, and it was getting really annoying.

I mean, in a perfect world, all email servers would be Exchange, and all would be protected by Forefront for Exchange, and all mobile devices would be windows mobile 6.1… But, at times, we must exist in conditions other than ideal 🙂

In a previous position, I worked with Postini, which was acquired by Google about 2 years ago.  I remember that it was a really accurate scanning engine, and that it cut out most of my spam, and had very few false positives.

With that in mind, I signed up for a new Gmail account, forwarded my other two internet email addresses to Gmail, then enrolled the Gmail account on my blackberry.  I deleted email forwarding for the two original accounts.

It has been one week now, and i have only gotten 2 spam messages sent to my blackberry.  So thanks Google!

Maybe someday we can convince Google to use Exchange, but now I’m only dreaming….

Another Way to Block Context Sensitive Adds on Web Pages

My friend, Joe Stocker, the illustrious Director of IT at Catapult Systems, pointed out another way of blocking contextual ads.  I think it’s a super cool idea:

From Joe:

Another way to do this is to download a copy of this hosts file which changes most ad companies to 127.0.0.1 (so your computer doesn’t route to their content). Ingenious, eh?

I got this from this MVP site:
http://www.mvps.org/winhelp2002/hosts.htm