DPM as a Distributed Application in SCOM

In Operations Manager 2012 there is something known as a Distributed Application (DA). The purpose of a DA is to give you the overall health of an application made up of different multiple objects. DA’s pull in objects that are already being monitored by SCOM. An example of using a DA could be to provide the health of a web application that consists of backend databases and front end web servers. Both the backend databases and front end web servers are objects that are monitored separately but together make up the entirety of the web application. Monitoring these alone lets you know what the health is of each object but when one of the objects is in a critical state it does not always help put two and two together that these objects make up the components of the web application.

When an infrastructure has multiple DPM servers a DA can be used to get an overall health of your DPM as a service vs… the health of each DPM server through state views trying to track down the root issue. We are going to create a new name for our multiple DPM servers. We need to do this because multiple DPM’s brought together in a DA become a service. We are going to call this "DPM Service". DPM as a DA can be useful for quick spot checking of your DPM service health. Using a DA also allows you to connect relationships between the health of objects that make up DPM. For example you can see the health of disks in the DPM storage pool, tape libraries, SQL databases, protected servers and the DPM servers.

In this post I am going to cover setting up DPM as a DA using the Distributed Application Designer (DAD) and show what it looks like after DPM is a DA. There are a few items that need to be covered before you can setup DPM as a DA. These are:

  • You need to have DPM Central Console installed in SCOM.
  • All DPM servers that will be a part of your DA DPM Service will need to have the SCOM agent installed.
  • Create a custom management pack to store your new DA in SCOM.

Setting up a DPM DA using the Distributed Application Designer

In the SCOM console go to: Authoring.

Right click on Distributed Applications and select Create new distributed application.

The Distributed Application Designer (DAD) will open.

Enter in the information about your DPM Service.

The fields you will need are highlighted in the following screenshot.

clip_image001

In the Template box, select the template for the starting point of the distributed application. Chose Blank (Advanced).

Choosing advanced is going to give us a blank template to work from and this is what we want.

Select your custom management pack that you made for this DA and click OK.

Now we need to create a couple of component groups. Let’s create the following:

  • Databases
  • Servers

Read the rest of this entry »

SCOM: Heartbeat Failure Alert Tuning

I recently deployed SCOM in a highly distributed network. Most of the edge locations had slow WAN links. These edge locations would often go offline. With the combination of the slow WAN links and them going offline SCOM would flood with alerts/emails on Health Service Heartbeat Failure and Computer Not Reachable monitors.

This had to be tuned out because these alerts were overwhelming for the team. Also as soon as these edge locations would go offline the team would be notified through other network location monitoring tools and from the staff at these edge locations.

These edge locations would often go offline for reasons of power outages or ISP’s going down. These edge locations could also be down for long periods from 2-3 days at a time. Fixing the issues were often out of the control of the team. Receiving alerts during these outages from the edge locations was not helpful. The team still needed alerts right away if servers at the corporate locations went offline. There are several ways to tune alerts for these monitors.

One way to tune Health Service Heartbeat Failure and Computer Not Reachable monitors is to adjust the heartbeat interval (default is 60 seconds) and the amount of missed heartbeats SCOM will tolerate. Note this would be a global change in SCOM across all monitored servers. To access these settings do the following:

In the SCOM console go to Administration>>Settings  in the right hand pane under Type: Agent you will see Heartbeat. Right click on Heartbeat and open the properties.  In the same pane under Type: Server you will see another Heartbeat. Right click on Heartbeat and open the properties. You can see this in the following screenshot:

clip_image001

Another way to tune the alerts on these monitors would be to go adjust the heartbeat interval on an individual server level. This would only be useful if you have a small amount of servers generating these alerts and know what servers they are. To access these settings in the SCOM console go to Administration>>Settings>>Agent Managed. Find your server/s. Right click on the server and select properties. Under the Heartbeat tab select the checkbox next to Override global agent settings and then adjust the Heartbeat interval.

clip_image002

For more information about both of those visit:

Heartbeat and Heartbeat Failure Settings in Operations Manager 2007

http://technet.microsoft.com/en-us/library/cc540380.aspx

Neither of those helped in my situation because we needed these alerts right away from one group of servers but not from another. Here is what I did to tune these monitors so that the team would not become overwhelmed by the alerts.

In this particular environment there were some things I need to point out before I go into the solution.

  • The team did not want to monitor heartbeat or ping basically connectivity to the edge servers at all. They were more interested in gathering performance data, status of the applications on those servers and more.
  • The servers that live in the edge had different sequence in the computer name vs. the servers that lived in the corporate locations. The naming schema was structured like this:
    • Corporate location # 1 server names: PROD100-xxV or PROD100-xxP.
    • Corporate location # 2 server names: PROD200-xxV or PROD200-xxP.
    • Edge server names: PROD404-xxV or PROD404-xxP (404 would actually match the number of that edge location. This would vary from edge to edge.).

The name schema was a big helping in breaking things out. So I basically created an edge server group in SCOM dynamically excluding all corporate locations. Here is what it looked like to build this:

clip_image003

Building the logic:

clip_image004

What it looks like in the group:

clip_image005

By doing that the members would consist of all servers from all edge locations without including any servers from corporate locations.  This member list was built dynamically so that the team did not ever have to worry about adding edge servers to the membership.

Read the rest of this entry »

Monitor Tomcat Running as a Windows Service with SCOM

In this post I want to detail my experience monitoring Tomcat with SCOM 2012.  In my case the application servers were Windows Server 2008 R2 running Tomcat 7 as a windows service. When Tomcat is running as a windows service SCOM does not automatically detect the Tomcat instance. So this blog post will focus around monitoring Tomcat Running as windows service and the steps you have to take to get this working.

 

There are two other things you need to take care of that I do not cover in this post. These are:

 

#1 The application servers that are hosting your Tomcat need to have the Agent Proxy enabled.

See this link on how to do this:

http://technet.microsoft.com/en-us/library/hh264858.aspx

or this link to enable proxy for all agents:

http://www.bictt.com/blogs/bictt.php/2012/08/24/scom-2012-enable-agent-proxy

 

#2 You will need to deploy BeanSpy to your Tomcat server first.

See step 5 of this post on how to do this:

http://blogs.catapultsystems.com/cfuller/archive/2012/03/26/operations-manager-2012-scom-lab-testing-monitoring-for-jee-components-tomcat-on-windows.aspx

 

BeanSpy is a WAR file. WAR files are used  to deploy web applications to Tomcat.

BeanSpy is essentially a web application deployed to Tomcat that SCOM uses to gather data. 

 

After you deploy BeanSpy  verify that it is working. Access this URL:

 

http://TOMCATSERVERNAME:8080/BeanSpy/Stats

 

If an XML page such as the one in the following screenshot appears then we have verified BeanSpy is working.

 

clip_image001

 

You can also view BeanSpy by accessing the web manager using this URL:

 

http://TOMCATSERVERNAME:8080/manager

 

clip_image002

 

 

Here are the steps to configure SCOM to monitor Tomcat running as a windows service:

 

Download OpsMgr_MP_Tomcat.docx and SC2012OM_JEE_MP.msi. from:

http://www.microsoft.com/en-us/download/details.aspx?id=29270

 

Run SC2012OM_JEE_MP.msi. This will extract all JEE management packs.

 

Now that the management packs are extracted. Import these JEE Monitoring packs into SCOM 2012 for Tomcat:

 

         Libraries monitoring packs:

             Microsoft.JEE.Library.mpb

             Microsoft.JEE.Templates.Library.mpb

 

         Tomcat monitoring packs:

             Microsoft.JEE.Tomcat.Library.mp

             Microsoft.JEE.Tomcat.5.mp

             Microsoft.JEE.Tomcat.6.mp

             Microsoft.JEE.Tomcat.7.mp

 

(NOTE: Only import MP’s for your version/s of Tomcat.)

 

As stated in the beginning of this post Tomcat running as a windows service will not automatically be discovered.

The following two screenshots demonstrate what you will see.

 

clip_image003

 

clip_image004

 

There are two options to discover the Tomcat servers when they are running as a windows service.

 

#1 Run your Tomcat instances as a process instead of a Windows service.

#2 Use some PowerShell scripts to discover the Tomcat instances.

 

In this post we are going to go with

 

Next you will need to use some PowerShell scripts to add the Tomcat servers. You will need these PowerShell scripts:

 

    JEEAppServerLibrary.ps1

    NewJEEAppServer.ps1

    RemoveJEEAppServer.ps1 

 

Read the rest of this entry »

Manually remove dependency from Microsoft.SystemCenter.SecureReferenceOverride

    I recently needed to uninstall the Exchange 2007 Management Pack from SCOM 2012.

    I could remove all the Exchange MP’s except Microsoft.Exchange.Server.2007.Library. This was getting stuck because the Microsoft.SystemCenter.SecureReferenceOverride was dependent on it. This was the message I would get when I tried to remove it.

    clip_image001

    Here are the “Dependencies” properties from Microsoft.SystemCenter.SecureReferenceOverride.

    clip_image002

    I did not want to delete the Microsoft.SystemCenter.SecureReferenceOverride management pack. That would be bad. I was stuck in a bind because I needed to remove the Microsoft.Exchange.Server.2007.Library Management pack but couldn’t.

    The reason the Exchange MP was referenced in the Secure Override MP is because the Exchange MP contained a Run As profile. Even after you remove accounts from the run as profile sometimes references get left behind. This is when things need to manually be cleaned up.

    Here are my steps to manually remove the dependency.

  • Within the SCOM console go to Administration>>Management Packs and export

the Microsoft.SystemCenter.SecureReferenceOverride MP.

  • The Microsoft.SystemCenter.SecureReferenceOverride will be an XML file open this in something like Notepad ++.
  • Locate the Microsoft.Exchange.Server.2007.Library MP and remove the reference. See the screen shot for a visual.

    clip_image003

  • Now in the same XML file locate the and increase the version number. This is required so that it will overwrite the existing Microsoft.SystemCenter.SecureReferenceOverride MP.

    clip_image004

  • Now in the SCOM console go to Administration>>Management Pack

and re-import the Microsoft.SystemCenter.SecureReferenceOverride MP that you modified. This should overwrite the existing Microsoft.SystemCenter.SecureReferenceOverride MP because it has a higher version number.

    After running through those steps I was able to remove the last Microsoft.Exchange.Server.2007.Library MP.

Virtual Machine Manager 2012 Cookbook published

I am happy to announce the Microsoft System Center Virtual Machine Manager 2012 Cookbook was published on March 26th 2013. I had the opportunity to be a  technical reviewer on this book. If you administer SCVMM you will want this book.

The author is Microfost MVP Edvaldo Alessandro Cardoso.

 

You can visit his blog here: http://virtualisationandmanagement.wordpress.com

 

Here is the cover and the link to purchase it:

image

http://www.amazon.com/Microsoft-Virtual-Machine-Manager-Cookbook/dp/1849686327

Protecting Server 2012 VMs on Server 2008 R2 Hyper-V go into Saved State

Problem:

A buddy of mine recently ran into an issue were a 2008R2 Hyper-V host was running Server 2012 virtual machines.

SCDPM 2012 SP1 was being used to back up all the virtual machines. Some of the virtual machines running Server 2012 would go into a saved state during the backup process. With virtual machine backups we want them to stay online as we back them up.

Now it is always assumed the issue is with SCDPM but something that is still not clear after many years of SCDPM being on the market is that SCDPM is often dependent on the health of the servers it is protecting especially VSS.

The problem is not with SCDPM in this case the issue is on the Hyper-V host itself. If another backup product was being used in this case that also utilizes VSS on the server the same issue would occur. Here is an example of this same issue occurring while using Symantec backup Exec to protect virtual machines running on Hyper-V.

http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/Server/Windows_Server_2008/Q_27833028.html

 

Troubleshooting:

There is a way to see what state will be used on your virtual machines the next time a VSS backup is performed on them.

This will give you more insight into what is going on with your VM backups. Note that these results are from the Hyper-V VSS writer and therefore if you used Windows Server Backup, Backup Exec, or SCDPM the result would be the same.

To check this follow these steps.

Open an elevated command prompt and type the following syntax below:

copy con checkbackup.txt

list writers

Press CTRL + Z (this will save the checkbackup.txt file to the path you are in)

clip_image001

diskshadow /s checkbackup.txt | find /i "caption: backup using"

You should see similar output seen in the screenshot below. Now you will know what virtual machines will back up in a saved or online state the next time a backup is run.

clip_image002

The next step would be to troubleshoot this at the hyper-v host level. Here are some common items you can look at.

  • Backup (Volume Snapshot) Integration Service is disabled or not installed.
  • The virtual machine has one or more dynamic disks.
  • The virtual machine has one or more non-NTFS based volumes.
  • The virtual machine Cluster Resource Group in a cluster setup is offline.
  • The virtual machine is not in a running state.
  • A ShadowStorage assignment of a volume inside the virtual machine is explicitly set to a different volume other than itself.
  • An App-V drive is installed on the virtual machine that creates a non-NTFS volume.

If one of the previously mentioned items are not the issue ultimately you will need to upgrade the Hyper-V host to Server 2012. Hopefully this post was helpful. Here is a link to another blog about the requirements for online Backup of a Hyper-V guest virtual machine via Hyper-V VSS Writer:

http://www.altaro.com/hyper-v/requirements-for-live-backup-of-a-hyper-v-guest-vm-hyper-v-vss-writer

Deduplication for Microsoft DPM using Crunch

With Microsoft DPM 2012 RTM and SP1 protecting deduplicated data on Windows Server 2012 volumes is possible but there is no deduplication on the DPM storagepool. This is a feature that has been needed and requested in the community for some time. DPM administrators have wanted a way to shrink the storage footprint for their DPMs.

 

Now with BridgeSTORs newest product Crunch deduplication is possible on the storagepool.  The nice thing about this product is that its sold on a Deduplication as a Service (DaaS™) model helping organizations avoid CAPEX. Crunch performs deduplication at the block and file-level. For more on how the deduplication works in Crunch check out this video they made.

 

http://vimeo.com/50332899

 

BridgeSTOR has moved from a physical appliance to a virtual appliance. Using a virtual appliance opens up many options of where to place the virtual machine such as a Hyper-V VM on the DPM server itself. Crunch not only offers dedupe but also serves as long-term disk storage for DPM and a way to push DPM data up to cloud storage. Here is a diagram they mad ethat shows where Crunch fits in the DPM architecture.

BridgeSTOR DPM

Crunch also has built in VTL so it allows for your DPM data to be stored to disk configured as virtual tape. The last thing I want to mention about Crunch in this post is that it does not require an enormous amount of hardware resources. Crunch does this by performing the deduplication at times of low system activity paging the hash table in and out of memory, eliminating the need for multiple processors and large RAM configurations.

 

I hope to do some testing of Crunch in the future to see what it can do! I will be posting another blog with my findings. For now you can visit www.bridgestor.com if you want to know more about it.

New System Center book & SC troubleshooting tool

This is an exciting week for System Center. First there is a new tool released by Microsoft called System Center 2012 SP1 Configuration Analyzer (SCCA). The System Center 2012 SP1 Configuration Analyzer works these System Center products in the stack:

 

  • Operations Manager
  • Virtual Machine Manager (VMM)
  • Service Manager
  • Orchestrator (plus Service Provider Foundation)
  • Configuration Manager
  • Data Protection Manager (DPM)

     

     

    It can be used to asses the health of  your System Center install and help in troubleshooting. Here is the link to download it: http://www.microsoft.com/en-us/download/details.aspx?id=36796 . This is for SP1 only. Before you install the SCCA you need to install the Microsoft Baseline Configuration Analyzer 2.0 (MBCA). The SCCA  documentation and installer says you need to install the Microsoft Baseline Configuration Analyzer 2.1 but 2.1 does not exist. Just install 2.0. 

    You can download the MBCA 2.0 here: http://www.microsoft.com/en-us/download/details.aspx?id=16475 .

    For more information on the System Center 2012 SP1 Configuration Analyzer visit:

    http://technet.microsoft.com/en-us/library/jj992580.aspx

     

    Now on to even more exciting news. The new System Center 2012 Operations Manager Unleashed book has been published!

     

    image

     

    This book contains information about SP1 along with tons of great content on SCOM 2012. It was authored by three solid Microsoft System Center Cloud and Data Center Management MVPs including Kerrie Meyler, Cameron Fuller, John Joyner. 

    The authors provide up-to-date best practices for planning, installation, migration, configuration, administration, security, compliance, dashboards, forecasting, backup/recovery, management packs, monitoring including .NET monitoring, PowerShell automation, and much more. I will be picking up my copy soon!

    You can buy it here: http://www.amazon.com/System-Operations-Manager-Unleashed-Edition/dp/0672335913

  • Export computer list from Active Directory

    Recently I needed to pull a large list of servers from Active Directory based on a common term in the computer name. This common term was not at the front of the name or at the end but in the middle of the computer name on each of these servers. Because of this I had to do a LDAP Query using the wildcard at the beginning and end of term. Another thing is that these servers were all over the place so they were not in a single OU. If they were in a single OU I could have simple done an export from the OU into a CSV file. I was able to run a custom LDAP query search in AD to pull a list of all the servers I needed but there was no way to export the results of this list into a CSV file.

    I ran across an LDAP tool that would let me run an LDAP search and then export those results into CSV format. It is LDAP Browser  (The free one) by Softerra. Here is the link: http://www.ldapbrowser.com/download.htm. They have a commercial tool and a free one with less features. Here are a few screenshots and what I did to get the list.

    Connect to your AD.

    Right click on the root of your directory and select Directory Search.

    clip_image001

    Input your LDAP Query and click Search.

    Once you get the results you want click Save Results. Save the results in the format you want i.e. Excel, CSV, text…..

    clip_image002

    This is what the original results will look like:

    clip_image003

    With some quick cleanup (my friend replace) it should look like so after:

    clip_image004

    MCSE: Private Cloud Certification

    I am happy to announce that today I passed the final exam to obtain the MCSE: Private Cloud certification.

    image 

    I was able to upgrade from the MCITP: Server Administrator requiring me to only pass two exams. Out of the 70-246 Private Cloud Monitoring and Operations with System Center 2012 and 70-247 Private Cloud Configuration and Deployment with System Center 2012 the 70-247 exam was the harder of the two.

     SNAGHTML30b6566

    To learn more about the MCSE: Private Cloud certification visit:

    http://www.microsoft.com/learning/en/us/private-cloud-certification.aspx

     

    Here is a blog post with Free “MCSE: Private Cloud” preparation materials:

    http://flo-microsoft-erlebnisse.tumblr.com/post/29484645395/free-mcse-private-cloud-preparation-materials

    Buchatech.com RSS Feed
    Buchatech Newsletter
    Subscribe For Updates
    Sponsors