SCOM PS Grid Widget – adding fake health state columns

The question of the day arrived from Jose (b|t):

Hey Oleg, wondering if you’d know whether it is possible to create a column using the createinstance method, whose type is a healthstate icon, the same way you have when you createfromobject? I’ve been trying to do that in many different ways, with no luck.

Do I know? Heck, I don’t! Actually, I had absolutely no time to get my hands dirty with SCOM visualization magic during last year. Fortunately, I had a chance to work with several SCOM presentation gurus from VIAcode team, so I learned some tricks. High time for the bloody reverse engineering exercise.

Quick search (just 2 hours of fun :)) takes me to Microsoft.SystemCenter.Visualization.Component.Library.DataProviders assembly, ScriptContext class. CreateWellKnownType public method looks very promissing. Some tests… reading code back and forth… trying to get a working sample… And here it goes – there are two ways to get a health state icon in the SCOM PS Grid widget:

Option 1: Based on enum values (0 – Uninitialized, 1 – Success, 2 – Warning, 3 – Error), maintenance mode will not be considered in this case.

$dataObjectA = $ScriptContext.CreateInstance("xsd://olegkapustin.com/MySchema");
$dataObjectA["Id"]="A"; 
$dataObjectA["Fake State"]=$ScriptContext.CreateWellKnownType("xsd://Microsoft.SystemCenter.Visualization.Library!Microsoft.SystemCenter.Visualization.OperationalDataTypes/MonitoringObjectHealthStateType",0)
$dataObjectA["Fake State Name"]="0 - Uninitialized";
$dataObjectA["Text 1"]="Hello";
$dataObjectA["Text 2"]="World";
$ScriptContext.ReturnCollection.Add($dataObjectA);  

$dataObjectB = $ScriptContext.CreateInstance("xsd://olegkapustin.com/MySchema");
$dataObjectB["Id"]="B";
$dataObjectB["Fake State"]=$ScriptContext.CreateWellKnownType("xsd://Microsoft.SystemCenter.Visualization.Library!Microsoft.SystemCenter.Visualization.OperationalDataTypes/MonitoringObjectHealthStateType",1)
$dataObjectB["Fake State Name"]="1 - Success";
$dataObjectB["Text 1"]="Stay with us,";
$dataObjectB["Text 2"]="World!";
$ScriptContext.ReturnCollection.Add($dataObjectB);

$dataObjectC = $ScriptContext.CreateInstance("xsd://olegkapustin.com/MySchema");
$dataObjectC["Id"]="C";
$dataObjectC["Fake State"]=$ScriptContext.CreateWellKnownType("xsd://Microsoft.SystemCenter.Visualization.Library!Microsoft.SystemCenter.Visualization.OperationalDataTypes/MonitoringObjectHealthStateType",2)
$dataObjectC["Fake State Name"]="2 - Warning";
$dataObjectC["Text 1"]="Goodbye";
$dataObjectC["Text 2"]="World";
$ScriptContext.ReturnCollection.Add($dataObjectC);

$dataObjectD = $ScriptContext.CreateInstance("xsd://olegkapustin.com/MySchema");
$dataObjectD["Id"]="D";
$dataObjectD["Fake State"]=$ScriptContext.CreateWellKnownType("xsd://Microsoft.SystemCenter.Visualization.Library!Microsoft.SystemCenter.Visualization.OperationalDataTypes/MonitoringObjectHealthStateType",3)
$dataObjectD["Fake State Name"]="3 - Error";
$dataObjectD["Text 1"]="Two hours of";
$dataObjectD["Text 2"]="research...";
$ScriptContext.ReturnCollection.Add($dataObjectD);

Outcome:
SCOM PS Grid Widget with icon

Option 2. Based on monitoring object’s state, maintenance mode is considered.

Get-SCOMClass -Name "Microsoft.Windows.Computer" | Get-SCOMMonitoringObject | %{
    $dataObject = $ScriptContext.CreateInstance("xsd://olegkapustin.com/MySchema");
    $dataObject["Id"]=$_.Id.ToString();
    $dataObject["Name"]=$_.DisplayName;
    $dataObject["HS"]=$ScriptContext.CreateWellKnownType("xsd://Microsoft.SystemCenter.Visualization.Library!Microsoft.SystemCenter.Visualization.OperationalDataTypes/MonitoringObjectHealthStateType",$_)
	$ScriptContext.ReturnCollection.Add($dataObject)
}

Outcome:

SCOM PS Grid Widget with icon from object

That’s it, Jose! Hope it helps.

 

Office 365 MP – some notes on security configuration

Okay, I’ve got a very short break between project A and project B. Hope, you’ll enjoy the result of 3 months development effort (unfortunately I cannot disclose any details, but those who watch for new MPs will probably understand what was the “project A” :). Hint: it is not related to SQL Server).

Meanwhile, let’s shed the light on some O365 MP staff. I saw several complaints about security configuration for this scom management pack and, indeed, this is not well-documented part.

So, Office 365 MP defines two Run As profiles:

Office 365 Subscription Password secure reference.

This one is used to store O365 management credentials which are used for authentication at the management portal via O365 API. I’ve seen some complaints about updating credentials via O365 MP subscription administration UI, but this can be worked around by editing Run As accounts directly at “Run As Account Properties” dialog.

Office 365 Subscription Proxy secure reference.

This one absolutely undocumented though very important. It is used by all data sources involved into both data collection and automatic alert closure:

Office 365 management pack Run As profiles

So, to make everything work, an account mapped to this profile should met following criteria:

  1. It should be a domain user;
  2. It should be able to login at the management server (otherwise SCOM will fail to run o365 monitoring workflows);
  3. It should be able to access Office 365 API endpoint (via https), so configure your firewalls, proxies, etc. There are no specific requirements for ports.
  4. It should be able to access SCOM data via SDK API to enumerate and close alerts. Usually the membership in “Operations Manager Operators” role is enough, but for unknown reason this doesn’t work in this case. So, ensure that you have granted “Operations Manager Administrators” role.

That’s it. And yes, I agree that synchronization and automatic closure logics could be somewhat more flexible and convenient.

 

How to add a good-looking description for SCOM Report?

Yesterday I was asked a question:

I would like to enter a description for a published SCOM report that is longer than the 512 character limit. I see that the Veeam reports have very detailed and long descriptions. Do you know how I can accomplish this?

Here is the answer: to get the long, detailed and good-looking description, we should add KnowledgeArticle element to the management pack and decorate it with appropriate attributes. The element should be a child of /ManagementPack/LanguagePacks/KnowledgeArticles.

So if we want to decorate our SCOM Report with the description like this:

SCOM Performance Report Description Screenshot

We should add the following code into the SCOM Management Pack (Note: I have removed many lines of code to make the screenshot more clear):

SCOM Report Knowledge Base Article Scrrenshot

Both screenshots show Microsoft Generic Report Library management pack. You may want to unseal it and study the code for further learning.

Enjoy!

 

SCOM Powershell Grid Widget for Mere Mortals

Just 2 hours left before I’ll open the door and will head to Houston for TechEd North America 2014. Great time to write a short blog post about an awesome feature introduced with Update Rollup #2 for SCOM 2012 R2  – Powershell Grid Widget.

Where to find

Download and install Update Rollup #2 as described in this KB article. Don’t forget to import new management packs from %SystemDrive%\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server\Management Packs for Update Rollups. One of the MPs will be Microsoft.SystemCenter.Visualization.Component.Library.mpb – it contains several hidden gems inside, one of them – Powershell Grid Widget.

PS Grid Widget Add Dialog

What’s that?

SCOM Powershell Grid Widget can be added to any SCOM Dashboards and takes PS script as configuration:

PS Grid Widget Script Setting

So, SCOM users are free to write their own scripts to collect data from any data source (not just SCOM) and have that data displayed in the SCOM console (both windows and web). And of course, these scripts can use SCOM cmdlets and SCOM SDK API to get information from SCOM itself.

Scripts for SCOM Powershell Grid Widget by example

Note: Please note my comments placed inside the code.

1. Hello World

Param($globalSelectedItems)

$dataObject = $ScriptContext.CreateInstance("xsd://olegkapustin.com/MySchema");
$dataObject["Id"]="1"; # Id should be string! String!! STRING !!!
$dataObject["First Name"]="Hello";
$dataObject["Last Name"]="World";
$ScriptContext.ReturnCollection.Add($dataObject);

Poweshell Grid Widget Hello World

2. ID should be unique

This will not work:

Param($globalSelectedItems)

$dataObjectA = $ScriptContext.CreateInstance("xsd://olegkapustin.com/MySchema");
$dataObjectA["Id"]="1"; 
$dataObjectA["First Name"]="Hello";
$dataObjectA["Last Name"]="World";
$ScriptContext.ReturnCollection.Add($dataObjectA);  

$dataObjectB = $ScriptContext.CreateInstance("xsd://olegkapustin.com/MySchema");
$dataObjectB["Id"]="1"; # This object will NOT be added due to duplicated IDs
$dataObjectB["First Name"]="Goodbye";
$dataObjectB["Last Name"]="World";

$ScriptContext.ReturnCollection.Add($dataObjectB);

This is correct:

Param($globalSelectedItems)

$dataObjectA = $ScriptContext.CreateInstance("xsd://olegkapustin.com/MySchema");
$dataObjectA["Id"]="1"; 
$dataObjectA["First Name"]="Hello";
$dataObjectA["Last Name"]="World";
$ScriptContext.ReturnCollection.Add($dataObjectA);  

$dataObjectB = $ScriptContext.CreateInstance("xsd://olegkapustin.com/MySchema");
$dataObjectB["Id"]="2"; # This object will be added - IDs are unique
$dataObjectB["First Name"]="Goodbye";
$dataObjectB["Last Name"]="World";

$ScriptContext.ReturnCollection.Add($dataObjectB);

Poweshell Grid Widget Hello World 2

3. Where does it run? Who am I? Can I write to disk?

Param($globalSelectedItems)

function debug($msg)
{
	#Can I write to disk? Absolutely!
	[string]::Format("{0} {1}",[datetime]::Now,$msg) | Out-File -FilePath C:\-Docs\PSGrid.txt -Append
}

$dataObject = $ScriptContext.CreateInstance("xsd://olegkapustin.com/schemas/PowerShellGridWidgetTest");
$dataObject["Id"]="1"; # Id should be string! String!! STRING !!!
$dataObject["Who?"]=[System.Security.Principal.WindowsIdentity]::GetCurrent().Name #Local You
$dataObject["Where?"]=[Environment]::MachineName; #Your local machine (surprise!)
$ScriptContext.ReturnCollection.Add($dataObject);  

debug("Hello Debug!")

4. Throwing exceptions

Param($globalSelectedItems)

throw "Hello World Not Found!"

SCOM Poweshell Grid Widget Exception

SCOM Poweshell Grid Widget Exception Details

5. Other widgets set the context

Now the one and only parameter comes to the stage:

#Hint #1: switch to another view and back to dashboard if you changed column names and/or order. 
#Hint #2: columns with null values will not be displayed
Param($globalSelectedItems)

$i = 0
foreach ($item in $globalSelectedItems)
{
	$dataObject = $ScriptContext.CreateInstance("xsd://olegkapustin.com/schemas/PowerShellGridWidgetTest");
	$dataObject["Id"]=$i.ToString();
	$dataObject["Object Id"]=$Item["Id"];
	$dataObject["Type"]=$Item.GetType().ToString();
	$ScriptContext.ReturnCollection.Add($dataObject);  
	++$i
}

Another way to explore the $globalSelectedItems parameter:

Param($globalSelectedItems)

$i = 0
foreach ($item in $globalSelectedItems)
{
	$item | Get-Member | Out-File -FilePath C:\-docs\psGrid_SelectedItem.txt
	$item.CollectionKeyProperty | Get-Member | Out-File -FilePath C:\-docs\psGrid_SelectedItem.txt -Append
	$item.CollectionKeyProperty | Out-File -FilePath C:\-docs\psGrid_SelectedItem.txt -Append # It has only Id 🙁
	break;
}

6. Getting related (hosted and contained) objects for the current context

Param($globalSelectedItems)

function debug($msg)
{
	#Can I write to disk? Absolutely!
	[string]::Format("{0} {1}",[datetime]::Now,$msg) | Out-File -FilePath C:\-Docs\PSGrid.txt -Append
}

foreach ($globalSelectedItem in $globalSelectedItems)
{
	$globalSelectedItemInstance = Get-SCOMClassInstance -Id $globalSelectedItem["Id"]
	foreach ($relatedItem in $globalSelectedItemInstance.GetRelatedMonitoringObjects())
	{
		debug($relatedItem.GetType().ToString())
		$dataObject = $ScriptContext.CreateFromObject($relatedItem, "Id=Id,State=HealthState,DisplayName=DisplayName,Path=Path", $null)
		$dataObject["ParentRelatedObject"] = $globalSelectedItemInstance.DisplayName
		$ScriptContext.ReturnCollection.Add($dataObject)
	}
}

SCOM PS Grid Widget In Action

Where to learn more?

Conclusion

The only thing I can write here is that I like this widget and hope to see more features like this. I like it so much, so I will use it in my demo during my DCIM-B415 session at TechEd. See you there!

 

SCOM Performance View vs. Performance Widget – What’s the Difference?

System Center Operations Manager has many talents. And the one I like most of all is its ability to bring surprises every day. Today’s surprise is how it delivers performance data to the user. If you know a lot about SCOM architecture, it will not take long to understand and absorb SCOM’s specialties. If not – you’ll be surprised :).

Performance data.

Most users think that there are two ways to get performance data collected by SCOM:

  1. Console
  2. Reports

That’s wrong. There are three ways:

  1. Console – performance view
  2. Console – performance widget
  3. Reports

Why three? Because performance views and performance widgets are different. Very different.

Difference #1: data source

SCOM writes collected data into 2 (two) databases – operational database and data warehouse. SCOM Performance view consumes data from the operational database, so you always get raw data. Performance widget is different – it consumes data from (surprise!) SCOM data warehouse.

Difference #2: aggregation

What? Aggregation in console? Yes! Performance view deal with raw data only, so when you try to view data for a long period of time, you get many samples. Long time range – many data points. Short time range – not-so-many data points. That’s it. Simple.

Performance widget is different. Take a look at the signature of SDK.Microsoft_SystemCenter_Visualization_Library_SinglePerformanceDataSeriesGet stored procedure in the data warehouse, and you will find a @NumberOfDataPoints parameter. Performance widget always requests 100 data points and expects to get 100 data points. So, if there are too many data points within the selected time range, the stored procedure will do “reduction” (i.e. re-aggregation) or will switch to hourly-aggregated values. Or it will do both. In the worst case, you’ll get daily aggregated values plus reduction.

Illustrations: SCOM Performance View vs. SCOM Performance Widget

Some performance counter, performance view, 8 hours:

SCOM Performance view 8 hours

Same performance counter, performance widget, 8 hours (looks alike):

SCOM Performance Widget 8 hours

Same performance counter, performance view, 24 hours:

SCOM Performance View 24 hours

Same performance counter, performance widget, 24 hours (looks very different):

SCOM Performance Widget 24 hours

 

Conclusion

SCOM Performance View and Performance Widget behave differently if the number of data points to be drawn is greater than 100. Is that a bug? Absolutely not. That is a feature which allows you to do some historical research right in the console. Just be aware of this behavior and keep in mind that you do not need to go for reporting if you need performance data for a long time range.

 

SCOM Standard Dataset maintenance workflow

As I’m going through my preparations for TechEd, I’m refreshing my knowledge about System Center Operations Manager. A while ago I went through all DW SPs involved into standard dataset maintenance and made some short notes about the implementation of this workflow. Here they are.

What’s a dataset?

Dataset is a set of tables, stored procedures and corresponding SCOM Write Actions. Dataset should be defined in the Management Pack using DataWarehouseDataSet element. SCOM Datasets are used to store historical monitoring data of certain structure. Datasets available out-of-box:

Dataset Database Schema Is standard?
Performance Perf Yes
State State Yes
Event Event Yes
Alerts Alert Yes
Client Monitoring CM Yes
APM apm No

 

What’s a STANDARD dataset?

SCOM provides a framework which implements a standardized approach for performing routine maintenance tasks like staging are processing, aggregation, partitioning, grooming and index maintenance. Dataset creation script should add necessary information into standard dataset configuration tables to enable correct functioning of maintenance workflow.

How does standard dataset processing work?

Standard dataset processing is implemented as a set of SQL Server Stored Procedures which are executed by “Standard Data Warehouse Data Set maintenance rule”. Maintenance frequency is 60 seconds by default.

The maintenance rule is targeted to “Standard Data Set” class and takes dataset ID as a parameter. The rule calls StandardDatasetMaintenance stored procedure.

The workflow.

Note #1: this is a rough description of what happens inside of StandardDatasetMaintenance stored procedure. Some details may be omitted.

Note #2: Table names and field names are italic, stored procedure names are red, calls for stored procedures defined by datasets themselvs are green.

Continue reading…

 

What’s wrong with optimized performance collection rules in SCOM?

I came across an interesting support case last week. The customer was asking why the optimization had been removed  from performance collection rules. Though I knew the answer, I spent some time on building the lab to showcase the impact of optimization. So, what does “optimization” means in terms of SCOM performance collection? Here is an excerpt from the MSDN article:

Unoptimized data results in a performance data item being returned once every specified polling interval. Optimized data results in a performance data item being returned only when the value delta has changed by the specified tolerance amount.

It clearly says that optimized performance collection rules will not return samples if they are more or less the same. It means that some samples will be literally filtered out and will not be written into both operational database and data warehouse. There are two implications from this kind of “optimization” – one good and one bad:

  1. Good – some space will be saved in both databases;
  2. Bad – There is no way to let data warehouse know that some samples have been collected, but not written, so aggregation logic will never know that a portion of data is missing. As a result, it will use only recorded values for calculation of hourly and daily aggregate values. As a result, aggregate values will be calculated incorrectly if optimization really works and suppresses samples.

Here is an illustration: two performance collection rules (one optimized and one raw) are collecting the same value. The value has a  constant value of 10 during first 30 minutes in each hour and a random value during last 30 minutes. Just take a look at how big is the difference between hourly aggregated values:

20140307_SCOM Performance Collection Optimized vs Raw

Conclusion: optimization of performance collection rules affects an accuracy of aggregated data in SCOM data warehouse. This is especially true for metrics that have periods of volatility and periods stability (due to physical limitations – like CPU Usage % cannot be higher than 100%, or due to workload seasoning – like business hours). Also, folks who are using data stored in the data warehouse for forecasting may get significantly incorrect predictions.

 

Survey 2013: How do you use SCOM reporting? – RESULTS

Here are results of the survey. I’m not going to provide any analysis here – I’ve made some conclusions, you may make your own.

Continue reading…

 

How to configure SQL Server 2012 Data Tools to support OpsMgr reports development

This is just a short note on how to configure SQL Server 2012 Data Tools to support OpsMgr reports development.

Note: Since I do not use Dundas Chart any more, I haven’t tried to install it. SSRS native chart control is the prefered option in my opinion. However, if you still need it, you may try steps described here.

  1. Copy Microsoft.EnterpriseManagement.Reporting.Code.dll from SSRS bin folder to C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\
  2. Add ManagementGroupId key to C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\PreviewProcessingService.exe.config: just copy <appSettings>…</appSettings> section from your ReportingServicesService.exe.config like this:
    <configuration>
      ...
      <appSettings>
        <add key="ManagementGroupId"
          value="Your management group's GUID goes here" />
      </appSettings>
    </configuration>
  3. Modify C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\RSPreviewPolicy.config as follows: open file in text editor and find first <CodeGroup> element. Add this code after the first child element:
      <CodeGroup 
       class="UnionCodeGroup" 
       version="1" 
       PermissionSetName="FullTrust" 
       Name="Microsoft Operations Manager Reporting Code Assembly" 
       Description="Grants the MOM Reporting Code assembly full trust permission.">
        <IMembershipCondition class="AllMembershipCondition" version="1" />	
      </CodeGroup>

    This is how it should look after modification:

    ...
    <CodeGroup 
     class="FirstMatchCodeGroup"
     version="1"
     PermissionSetName="Nothing">
      <IMembershipCondition 
       class="AllMembershipCondition"
       version="1" />
      <CodeGroup 
       class="UnionCodeGroup" 
       version="1" 
       PermissionSetName="FullTrust" 
       Name="Microsoft Operations Manager Reporting Code Assembly" 
       Description="Grants the MOM Reporting Code assembly full trust permission.">
        <IMembershipCondition class="AllMembershipCondition" version="1" />	
      </CodeGroup>
    ...

That’s it! Happy coding!

PS: Instructions for BIDS 2008 are here. If you still want to use BIDS 2005, please refer this post and keep in mind that SQL Server 2005 is out of mainstream support.

PPS: If you experience any issues with your old shared data source – try to recreate it.

 

#WorstPractice – SCOM Management Pack Design: Too narrow, too wide, too complex, too simple

When you choose shoes, you put it on and just feel if it fits or not. What about SCOM Management Pack? Okay, you know how to put it on, how to deploy it into your lab environment. But how to “feel” that it is what YOU need? …
Continue reading…