MOM Essentials 4: Extending Microsoft Operations Manager (MOM) 2005 - Part 2Paul Collins

Microsoft UK

Agenda

Creating Custom Reports

Managing Server Security

Creating Custom Reports

Agenda

SQL Server 2000 Reporting Services

MOM data flow

Reporting wizard

Complex reports and large data sets

Tips and tricks

Reporting database schema

Packaging reports for distribution

SQL Server 2000 Reporting Services

SQL Server CatalogSQL Server Catalog

Report ServerReport Server

XML Web Service InterfaceXML Web Service Interface

Report ProcessingReport Processing

DeliveryDelivery

Delivery TargetsDelivery Targets(E-mail, SharePoint,(E-mail, SharePoint,

Custom)Custom)

RenderingRendering

Output Formats Output Formats (HTML, Excel, PDF, Custom)(HTML, Excel, PDF, Custom)

Data ProcessingData Processing

Data SourcesData Sources(SQL, OLE DB, XML/A, (SQL, OLE DB, XML/A, ODBC, Oracle, Custom)ODBC, Oracle, Custom)

SecuritySecurity

Security ServicesSecurity Services(NT, Passport, (NT, Passport,

Custom)Custom)

OfficeOffice Custom ApplicationCustom ApplicationBrowserBrowser

SQL Server 2000 Reporting Services

Free add-on to SQLhttp://www.microsoft.com/sql/reporting/default.asp

–Version for Standard and Enterprise

–You can only use the version that matches the version of SQL Server you own

Designed by SQL Server team

Report creation is Visual Studio 2003–Recently bought Active Views to make ad-hoc reporting easier

Very scalable

Very powerful

Can be used for all reporting needs – not just MOM

MOM Data Flow

PerformancePerformance

Alerts/EventsAlerts/Events

Service DiscoveryService Discovery

Rules, etc.Rules, etc.

DTS

Database

MPReporting Database

1.1. Management Pack is installedManagement Pack is installed It containsIt contains

Performance Collection RulesPerformance Collection RulesEvent Collection RulesEvent Collection RulesAlert RulesAlert RulesService DiscoveryService DiscoveryMOM Configuration InformationMOM Configuration Information

3.3. The DTS transfers the data using a Windows Scheduled Task The DTS transfers the data using a Windows Scheduled Task

on the Reporting Serveron the Reporting Server

ReportsReports

4.4. Reports are run on the data in the reporting databaseReports are run on the data in the reporting database

2.2. Data is collected in the MOM databaseData is collected in the MOM database

MOM Data Flow

MOM operational databaseMOM operational database(Onepoint)(Onepoint)

Periodic snapshotPeriodic snapshotRows are moved every time by DTSRows are moved every time by DTS

AttributesAttributesComputer Group MembershipComputer Group MembershipComputer to Rule MembershipComputer to Rule Membership

....

v

Additive factsAdditive factsRows are added every time by DTSRows are added every time by DTS

Samplednumericdata=Performance CounterSamplednumericdata=Performance CounterAlertsAlertsEventsEvents

MOM ReportingMOM Reporting(SystemCenterReporting)(SystemCenterReporting)

The first run of the DTS creates the views in the reporting database- the Transaction log can grow to 2-3 times the amount of data being transferred during one run

Data is stored in the Reporting Server in two different ways

Reporting OverviewBased on SQL Reporting Service and the System Center Data Warehouse:

– Long term data storage– Customization– Dynamic/Sophisticated reports – Per report security– Exporting data to other formats

Service specific reports out of box– Summary reports– Capacity and performance trend graphs– Operations reports– Resources– Availability and Reliability

Capabilities– View or print – Publish to Web site– Schedule generation offline

System Center Data Warehouse

Excellent Tool for Managing High Volumes of Data–Long term offline storage and analysis of data

– Default is 13 months

– In built data transformation and management functionality

– Improved grooming for both data warehouse and DB

–Star schema for better analytics

Reporting

System Center Data Warehouse

Reporting ImprovementsFeature Component MOM 2000 MOM 2005

Report Manager Job Engine Access SQL Reporting Svc

DataWarehouse Built-in w/ Grooming

Design Reports Create / Modify

Import / Export (RDL)

Rendering Dynamic Parameters

Charts

Drill Through/Hide/ Sort

Output Excel, XML, PDF

HTML Archive

Management Cache / Snapshot / Search / Linked

Subscription Email, Data-Driven

Scalability Web Farm

Security Windows / Roles

Programmability API, Web Service, WMI

Partial supportPartial support Full supportFull support

Pre-RequisitesWin2000 with SP4, all editions Windows 2003, all editions, XPASP.NET 1.1, IIS 5.0 or later installed and configured, MDAC 2.6 or higher.SQL Server 2000 SP3a. For Windows 2003, the computer must be configured as an application server. For Windows 2003 to use the network service account to run the ReportServer service, you need SQL Server QFE 859. Install Visual Studio 2003Install SQL 2000 Reporting Services according to your SQL VersionInstall System Center Data WarehouseDefault website accessible through http://<servername>/ReportserverSystem Center Data Warehouse Reports through http://<servername>/reports

SQL Server 2005 - Report Builder included

Reporting Services Walkthrough

Tony Clarke

Microsoft UK

Building a Simple Report

Building a simple report is easy using the reporting wizard

The reporting wizard allows you to select all of the required options for your report including:

–Data source

–SQL Query

–Report look and feel

–Fields to show on your report

A Simple Report

Tony Clarke

Microsoft UK

Report Features

Underlined Underlined infromation is infromation is

a link to a a link to a detailed reportdetailed report

Print the Print the parameters parameters

selectedselected

Logo can be Logo can be replaced replaced globallyglobally

This is a group This is a group header. Sorting header. Sorting works best on works best on

this levelthis level

The next group The next group header is only header is only

visible after visible after expandingexpanding

Creating more Complex Reports

Use the Microsoft reports as a starting point when creating more complex reports

Save an existing report to RDL and open in SQL Reporting Services

Modify the look and feel as required

Edit the report to show the data you would like to see

A More Complex Report

Tony Clarke

Microsoft UK

Working With Large Volumes Of DataParameter Selection

SELECT *FROM fn_ListComputerGroups() fn_ListComputerGroups_1

SELECT SUBSTRING(Name, 1, 100) AS CompGroup FROM SC_ComputerRuleDimension_View CRD UNION SELECT '<ALL>'

xxx FUNCTION dbo.[fn_ExpandString](@inputstring nvarchar(1024))RETURNS nvarchar(1024)AS BEGIN DECLARE @ExpandedString nvarchar(1024) SET @ExpandedString = (CASE WHEN @inputstring = '<ALL>' THEN '%' ELSE @inputstring END) RETURN (@ExpandedString)ENDGOSET QUOTED_IDENTIFIER OFF GOSET ANSI_NULLS ON GO

WHERE(b.FullComputerName

LIKE dbo.fn_ExpandString(@Computer))

Step 1: Filling the drop-down with data

Step 2: Adding the Wildcard “<ALL>”

Step 3: Referencing the expandstringfunction in your query

Step 4: Expanding the String

Working With Large Volumes Of DataShow only relevant Information

Toggle field visibility by logical group headers

On the advanced properties of this Row visibility is toggled and made dependent on the upper columnDo it by row, not by the single field

This field has in the advanced properties visibility set to collapsed (+) as a default

Working With Large Volumes Of Data

Tony Clarke

Microsoft UK

Report Planning

Data collectionData should be collected by MOM that you can Report on. If not already done you must create the rules to collect the data

Using Parameters to handle large data volumesReport header allows choice of parameters

–FilteringA filter is a field where the whole results are impacted

–SortingAllow to sort the columns in a Report

–GroupingAllows to display data in groups and reduce the amount of data presented.

Things not to do

Don’t create a simple 100 pages long list- develop the Report based on a scenario e.g. IIS Server Reports were planned as:“give me all IIS Servers with the supported options and capacity to decide where I host that application”or “show me what Server has which Version of ASP .NET installed”

The faster you get to your result the better – details can be on a linked Report

Reporting Services Settings

This is the setting in This is the setting in Visual Studio Reporting that Visual Studio Reporting that controls where to put the controls where to put the Reports on the Reporting Reports on the Reporting WebsiteWebsite

Set Data Source to SCDWSet Data Source to SCDW

Reason: If you deploy the Reason: If you deploy the Reports you want to have the Reports you want to have the Data Source installed on the Data Source installed on the Target System. Target System.

As this is the MOM Data source As this is the MOM Data source you can re-use it.you can re-use it.

Please use this TargetFolder for your

Custom Reports

Tips and Tricks

Don’t install SQL Server 2000 Reporting Services to be the default Website – it might break other WebsitesTest export the Report first to PDF, then to Excel to see the results during your developmentSQL Server 2000 Reporting Services does not support multi-select in Parameters. Use Computergroups insteadReports execute automatically when opened – watch out what you put in as Parameter defaultsSQL Server 2000 Reporting Services does not support multiple queries for a single group – if a query is getting to complex use Stored Procedures and call the from the Report

Tips and Tricks – contd.

Convert all UTC date from DB to Local date using this function in the SQL Query dbo.fn_ToLocalDate(Date, GETUTCDATE(), GETDATE())

display the Server/Agent Name with Domain name attached eg. COALESCE(CD.ComputerDomain_PK+'\'+CD.ComputerName_PK,CD.ComputerName_PK)

Have a no data text available in the description of the Report. The no data areas offered by Rosetta are too small. When you collect data explain which Rules need to be enabled for this Report in the no data text.

Use “Begin Date” and “End Date” as standard parameters

Print Sort By and Sort Order as standard parameter on the Report together with all filters entered

Use SCDW as datasource name

Have a runtime of no longer than 30 sec - Users don’t like to wait

Querying Data

The reporting database schema is documented in the SDK

Computer Rule to PRG Relationship

SC_ComputerRuleToProcessRuleGroupFact_view

PK SMC_InstanceID int identity

ComputerRule_FK bigint DateAdded_FK bigint DateTimeAdded datetime ProcessRuleGroup_FK bigint TimeAdded_FK bigint ConfigurationGroup_FK bigint DateTimeOfTransfer datetime

SC_ComputerDimension_view

PK ComputerDomain_PK nvarchar(100)PK ComputerName_PK nvarchar(255)

ComputerID uniqueidentifier DNSName nvarchar(512) IsAgent bit IsCollector bit SMC_InstanceID int identity ComputerType int DateTimeLastContacted datetime Description nvarchar(255) FullComputerName nvarchar(512) SC_TimeDimension_view

PK Hour_PK intPK Minute_PK intPK Second_PK int

AMPM nvarchar(2) SMC_InstanceID int identity TimeOfDay datetime

SC_DateDimension_view

PK DateDay_PK intPK DateMonth_PK intPK DateYear_PK int

Date datetime SMC_InstanceID int identity

SC_ProcessRuleDimension_view

PK ProcessRuleID_PK uniqueidentifier

IsRuleGroup bit ProcessRuleName nvarchar(255) ProviderDetail_FK bigint SMC_InstanceID int identity

Reporting Schema

Dimension Tables e.g.–SC_ComputerDimension

–SC_AlertLevelDimentions

Fact Tables e.g.–SC_AlertFact_Table

–SC_EventFact_Table

Periodic Snapshot Fact Tables e.g.–SC_ClassAttributeInstanceFact_Table

–SC_ProcessRuleMembershipFact_Table

Deployment of Reports

Command line tool: RptUtil.exe creates .xml file which can be imported using the MOM Admin consoleThe XML file can contain 1-n Reports

/action: Action - import or export./file: Full path to the import or export file./url: Url of the report server./reportpath: Path to the report or report folder to be exported./fromdsref: Name of the "from" datasource reference to fixup./todsref: Name of the "to" datasource reference to fixup./datasource:Name of the datasource to fixup./dwserver: Name of the datawarehouse server used to fixup the datasource. /dwdb: Name of the datawarehouse database used to fixup the datasource.

cd "E:\Program Files\Microsoft System Center Reporting\Reporting"rptutil.exe /file:e:\myalertlatency.xml /nowarn /reportpath:"/custom reports/alert logging latency"

Example for a Batch file to export a Report to xml:

Summary

Build your queries first

Use a Reports folder outside of “Microsoft Operations Manager Reporting”

Do not overload Reports with too much data – use linked reports if necessary

Use a template so your reports have a consistent look and feel

Test your output to Excel and PDF

Technical Resources

Developing Custom MOM ReportsDeveloping Custom MOM Reports– http://www.microsoft.com/technet/prodtechnol/mom/mom2005/http://www.microsoft.com/technet/prodtechnol/mom/mom2005/

Library/cf1e57a0-ecb1-4f42-a8ef-4d43aa3e8d44.mspx?mfr=true Library/cf1e57a0-ecb1-4f42-a8ef-4d43aa3e8d44.mspx?mfr=true

Download custom management pack Download custom management pack – http://www.microsoft.com/downloads/details.aspx?http://www.microsoft.com/downloads/details.aspx?

familyid=c5b42e5b-68ed-45ea-8864-familyid=c5b42e5b-68ed-45ea-8864-a9d4087d261d&displaylang=en a9d4087d261d&displaylang=en

Information about SQL Server viewsInformation about SQL Server views– http://msdn.microsoft.com/library/default.asp?url=/library/en-us/http://msdn.microsoft.com/library/default.asp?url=/library/en-us/

createdb/cm_8_des_06_9mlv.asp createdb/cm_8_des_06_9mlv.asp

MOM SDK 2.0MOM SDK 2.0– http://www.microsoft.com/mom/downloads/sdk/default.asp http://www.microsoft.com/mom/downloads/sdk/default.asp

SQL Server Reporting ServicesSQL Server Reporting Services– http://www.microsoft.com/sql/reporting http://www.microsoft.com/sql/reporting

Managing Server Security

Three Main Factors For Effective Application Monitoring

Exception Monitoring–What errors are occurring in my application?

Performance Monitoring–How is my application performing?

Security– Is my application secure?

Collecting Security Information

Is my application fully patched?– SMS, MBSA

Who is accessing my application?– Security Auditing (event log\app log)

What is my configuration– Registry, DCM

Types of Security data

Pro-active alerting– i.e. Critical changes to groups, multiple logon failures, rogue

processes etc.

Auditing–Workstation access, file access, who made changes and when

Configuration–What is my current configuration, do I meet best practices?

MBSA Management Pack

Vulnerability Assessment (VA) Check

– Give our customers advance, proactive, and regular notification of any Vulnerabilities

Patch Alerting

– Notify MOM administrators that their servers require patching

– Integrates with SMS

– DOES NOT deploy or install patches

VA checks in 4 Areas

– Internet Explorer

– IIS Server

– SQL Server

– Windows

Patch Scanning

– Leverages mssecure.cab

Security Event Log

Focus on Domain Controller Logs

Turn on auditing for logon events

Create rules based on Event ID’s – e.g. 529 (logon failure)

Use filter rules to block system accounts– i.e. NTService$

Consolidate rules to identify attacks– i.e. 15 x event id 529 in 5 minute period may indicate an attack

Security Event Collection Issues

Very large volume of events can impact database size and may effect performance

–Consider second management group with separate database and multi-home agents

Lots of Security Alerts may take emphasis away from operational alerts

–Only display critical alerts, use reports for analysis

System Controls MP for MOM 2005Auditing & Reporting Services for Windows Server Security Events

Compliancy Quandrum

Audit Control SEC Event/s FISMA

User Account Changes

Sensitive Data Access

Logon Failures

Privilege Use

404-3.1

404-3.1

404-3.1

404-3.1

642

560; 566

529;675

577;578

AC-3

SI-7, SI-12

AC-7

AU-9

Above diagram is an example for reference only and does not represent actual mappingsAbove diagram is an example for reference only and does not represent actual mappings

Security Best Practices and Governmental Regulations all require some level of auditing for Security Events.

Requires Controls, Segregation of Duties and Reporting

Event Collection & Control mapping add Clarity

Windows Server Security Auditing

System Controls Management Pack

Security Event collection (Parameter Extraction)

Cross-regulation applicability (FISMA, SOX, GLBA)

Predefined Control Rules (ie Domain Admin Member Added)

– Knowledge Content supplemented by Randy Franklin Smith’s Online Event Encyclopedia available at www.UltimateWindowsSecurity.com

Expanded operational views (over 50 event views)

Security Auditing Tasks (SCA, Policy Editor, secedit /analyze)

Comprehensive reporting (Detail & Summary)

Sample Auditing Scenarios

Scenario Scenario Description

Account Management Monitor User and Group Account Management Activities

Account Misuse Monitor Logon Failure Activity

Audit and Accountability Audit Confidentiality, Integrity and Availability of the Security Event Log

Incident Response Monitoring and Alerting for specific Controls

Privilege Use Monitor User Privilege Rights Use for Key Controls

Employee Provisioning Monitor account activation, deactivation, creation and deletions

Policy Changes Monitor User and Group Policy Changes

Sensitive Data Access Monitor User Access to Sensitive Folders and Files

Scenarios are supported via collection, alerting and reporting features of the SCMPScenarios are supported via collection, alerting and reporting features of the SCMP

SCMP Benefits

Leverages MOM Infrastructure

Rapid Deployment & Extensibility

Facilitates Internal Control Auditing

Authoritative Knowledge Content

Comprehensive Reporting Services

Centralized Security Event Collection

Collection Rules

Control Rules

Rule Knowledge

Operational Views

SCMP Security Reports

SCMP Reporting

SCMP Demo

System Controls Event Collection

Operational Views

Custom Control Auditing

System Controls Reporting

Tony Clarke

Microsoft UK

Summary

Monitoring Security Events Best Practice–Create a separate Management Group

–Multi-home the agents

Ensure the appropriate Audit Policy settings have been defined and implemented

Enable only those rules that are required

Create your own rules or use a 3rd Party option e.g. Secure Vantage or NetIQ

MOM Resources

Microsoft Operations Manager http://www.microsoft.com/MOM

Getting Started Resourceshttp://www.microsoft.com/MOM/Beginners

– Technical Walkthrough– Key Documentation– MOM Evaluation Download

Partner Product Cataloghttp://www.microsoft.com/MOM/ManagementPacks

MOM Communityhttp://www.microsoft.com/MOM/community/

Solution Acceleratorshttp://www.microsoft.com/mom/evaluation/solutions/default.mspx

TechNet Information

FREE fortnightly technical newsletter: “The TechNet Flash”

FREE regular technical events hosted across the UK

FREE quarterly technical magazine – “TechNet”

FREE weekly UK & US led technical webcasts

FREE comprehensive technical web site

Monthly CD / DVD subscription with the latest technical tools & resources and full-version evaluation and beta software. 30% off until 31 March 2006

We would like to bring your attention to the key elements of the TechNet programme; the central information and community resource for IT professionals in the UK:

To subscribe to the newsletter or just to find out more, please visit www.microsoft.com/uk/technet or speak to a Microsoft representative during the break

http://www.microsoft.com/uk/technet