DAT300DAT300

Notification Services Notification Services Programming: Introduction  Programming: Introduction 

Tete Mensa-AnnanTete Mensa-AnnanProgram ManagerProgram ManagerSQL ServerSQL ServerMicrosoft CorporationMicrosoft Corporation

Focus of this sessionFocus of this session

• Goal:Goal:– Explain Notification Services concepts and Explain Notification Services concepts and

featuresfeatures

• Non-Goal:Non-Goal:– Teach how to develop Notification ApplicationsTeach how to develop Notification Applications

AgendaAgenda

• What is SQL Server Notification ServicesWhat is SQL Server Notification Services– IntroductionIntroduction

• ArchitectureArchitecture– Under the hoodUnder the hood

• Instances, Applications, VersioningInstances, Applications, Versioning– Important conceptsImportant concepts

• DeploymentDeployment– ScenariosScenarios

What is the SQL Server What is the SQL Server Notification Services Platform?Notification Services Platform?• A platform for developing and deploying highly A platform for developing and deploying highly

scalable, rich Notification Applicationsscalable, rich Notification Applications

• A rich yet simple declarative model used to describe Notification A rich yet simple declarative model used to describe Notification ApplicationsApplications

• An efficient and scalable server, based on SQL Server 2000 An efficient and scalable server, based on SQL Server 2000 and .NET Framework, that execute Notification Applicationsand .NET Framework, that execute Notification Applications

– You write Notification Application - Notification Services You write Notification Application - Notification Services hosts and executes ithosts and executes it

Notification ApplicationsNotification Applications

A new class of scalable applications A new class of scalable applications that enable new scenarios delivering that enable new scenarios delivering personalizedpersonalized and and timelytimely information information to to any device.any device.

PersonalPersonal

Any deviceAny device

TimelyTimely

Notification ApplicationsNotification Applications

Events(thousands)

Notifications(millions)

Match

Subscriptions(millions)

Activity Generation Delivery Receipt

SQL Server on Windows,Oracle on Solaris,DB2 on S/390.

Custom Data Store.

SQL Server 2000 Notification Services

Exchange Server,WinRTC,SMS Aggregator,

Custom Mechanism.

Outlook,XP Messenger,Pocket PC,SmartPhone 2002,SMS capable phone,

Custom Mechanism.

Information flow

Generation vs. DeliveryGeneration vs. Delivery

Why Notification Services?Why Notification Services?

• Build and deploy application quicklyBuild and deploy application quickly(Time To Market)(Time To Market)– You customize within Notification Services frameworkYou customize within Notification Services framework– You focus on application, not platform designYou focus on application, not platform design

• RichnessRichness– TimeZones, Recurrence, Fall behind, RetryTimeZones, Recurrence, Fall behind, Retry

• PerformancePerformance– SQL matches Events and Subscriptions efficiently SQL matches Events and Subscriptions efficiently – A true ‘Internet’ scale solutionA true ‘Internet’ scale solution

AgendaAgenda

• What is SQL Server Notification ServicesWhat is SQL Server Notification Services– IntroductionIntroduction

• ArchitectureArchitecture– Under the hoodUnder the hood

• Instances, Applications, VersioningInstances, Applications, Versioning– Important conceptsImportant concepts

• DeploymentDeployment– ScenariosScenarios

Subscription Management ApplicationSubscription Management Application

SubscriptionsSubscriptionsSubscribersSubscribers

CustomCustom

Eve

nt

Eve

nt

Co

llector

Co

llector

Event Provider HostEvent Provider Host

File SystemFile System

SQL ServerSQL Server

NotificationNotificationGeneratorGenerator

SQL MatchSQL MatchRulesRules CustomCustom

FormatterFormatter

DistributorDistributorSMSSMS

EmailEmail

FileFile

CustomCustom

EventsEvents

NotificationsNotificationsEvent BatchesEvent Batches

Notification BatchesNotification Batches

Subscriber Subscriber DevicesDevices

App. specificApp. specific

StandardStandard

XSLT XSLT FormatterFormatter

Architecture OverviewArchitecture Overview

Architecture DetailsArchitecture Details

Subscription Management Subscription Management ApplicationApplication

Subscription Management ApplicationSubscription Management Application

• Allows subscribersAllows subscribersto manage subscriptionsto manage subscriptions

• Can be written in:Can be written in:– Managed code: ASP .NET, Managed code: ASP .NET,

C#, VB.Net, etcC#, VB.Net, etc

– Unmanaged code: ASP, Unmanaged code: ASP, C++, VB6, etcC++, VB6, etc

• Deal with three classesDeal with three classes– SubscriberSubscriber• Represents Represents whowho gets informationgets information• Usually one entry per personUsually one entry per person

– SubscriptionSubscription• Represent Represent what & whenwhat & when information is sentinformation is sent• Multiple subscriptions per subscriberMultiple subscriptions per subscriber

– SubscriberDeviceSubscriberDevice• Represents Represents wherewhere to send the information to send the information• Multiple devices per subscriberMultiple devices per subscriber

Subscription Management ApplicationSubscription Management Application

Architecture DetailsArchitecture Details

Event ProvidersEvent Providers

Event ProvidersEvent Providers

• Event Providers collect data changes and submits them as Event Providers collect data changes and submits them as Events in Event BatchesEvents in Event Batches

• Notification Applications have one or more Event ProviderNotification Applications have one or more Event Provider

• Event Providers map to Event ClassesEvent Providers map to Event Classes

• Notification Services ship with two standard providersNotification Services ship with two standard providers

– File System Watcher Event ProviderFile System Watcher Event Provider– SQL Server Event ProviderSQL Server Event Provider

Custom Event ProvidersCustom Event Providers

• Hosted Event ProviderHosted Event Provider– Run in-proc with Notification Services Windows ServiceRun in-proc with Notification Services Windows Service– Must implement IEventProvider or IScheduledEventProvider interfacesMust implement IEventProvider or IScheduledEventProvider interfaces– Less custom code by leveraging Initialize, Run, Terminate, ScheduleLess custom code by leveraging Initialize, Run, Terminate, Schedule– Starts and Stops with Notification Services Windows ServiceStarts and Stops with Notification Services Windows Service

• Non-Hosted Event ProviderNon-Hosted Event Provider– Can run on alternate platform or remote systemCan run on alternate platform or remote system– Starts and Stops independently from Notification Services Windows ServiceStarts and Stops independently from Notification Services Windows Service

• Event Submission MethodsEvent Submission Methods– Event ObjectEvent Object– XMLLoader ObjectXMLLoader Object– Event Submission SQL Server Stored ProceduresEvent Submission SQL Server Stored Procedures

• A Custom Event ProviderA Custom Event Provider– Accepts event data from an external sourceAccepts event data from an external source– Creates and populates an event object with event dataCreates and populates an event object with event data– Calls Event Collector “Write” for each eventCalls Event Collector “Write” for each event– Calls Event Collector “Commit” to submit the batch of eventsCalls Event Collector “Commit” to submit the batch of events

EventsEventseventseventsWriteWriteWriteWriteWriteWrite

CommitCommitProvider HostProvider Host

Ev

ent

Ev

ent

Co

llecto

rC

ollec

tor

CustomCustomEvent ProviderEvent Provider

EventEventIBMIBM 102 ½102 ½MSFTMSFT 65 ¼65 ¼ORCLORCL 8 ¼ 8 ¼

IBMIBM 102 ½102 ½

MSFTMSFT 65 ¼65 ¼

ORCLORCL 8 ¼8 ¼

Event Object SubmissionsEvent Object Submissions

XMLLoader Object SubmissionsXMLLoader Object Submissions

• A Custom Event ProviderA Custom Event Provider– Accepts event data from an external sourceAccepts event data from an external source– Creates an XMLLoader object; optionally initialized with SQL Creates an XMLLoader object; optionally initialized with SQL

annotated XML event schemaannotated XML event schema– Called with an XML document or stream Called with an XML document or stream

• Batch size = the number of events within the document or streamBatch size = the number of events within the document or stream

EventsEventseventsevents

LoadXMLLoadXML

Provider HostProvider Host

XM

LX

ML

Ev

ent

Ev

ent

ProviderProvider

Event Submission SQL Server Stored Event Submission SQL Server Stored ProceduresProcedures

Un-Hosted Custom SQL Server Stored Procedures Provider:Un-Hosted Custom SQL Server Stored Procedures Provider:

DECLARE @eventQuery DECLARE @eventQuery DECLARE @postQueryDECLARE @postQuery

SET @eventQuery = SELECT stocksymbol, stockprice from RDT1SET @eventQuery = SELECT stocksymbol, stockprice from RDT1UNION SELECT stocksymbol, stockprice from RDT2UNION SELECT stocksymbol, stockprice from RDT2

SET @postQuery = DELETE RDT1 DELETE RDT2SET @postQuery = DELETE RDT1 DELETE RDT2

EXEC NSEventSubmitBatchStockEvent @providerName, @eventQuery, @postQueryEXEC NSEventSubmitBatchStockEvent @providerName, @eventQuery, @postQuery

EventsEvents

SubmitSubmit

SQ

L S

tored

SQ

L S

tored

Pro

cP

rocProviderProvider

Raw DataRaw DataTable 1Table 1

Raw DataRaw DataTable 2Table 2

Architecture DetailsArchitecture Details

GeneratorGenerator

GeneratorGenerator• Matches events and subscriptions to produce Matches events and subscriptions to produce

notificationsnotifications• You write “match rules”You write “match rules”

with full power of SQLwith full power of SQL(Transact-SQL, sp, transactions…)(Transact-SQL, sp, transactions…)

MatchMatchRuleRule

GeneratorGenerator

SELECT Notify(SELECT Notify(S.SubscriberId, S.SubscriberId, S.DeviceName,S.DeviceName,S.Locale, S.Locale, S.Symbol, S.Symbol, E.Price) E.Price)

FROM FROM StockEvents E, StockEvents E, StockSubscriptions S StockSubscriptions S

WHERE WHERE E.Symbol = S.Symbol AND E.Symbol = S.Symbol AND E.Price >= S.TriggerPriceE.Price >= S.TriggerPrice

SubscribersSubscribers

Stock Stock SubscriptionsSubscriptions

Subscriber Subscriber DevicesDevices

Stock Stock EventsEvents

Stock Stock NotificationsNotifications

Rule ClassificationsRule Classifications

• Notification generation rulesNotification generation rules– Subscription Event rulesSubscription Event rules

• Multiple per Event ClassMultiple per Event Class• Multiple per Subscription ClassMultiple per Subscription Class• Notify when MSFT stock price Notify when MSFT stock price

goes above 70.00goes above 70.00

– Subscription Scheduled rulesSubscription Scheduled rules• Multiple per Subscription ClassMultiple per Subscription Class• TimeZoneTimeZone• RecurrenceRecurrence• Send MSFT stock price at 1PM Send MSFT stock price at 1PM

Mon-FriMon-Fri

• Chronicle state change rulesChronicle state change rules– Event Chronicle RuleEvent Chronicle Rule

• One per event classOne per event class• Used to maintain Event state Used to maintain Event state

like Latest stock priceslike Latest stock prices

– Subscription chronicle updatesSubscription chronicle updates• (usually embedded within (usually embedded within

Event & Scheduled Event & Scheduled Subscription Rules)Subscription Rules)

• Last time document changes Last time document changes were sent to subscriberwere sent to subscriber

Rule Firing SequenceRule Firing Sequence• Rules firingsRules firings

– Event Chronicle Rules Fired firstEvent Chronicle Rules Fired first– Subscription Event Rules fired second (Subscription Chronicle Updates)Subscription Event Rules fired second (Subscription Chronicle Updates)– Subscription Scheduled Rules fired third (Subscription Chronicle Updates)Subscription Scheduled Rules fired third (Subscription Chronicle Updates)

• Two in-order application processing options:Two in-order application processing options:– Quantum orderingQuantum ordering

• All event batches for quantum processed firstAll event batches for quantum processed first• All scheduled subscriptions processed secondAll scheduled subscriptions processed second• Never do more work than one Quantum worth at a timeNever do more work than one Quantum worth at a time

– Strict event batch orderingStrict event batch ordering• Scheduled subscriptions due before event batch arrival are processed firstScheduled subscriptions due before event batch arrival are processed first• Event batch is processedEvent batch is processed• Scheduled subscriptions due after event batch arrival are processed until end of Scheduled subscriptions due after event batch arrival are processed until end of

QuantumQuantum

Generator Falling BehindGenerator Falling Behind

• Generator schedule = QuantumGenerator schedule = Quantum

• Application configurable Quantum limitsApplication configurable Quantum limits– Event Chronicle Rule firing limitEvent Chronicle Rule firing limit

• Don’t process stock event batches older than 24hrsDon’t process stock event batches older than 24hrs• Don’t process news stories older than 12 hoursDon’t process news stories older than 12 hours• Don’t process traffic events older than 2 hoursDon’t process traffic events older than 2 hours

– Subscription Rule firing limitSubscription Rule firing limit• Don’t fall behind more than 15 minutes for basketball notificationsDon’t fall behind more than 15 minutes for basketball notifications• Don’t fall behind more than half hour on weather notificationsDon’t fall behind more than half hour on weather notifications• Don’t fall behind more than 20 minutes on traffic eventsDon’t fall behind more than 20 minutes on traffic events

Architecture DetailsArchitecture Details

DistributorDistributor

DistributorDistributor

• Formats and delivers NotificationsFormats and delivers Notifications– Transforms Notification using XSLT or Custom Content Transforms Notification using XSLT or Custom Content

Formatter moduleFormatter module– Applies device, country and region metadata to format Applies device, country and region metadata to format

the notification body and envelopethe notification body and envelope– Sends the Notification via a delivery protocolSends the Notification via a delivery protocol

NotificationsNotifications

DistributorDistributor SMSSMS

SMTPSMTP

SMSSMS

CustomCustom

ContentContentFormattingFormatting

Distributor OverviewDistributor Overview

• Generator Creates NotificationsGenerator Creates Notifications– Per Notification ClassPer Notification Class– Raw notifications as rows in SQL tablesRaw notifications as rows in SQL tables

• Distributor LoopDistributor Loop– Periodically scan for notification batches ready to be Periodically scan for notification batches ready to be

distributeddistributed– Format raw SQL notification data Format raw SQL notification data – Assemble formatted notifications into protocol packetsAssemble formatted notifications into protocol packets– Deliver the protocol packetDeliver the protocol packet– Handle delivery failuresHandle delivery failures

Content FormattingContent Formatting

• Format raw SQL Notification dataFormat raw SQL Notification data– Language-ContryRegion and DeviceType Language-ContryRegion and DeviceType

• Specified per NotificationClassSpecified per NotificationClass– Stock notifications are formatted differently from Flight notificationsStock notifications are formatted differently from Flight notifications

• Built-in XSLT Content FormatterBuilt-in XSLT Content Formatter– Applies a XSLT to NotificationsApplies a XSLT to Notifications

• Custom Content FormatterCustom Content Formatter– Must implement IContentFormatterInterfaceMust implement IContentFormatterInterface– Initialize, ContentFormat, Close methodsInitialize, ContentFormat, Close methods

Digest and MulticastDigest and Multicast

• Digest: several individual notifications for the same subscriber are Digest: several individual notifications for the same subscriber are combined into one combined into one – E.g. Stock notifications for MSFT, IBM, CSCO,… for a single subscriber are E.g. Stock notifications for MSFT, IBM, CSCO,… for a single subscriber are

combined to be sent as one notificationcombined to be sent as one notification– Enabled per notification classEnabled per notification class– Customer Satisfaction Customer Satisfaction

• Multicast: same notification sent to multiple subscribers Multicast: same notification sent to multiple subscribers – E.g. Weather report for Seattle is the same for all recipients E.g. Weather report for Seattle is the same for all recipients – Enabled per notification class Enabled per notification class – Formatting is done onceFormatting is done once– Underlying protocol can choose to physically send one message, or to send each Underlying protocol can choose to physically send one message, or to send each

message separatelymessage separately– Huge performance gain for distributor Huge performance gain for distributor

Retry LogicRetry Logic

• Handling delivery failures: If a delivery operation fails, the Handling delivery failures: If a delivery operation fails, the distributor can make multiple attempts distributor can make multiple attempts

• The developer specifies the retry schedule per notification class The developer specifies the retry schedule per notification class (per protocol)(per protocol)– Series of delay intervals Series of delay intervals

• Retry attempts will continue until the entire retry schedule has Retry attempts will continue until the entire retry schedule has been completed, or until the notification expiresbeen completed, or until the notification expires

• Notification is flagged as done and failed if all retry attempts failNotification is flagged as done and failed if all retry attempts fail– Vacuuming cleans up both successful and failed notificationsVacuuming cleans up both successful and failed notifications

AgendaAgenda

• What is SQL Server notification ServicesWhat is SQL Server notification Services– Marketing Fluff IntroductionMarketing Fluff Introduction

• ArchitectureArchitecture– Under the hoodUnder the hood

• Instances, Applications, VersioningInstances, Applications, Versioning– Important conceptsImportant concepts

• DeploymentDeployment– ScenariosScenarios

Notification Services InstanceNotification Services Instance• An Instance hosts one or more Notification Applications that An Instance hosts one or more Notification Applications that

share Subscribersshare Subscribers

• Each Instance runs as a multithreaded Windows ServiceEach Instance runs as a multithreaded Windows Service– Hosts Event Providers, Generator, Distributor Hosts Event Providers, Generator, Distributor

• Multiple Instances per computerMultiple Instances per computer– Test and productionTest and production– Separate Windows Services,Separate Windows Services,

Databases, trust boundariesDatabases, trust boundaries

• Instance may span multipleInstance may span multiplecomputerscomputers– E.g., multiple DistributorsE.g., multiple Distributors NewsStock Sports

Subscribers

Internet01

Notification ApplicationNotification Application

• An Notification Application definesAn Notification Application defines– One or more Event Classes, Subscription Classes (Rules), and One or more Event Classes, Subscription Classes (Rules), and

Notification ClassesNotification Classes

• Each Notification Application is a separate SQL Server Each Notification Application is a separate SQL Server databasedatabase– Event, Subscription, Chronicle and Notification dataEvent, Subscription, Chronicle and Notification data

• System ElementsSystem Elements– Event Providers (input), Generator (rule evaluation), Distributors Event Providers (input), Generator (rule evaluation), Distributors

(output)(output)

Notification Services VersioningNotification Services Versioning

• Different Notification Services Instances running on the Different Notification Services Instances running on the same system may run different versions of Notification same system may run different versions of Notification ServicesServices

• Each Notification Services Instance must run the same Each Notification Services Instance must run the same version across all computersversion across all computers

• Multiple versions of Notification Services may be Multiple versions of Notification Services may be installed side-by-sideinstalled side-by-side

• Installation of a new version does not remove or Installation of a new version does not remove or replace existing versionsreplace existing versions

AgendaAgenda

• What is SQL Server notification ServicesWhat is SQL Server notification Services– Marketing Fluff IntroductionMarketing Fluff Introduction

• ArchitectureArchitecture– Under the hoodUnder the hood

• Instances, Applications, VersioningInstances, Applications, Versioning– Important conceptsImportant concepts

• DeploymentDeployment– ScenariosScenarios

Core Deployment ScenariosCore Deployment Scenarios

SQL / NS SQL NS

Availab

ility

NS NS

SQL SQL

SQLNS

SQLNS

Scenario 1 Scenario 2

EP

GEN

DIST

SQL

Scenario 3

Scenario 4 Scenario 5

Scalability

SecuritySecurity

• Designed with Security in mindDesigned with Security in mind– Windows AuthenticationWindows Authentication– SQL Server AuthenticationSQL Server Authentication

• Typical Accounts (low privileges)Typical Accounts (low privileges)– Notification Services Windows Service AccountNotification Services Windows Service Account

• Domain\NSService + SQL Server NSService LoginDomain\NSService + SQL Server NSService Login

– SQL Server Service AccountSQL Server Service Account• Domain\SQLServiceDomain\SQLService

– ASPNET AccountASPNET Account• Domain\ASPNET + SQL Server ASPNET LoginDomain\ASPNET + SQL Server ASPNET Login

• DBA AccountDBA Account Operations AccountOperations Account• Domain\DBADomain\DBA Domain\OpsDomain\Ops

SummarySummary

• Notification Services is a platformNotification Services is a platform

• Enables rapid development of robust scalable Enables rapid development of robust scalable Notification ApplicationsNotification Applications

• Easy to use Declarative Programming ModelEasy to use Declarative Programming Model

• A True .Net Enterprise Server ProductA True .Net Enterprise Server Product

Questions and AnswersQuestions and Answers

Don’t forget to complete the Don’t forget to complete the on-line Session Feedback form on-line Session Feedback form on the Attendee Web siteon the Attendee Web site

https://web.mseventseurope.com/teched/https://web.mseventseurope.com/teched/