New Features of Message New Features of Message Queuing in Windows .NET Queuing in Windows .NET Server 2003Server 2003

Shai KarivShai KarivProgram ManagerProgram ManagerMSMQMSMQMicrosoft CorporationMicrosoft Corporation

Before We Start…Before We Start…

Win06: Tips & Tricks, later today!Win06: Tips & Tricks, later today! MSMQ background:MSMQ background:

MSMQ 1.0 in Windows NT 4.0MSMQ 1.0 in Windows NT 4.0 MSMQ 2.0 in Windows 2000MSMQ 2.0 in Windows 2000

Platform integration: AD, Clustering, MMCPlatform integration: AD, Clustering, MMC Add-on services like TriggersAdd-on services like Triggers

MSMQ 3.0 in XP and .NET Server 2003MSMQ 3.0 in XP and .NET Server 2003 Internet messagingInternet messaging 1-to-many messaging1-to-many messaging Integrating Triggers, extending Admin APIIntegrating Triggers, extending Admin API

System.MessagingSystem.Messaging

AgendaAgenda

Reliable 1-N MessagingReliable 1-N Messaging Distribution ListsDistribution Lists Reliable MulticastReliable Multicast

Reliable Internet MessagingReliable Internet Messaging Messaging Over SOAP/HTTPMessaging Over SOAP/HTTP Firewalls and Load BalancingFirewalls and Load Balancing

Admin APIAdmin API Deployment and StorageDeployment and Storage TriggersTriggers

Distribution ListsDistribution ListsOverviewOverview

Software “fan-out”Software “fan-out” Pub/Sub primitivesPub/Sub primitives End-to-end semantics forEnd-to-end semantics for

Delivery guaranteesDelivery guarantees SecuritySecurity TransactionsTransactions

Implementation: 1-1 “internally”Implementation: 1-1 “internally”

Distribution ListsDistribution ListsPublic DLsPublic DLs

Registered in Active DirectoryRegistered in Active Directory Leverage AD Group objectLeverage AD Group object

Programmable via ADSI callsProgrammable via ADSI calls Manageable via AD MMC snap-insManageable via AD MMC snap-ins

New format name: New format name: DL=<GUID>DL=<GUID> Elements can be other DLsElements can be other DLs

Dim dest as New MSMQDestinationDim dest as New MSMQDestination

dest.FormatName = “DL=<GUID>”dest.FormatName = “DL=<GUID>”

Message.Send destMessage.Send dest

Distribution ListsDistribution ListsPrivate DLsPrivate DLs

Client-managed “DL”:Client-managed “DL”: List of format namesList of format names Including new format names: DL, HTTPIncluding new format names: DL, HTTP Non-discoverableNon-discoverable

Dim dest as New MSMQDestinationDim dest as New MSMQDestination

dest.FormatName = “DIRECT=OS:host1\dest.FormatName = “DIRECT=OS:host1\q”,”PRIVATE=…”q”,”PRIVATE=…”

Message.Send destMessage.Send dest

Distribution ListsDistribution ListsTransactionsTransactions

Identical to queue transactionsIdentical to queue transactions Multiple transactions to same DL: In-Multiple transactions to same DL: In-

Order deliveryOrder delivery If sent from the same computerIf sent from the same computer

No in-order guarantee for messages No in-order guarantee for messages sent to different DLssent to different DLs Even if DLs contain the same queuesEven if DLs contain the same queues

Distribution ListsDistribution Lists

AgendaAgenda

Reliable 1-N MessagingReliable 1-N Messaging Distribution ListsDistribution Lists Reliable MulticastReliable Multicast

Reliable Internet MessagingReliable Internet Messaging Messaging Over SOAP/HTTPMessaging Over SOAP/HTTP Firewalls and Load BalancingFirewalls and Load Balancing

Admin APIAdmin API Deployment and StorageDeployment and Storage TriggersTriggers

Reliable MulticastReliable MulticastOverviewOverview

Hardware “fan-out”: Efficient send to Hardware “fan-out”: Efficient send to multiple recipientsmultiple recipients

Expose and leverage IP multicastExpose and leverage IP multicast MSMQ provides offline, persistencyMSMQ provides offline, persistency

Provide building blocks for Pub/SubProvide building blocks for Pub/Sub Scenario: real-time market data push Scenario: real-time market data push

(financial sector)(financial sector) Protocol: Pragmatic General MulticastProtocol: Pragmatic General Multicast

Reliable MulticastReliable MulticastPGM: Pragmatic General MulticastPGM: Pragmatic General Multicast

Joint Microsoft/Cisco effortJoint Microsoft/Cisco effort Public specificationPublic specification Cisco routers provide optimizationCisco routers provide optimization

Reliable multicast transportReliable multicast transport Receiver either gets all transmission and Receiver either gets all transmission and

repair datarepair data Or able to detect unrecoverable data lossOr able to detect unrecoverable data loss

Members may join/leave at any timeMembers may join/leave at any time

PGM Data FlowPGM Data FlowPGM host sends a multicast packetPGM host sends a multicast packet

SenderSender

ReceiverReceiverSSRR

= Multicast ODATA= Multicast ODATA

RRRR

RR

RR

PGM Data FlowPGM Data FlowA recipient sends (unicast) NAKA recipient sends (unicast) NAK

Unicast NAKUnicast NAK

SenderSender

ReceiverReceiverSSRR

RRRR

RR

RR

PGM Data FlowPGM Data FlowRouter multicasts NCF in LANRouter multicasts NCF in LAN

SenderSender

ReceiverReceiverSSRR

= Multicast ODATA= Multicast ODATA

RRRR

RR

RR

PGM Data FlowPGM Data FlowRouter unicasts NAK back to senderRouter unicasts NAK back to sender

SenderSender

ReceiverReceiverSSRR

RRRR

RR

RR

PGM Data FlowPGM Data FlowSender multicasts repair dataSender multicasts repair data

SenderSender

ReceiverReceiverSSRR

= Multicast ODATA= Multicast ODATA

RRRR

RR

RR

Reliable MulticastReliable MulticastConfiguration and SecurityConfiguration and Security

Receiver configuration: Receiver configuration: New queue property: New queue property: MulticastAddressMulticastAddress A queue can join one multicast groupA queue can join one multicast group

Sender configuration:Sender configuration: Max send rateMax send rate Max time the repair data is availableMax time the repair data is available

Security:Security: Existing authN/authZ (per queue)Existing authN/authZ (per queue) Encryption only at application levelEncryption only at application level

Reliable MulticastReliable MulticastProgramming modelProgramming model

Receiver side:Receiver side: New queue property: MulticastAddress New queue property: MulticastAddress

can be set programmatically…can be set programmatically… When creating the queue or later onWhen creating the queue or later on

Sender side:Sender side: New format name: New format name: MULTICAST=<IP>MULTICAST=<IP> No additional end-to-end semanticsNo additional end-to-end semantics

Dim dest as New MSMQDestinationDim dest as New MSMQDestination

dest.FormatName = “MULTICAST=<address:port>”dest.FormatName = “MULTICAST=<address:port>”

Message.Send destMessage.Send dest

Reliable MulticastReliable Multicast

AgendaAgenda

Reliable 1-N MessagingReliable 1-N Messaging Distribution ListsDistribution Lists Reliable MulticastReliable Multicast

Reliable Internet MessagingReliable Internet Messaging Messaging Over SOAP/HTTPMessaging Over SOAP/HTTP Firewalls and Load BalancingFirewalls and Load Balancing

Admin APIAdmin API Deployment and StorageDeployment and Storage TriggersTriggers

Reliable Web MessagingReliable Web MessagingGoalsGoals

Same programming modelSame programming model Use HTTP as transportUse HTTP as transport SOAP-based protocolSOAP-based protocol Secure messaging across firewallsSecure messaging across firewalls Support Network Load BalancingSupport Network Load Balancing

Reliable Web MessagingReliable Web MessagingProgramming modelProgramming model

Open queue, send messageOpen queue, send message Queue Queue cancan be HTTP-addressed be HTTP-addressed New format name:New format name:

DIRECT=HTTPS://host/MSMQ/qDIRECT=HTTPS://host/MSMQ/q Open queue, receive messageOpen queue, receive message New message properties leverage New message properties leverage

SOAP supportSOAP support SoapHeader, SoapBodySoapHeader, SoapBody SoapEnvelope, CompoundMessageSoapEnvelope, CompoundMessage

Reliable Web MessagingReliable Web MessagingSOAP Reliable Messaging ProtocolSOAP Reliable Messaging Protocol

SRMP: Specification for general-SRMP: Specification for general-purpose reliable messagingpurpose reliable messaging Published specificationPublished specification Extension of WS-RoutingExtension of WS-Routing Implemented by MSMQ 3.0Implemented by MSMQ 3.0

SOAP header is used to provide end-SOAP header is used to provide end-to-end information needed for reliable to-end information needed for reliable messagingmessaging

SRMP Message ElementsSRMP Message Elements

WS-Routing: WS-Routing: pathpathWS-Routing: WS-Routing: pathpath

PropertiesPropertiesPropertiesProperties

ServicesServicesServicesServices StreamStreamStreamStream

Receipts:Receipts:

DeliveryDelivery

CommitmentCommitment

StreamStream

Receipts:Receipts:

DeliveryDelivery

CommitmentCommitment

StreamStream

SOAPSOAPBodyBody

SecuritySecuritySecuritySecurity

SOAP SOAP HeaderHeader

SOAP BodySOAP BodySOAP BodySOAP Body

SRMP Path, PropetiesSRMP Path, Propeties<SOAP-ENV:Header xmlns=http://schema.xmlsoap.org/srmp<SOAP-ENV:Header xmlns=http://schema.xmlsoap.org/srmp

xmlns:rp="http://schemas.xmlsoap.org/rp/“> xmlns:rp="http://schemas.xmlsoap.org/rp/“>

<<rp:pathrp:path SOAPENV:mustUnderstand=“1”> SOAPENV:mustUnderstand=“1”> <<rp:actionrp:action> MSMQ:Message <> MSMQ:Message </rp:action/rp:action>> <<rp:torp:to> > http://server/somethinghttp://server/something < </rp:to/rp:to>> <<rp:idrp:id>> uuid:2288926@ac3fd49c-e7d5-4354-ba8d-3e13fc6f677cuuid:2288926@ac3fd49c-e7d5-4354-ba8d-3e13fc6f677c <</rp:id/rp:id>> <<rp:relatesTorp:relatesTo>> uuid:2288926@ac3fd49d-e7d5-4354-ba8d-3e13fc6f677cuuid:2288926@ac3fd49d-e7d5-4354-ba8d-3e13fc6f677c <</rp:relatesTo/rp:relatesTo>> <</rp:path/rp:path>> <<propertiesproperties SOAP-ENV:mustUnderstand="1“> SOAP-ENV:mustUnderstand="1“> <<expiresAtexpiresAt>20380119T031407<>20380119T031407</expiresAt/expiresAt>> <<sentAtsentAt>20010509T130502<>20010509T130502</sentAt/sentAt>> <</properties/properties>></SOAP-ENV:Header></SOAP-ENV:Header>

SRMP ServicesSRMP Services<services SOAP-ENV:mustUnderstand="1"><services SOAP-ENV:mustUnderstand="1"> <durable/><durable/> <deliveryReceiptRequest><deliveryReceiptRequest> <sendTo><sendTo> http://www.we-love-books.org/receiptshttp://www.we-love-books.org/receipts </sendTo></sendTo> </deliveryReceiptRequest></deliveryReceiptRequest> <commitmentReceiptRequest><commitmentReceiptRequest> <sendTo><sendTo> http://www.we-love-books.org/deliverydonehttp://www.we-love-books.org/deliverydone </sendTo></sendTo> <negativeOnly/><negativeOnly/> </commitmentReceiptRequest></commitmentReceiptRequest></services></services>

SRMP StreamsSRMP Streams<stream SOAP-ENV:mustUnderstand="1"><stream SOAP-ENV:mustUnderstand="1"> <streamId> <streamId> uuid:84b9f5d0-33fb-4a81-b02b-5b760641c1d6uuid:84b9f5d0-33fb-4a81-b02b-5b760641c1d6 </streamId></streamId> <current>1</current><current>1</current> <start><start> <sendReceiptsTo> http://sender/streamAcks/ <sendReceiptsTo> http://sender/streamAcks/

</sendReceiptsTo></sendReceiptsTo> </start></start></stream></stream>

<stream SOAP-ENV:mustUnderstand="1"><stream SOAP-ENV:mustUnderstand="1"> <streamId><streamId> uuid:84b9f5d0-33fb-4a81-b02b-5b760641c1d6uuid:84b9f5d0-33fb-4a81-b02b-5b760641c1d6 </streamId></streamId> <current>7</current><current>7</current> <previous>5</previous><previous>5</previous></stream></stream>

SRMP ExtensibilitySRMP ExtensibilityMSMQ support for extensibilityMSMQ support for extensibility

MSMQ creates the SRMP elementsMSMQ creates the SRMP elements When destination format name is URLWhen destination format name is URL

Receive-side applications Receive-side applications cancan access access the message as XML documentthe message as XML document SoapEnvelope, CompoundMessageSoapEnvelope, CompoundMessage

Send-side applications Send-side applications cancan extend the extend the SOAP messageSOAP message SoapHeader, SoapBodySoapHeader, SoapBody

Internet Messaging / Internet Messaging / SRMPSRMP

Reliable Web MessagingReliable Web MessagingRedirecting SRMP messagesRedirecting SRMP messages

HTTP requests can be redirectedHTTP requests can be redirected SRMP must support redirectionSRMP must support redirection

Redirecting SRMP messages requires Redirecting SRMP messages requires store and forwardstore and forward

MSMQ 3.0 supports store, redirect and MSMQ 3.0 supports store, redirect and forwardforward Using XML files for “mapping”Using XML files for “mapping” Message content does not changeMessage content does not change

Reliable Web MessagingReliable Web MessagingMapping fileMapping file

<from><from> External (known) name of the destinationExternal (known) name of the destination

<to><to> Local resolution for the destinationLocal resolution for the destination

<redirections><redirections>

<redirection><redirection>

<from>https://z.com/msmq/orders</from><from>https://z.com/msmq/orders</from>

<to>https://server1/msmq/q1</to><to>https://server1/msmq/q1</to>

</redirection></redirection>

</redirections></redirections>

Reliable Web MessagingReliable Web MessagingRouting SRMP messagesRouting SRMP messages

Enterprise: MSMQ uses Active Enterprise: MSMQ uses Active Directory for dynamic routingDirectory for dynamic routing

MSMQ does not use Active Directory to MSMQ does not use Active Directory to resolve DIRECT format nameresolve DIRECT format name

Redirect mapping allows Redirect mapping allows connectionless SRMP messagingconnectionless SRMP messaging Useful for workgroup mode as wellUseful for workgroup mode as well

ReceiverReceiverSenderSenderMSMQ MSMQ hosthost

Mapping tableMapping table

Internet Messaging / Internet Messaging / MappingMapping

AgendaAgenda

Reliable 1-N MessagingReliable 1-N Messaging Distribution ListsDistribution Lists Reliable MulticastReliable Multicast

Reliable Internet MessagingReliable Internet Messaging Messaging Over SOAP/HTTPMessaging Over SOAP/HTTP Firewalls and Load BalancingFirewalls and Load Balancing

Admin APIAdmin API Deployment and StorageDeployment and Storage TriggersTriggers

Secure Web MessagingSecure Web MessagingRequirementsRequirements

Do not allow direct TCP sessions from Do not allow direct TCP sessions from Internet to IntranetInternet to Intranet

Do not expose names of internal Do not expose names of internal resources: computers, queuesresources: computers, queues

Do not store sensitive data on Do not store sensitive data on computers in the DMZcomputers in the DMZ

Do not forward unauthenticated Do not forward unauthenticated messages from DMZ to Intranetmessages from DMZ to Intranet

Secure Web MessagingSecure Web MessagingTopologyTopology

Internet DMZ

FirewallFirewall FirewallFirewall

HTTP Proxy

MSMQServer

https://my_company.com/orders

Authenticate

Map

Store

Forward

https://ic/msmq/InQueue

https://your_company.com/acks

Intranet

<sendReceiptsTo>

http://my_company.com/receipts

</sendReceiptsTo>

MSMQ Client

Reliable Web MessagingReliable Web MessagingNLB OverviewNLB Overview

Network load balancing solutions:Network load balancing solutions: Hardware (e.g. Cisco LocalDirector)Hardware (e.g. Cisco LocalDirector) DNS round-robinDNS round-robin Windows Network Load BalancingWindows Network Load Balancing

NLB assumes state-less servicesNLB assumes state-less services Seamless for low QoS messaging appsSeamless for low QoS messaging apps Special solution required for high QoS Special solution required for high QoS

messaging appsmessaging apps

Reliable Web MessagingReliable Web MessagingNLB: Low QoS messagingNLB: Low QoS messaging

NLBNLB

Node1Node1

My_MSMQ_AppMy_MSMQ_App

Node1Node1

My_MSMQ_AppMy_MSMQ_App

Node2Node2

My_MSMQ_AppMy_MSMQ_App

Node2Node2

My_MSMQ_AppMy_MSMQ_App

Node3Node3

My_MSMQ_AppMy_MSMQ_App

Node3Node3

My_MSMQ_AppMy_MSMQ_App

NLB ClusterNLB Cluster

DIRECT=HTTP://NLB_URL/MSMQ/q1DIRECT=HTTP://NLB_URL/MSMQ/q1

oror

DIRECT=TCP:<NLB_IP>\q1DIRECT=TCP:<NLB_IP>\q1 (works also on Windows 2000)(works also on Windows 2000)

Reliable Web MessagingReliable Web MessagingNLB: High QoS messagingNLB: High QoS messaging

NLBNLB

NLB ClusterNLB Cluster

MSMQ Client1MSMQ Client1

App_inqueueApp_inqueue

MSMQ Client2MSMQ Client2

App_ordersApp_orders

Node1Node1Q1->client1\q1Q1->client1\q1Q2->client2\q9Q2->client2\q9

Node2Node2Q1->client1\q1Q1->client1\q1Q2->client2\q9Q2->client2\q9

Node3Node3Q1->client1\q1Q1->client1\q1Q2->client2\q9Q2->client2\q9

AgendaAgenda

Reliable 1-N MessagingReliable 1-N Messaging Distribution ListsDistribution Lists Reliable MulticastReliable Multicast

Reliable Internet MessagingReliable Internet Messaging Messaging Over SOAP/HTTPMessaging Over SOAP/HTTP Firewalls and Load BalancingFirewalls and Load Balancing

Admin APIAdmin API Deployment and StorageDeployment and Storage TriggersTriggers

Admin APIAdmin APIOverviewOverview

Programmable control of MSMQ Programmable control of MSMQ objects:objects: MSMQ serviceMSMQ service Application queuesApplication queues Internal transmission queuesInternal transmission queues

Local and remoteLocal and remote COM object modelCOM object model Add-on available for MSMQ 2.0Add-on available for MSMQ 2.0

Admin APIAdmin APIExtension to existing objectsExtension to existing objects

MSMQApplication extensions includes:MSMQApplication extensions includes: ActiveQueuesActiveQueues PrivateQueuesPrivateQueues DirectoryServiceServerDirectoryServiceServer IsConnectedIsConnected UsedQuotaUsedQuota ConnectConnect DisconnectDisconnect Machine (can manage remote hosts)Machine (can manage remote hosts)

MSMQQueue extension:MSMQQueue extension: PurgePurge

Admin APIAdmin APINew objectsNew objects

MSMQManagement: MSMQManagement: base class for outgoing and target base class for outgoing and target queuesqueues MachineMachine FormatNameFormatName MessageCountMessageCount QueueType (private, public, …)QueueType (private, public, …)

MSMQOutgoingQueueManagementMSMQOutgoingQueueManagement NextHops (routing info)NextHops (routing info) EodGetSendInfo (transaction info)EodGetSendInfo (transaction info) EodResend (transactional protocol)EodResend (transactional protocol) PausePause ResumeResume

MSMQQueueManagementMSMQQueueManagement UsedQuotaUsedQuota EodGetReceiveInfo (transaction info)EodGetReceiveInfo (transaction info)

Admin APIAdmin APICode exampleCode example

Pause an outgoing queue on Pause an outgoing queue on “RemoteMachine”, inspect its content “RemoteMachine”, inspect its content and resume it.and resume it.

Dim qmgmt as New MSMQManagementDim qmgmt as New MSMQManagement

qmgmt.Init Machine:=“RemoteMachine”,qmgmt.Init Machine:=“RemoteMachine”,

PathName:=“TargetMachine\q”PathName:=“TargetMachine\q”

Dim outqmgmt as MSMQOutgoingQueueManagementDim outqmgmt as MSMQOutgoingQueueManagement

Set outqmgmt = qmgmtSet outqmgmt = qmgmt

outqmgmt.Pauseoutqmgmt.Pause

MsgBox “messages in q:” & outqmgmt.MessageCountMsgBox “messages in q:” & outqmgmt.MessageCount

outqmgmt.Resumeoutqmgmt.Resume

AgendaAgenda

Reliable 1-N MessagingReliable 1-N Messaging Distribution ListsDistribution Lists Reliable MulticastReliable Multicast

Reliable Internet MessagingReliable Internet Messaging Messaging Over SOAP/HTTPMessaging Over SOAP/HTTP Firewalls and Load BalancingFirewalls and Load Balancing

Admin APIAdmin API Deployment and StorageDeployment and Storage TriggersTriggers

Storage & DeploymentStorage & DeploymentEnterprise deploymentEnterprise deployment

Simplify MSMQ requirements in Active Simplify MSMQ requirements in Active Directory environment:Directory environment: No MSMQ servers on domain controllersNo MSMQ servers on domain controllers MSMQ clients access Active Directory MSMQ clients access Active Directory

directly via LDAPdirectly via LDAP Old requirements still apply for NT 4.0 and Old requirements still apply for NT 4.0 and

Windows 2000 clientsWindows 2000 clients

Storage & DeploymentStorage & DeploymentNo deployment on DCNo deployment on DC

SenderSender RecipientRecipient

MSMQMSMQ

Any Any Domain Domain

ControllerController

Existence & Existence & AuthenticationAuthentication

Open Open queue on queue on recipientrecipientfor send for send

Storage & DeploymentStorage & DeploymentUnlimited storageUnlimited storage

Problem: MSMQ 1.0/2.0 message store: Problem: MSMQ 1.0/2.0 message store: 2 GB limitation2 GB limitation Win64 doesn’t solveWin64 doesn’t solve

Goal: lift the 2 GB limitGoal: lift the 2 GB limit Implementation: Extend the current Implementation: Extend the current

MSMQ proprietary message storeMSMQ proprietary message store Non-goal: Solve the 4 MB message size Non-goal: Solve the 4 MB message size

limitlimit

AgendaAgenda

Reliable 1-N MessagingReliable 1-N Messaging Distribution ListsDistribution Lists Reliable MulticastReliable Multicast

Reliable Internet MessagingReliable Internet Messaging Messaging Over SOAP/HTTPMessaging Over SOAP/HTTP Firewalls and Load BalancingFirewalls and Load Balancing

Admin APIAdmin API Deployment and StorageDeployment and Storage TriggersTriggers

TriggersTriggersOverviewOverview

Problem: Generic queue listener code Problem: Generic queue listener code is frequently rewrittenis frequently rewritten

Solution: Solution: High performance, built-in, 24x7 listener High performance, built-in, 24x7 listener

service to invoke components on service to invoke components on message arrivalmessage arrival

Invokes EXEs or COM componentsInvokes EXEs or COM components .NET components need to be wrapped as COM .NET components need to be wrapped as COM

componentscomponents No need to call MSMQ APINo need to call MSMQ API

TriggersTriggersNew featuresNew features

Integrated setupIntegrated setup Integrated administrationIntegrated administration Integrated clustering model Integrated clustering model

(active/active)(active/active) Message retrieval:Message retrieval:

PeekPeek ReceiveReceive Transactional receiveTransactional receive

TriggersTriggers

More Information…More Information…

Win06: Tips & Tricks, later today!Win06: Tips & Tricks, later today! MSMQ web site: MSMQ web site:

www.microsoft.com/MSMQwww.microsoft.com/MSMQ MSDNMSDN

Call-To-Action: Check out Call-To-Action: Check out System.Messaging in the .NET System.Messaging in the .NET Framework SDKFramework SDK

© 2002 Microsoft Corporation. All rights reserved.© 2002 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.