Death of the batch job - NServiceBus Sagas

transcript

Dennis van der Stelt

all your batch jobs are belong to us

Software Architect

http://dennis.bloggingabout.net/

dennis@bloggingabout.net

Particular Software engineer

death of the batch job

@dvdstelt

#nservicebus

AGENDA

NServiceBusIn Particular

NServiceBus

It’s not like WCF, which does RPC

But closes to WCF than to BizTalk

BUS TOPOLOGY

MessagingWhat is it and why do I need it?

reduce

coupling

SpatialTemporalPlatform

coupling aspects

PLATFORM

Also known as ‘interoperability’

http, json, xml, xsd, etc…

TEMPORAL

Store Front End Shipping Service

TEMPORAL

Store Front End Shipping ServiceShipping Service

TEMPORAL

Store Front End Shipping ServiceOrder Queue

TEMPORAL

Messaging

Reduces spatial coupling

XML/JSON for platform coupling

Asynchronous for temporal coupling

Messaging using WCF

PERFORMANCE

RPC versus Messaging

NServiceBus SagasA pattern by relation database community

NServiceBus Sagas

Sagas Recap“What have you done” – Within Temptation

HANDLING MESSAGES

Behavior like normal message handlers

class MySaga : IHandleMessages<MyMessage>{public void Handle(MyMessage message){…

STARTING SAGAS

Extends the default IHandleMessages<T>

class MySaga : IAmStartedByMessages<MyMessage>{public void Handle(MyMessage message){…

STORING STATE

Extends the default IHandleMessages<T>

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>

{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

CORRELATING MESSAGES TO SAGA INSTANCE

{protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MySagaData> mapper){mapper.ConfigureMapping<MyMessage>(m => m.MyProperty).ToSaga(s => s.MyStateProperty);

public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

REQUESTING TIMEOUTS

Reminders to the Saga itself

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IHandleTimeouts<MyTimeout>

{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;RequestTimeout<MyTimeout>(TimeSpan.FromSeconds(10));

public void Timeout(MyTimeout state){…

SENDING MESSAGES

Reminders to the Saga itself

{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

this.Bus.Send(new MyCommand());this.Bus.Publish(new MyEvent());

SAGA STATE

Memento

class MySagaData : IContainSagaData{public virtual Guid Id { get; set; }public virtual string Originator { get; set; }public virtual string OriginalMessageId { get; set; }

[Unique]public virtual Guid MySagaId { get; set; }

ALTER TABLE [dbo].[MySagaData] ADD UNIQUE NONCLUSTERED ([MySagaId] ASC) ON [PRIMARY]

SAGA STATE

Memento

public virtual IList<Product> Products { get; set; }}

public class Product{public virtual Guid ProductId { get; set; }

SAGA STATE

Memento

public virtual IList<Product> Products { get; set; }}

public class Product{public virtual Guid ProductId MyUniqueId { get; set; }

Death to the batch jobBecause we don’t want to depend on operations ;-)

scheduled tasks

Your CEO had insomnia and was using the system in the middle of the night. The batch job failed somewhere in

the middle of updating 74 million records…

You need to figure out which row it failed on (how?), why it failed, correct the issue, then start the job again from where it left off, because if you have to start from the beginning, it won't get done before peak hours in

the morning.

because that sounds better than batch job

BATCH JOB

All customers that ordered $5000 in the last year, get preferred status

DateTime cutoff = DateTime.Today.AddDays(-365);

foreach(var customer in customers){var orderTotal = customer.Orders.Where(o => o.OrderDate > cutoff).Sum(order => order.OrderValue);

customer.Prefered = orderTotal > 5000;}

Tromsø, Norway

what if

we can see things before they happen?

customer preferred status

Dev: Let's say Steve orders something for $100. At what point does that amount no longer count toward Steve's preferred status?

BA: That's easy, after 365 days!

-$300-$100

DURABLE TIMEOUTS

Our way to predict the future

2015 2016

DURABLE TIMEOUTS

public void Handle(OrderPlaced message){

if(this.Data.CustomerId == 0)this.Data.CustomerId = message.CustomerId;

this.Data.RunningTotal += message.Amount;this.RequestTimeout<OrderExpired>(TimeSpan.FromDays(365),

timeout => timeout.Amount = message.Amount);

CheckForPreferredStatusChange();}

public void Handle(OrderExpired message){

this.Data.RunningTotal -= message.Amount;CheckForPreferredStatusChange();

DURABLE TIMEOUTS

private void CheckForPreferredStatusChange(){

if(this.Data.PreferredStatus == false && this.Data.RunningTotal >= 5000){

this.Bus.Publish<CustomerHasBecomePreferred>(evt => evt.CustomerId = this.Data.CustomerId);

}else if(this.Data.PreferredStatus == true && this.Data.RunningTotal < 5000){

this.Bus.Publish<CustomerHasBecomeNonPreferred(evt => evt.CustomerId = this.Data.CustomerId);

Best PracticesThe silver bullets?

Rule #1 : Don’t query data

Never ever, ever, ever query data

- From the saga to another data source

- Owned by saga with a 3rd party tool

Business eventsTimeoutsStarting sagas

out of order

ASYNCHRONOUS COMMUNICATION

Order of message delivery

OrderAccepted

OrderBilled

OrderAccepted

OrderBilled

OrderAccepted

OrderBilled

OrderCancelled

How to solve?

race conditions do not exist

STARTING SAGAS

What business events can start a Saga?

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IAmStartedByMessages<YourMessage>,IAmStartedByMessages<AnotherMesssage>

{public void Handle(MyMessage message){…

if (VerifyState())MarkAsComplete();

404 ON SAGAS

What if the Saga was already MarkedAsComplete

public class SagaNotFoundHandler : IHandleSagaNotFound{

IBus bus;

public SagaNotFoundHandler(IBus bus){

this.bus = bus;}

public void Handle(object message){

bus.Reply(new SagaDisappearedMessage());}

AUTO CORRELATION

Avoids the need for mapping responses

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IHandleMessages<Response>

{protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MySagaData> mapper){mapper.ConfigureMapping<MyMessage>(m => m.MyProperty).ToSaga(s => s.MyStateProperty);// Mapping for Respose not needed!

public class RequestHandler : IHandleMessages<Request> {public void Handle(Request message) {Bus.Reply(new Response());

AUTO CORRELATION

Avoids the need for mapping responses

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IHandleMessages<MyTimeout>

{public void Timeout(MyTimeout state){if (state.RasiedAtUtc == …

public class MyTimeout{public DateTime RaisedAtUtc { get; set; } = DateTime.UtcNow;

Can be a POCO

No `virtual` as it’s serialized to string format

CONCURRENCY

[LockMode(LockModes.Read)]class ShoppingCartSagaData : IContainSagaData{…

[RowVersion]public virtual int RowVersion { get; set; }

[Unique]public virtual Guid CartId { get; set; }

CONCURRENCY

DELETE FROM ShoppingCartSagaDataWHERE Id = '1C669378-3508-4A20-8EFD-A5C60168893E'AND Originator is null AND OriginalMessageId = 'd6436b7-eb17-4de5-8b92-a5c60168888c'AND CartId = '49784AF0-0854-4FF9-B8CD-E57CC8EE6CFC‘AND …AND …AND …

DELETE FROM ShoppingCartSagaDataWHERE Id = '1C669378-3508-4A20-8EFD-A5C60168893E'AND RowVersion = 1

Sagas is business agility

http://bit.ly/nlnsbug

Please…stay in touch!http://dennis.bloggingabout.net

dennis@bloggingabout.net

Death of the batch job - NServiceBus Sagas

Technology