Death of the batch job - NServiceBus Sagas

Dennis van der Stelt

all your batch jobs are belong to us


Software Architect

http://dennis.bloggingabout.net/

[email protected]

Particular Software engineer

death of the batch job

@dvdstelt

#nservicebus

http://bloggingabout.net/blogs/dennis/


AGENDA


NServiceBusIn Particular


NServiceBus

It’s not like WCF, which does RPC

But closes to WCF than to BizTalk


BUS TOPOLOGY


MessagingWhat is it and why do I need it?

reduce

coupling


SpatialTemporalPlatform

coupling aspects


PLATFORM

Also known as ‘interoperability’

http, json, xml, xsd, etc…


TEMPORAL

Store Front End Shipping Service


TEMPORAL

Store Front End Shipping ServiceShipping Service


TEMPORAL

Store Front End Shipping ServiceOrder Queue


TEMPORAL



TEMPORAL



Messaging

Reduces spatial coupling

XML/JSON for platform coupling

Asynchronous for temporal coupling

demo

Messaging using WCF


PERFORMANCE

RPC versus Messaging


NServiceBus SagasA pattern by relation database community


demo

NServiceBus Sagas


Sagas Recap“What have you done” – Within Temptation


HANDLING MESSAGES

Behavior like normal message handlers

class MySaga : IHandleMessages<MyMessage>{public void Handle(MyMessage message){…

}}


STARTING SAGAS

Extends the default IHandleMessages<T>

class MySaga : IAmStartedByMessages<MyMessage>{public void Handle(MyMessage message){…

}}


STORING STATE

Extends the default IHandleMessages<T>

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>

{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

}}


CORRELATING MESSAGES TO SAGA INSTANCE


{protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MySagaData> mapper){mapper.ConfigureMapping<MyMessage>(m => m.MyProperty).ToSaga(s => s.MyStateProperty);

}

public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

}}


REQUESTING TIMEOUTS

Reminders to the Saga itself

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IHandleTimeouts<MyTimeout>

{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;RequestTimeout<MyTimeout>(TimeSpan.FromSeconds(10));

}

public void Timeout(MyTimeout state){…

}}


SENDING MESSAGES

Reminders to the Saga itself


{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

this.Bus.Send(new MyCommand());this.Bus.Publish(new MyEvent());

}}


SAGA STATE

Memento

class MySagaData : IContainSagaData{public virtual Guid Id { get; set; }public virtual string Originator { get; set; }public virtual string OriginalMessageId { get; set; }

[Unique]public virtual Guid MySagaId { get; set; }

}

ALTER TABLE [dbo].[MySagaData] ADD UNIQUE NONCLUSTERED ([MySagaId] ASC) ON [PRIMARY]


SAGA STATE

Memento



public virtual IList<Product> Products { get; set; }}

public class Product{public virtual Guid ProductId { get; set; }

}


SAGA STATE

Memento



public virtual IList<Product> Products { get; set; }}

public class Product{public virtual Guid ProductId MyUniqueId { get; set; }

}


Death to the batch jobBecause we don’t want to depend on operations ;-)



scheduled tasks

Your CEO had insomnia and was using the system in the middle of the night. The batch job failed somewhere in

the middle of updating 74 million records…

You need to figure out which row it failed on (how?), why it failed, correct the issue, then start the job again from where it left off, because if you have to start from the beginning, it won't get done before peak hours in

the morning.

because that sounds better than batch job


BATCH JOB

All customers that ordered $5000 in the last year, get preferred status

DateTime cutoff = DateTime.Today.AddDays(-365);

foreach(var customer in customers){var orderTotal = customer.Orders.Where(o => o.OrderDate > cutoff).Sum(order => order.OrderValue);

customer.Prefered = orderTotal > 5000;}


Tromsø, Norway


what if

we can see things before they happen?


customer preferred status

Dev: Let's say Steve orders something for $100. At what point does that amount no longer count toward Steve's preferred status?

BA: That's easy, after 365 days!


-$300-$100

+$300

DURABLE TIMEOUTS

Our way to predict the future

2015 2016

+$100


DURABLE TIMEOUTS

public void Handle(OrderPlaced message){

if(this.Data.CustomerId == 0)this.Data.CustomerId = message.CustomerId;

this.Data.RunningTotal += message.Amount;this.RequestTimeout<OrderExpired>(TimeSpan.FromDays(365),

timeout => timeout.Amount = message.Amount);

CheckForPreferredStatusChange();}

public void Handle(OrderExpired message){

this.Data.RunningTotal -= message.Amount;CheckForPreferredStatusChange();

}


DURABLE TIMEOUTS

private void CheckForPreferredStatusChange(){

if(this.Data.PreferredStatus == false && this.Data.RunningTotal >= 5000){

this.Bus.Publish<CustomerHasBecomePreferred>(evt => evt.CustomerId = this.Data.CustomerId);

}else if(this.Data.PreferredStatus == true && this.Data.RunningTotal < 5000){

this.Bus.Publish<CustomerHasBecomeNonPreferred(evt => evt.CustomerId = this.Data.CustomerId);

}}


Best PracticesThe silver bullets?


Rule #1 : Don’t query data

Never ever, ever, ever query data

- From the saga to another data source

- Owned by saga with a 3rd party tool


Business eventsTimeoutsStarting sagas

out of order


ASYNCHRONOUS COMMUNICATION

Order of message delivery

OrderAccepted

OrderAccepted

OrderBilled




OrderAccepted

OrderAccepted

OrderBilled




OrderAccepted

OrderAccepted

OrderBilled

OrderCancelled

How to solve?

race conditions do not exist


STARTING SAGAS

What business events can start a Saga?

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IAmStartedByMessages<YourMessage>,IAmStartedByMessages<AnotherMesssage>

{public void Handle(MyMessage message){…

if (VerifyState())MarkAsComplete();

}}


404 ON SAGAS

What if the Saga was already MarkedAsComplete

public class SagaNotFoundHandler : IHandleSagaNotFound{

IBus bus;

public SagaNotFoundHandler(IBus bus){

this.bus = bus;}

public void Handle(object message){

bus.Reply(new SagaDisappearedMessage());}

}


AUTO CORRELATION

Avoids the need for mapping responses

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IHandleMessages<Response>

{protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MySagaData> mapper){mapper.ConfigureMapping<MyMessage>(m => m.MyProperty).ToSaga(s => s.MyStateProperty);// Mapping for Respose not needed!

}}

public class RequestHandler : IHandleMessages<Request> {public void Handle(Request message) {Bus.Reply(new Response());

}}


AUTO CORRELATION

Avoids the need for mapping responses

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IHandleMessages<MyTimeout>

{public void Timeout(MyTimeout state){if (state.RasiedAtUtc == …

}}

public class MyTimeout{public DateTime RaisedAtUtc { get; set; } = DateTime.UtcNow;

}

Can be a POCO

No `virtual` as it’s serialized to string format


CONCURRENCY

[LockMode(LockModes.Read)]class ShoppingCartSagaData : IContainSagaData{…

[RowVersion]public virtual int RowVersion { get; set; }

[Unique]public virtual Guid CartId { get; set; }

}


CONCURRENCY

DELETE FROM ShoppingCartSagaDataWHERE Id = '1C669378-3508-4A20-8EFD-A5C60168893E'AND Originator is null AND OriginalMessageId = 'd6436b7-eb17-4de5-8b92-a5c60168888c'AND CartId = '49784AF0-0854-4FF9-B8CD-E57CC8EE6CFC‘AND …AND …AND …

DELETE FROM ShoppingCartSagaDataWHERE Id = '1C669378-3508-4A20-8EFD-A5C60168893E'AND RowVersion = 1

Sagas is business agility

http://bit.ly/nlnsbug


Please…stay in touch!http://dennis.bloggingabout.net

[email protected]

Date post:	22-Jan-2018
Category:	Technology
Upload:	dennis-van-der-stelt
View:	701 times
Download:	4 times

Death of the batch job - NServiceBus Sagas

Technology