MassTranist 101 – What is MassTransit

After playing with MT I have noticed that there is a distinct lack of entry level documentation that may put developers that are not up to the calibre or the uber nerd* MT members. so i have offered to attempt to document or at least given some guidance to help people get into it.
So i am going to start from the very start and try to aim a t a n intermediate level .net that is comfortable with web service and distributed communication etc but may be a bit hazy of what exactly terms like SOA, ESB and Asynchronous Durable Messaging mean.. 😉

*Chris, Dru and Greg and i mean nerd in the good “I am not worthy” sense

I plan to do some basic “This is what this does” type posts too, so there is some step by step articles to get people up and running fast, but for starters here is the proposed initial doc:

The official synopsis of MassTransit, from its website is as follows:
“MassTransit is lean service bus implementation for building loosely coupled applications using the .NET framework.
The lean implementation is supported by the YAGNI principle. By focusing on a tight set of specific concerns, the touch points between Mass Transit and the application are minimized resulting in a clear and concise set of interfaces.”
But what does it actually do?
Well as the brief states MassTransit is a service bus for building loosely coupled applications. Let’s break that down.
So what is a service bus and how does it aid in loose coupling? Well hopefully the follow example may help you understand:
You may have 1 or more systems/components communicating with out any real knowledge of who or what each systems/components is. For example if I what to Create a New Purchase Order for a client I may send a specific message, for example a message called “CreateNewPurchaseOrderRequest” (with the company and order details inside that message) MassTransit picks up this message and forwards on to which ever system(s) is registered to deal with that type of message. Now my sending component does not know or care which system receives and deals with that message, it has done its job of raising the purchase order.
Ideally there is another system that is ready to deal with that type of message. It is given the message from the buses queue and the service processes the data in the message; persisting, starting workflows etc. It may even send a message back to the bus, say a “CreateNewPurchaseOrderResponse“, to acknowledge that, yes, the purchase order has been successfully created. The original sender of the CreateNewPurchaseOrderRequest may (or may not) be interested in this response and can subscribe to the message type and act on it accordingly.
What this does require is that the bus is alerted of who wants to receive what type of messages and also that when a message is created and ready to be sent, it is passed to the bus.

Now this scenario raises a bunch of questions
Why not use method calls/web services?
• A service bus like MassTransit allows you to send and forget. You give the bus a message and let the bus decide where to send the message. This means you do not need to know that HR is now looking after message X and the Logistic is dealing with message Y differently. You don’t care. You also don’t need references to those other systems. Changes to other systems are much less likely to affect your system.
Why the ignorance?
• What happens if the business now decides that when we raise a Purchase Order we also need to alert Logistics, Warehousing, Marketing and the original Sales department? If you where using traditional web services this means you would need to go into you code add references to those web services and make class to them from the raising application. With a service bus you raise the message and let the the bus forward the message on t the interested parties. It means the component focuses on the job it is supposed to do and not worry about departmental interactions. This concern is now that of the Bus
Why messages?
• As business and their technical solution grow, maintaining those technical solutions can become difficult. Architects have been using SOA approaches to help delineate logical service and use messages as the means of moving data around. A message in itself guides the bus to what its intended purpose is. A CreateNewPurchaseOrderRequest is quite self explanatory and very specific. Well constructed messages and services means disparate components can talk to each other, showing their intention but hiding the implementation.

Benefits of MassTransit
• Loose coupling/Recipient ignorance: Send and forget. In the same vein you can subscribe to a message type and not care where it was sent from.
• Asynchronous communication: no need to hold up connections and thread waiting for replies. You can send the request and if you are interested in the response subscribe to the message type
• Durable and reliable messaging thanks to the message queue backbone (e.g. MSMQ)
• Light weight : MassTransit.ServiceBus.dll, MassTransit.ServiceBus.MSMQ.dll & MassTransit.WindsorIntegration.dll combined are under 100KB
• .Net Type safe. You can subscribe to message based on the .net type of the message. No need to interrogate XML files or manually transform .net types to xml etc
• Castle integration: The stack is built using best practices such as IoC and is ready to integrate with the popular .net Library “Castle”
• Open Source: The source code, including unit tests and samples, are available openly.

Who would want to use MassTransit?
• A designer or developer who’s systems are predominantly in .Net realm.
• A designer or developer that wants to have a means to communicate to separate systems in a loosely coupled and possibly asynchronously manner
• A designer or developer that wants to have a reliable and durable means of communication
• A designer or developer that wants to have an easy means to set up a messaging bus!

What does this require of the developer?
• You will need a supported Messaging Implementation such as MSMQ
• You will need to base your communication on a message construct i.e. each message has a specific intent and relevant data. Passing entities is not explicit enough as it does not show intent.
• You will need to reference MassTransit and use the library accordingly! (see the samples for more info)

I hope this gives some basic back grounds as to what MassTransit is and does. Any question please email me or for technical issues go here

Rusty on the ol’ SQL

It took me a good 15 minutes to figure out how to get a list of table names with schema’s from sql 2005..jeez. not good.
I would not be surprised if its actually wrong. Anyway, whats with hiding the sys tables? Weird.

select ss.name ‘schema’, so.name ‘table’
from sysobjects so
inner join sys.schemas ss
on so.uid = ss.schema_id
where so.xtype = ‘U’
order by ss.name, so.name

Mass Refactoring of a Poor DDDD Stack

We are at a cross roads on our current project where:
• We are hitting limitation of our current architecture and framework
• We need to be able to facilitate service orientation and be come part of an SOA environment
• We are not completely swamped with work
• Bugs are lingering in a hard to understand convoluted stack

We have an NHibernate /Oracle based distributed (client – server) application.
Current concrete problems:
• We pass entities around across service boundaries. These entities can be huge and are therefore very slow to pass around and we hit upper limits on WCF message size (semi)regularly. The screens we are displaying show a lot of related data and the load time for some pages is insanely slow.
• Our application has some smarts in it that can be argued are actually domain logic, not application logic.
• We have a lot of code repetition “copy and paste mania” through out the stack
• We have a tightly coupled stack; using doubles/mocks/fakes at times is near on impossible meaning test are integrated, brittle and slow
• Services expose too many finely grained functions, many of which are not appropriate as public calls.
• We have one domain which needs to be spilt to shared kernels
• We have a very anaemic domain. Our domain objects are treated a bit like strongly types datasets with the service controlling business logic and flow.
• Poor separation of concerns both vertically and horizontally (i.e. mega classes)
• subtle and painful circular references
• Logic in views

DDD
• Clearly define boundaries. Implicit knowledge should not be required. Code should be fluent and readable. Refactoring is therefore a necessary component of building a fluent domain
• Flesh out the actual business problem and map it completely and deeply into the domain code. Reading a domain entity class it should obvious what it can do and what is cant.
• Assert your assumptions, use DBC, predicates, valuators and factories, especially with complex entity construction
• Domain entities are smart. Do not let the smarts of these domain object leak into the repositories or services (ie an anaemic domain)
• Repositories are just that, effectively a bucket holding domain objects. If you want to do complex DB logic use a report data access call, don’t muddy the repositories with reporting.
• Services are aggregators of domain functionality. They control flow and coordination between Repositories, Factories and Domain objects. They do not hold leaked domain logic.
• Reports are anything that is not domain object based: Actual reports, Scheduling, Complex db searching etc. These may return objects but they are not smart domain entities, nor are they are domain value types and so should not pollute the repositories.
• Value types are immutable and so consider whether they are suitable to be structs (e.g. time code)
• Reference Value types are read-only (i.e. only selectable form the database, never CUD) and so do not need mapped “created/last modified“ properties. These may be necessary in the DB (for standardisation) but this does not mean they need to be leaked into the domain

Distribution
• The message and service should be verbose enough to describe the functionality.
• Use messages. Don’t pass entities around across boundaries.
• Messages are immutable, don’t attempt to modify request messages on the recipient, nor should you modify response on the caller
• Feel free to reuse structures ie DTO but don’t reuse messages at all. A message is bound to a service and a function
• A message is the object passed across the wire which holds a DTO, which may be in turn made up of more DTOs. A message implements either IRequestMessage or IResponseMessage
• On entry to the server side any instrumentation/auditing/logging/security checks etc should be handling in a standardised reusable manner, consider AOP or delegate passing.
• On entry to the server side transactions/sessions should be handled as this is the entry point to the domain and a service call should be considered atomic.
• Faults, not exceptions, should be passed back to the caller.
• Consider whether the use of an ESB could improve the use of your domain. Message based communication is necessary if this is to be implemented.

Application
Logic in an application is bound to application specific flow. The exception here is validators. [I am yet to figure out how best to organise validation across boundaries. Currently my thinking is keep it very separate with well constructed messages helping]

General
Do not pollute any part of the stack with cross cutting concerns. Use AOP. The entry and exit of a method is usually more than enough context for things like security and logging. If it is not then rethink the size of your methods!
Loosely couple yours stack. Use IoC. A standard IoC container can ease testing and un-clutter the complexity of object creation.

Concerns I need to keep at the back of my mind if we switch to something like the guidelines above:
• Client and Server side Authentication and authorisation. How will we pass security tokens around?
• Concurrency, how will I address 2 user working on the same asset at the same and race conditions?

Bring on the refactoring, I just wish we did it as we went! Please learn from our lessons.

IoC 101

Keeping with the theme, here is a great introduction blog post from StructureMap creator Jeremy D Miller on Inversion of Control and Dependency Injection.

I have never used StructureMap on any live apps, which is a shame as i have followed its life cycle over the past few years as it documentation, mainly thru JDM’s blog posts, have been very interesting. If you want to see how to integrate something like StructureMap into your existing code base check out Rob Conerys re factoring session with Jeremy . Actually check out the whole video series its great to watch as you get to see someone make mistake and fix them. People don’t actually learn very well from watching experts, they learn better from watching people adapt after making small errors. wow.. 4 years of Pysch just paid off! so check it out and watch rob talk with Gods of .Net (JDM, Ayende etc)

Design Patterns 101

How have i missed these!!!
JP Boo hoo hoo hoo 😉 has some great intro’s to Design Patterns on DNR TV. I am just watching the first of the series now so cant yet comment on the rest but so far so good. He presents it really well, but beware if you are not a ReSharper user it may be a bit of a tornado.

Check it out!
DotNetRocks TV Archives
John Paul’s Design shows: 63, 65, 68, 71, 92

BTW Word on the street is JP’s .Net courses are top-shelf-single-malt awesomeness. I have been recommended them from guys i respect in the community but have not had the chance to attend (get to the UK mate). If you get the chance to go to one at least investigate.

JP’s Blog

TDD 101

Local London ALT.Net-er and MVP Ian Cooper has put up a good TDD post with indications of more to come. I would recommend all to read this even if you regularly employ TDD and espeically if you are still coming to terms with the process. It covers basic concepts and acronyms that you should be aware of.
Check it out!

The book he mentions is very good for jumping in to TDD. I bought and read it on Ian’s recommendation earlier this year and highly recommend it as a way to get your feet wet or to show colleagues how to begin the TDD process. Check it out here:
Kent Beck’s book Test-Driven Development, By Example

VAR Abuse

Why do you use C#? Is it just because that’s what you where told to use? Are you one of those that hates VB.net with out any reasons other than, “its what hobbyist coders use”?.
I have a feeling there are some people that just don’t “get it”. Unfortunately Jeff Atwood seems to be one of those. I have been reading his blog for years and he frequently brings up great points ideas and thoughts about software in general.
Unfortunately I have to strongly disagree with the path of one of his recent blogs: Department of Declaration Redundancy Department

To me that is “var abuse”. If you are using C#, IMO, you should be using it as a statically typed language and use feature like var as a get out of jail free card when it is needed, or will provider significant “value add”.
Using it everywhere make your code weak, in the world of C#.
If you want to use a dynamically typed/inferred language, use one! Don’t try to bend a statically typed language in nasty ways to do your work for you.

My major concern real here is maintenance. Introducing var as common place syntax means you are diluting two worlds. Coding standards may go by the way side for no reason other than, “I can use var so I will use var”.
Would you mix you HTML, CSS, JavaScript, ASP.Net script tags and code behind all in one file? If so, then var is for you! Just don’t expect a job from me any time soon.

The irony is the people that say “yeah i use var all the time” are often the ones that say, “yeah but i would never use vb”..well why not?

My original reply:

comment like :
“Agreed. I use var all the time. It could be even more concise if they dropped the ‘var'”
Make me ask: “Why are you coding in C#?”
Find a language that supports what you want! Try Iron python if you what to stay on the CRL/DRL.
I am keeping my “IFoo foo = new Foo();” syntax, thank you very much.
It concerns me that C# is trying to be the everything language. I give it about 3 years before no one whats to code in it anymore due to the lines be sooo blurred.
Use C# how is it supposed to be used, then use Boo, Iron python etc for more dynamic approaches.
Would you put all you JavaScript, CSS, ASP.Net and HTML in one file? I see this and being a similar scenario. 4 DIFFERENT “languages” solving different problems.
my $0.02
RhysC

TDD Wisdom

Just read an old response to a CSLA.Net test question by Chad Myers:
“So far, the best (most easily maintainable, most adaptive, most flexible) designs I’ve ever created were ones done with tests in mind. IMHO, tests are your first and best customer of your API and expose design smells quickly.

So please don’t dismiss ‘testability’ so easily as some secondary, ancillary concern because it’s one of the best ways to flesh out potential problems in your design, rather than giving up and masking it by lifting the CLR’s skirt and messing around in order to test your code.”

So wise… like a miniature Buddha covered in hair…

Having a web presence

The amount of time I want to contact someone in the technical field that I don’t know personally is not that often, but it surprises me when I can’t find their information. There are some guys out there that have technical blogs I enjoy reading, with no web site or contact details to speak of.
If the blog does not enable comments then there is no easy way of me to contact these guys.

I suppose there are a few level of anonymity:
No web presence: Does not know the net is 😉
No public web presence: No active blog, web site etc
Public and un-contactable: Possibly a blog or old posts on a mailing list archive with an invalid email address.
Public: Real details easily found. Usually these people also have an active blog or some sort of regular contact to the rest of the world.

Now I understand having a public profile takes effort. It can in fact be a hassle as you have a website or blog your have to maintain; you open yourself up to spam and other unwanted attention. However recently I have been meeting some very interesting and sometimes infamous people and I am genuinely surprised how many of them have
A) no really public presence such as a web site, a LinkedIn profile, Twitter account or even an active Blog
B) Or a web site that just sucks. I sometimes think none is better than crap, as Dad used to say “Do it properly or don’t do it all!”.

I do not consider my self a pioneer of the IT world nor someone that has any real public following, but was surprised how many people from the Alt.net community knew who I was (even if by my RhysC handle). If any of those guys wanted to find out details about me, I would like to think it would be pretty easy, and a few of them have. This is a good thing! I am a contractor! I want people to be able to find out about me, and when they do see my web site, I don’t want them to think. “Geee.. that sucks!”

So this is me laying it out, to you as a IT geek, sort it out! Especially if you are a contractor or a high profile senior IT employee. Get an online profile. Share with the world your thoughts. You didn’t get where you are by being a dumb ass, people may actually want to hear your thoughts, contact you, offer you a job… who knows 

NOTE: oh yeah, my UK company is FullStack
and the Aussie company i work with is ArtemisWest. AW does need a bit of polish, however we have had some big things happen in personal lives so work goes on the back burner… we work to live, not the other way around!

NHibernate errors: 101

We seem to be having a bit of trouble with NHibernate of late. Below are some basic check and strategies to get the most out of the ORM. This document is very basic but covers 90% of the issues I have had to fix with NHibernate on this project. For some background i believe I am the only one to have work with NHibernate previously and had come in well after the DAL had taken shape, so some basic mistake have been uncovered as part of our learning process.

Basic checks
Check Mapping files are embedded resources (right click the file in VS and select Properties. Ensure the build action is Embedded Resource)

Check the class is marked as Serializable

[Serializable]
public class Customer : DomainEntity<int>, ICustomer
{…}

Check the properties are Virtual

public virtual Orders Orders
{
get
{
return this._orders;
}
set
{
this.orders = value;
}
}

Lazy loading exceptions

Check the object loaded is occurring in a session correctly and the session is being properly closed & committed
Check the mapping for the object and the children you are trying to load. If the Mappings are set to lazy=true, you will have to explicitly load them or access a property on the object to hydrate the object graph to the next lazy node down the object graph.
Check your session is not closing prematurely, either by code or by an internal error such as DB timeouts. Nested session will all be committed at the committing of the upper most transaction. Be wary of overly nested transaction as this may lead to time out and a possible “number of connection/cursor” exceptions throw from the database or it drivers.

Concurrency Issues
NHibernate is finicky with objects being loaded multiple times and associated in different sessions. This (in my cases) has usually come down to poorly factored code. E.g. Make sure you are not trying to put the same object in to a collection twice from 2 different sessions etc. I don’t have any great advice here as this is usually handled on a case by case basis, but is a good sign you need to rethink your design.

Rules of thumb
Reference data should be cached, immutable and eager loaded. This makes the pretty basic assumption that no relationships exist off to other objects (i.e. no children or other mapped entities)
Complex domain data should generally be lazy loaded, immutable and not cached. You can set up lazy/eager loading strategies on a per property basis. Eager loading simple types is fine, be careful with complex relationships as you should be catering for the most minimal requirements. Excessive use of eager loading will severely affect performance.
Again, be careful with eager loading. If you eager load a child there may be any number levels of eager loading down the object graph. Eager loading should generally be avoided unless there is a business reason to do so. Instead explicitly load each object or object collection explicitly in the business code that deals with the situation, ie

//this occurs with in a session to get all the Orders of the given customer

this.TheOrderDao.Initialize(customer.Orders); //Using NHibernateUtil.Initialize

Preferentially walk the object graph as opposed to using find by id or overly simple HQL. I.e. to get the customer orders use customer.Orders as opposed to a query that looks for order based on a customer ID. NB: This must be done in-session if objects are marked as lazy.
Use ICriterion styled searches to do simple object searches, ie: Find all customers with city location of London
Use HQL queries to get type objects back from queries. These are general object based queries where a collection of a mapped type is returned but may need other joined tables for querying purposes. Ie return all the product categories for all the orders of customer x for the last 365 days. These queries can usually be though of as follow the format select customer.* from customer inner join….
Only use stored procs for set based queries to
aid in speed if walking the object graph would be extraneous or load too much redundant data
to return streamlined DTO/DataSet for non type queries or queries that would otherwise return overly large object graphs when only a small portion of the graph is required. (These are the exception rather than the norm.)

HTH

Rhys