The number of available build tools in .Net grows daily it seems… and i feel i am falling behind.
MSBuild and NAnt are the standards in the .Net world but the uprising of Ruby has formed Rake, which is also usable in .Net. Rake is a Ruby implementation of the build tool Make which has inspired the .Net equivalent Boobs Bake from Ayende and now James Kovacs has PSake (pronounced sake as in the Japanese wine)….
I really will have to look into these… jeez i still haven’t played with Power shell and my Ruby and Boo skills have a lot to be desired!
Monthly Archives: August 2008
The satisfaction of refactoring
I am in the process of heavily re factoring our service facade (ie WCF) layer. the first file i got hold over i have already removed 60% of the LOC and got a significant number of the methods to one line
eg:
public GeoLocationFindByIdResponse GeoLocationFindById(GeoLocationFindByIdRequest aTransportObject)
{
return FindEntityById<IGeoLocation>(aTransportObject.Payload,this.Domain.GeoLocationFindById);
}
that’s a big improvement from the 36 liner that it replaces and a whole lot easier to read.
One of our major issues was cut and paste code of logging and auditing/instrumentation. As we are not using AOP I have decided to go with passing delegates. It has actually sped up the method calls too as i have re factored out redundant methods calls and repeated logging. Now all of fault handling and the “aspect” like stuff can be handled in one place and the code is a whole lot easier to maintain.
Cool, i am happy with that!
Setting up a Greenfields project
My last post talked about technical investment. well like financial investment, its usually better to invest early.
Lee recently posted about Standards and Success Criteria. I think this is crucial for project success and hopefully this post is a concrete example of what I think are standards when beginning a project.
Setting up your solution well early can make life a lot easier. Some of the main concerns to me when initially setting up are as follows:
Naming
Establish a naming convention early on. It does not really matter what that convention is, just define it. There are some .Net guideline already established, use these if you don’t already have one (iDesigns coding standard is pretty good starting place).
Be sure to talk to other developers and stake holders about the names you use and change them as needed. It will much easier to change these names early one than it will later on when multiple Dev’s are in the mix and releases are already being made.
Use the correct names for commonly know things. One of the key areas I often see is pattern names used incorrectly. Make sure you understand what the pattern is if you are going to use the terms. Every self respecting developer should have GoF (or at least read it).
Libraries & tools
Define what Core and 3rd party libraries and frameworks you are going to use at the start of the project. If you are going to use .Net 3.5 and WCF use it from the beginning and use it properly.
Libries you need to consider include: Test/Mock libraries, Persistances/ORM, communication/ESB, UI, Logging, Framework (IoC, AOP etc) etc.
Tools include: Soruce control, analysis tools, code gen, IDE, CI, build, test runners/coverage, IDE plugins, documentation generators
This is also a good chance to be an early adopter. If there is a new beta release of a product you are interested in and the release is schedule for before your release, why not start using it now? Sure you will have to weigh this up yourself, however this has proved successful to me in the past.
One caveat here is make sure you know how to use the library before jumping on board. It is only after you have learnt the library that you know how steep the learning curve is. Personally I found it easier to learn a whole new language (Python) than picking up an ORM (NHibernate). Don’t drag your team through hell because you wanted to play with the latest toy.
Build process
Establish a build process early on. Set you team up for success. You may not have CI set up yet but that does not mean you cant have the solution ready to go. Have your build scripts set up before you have any significant code base. Use the build script.
Ensure the build script is doing everything is should, you should not have to move dll’s or copy config files etc. It should do it all.
Project Processes
Define how the project will be run from day one and do it properly. The term Agile is so flippantly thrown around, yet so rarely done properly… I find i quite annoying. To me its like saying I drive a Lexus when I am rollin’ in a Toyota. There is nothing wrong with a Toyota, and it may be the best vehicle for the job, just call it what it is.
Tests
Whether you are TDD or not, I assume you are writing some sort of automate-able tests, whether they are strictly unit test or otherwise. Integrate this in to your build process. As we are talking about Greenfield projects here, define your expected test coverage up front. Now this is a doable edged sword, but i think it is worth while. Also encourage your Dev to truly embrace TDD and correct UNIT testing using mocks/fakes/stubs.
On a side note, I was talking to a Java mate yesterday and apparently there is a a pug in for IntelliJ that runs your test in a back ground thread every time you build in the IDE. With the multi-processor machines we uses nowadays, why not? It got me thinking, i could pretty easily set up something like that too for my local machine.
Follow coding best practices
The first port of call in doing this is setting warnings as errors and setting the error level as high as it can go in each of your project files in the solution.
Get FXCop and use it. if there are things you don’t like then document and removed the waring, but start with it.
Some will disagree with me on this, but on this subject I say F*^K ’em. Not having these on just sets up a slippery slope. I would rather deal with the issues as they come up, not retro fix.
Follow design best practices
Terms like IoC, AOP & TDD are not just cool buzz word, they are best practices there to make YOUR life easier. Using IoC make TDD easier. Using TDD encourage the use of IoC. AOP cleans up your code from so the business problem at hand is not cluttered with technical sideshow concerns.Use these, embraces these, understand these and introduces them early on into the picture. Create a pit of success.
Also be sure to use a metrics tool like NDepend to check on your solutions health. Again, dealing with issues early may help stem some nasty heads from popping up later.
Defining future Standards and Success Criteria
This is pretty broad and somewhat recursive, but there will be personally issues that you are concerned with and project specific thing that should be address early on in the picture
Technical Investment
A recent post on Technical debt on InfoQ got me thinking about technical investment.
In the age of YAGNI, ominous PM’s and tight budgets, often time is not spent on investing technically in the team or the business. We build stuff as we need it and if we can reuse something we try to, but often end up hacking it to fit our new needs, if the shoe does not quite fit.
At my current place of work we are “post live” and in a situation where investing technically makes great business sense. Our project is some what of a flag ship project as the company is
A) New to .Net, coming from a Delphi background
B) New to NTier distributed development, coming from a rich client with db connection
C) Looking to move to an SOA architecture across the company to more cleanly delineate the business logic get some reuse out of the business processes.
Because of these factors there have been mistakes that have been made. The application we have built works but it is high in technical debt. It is just holding itself together, push it just a little and it will come crumbling down.
A lot of this is to do with hurried framework design and best practices ignorance. We have acknowledged our mistakes, but that is not enough. Over the next few months this application stack is to be used as the basis for other NTier development in the new and growing .Net department, we need to make sure the other teams don’t copy our mistakes. We need to make it easy for them to succeed!
IMO we need to start with re factoring our core libraries, specifically Data Access, Logging and Service Boundaries. These are areas that will be needed for all of the applications in the future so why not get this stuff right.
Are we using WCF as intended?
Are we getting the most out of NHibernate?
Can we reduce the noise from our non business aspects that have leaked in to our code?
Is this Gold plating? Well, yes i guess it is and i would not recommend it normally. However given the circumstance i think the cost of not getting this rights is too great.
So how would I do it if i was in the same position tomorrow as the guys were at the start of this project? Maybe that’s the next post…
NDepend in action
Nice post here breaking down the metrics on the latest NHibernate release
interesting results…
Domain object mapping
I have been using Nhibernate now for almost 2 years and to be honest for most of the time really missed the point of using an ORM. I think alot of this stems for the M$ influence mindset of building from the DB up. This is not necessarily a bad thing and to be honest is often a necessity if you are using an existing DB or working disparate to the DBA or owner. However as i explore the DDD world i see the benefit of build from the domain out, especially in complex domains that are not just crud operations.
Most of my time using an ORM i have just been using it as a glorified typed data set. The object would contain data in the form of properties that mapped to the structure of the table it is mapped to. This really is not using the full strength of an ORM, and to be honest if this is all you are doing then a code gen option like Net Tiers is probably a much better path (faster, less stress easier).
However now I feel the ORM really is a way of abstracting your domain from your persistence mechanism. Unfortunately it took an nasty database to show me the light.
I dont mind admitting my mistakes, usually its in a less public forum, however these are mistake I see frequently that i dont know if the authors are aware they are making.
So here’s some guidelines that i am now trying to follow, take them for what they are.
Properties != Columns
Column and property names do not need to be the same. I have often seen the suffix “Flag” for Boolean columns; however this is less IMO readable than the “Is” prefix in managed code. do not feel the need to map names directly. use the most suitable domain representation
Incorrect Mapping of Types
I have seen properties that are of type string because the underlying table has a char(1) for Y/N flags. there is then a property that interrogate the string flag property and return a bool depending on the result… yuck. Why not use the mapping correctly
this also applys to numeric types and strings. There are more numeric types than int, so use the most appropriate one. if a string has a max limit on the DB of 30, enforce that in your map and in your class.
Inappropriate use of the public keyword
The public key word for some reason seems to be a default accessor. it should only be used when you w this aspect of your code.
Classes
If you have a class that is in way relevant to the outer world, do not mark it as public. Many mapping classes fall in to this category. I dont want to see Foo.FooBar[0].Bar… this show me implementation details that i dont care about and just add noise. If you need the join class but it serves no public functionality encapsulate it by providing a Bar collection property that transverses the join classes to give Foo.Bar[0]. You may want to consider if your loading strategy for this relationship is appropriate (ie not lazy) if you are using this out of session.
Bottom line is: ORMs are there to provide DB/Persistence ignorance not highlight the DB structure.
Properties
Properties that are mapped to columns may not always be publicly accessible. There may be time when if you update property A then B must be considered too. Dont make these setters public, or you give the impression that changing them freely is OK. Use a method to populate the properties so you can also encapsulate business logic associated to these.
Methods
This point follows on from the preceding point. You domain objects do not have to be buckets designed to carry data. They can and should have behaviour associated to them. Not doing so leaks your logic out of the domain and results in duplicated logic and hard-to-maintain uber services. Dont let this happen.
Fear of re factoring
It is often difficult to change a database especially in post live situations. this does not mean your domain has to reflect old/incorrect data structures. Feel free to remove mapped columns form your classes and mapping files if they no longer make sense in the domain. just because one area of development is wrong it does not mean it has to leak out to the other areas.
Light Handed Mappings
That title doesn’t really makes sense, what i really mean is be heavy handed with your mappings, classes and tables. Assert your assumptions. If a column can not be null, assert that in the DB, maps and in the managed code. If you know the class a will always need reference to its collection of class B then set lazy loading to false. Ignorance is a common excuse, but its one i am trying to minimise
Redundant Repositories
A repository is generally only required for an Aggregate Root. Define your boundaries and remove redundant Repositories. Do you really need an ICustomerAddressRepository? get rid of it!
Confusing the purpose of a Repository
This may be more than a ORM issue but a design issue. A Repository is a store for entities. It does not do reporting. It does not do complex SQL searches that return weird and wonderful datasets. I “like” Greg Youngs notion* (which i initially thought was wildly over the top, but goes to prove a point) of just saving the id of aggregate root and then serialise the object graph in to the other column of the table. This database is not for reporting, its for serving the domain.
Although your approach man be a little less drastic, this highlights a point. If you cant think of your repository as an in memory store then you are not using like a repository. In fact my current thought is your should be easily be able to inject an in memory store that uses the I[Foo]Repository interface and work exactly as intend , albeit a smaller set of data (eg for testing). I may be going a bit OTT here and would like feedback.
*i hope i am not miss-quoting Greg
Hopefully that has clarified some of the errors i have done/seen repeatedly in the past. i would like to here feedback. have i yet again missed a point? am i being over the top?
lay it on 🙂
MassTranist 101 – What is MassTransit
After playing with MT I have noticed that there is a distinct lack of entry level documentation that may put developers that are not up to the calibre or the uber nerd* MT members. so i have offered to attempt to document or at least given some guidance to help people get into it.
So i am going to start from the very start and try to aim a t a n intermediate level .net that is comfortable with web service and distributed communication etc but may be a bit hazy of what exactly terms like SOA, ESB and Asynchronous Durable Messaging mean.. 😉
*Chris, Dru and Greg and i mean nerd in the good “I am not worthy” sense
I plan to do some basic “This is what this does” type posts too, so there is some step by step articles to get people up and running fast, but for starters here is the proposed initial doc:
The official synopsis of MassTransit, from its website is as follows:
“MassTransit is lean service bus implementation for building loosely coupled applications using the .NET framework.
The lean implementation is supported by the YAGNI principle. By focusing on a tight set of specific concerns, the touch points between Mass Transit and the application are minimized resulting in a clear and concise set of interfaces.”
But what does it actually do?
Well as the brief states MassTransit is a service bus for building loosely coupled applications. Let’s break that down.
So what is a service bus and how does it aid in loose coupling? Well hopefully the follow example may help you understand:
You may have 1 or more systems/components communicating with out any real knowledge of who or what each systems/components is. For example if I what to Create a New Purchase Order for a client I may send a specific message, for example a message called “CreateNewPurchaseOrderRequest” (with the company and order details inside that message) MassTransit picks up this message and forwards on to which ever system(s) is registered to deal with that type of message. Now my sending component does not know or care which system receives and deals with that message, it has done its job of raising the purchase order.
Ideally there is another system that is ready to deal with that type of message. It is given the message from the buses queue and the service processes the data in the message; persisting, starting workflows etc. It may even send a message back to the bus, say a “CreateNewPurchaseOrderResponse“, to acknowledge that, yes, the purchase order has been successfully created. The original sender of the CreateNewPurchaseOrderRequest may (or may not) be interested in this response and can subscribe to the message type and act on it accordingly.
What this does require is that the bus is alerted of who wants to receive what type of messages and also that when a message is created and ready to be sent, it is passed to the bus.
Now this scenario raises a bunch of questions
Why not use method calls/web services?
• A service bus like MassTransit allows you to send and forget. You give the bus a message and let the bus decide where to send the message. This means you do not need to know that HR is now looking after message X and the Logistic is dealing with message Y differently. You don’t care. You also don’t need references to those other systems. Changes to other systems are much less likely to affect your system.
Why the ignorance?
• What happens if the business now decides that when we raise a Purchase Order we also need to alert Logistics, Warehousing, Marketing and the original Sales department? If you where using traditional web services this means you would need to go into you code add references to those web services and make class to them from the raising application. With a service bus you raise the message and let the the bus forward the message on t the interested parties. It means the component focuses on the job it is supposed to do and not worry about departmental interactions. This concern is now that of the Bus
Why messages?
• As business and their technical solution grow, maintaining those technical solutions can become difficult. Architects have been using SOA approaches to help delineate logical service and use messages as the means of moving data around. A message in itself guides the bus to what its intended purpose is. A CreateNewPurchaseOrderRequest is quite self explanatory and very specific. Well constructed messages and services means disparate components can talk to each other, showing their intention but hiding the implementation.
Benefits of MassTransit
• Loose coupling/Recipient ignorance: Send and forget. In the same vein you can subscribe to a message type and not care where it was sent from.
• Asynchronous communication: no need to hold up connections and thread waiting for replies. You can send the request and if you are interested in the response subscribe to the message type
• Durable and reliable messaging thanks to the message queue backbone (e.g. MSMQ)
• Light weight : MassTransit.ServiceBus.dll, MassTransit.ServiceBus.MSMQ.dll & MassTransit.WindsorIntegration.dll combined are under 100KB
• .Net Type safe. You can subscribe to message based on the .net type of the message. No need to interrogate XML files or manually transform .net types to xml etc
• Castle integration: The stack is built using best practices such as IoC and is ready to integrate with the popular .net Library “Castle”
• Open Source: The source code, including unit tests and samples, are available openly.
Who would want to use MassTransit?
• A designer or developer who’s systems are predominantly in .Net realm.
• A designer or developer that wants to have a means to communicate to separate systems in a loosely coupled and possibly asynchronously manner
• A designer or developer that wants to have a reliable and durable means of communication
• A designer or developer that wants to have an easy means to set up a messaging bus!
What does this require of the developer?
• You will need a supported Messaging Implementation such as MSMQ
• You will need to base your communication on a message construct i.e. each message has a specific intent and relevant data. Passing entities is not explicit enough as it does not show intent.
• You will need to reference MassTransit and use the library accordingly! (see the samples for more info)
I hope this gives some basic back grounds as to what MassTransit is and does. Any question please email me or for technical issues go here
Projects measuring success..
Intention revealing interfaces
In the argument of “Intention revealing interfaces” vs “Reusable DTO/message” i will 99% of the time lean heavily toward the “Intention revealing interfaces” side.
I have just encountered code where i can see a (somewhat) noble intention of trying to keep the number of messages available in an assembly down to a minimum. Considering the system(s) mainly passes entities across boundaries, message are not common currency.
Unfortunately i don’t think enough thought was put into the decision. Leaving the notion of passing entities around as another issue, the fact that this DTO is now being used as a request AND response object in MULTIPLE service functions means the intention of each DTO is very unclear.
There are multiple fields on the DTO that are not used by the service and there is a magical “ObjectPacket” property that is of type object (as in System.Object) that is actually quite important. The reason the property’s type was left as object is because of “reuse”, as the different functions do different things with that property.
OMG.
Many hours, nay days, of bug fixing could have been saved if this DTO was split in to just 4 classes, 2 response and 2 requests. These could have been generic enough for the remained of the functions and revealed, along with the service function names* the intention of the call.
Thinks of other when you names stuff (classes, components, service, methods and parameters) and think of whether re usability is actually a benefit before bundling on bonus properties to a transfer object. 😉
*yes this is a JBOS interaction
VS2008/3.5 SP1 == Fail
Ok, so its been about a week of running SP1 on vs2008 and all i can sy is: “WTF?!?” There is no need to list the bugs, as the interweb is chocka full of them (including a couple I have posted to the conect team), but I must say that I have been incredibley underwhelmed. When i install an SP i dont expect everything to break. This is the worst release i have encountered from M$ in a while. I am really dissapointed in a team that i thought was on the up and heading in a new positive direction. 😦
Rhys Campbell 