I’ve been meaning to set up some backups for ages and at last today I got around to it. I have a bunch of computers in the house and they kind of back each other up but if my house goes up in smoke or if some sort of subterranean mole creates invade Alberta in revenge for the oil sands I’m out of luck.

I really like this company BackBlaze who offer infinite backups for $5 a month. I have some issues with them which I’ve outlined in another blog post which is, perhaps, a bit too vitriolic to post. Anyway they have little client which runs on OSX or Windows and steams files up to their data center. What they lack is support for UNIX operating systems and I wanted a consistent backup solution for all my computers so something platform portable was required. They also had some pretty serious downtime a while ago which I felt highlighted some architectural immaturity.

My next stop was this rather nifty S3 BackupSystem. Again this was windows only and couldn’t backup network drives so it was out.

Finally I came acorss s3sync.rb a ruby script which was designed to work like rsync. It is simple to use and can be shoved into cron with narry a thought. At its most basic you need to do:

Set up Keys

or set the variables in

Backup /etc to S3

Restore /etc from S3

This assumes you have set up S3 already. If not head over to Amazon and set one up. When you sign up take note of the keys they give out during the sign up. If you fail to do so they can still be accessed once the account is set up from Security Credentials under your account settings. S3 is pretty cheap and I think you even get 15gig for free now. If that ins’t enough then it is still something like 15 cents a gig/month in storage and 10 cents a gig in transfer. Trivial to say the least.

Yet another TFS post! Is it because I believe that TFS is the cat’s meow and that you should all run out an buy it right now? Not so much but it is starting to grow on me and I can admit that building .net projects building it is okay. There is really a need for a collection of workflow tasks and some better generic templates. I’m going to see if I can convince my boss to let me open source some of the stuff I’m building for running NUnit tests and the such.

One thing which is missing from TFS is the ability to embed build information in the binaries which are built. I always like this because it is easy to get clients to right click on a file and tell you the version number which will give you an idea as to where to start looking at their problem. The version numbers in .net builds are set using attributes which are commonly defined in Properties/AssemblyInfo.cs. The quickest solution is to update the .cs files before running the compilation step of the build. To do this I created a new task to plug into the windows workflow process that is run during a build in TFS2010. Funnily enough Jim Lamb used setting the build numbers as his example for creating a custom TFS task.

I started with the project he provided and made changes to it because my build numbers are slightly different from his. Jim uses the date as the build number, that’s reasonable but I want to know the exact changeset. I also changed what was passed into and out of the task. It made sense to me to pass in the IBuildDetails object rather than trying to couple task together to get out of the object what I wanted.

First change the in argument to BuildDetail

[RequiredArgument]
public InArgument BuildDetail
{
get;
set;
}

Then the execute to set up all the fields I want

protected override void Execute(CodeActivityContext context)
{
String filePath = FilePath.Get(context);

if (String.IsNullOrEmpty(filePath))
{
throw new ArgumentException( “Specify a path to replace text in”, “FilePath”);
}

if (!File.Exists(filePath))
{
throw new FileNotFoundException(“File not found”, filePath);
}

IBuildDetail buildDetail = BuildDetail.Get(context);
if (buildDetail == null)
{
throw new ArgumentException(“Specify the build detail”, “BuildDetail”);
}

// ensure that the file is writeable
FileAttributes fileAttributes = File.GetAttributes(filePath);
File.SetAttributes(filePath, fileAttributes & ~FileAttributes.ReadOnly);

string contents = File.ReadAllText(filePath);

contents = SetAssemblyConfigurationString(contents, buildDetail.BuildNumber);
contents = SetVersionNumberString(contents, buildDetail.SourceGetVersion);
contents = SetVersionFileNumberString(contents, buildDetail.SourceGetVersion);
contents = SetAssemblyCopyrightString(contents);

File.WriteAllText(filePath, contents);

// restore the file’s original attributes
File.SetAttributes(filePath, fileAttributes);
}

Slap it into the workflow, set the properties and you’re good to go. I put my task after Get Workspace in a branched version of the default workflow.

One additional thing to remember is that you need to check a compiled version of your task into TFS somewhere and point the build controller at the location. Thanks to this all the binaries which come out of my build are stamped properly.

I have been working the last couple of days on doing web deployment as part of one of my TFS builds. There are quite a few posts out there on how to do it and so far every last one of them is wrong. Technically they are fine, doing what they say will for sure deploy a package to your webserver*. However they fail to grasp the deploy only on successful build model. Why would I want to deploy something to a webserver which I had tests proving it didn’t work? All that is gonna get you is trouble, and I have enough trouble around these parts. In order to get the behaviour I wanted I needed to edit the build workflow.

First step is get your build to produce the zip file and other files associated with deployment. To do that you need only add

/p:CreatePackageOnPublish=true /p:DeployOnBuild=true

to the MSBuildArgumetns in your build definition. Here is a picture because it makes this post look longer and easier to follow:
Woo, picture!

Next I branched an existing build template. These templates are located, by default, in BuildProcessTemplates folder at the root of your TFS project. You can put them anywhere you want, however, and you might want to put them in the same directory as your solution files, just to keep them spatially proximal. You want to branch rather than creating a new one because the default template is amazingly complex and recreating it would be painful. The default workflow is actually pretty reasonable and I have only minor quibbles with how it is put together.

Next we’re going to to add some new parameters to the build so that people can easily configure the deployment from the build definition. To do this open up your newly branched template and click on arguments. I have yet to figure out how you can put these arguments into categories from the UI but from the XML it is pretty easy. I don’t care enough to put things into categories at this time, although it probably took me longer to write this sentence about not caring than it would have taken to just do it.

As you can see I added 6 new properties

• PerformDeploy ““ A boolean which denotes if we should attempt a deploy at all. This keeps our project generic so it can be used on anything.
• DeploymentURL ““ The URL to which the deployment should run. I’m stuck on IIS6 so this is the MSDeployAgentService URL
• DeploymentUserName ““ The username to use during the deploy
• DeploymentPassword ““ The password to use during the deploy. You might want to be more creative about how you get your password to comply with whatever rules exist at your site.
• DeploymentApplicationPath ““ The path in IIS to which deployment should be run
• DeploymentCommand ““ The name of the *deploy.cmd file to run. This should

Now for the fun part: adding the new workflow item. If you have a slow computer, say anything slower than the computer from Voyager then you’re going to be sorry you ever opened this workflow. Scroll to the very bottom where there is a task for dealing with gated checkins. Our new task is going to go after this. Drag and If from the toolbox. In the conditional we’re going to check that we had no build failures and that we should run a deploy:

BuildDetail.TestStatus = Microsoft.TeamFoundation.Build.Client.BuildPhaseStatus.Succeeded And PerformDeploy

Now drop in an invoke process to the Then. In the arguments put

String.Format(“/M:{0} /U:{1} /P:{2} “”-setParam:name=’IIS Web Application Name’,value=’{3}’”” /Y “, DeploymentURL, DeploymentUserName, DeploymentPassword, DeploymentApplicationPath)

File name:

Path.Combine(BuildDetail.DropLocation, DeploymentCommand)

I also dropped a couple of WriteBuildMessage actions on there because that is how I roll. Remember this: the longer the build log the more important your job. Let’s see that junior developer guy who only gets paid half what you do figure this 29Meg log file out.

That is pretty much it, save it, commit it and create a new build definition which uses it. You can either run it on commit or manually. I am running mine manually because I don’t trust our unit tests to prove correctness yet.

*Assuming you have everything configured properly and you’re not relying on black magic or even mauve magic.

I had a bug to fix earlier this week which was really just a UI bug. The details aren’t all that important but it was something like a field in a popup div wasn’t being properly updated. This felt a lot like something which could use not just a test in the back end to ensure the correct data was being passed back but also a test in the front end to make sure it was being displayed. As our front end is all webby, as most front ends tend to be these days. Years ago I did some work with Mercury QuickTest as it was then called(I believe it is now owned by HP which makes it pure evil, just like HP-UX). I looked at a couple of tools for automating the browser and decided on selenium partially because it appeared to be the most developed and partially because I couldn’t refuse a tool with an atomic mass of 78.96.

This is the part where I warn you that I really have no idea what I’m doing, I’m no selenium expert and I’ll probably change my mind about how to best implement it. I’ll post updates as I change my mind.

I started by installing the selenium IDE and, on another computer, the remote control. The IDE is just a plugin for firefox while the RC is a java based tool which happily runs on windows. I opened up a port for it in the windows firewall”¦ oh who am I kidding I turned the damn thing off.

I recorded a number of actions using the IDE and added a number of verifications(you can add a check by right clicking on an element and selecting one of the verify options in there). I don’t consider these to be unit tests since they tend to cross over multiple pages. What I’m creating are work flow tests or behavioral tests. These could be used as part of BDD in conjunction with a tool such as Cucumber or SpecFlow. As such I don’t really care how long these tests take to run, I don’t envision them being run as part of our continuous integration process but rather as part of hourly builds. If things really get out of hand(and they won’t our application isn’t that big) then the tests can be distributed over a number of machines and even run in the cloud on some Amazon instances.

In the Selenium IDE I exported the test to C# so they could be run in our normal build process. Unfortunately the default export template export nunit tests and our project makes use of the visual studio test suite. There are no technical reasons for not having both testing frameworks but I don’t want to burden the other developers too much with a variety of frameworks. So my job for tomorrow is to explore alternatives to the standard export. I am also not crazy about having to maintain the tests in C#, it requires that a developer be involved in testing changes when it should really be easy enough for a business person to specify the behaviour. I have some ideas about running the transformations from Selenium file format(which is an html table) into C# using either T4 or the transformation code extracted from the IDE running in a server side javascript framework.

I’ll let you know what I come up with. Chances are it will be crazy, because sensible isn’t all that fun.

Useful Blogs

• Design of Selenium tests for Asp.netA blog with some specific suggestions about how to work with selenium and asp.net. I’m not in 100% agreement with his ideas but I might change my mind later.
• Adam Goucher’s blog ““ Adam is a hell of a nice guy who has been helping me out with Selenium on the twitter.

As you all should I am using web.config transformations during the packaging of my ASP.net web applications. Today I was working with a project which didn’t have transformations defined previously so I thought I would go ahead and add them. All went well until I built a deployment package and noticed that my transformations were not being applied. Looking at the build log I found warnings like these

C:\Users\stimms\Documents\Visual Studio 2010\Projects\SockPuppet\Web.Debug.config(5,2): Warning : No element in the source document matches '/configuration'  
Not executing SetAttributes (transform line 18, 12)  
Output File: objDebugTransformWebConfigtransformedWeb.config

This started a fun bit of debugging during which I removed everything from the web.config and built it from the ground up. Eventually I traced the problem to a hold over from some previous version of .net, in the Web.config the configuration was defined as

<configuration xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0">

Changing that to just

<configuration>

Solved the problem. I imagine this is some sort of odd XML namespace issue, hopefully if you run into this you’ll find this post and not waste an hour like me.

I have an autonomous component in my NServiceBus implementation which needs to talk to a MySQL database. When I first implemented the handler I configured the end point to be transactional, mostly because I wasn’t too sure about the difference between configuring AsA_Service and AsA_Client and transactions sounded like they might be something I would like. What the transactional endpoint does is wrap the endpoint in a distributed transaction. A distributed transaction is a mechanism which allows you to share a transaction between a number of databases so that if you’re writing to several databases when you commit to one database and it fails then the transactions in the other databases will rollback.

However when I went to test the handler it failed with an error:

MySql /Net connector does not support distributed transactions

I solved it by configuring the endpoint AsA_Client. The problem with that configuration is that the handler wipes the queue on startup which isn’t ideal. None of the built in configurations are quite right for our situation so we override the configuration as described here: http://www.nservicebus.com/GenericHost.aspx.

I make heavy use of Red Gate’s excellent SQL Compare tools. I know I’m a bit of a shrill for them but they are time savers when dealing with multiple environments(development, testing, production) which is a pretty common occurrence in any sort of agile development. One flaw in them is that they often mess up the sequences in the destination database. Say you have a table Students with 15 records in it in development and 30 in production then performing a copy often brings along the sequence even if you don’t select syncing that table. This results in duplicate key errors whenever a new record is inserted.

For weeks I have been saying “I should write a script to check and fix that”. Well I finally did it

SET ANSI_NULLS ON  
GO  
SET QUOTED_IDENTIFIER ON  
GO  

create PROCEDURE dbo.FixTableIdentities  

AS  
BEGIN  
 SET NOCOUNT ON;  
 declare @currentKeyValue int  
 declare @currentIdentityValue int  
 declare @toRun nvarchar(500)  
 declare @tableToCheck nvarchar(500)  
 declare @idColumnCount int  
 declare db_cursor cursor for select name from sysobjects where type='U'  

 open db_cursor  
 fetch next from db_cursor into @tableToCheck  

 while @@FETCH_STATUS = 0  
 BEGIN  
 select @idColumnCount = count(*) from syscolumns where id=object_id(@tableToCheck) and name='id'  
 if(@idColumnCount = 1)  
 BEGIN  
 select @currentKeyValue = ident_current(@tableToCheck)   
 set @toRun = N'select @currentIdentityValue = max(id) from ' + @tableToCheck;  
 EXEC sp_executesql @toRun, N'@currentIdentityValue int OUTPUT', @currentIdentityValue OUTPUT;  
 if(@currentIdentityValue @currentKeyValue)  
 BEGIN  
 DBCC CHECKIDENT (@tableToCheck,reseed, @currentIdentityValue)   
 END  
 END  
 FETCH NEXT FROM db_cursor into @tableToCheck  
 END  
 CLOSE db_cursor  
 deallocate db_cursor  
END  
GO

When run this procedure will go through all your tables and ensure that the id column is in sync with the sequence. At the moment it just looks at the column called id and manipulates that.

I recently asked a question over at stackoverflow about the productivity gains in various languages.

Does anybody know of any research or benchmarks of how long it takes to develop the same
application in a variety of languages? Really I’m looking for Java vs. C++ but any
comparisons would be useful. I have the feeling there is a section in Code Complete
about this but my copy is at work.

I was really asking because I wanted to help justify my use of the Pellet semantic reasoner over the FaCT++ reasoner in a paper.

What emerged from the question was that there really was not much good research into the topic of language productivity and that any research which had been done was from the 2000 time-frame. What makes research like this difficult is finding a large sample size and finding problems which don’t favour one class of language greatly over another. That got me thinking, what better source of programmers is there than stackoverflow? There are developers from all across the spectrum of languages and abilities; there is even a pretty good geographic disbursement.

Let’s do this research ourselves! I propose a stackoverflow language programming contest. We’ll develop a suite of programming tasks which try as hard as possible to not focus on the advantages of one particular language and gather metrics. I think we should gather

• Time taken to develop
• Lines of code required
• Runtime over the same input
• Memory usage over the same input
• Other things I haven’t thought of

I’ll set up a site to gather people’s solutions to the problems and collate statistics but the problems should be proposed by the community. We’ll allow people to checkout the problem set, time how long it takes to the to complete it and then submit the code for their answers. I’ll run the code and benchmark the results and after, say two weeks of having the contest open, publish my results as well as the dataset for anybody else to analyze.

In the code base I’m working with we have a number of objects which augment existing objects. For instance I have a User object which is generated by my ORM so it looks like

 string userName;  
 string firstName;  
 string lastName;  
 int companyID;  
 int locationID;

In order to display user objects it is useful to have the name of the company and the location which are stored in another table. To limit the amount of stuff being passed around we defined an ExtendedUser which extends User and adds the fields

 string companyName;  
 string locationName;

Creating these extended classes requires passing in a base class and then pulling all the properties off of it and assigning them to the extended class. This is suboptimal because it means that when a new property is added to the bass class it has to be added to the code which extracts the properties in the extended class. To address this I created a method which iterates over the properties in the base class and assigns them to the extended class.

public static void CopyProperties(this object destination, object source, List<string> ignoreList)  
 {  
 foreach (var property in destination.GetType().GetProperties())  
 {  
 if (source.GetType().GetProperties().Select(p => p.Name).Contains(property.Name) && !ignoreList.Contains(property.Name))  
 {  
 var sourceProperty = source.GetType().GetProperty(property.Name);  
 if (property.CanWrite && sourceProperty.GetType() == property.GetType() && sourceProperty.GetValue(source, null) != null)  
 property.SetValue(destination, sourceProperty.GetValue(source, null), null);  
 }  
 }  
 }

If you have sharp eyes you’ll notice that I’ve defined this method as an extension method. This allows me to do insane things like

ExpandedClass expandedClass = new ExpandedClass();  
expandedClass.CopyProperties(cls);  
expandedClass.LocationName = GetLocationNameFromID(cls.LocationID);  
expandedClass.CourseName = GetCourseNameFromID(cls.CourseID);  
expandedClass.InstructorName = GetInstructorNameFromID(cls.InstructorID);  
expandedClass.CompanyName = GetCompanyNameFromID(cls.CompanyID);

I can also do this for any other two classes which share property names.

Rob Conery, who is a giant in the ASP.net MVC world(he wrote the ASP.net store front and is also the author of a 200 page book on inversion of control) is calling for suggestions about an active record persistence engine. I wanted to present how I think it should be done which is just a bit too long for a comment on the tail end of Rob’s blog. I’ve been reading a lot lately about areas in MVC2 and the portable areas project which is part of MVC contrib project. Now the portable areas aren’t yet finalized but the idea is that these areas will be able to be dropped into a project and will provide some set of controllers and views which will provide a swack of functionality.

The example I’ve seen bandied about is that of a forum. Everybody has written a forum or two in their time now you can just drop in an area and get the functionality for free. I can see a lot of these components becoming available on codeplex or git hub. Component based software like this is “the future” just like SOA was the future a few years ago. The problem with components like this is that it is difficult to keep things consistent across the various components. At one end of the spectrum of self containment If each component is self contained then it has to provide for its own data persistence as well as any other services it consumes.

I have helpfully harnessed the power of MS Paint to create an illustration of the spectrum between a component being self contained and being reliant on services being provided for it. If anybody is interested my artistic skills are available for hire. The further to the left the more portable the further to the right the more reliant the components are on services being provided for them and the less portable. We want to be towards the left, because left is right.

If these components really are the future then we need to find a way to couple the components and provide communication between them. This is where MEF steps up to the plate. What I’m proposing is that rather than spending our time creating unified interfaces for storing data we create a method agnostic object store. Components would call out to MEF for a persistence engine and then pass in whatever it was they wanted to save. The engine should handle the creation of database tables on the fly or files or web service callouts to a cloud. That is what I believe should exist instead of a more concrete IActiveRecordEngine.

What’s the advantage? We keep the standard interface for which Rob is striving but we can now have that interface implemented by a plugable component rather than having it hard coded into a web.config.

The second part of Rob’s post is about creating opinionated controllers. I have to say I’m dead against that. I agree with the goal of implementing the basic CRUD operations for people, in fact I’m in love with it. What I don’t like is that it is implemented in a base class from which my controllers descend. If I’m reading the post correctly then the base controller is implementing actual actions. It is dangerous to implement actions willy nilly, actions which could be dangerous and people wouldn’t even realize the actions exist. Chances are very good that users are just going to leave the actions implemented rather than overriding them with noop actions.

Another option is that I’m reading this post incorrectly and the methods in the base class are private and not actions. I like that a lot more, but even more I like generating template controllers. Subsonic 3 follows this methodology and it is really nice to be able to twiddle with bits of the implementation. What’s more the generation doesn’t have to stop at the controller. If the implementation in the controller is known then why not generate the views as well?

All in all I like the idea of improving the object store support in ASP.net MVC but I would like it to be as flexible as possible.