Well, we all make mistakes from time to time and this weekend I made a good one.
I’ve written a few posts lately on why I think small businesses need Team System. This weekend I got a comment from Ben Scheirman on my post about the cost of Team System licensing. Ben’s comment was in favor of using free and open-source options such as Subversion, Basecamp, Fogbugz, Cruise Control, and NUnit instead of Team System. Ben also feels that, in addition to the price, these are superior products to the offering from Microsoft.
So far, it’s pretty non-controversial, right? So where’s my mistake?
Well, I deleted the post. Yup. Deleted it. I responded to Ben privately by email explaining why I’d deleted the post and why I thought he might not be 100% right but the bottom line is that I deleted the post.
That wasn’t my best decision ever but here’s what I was thinking. I make my living by teaching customers how to adopt, use, and customize Visual Studio Team System and my blog posts are a big part of how I advertise. While we’re at it, I might as well say that I also happen to really like the product. (More on that later in this post.) On a blog that I use to sell services related to Visual Studio Team System, I didn’t especially want to plug the competition. Plus, on a post about licensing, I didn’t want to have a long, public argument about Team System vs. Everything Else. In addition to those reasons, having a public argument just isn’t my style – especially an argument where I would have to publicly rebut someone and on a topic where I don’t think there’s necessarily a Single True and Right Answer.
So, I deleted the post. The problem is that doing so slammed the door on the whole debate and made it seem like I might be hiding something or think that Team System couldn’t stand up to criticism. Not the case. I made the wrong choice and it caused a bit of a kerfuffle.
Moving on.
It shouldn’t be a surprise that I’m a fan of Microsoft Visual Studio Team System and Team Foundation Server and I happen to think that it has a lot going for it. I think it’s a great product and that’s why I specialize in it and recommend it to my customers but like a lot of things in technology, it’s Apples vs Oranges. There are trade-offs with each choice but at the end of the day, it’s up to you to decide which one you like and, once you’ve made your decision, the best you can say is that you picked the product that’s right for you.
Going back to Ben’s issues with Team System, my rebuttal to his points mostly revolve around Team System being an integrated solution. Here are Ben’s issues with Team System and my response to each.
Issue #1: Using Git and Subversion for source control introduces “far less friction” when only occasionally connected
Ben suggests that having central commanding server that locks your files is not a good approach for distributed teams. Assuming that you’re completely unproductive and unable to code as soon as you pull your network cable, then yes. That’d be pretty bad. Fortunately, that’s not the case.
The ability to add, edit, and delete files offline (i.e. without a connection to TFS Source Control) was added to VS2005 via the Team Foundation Server Power Tools and is now part of the Team Explorer 2008 default installation. Make your disconnected code changes and when you’re back in the office again, click the “go online” button in Visual Studio and your changes are sync’d back to TFS.
Issue #2: “Most people…don’t like managing work items within Visual Studio” because the user experience is “horrible”
Ben specifically mentions a “dropdown box that is 1600px wide” as being a problem and that the user experience is only slightly better when you’re using Team System Web Access. Well, if you don’t like the interface, you can edit it. The user interface for displaying/editing work items is editable through an HTML-like XML configuration file. If you don’t like how it looks or perhaps the workflow isn’t exactly what you want; hop into the XML and change the layout or maybe change the control that is used to display a data field. If you can’t make it work by tweeking the XML, you can also write your own user controls that can be displayed within your work items.
Team System is extremely customizable. If something isn’t working the way you want, change it.
Another option is to use the integration between TFS Work Items and Excel and Microsoft Project. If I need to create 100 different work items quickly, my choice is probably going to be Excel. Make the data work the way you want -- cut, paste, delete, change values – when you’re done, click publish and your changes get written back in to TFS. (You can do the same kind of thing in Microsoft Project, too.)
If you’re always creating the same kind of work items, check out Work Item Templates in the Power Tools for creating work items from within Visual Studio or, if you want to create Work Items without having to start up a browser or Visual Studio, try out “Work Item Stencils” inside of my tool, Dubbelbock.
Issue #3: Subversion, Git, Basecamp, CruiseControl, NUnit, etc is free
Sure it’s free but you have to install, manage, and maintain n applications and make those applications all work together. It might be possible to make them work together but the point is that they weren’t BORN to work together.
Team System and Team Foundation Server were written together and are engineered to work together. Plus, if you’ve got a problem you can call up Microsoft Support and get answers. You can get answers from the open-source community but if you aren’t paying them for the product, what’s your guarantee that they’ll support you on the schedule that your company needs?
Time is money. Is the money that you save by not buying Team System worth the time might lose supporting the free product? It might be. It might not be. Personally, I don’t think so and, in any case, I think the integration of all of the non-source control features with TFS’s source control repository is worth the cost.
Issue #4: Team System’s Automated Build System doesn’t integrate natively with NUnit
Ben’s right. If you want to run NUnit unit tests from a Team Build, you have to do some customization and probably use a 3rd-party component. But the idea with Team Foundation Server and Team Build is that you’re not going to be using NUnit but instead you’ll be using the unit testing features of Visual Studio (MSTest). Does Cruise Control automatically run MSTest unit tests? My guess is that it’s entirely possible but you have to make it do the work. Not a huge deal but just not available out of the box.
The same goes for running NUnit tests from a Team Build. It’s not that hard. You just have to customize it.
If you choose to use NUnit tests with Team Build, what you’ll miss is the integration between MSTest unit tests, your project plan, your defect information, Team Build, and source control and the business intelligence that is gathered by the tool. (I’ll talk more about that in a moment.)
Issue #5: “MSTest has always lagged behind the other free [unit] testing frameworks”
Ben’s point is why use Visual Studio unit tests (MSTest) when you can use NUnit for free. Well, if you’re using Visual Studio 2008 Professional, you’ve already got unit tests as part of the IDE. So, if you want to run the tests, you just run them out of a dialog within the tool. If you want to set a breakpoint and debug your code while you’re running your tests, start the tests in Debug mode from within Visual Studio.
You can do the exact same thing in NUnit but you’ll need to install some extra stuff in order to get the integration. Not the end of the world but it’s about the native integration.
If you’re using a Team Edition of Visual Studio (Team Suite, Team Test, Team Developer), you also get integrated code coverage and code profiling. Once again, if you want code coverage with NUnit, you can do it but it’s not integrated in to the tool.
If you’ve got Team Suite or Team Test, you also have the ability to write Web Tests against that exercise and validate your web applications. Once you’ve created the Web Tests, you can then turn those in to Load Tests. This is all integrated in to Visual Studio. You can run these tests all together. You can easily run these web and load tests from a Team Build.
Summary: Why is integration important?
What are the key pieces of Team System? On the server side, Team Foundation Server (TFS) gives you source control, work item management for managing your project plan and bugs, automated builds, and reporting. On the client side, you can run and debug a number of types of unit tests directly and natively through the Visual Studio user interface.
All of these pieces of Team System work well together because they’re part of the same product.
There’s also the added benefit of being able to associate your code check-ins with a work item such as a task or a bug. Associating your work items to your check-ins gives you traceability on your project. Want to see all the check-ins that were required to implement a feature? The data’s there. Want to see the bugs that were fixed on a check-in? The data’s there.
Now let’s say that a continuous integration build gets kicked off because of your check-in. TFS will associate all the new check-ins that went in to that build with the instance of the build. Since the check-ins are associated to the build and the check-in is associated to your work items, you can now say what bugs and tasks went in to a build. If the build fails, TFS can assign a Bug to the developer that triggered the failed build. Need to QA a build? As you create bugs, associate them to the build number.
You can associate just about anything in TFS to just about anything else in TFS. These associations then get tracked by TFS and are put into a data warehouse for you. TFS also gives you a number of canned reports so that you can see the status of your project. You can also connect to the data warehouse directly using Excel or write your own reports using SQL Server Reporting Services – this allows you to answer the questions that you have about your project status so that you can know when things are going well – but more importantly – know when things are going off the rails. One of my favorites is the “Quality Indicators” report. This shows you trends in code churn, code coverage, unit test passes/failures all graphed together over time on a per-build basis. Amongst other things, this lets me know how my code coverage numbers are trending and, if they’re going down, I can go talk to my team to find out why they’re going down.
Having all this intelligence about the status of your project is one of the biggest reasons why “Team System helps increase the predictability of success” and you don’t have to cobble a bunch of disparate tools together to make it happen.
-Ben
posted @ Monday, July 27, 2009 2:39 PM