Team Foundation Server (TFS) and Visual Studio Team System (VSTS)

What is TFS?

It would not be correct to state that TFS is the replacement for Microsoft’s earlier source/version control software (VSS – Visual Source Safe), as I believe TFS is much more that just a source/version control software. If used correctly, it will help Application Lifecycle Management (ALM) and provide a platform for collaboration and when used with Sharepoint it really completes the entire picture (obviously there are other products and companies out there which provide similar capabilities, but this blog topic is going to focus on TFS).

A few of TFS salient features include

• Source/Version control (how obvious)


• Automatic Build process via Team Build which uses Microsoft MSBuild (to all those java developers MSBuild is equivalent to Ant, Maven, although to Ant’s credit there is a Microsoft version for Ant, called “nAnt” – how creative, if it was me I would have named it “bug” )


• Create Work Items and tie them to the different development code modules. Bug, Risk, Scenario, Task, Quality of Service Requirement, Requirement, Change Request etc they all constitute examples of work item types. NOTE: depending on the Process template that you select during the “Team Projects” creation wizard in TFS, some work items may not be available for use. Example: “Requirement” is available only if you select “MSF for CMMI Process Improvement” process template versus “MSF for Agile Software Development” process template. Having said that, TFS allows you to create new Work Item Types as well as create custom process templates. My frank opinion about all this custom stuff is this, try to use the out of the box process templates provided by TFS or use some already built process templates, e.g. there is one for SCRUM, do not be a hero and try to create your own process template as that would be a project in itself, unless you have a home grown Process Template that you inherit before the Project starts do not be a hero. Adding some new Work Types to existing Process Templates is not a big deal; but do not create a custom Process Template on your own or do that before any project work starts; else you will not be able to utilize the biggest benefit that TFS provides which is rapid application Lifecycle management/development with a common platform for team collaboration, as you will spent most of your time tweaking the custom Process Template assuming that its getting refined as you go along with the application development, not a good idea).


• Create work items automatically based on events, for example, say your built fails, that event will create a work item so that you can diagnose the cause of the problem, pretty neat.


• Integrates Testing efforts into the overall application development and provides one central repository for accomplishing the same (refer to my section on “What is VSTS?”)


• Provides tons and tons of out of the box reporting.


• TFS also provides Process Templates, two which are provided out of the box include




• MSF for Agile Software Development


• MSF for CMMI Process Improvement

I do plan to write a blog on Microsoft Solution Framework (MSF), which I think is a really good Framework to adapt and I think Microsoft did a good job of providing Process Templates with default Plans and Specifications which a Project can use and adapt to suite their needs, its this sample templates, one for Agile development and another one based on CMMI, is what I like about MSF; it really guides you at different stages within an Application Lifecycle Management/Development. I plan to dedicate a blog for MSF.

What is VSTS?

Visual Studio Team System (VSTS) is the big brother of Visual Studio.NET. It comes in various editions with each version providing increment features. If you want to try it out download the “Team Suite” edition as it contains all the features of other editions.

Visual Studio Team System is tightly integrated with TFS and provides capabilities to interact with TFS directly from the development IDE used by developers for application development. Team Explorer is one such plug-in that you use a lot. Team Explorer is used to access the TFS. In TFS you create what’s called “Team Projects” (do not confuse this with Visual Studio Projects, as Team Projects can contain more than just .NET code, example – project documents like Coding Standards, System Design documentation, build reports, testing reports, work items etc). Think of “Team Project” as a logical name for your overall project used by everyone for project team collaboration, in fact when you create a Team Project, it will also ask you to create a “Team Project Site” within WSS (Windows SharePoint Services – do not confuse this with the big brother SharePoint Server (MOSS), this is just the mini-me version that used to come before as a service within Windows Server 2003, but Microsoft now removed this from Windows Server 2008 and provide the same as a separate download, in short if you install Windows Server 2008 it does not install WSS, you will have to download this software separately (maybe Microsoft realized that by doing that both products(Windows Server and WSS) lifecycle can be decoupled from each other. TFS can also be integrated with SharePoint Server (MOSS), but you will have to install the TFS and SharePoint Server on separate physical boxes which could sometimes become overkill for small/medium size projects).

One thing that VSTS provides which the Visual Studio.NET does not provide is IDE capabilities for performing Testing and integrating that with TFS. Now you can do unit testing with Visual Studio.NET but that’s just stand-alone and is not integrated with the overall picture of testing. The edition that supports testing is “Test Edition”, however if you have the “Team Suite” edition you are all set. Each edition of VSTS is there for a purpose. The “Test Edition” was meant for testers, I recommend buying the “Team Suite”, that way you are all set (I usually go to an electronic store with a budget to buy a camera with bare minimum features and I end up buying the most expensive one, that way I know the basic features are there and I have some more, which for all practical purposes I will never use but it’s the peace of mind, a little secret, I still do not know all the features that came with my camera, all I know is how to zoom and click and connect to USB drive of my computer to share and archive the same).

VSTS “Team Suite” as well as “Test” edition provides means for you to create Test Projects. Test Projects are like any other project you have created within Visual Studio.NET in the past (Examples of other Projects are: Windows Forms Projects, ASP.NET Web Application Projects) but are used specifically for testing just like how ASP.NET Web Application Project is used for building dynamic web applications. Once the Test Project is created there are various Test Templates that you can use to add to the Test Project

• Unit Test


• Database Unit Test


• Web Test


• Load Test


• Ordered Test


• Generic Test


• Manual Test (text format)


• Manual Test (word format)

A few features of VSTS that I feel emphasizing include:

• Helps you select a code class and method and create a test stub by few clicks


• You can also define Code Coverage which will tell you how much code within the application modules got tested, you can also define color coding to highlight areas of code that has not being tested yet.


• It’s integrated with TFS; you can publish test results to TFS for other team members to review.


• Its integrated to work with the overall Project Management process e.g. you can attach a test result with a work item defined in the TFS “Team Projects”


• There are many more features and this is not a complete list ………………

The various Test Templates are explained below in details

Unit Test

Once you have finished your class and say you wish to test the class methods, you can create a Unit Test which can create stubs that would invoke your class methods. The steps that you need to perform are pretty simple, all you have to do is select the class/method you wish to unit test and right click to select the correct context menu option in order to create a unit test stub class. With the Code Coverage enabled you will be able to monitor what percentage of your code was tested. The Unit Test class that gets created is like any other C#/VB.NET class which you can modify for further control on your testing. For Example, you can attach a data source and abstract the values that you are passing to various methods into a database table (something that I recommend you do as this will help you refine your testing as you progress with your coding efforts).

Database Unit Test

This is basically to test the database stored procedures and functions

Web Test

I once worked on a project where I had to screen scrap an application using Windows C++ SDK and most of my time was spend in just writing a utility to record and playback the mouse clicks/positions and key strokes and I never achieved perfection, but the job was done as the scraping exercise was a one time exercise (thank god). Well if you have developed an ASP.NET web application you can use the Web Test to do testing of that Web application. The steps are pretty sleek

• Start the VSTS recording to record each and every action that you perform on the web application. (I would recommend creating Web Test that test a few web pages of the application that cover some business functionality, that way you can perform incremental testing and not get slowed down by creating one big Web Test, in short do what you do when you unit test a class, one class at a time, in this case one small business flow at a time, which should translate into one or two web page interactions)


• Stop the recording. This will result in VSTS creating a map which converts all the web interaction into a tree structure which you can view from the “Web Test Editor” window.

Within this Web Test Editor Window you can configure the following

• You can add new Web Request in the flow, delete exiting Web Request or edit properties of exiting Web Request
  
• In each Web Request (which represents a web page logically). You can set properties that are based around the HTTP request. For Example, you can define whether the request is a “GET” or a “POST”; you can define what the request timeout is etc.
  
• You can define Extraction Rules for the request, think of this like reading the HTTP response. You can define extraction rules to do that extraction from the HTTP response.
  
• You can define Validation Rules, this is where you can compare the extracted value with say the “expected” value which you can retrieve from a database or a file and make a determination to pass or fail the test using Asset statements.
  
• You can also define properties that apply to the entire Web Context, for example, you can define Username and Password at the Web Test level so that it applies to all the web pages within a web context.
  

You can also test Web Services using Web Test. The steps invoked are a bit different since there is not much to record. But you can record and stop to create the Web Request Tree and then access the Tree in “Web Test Editor” window to configure testing the exposed web methods. At that point of time your testing experience will be similar to what you had when you were testing individual class methods during Unit Testing.

It’s also possible to create coded Web Test. In order to convert a recorded Web Test into a Coded Web Test you will have to right click the recorded Web Test in the Web Test Editor and select the “Generate Code” option from the context menu. VSTS will basically create C#/VB.NET classes that would act as a stub for the Web interaction that got recorded. Once these classes get created, there are events/methods defined within the classes that you can consume/override to have granular control on the web test. For Example, you can create Validation Rules/ Extraction Rules using C#/VB.NET classes that are provided for the same and add them dynamically through the exposed API. You can override events like PreWebTest, PostWebTest, PreRequest, and PostRequest where you can write your custom code.

For Example, you can write code in the “PreRequest” event to log the parameters passed to the web request.

When I was working on a project, to custom built an application that would interact with a web page, I had to do everything manually and use my Windows C++ SDK skills (remember Spy++ guys) to interact with Internet Explorer, now VSTS Coded Web Test encapsulates that and provides us managed code classes in C#, VB.NET to achieve the same. It’s a great bonus for all of us who want to have more control on the testing without spending days trying to use Windows C++ SDK and all the complexity involved with pointers, memory management, processes and what not.

Load Test

I remember I was working on one project where I had to write custom program that would simulate the interaction with the web application. This program was multi-threaded and was tightly coupled to the custom Web application I wanted to test. I used this program to perform load testing as I had this program run from multiple machines with each machine running multiple threads, there by simulating multiple users assessing the application and hence achieving my load/stress/performance testing efforts. I did not take the program to the next level, which is to create a windows service that can run from each machine and in turn can co-ordinates with a central server Windows Service. VSTS Load Test provides all this out of the box. In the VSTS terminology, we have a “rig” that consist of a controller and multiple “agents”, each agent is deployed on a machine that is dedicated for testing and the controller is like a co-coordinator (no testing is done by the controller; it’s the agents who perform the actual testing). The concept is pretty easy to understand. If you have built distributed application then you can easily understand what this architectural approach of rig-controller-agent is. The controller uses a SQL Server database to store the data that it collects during load testing from each agent. You can use this information to see trends and perform further analysis to identify potential bottlenecks.

When you add Load Test to a Test Project you will have to define Scenarios. For each Scenario you will have to provide the following information

• Test Mix – in short what test (Unit Test/Web Test) you will like to include in the Load Test. Define properties like - how many users to simulate, how much Think Time etc


• Browser Mix – specify different browsers to use for testing


• Network Mix – DSL, Broadband, Dialup etc

Once you run the Load Test, the generated Test Results have tons and tons of reports that you can use for analysis. You can create a Results Store in a SQL Server Database for persistence. Typically, when you install the controller on a machine, you will also be asked to create the Results Store (in short what SQL Server instance to use for persistence). Do not confuse the Results Store database with what TFS uses to store the published Test Results for a build that contains .NET solutions with projects that include tests (e.g. Unit Test, Web Test etc) as a part of the Built. TFS has its own database schema that it uses to store published Test Results which is not the same as the SQL Server database we use for storing Load Test Results; in short the load test results are typically stored in a separate SQL Server database and/or schema than the TFS database. You can publish any Test Results for any test (Web Test, Load Test, Unit Test etc.) to TFS for other team members to take a look at the test results. As soon as you publish the Test Results to TFS, that’s when the TFS database comes into picture. I know it sounds confusing, but it’s important to know which databases come into picture and when.

Ordered Test

If you want to run a set of test in a specific order, then you can use Ordered Test list. I was a bit confused when I started using Ordered Test list because there is something called “Test List Editor”, where you can group all the tests within a group referred to as “Test List”. At first I thought the order in which you add individual tests (e.g. Unit Test, Generic Test, Web Test etc) will define what order the tests will get executed. This is not true. If you wish for that, then you will have to create an “Ordered List” and then add the “Ordered List” to the “Test List”. I am not sure why Microsoft designed it that way, when you can just add an attribute in the “Test List” and do a ordering that way (I am a simple man, with simple solutions, I am pretty sure there are good reasons for Microsoft to do it that way, and its my lack of imagination, or they already have what I am saying and its just that I did not see it)

One thing to note about “Test List”, if you wish to include a “Test” from your Test Project into a build, you will have to create a “Test List”, as the Wizard for creating a Build Definition file does not see the Test directly, although there are other options like including tests from a “DLL” which can be used as an alternative, if you really do not wish to create a “Test List”

Generic Test

You select this “Test” type if you wish to include a wrapper around a third-part test. There are some guidelines/rules that the third-party test has to adhere to in order to participate in the overall testing efforts.

Simple guidelines/rules like

• Must be able to execute from command line


• Must return a Boolean value of either True or False


• Ensuring that the third-part test writes test results into a XML file using a XSD schema provided by VSTS. This allows VSTS to display the test results within its IDE (e.g. in its “Test Results” window)

Lets say you wish to send information to another executable, how would you design that interaction (assuming that you are not doing inter-process communication and all that complexity that goes with it). You would basically use XML to send inputs and XML to read outputs from the third-party executable, and you will define schemas to control the schematics of the XML; this is what the Generic Test does.

Manual Test (text format as well as word format)

If your project does not have a budget for doing automated testing, you can use this Test type to perform manual testing. VSTS provides sample boiler plate for Unit Test Specifications which you can use to document the steps that you are going to perform manually for this test and to capture the expected and actual results. This is the traditional way of doing testing. The benefit you get with VSTS/TFS is that by using a common platform for collaboration, you can increase team co-ordination and/or communication (Example, the Manual Test could be tied to Work Items within TFS, this allows the tester to co-ordinate with the developer(s) involved with the code the tester is testing, a work item could be a Task, Bug etc).

Where does TFS and VSTS Testing Meet?

TFS is providing you a platform for team collaboration at a level which I believe will help you develop an application in a much cohesive way. VSTS is very much integrated with TFS. Areas where VSTS integrates with TFS include

• You can use VSTS as an IDE (more specifically the Team Explorer) to navigate the “Team Projects” that you have created in TFS.


• You can configure automated builds from VSTS.


• You can publish “Test Results” from VSTS to TFS (you can also do that using the command line utility “MSTest” but the IDE provides a click and publish capability with ease of use in mind)


• Obviously you can use VSTS for application development and use the TFS “Source Control” feature to check-in and check-out code for versioning

There are so many features in TFS/VSTS that I will end up writing a book. I do plan to write another blog for Microsoft Solution Framework (MSF) and explain how it ties into TFS/VSTS.