Jennifer Marsman, one of our beloved Heartland District Developer Evangelists, arranged for Mark Mydland, Principle Group Manager with Microsoft in Redmond for the VSTS Tester Edition to tour our area and deliver a presentation on VSTS Tester 2010. This also doubled as the November Central Ohio .Net User Group (CONDG) meeting. Thanks, Microsoft for the food and swag.
Along with some great product development stories, Mark highlighted the following:
Playback - often, the written repro scenario provided along with the defect isn't clear (if it even exists). This feature enables developers and other roles to watch a video of the functional / UI test from the Tester live, in action.
Test Case vs. Actual Capture - for manual tests, a list of steps involved with the test can be compared to what steps the tester actually took.
Work Item Categories - classify a work item type into categories with one work item serving as the default.
Cut/paste screen shots - easy to [Alt]+[Prnt Scrn] and paste a defect screen shot into a TFS defect work item.
Testing Activity Center - a UI for the layperson tester. No Visual Studio...just a lightweight app to manage test cases and iterations, execute automated tests, and manage defects.
Thread Debugger Agent - from within the playback for a test case, you can step through the code. It only works with events in the CTP but it will include methods and properties for RTM. It works by attaching to CLR profiler, registering for events and then logging them to the test case/defect. This is tied to a specific version of the asset stored within TFS. Wow.
Distributed Test - agents run as Windows services assigned to a single controller to execute and instrument (collect data metrics) distributed testing scenarios. Included in this instrumentation, called Data and Diagnostics, can concurrently capture things such as the Event Log, System Information, custom logging files, PDBs, and even a video recording. Each agent may possess configuration elements to accommodate environmental differences e.g dev/test/staging...different database connection, etc.
PEX - this is a tool which while available in 2010, is downloadable for 2008 from the link. This neat feature executes your methods and recursively discovers all the code paths. It does this by throwing varying values and types at your method attributes e.g. nulls, strings, etc. This produces a Pex Exploration Results. Then, one can use this to generate test harness classes leveraging this sample set. Wow.
Chess - Not here for the CTP but it should come online for a future Beta release. Chess sets up test cases and allows you to debug and break on any context change. Chess intercepts requests to the thread scheduler and re-executes them. Using some magic algorithm, Chess continues to execute thread requests in differing order. This helps determine and test if your code is thread-safe and yields the expected outcome regardless of the order in which it was executed.
Coded UI Test - A new test type which will produce managed code based off a UI recorder.
Thanks for visiting, Mark and thanks for bringing such talent to the district, Jennifer!
Comments