Test automation has been a grey area for quite some time. I did not think it was fun nor saw the real benefits. Today I have to revalue those thoughts. For about a year now I have been working with the test automation framework Fitnesse. This is my evaluation:
What is Fitnesse?
Fitnesse is, as mentioned earlier, a test automation framework. It consists of two parts: wiki pages and fixture code. As far as I know the fixture code is in Java only. We have only used a small part of the framework. Fitnesse is far more complex and versatile.
As my speciality lies in test and not programming, I had to get a lot of help from the developers so write the fixture code. Working close to the developers like this had both its benefits and drawbacks. At the same time as the developers got a good insight to my tests and way of thinking, it took a lot of time to produce the fixture code as the tasks were not always of the highest priority for them. If you want to get the most out of Fitnesse, the tester should be able to write decent Java code on their own.
The wiki pages were easy to write and manage. Well, except for the symbolic links. Though when I got them in place they worked well and were most useful. We used symbolic links to create test sets.
We used the framework to automate most of our test cases. We only left out the usability, performance and gui test cases as those were used by testers who did not have access to Fitnesse.
I believe that Fitness could be used to document a most test cases. Not only those that are automated. Let’s say that you write 10 test cases in Fitnesse. One test case per wiki page. 7 of them are suitable to automate, so you do that. To let the framework know that it is a wiki page containing an automated test case, you change its property to “Test”. Now when you run your automated tests, the framework will ignore all pages that have not had the property changed to “Test” and only run those that have. By this we have achieved that you can keep all automated and none automated test cases in the same place giving you a great overview of the coverage.
I only thought of this late in the project and really would have liked to try it out more. However all unfinished automated test cases have had the “Normal” property for quite some time so it should work just fine.
What is the key benefit I’m reaching for? Well, I believe that many projects mix automation with the more common “manual” testing (and they should) struggle with having multiple tools. You have some test cases in an automation tool and some in QC (or other similar tool). That does not give you any clear overview, does it? That is why I became so curious to whether Fitness could solve this or not.
Yes, I do see drawbacks too. It is not suitable in any way to store test results from “manual” testing I Fitnesse. So you still need a tool for that. Also I cannot promise that moving from one “manual” test case to another will be all that smooth.
Bottom line is that I liked Fitnesse as test automation framework and really would like to learn more about its possibilities. And perhaps explore a further usage to support other test approaches.
Read more and download Fitnesse at http://fitnesse.org/