Archive for December, 2011

 

Software testing is not only about verifying functions. A system is so much more than functions, if it’s suppose to be useful anyway. The system has to live in a context and in a world with many different users with many different needs and behaviors.

 

There are many stakeholders, not only the end users. All these stakeholders have different requirements and wishes for what the system should handle or support. It can be deliverables or qualities that must be in the system to make it possible for these people to do their job. For example, if an operating technician should be able to maintain a good level of security there must be routines and support for back up. Taking new back up and restoring an old back up just have to work. These things are crucial for an operating technician while the market department hasn’t dedicated this even the slightest thought. The market department on the other hand, might be anxious about the coolness of what is delivered in the next version, so they can beat the market by surprise in their next press release. Yes, there are many different stakeholders and many different wishes.

Example of stakeholders and their requirements/wishes

 

How do we do to cover more than just functions?

 

First of all I think we have to look beyond requirements specifications, meet all the different stakeholders and discuss what’s important for them. There is a lot of support in lists of test types like performance test, load test, usability test, installation test etc. Collect a list of your own, with suitable test types and go through them for every project to determine which applies for the test object. My list, of absolutely necessary test types that should be included in almost every project, is:

 

–          Functional test

–          Usability test

–          Platform test

–          Security test

–          Access level testing

–          Performance test

–          Stress test

–          Stability test

–          Installation test

–          Recovery test

–          Storage test

–          Volume test

 

There might be other types of test that can be applicable depending on the project and system under test. There might also be the opposite, several of the test types mentioned above might not be applicable or require very little effort. Often this is related to time and money but also what is valuable to test. But, just by discussing an area you have lifted up the different qualities and then the project can make a decision whether the area should be tested or not.

 

When you come to the point where you have identified all stakeholders and their requirements and whishes, you probably have an enormous amount of work before you. To cover all is impossible and most likely not meaningful. Therefore, your next task, possibly even more difficult, is to prioritize. Whose requirements are the most important? We can look deeply into this but the thing I think is the most significant is to make sure you communicate a lot with all stakeholders.

 

Someone whose list is very long and full of details is Rikard Edgren. The last time I listened to him his list was 77 test ideas long. See http://thetesteye.com.

 

What does your list look like?

What smart test types do you have in mind and which to you think is absolutely necessary for most systems/projects?