I tried a couple of ways of doing “testing days” or really focused testing sessions. The common denominator might be that testing should be done really focused under a short time period. Administration and planning are done to a minimum and bug reports is often enough as documentation once the testing is done. The expectations are usually that a lot of issues will be found that the usual, often requirement focused testing with few testing opportunities, won’t find. Let’s leave this for now… what I want to write about is that I’m starting to find a way of doing this that I really like.

In my example I will tell you how I and one other person (a project leader) did one of these testing sessions. We tested an application, available for both Android and Iphone, which I knew almost nothing about. Let’s call the application “AppX”. What I will show you below is what we did but I changed it a bit to anonymise the application.

 

We spent 2 hours like this:

• 15 min test planning
• 1h 25 min testing
• 20 min test reporting

 

The goal was to test as much as possible and to find as many critical bugs as possible. All testing should be visualized as much as possible, in test logs, notes, pictures etc.

The result was 15 new bugs and this is what it looked like:

 

What we did?

 

The notation I used:

 

I really like this way of working. Why don’t you try it!?!

 

 

“Technical tester”, “Automation Expert” and “Test Engineer” are sought after skills. Everyone wants them, everyone needs them. We go from heavy manual tests in waterfall projects to Agile development with rapid and automated testing. In the agile world the test cycles becomes more frequent and the regression tests really heavy since they need to be run more often. This requires a lot of manual testers or automation. The automation itself makes the tests become more technical. The number of testing tools is increasing, but they also increase in complexity. We can record all types of protocol language, we can, with a little fix in the script code, manipulate our test case to be something completely different. We have well thought structures for automatic verification where modules are reused. We feed the test tool with generic test data, etc.. This requires an entirely different kind of knowledge and experience than the tester who grew up with manual click-step-by-step tests.
The knowledge about testing is more spread and people understand that it is important and want to go for it. This is of course great and after striving for this for years, it feels good when things are finally happening. But “people” are often managers, project management, or similar, that primarily have been, and are, focused on development. This makes it easily to focus on building something smart, efficient and ­– that’s right, automated. But there are very few good testers to come by.
Having a tester’s way of thinking AND technical skills AND be able to automate … well, then the question might be if we are looking for developers more than testers. If you are lucky and find a developer who also has experience in testing, it is quite likely that you have found a developer who does not want to test but who was forced to the job. The same goes for the opposite. There are a lot of manual/non-technical testers who have been forced to work with more technical tests, because that was what the team had to do. The tester is not interested and feels uncomfortable when he or she lacks experience and knowledge. Where can I find the modern tester who can hold both types of knowledge and engagement? I know some but they are few!

 

It’s not just about fully automated tests. I have had the privilege to work in teams who had a type of test-developer. That person made a lot of scripting, generating test data, developing small tools to support testing or test case management and scripts to semi-automate testing.
Is it courses and training that is missing? Is it still the test profession’s reputation as a low status job, which prevents developers from wanting to be a tester? Is it our old-fashioned way of thinking of how the test should be conducted, that makes it difficult to venture into the future on a broad front? Or is it that technical testing and automation is the wrong way to go?

I don’t know … I just know that it’s very hard to get really skilled testers who also possesses deep technical knowledge and experience in test automation.

 

Usually the screens on the train shows news and the weather. Today we got some strange news, 9 years old news!

 

A while ago I saw a video with Jan Jaap Cannegieter where he said something like “it doesn’t have to be a Tester, it could be a developer with passion for test”. After that I started to think about what is the different? When do you become a developer with passion for test instead of a tester and vice versa.

• Is it depending on your title and tasks at work?

• Is it about your ability/knowledge about programming?

• If you are a really good programmer but also a really good tester. What are you then?

• Is it only about where your heart lies?

• If you are a developer that works a lot with testing, perhaps often more than developing, are you then still called a developer? or are you then a tester, whether you want to or not?

 

One thing I know is that I am a tester and not a developer, maybe a tester with a passion for development!

 

 

 

This is my backlog:

This is my “Done Bowl” (just emptied):

Sometimes if feels quite unstructured with all these sticky notes but there are some really great benefits

1. It’s easy to sort them into different groups and priority.

2. The really really urgent tasks I can put on my screen so I won’t miss it.

3. I don’t need my computer to write new tasks.

4. When I leave them on my desk I really do leave them so they won’t stress me.

5. They are really visual and the amount of tasks is really obvious as well.

6. Thanks to the “Done bowl” even the tasks that are done are really visible so they make me feel good and happy.

7. They make my life more colorful.

 

Don’t underestimate the power of sticky notes!

 

 

Software testing is not only about verifying functions. A system is so much more than functions, if it’s suppose to be useful anyway. The system has to live in a context and in a world with many different users with many different needs and behaviors.

 

There are many stakeholders, not only the end users. All these stakeholders have different requirements and wishes for what the system should handle or support. It can be deliverables or qualities that must be in the system to make it possible for these people to do their job. For example, if an operating technician should be able to maintain a good level of security there must be routines and support for back up. Taking new back up and restoring an old back up just have to work. These things are crucial for an operating technician while the market department hasn’t dedicated this even the slightest thought. The market department on the other hand, might be anxious about the coolness of what is delivered in the next version, so they can beat the market by surprise in their next press release. Yes, there are many different stakeholders and many different wishes.

Example of stakeholders and their requirements/wishes

 

How do we do to cover more than just functions?

 

First of all I think we have to look beyond requirements specifications, meet all the different stakeholders and discuss what’s important for them. There is a lot of support in lists of test types like performance test, load test, usability test, installation test etc. Collect a list of your own, with suitable test types and go through them for every project to determine which applies for the test object. My list, of absolutely necessary test types that should be included in almost every project, is:

 

-          Functional test

-          Usability test

-          Platform test

-          Security test

-          Access level testing

-          Performance test

-          Stress test

-          Stability test

-          Installation test

-          Recovery test

-          Storage test

-          Volume test

 

There might be other types of test that can be applicable depending on the project and system under test. There might also be the opposite, several of the test types mentioned above might not be applicable or require very little effort. Often this is related to time and money but also what is valuable to test. But, just by discussing an area you have lifted up the different qualities and then the project can make a decision whether the area should be tested or not.

 

When you come to the point where you have identified all stakeholders and their requirements and whishes, you probably have an enormous amount of work before you. To cover all is impossible and most likely not meaningful. Therefore, your next task, possibly even more difficult, is to prioritize. Whose requirements are the most important? We can look deeply into this but the thing I think is the most significant is to make sure you communicate a lot with all stakeholders.

 

Someone whose list is very long and full of details is Rikard Edgren. The last time I listened to him his list was 77 test ideas long. See http://thetesteye.com.

 

What does your list look like?

What smart test types do you have in mind and which to you think is absolutely necessary for most systems/projects?

 

 

Once upon a time there was this company who struggled with a poorly built system. Every day the users got more and more annoyed on the bad interface. One day enough was enough and the kind boss asked a prominent consultant to build them a new system. The developer consultant arrived with his shiny laptop and started working. It was not long before the consultant started to struggle. The environment and business logic turned out to be a lot more complex than he predicted and none from the company side could give him any straight answers on how things should work. After many weeks of swift coding the system was ready for the users to test. And once again the consultant was faced with the users’ unavailability.

It was time to call in a tester. The humble tester arrives and is quickly introduced to the system and the users. Only two weeks left until deployment and no tests run, the tester decided to use the users as testers. One by one the users were booked for an hour each. The tester arrives to the meeting with a few bullets that needs to be tested and tell the users to use real data when testing. The users slowly start to familiarize themselves with the system and a lot of feedback is recorded by the tester.

The developer is happy to finally receive feedback.

The kind boss is happy the quality will be improved.

The users are happy that their opinions are raised.

The tester is happy that the system has been tested, the users introduced to the system through coaching and that everyone now seems comforted that this new system will save their day.

I doubt 200 test cases could have done a better job in this case.

 

 

Do you need something fun for your test conference or need to cheer up your test team?

 

We are looking for new oppertunities to perform the “The Supertester – A Slightly True Story”.

 

The Supertester – A slightly true story” is a theater performance where we try to mix serious test issues and common prejudices with laughs. We will take the roles and personalities of project managers, testers and super heroes to a new level. We will use prejudice and radical opinions in our effort to entertain and inspire.

You will meet the project manager who blame the test department for everything that goes wrong. The project manager loves all the new and cool development methods. He is not making it easy for the poor shy little company tester who lost the last shred of self confidence the first day at work. Can she be saved by the magnificent Supertester and once again be restored to a strong tester who stand up for her profession? Will the powers of the Supertester be enough to save the organisation?

 

Join us in a twisted but still slightly true world of testing.

 

You can read more about the show under The Show.

If you are interested don’t hesitate to contact us at linda.hoff (at) gmail.com or anna.o.hoff (at) gmail.com

 

 

 

”I have a dream where people look beyond waterfall steps and agile manifests.
I have a dream where scripted testers and exploratory testers are living together in harmony.
I have a dream where testers stand up proudly and talk about their job and progress.”

From “The Supertester – A Slightly True Story”

 

The last couple of years, I’ve experienced two sides within the test world being formed. One side that is for scripted testing and ISTQB and then there are those who are against ISTQB and completely into exploratory testing. On test conferences and seminars I’ve heard test guru insult “the other side”. Those who are for ISTQB are narrow-minded academics that spends way too much time on statistics and administration, which is a completely waste of time and of no use at all. What is a certification about anyway? Is it really a proof of a skilled tester? The others, the ones that are into exploratory testing, they are fuzzy, unstructured and count on gut feeling way too much and they just laugh science in the face. But, who is right?

I don’t know who’s right but I know what I think. No one! Those who’s right, according to me, are the ones not taking sides, but sees the advantages of both approaches, pick the cherries and uses a little from both sides.

Below we can see a fairly typical picture for describing scripted testing and exploratory testing. The picture shows a scale where one can do more or less of the two ways of testing.

At previous workplaces of mine and also as a consultant I have had the privilege to see different organizations and projects and it is only at one single occasion that I’ve seen pure scripted testing. There where one person who was responsible for designing and writing new test cases but also updating them. The test team consisted of a couple of testers which only task was to execute test cases and report bugs. We ran a complete function test every week. It’s the same with the other side of the scale. I only experiences 100% exploratory testing once. We worked according to Scrum and all new development was tested with exploratory testing. The regression tests, on the other hand, were a kind of scripted test (the test steps were not that limited, especially when it came to test data). At all other projects and organizations I’ve been to, testing have been a mix of techniques and approaches. The test cases are completed parallel with execution, some test cases are very detailed and others invite creativity to take full advantage of the testers’ experience. Do I then feel that the tests have been better or worse at some particular place? The organizations that succeeded to pick the best parts out of several methods and found a harmonious mix, are the ones I think have had the best quality, the best control over their work and orderliness. They also had the most satisfied test team, which was proud about their work and gladly embarked on new bug hunting adventures every day. The places I feel didn’t work well at all, are those where management had to nag about some particular method/process every week and they had to force the team to use this process or some fancy tool, but all these stuff didn’t fit the products worked on or the personality of the team members. These organizations spent a lot of energy discussing why/why not to use this particular method and this steals a lot of focus from the day to day work and no one gets any peace or inspiration to perform some really good work.

Another aspect that I have been thinking about is the testers’ personality. Not everyone is suitable as exploratory testers. I think most people agree that they don’t, but is it then right to limit the title “Tester” to solely creative and imaginative persons? No, I don’t think we should do that. We are all different and that’s something we should take as much advantage of as possible. There are people that love documentation, accuracy and that manages, well even enjoy, repetitive tasks and possesses the ability to keep the same focus and commitment the first time they run a test cases as the fiftieth time. Both personalities are worth a lot so why not use them both.

Just think of Thorkil Sonne and his “Testspecialisterne” (test consultants diagnosed with autism). Here it just doesn’t work with exploratory testing. So, don’t they do a great job then? Sure they do! In the right situation, with suitable tasks, they do hundred times better work than most of us should have done. They are needed. But everyone can’t be like them, then a lot of test scenarios would be missed, where exploratory testing might be the only way to find them. One part of testing is about predicting situations and then imagination and exploratory mindset is very valuable. Without that our tests would be limited to requirements and documentation and then we probably would miss serious bugs and issues.

No, both sides are needed! Stop fighting and do something really good together!

 

” All those easily charmed project managers who try all new cool methods and tend to forget everything they ever learnt in school or in previous projects. All scrum fan boys who read the agile manifest saying that no one ever more needs to document or follow a boring process! All administration nerds that documents for months and months in their  waterfall projects with all the dead fish floating down the waterfall with them and all those theoretical brain heads that memorized all the answers to the ISTQB-certification. And still no one has a clue.”

From “The Supertester – A Slightly True Story” by Linda & Anna Hoff

Enjoy!

http://www.eurostarconferences.com/content/supertesters.aspx