This post was transformed twice: first, I turned it into an article in Tea Time with testers magazine, than it became a presentation in QA&Test 2013 conference at Bilbao, Spain.
The conference was awesome. Hospitality was great. I had an opportunity to get to meet and share ideaswith many cool testers from all over the world. I was also able to experience presenting at an International conference.
The main idea behind the work is the need to connect our Stress tests to the user’s needs . To do that, I suggest categorizing our Stress tests and failures into 3 main categories: Multiple experiments, Stability over time and Load.
While working on the presentation I added a few aspects which are connected to the failure classification and Stress test planning:
· Assessment of risk when selecting risky flows for multiple experiments.
· Taking in account the impact of the product on the system stability.
· The need to find good oracles beyond the official requirements when defining the load and stability targets.
· Perform load tests of few types :
o The largest amount of data or actions which has meaning for the users
o The full capacity of the product – in order to spot degradation in the capacity before they has impact on the users.
· Use good logging mechanism to gather data on all the experiments that your stress performs.
· Monitor the system resources in order to quickly find stability issues .
Since I was scheduled to present on the last day of the conference, I had some time to get inspiration from a few people that I met during the 1st two days. The night before the presentation, I changed the summary slide from a list into a mind map that summarizes my takes on the subject.
I am publishing the mind map and would like to ask you to review it and contribute to my initial work on that. I promise to give you credit if you’ll provide meaningful input.