Sunday, January 24, 2021
Friday, July 15, 2016
Experience report of a Tester in (fr)agile world
The following video shows an experience report of a tester working in different scrum team by my friend Yaakov Morgenstein, a Tester and scrum master.
This short talk given during the latest Jerusalem SW testing meetup that Shmuel Gershon and I organize from time to time.
A nice taste from a lightning talks meetup.
Enjoy!
Friday, June 17, 2016
White elephants
According to the legend, the kings of Siam would gift a white elephant to courtiers who had rendered themselves obnoxious, in order to ruin the recipient by the cost of its maintenance. Based on that, the term “white elephant” is used to describe a possession which its owner cannot dispose of and whose cost, particularly that of maintenance, is out of proportion to its usefulness.
It’s very easy to spot a real white elephant. They are rare and stand out from the background. However, Testing projects tend to attract lots of white elephants which are less easy to identify.
I would like to share with you my observation of the white elephants life cycle in testing projects.
Birth
The birth of white elephants is usually a result of a match between cool ideas and resources. A future parent with a cool idea pitches it to the herd, which are carried away by it. The cooler the idea is, the bigger the resources allocated to create and maintain it. Many times the declared goal of the cub creation is to save efforts in the long term. Different ideas takes different forms. Common examples are tools that automate manual tasks, enforce a better process, save testers from writing code, etc.
Childhood
The early days of the white elephant are exciting. Brainstorming happens, designs are created and an
artist is invited. He is freed from his usual less cool tasks, to give life and nurture the young elephant. New technologies and approaches are learnt. A lot of fun happens all around.
artist is invited. He is freed from his usual less cool tasks, to give life and nurture the young elephant. New technologies and approaches are learnt. A lot of fun happens all around.
On the elephant’s first birthday, the POC is ready and a demo is presented to the herd. The herd is
excited. The elephant shows a great potential. The proud parents beam and relatives from distant places come to observe the miracle taking his first steps.
Youth
The elephant’s caretakers keep raising the cub. They take him outside so he can start living independently and begin contributing to the community. Often, the community members find out that the young beast is not stable enough. The community is very happy to receive him and show a high tolerance of his childish behavior in the early stages. This tolerance may be challenged by the elephant’s habit of requiring too much attention and performing incomplete work.
Adulthood
While the white elephant allegedly reached adulthood it still does not act like an adult. On the one hand, the people enjoy his company and the cool atmosphere that he spreads around him and they are still hoping he fulfills his potential. On the other hand, each time they try to use him, they are disappointed. Often they continue using the ordinary work elephants and return the white one to his caretaker who still spends too much time nurturing him. On the outside, the picture looks much more ideal. Every guest is taken to see a demo of the white elephant working. His caretaker takes him to conferences within and outside the organization. Prestige is acquired as well as prizes. People still hope the elephant will fulfill the purpose of its creation, despite the growing concerns.
Death
Most of the white elephants dies naturally when the community gets rid of them. The community reaches a point where they lose hope of benefiting from the contribution they had imagined and understand that they do not gain real benefits from the white elephant. Getting rid of the beast is very painful to the people, but they understand that there is no choice. Many times, in order to ease the pain, the birth of elephant 2.0 is declared. The parents plan to make him a better elephant that won’t turn out to be a white one. A soon as the white elephant is gotten rid of, the pain is reduced.
In very rare cases, instead of killing the white elephant, the people are able to transform it. In most cases such attempts only consume more resources and delay the inevitable.
Spotting White elephants
No modern white elephant is designed to be one. The guy who can help us spot a white elephant, so we can kill him in early stages, is called Roi, or in his full long boring name: Return On Investment calculation.
When the community forgets to invite Roi to the party, no one is there to set real expectations of the elephant. The most romantic time, the birth of the cool idea, is also the time to make the non-emotional calculations of ROI. It is the time to think about the possible lowlights the new approach is based on.
I suggest that as much as our idea is cool and looks good, so should our ROI calculation be ruthless, to make up for our strong bias in support of our idea.
Doing the ROI calculation once and forgeting about it, is a common mistake. The unexpected happens and turns our elephant into a white one. As more resources are consumed working on the idea, we are more biased towards it and we start to “throw good money after bad”. We tend to take into account the sunk costs. which is not a best decision making practice.
Monitoring the ROI, and continuously comparing our expectation to the reality, can help us identify our white elephant early, so we will be able to get rid of him before he consumes too much of our resources.
Can you spot the white elephant near you?
Tuesday, May 13, 2014
Multi-purpose Testing Tool Kit
I would like to reveal my secret toolkit which
I use to tackle any test challenge.
These three thinking tools are shaped to work on every Testing task from small to big. Sometimes, using these tools makes the difference between the ordinary and the expert testers’ way of thinking.
What is in my kit?
I keep the following three thought tools at
hand:
·
Definition of Quality
·
Risk Formula
·
Awareness to ROI
Definition of Quality
Every action that we do as testers should have
connection to the quality of the product. We
To decide whether an activity is connected to
the quality of the product - we must first align ourselves on “what is quality”.
My preferred definition for quality is “value
to someone” (Jerry Weinberg). Cem Kaner adds the extension “who matters”, which
is useful for focusing on the needs of the person that we are testing for. The
benefit of such a basic definition is that it can be used to criticize less
basic baselines of quality, like “adhering to the official requirements” or quantitative
measurements which are used in the organization process.
The Risk Calculation Formula
Our testing is an activity to mitigate risk.
We test in order to provide information about the quality of the product, so
that defects with the higher risk to the product quality will be fixed.
A basic formula of calculating a risk is:
Using this formula helps us to compare between
the different risks that we want to address. We will try to focus our work first
on areas which have the highest probability of having issues that will have the
highest impact on quality.
[Late note: My coulege, Amit Wertheimer point out in the hebrew testing forum of Tapuz that it is important to remember that the caculation is based on our limited analisys and not on accurate certain factors ]
[Late note: My coulege, Amit Wertheimer point out in the hebrew testing forum of Tapuz that it is important to remember that the caculation is based on our limited analisys and not on accurate certain factors ]
Awareness of ROI
Testing is an economic activity. Testing consumes
resources and is done in order to supply the one who send us to test with
information that he values. Whether it is issues that he chooses to fix or risk
assessment that will allow him to determine the shipping time of the product.
ROI - return on investment, means that
something we do is “worth it” and does not cost us more than what we will get
from it. Of course, since Testing is an activity of learning and gathering
information, we can’t know ahead of time whether we will learn something that is
“worth it”. Here, performing Risk analysis will help us to decide if a testing
activity has a good probability of resulting in positive ROI or not.
Examples
Let’s look at some examples for using the
tools:
A tester gets a new version of the SW and has
to decide what to test first. The risk formula will guide him by viewing
the risk factors: Looking at the Probability of possible
failures, he will determine the areas which have higher probability of having
defects - areas that been changed. Areas that tend to be buggy and so on... On
the other hand he will look at the impact of possible issues that he can
test for: what flows are the most critical for the product users, what type of
defects are more costly to the firm. The first areas that he will pick to test
will be the ones that will calculate the highest risk - probability
to fail, multiplied by the impact the quality. These areas will
have the most ROI as he assesses them as the most valuable.
This example can be true also when planning a
test strategy for a whole product for a whole team.
Discussing Whether to fix a Defect or not
If you consider quality, risk and ROI, you
understand that fixing a bug is not always the right choice.
A smart tester will use the risk formula
and will guide the discussion to consider the impact on the quality,
the probability that a user will be effected by the defect on the one
hand, and on the other hand - the probability that fixing the defect
will cause other defects. This way we can estimate the total cost of the
fix in terms of risk and decide whether the defect fix has positive ROI.
Sometimes, a proposal to fix a defect is
actually a choice between different quality attributes. For example,
making the product more secure may reduce the performance. Examining the impact
on the “someone who matters”, usually our user, will help us make a
choice.
ROI is a key thinking tool to examine investments in infrastructure,
tools and automation. A realistic estimation of how much the tool will cost -
in terms of development, integration, maintenance and fees versus the benefits we
expect to gain, such as time saving, additional test coverage, will help us decide
whether to “go for it” or not.
Some Critique of the Above
Essentially, all models are wrong, but some
are useful - said George E. P. Box. This is true for the thought models that I
just described. Since Testing is a learning activity, and learning is unpredictable,
sometimes wandering around may be the right thing to do. Low risk and low ROI
activity that we barely connect to the product quality in the first place, may
lead us to critical information that has high ROI and high risk which is
connected to the product quality.
What is your secret
tool? Do you use a different version of similar tools? Let me know.
Sunday, January 12, 2014
Professional meeting? JeST DO IT
This post is about my experience of organizing Tester’s meeting. It is true as well for any other meeting, be it Software Developers, Social workers or a meeting of Puppeteers.
The 3rd meeting of JeST –
the Jerusalem workshop on SW Testing is just over. Such a meeting is a great
opportunity to meet fellow testers, and share experiences and ideas.
I am the proud founder of JeST, together with Shmuel Gershon, Asher
Samuels and Alon Waisbard (look in Twitter for @sgershon, @absamuels and @awaisbard
). I am telling you that because less than a year ago, organizing such a
meeting, looked to me as a distant vision, something that will happen in the
far future when I will develop the complex skills necessary to organize such an
event.
But it doesn’t.
A moment during JeST2 |
Organizing a meeting is so easy, that I must share how easy it is, with
you. The main issue to overcome is the mental block and just do it. To
help you overcome this block, I prepared a list of the things that you don’t
need, in order to hold such a meeting:
1)
A budget
You will be able to find a place to host a
short meeting. You don’t need to rent a conference hall. It can be the room in
the company of one of the meeting participants, or one’s home living room.
Refreshments can be brought by the host or by the participants. If it works for you and the potential
participants, you can schedule a meeting in a quiet restaurant and address the
need for both the location and refreshments.
2)
Sponsors
Since you don’t need a heavy budget, you
don’t need an organization to support your meeting. Although I welcome
cooperation and networking, which naturally happens in such meetings, I prefer
to remain independent and stay away from organizations that try to use the
meeting for their own promotion.
A thank you note should be enough as a return
of the hosts good will.
3)
Heavy logistics
You don’t need a full-time administrator to
organize such a meeting. Split the tasks between a few friends and it will not
consume too much time. Use social networks to spread the word. Meetup service can simplify the process even
more, in return for a payment of a few bucks.
(Late note: there are many Meetup groups which are allready dedicated to promote events like yours. See for example: ILTechTalks group. All you need is to suggest your event in such group, free of charge)
(Late note: there are many Meetup groups which are allready dedicated to promote events like yours. See for example: ILTechTalks group. All you need is to suggest your event in such group, free of charge)
4)
Professional presenters and
presentations
Not everyone who has interesting things to
say is a professional presenter and the length of a
presentation does not make
it better (most times it’s the opposite). For our meetings, we chose a format
of lightning talks
followed by a discussion. One time, we
had a “regular” presentation, which was great too, but the most significant
part for most of the participants was the discussions that we held.
JeST3 agenda |
5)
A large number of participants
Rating is overrated. While it is nice to see
many people come to “your” meeting, a large audience has its drawbacks too – the
atmosphere is less intimate, it’s harder to facilitate the discussion, and not
every participant feels free to express his opinion. When I started to organize
the first JeST meeting, I was worried that the meeting would not draw a “respectful” number of participants. I did the “Worst case scenario” drill and came
to conclusion that if only me and the other 3 folks that shared the idea of organizing
the meeting would come, it would not be a waste of time. At the end the meeting turned out to be a
success, also by the number of participants.
All the above are not a “must have” in order
to organize your testers meeting. All you need is Testers and a love of testing.
Friday, November 8, 2013
A small cool macro that makes Mind Maps and Spreadsheets better friends
In
case, like me, you belong to the Mind Maps lovers’ group, there’s good chance
this tool will interest you.
I will not keep secrets from you. I wrote an Excel VBA Macros that provides the wish list above. Feel free to use it, just copy it into your VBA editor.
Note: while this tool is great for porting the Mind Map data into Excel, once you’ve used it, the data won’t be easily exported back to MindMap. If you find it necessary, you can create a VBA macro that will help to do that.
Sub moveAndIndent()
Sub GroupIt(Optional LastRow)
I like
Mind Maps because they are easy to create and evolve. They represent data in a
way that makes sense to human beings.
When
you want to add a leaf to any branch of the data structure, you don’t need to mess
around, as it is just a natural development of the idea representation.
On
the other hand, Spreadsheets have their own advantages. They are able to perform
calculations on the data, and sort and filter it.
I
like to combine both Mind Maps and spreadsheets in my work. I summarize ideas
in a Mind Map and move it to an Excel™ spreadsheet in order to use it in a way
that involves calculations and filtering.
I
use Xmind for creating Mind Maps. Moving the data from the Xmind application to
an Excel sheet is very easy: copy-paste the central subject into the sheet.
However, the data format in the target sheet is not very useful for my goal –
each hierarchy is placed in a new column, as you can see in picture #1.
I would be happier if I were able to have all the
data in same column, indented by the hierarchy, as
you can see in picture #2.
It would be even cooler if we were able to use
Excel’s “grouping” feature so we could see the exact hierarchic level of the
data, see picture#3.
Pic#1 |
Note: while this tool is great for porting the Mind Map data into Excel, once you’ve used it, the data won’t be easily exported back to MindMap. If you find it necessary, you can create a VBA macro that will help to do that.
Pic#3 |
Pic#2 |
I created three macros: one that moves the data to one column and indents it according the hierarchy. The other groups the data according to the indentation and the third one which calls both macros.
Before that you run the macro, make sure that the cells that conatins the data are in the "selection" as you can see in the following video:
If you have any questions, Tweet me: @testermindset
Enjoy!
The VBA code:
Before that you run the macro, make sure that the cells that conatins the data are in the "selection" as you can see in the following video:
If you have any questions, Tweet me: @testermindset
Enjoy!
The VBA code:
Sub moveIndentGroup()
' This macro call the 2 other Macros in order to perform all actions usingcommand
LastRow = Selection.Cells.Rows.Count
Call moveAndIndent
Call GroupIt(LastRow)
End Sub
Sub moveAndIndent()
Dim rCell As Range
Dim rRng As Range
Set rRng = Selection
For Each rCell In rRng.Cells
If ((rCell <> "" Or rCell <> 0)) Then
Cells(rCell.Row, 1).Value = rCell.Value
Cells(rCell.Row, 1).IndentLevel = rCell.Column - 1
If rCell.Column > 1 Then rCell.Value = ""
End If
Cells(rCell.Row, 1).HorizontalAlignment = xlLeft
Next rCell
End Sub
Sub GroupIt(Optional LastRow)
If IsMissing(LastRow) Then LastRow = Selection.Cells.Rows.Count
For j = 1 To 5
For i = 1 To LastRow
If Cells(i, 1).IndentLevel = j Then
FirstCell = i
LastCell = i
While (Cells(FirstCell, 1).IndentLevel <= Cells(LastCell + 1, 1).IndentLevel) And LastCell <= LastRow
LastCell = LastCell + 1
Wend
Range(Cells(FirstCell, 1), Cells(LastCell, 1)).Select
Selection.Rows.Group
i = LastCell + 1
End If
Next i
Next j
End Sub
Location:
ירושלים, ישראל
Sunday, November 3, 2013
Dealing with Stress take 2
In
case that you are reading this post, there is a chance that you read my blog post about Stress.
This
post was transformed twice: first, I turned it into an article in Tea Time with testers magazine, than it became a presentation in QA&Test 2013 conference
at Bilbao, Spain.
The
conference was awesome. Hospitality was great. I had an opportunity to get to
meet and share ideas
with many cool testers from all over the world. I was also
able to experience presenting at an International conference.
Besides
the format change – from a blog post to an article and then into a presentation,
the ideas themselves emerged and developed as I got a lot of feedbacks while
working on the material.
The
main idea behind the work is the need to connect our Stress tests to the user’s
needs . To do that, I suggest categorizing our Stress tests and failures into 3
main categories: Multiple experiments, Stability over time and Load.
While
working on the presentation I added a few aspects which are connected to the failure
classification and Stress test planning:
·
Assessment of risk when
selecting risky flows for multiple experiments.
·
Taking in account the impact
of the product on the system stability.
·
The need to find good
oracles beyond the official requirements when defining the load and stability
targets.
·
Perform load tests of few
types :
o
The largest amount of data
or actions which has meaning for the
users
o
The full capacity of the product – in order to
spot degradation in the capacity before they has impact on the users.
·
Use good logging mechanism
to gather data on all the experiments that your stress performs.
·
Monitor the system
resources in order to quickly find stability issues .
Since
I was scheduled to present on the last day of the conference, I had some time to
get inspiration from a few people that I met during the 1st two days.
The night before the presentation, I changed the summary slide from a list into
a mind map that summarizes my takes on the subject.
I
am publishing the mind map and would like to ask you to review it and contribute
to my initial work on that. I promise to give you credit if you’ll provide
meaningful input.
Labels:
Confrence,
Mind map,
Presentation,
QA&Test,
Stress
Location:
ירושלים, ישראל
Subscribe to:
Posts (Atom)
First column subject is about the definition of quality.
Link to the full magazine issue: https://bit.ly/39Wm3BV