Read the requirements (again)!

This week I helped my 7 years old Son, Harel, in a school project – building a model of Noah’s Ark. Since I liked it to be educational and to teach him a bit about approaching a project, I asked him to list all the Ark's requirements directly from the source. We opened the Bible and read (I am quoting here the original Hebrew text and the American standard version, Genesis 6,13):

עֲשֵׂה לְךָ תֵּבַת עֲצֵי-גֹפֶר, קִנִּים תַּעֲשֶׂה אֶת-הַתֵּבָה; וְכָפַרְתָּ אֹתָהּ מִבַּיִת וּמִחוּץ, בַּכֹּפֶר. טו וְזֶה, אֲשֶׁר תַּעֲשֶׂה אֹתָהּ: שְׁלֹשׁ מֵאוֹת אַמָּה, אֹרֶךְ הַתֵּבָה, חֲמִשִּׁים אַמָּה רָחְבָּהּ, וּשְׁלֹשִׁים אַמָּה קוֹמָתָהּ. טז צֹהַר תַּעֲשֶׂה לַתֵּבָה, וְאֶל-אַמָּה תְּכַלֶּנָּה מִלְמַעְלָה, וּפֶתַח הַתֵּבָה, בְּצִדָּהּ תָּשִׂים; תַּחְתִּיִּם שְׁנִיִּם וּשְׁלִשִׁים, תַּעֲשֶׂהָ. יז וַאֲנִי, הִנְנִי מֵבִיא אֶת-הַמַּבּוּל מַיִם עַל-הָאָרֶץ, לְשַׁחֵת כָּל-בָּשָׂר אֲשֶׁר-בּוֹ רוּחַ חַיִּים, מִתַּחַת הַשָּׁמָיִם: כֹּל אֲשֶׁר-בָּאָרֶץ, יִגְוָע. יח וַהֲקִמֹתִי אֶת-בְּרִיתִי, אִתָּךְ; וּבָאתָ, אֶל-הַתֵּבָה--אַתָּה, וּבָנֶיךָ וְאִשְׁתְּךָ וּנְשֵׁי-בָנֶיךָ אִתָּךְ. יט וּמִכָּל-הָחַי מִכָּל-בָּשָׂר שְׁנַיִם מִכֹּל, תָּבִיא אֶל-הַתֵּבָה--לְהַחֲיֹת אִתָּךְ: זָכָר וּנְקֵבָה, יִהְיוּ. כ מֵהָעוֹף לְמִינֵהוּ, וּמִן-הַבְּהֵמָה לְמִינָהּ, מִכֹּל רֶמֶשׂ הָאֲדָמָה, לְמִינֵהוּ--שְׁנַיִם מִכֹּל יָבֹאוּ אֵלֶיךָ, לְהַחֲיוֹת. כא וְאַתָּה קַח-לְךָ, מִכָּל-מַאֲכָל אֲשֶׁר יֵאָכֵל, וְאָסַפְתָּ, אֵלֶיךָ; וְהָיָה לְךָ וְלָהֶם, לְאָכְלָה. כב וַיַּעַשׂ, נֹחַ: כְּכֹל אֲשֶׁר צִוָּה אֹתוֹ, אֱלֹהִים--כֵּן עָשָׂה. וַיֹּאמֶר יְהוָה לְנֹחַ, בֹּא-אַתָּה וְכָל-בֵּיתְךָ אֶל-הַתֵּבָה: כִּי-אֹתְךָ רָאִיתִי צַדִּיק לְפָנַי, בַּדּוֹר הַזֶּה. ב מִכֹּל הַבְּהֵמָה הַטְּהוֹרָה, תִּקַּח-לְךָ שִׁבְעָה שִׁבְעָה--אִישׁ וְאִשְׁתּוֹ; וּמִן-הַבְּהֵמָה אֲשֶׁר לֹא טְהֹרָה הִוא, שְׁנַיִם--אִישׁ וְאִשְׁתּוֹ. ג גַּם מֵעוֹף הַשָּׁמַיִם שִׁבְעָה שִׁבְעָה, זָכָר וּנְקֵבָה, לְחַיּוֹת זֶרַע, עַל-פְּנֵי כָל-הָאָרֶץ

Make thee an ark of gopher wood; rooms shalt thou make in the ark, and shalt pitch it within and without with pitch. And this is how thou shalt make it: the length of the ark three hundred cubits, the breadth of it fifty cubits, and the height of it thirty cubits. light shalt thou make to the ark, and to a cubit shalt thou finish it upward; and the door of the ark shalt thou set in the side thereof; with lower, second, and third stories shalt thou make it. And I, behold, I do bring the flood of waters upon this earth, to destroy all flesh, wherein is the breath of life, from under heaven; everything that is in the earth shall die. But I will establish my covenant with thee; and thou shalt come into the ark, thou, and thy sons, and thy wife, and thy sons wives with thee. And of every living thing of all flesh, two of every sort shalt thou bring into the ark, to keep them alive with thee; they shall be male and female. Of the birds after their kind, and of the cattle after their kind, of every creeping thing of the ground after its kind, two of every sort shall come unto thee, to keep them alive. And take thou unto thee of all food that is eaten, and gather it to thee; and it shall be for food for thee, and for them. Thus did Noah; according to all that God commanded him, so did he. And Jehovah said unto Noah, Come thou and all thy house into the ark; for thee have I seen righteous before me in this generation. Of every clean beast thou shalt take to thee seven and seven, the male and his female; and of the beasts that are not clean two, the male and his female."

So Harel took a pencil and listed the requirements of his model in his own words:

1. It should be made pales from a certain wood type

2. The ark should be sealed (with pitch)

3. Size in cubits will be 300(Length) X 50(width) X 30(height)

4. A window should be on the upper floor in size of one cubit

5. A door should be on the side of the Ark

6. The Ark should have 3 floors

7. In addition: The Ark will hold Noah’s family and the beasts: a pair of each kind, but seven of each “clean” kind.

When he was done listing, we noticed a few things which are not usually shown in the typical drawings or models of this favorite subject: I never saw anyone painting the Ark in black, however, it’s very likely that after Noah pitched the Ark from outside and inside he didn’t paint it brown and draw wood parts on it, like is shown in most pictures. No one also painted seven sheep or giraffes although these animals are from the “clean” type according to the tradition and Noah might have brought seven of them. We also noticed that the triangle roof and the ramp are optional components. Then we went to work – fulfilling the requirements without the bias we had from other sources, and learned a lesson about getting the requirements from the original source.

Harel demonstrates great requirements reading skills. I am not sure what is the chance that my son will be a tester in his future (currently he wants to have a career as a gaming SW tester, which in his eyes is the perfect profession), but for me this experience is an additional reminder of the danger of executing a test from a test plan without reading again the requirements.

The traditional test execution process (unlike ET), is built as a linear line of of turning the requirments and SW design into test cases and then executing them, but our mind is not linear. The ideas and understanding of the product evolve with time and experience, and our original perspective changes - for better, since it's more mature with time, and for worse, since it's likly to disconnect from the original sources of knowledge. Returning to the requirements at each stage of the execution will reconnect us to our mission definition and will give us an opportunity to relearn the material when our mind is more ready.

My recommendation to everyone who opens a bug or executes a test is always: Read the requirements (again)!

The Ark model.

We didn’t paint it black (we thought brown is nicer). We placed two

sheep and not seven (resources limitation), but we did it consciously.

Observations, Art, practicing and inspiration from other blogs

Practicing in the art of description.

Recently, I have read few blogs that introduce the idea of practicing in describing art in order to improve the skill of description.

Describing what we see is essential part when we test complicated systems. Beyond being accurate, objective and concise, you need to take the system generalist role - be able to conclude.

Parimala Shankaraiah talks in her refreshing blog "curios tester" about painting the picture - she describes how she addressed the challenge in Marlena Compton blog post “Visionary Testing” . I am quoting Marlena:

I challenge you to find a work of art be it a painting, sculpture, installation or anything you deem “art-worthy” and study it. This can be in a museum, a coffee house or your mom’s living room. Once you feel you have an understanding of what you are looking at, try to communicate your understanding with words.

Not only that I decided, like Parimala to address it and post my notes in my blog, but I also told Parimala that I want to do that. She replied to me that I’ll let her know how I did this exercise, so now I am obligated to try this :-). Hopefully you will see that in few lines.

Another interesting thing was that Parimala wrote about different perceptions of different people, and that Interpretation requires more knowledge about the context.
We could see a deomonstration of of that if we will look at the example that Marlena gave in her post. she shows a picture named “Esther before Ahasuerus”. She describes Esther (the women in the picture) as “another chick in a dress who was fairly bitter about life in general”.

I guess that when Parimala(She is an Indian women) read that post, she might agree or not, but when I did, that description looked so out of the context. I also saw a women and a king, but reading the picture name rang all of my bells. As an Israeli and Jewish, I have heard the story behind the scene so many times, so for me Esther is not a “bitter chick” but a great women in a middle of a brave act that saved our people from destruction. Eventually, that gave us a great holiday - Purim, that we celebrate every year, having lot of fun.

As for the exercise of picking and describing a picture?

I picked the “The Raft of the Medusa” (1819) by Theodore Gericault. I saw the picture around 13-14 years ago, when I visited the museum Deorsey in Paris. I remember that I was very impressed, but don’t recall any specific observations when I saw it. Forgive me if it will be too depressing, but this is a really impressive piece of art. Let's go to my lab notes:








What I saw?

A raft full of survivors from a ship sink.
Survivors pack fills the raft with desperate and terrified people. exception for that is the black man that waves towards the horizon
the view behind is depressing dark skies with clouds that promise a storm that could sink the raft

What I inferred?
Depression. The end is coming. The black waving man looks like an exception that should be examined carefully.


What I saw?
On the lower left side, fainted, almost dead bodies. Near them a desperate man sitting hopeless perhaps mourning for them
On the upper right side, near the mast, few terrified people, probably seeing their death in their eyes. Going right – we could see the black man waving. His look is hidden, but we could tell that he is looking forward something (rescue? Freedom?). beneath him another man, bent is waving to the same direction few other figure point to that direction too

What I inferred?
The situation puts the different people in different states: some are lost their consciousness, some are mourning, some are terrified and some are hoping for rescue

What I saw?
Examining the different figures one by one, There is a gradual progress from one side of the people that lies down to the black man which stands up and waving. Some of the figures shows both the “hope” and the “terrified” situation

What I inferred?
A fragile situation, that moves like a raft, between desperation and hope.

Create your tester portfolio

The following presentation that I have created with my friend Shmuel Gershon, suggest ways for novice testers to gain experience by doing open source and crowd source testing and. We did not invent the wheel here, but we discuss and give some guiding that could be useful for novice testers.
This presentation was a result of a request that we got from few Tech career SQA course graduates, for some training that will increase their value in the eyes of potential employers. they suggested that we will teach them the ITCQB certification exam. however, we thought that we could do something more meaningful by giving them this workshop.

link to our work:

Create Your Tester Portfolio

View more presentations from sgershon.

and as a shared document in scribd

The Beauty of technique and the bias Beast

Recently, I see around me more emphasis on test design techniques. People get trained in common techniques, and I hear voices that call for more documentation of the techniques used in test design documents.
I would like to have a closer look on the trend by analyzing a simple but extreme example to demonstrate the hidden pitfalls that could be found in this path.

Let's take a look at the lovely pair of equivalent partitioning and Boundary values analysis. Both are usually the first most basic techniques to be mentioned when testing design techniques are discussed.

Equivalent partitioning method is derived from the fact that testing all possible values is not possible, and that one representative value from each partition is sufficient in order to have good enough testing.
Boundary values analysis come to compensate the weakness of Equivalent partitioning. Although that the specification does not specify a different behavior for processing boundary values, we do know that this areas are likely to fail due to architecture or programming bugs.

Let's apply the techniques for deriving data for our test design. Assume that we have to test the following function:

MemoryCopy(Source,Length,Destination)

The requirments specification says:

This function is used by software other components to copy memory cells, given the source memory cell number, the length of data to copy and the destination cell to copy the cells group to.

The environment of operation of the function is a memory area consists of 0 up to N memory cells that could contain any data.

Using the equivalent partitioning approach, we will analyze our environment: We have 3 data partitions: below Zero, Zero up to N and N up to infinite. Each one of our parameters could get one of them. Using pair wise planning will lower the amount of tested combinations so to get good enough coverage for the function will run the following test cases and call it TDD#1:

Case#	Source	Length	Destination
1	0-N	0-N	0-N
2	N+1-infinite	0-N	0-N
3	0-N	Negative	0-N
4	0-N	N+1-infinite	0-N
5	0-N	0-N	Negative
6	0-N	0-N	N+1-infinite
7	Negative	0-N	0-N

Note, that in this case, only one out the seven cases is a positive test case (Case#1), since the specification defines only one valid partition.

Since we already know that we must not use this technique without the complementary boundary values, we will specify the boundary values that we must check: instead of using 0-N we will use 0, 1,N-1 and N which are the boundary values within the valid partition range, -1 and N+1 as the boundary values of the invalid partitions. Thanks to all pairs algorithm, we end up with the following 44 cases(The negative ones are marked with ~) that we will call TDD#2:

Case#	Source	Length	Destination		Case#	Source	Length	Destination
1	N-1	1	0		12	1	N-1	N-1
2	N	N	N-1		13	N-1	0	N-1
3	N	0	N		14	1	0	0
4	0	1	N-1		15	0	0	1
5	N	N-1	0		16	0	N	0
6	N-1	N-1	1		17	1	N	1
7	0	N-1	N		18	~-1	N	0
8	1	1	1		19	N	N	~-1
9	N-1	N	N		20	N-1	~-1	N-1
10	N	1	1		21	~-1	0	1
11	1	1	N		22	0	N	~N+1
Case#	Source	Length	Destination		Case#	Source	Length	Destination
23	N	~-1	1		34	~-1	N	N
24	0	~-1	0		35	N-1	0	~-1
25	1	~-1	N		36	~N+1	0	N
26	~N+1	N-1	N-1		37	0	N-1	~-1
27	N-1	~N+1	N		38	N	1	~-1
28	~N+1	1	0		39	1	1	~-1
29	0	~N+1	1		40	~N+1	N-1	1
30	~-1	1	N-1		41	~N+1	N	0
31	N-1	1	~N+1		42	1	~N+1	0
32	N	~N+1	N-1		43	N	0	~N+1
33	1	N-1	~N+1		44	~-1	N-1	N

So far so good, but we just been trapped…

While in terms of valid, invalid data ranges and their boundaries we did state of the art coverage, we forgot the main functional attributes of our function while it copies data from source to destination, and has to maintain data integrity.

In order to look up what we missed, let's go back to our tester natural mind set and do analysis without bounding ourselves to technique.

Getting the mention above specification on the first time, we will use our drawing board to draw a typical environment, and use our best test technique abstraction ability to imagine the possible cases and list them. We'll give attention to the partitioning and the boundaries when they will come in our path, but we will defer the full discussion of them to a later stage, so to remain focus on the functionality.

Figure - drawing board

Having the drawing in front of us, we would imagine and note the different possible cases:

Let's move memory cells 0 -3 to cells N-3 up to N. Now, intuitively, we will move them to N-2 (cases 1 and 2).

But even nicer would be to copy cells 0 - 3 to cells 1-4 since potential data loss could be here when some of the destination data overlaps the source data. In this point we'll take the idea further and imagine how will the function will copy the data (cases 3 and 4).

In case that data is copied cell by cell, it should better take rightmost cell (in our example "D" from cell 3) and copy it to the rightmost cell of the destination (cell 4) and so on till it reaches the leftmost cell, so to avoid loss of data. From the other hand if we will try to copy cells 1-4 to cells 0-3, the function must start with the leftmost cell to avoid the same data loss.

Let's sum our findings into test cases (we will call it TDD#3):

Case#	Source	Length	Destination
1	0	0+X	N-X
2	0	0+X	N-X+1
3	0	Y	1
4	1	Y	0
5	Valid	Invalid	valid
6	invalid	Valid	Valid
7	Valid	Valid	Invalid

We'll have to define x and y and in some cases to expand more the definition of invalid data , but it's clear that TDD#3 is much more powerful than the previous two, since it's cover functional aspects of the software.

O.K. the reader will say, you waste our time demonstrating a totally improper use of a test technique which relates to data when the actual focus has to be on the functionality. Would you like that we will add a warning against improper use of a technique on the package?

Yes. We must warn against improper use. when the focus wanders from the cognitive research and analysis to the BKM area, it is likely that we and our reviewers will be less criticizing. If in our example we were starting from the simple abstraction on the drawing board, and only then we will look inside the "techniques basket", picking the ones that we think most adequate for our proposal, we will also give it the proper weight, watching from the technique blindness.
In our example of the last Test design, we might think that few boundary values are missing and that we should relate to both invalid partitions in the invalid input cases, but we will understand that these are only second priority items. since the whole BVA technique is blind for relation betwen the variables, starting with the technique will lead our mind in the less relevant direction.

Changing the nature of this blog

I have changed my mind!

I don't see any point to dedicate the blog only for reviews. Allthough I am not going to refrain from reviewing other's work at all. So I have changed the title.

I would like to answer Pradeep Soundararajan call ,and go on blogging my own ideas too. I thought about that for a long time, but his last post gave me the push.

Hope to be able and do that soon.

Online test magazines

The first magazine I would like to link to is Testing Experience:
http://www.testingexperience.com/

This magazine already has it's 4th issue, and seems to collect articles, which expands the knowledge in many areas. So far, I found in every issue 1 or 2 articles which I found interesting, and expanding my knowledge in new domains of of our profession. The magazine also, has it's news section, and the last one, just announced breaking news on....a new quality testing certification... Hey! is this the new gospel of testing? I don't want to start here a discussion about certifications, but this just seems to me as too much. So for the sake of balance, ill point you to a video from STAREAST 2008, which address this subject , and probably will make you smile:
The Angel and Devil of Software Testing http://uk.youtube.com/watch?v=oHtydNQU1f8

The other magazine that I want to review is Quality Matters:
http://www.quality-matters.org/

This magazine has published already it's 1st issue, and seems to concentrate on the foundations of testing. The feature subject of this issue was risk based testing by Rex Black, which is one of the recognized test experts in the industry, allthough too much related to the factory school. The article addressed the foundations of RBT, and could be a good summary of foundations of this approach. So was another article about team development.

Lessons Learned in Software Testing (book)

Lessons Learned in Software Testing A context-Driven approach, by Cem Kaner, James Bach, Bret Pettichord.


My testlink rating: 5 stars



http://www.amazon.com/Lessons-Learned-Software-Testing-Kaner/dp/0471081124





I thought that it would be appropriate to start this blog with a recommendation on the most valuable( in my opinion), book in the field. This book sits on my cubic shelf, and I use my free time to have one more glance in one more lesson. the book is great for any level, from the most beginners to experts and managers. The thing that I like at most in this book is that it's divided into short lessons, cutting off the accompanying bullshit from the valuable information.

Additional related link - James Bach homepage http://www.satisfice.com/