Testing Computer Software by Cem Kaner, Jack Falk and Hung Q. Nguyen

Three sentence summary

This book is about doing testing when your coworkers don’t, won’t and don’t have to follow the rules (p. vii). Realistic test planning is dominated by the need to select a few test cases from a huge set of possibilities (p. 17). Testing is best conceived as a group that provides technical services and information. (p. 343)

Book Club

This is the fifth book that my team at work is reading for our book club. We were reading this book from August to September 2022. We had a meeting every month to discuss the book. I like talking with people about good books as much as I like reading them.

Random thoughts

I’ve already started reading this book many years ago, and gave up. I remember thinking it was very hard to read. I didn’t find it so hard to read now.

This is one of the rare books that I’m reading from a paper copy. I usually read e-books. This one is pretty old. It has the very familiar smell of an old book. The smell takes me back to my childhood when I was reading a lot of comics that smelled exactly like that.

I was planning to read the book while traveling the Croatian seaside this summer. So, the book visited three islands with me, Brač, Hvar and Čiovo. Unfortunately, I didn’t have any time to read the book there.

Chapters

When I read a book, I like to write a short summary of each chapter. It’s usually a quote from the chapter. For some chapters a have a few random thoughts or quotes.

Section 1 - Fundamentals

1 An example test series

Summary: This chapter introduces the book by illustrating how an experienced tester could approach the early testing of a simple program. (p. 1)

2 The objectives and limits of testing

Summary: Realistic test planning is dominated by the need to select a few test cases from a huge set of possibilities. No matter how hard you try, you will miss important tests. As we see it, testing is the process of searching for errors. Good test cases are more likely to find errors, or more likely to find serious errors, than poor test cases. (p. 17)

3 Test types and their place in the software development process

Summary: This chapter is a general overview of the field of testing. (p. 27)

Quote: Some readers have identified this as the most boring chapter of the book. People who stopped reading this book tell us they stopped here. (p. 28)

As far as I can remember, in my entire life, there are only two books that I’ve started reading but didn’t finish. About a decade ago, I started reading this book but gave up. Probably in chapter 3. 😅 The other book is Crime and Punishment by Fyodor Dostoevsky. (It was required reading in high school.)

Quote: In bebugging a program, someone deliberately introduces bugs into it before the person who will test it starts testing. (p. 49)

I don’t think I’ve ever seen this done.

4 Software errors

Summary: This brief chapter defines “quality” and “software error”. Then (…) we describe thirteen categories of software errors. (p. 59)

5 Reporting and analyzing bugs

Summary: How well you report a bug directly affects how likely the programmer is to fix it. (p. 65)

Section 2 - Specific testing skills

6 The problem tracking system

Summary: Here we describe what happens to the Problem report after you report it. This chapter provides the basic design of a problem tracking database and puts it in perspective.

Random thoughts: This chapter might be the most obsolete part of the book. It assumes you have to create a task/bug tracking system. Many such systems exist today and it’s very unlikely the company isn’t already using one.

7 Test case design

Summary: This chapter is about creating good black box test cases. (p.123)

8 Testing printers (and other devices)

Summary: This chapter is about configuration testing in general and about printer testing in particular. (p. 143)

Random thoughts: I’ve never needed to test a printer. The vast majority of this chapter is about that. I guess an equally important device to test these days would be a phone.

9 Localization testing

Summary: Localization is even more important now than when the book was written. In the chapter they focus on West European languages and DOS. Today localization testing is much harder. There are three major desktop and two major mobile operating systems. Programs are translated to many more languages, some of them quite tricky (right-to-left, script…)

10 Testing user manuals

Summary: The product includes more than the program. For example, most products also include documentation, packaging, samples and service (such as technical support). (p. 179)

11 Testing tools

Summary: This is a basic introduction to black box testing tools: what you might want to accomplish with them, what types of things they can do, and what some of their limitations are. (p. 189)

Random thoughts: Fundamental tools have not changed in a couple of decades (since 1999). We still use a computer, text editors, spreadsheets, diff tools, file viewers, screen captures, inspectors… (pp. 189-190)

12 Test planning and test documentation

Summary: This chapter … discusses the general strategy and objectives of test planning. We regard this chapter as the technical centerpiece of this book. (p. 203) Remember your primary task - finding bugs and getting them fixed - not designing or filling out forms. (p. 253)

Random thoughts: I found this chapter pretty boring to read.

Section 3 - Managing testing projects and groups

13 Tying it together

Summary: This chapter ties together the organizational and strategic issues. (p. 255)

Random thoughts: It’s been a long time since I’ve tested a piece of software that went through all phases (idea, alpha, beta…). The book suggests that is the usual case. Teams start deploying very early these days, with incremental changes.

14 Legal consequences of defective software

Summary: QC should recognize product defects that are so serious that the company will be sued over them. QC should be able to use the law as ammunition, in an argument to get a serious defect fixed. (p. 303)

Random thoughts: This was a very boring chapter. It is also focused on US law, making it less useful for people outside the US.

15 Managing a testing group

Summary: Testing is best conceived as a group that provides technical services and information. (p. 343)

Random thoughts: I’m not a manager, but I find this chapter very interesting and useful. Especially a very short career growth section, a topic that I’m very interested in.

Quote: It seems to us that every company already has a proper group to set standards, evaluate and train staff, and generally monitor and work to improve every phase of product development. That group is called Management. Management is the real quality assurance group in every company. (p. 347)

I couldn’t agree more.

Appendix: Common software errors

Summary: Check a comprehensive “standard” list to get further ideas for testing a program. (p. 363)

Random thoughts: I thought the appendix would be boring. It is actually an interesting read.