I like being corrected when it improves my performance or keeps me from looking silly in public, but it still grates a little. There are people in our community who like to practice using words very precisely and I think this is a good impulse, but I find it too easy to fall into abstract semantic arguments. As such, I’m a little burnt out on the topic.

Yet, when I run across a post Alan Page put up, which mentions a post by Matt Heusser, where they grapple with “test strategy“, “test plan“, “test design” I find I do have some energy to put towards the topic.

My first reaction to all this is, “Seriously? We are this far into “software testing” and we don’t have a common vocabulary?” OK, everybody put down your other work and lets bang this out. Such issues can’t continue to hold us back. Quick solution – let’s use the dictionary definition of these terms for now. They’re pretty good and will enable us to move forward.

In Alan’s essay certification is tossed out as a solution. To Alan’s credit he quickly tosses it away, but why not mention the AST’s work to create a testing dictionary project? Better yet, why not volunteer for it? Best solution I’ve heard of to date. Let’s stop talking about certification as a solution to any of the big problems facing us. That energy is best spent elsewhere.

Later Alan says:

“Also – to be ultimately clear, there’s nothing wrong with what Matt says, we just have different interpretations of what a test strategy is – no big deal.”

It is a big deal, because it takes energy to talk about this over and over and over again, without producing a solution. A solution that allows us to move on.

Reading a book on troubleshooting, the author made the distinction between an “exercise” and a “problem”. An exercise is basically a situation where you know all you needed to know and just have to “run the numbers” to get the solution. A problem is where initially you aren’t sure what to do. It requires some thinking, some research, a little exploration, maybe talking to people. You come up with some hypotheses and start working through them.

Is a “test plan(strategy)(design)” an exercise or a problem?

If you are writing one for a maintenance release, patch or something of that ilk , it is probably an exercise. Writing one for a 1.0 or a major overhaul? Problem. Between we have the spectrum that makes for great after work discussions.

We don’t need to talk about exercises here. Problems? Proceed.

Alan continues:

“For me, a test strategy is a description of the general approach a team will take to testing over a product cycle. It describes a set of approaches and techniques a team will use, why those approaches are appropriate, and how they will benefit the team and product. I (in my definition) avoid details of implementation, and instead, use the strategy message to align a test team and get everyone on the same page with values, approaches, and areas of growth.”

Alan is far more ambitious than I with his test strategies. You are going to improve the product and the team? I tend to separate the two as each has different needs, cycles, temperaments, and vacation schedules. Given this ambition Alan makes the test strategy too hard. Separate the two.

“To meet these goals, a strategy basically has a bunch of testing ideas (ideas include everything from approaches to processes to tools) – some are challenging, but all definitely achievable. Some ideas are part of the day-to-day workflow of the team, but the team doesn’t address every single idea every day. Instead, the team bounces around between ideas as appropriate to the context of the product cycle, feedback, etc.Sometimes, owners are assigned to own ideas (or pieces of the strategy) throughout the product cycle, while other ideas cycle between team members.”

I would  make distinctions between “test ideas”, “approaches”, and “processes”. It feels like this is the same point where Alan and I both have a problem creating these documents. You have all this stuff mashed together and teasing it apart starts to look like a project in itself. I start wondering why I’m writing another one of these damn documents I don’t really like.

From the list Alan gives, it feels like things such as day-to-day workflow should be decided elsewhere. I like the mention of context but wouldn’t mind more description of his to see how his strategy is a response to it. Interesting mention of ownership of ideas. It would be a flexible team that could pass these around easily given shifting availability and feedback from the testing project.

To bring this home, lets buy a dictionary for now, signup for the AST testing dictionary project, keep the testing project and building our teams as separate activities, share some actual plans that worked for you, and then move on to solve some problems.