I'm Not Ready for School... Yet

Michael Bolton has responded to my plea for schools to 'go away'.
The context-driven school obviously exists. People keep telling me it does. I can't deny that. God bless it and all who sail in it etc. etc. It is the definitions of OTHER schools by the context-driven school that I'm discouraging. That wasn't clear, I suppose.

Michael, like James, seems keen to set me up in my own little school (of one :-))

I'm not offended and it's kind of flattering. I have a thick skin and a sense of humour. My response to James' post posits my view that some people who do not have thick skins and senses of humour will be offended if you label them. I also tried to explain why I think it is an unhealthy activity for one school to define the 'other schools'. Telling people how they think isn't very helpful.

Michael writes, "maybe...Paul means no one wants to be tagged as a member of a school against his or her will, or by someone else." My point exactly.

Do I want to be part of a school? The Context-Driven (CD) school? Hmmm, let's see now...

Michael says, "the belief that high-formality approaches are universally appropriate is the result of crooked thinking, just as the belief that they're universally inappropriate would be too". I agree. This thinking suits me nicely. I like it.

Naturally, we should extend this to cover all approaches and contexts. So I think I'm agreeing with the the CD guys in believing that there are no always-good practices (and maybe no always bad ones either? (Actually, I CAN think of some always-bad things to do ;-) so let's park that bit). I can't think of a situation where the no always-good practices statement would be proven untrue. Too many negatives there, but I'm so confident it's true that I would call it Axiomatic.

I use Axiom like this: An Axiom is something believed to be true, but cannot be proven in any practical way. It could be disproven by experiment or experience and we should be prepared to be proven wrong. We welcome attempts to do this. This usage is consistent with many other famous examples: the Definitions in Euclid's Elements, Newton's laws of motion, the US Declaration of Independence and many other 'position statements' present sets of beliefs without proof or corroboration. Most have subsequently been shown to be imperfect, but they continue to work for most practical purposes.

To distinguish it from Context-Driven, let me call this axiomatic notion Context-Sensitive (CS).

But the CD guys do not believe in axioms. They say nothing is certain (even death and taxes?). All rules, principles, approaches etc. are heuristic. Heuristics have value in some contexts, but are limited in application, usefulness, accuracy etc. They are limited or fallible in known ways. This I appreciate. But this leaves me in a curious position.

I can't call myself CD because I have more confidence in one of their key values than they do. I'm more CD than they are :) I know this is facetious, but I'm serious about the CS Axiom. I've thought this way since 1993, I reckon. I believe it would be dishonest and perverse to think any other way.

As an Axiom, however, I don't find a CS Axiom so useful. It's too broad. Intuitively, I think there's some kind of sub-division. It also relates directly to approaches, and it's really hard to make any axiomatic statements about approaches, even in general. (I think ends can be 'axiomatic', but not means).

In the Testing Axioms I've proposed, several relate directly to this notion of operating within one's context. The stakeholder, test basis, test oracle, fallibility, scope management, design, environment, all directly or indirectly relate to context. Context isn't fully described by them obviously - it's hard to think of axiomatic statements about the technology, culture, risk etc. for example.

So here's my proposal for Axioms restated in a different way. If a set of test Axioms exist, I think it is possible to word them in a way that all testers would regard them axiomatic. That is, the Axioms might be wrong, but for most practical purposes, we can safely assume they are correct. If anyone devises a situation where the Axioms are violated, we think again: Maybe we scrap or change the Axiom. Maybe we qualify the Axiom by refining our scope of applicability.

Test Axioms could then apply to all testing schools. They are context, approach, school independent. In fact, they don't reference anything that could be interpreted as a specific object under test. In my time, I've written tests for software, software systems, interfaces, hardware, firmware, buildings, furniture, telephones, concrete, people. The Axioms still apply. They must represent a core set of values that testers accept. If testers cannot agree on anything, they should still share these core values.

For me, all this adds up. Even the guys with the most strident, extreme views in testing must have some things in common, surely?

Now, all this is well and good, but as is pointed out so often and to our continuing embarrassment, we can't even agree on the definition of test. Just about every consultant and practitioner has their own. The only way out of this is to adopt a context-, process-, technology-neutral (and other things-neutral) definition.

The best candidate I've found so far, is the one from the American Heritage Dictionary: "A procedure for critical evaluation; a means of determining the presence, quality, or truth of something; a trial". (Find it on dictionary.com) There are three variants, but that's reasonable and helpful - it's a more rounded definition. It's independent of most of the things we find ourselves debating all the time. It could, in principle, apply to testing anything. Best of all - it wasn't invented by a software tester.

So how do Axioms and Schools stack up?

Here's my Equation of Testing (strategy):


This is not a well-formed idea. It's just a basic recipe. Perhaps this is as sophisticated as it gets. It's derived from many, many discussions with many people over the years. Regular discussions with Neil Thompson have helped in particular.

The TESTING VALUES, some specialised definitions (including that of test, perhaps) and some approaches unique to your School, come from your School. I'd like to submit that this equation applies to ALL schools.

Am I interested in joining a School? Not yet. I don't think I'm old enough.

I look forward to talking with Michael at Eurostar in the Hague next week.


>Now, all this is well and good, but as is pointed out so often and to our continuing embarrassment, we can't even agree on the definition of test. Just about every consultant and practitioner has their own. The only way out of this is to adopt a context-, process-, technology-neutral (and other things-neutral) definition.

This isn't the only way out. It's not even a way out. As long as people are involved, there will be contexts and technologies and other things, and they won't be neutral.

You might like to check out
the work of John Wilkins
and the idea of a universal language. The effort began in 1668, and we still appear to be stuck. Wittgenstein knew why.

"Wittgenstein tried to show philosophers the way out of the fly bottle of philosophy by getting them to pay closer attention to the meanings of words. Just try to define a simple word like 'game'. Wittgenstein pointed out that if you say a game has to involve luck or have a winner or be fun, you will always be able to find an example of a game that doesn't fit your definitions. When philosophers give words overly precise or restrictive definitions, when they ignore the complexity and the irredeemable vagueness of language, they often fall into confusion."

-John Mighton, The End of Ignorance

---Michael B.

Paul says: Since I wrote that statement, it's clear that it needs a little more clarification. And the thinking I'm using applies to Axioms as much as definitions.

The motive for seeking some common, agreed definitions is to enable some clearer communication and common focus. Why are we having this (and other) conversations? We must have something in common - since we both use the word test in almost every utterance, that would be a good starting definition. I understand there will be peripheral ideas that don't fit - that's why we need to define the scope of the definition.

So, I agree absolutely about a definition being prone to exceptions. So all shared definitions need some qualifying scope to be complete. Now we could spend our lives trying to define to the nth degree the scope in which a proposed definition applies. But unfortunately none of us have the time. Sure, I guess we aren't talking about pregnancy tests, or DNA tests, or olympic trials or memory tests or ... whatever. These could all be called testing - but I don't think that's what we're talking about, is it? Do I really need to identify all the exceptions to use that word with confidence?

If I use the American Heritage definition of test, and I then use the qualification that we're talking about testing 'systems' - then we get into really deep water - count the definitions of system on dictionary.com. This discussion could take some time.

But we can still have an intelligent conversation mutually trusting each other to know when we are drifting into a periphery where our common understanding might disintegrate. I think we're smart enough to know when we're in that risky area.

Call me a school (or idealist or fool) if you must, but I think deep down there are a set of common beliefs and fuzzy (until agreed) definitions held dear by testers in our field (define field please :-) ). Then there are values that we share and some values we don't. I think it's consistent to vary or further qualify our definitions once we drift into values or schools, and I appreciate it might be expedient to have different definitions for different contexts. But underpinning all this, I still think it helpful to have a common notion of what we means by some key things. Then we can move away from the angels on a pin arguments (or whatever) and talk about what we really need to talk about.

So I'm happy for others to invent their definitions. But if, next time we talk, I ask you to precisely qualify your definition of test (and identify all exceptions) - I suspect you'd be a tad frustrated. This is one reason why I think it's sensible to use a definition of test not created by a testing consultant. We both can take it at face value and trust each other to know what we're both talking about - because a modicum of English comprehension is all that's required to make some progress.