There are some phrases in English that, as often as not, come off sounding obligatory and/or insincere. The phrase “I’m honored…” comes to mind (particularly if someone is accepting an award in front of a room full of people).
Be that as it may, I genuinely felt really honored last night and again today by a couple comments James Bach has said about me, including these:
Here’s the quick background: (1) James knows much more about software testing than I do and I respect his views a lot. (2) He has a reputation for not suffering fools gladly and pretty bluntly telling people he doesn’t respect them if he doesn’t respect the content of their views. (3) in addition to his extremely broad expertise on “testing in general” James, like Michael Bolton, knows a lot about pairwise and combinatorial testing methods and how to use them. (4) I firmly (and passionately) believe that pairwise and combinatorial testing methods are (a) dramatically under-appreciated, and (b) dramatically under-utilized. (5) James has published a very good and well-reasoned article about some of the limitations of pairwise testing methods that I wanted to talk to him about. (6) I co-wrote an article that IEEE Computer recently published about Combinatorial Testing that I wanted to discuss with him. (7) James and I have been at the STP Conference in Boston over the past few days. (8) I reached out to him and asked to meet at the conference to talk about pairwise and combinatorial testing methods and share with him my findings that – in the dozens of projects I’ve been involved with that have compared testers efficiency and effectiveness – I’ve routinely seen defects found per tester hour more than double. (9) I was interested in getting his insights into where are these methods most applicable? Least applicable? What have his experiences been in teaching combinatorial testing methods to students, etc.
In short, frankly, my goals in meeting with him were to: (a) meet someone new, interesting and knowledgeable and learn as much as could and try to understand from his experiences, his impressive critical thinking and his questioning nature, and (b) avoid tripping up with sloppy reasoning (when unapologetically expressing the reasons I feel combinatorial testing methods are dramatically under-appreciated by the software testing community) in front of someone who (i) can smell BS a mile away, and (ii) doesn’t suffer fools gladly.
I learned a lot, heard some fantastic war stories and heard his excellent counter-examples that disproved a couple of the generalizations I was making (but didn’t dampen my unshaken assertions that combinatorial testing methods are wildly under-utilized by the software testing community). I thoroughly enjoyed the experience. Moving forward, as a result of our meeting, I will go through an exercise which will make me more effective (namely carefully thinking through and enumerating all of the assumptions behind my statements like: “I’ve measured the effectiveness of testers dozens of times – trying to control external variables as much as reasonably possible – and I’m consistently seeing more than twice as many defects per tester hour when testers adopt pairwise/combinatorial testing methods.”
His complement last night was private so I won’t share it but it ranks up there in my all time favorite complements I’ve ever received. I’m honored. Thanks James.