Why I have troubles signing Stop #ISO29119

In the last few days the noise around ISO 29119 is metaphorically deafening. And the case the proponents of the petition to suspend/revoke 29119 sounds quite reasonable. They claim that there is no wide consensus, that not all sides have been heard. They fear that this standard is there to create business for big consultancies and not to further the software testing profession. Many of their arguments are sound and do resonate with me as well. I agree that, as to what I have seen, the standard only covers some parts and a very particular approach to testing. And I agree, that judging the whole standard based on its content is not as easy as one would like, as you have to pay quite some money for it.

So why is it then, that I have troubles signing that petition? I am a freelance consultant as well (not only, admittedly), not in line with everything the ISTQB or other organizations say about testing. I am affected in my job as well as anybody else in the testing profession. And I do take my profession seriously. So given that the arguments seem sound and I am affected, I should not hesitate. Yet I do.

I do, because of the ongoing discussions I have been following in the social media the last few days. I do, because of the fact that I have heard a lot about “that has to stop” but not a lot about “that is what we offer”. I do, because I do not know how many of the commentators actually reviewed the full standard, which is not freely available. I do, because I myself had not the opportunity to review it. I do hesitate, because the discussions in the last few days made me doubt the motives of some of the proponents.

My impression is, that contrary to the claims of the initiators of the petition they do not seek an improvement of the standard. At least some of them, sorry for generalizing. My impression is that some just want to stop any standardization process. Fair enough, but then say it. My impression is that it is not about additional inclusion of modern or maybe even controversial ideas, furthering the development of the art and engineering of software testing. My impression is that it is about replacing just with their views. My impression is that it is not us and them and a lot of others. My impression is that it is us vs. them and the rest of the world who have different opinions. So, if my impression would be true, some of the proponents do not have any better intentions as they ascribe to the makers of ISO 29119.

The cynical person in me suggests that some of them are just basically afraid that they could lose their business of selling “non-standard” testing.  That their intentions are not primarily about testing, but about their personal gain. To be fair: consulting to get companies standard ready is a lucrative business, and it would not be the first standard where quite a lot of people  suspect strong industry interest (formulating cautiously, not wanting to get sued). I have decided to dismiss this thought, as I have come to hold many people proposing to stop ISO 29119 and their ideas in high regard.

I personally get wary about nearly every kind of evangelism of methodologies. And I know that this in itself can be considered dogmatic. But I have made the observation that the freedom and inclusion demanded by avant-garde thinkers is often not given to others by the very same people. And I am deeply convinced that one must allow others the same freedom of opinion and leeway of acting as one is asking for him or herself. I really like Robert Martins article (http://blog.8thlight.com/uncle-bob/2012/04/18/After-The-Disaster.html) on standards, as it is thoughtful and draws a very accurate picture as to what could happen. I figure that this is the last thing the proponents of Stop 29119 want to happen. I do not want that to happen. I believe nobody in the testing profession wants that to happen.

I therefore strongly believe that we have to come out of this “we are cooler than you”, “we are more experienced than you”, “certification is good, because …”, “certification is bad, because” behavior and try to find common ground. That common ground might not be all encompassing. That common ground might be the most minimal consensus a diverse, multi-national, multi-cultural, multi-whatever-there-is community can come up with. But I reiterate: just being against is not enough. And I have seen few things beyond that in the last few days. That’s why I have troubles signing the petition, even though I see a lot of valid points in the arguments put forward.

Ps: I will invite some proponents of both sides to comment on that in the hope to get andprovide more clarity.

pps: I found a bug in WP when trying to publish this article :)

This post is also available in: German

10 thoughts on “Why I have troubles signing Stop #ISO29119

  1. I’m one of the signatories of which you speak. Although I did not design or produce the petition

    In one sense, it won’t make any difference to my business if 29119-1, 29119-2, and 29119-3 are left to stand, and if 29119-4 and 29119-5 move from draft to accepted. Rapid Software Testing is about actual testing skills—exploration, experimentation, critical thinking, scientific thinking, articulate reporting, and so forth. That doesn’t compete with 29119, in the same kind of way that a fish restaurant doesn’t compete with the companies that make canned tuna. We object to people manipulating the market and the ISO standards development process to suggest to the wider world that canned tuna the only food that is fit for people to eat. I discuss that here: http://www.developsense.com/blog/2014/08/rising-against-the-rent-seekers/

    In another sense, 29119 could be fantastic for my business. It would offer me a way to extend the brand: how to excellent, cost-effective testing that stands up to scrutiny in contexts where some bureaucrat, a long way away from the development project, was fooled into believing that 29119 was important. At the moment, I’m happy to refer that kind of business to colleagues of mine, but I suspect that it would be something of a gold mine for me. Yet still I oppose it, because what’s in my interest may not be in the interests of my clients and of society at large.

    Let me be specific: There are existing standards for medical devices, for avionics, and the like. Those standards matter, and many of them are concise, well-written, and were created by genuine collaboration among interested parties. Testers who are working on medical devices or on avionics software have a limited number of minutes in the working day. As someone who flies a lot, and as someone who is likely to require the help of medical devices in the foreseeable future, I would prefer that those testers spend as many minutes as humanly possible actually investigating the software, rather than complying (authentically, pathetically, or maliciously) to an unnecessary standard for process modeling, documentation, and strategizing (a standard for developing a strategy—imagine that!).

    “I have made the observation that the freedom and inclusion demanded by avant-garde thinkers is often not given to others by the very same people.”

    If you believe that, how would you feel if those people tried to impose a standard on how you do your work?

    But I reiterate: just being against is not enough.

    Yes it is. Do you have to be for something as an alternative to being against lung cancer?

    But as a matter of fact, I am for something that is more important than any standard: freedom and responsibility for the quality of my work (as I hope you’re for freedom and responsibility for the quality of your own work). That includes the responsibility to make my work capable, credible, open to scrutiny, and as cost-effective as possible. I must be responsible to my clients, to my craft, and to society as a whole. In my view, those responsibilities do not and should not include compliance with unnecessary, time-consuming, unrepresentative standards created by self-appointed documentation and process-model enthusiasts.

    I hope you feel the same way.

    —Michael B.

  2. Interesting take on the debate.

    I have signed the petition and I have no vested interest in regards to being a person who sells consultancy or training or otherwise. I signed the peitition to get it suspended so a discussion with the whole of the testing community can happen and not a select few.

    My question to you is why is there no software development standards? There are good approaches andas you put methodologies but there is no all encompassing standard that dictates the process and documentation that ‘has’ to be done. Why is that the case? Is it that the craftmanship of designing and developing software is very much a creative skill along with a large amount of tacit knowledge that is hard to define in terms of a set of processes and practices. I see the profession of a tester in a similar way. How can you create a set of processes around something which is heavily dependant on creative and critical thinking? The standards as they stand, and yes I have read them, do not reflect how modern software development is being implemented. They see testing as a phase rather than as a activity that happens throughout software development. They isolate the role of testers and do not mention dev/test working together.

    So Bernhard as someone like myself who would gain nothing if the standards were not published or if they were, still signs the petition based solely on the reason that I care about testing does that affect your decision why you do not sign?

    • Hi,

      thanks for that thoughtful comment. A side note: there is an ISO standard on software development (ISO 12207 – Systems and software engineering — Software life cycle processes) and it is a process standard. And I’m not surprised that it is not commonly known :) But it is fairly valid (in parts) with the automotive industry (german, that is) through the SPICE Assessment standards.

      I do believe that software testing is a creative process (highly so). I do also believe that there are a bunch of methods and tools out there, one should strive to learn over time as a tester. I believe that there is no one size fits all approach.
      That said, my understanding of standards (in our area – mind you, not talking about medical stuff or so) is one of a toolbox. If I need a hammer, it is in there. A drill – there as well. I have never used a hammer to drill a hole (well, maybe I tried when I was little, but I’m pretty sure I got the idea quickly – my mother still tells the story of me building a drill using Matador and a pencil sharpener and – yep drilling holes in the wall of my room – ok off topic).
      ISO 15504 is a standard I’m quite familiar with. I like that one. Basic question there. Whatever you do, is it fit to achieve the intended result. In my mind a great question.

      On your question: yes it does affect my decision. The whole discussion about it does. As I said, a lot of arguments resonate well with me. But I also want to make an informed and educated decision. And I still see some worth in trying to have an inclusive, diverse, common approach to testing where the educated tester can choose based on situation.

      • Bernhard, when you say this:

        And I still see some worth in trying to have an inclusive, diverse, common approach to testing where the educated tester can choose based on situation.

        I agree. One of the essential premises of the context-driven school is that there are good practices in context, but that there are no best practices. What you’re calling
        an inclusive, diverse, common approach to testing is at its essence the open market of ideas.

        Standardization by the rent-seekers is an attempt to suppress that. I trust you’re aware of this: http://www.nist.gov/standardsgov/definestandards.cfm. Let me quote the second-last paragraph:

        Still another classification scheme distinguishes between voluntary standards, which by themselves impose no obligations regarding use, and mandatory standards. A mandatory standard is generally published as part of a code, rule or regulation by a regulatory government body and imposes an obligation on specified parties to conform to it. However, the distinction between these two categories may be lost when voluntary consensus standards are referenced in government regulations, effectively making them mandatory” standards.

  3. I think the problem is that 29119 calls itself a ‘Standard’, which sets an expectation that it is a standard that all should be held to!

    I have seen various posts, including one from one of the authors, that says 29119 is not something they expect people to be held to and you can take it, or any component of it, or leave it; so it’s not a standard, but more a guidline?

    I personally don’t think an ‘Industry Standard’ can be applied to Software Testing. Each company I have worked with has used different methods and approaches to varying success. Sometimes a successful approach in one situation will fail in another. There are so many different methodologies that suit different products in different companies and even in different teams: Agile isn’t for everyone; Waterfall can mean each team in the SDLC can adopt a different methodology and still be successful, and so on. I agree that variety can cause confusion and may require someone to bring it under control. But: In testing there is no one size fits all; and I wouldn’t want there to be.

    One of the aspects of testing is the search for: Something new; New ways of working; and Improvement. Whether that improvement and way of working is applied to the test subject or the method used to test the subject, we are always looking at what we do differently and with an eye on identifying potential issues and improvements. When a ‘standard’ takes years to compose it is already out of date. I’d say that from year to year the industry is evolving and maturing, even quicker in some areas.

    There are initiatives around to build taxonomies, glossaries, dictionaries, so that we can all agree on the language used in order to understand each other and help new testers to understand the career they have entered: This I applaud, but it has to be a living database. Note: With respect to the language of testing, we fight an eternal battle with forces outside of testing. e.g. ITIL brought in contradictory terms for testing when compared to ISTQB terms.

    I have not been able to justify paying out for the whole 29119 download, I can only comment on what the standards website tells me. I do understand that the 29119 includes a list of terms and meanings. Again I don’t know how this maps to previous attempts to define the language. I would hope that 29119 doesn’t contradict everything before it, and as it’s based on previous ‘standards’ I expect it doesn’t.

    In my view a standard of how to test cannot cover all situations and environments. However providing a common language does acheive at least part of the intent of the standard. It makes everyone talk the same language that is understood globally.

    At the end of the day, I am not worried about 29119, just like I was not worried about the IEEE standard that is refered to by ISTQB. I’ve tried to use it but it’s never been practical in any of my roles over the last 15 years of testing. I do try to enforce the language of ISTQB though, just to keep a clear understanding.

    If the standard is as bad as people claim, it will become another peice of shelfware, another bookshelf filler. However if it is as good and usable as the working group hope their time produced, we’ll see it used, and we’ll all be certified in 29119 in 5 years. I personally think it will be the first.

  4. The idea of a common language that is understood globally is one of the more farcical arguments promoted by the standards enthusiasts. The standard is written in one language… and then translated into others! Nuances and shades of meaning creep into those translations. For example:

    „Das Ziel der ISO/IEC 29119 ist es, einen endgültigen Standard für Software-Prüfungen zur Verfügung zu stellen, welches Vokabular, Prozesse, Dokumentation, Techniken und ein Prozess-Bewertungsmodell für Software-Testing beschreibt und innerhalb jedes möglichen Softwareentwicklungs-Lebenszyklus verwendet werden kann.“

    (Source: Wikipedia.de, August 29, 2014)

    Does that mean the same as this?

    “The aim of ISO/IEC 29119 Software Testing is to provide one definitive standard for software testing that defines vocabulary, processes, documentation, techniques and a process assessment model for software testing that can be used within any software development life cycle”.

    Source: the 29119 standards site on or about June 6, 2012 (quoted on by James Christie on his blog). However, the discerning reader will note that the site now says this:

    “ISO/IEC/IEEE 29119 Software Testing is an internationally agreed set of standards for software testing that can be used within any software development life cycle or organisation. By implementing these standards, you will be adopting the only internationally-recognised and agreed standards for software testing, which will provide your organisation with a high-quality approach to testing that can be communicated throughout the world.”

    The differences in meaning and time show up before we’ve even opened the official documents.

    http://www.developsense.com/blog/2011/06/common-languages-aint-so-common/

    I do try to enforce the language of ISTQB though, just to keep a clear understanding.

    How do you think of exploratory testing? As this?

    “Exploratory testing is concurrent test design, test execution, test logging and learning, based on a test charter containing test objectives, and carried out within time-boxes.”

    (Source: ISTQB syllabus)

    Or as this?

    “Exploratory software testing is a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the value of her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”

    (Source: Cem Kaner’s Web site)

    If “exploratory testing” is the former, what do you call the approach that you’re using when you’re testing to investigate a mysterious situation that has suddenly occurred when you’re in the middle of a formally scripted test?

    A note on the citations: apparently if you put too many URLs into a comment, the spam filtering feature on this blog treats the comment as spam. I’m happy to provide URLs for each one of the citations above.

    —Michael B.

    • I might not have been clear when refering to a common language. I wasn’t refering to German v English v French etc. I guess vocabluary would have been a better term to use.

      With regards to the two descriptions of exploratory testing: (as and example)
      I find it difficult to pick out of of the two any substantial difference. Other than timeboxed v throughout the project.
      They both refer to design, execution, learning, logging/test interpretation as parrallel/concurrent activities.

      One refers to a test charter. Which all testing should be constrained by, if you have one, and the other specifically calls out optimizing work, again that’s implicit in both.

      So I don’t see the difference.

      Taking my ‘blindness’ aside, isn’t that the point of using a common vocabluary, so when you discuss exploratory testing, I am thinking about the same thing.

      Whilst I base my use of testing terms in ISTQB, I also continually refine and mature my understanding of the terminology. I don’t believe ISTQB vocab’ is that far removed from current usage of the same terms and definitions.

      I experience mis-communication due to terminology misuse all the time. Mostly due to people who are not test professionals perspective being skewed. e.g. and engineer who thinks UAT is when he hands it to the delivery team. They after all are his users. But the delivery team think of UAT as with the customer. I try and try again to get them to understand, and document it, but it’s always someone new. I guess that’s always going to be a problem, new people not knowing where to go to learn the lingo. Unless there is a certification or ‘standard’ they are supposed to follow.

      I still don’t like standards for generic testing though. Standards of a methodology, possibly, but not generic.

      • So I don’t see the difference.

        Do you see how this demonstrates the point? If there were a one-to-one correspondence between words and meaning, you would see the difference. But there is no such correspondence. What I wrote is one thing; what you interpreted is another. There are good reasons for that; you and I have different degrees of experience in different contexts, and the words we choose and the the meanings we perceive reflect that.

        I don’t believe ISTQB vocab’ is that far removed from current usage of the same terms and definitions.

        It isn’t… for people who agree with the ISTQB vocabulary. Let me explain the difference that you’re apparently not seeing.

        For the last dozen years or so those of us who study testing and the way people perform it in the real world (rather than an idealized process model) have described exploratory testing as an approach; not as a technique, and not as an activity (as the ISTQB appears to describe it). What’s the difference? An approach modifies a technique or an activity. You can perform pretty much any technique in a scripted way or in an exploratory way. The ISTQB’s description of exploratory testing is designed to provide a basis for shallow agreement, and to put aside the more nuanced, deeper understanding of what we do as testers. Consider this: http://www.developsense.com/resources.html#exploratory

        I experience mis-communication due to terminology misuse all the time.

        I think it’s more likely that you experience terminology misuse due to miscommunication.

        I guess that’s always going to be a problem, new people not knowing where to go to learn the lingo. Unless there is a certification or ‘standard’ they are supposed to follow.

        No standard, and certainly no certification based on a 40-question multiple choice test, will help with that in any significant way, since the “standard” is, in my view shallow and misrepresents testing. In order to have any significant understanding a language and a culture, you must become immersed in that culture.

        And funny you should mention it: I wrote specifically about the UAT issue here: http://www.developsense.com/presentations/2007-10-PNSQC-UserAcceptanceTesting.pdf

        —Michael B.

  5. For each writer and reader of thoughtful blogs about testing and 29119, there are thousands of uninformed testers, managers, executives and companies. These are the folks that do not (or cannot) think things through. These are the folks that don’t/can’t see a standard as just “one of a toolbox”. Rather, they see a standard according to the definition: A required or agreed level of quality or attainment. And that is part of why 29119 scares me. Because of a world full of followers that might make my life miserable.

  6. Bernhard,
    Reading your blog post I felt as if I was reading something I had wrote. As if I had somehow written down my thoughts in your blog! :)

    Oddly enough I share the exact same skepticism you do regarding signing the petition. And I have been abstaining for very similar reasons. There is a discussion on the LinkedIn group Software Testing and Quality Assurance titled “ISO/IEC/IEEE 29119 – why the fear and opprobrium” where we have been discussing exactly this. Linked below:
    https://www.linkedin.com/groupItem?view=&gid=55636&type=member&item=5910761277268963330&commentID=-1&trk=groups_item_detail-b-jump_last#lastComment

    I have posted in this group some of the very same points you are making in your blog post regarding the stop campaign. I too question the motives (not everyones) but certainly enough to make me wonder.

    Again, thanks for the post it was good to know that my opinion is shared by others. Bravo Zulu!

    Freddy Vega

    PS – that was some hard math on the capcha! :)

Leave a Reply

Your email address will not be published. Required fields are marked *

*