The last weeks have been very busy ones with a number of events and discussions. And while checking in with colleagues and friends, I was also very busy in trying to remediate some parts of a somewhat challenged software project. And yes, it was a Scrum project, carried out by people from a very well renowned company.
The project was about exchanging one rather old and definitely outdated and user-unfriendly component with a state of the art solution. The management of the customer decided to go with a very well known company, whose employees have been an integral part of the development of Agile methodologies since years.
The project progress
The project setup was a pretty classical Scrum one, and we were able to look into the show cases every other week. And what we saw, was pretty much what we expected. A usage of accepted practices, solid agile methodology, a nice and working deployment pipeline. What we also saw, and where we did not react in time: the deployment pipeline ended with the component test. Yet the component we discuss here, had to be integrated into a larger system. Even though the conditions to integrate and the environment to do so where discussed with the team several times, it just did not happen.
Admittedly, the task would have been very cumbersome. The environment was and is complex and tricky, to say the least. But here is fail number one: we did not insist on the regular full integration!
Who is the user?
The team focused strongly and rather remarkably on the end users to understand their needs and wishes for the new product. This was excellently done and, from that end-user perspective the project can be called a success. So why am I here writing about a failure. Because the team, even though they had a lot of talks with relevant stakeholders missed out a group of users. The operation guys, who in future had to run and maintain that PoS (piece of software). In classical project management (and also in requirements management) we would call that a botched stakeholder analysis. And as we know from requirements engineering: missed stakeholders are missed requirements. A point which would cause another fail.
Operators are Users too
Before anybody comes up with DevOps suggestions: since this was a one off project, this was not an option. When the development had been done, it had to be handed over to operations. To be fair: it is not new or uncommon that Operations is not thought about when developing software. Yet you don’t want to see a miss like that.
Operations people, in my experience, belong to the way more ordered and structured people in IT. Seems to me that you have to be structured, if you manage thousands of servers and databases, no matter how sophisticated the tools have become. That´s the reason why they typically can state very clearly, what they expect from software. Most of them have a standard catalogue of minimal operational requirements, covering things like logging, alerting and other attributes relevant for efficient operations. You might also be interested in their view on preferred and available systems and system parameters, as they will affect your software.
Of course, you can either knowingly or unknowingly ignore these things. You might come up with an even better solution for the typical needs of operations and they will be very happy. If your team has some datacenter operations experience, that might truly be the case. Yet, if not, it is more likely that you deliver software, which is considered hardly operable.
Let’s go with an easy example of fail: in normal operations your application should log only the bare minimum (or what is considered as that). On the one hand for performance reasons, on the other hand because we want to limit the disk space usage. There is a reason why they gave us log levels. And they should be easily configurable for an operator, to turn them to finer granularity when needed. You should also make sure that your log files don’t clog up the system over time. Size limits, rolling files and a maximum number of file generations will do the trick here.
Admittedly, one can say that monitoring and cleaning disks is part of the housekeeping. Yet you tell your children to clean up their room themselves as you do not want to clean after them – you do it anyways, but you still hope to save a bit of time.
Communication over Documentation
The Agile Manifesto states very clearly “both are important, we just think left is more important than right”. I always wonder why people do not care to read and understand what that means. It means: you have to have reasonable documentation, where needed. And by the way: an agile team might like it or not. The need is defined by the customer. Like everything else, it is negotiable, yet it is the customer’s call.
Specifically, we did not create a deployment pipeline up to the integration system. Ok, we can do that. However, what we cannot do is: missing out in using a standard deployment method and not documenting how to install and configure it. It doesn’t surprise anyone that, when the time came to integrate it (basically, at some point in time the team realized, that this step should be done), it took nearly two days and a lot of work to get it set up. This is a very bad value for a piece of software which deploys automatically in ten minutes. Why was that the case?
It was because the system test did not account for surrounding systems. Therefore, the configuration tasks for that were just left out. And also, to some extent, forgotten. On the other hand, as there was no documentation on how the software has to be installed, the guys knowing the whole environment and dependencies (btw., from a not very good, but yet at least existing documentation) could not help easily and fast. Not being approached once over the whole project duration, it also took them some time to understand, what the component should do and where it would fit into the big picture. So here is another fail. Do not forget that someone else has to work with your software in future.
As we see, plainly focusing on the end users doesn’t do the trick to cover the requirements. You should never forget about other (potential) stakeholders and user groups. The fails we observe here can happen in a non-agile project as well. The important message is that the application of agile methods doesn’t automatically deliver you good software. The basics of project and requirements management, like understanding the context of your software and identifying the stakeholders are also required when using agile methodologies and should applied accordingly.