Automation practices in Agile Software Development have an important role. But what about Agile outside IT? What to automate? How to automate?
In all successful organisations and effective teams I’ve been part of, while adopting Agile Software Development, automation practices always played a fundamental role. Both in terms of speed, quality, and safety.
Those that immediately come to my mind are Automated Testing, Continuous Integration (with Trunk Based Development), Continuous Delivery (with automated remediation plans), and DevOps.
Automation practices are so important that even when Lean and/or Agile are adopted outside IT, their fundamental benefit cannot be ignored.
All the tasks that involve creating new solutions, designing new things, and more in general using human creativity and ingenuity, are not good candidates for automation. Nor are those tasks that involve exploring, experimenting, discovering, guessing, and more in general those that require using human imagination, intuition, and insights.
Also, all the tasks that involve making sense of things collectively, building and reaching consensus, dealing with human beliefs and values and principles, dealing with power or emotions, taking important decisions, they are not good candidates for automation either.
All these tasks deeply involve what I call the human element, and that is why for them a real person needs to be in control, be in the loop, make the final call.
Before Agile Software Development, automation was often approached with a big-bang top-down large initiative. Or trying to buy silver bullet tools promising miraculous results.
The new approach to automation of Agile Software Development has been a huge leap forward, and very successful. Automation has been driven and created by those doing the work. Starting small and gradually evolving custom made solutions. The role of the tools has been that of a supporting role, leaving to those doing the work the protagonist role they need to make it work.
Here you can find my suggestions on Agile technical practices outside IT, that are relevant also for automation practices outside IT: Agile Technical Practices outside IT.
Step 1 – Documentation
Once a candidate task (usually a repetitive, time consuming, low-variation task) has been selected, the first step is to document it for someone else to carry out the task manually, following the documentation.
This will allow others in and out of the team, with limited knowledge of the details of the task, to carry out the task anyway. You can think for example about the difference between someone knowing how to create a formula and its proof Vs someone knowing only how to apply the given formula. The latter is all that is needed to follow the documentation.
The tool used to document the steps should be like a Wiki, a tool that doesn’t make a distinction between the author and the readers, instead everyone is an active reader (both author and reader at the same time). Those using the documentation to carry out the task will ask questions to fill the gaps and to clarify any ambiguity in the documentation, and so evolving, updating and improving it.
It is important that this documentation is valuable in itself and it is actually used and useful. This will lead to improve the documentation and will prove that there is a need for help to carry out the task, and that the delegation of the task to others actually works. And so that there are actual changes to succesfully automate it.
Step 2 – Scripting (checklist / flow-diagram)
The next step is to streamline the documentation, making the instructions simpler and more straightforward. Eliminating any part that can be subjective. Providing clear unequivocal criteria for every decision that needs to be taken. Bringing down almost to zero the level of interpretation and the judgment calls that the reader needs to make in order to carry out the task. Providing clear instructions also to deal with error conditions that can happen on the way, as well as to deal with variations that there may be in the inputs to the task.
This format of this streamlined documentation is much easier to follow, leave much less room for error, requires much less knowledge from the reader. So if it really works, a larger group of people will be able to follow its instructions.
As before, it is important that this documentation is valuable in itself and it is actually used and useful. This will prove that there are not too many variations in the task that make it not a good fit to be automated. And that the instructions to carry out the task have an actual chance to be automated effectively (by an algorithm, a smart robot with physical abilities, or basic cognitive abilities that can be trained into commercially available AI).
Step 3 – The actual automation
After the scripted documentation from the previous step has been proved to work well repeatedly, without tweaks or non-scripted interventions, then it is time for the actual automation.
The process of automation ofter requires that part of the resources (raw material, or artefacts) used in carrying out the task follow some process of standardisation to reduce variation and make the automation possible and easier.
This step is the proof that the automation is actually feasible, that it works well, that is stable and sustainable, and that is beneficial and effective. The automation should be done by those initially responsible for carrying out the task, the experts. Their understanding and knowledge are fundamental in judging the solidity of the automation and its sustainability.
Because the work we do and the way we work is in constant evolution, also the automated tasks will evolve over time, this is why the skills to maintain and evolve the automation introduced should be embedded and available inside the team.
The smaller is the initial task to be automated, the faster this step is reached, and the sooner it can be determined if it really can be automated successfully.
Common reasons for a task automation to fail or to be unsuccessful are: some part of the task is too hard/expensive to automate; human judgment cannot be removed from the task; variations in the task inputs or outputs cannot be removed without losing to much value.
In all the other cases, automation will provide a significant increase in speed, a significant reduction in human errors, and a significant increase in the volume of work that can be carried out. We are talking about several orders of magnitude.