Chasing a vision
Throughout my experience as a software engineer – and having spent most of my career working as a consultant in client-facing roles – I have come to realise that companies usually look to software engineering for solutions to problems that don’t actually concern software engineering.
Developing software is easy for those who master the craft. As soon as you've got a good requirement and a sustainable cadence, it's just a matter of translating those requirements into lines of code.
Of course this is an oversimplification – especially in the context of tech-centric businesses where all the knowledge areas (such as business, design, tech and marketing) are all mixed up into interdisciplinary business units – but humour me on this one.
Finding the source of the issue
Assuming that we know how to write good software and release it to our end users, the real issues actually exist before the first line of code is written. These problems usually grow over time and penetrate software engineering processes, generating some of the pain points that are often discussed as being ‘contemporary software engineering issues’ when, in my humble opinion, they're business issues.
When I say the actual issues exist even before the first line of code is written, I'm referring to lack of alignment between everyone involved in the vision that they want to achieve.
It's not hard to understand the reasoning behind it since software engineering – like most digital products and services – is a complex and abstract area. Complexity in this case means that there are multiple correct solutions to the same problem, and abstract means that there's a lot of room for ambiguity when it comes to describing expectations.
Intuitively we can understand that it's simpler to communicate our vision when things are concrete, because you've got more tools to describe the expected output and therefore you rely less on other people’s interpretation. The fancy name for this is ‘a low representational gap’.
For example, if I want to build a car, I can create a super-detailed 3D model to describe how each physical component should work. Try to do that with a web page and you'll soon realize that it's harder than you think. The same applies to problems that only have a few possible solutions. It's easier to create alignment when there is only one possible path forward.
A simple answer to a complex problem
The actual output of software usually depends on other people's interpretations of a description of complex and abstract expectations. That alone can help you imagine how challenging our problem is.
Now, I'm not advocating for extensive documentation – been there, done that, and it will most likely fail. Plus, the fact you'll spend tons of time producing it and the problem still won't become less complex or less abstract.
The good news is that the solution to these problems is simple – but unfortunately it’s not easy. The solution is that people need to talk to each other.
Plain, simple conversations have been proven to be the best tool to create alignment over an abstract vision, helping to empower people so they can work autonomously to solve complex problems.
And as creating alignment over an abstract vision is the topic I want to cover in this article, let's jump into an overview of the processes and tools I've seen in use over the years.
Sequential or iterative?
In my experience, creating alignment is carried out either sequentially or iteratively. The main difference between these two approaches is not so much the order of things or how much time you spend on them, but rather how comfortable the team or organisation is with working in unpredictable environments.
The sequential approach relies on the assumption that dedicating time before jumping into building a new product or service will make the overall process quicker because you'll be able to fail faster before bootstrapping a new team or initiative which requires upfront investment. The premise is that you will ‘waste less’ carrying out activities like market research and prototyping before moving into active development. This is ultimately the idea of pursuing as much certainty as possible to ensure only relevant experiments are tested. The best-known example of this approach is the waterfall model of software development, but activities such as exhaustive market research before developing a new product are also examples of a linear, ‘waterfall’ approach.
Iterative approaches rely on the assumption that team dynamics play a strong role in product success. Therefore, allowing a team to work together as early as possible will allow them to fail and learn. This helps the team to develop a strong dynamic which maximises product success because when relevant opportunities present themselves, the team is mature enough to seize them. Design Sprint, Lean Inception and Lean Service Creation are examples of iterative approaches.
Why not both?
It might be appealing to choose one of the approaches and just leverage its benefits, but I truly believe that the greatest value is achieved when we understand and leverage the best of both approaches. This is what most successful digital organisations have been doing for a while: they run both approaches in parallel, creating an uninterrupted workflow based on continuous discovery. They rely on research and on stopping irrelevant experiments, creating delivery teams that understand their job is also to challenge assumptions and pursue facts. This leads to teams that are always experimenting and testing hypotheses rather than accepting any assumption as a fact.
In summary, these organisations adopt a research-oriented approach to reduce the number of experiments they need to carry out and then transition to iterative ways of working when developing an actual product. Regardless of the research carried out, successful organisations embrace the unpredictability that is a natural characteristic of digital products – especially consumer-facing ones.
The importance of eliminating irrelevant hypotheses is intuitive because it's tied to the elimination of waste, and this is the main benefit that a sequential approach brings. On the other hand, the iterative approach contributes to the development of two fundamental capabilities: resilience and agility.
Resilience and agility make teams good at pivoting and fearless when facing new challenges or opportunities. This ultimately leads to teams that are capable of maintaining and updating a product vision over time.
By building a vision that relies on both sequential and iterative development approaches we reduce the likelihood of wasting time and money, and also create innovative teams that develop products based on continuous learning and discovery.
- Caique PeixotoTech Advisor & Senior Software Engineer