What’s Happening to Agile?

Tech Talk

All Blog Posts

Search for ‘Agile’ on Medium — 4 of the top 5 results are titled something along the lines of “Why Agile is Bad”.

Personally, one of the things I found so compelling about the tech industry was the Agile Methodology. I wanted to dig into some of these frustrations a bit and look at what alternatives have been suggested.

What are the Common Complaints About Agile?

The list is wide and varied, but there are some issues that crop up more than others. Three common complaints are:

  • Not enough room for innovation
  • Strict time pressures affect quality
  • Continuous development of features for the sake of it

What is Agile all About?

Before we delve into the issues above, let’s take a look at what Agile is all about, so that we can understand whether the issues are genuinely to do with the methodology itself, or poor execution.

Traditionally, software development companies used the Waterfall Model. In a nutshell, this way of working means the customer sees the completed product all at once, at the end of production. The obvious problems this can lead to is the customer not getting entirely what they asked for and it being too late to change things by the end. Production times running massively over schedule is another common Waterfall issue, due to unforeseen issues which gradually push the deployment date back and back and back.

The Agile Manifesto was the result of what you might call a summit of the thought leaders in the software development world. Agile Methodology aims to solve these two common issues which arise when using the Waterfall Method by focussing on delivering the following…

Client Satisfaction

The first principle of the Agile Manifesto is…

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

The classic issue with Waterfall planning is that teams take too long to deliver work that wasn’t what the stakeholder wanted in the first place.

Agile forces us to fail fast by delivering early, and allowing for change. Teams welcome changing requirements and pivot to deliver what’s needed.

Better Planning

Since the dawn of development (and before), estimates of how long things take have been consistently, and demonstrably, wrong. Nobel prize winner Dan Kahneman actually wrote about it in his book Thinking, Fast and Slow. It’s called the Planning Fallacy.

We’re wrong because we base our estimates on the best-case scenario, which, naturally, never happens. This becomes a problem in a Waterfall scenario when work backs up so much the project live date begins to get pushed.

Agile can’t suddenly make us better at estimating, but by tight interval control, we can reduce the risk of estimates that miss the mark to a maximum impact of a 2-week sprint.

There are a number of different features of Agile, but for me, these are two of the biggest benefits.

So, What’s the Problem?

According to a Stack Overflow survey in 2018, 85.9% of 101,592 international surveyed software developers use an Agile process. It’s even making its way into a third of marketing teams.

The concept of Agile is no longer the cutting edge. It’s commonplace. So with almost everyone doing it, all interpreting Agile in their own ways, and using different frameworks (e.g. Scrum, Kanban, DSDM, Lean…. to name a few), not every company will be doing the same thing and not everyone will be getting it right the first time. That means right now, not everyone loves Agile.

Whether you agree or not, Agile is clearly a method that that lots of people in tech like the sound of. So where does it go wrong? Let’s take a look at those common complaints in more detail.

Not enough room for Innovation

One of the touted benefits of Agile is that it empowers teams to innovate and take ownership of work.

However, it seems like it doesn’t always work out that way though. Even with the high number of companies working Agile, many developers still report feeling dissatisfied.

Stakeholders invest time into their ideas, and will have their own vision for how their features should look and work. Often, requirements are so prescriptive that individuals working on the project can be left with little room to provide input to solutions.

(This article was originally published on our Medium account here, follow us on Medium for more technical articles.)

It’s a sure-fire recipe for disengaged colleagues, which also means worse outcomes for the end-user.

What is the alternative? The best teams have processes built-in to allow for innovation. In reality, this might look like a monthly innovation meeting where developers have a chance to suggest new ideas and explore new concepts together. It might look like a slightly less specific list of tasks, and more spike tasks. It might mean that developers are encouraged to take more of a consultative approach, where ideas and different suggestions are welcomed. Take a look at Pete Truran’s article Making a Good Team Great where he discussed psychological safety in teams, for more info on creating the best environment for sharing ideas.

Strict time pressures affect quality

With a set sprint (e.g. two weeks) to deliver a working increment, teams need to find the most efficient solution. This has numerous benefits — the customer gets to see working software quickly, and this can be built on in later releases. Good teams will also manage expectations effectively. Sometimes a feature will take longer than expected, and it’s up to the team to communicate this early. The risk with this strict timing, however, is that often the simplest solution wins.

This is all good until that simplest solution is not the best quality solution. When time pressures begin to affect the quality of work produced, the product owner is forced to choose whether to “accept” the feature, or delay other work that would come next. Left unchecked, this can lead to stakeholders and users feeling short-changed.

What can be done? The key to dealing with the time pressures of delivering solutions within a sprint is to simply not over-fil your sprints. It might be tempting when you are running a little behind, or you have stakeholders adding pressure to see results quickly, but squeezing more and more into a sprint is just going to put you further and further behind, and corners are going to be cut. Be realistic, expect to run into issues, and always keep the bigger picture in mind. Ask yourself — Is this the best solution for the customer? Or is it just the quickest? Which is more important in the big picture?

Increasing complexity

This post by John Cutler has a great explanation of how features become more time consuming to maintain the more they’re worked on, due to the added complexity that they accrue over time. It becomes more expensive to make smaller changes, due to the growing complexity of the system.

The majority of the benefit from a feature comes when it is first implemented, or in it’s first few iterations. But naturally, potential improvements arise and stakeholders compete for more functionality.

Using Agile, iterations come thick and fast. As layers of complexity are added, and more hours are invested in the feature, the incremental increases in benefit become smaller. Without being aware of this, you can end up putting in a huge amount of effort just to gain minimal benefit to the end-user.

So, how do we know when to stop? With every new iteration, take a step back. How does the effort compare to the benefits? Are we developing this just so we can show the customer new features? Or is this essential for the product? This, I guess, is the opposite of the previous issue of choosing efficiency over quality. Sometimes, in an effort to find quality, we go too far and throw too much complexity into the feature. It is about finding the middle ground, stopping to think about the benefits to the client, about looking at the big picture.

Not Convinced? What are the Alternatives?

I wanted to find some examples of teams shouting about a process they’ve found that really works and whether there are any viable alternatives. Two companies with their own effective processes are Hubspot and Basecamp.

Basecamp — “ShapeUp”

Basecamp’s head of strategy recently published an ebook on their dev process, where he detailed exactly how they plan and implement new features. They still work in iterations, even if they don’t call them sprints. But they’re 6 weeks, which is mammoth by today’s standards.

They steadfastly refuse to mandate any form of tracking software. Instead, they take pitches from stakeholders every 6 weeks and people track their ideas in whatever way makes sense to them. Sounds like an ISO nightmare, but you can’t argue with their work. Basecamp is great to use.

The really beautiful thing about Basecamp’s methodology is the idea of appetite. In their own words:

Instead of asking how much time it will take to do some work, we ask: How much time do we want to spend? How much is this idea worth? This is the task of shaping: narrowing down the problem and designing the outline of a solution that fits within the constraints of our appetite.

No estimates. Just a rough idea and a set amount of time for teams to come up with a solution and implement it. It’s a really interesting concept and got me wondering whether there were other companies that have developed their own process in the same way as Basecamp.

HubSpot — “Science Fair”

Another cool one I found was Hubspot. They found scrum too transactional and didn’t give developers/product owners enough room to innovate.

So they tried something pretty radical: They got rid of pretty much everything except the product demo.

Their methodology is called Science Fair. Essentially, every month, each team gets a chance to show the wider group what they’ve been working on for the past month, and how this work will affect the end-user. They’ve found that this builds trust between front line teams and leadership — management don’t seek to judge or criticise, and teams don’t try to hide their struggles and challenges.

The only ‘guideline’ that hasn’t changed over the years is that whatever is presented at Science Fair has to show real customer impact.

One more thing Hubspot have done to make these feel more collaborative is to introduce the idea of themes. This helps rally people around a common goal, rather than teams competing to have the best idea.

Conclusions

It’s clear that Basecamp and Hubspot have found processes that work for them. It’s also clear that some people experience issues with Agile. But let’s go back to those original principles for a second.

Here’s a snippet from agilemanifesto.org. It’s the opening paragraph of the original manifesto.

Note the distinct lack of any prescribed methodology, or any mention of sprints, backlog, and so on. There’s no one looking at that and saying “Nah. Documentation is more useful than working software.”

And neither is Basecamp or Hubspot. Maybe they don’t call it Agile, but by working out their own system, keeping the pieces that work and getting rid of inefficient processesare Basecamp, Hubspot and co. not following those principles in their own way?

I’d argue they are.

With nearly 90% of developers using Agile practices now, there are bound to be some places that don’t get it right straight away. And that’s where issues arise. Businesses trying to follow scrum to the letter, or using Agile as a tool to keep a closer eye on teams will struggle to see the benefit. But taking it back to basics, Agile principles are still as valid as they’ve ever been, and I’d wager will be for some time to come.

***

This article was originally published on our Medium account here, follow us on Medium for more technical articles.

***

To learn more about Harry and the rest of our team, take a look our staff profiles in About Us.


Everything You Need to Know About Choosing a Learning Management System

What is ‘Software Architecture and Infrastructure’ and Why Does it Matter?

Back to All Blog Posts