Design is not a linear process. At the start of a design, it is rare for all the requirements to be known, least fully understood, nor how they might be addressed. Design requires a period of discovery and a continuous process of incremental iteration, which incorporates learning, experimentation and even (early) failure. Design, by its very nature, is an emergent process.
The methods and techniques associated with this emergent process are numerous, some new and some not so new. They cover four broad areas: (1) the sense-making and creative design activities themselves; (2) the coordination and management of these activities; (3) where applicable, the coordination and management of the activities across different work streams; and (4) the management of the lifecycle of the ‘thing’ being designed, from creation through to redesign or retirement. The required mindset, methods and techniques are sometimes collectively referred to as “Iterative, Incremental Development (IID)”, or more commonly today as “Agile Development”.
In product design, agile development is well established, to the extent that it doesn’t really have a name – mention “agile” and you will probably get a blank look. Ideas are sketched out and models built, long before any significant resources are committed to the development of the final product. Product lifecycle management is also a well established discipline.
Unfortunately, in the case of services and service design such practices are far less well established. Historically a great deal of ‘service design’ has been technology driven, i.e. the service was indirectly designed through the specification and creation of an IT solution. And in the world of IT, there has historically been a strong tendency to work in a much more linear fashion, largely driven by the use of external suppliers and the need to manage costs. Typically an IT project would be spun up, a set of requirements would be defined by the customer upfront, against which a solution would then be developed and built. An approach which is often referred to as “waterfall development”.
Fortunately in recent years, the more formal design of services has taken root, challenging the old technology driven approaches and introducing improved ways of working, by taking the (agile) design practices found in the product world and applying them to the design of services. And at the same time the world of IT has also changed, with the greater adoption of more agile development approaches, a shift that was driven by the creation of the Agile Manifesto in 2001 (from whence the use of the term “agile” comes from) and the Agile Alliance in the same year.
You would expect these two changes to have created a perfect marriage. But despite the move to more agile ways of working within IT, this has not removed the tendency to focus on the design of the technology rather than the service/s it is enabling. If anything, certain agile software development practices (or at least their poor implementation) have actually made the situation worse.
Before discussing agile development further, it’s worth looking at waterfall development in a bit more detail and the problems it can create.
Waterfall Development
Waterfall adopts a pure linear approach to the design and development process. Each stage is defined, or delivered in full, before moving onto the next stage.
The development lifecycle typically passes through a series of “control gates”, or “approval stages”. The work completed at each stage is considered complete and there is very little room for iteration.
Unfortunately what often gets delivered under Waterfall is frequently overly complex and costly, and fails to meet the actual needs of the customer. This is because the design has not had the opportunity to properly emerge – there has been no prototyping, no experimentation. It also takes much longer, because one stage has to complete before another stage can begin. This is another reason why the customer needs are not met – by the time it delivers something, the world has often moved on.
Despite its recognised weaknesses, Waterfall is still widely used, where service design is still being largely driven by the design of the enabling technology. Its linear simplicity appeals to many, particularly managers and those responsible for procuring resources and agreeing finance. For some, the sense of occasionally going backwards, to go forwards, is an anathema.
Agile Development Approaches
As emphasised earlier, agile development is not a single approach towards product, service or operational design and development. Rather it should be viewed foremost as a mindset. Against this mindset there are a wide range of applicable methods and techniques, each appropriate to a given context and that differ in their degree of agility and formality.
An agile development may be project or continuous delivery based, i.e. rather than discrete change projects, a service is developed and maintained by a permanent team, delivering changes in a continuous flow. The project or continuous delivery team may be semi-autonomous, taking collective responsibility for what is worked on and how. A change programme may have a mixture of conventional project-based work-streams and continuous delivery work-streams.
In selecting an appropriate method, or mix of methods, there are two primary factors that determine how ‘agile’ a development needs to be – the degree of complexity involved and as a consequence, the degree of emergence.
The more complex the service, the more effort needs to be applied to understanding the design space, or ‘bounded context’, and the greater degree to which requirements need to be allowed to emerge, e.g. through probing, the testing of hypotheses, the use of mock-ups and the building of prototypes.
A useful tool to help determine the degree of complexity that exists, and the right design approach to adopt, is the Cynefin decision support framework. This makes a distinction between “clear”, “complicated”, “complex” or “chaotic” bounded contexts, all of which require different design approaches, from the simple adoption of well established best practice through to the need to adopt more novel approaches.
With greater complexity also comes greater uncertainty. A long established technique used to express the level of uncertainty that exists, found for example in classic approaches to project management, is the “Cone of Uncertainty”. The wider the cone, the greater uncertainty exists as you move forward in time, again increasing the degree of agility required, e.g. in technical approach, planning, budget setting and forms of supplier contract.
For the purposes of illustration, I will now cover three basic agile methods for the coordination and management of the work, often nested within each other – “Iterative Waterfall”, “Double Diamond” and “Scrum”, applied to a moderately complex service operation and set of delivered services.
Iterative Waterfall (not to be confused with the pure form discussed earlier) covers the general management of the entire development lifecycle, whereas the Double Diamond framework and Scrum are intended for the individual stages and well-bounded elements of the work. Within this overall management framework, creative agile design methods and techniques are applied, appropriate to the design task in hand.
Iterative Waterfall
As the name implies, Iterative Waterfall is the adaptation of a pure linear approach to development, through the incorporation of several agile techniques – concurrent development, iterative development and incremental delivery:
Concurrent Development. This is based on the principle that you can start a stage before the preceding stage has completed. The trigger is simply the point at which you have sufficient information to work with. This has two advantages – it reduces significantly the overall end-to-end development time and allows early iteration, even before the previous stage has completed, again greatly speeding up the process.
Concurrent development is found extensively in the engineering world, where it is known as “Concurrent Engineering” (CE). CE became standard engineering practice, certainly in the UK, from about the mid-1980s.
Iterative Development. This is the application of ‘system feedback’ – taking what you have learnt in one stage and feeding it back into the earlier stage; and also refining the design back through the preceding stages. It is the mechanism by which the design evolves.
Most iteration occurs between adjacent stages, but it is quite possible to discover something, say at detailed design, that fundamentally changes some high level aspect of the design. Iteration in the early stages of the development lifecycle is achieved through the use of mock-ups and prototypes.
Incremental Delivery. This is the principle that you deliver something early and often, learning continuously as you go. Typically you aim initially for a ‘minimum viable product’ (MVP) – defined as the minimum amount of utility or functionality that allows you to expose the service to a customer and obtain feedback. Hopefully you find yourself on the right track, if not, it provides the opportunity to fail early, learn and re-orientate.
Incremental delivery can work across various dimensions, singularly or in combination, e.g. features, customer group and location. You may choose to deploy a service in only one location, for only certain customers and without all the intended features.
Iterative Waterfall is intended to be applied across the end-to-end design process, from Business Model through to service operating model design and the design of the individual services; and for moderate degrees of emergence and complexity.
Double Diamond
The “Double Diamond” framework, developed by the Design Council in 2004 provides a very useful general approach to framing the work that needs to be done within each design stage, or across adjacent stages, e.g. the development of the business model, high level operating model design and the design of the individual services.
The two diamonds, in it’s standard form, represent two design phases – “Discover and Define” and “Develop and Deliver”. The diamond representation emphasises that these two phases are non-linear, passing through a cycle of divergent and convergent thinking.
The Double Diamond framework is very adaptable. You can frame any phase of design as a divergent/convergent diamond. For example, I most often use it to separately frame discovery and define, effectively using three diamonds instead of two. And you can also imagine it as a continuous chain, i.e. cycles of evolutionary development, existing throughout the lifecycle of the product or service.
Scrum
Scrum is an agile workflow management framework that is specifically designed for those situations where the design requirements are not well understood, i.e. where they are emergent; and where there is a high degree of flexibility in the final design, e.g. when designing custom software solutions.
Although developed originally as an agile software development method, Scrum can be applied more broadly to the design of services. Also, Scrum does not cover the entire development lifecycle, rather it is intended to be used within individual stages.
Scrum is one of those methods that gets a bit lost in its own jargon, but the key features are:
Multidisciplinary Teams. Scrum uses an integrated, team, with permanent members from the business working alongside the development team, ideally located in one place, to ensure continuous engagement. The team is semi-autonomous, i.e. it collectively plans and controls the production process, it is non-hierarchical.
Sprints. These are the basic unit of development in Scrum. Sprints are time-boxed, i.e. restricted to a specific duration. Time-boxing is a common feature of many agile development methods. Sprints are typically between one week and a month in duration.
Scrums. These are short regular meetings, typically daily and performed standing up (to help keep them energetic and short), to collectively discuss and coordinate the work across the development team, especially regarding overlap and integration.
Product Backlog. This is an ordered list of the items to be worked on, the sequence in which they will be worked; and an assessment of their business value. Items can be represented in many different forms, e.g. sub-products, user stories or simply a list of service features, bug fixes and non-functional requirements.
Visual Control. Scrum borrows the concept of visual control used in Lean manufacturing. A task, or “Kanban Board”, is used to visualise the workflow, by showing: the Product Backlog; those items to be worked on in the next Sprint (Sprint Backlog); and those items included in the current Sprint. The current Sprint is broken done into the stages of work, e.g. design, build, deliver and completed.
Using Scrum, like any method, needs some care. Done badly, it can seriously impede the design process. Probably the biggest problem is associated with Sprints and the use of a fixed sprint duration within and across work-streams, irrespective of the nature of the work being undertaken. A very long established principle, in classic project management, is that the technical approach drives the management approach, not the other way around. The length of a sprint should be selected based on optimising the flow of work for the design activity in hand.
Implementing Agile Development
Agile development methods can be very difficult to introduce in traditional hierarchical organisations. This is best done as part of a broader drive towards the creation of a more agile organisation. But also, every organisation is different. You need to find out what works for you and this will change over time as the organisation adopts new ways of working. And if your organisation has only ever used Waterfall, Iterative Waterfall is a good place to start, to get the basic iterative and incremental development principles in place. That in itself may require considerable change within your organisation, particularly if you are dependent on suppliers. Agile development does not work well with fixed priced contracts. Budgeting practices also typically need to change, with the introduction of greater flexibility and agility in the allocation of funds to projects or continuous delivery service development pipelines.
It is important to allow for the use of different development approaches. Create one overall development framework, with various approaches within this, that can then be matched to a given design context. I would tend to avoid the so called scaled agile frameworks that have appeared in recent years. These tend to be restricting rather than liberating, almost representing the polar opposite of agile ways of working.
And finally, agile development is not an excuse for adopting an unstructured approach to design and development. The opposite is true. Away from the simplicity of a linear approach, a great deal of structure, planning, discipline and ’emotional intelligence’ is required to make it work well.
Further Reading….
Unfortunately agile development is one of those areas where people can become a bit evangelical and, ironically, introduce a lot of unnecessary complexity. Also, the dominate nature of IT means that a lot of books and articles are focused on agile software development. This is useful in its own right but would be even more useful if the material that has more general applicability was written with this in mind. Instead, the reader is often left to do this translation themselves.
A further problem is that few people make any connection with the agile development methods long applied beyond the world of IT. So quite a lot of the material can come across as rather naive, and/or a poor rework of long established theories and practices. A prime example of this is the area of Product Management.
Craig Larman, “Agile & Iterative Development”
Provides quite a nice history of IID and agile development, as it applies to software development; and a summary of the early methods and techniques. It’s got a nice bit on the origins of “Waterfall”. A classic tale of misinterpretation in application. A bit dated now, but still well worth a read.
Hirotaka Takeuchi and Ikujiro Nonaka, “The New New Product Development Game”, HBR, 1986
This article from 1986 describes the development of concurrent engineering methods and shows how many of the practices we more closely associate with the IT industry and Agile, have in fact been established for decades elsewhere, in other product-based industries.
From two of the co-creators of Scrum, a free on-line resource that provides the detail in a clear and well presented format. Better than many books on the subject. I would read the 2017, as well as the latest 2020 edition. Some of the recent changes seem to me to be a bit of a backward step.
David J. Anderson, “Kanban: Successful Evolutionary Change for Your Technology Business”
This is about the method, not the visual management technique of the same name. An alternative to Scrum. It has the advantage of not using fixed time-boxed “Sprints”, which in many contexts is more appropriate, e.g. within complex change programmes.
A beginners guide to building Scrum task or Kanban boards.
A stimulating article on the use of MVPs. You can adapt these ideas to many different types of service – they are not limited to the development of on-line services.
Jonathan Smart, “Sooner, Safer, Happier”
A great book on the mindset (both of the individual and organisation) required for the successful adoption of agile development. Again, principally written in the context of software development but equally applicable to the design of services. That said, it does tend to fall into the usual trap of focusing on the software product, rather than on the design of the service or services it is enabling.
First Published: 24/05/2016 Last Updated: 29/12/2021