Design is not a linear process. At the start of a design, it is rare for all the requirements to be known, least fully understood. Design requires a period of discovery and iteration, which incorporates learning, experimentation and even (early) failure. Design, by its very nature, is an emergent process.
The emergent nature of design requires a non-linear approach to development. The group of methods and techniques that adopt such an approach have traditionally been referred to as ‘Incremental and Iterative Development’ (IID), but more recently as agile development, or simply “Agile”.
In product design, agile development is well established. Ideas are sketched out and models built, long before any significant resources are committed to the development of the final product. And of course in other areas of design, including the creative arts, such approaches are the norm.
Unfortunately, in the case of services and particularly IT design and development, the use of agile methods and techniques is far less well established. Historically, a linear approach has been adopted, whereby requirements are defined, against which a solution is then developed and built – referred to as “Waterfall” development.
Waterfall adopts a pure linear approach to the design and development process. Each stage is defined, or delivered in full, before moving onto the next stage.
The development lifecycle typically passes through a series of ‘gates’ (approval stages). The work completed at each stage is considered complete and there is little room for iteration.
Unfortunately, what often gets delivered under Waterfall is frequently overly complex and costly, and fails to meet the actual needs of the customer. This is because the design has not had the opportunity to properly emerge – there has been no prototyping, no experimentation. It also takes much longer, because one stage has to complete before another stage can begin. This is another reason why the customer needs are not met – by the time it delivers something, the world has often moved on!
Despite its recognised weaknesses, Waterfall is still widely used. Its linear simplicity appeals to many, particularly managers and those responsible for procuring resources and agreeing finance. For some, the sense of occasionally going backwards, to go forwards, is an anathema.
Agile Development Approaches
Agile is not a single approach towards product, service or operational design and development, rather it is a collection of methods, that differ in their degree of agility and formality.
In selecting an appropriate method, or mix of methods, there are two primary factors that determine how agile a development should be – degree of emergence and degree of complexity.
The more emergent the requirements and the more complex the service, the more agile the development approach needs to be – to accommodate these two characteristics and to ensure a successful design.
I cover two methods here, one often nested within the other – Iterative Waterfall and Scrum. Iterative Waterfall covers the entire lifecycle, whereas Scrum is best suited to individual stages and well-bounded elements of the system.
As the name implies, Iterative Waterfall is the adaptation of the linear Waterfall approach to development, through the incorporation of a number agile techniques – Concurrent and Iteration Development; and Incremental delivery.
Concurrent Development. This is based on the principle that you can start a stage before the preceding stage has completed. The trigger is simply the point at which you have sufficient information to work on. This has two advantages – it reduces significantly the overall end-to-end development time and allows early iteration, even before the previous stage has completed, again greatly speeding up the process.
Concurrent development is found extensively in the engineering world, where it is known as Concurrent Engineering (CE). CE became standard engineering practice, certainly in the UK, from about the mid-80s.
Iterative Development. This is the application of ‘system feedback’ – taking what you have learnt in one stage and feeding it back into the earlier stage; and also refining the design back through the preceding stages. It is the mechanism by which the design evolves.
Most iteration occurs between adjacent stages, but it is quite possible to discover something, say at detailed design, that fundamentally changes some high level aspect of the design. Iteration in the early stages of the development lifecycle is achieved through the use of mock-ups and prototypes.
Incremental Delivery. This is the principle that you deliver something early and often. Typically you aim initially for a Minimum Viable Product (MVP) – the minimum amount of utility or functionality that allows you to launch the service and obtain customer feedback.
Incremental delivery can work across various dimensions, singularly or in combination, e.g. features, customer group and location. You may choose to deploy a service in only one location, for only certain customers and not supporting all the intended features.
Iterative Waterfall is best applied across the end-to-end design process, from Business Model through to the delivery of a service or service operating model; and for moderate degrees of emergence and complexity.
Iterative Waterfall is also well suited to situations where a standard design solution is being matched to a set of requirements; where the objective is to achieve the best match against the available options; as in the case when using Commercial Off-The-Shelf (COTS) software packages.
Scrum is an agile development framework that is specifically designed for those situations where the design requirements are not well understood, i.e. where they are emergent; and where there is a high degree of flexibility in the final design, e.g. when designing custom software solutions.
Although principally a software development method, Scrum can be applied more broadly. Also, Scrum does not cover the entire development lifecycle, rather it is intended to be used within individual stages, specifically for product development.
Scrum is one of those methods that gets a bit lost in its own jargon, but the key features are:
Multidisciplinary Teams. Scrum uses an integrated, team, with permanent members from the business working alongside the development team, ideally located in one place, to ensure continuous engagement. The team is ‘self-organising’, i.e. it collectively plans and controls the production process, it is non-hierarchical.
Sprints. These are the basic unit of development in Scrum. Sprints are time-boxed, i.e. restricted to a specific duration. Time-boxing is a common feature of all agile development methods. Sprints are typically between one week and a month in duration.
Scrums. These are short regular meetings, typically daily and performed standing up (to help keep them short), to discuss and coordinate the work across the development team, especially regarding overlap and integration.
Product Backlog. This is an ordered list of the items to be worked on, the sequence in which they will be worked; and an assessment of their business value. Items can be represented in many different forms, e.g. sub-products, user stories or simply a list of features, bug fixes and non-functional requirements.
Visual Control. Scrum borrows the concept of visual control used in Lean manufacturing. A task board, or “Kanban” (not to be confused with the method), is used to visualise the workflow, by showing: the Product Backlog; those items to be worked on in the next Sprint (Sprint Backlog); and those items included in the current Sprint. The current Sprint is broken done into the stages of work, e.g. design, build, deliver and completed.
Velocity. This is the development speed that a team is capable of in a Sprint; and provides a guideline for what can be achieved in future Sprints.
Implementing Agile Development
Agile development methods can be very difficult to introduce in traditional hierarchical organisations; and they are best introduced as part of a broader drive towards the creation of a more agile organisation. But also, every organisation is different. Find out what works for you. And if your organisation has only ever used Waterfall, introduce Iterative Waterfall first, before more agile methods such as Scrum. That in itself may require considerable change within your organisation, particularly if you are dependent on suppliers for your development capability. Learn to walk before attempting to run.
Allow for the use of different development approaches. Create one overall development framework, with various approaches within this, e.g. Iterative Waterfall, Scrum and Package-based Development, that can then be matched to a given project and/or product.
It is also important to understand the relationship between agile development methods and project management – you will often see agile development methods such as Scrum being described as a project management method, which they are not.
The development approach adopted (and its elements of coordination and control) needs to sit within an overall project management framework, e.g. for the management of the relationship and dependencies across other work-streams and projects. And this framework in turn needs to be able to accommodate the use of different development approaches.
If we use PRINCE2 as an example, this commonly used project management framework makes no assumptions about the development approach being used – it requires this to be defined; and then the specific application of PRINCE2 is defined around it, including the setting of appropriate project cost and time tolerances, reflecting the degree of uncertainty that exists.
And finally, Agile is not an excuse for adopting an unstructured approach to design and development. The opposite is true. Away from the simplicity of a linear approach, a great deal of structure, planning, discipline and emotional intelligence is required to make it work well.
Unfortunately Agile is one of those areas where people can become a bit evangelical and, ironically, introduce a lot of unnecessary complexity. The ‘signal to noise’ ratio is such that there are very few books and on-line articles that I can recommend.
From two of the co-creators of Scrum, a free on-line resource that provides the detail in a clear and well presented format. Better than many books on the subject.
A beginners guide to building Scrum task boards.
A stimulating article on the use of MVPs. You can adapt these ideas to many different types of service – they are not limited to the development of on-line services.
First Published: 24/05/2016 Last Updated: 27/08/2018