Guidelines for building evolutionary architectures

[...] As much as we like to talk about architecture in pristine, idealized settings, the real world often exhibits a contrary mess of technical debt, conflicting priorities, and limited budgets. Architecture in large companies is built like the human brain — lower-level systems still handle critical plumbing details but have some old baggage. Companies hate to decommission something that works, leading to escalating integration architecture challenges.

Retrofitting evolvability into an existing architecture is challenging — if developers never built easy change into the architecture, it is unlikely to appear spontaneously. No architect, no matter how talented, can transform a Big Ball of Mud into a modern microservices architecture without immense effort. Fortunately, projects can receive benefits without changing their entire architecture by building some flexibility points into the existing one.

Remove needless variability

One of the goals of Continuous Delivery is stability — building on known good parts. A common manifestation of this goal is the modern DevOps perspective on building immutable infrastructure.

While immutability may sound like the opposite of evolvability, quite the opposite is true. Software systems comprise thousands of moving parts, all interlocking in tight dependencies. Unfortunately, developers still struggle with unantecipated side effects of changes to one of those parts. By locking down the possibility of unantecipated change, we control more of the factors that make systems fragile. Developers strive to replace variables in code with constants to reduce vectors of change. DevOps introduced this concept to operations, making it more declarative.

Immutable infrastructure follows our advice to remove needless variables. Building software systems that evolve means controlling as many unknown factors as possible.

Make decisions reversible

Many DevOps practices exist to allow reversible decisions — decisions that need to be undone. For example, blue/green deployments, where operations have two identical (probably virtual) ecosystems — blue and green ones — common in DevOps.

Feature toggles are another common way developers make decisions reversible. By deploying changes underneath feature toggles, developers can release them to a small subset of users (called canary releasing) to vet the change. [...] Make sure you remove the outdated ones!

[...] Service routing — routing to a particular instance of a service based on request context — is another common method to canary release in microservices ecosystems.

Make as many decisions as possible reversible (withou over-engineering).

Prefer evolvable over predictable

Unknown unknowns are the nemesis of software systems. Many projects start with a list of known unknowns: things developers know they must learn about the domain and technology. However, projects also fall victim to unknown unknowns: things no one knew were going to crop up yet have appeared unexpectedly. This is why all Big Design Up Front software efforts suffer — architects cannot design for unkown unknowns.

While no architecture can survive the unknown, we know that dynamic equilibrium renders predictability useless in software. Instead, we prefer to build evolvability into software: if projects can easily incorporate changes, architects don't need a crystal ball.

Build anticorruption layers

Projects often need to couple themselves to libraries that provide incidental plumbing: message queues, search engines, and so on. The abstraction distraction antipattern describes the scenario where a project "wires" itself too much to an external library, either commercial or open source. Once it becomes time for developers to upgrade or switch the library, much of the application code utilizing the library has backed-in assumptions based on the previous library abstractions. Domain-driven design includes a safeguard against this phenomenon called an anticorruption layer.

Agile architects prize the last responsible moment principle when making decisions, which is used to counter the common hazard in projects of buying complexity too early.

[...] Most developers treat crufty old code as the only form of technical debt, but projects can inadvertently buy technical debt as well via premature complexity.

In the last responsible moment answer questions such as "Do I have to make this decision now?", "Is there a way to safely defer this decision without slowing any work?", and "What can I put in place now that will suffice but I can easily change later if needed?"

[...] Building an anticorruption layer encourages the architect to think about the semantics of what they need from the library, not the syntax of the particular API. But this is not an excuse to abstract all the things! Some development communities love preemptive layers of abstraction to a distracting degree but understanding suffers when you must call a Factory to get a proxy to a remote interface to a Thing. Fortunately, most modern languages and IDEs allow developers to be just in time when extracting interfaces. If a project finds themselves bound to an out-of-date library in need of change, the IDE can extract interface on behalf of the developer, making a Just In Time (JIT) anticorruption layer.

Controlling the coupling points in an application, especially to external resources, is one of the key responsibilities of an architect. Try to find the pragmatic time to add dependencies. As an architect, remember dependencies provide benefits but also impose constraints. Make sure the benefits outweigh the cost in updates, dependency management, and so on.

Architects must understand both benefits and tradeoffs and build engineering practices accordingly.

Using anticorruption layers encourages evolvability. While architects can't predict the future, we can at least lower the cost of change so that it doesn't impact us so negatively.

Build sacrificial architectures

[...] At an architectural level, developers struggle to anticipate radically changing requirements and characteristics. One way to learn enough to choose a correct architecture is build a proof of concept. Martin Fowler defines a sacrifical architecture as an architecture designed to be thrown away if the concept proves successful.

Many companies build a sacrificial architecture to achieve a minimum viable product to prove a market exists. While this is a good strategy, the team must eventually allocate time and resources to build a more robust architecture [...].

One other aspect of technical debt impacts many initially successful projects, elucidated again by Fred Brooks, when he refers to the second system syndrome — the tendency of small, elegant, and successful systems to evolve into giant, feature-laden monstrosities due to inflated expectations. Business people hate to throw away functioning code, so architecture tends toward always adding, never removing, or decomissioning.

Technical debt works effectively as a metaphor because it resonates with project experience, and represents faults in design, regardless of the driving forces behind them. Technical debt aggravates inappropriate coupling on projects — poor design frequently manifests as pathological coupling and other antipatterns that make restructuring code difficult. As developers restructure architecture, their first step should be to remove the historical design compromises that manifest as technical debt.

Mitigate external change

Most projects rely on a dizzying array of third-party components, applied via build tools. Developers like dependencies because they provide benefits, but many developers ignore the fact that they come with a cost as well. When relying on code from a third party, developers must create their own safeguards against unexpected occurrences: breaking changes, unannounced removal, and so on. Managing these external parts of projects is critical to creating evolutionary architecture.

We recommend that developers take a more proactive approach to dependency management. A good start on dependency management models external dependencies using a pull model. For example, set up an internal version-control repository to act as a third-party component store, and treat changes from the outside world as pull requests to that repository. If a beneficial change occurs, allow it into the ecosystem. However, if a core dependency disappears suddenly, reject that pull request as a destabilizing force.

Using a Continuous Delivery mindset, the third-party component repository utilizes its own deployment pipeline. When an update occurs, the deployment pipeline incorporates the change, then performs a build and smoke test on the affected applications. If successful, the change is allowed into the ecosystem. Thus, third-party dependencies use the same engineering practices and mechanisms of internal development, usefully blurring the lines across this often unimportant distinction between in-house written code and dependencies from third parties — at the end of the day, it's all code in a project.

Updating libraries versus frameworks

Because frameworks are a fundamental part of applications, teams must be aggressive about pursuing updates. Libraries generally form less brittle coupling points than frameworks do, allowing teams to be more casual about upgrades. One informal governance model treats framework updates as push updates and library updates as pull updates. When a fundamental framework (one whose afferent/efferent coupling numbers are above a certain threshold) updates, teams should apply the update as soon as the new version is stable and the team can allocate time for the change. Even though it will take time and effort, the time spent early is a fraction of the cost if the team perpetually procrastinates on the update.

Because most libraries provide utilitarian functionality, teams can afford to update them only when new desired functionality appears, using more of an "update when needed" model.

Prefer continuous delivery to snapshots

Continuous Delivery suggested a more nuanced way to think about dependencies, repeated here. Currently, developers only have static dependencies, linked via version numbers captured as metadata in a build file somewhere. However, this isn't sufficient for modern projects, which need a mechanism to indicate speculative updating. Thus, as the book suggests, developers should introduce two new designations for external dependencies: fluid and guarded. Fluid dependencies try to automatically update themselves to the next version, using mechanisms like deployment pipelines. For example, say that order fluidly relies on version 1.2 of framework. When framework updates itself to version 1.3, order tries to incorporate that change via its deployment pipeline, which is set up to rebuild the project anytime any part of it changes. If the deployment pipeline runs to completion, the fluid dependency between the components is updated. However, if something prevents successful completion — failed test, broken diamond dependency, or some other problem — the dependency is updated to a guarded reliance on framework1.2, which means the developer should try to determine and fix the problem, restoring the fluid dependency. If the component is truly incompatible, developers create a permanent static reference to the old version, eschewing future automatic updates.

None of the popular build tools support this level of functionality yet — developer must build this intelligence atop existing build tools. However, this model of dependencies works extremely well in evolutionary architectures, where cycle time is a critical foundational value, being proportional to many other key metrics.

Version services internally

[...] Developers use two common patterns to version endpoints, version numbering or internal resolution. For version numbering, developers create a new endpoint name, often including the version number, when a breaking change occurs. [...] The alternative is internal resolution, where callers never change the endpoint — instead, developers build logic into the endpoint to determine the context of the caller, returning the correct version. The advantage of retaining the name forever is less coupling to specific version numbers in calling applications.

In either case, severely limit the number of supported versions. The more versions, the more testing and other engineering burdens. Strive to support only two versions at a time, and only temporarily.

Neal Ford, Rebecca Parsons & Patrick Kua, "Building Evolvable Architectures", in Building Evolutionary Architectures: Support Constant Change, 107-119.