Advices on refactoring

Brevity is the soul of wit, but clarity is the soul of evolvable software.

Martin Fowler, "Refactoring: A First Example", in Refactoring: Improving the Design of Existing Code (2nd Edition), 33.

I always have to strike a balance between all the refactorings I could do and adding new features. At the moment, most people underprioritize refactoring — but there still is a balance. My rule is a variation on the camping rule: Always leave the code base healthier than when you found it. It will never be perfect, but it should be better.

Martin Fowler, "Refactoring: A First Example", in Refactoring: Improving the Design of Existing Code (2nd Edition), 34.

I'm talking about improving the code — but programmers love to argue about what good code looks like. [...] If we consider this to be a matter of aesthetics, where nothing is either good or bad but thinking makes it so, we lack any guide but personal taste. I believe, however, that we can go beyond taste and say that the true test of good code is how easy it is to change it.

Martin Fowler, "Refactoring: A First Example", in Refactoring: Improving the Design of Existing Code (2nd Edition), 43.

The key to effective refactoring is recognizing that you go faster when you take tiny steps, the code is never broken, and you can compose those small steps into substantial changes.Remeber that — and the rest is silence.

Martin Fowler, "Refactoring: A First Example", in Refactoring: Improving the Design of Existing Code (2nd Edition), 44.

Putting evolutionary architecture into practice

Organizational factors

Teams structured around domains rather than technical capabilities have several advantages when it comes to evolutionary architecture and exhibit some common characteristics.

• Cross-functional teams: the goal of a domain-centric team is to eliminate operational friction. In other words, the team has all roles needed to design, implement, and deploy their service, including traditionally separate roles like operations.
• Organized around business capabilities: many architectural styles of the past decade focused heavily on maximizing shared resources because of expense. Shared resource architecture has inherent problems around inadvertent interference between parts. Now that developers have the option of creating custom-made environments and functionality, it is easier for them to shift emphasis away from technical architectures and focus more on domain-centric ones to better match the common unit of change in most software projects.
• Product over project: product teams take ownership of quality metrics and pay more attention to defects. This perspective also helps provide long-term vision to the team.
• Dealing with external change: a common practice in microservices architectures is the use of consumer-driven contracts, which are atomic fitness functions. Maintaining integration protocol consistency shouldn't be done manually when it is easy to build fitness functions to handle this chore. Using engineering practice to police practices via fitness functions relieves lots of manual pain from developers but requires a certain level of maturity to be successful.
• Connections between team members: the motivation to create small teams revolves around the desire to cut down on communication links. Each team shouldn't have to know what other teams are doing, unless integration points exist between the teams. Even then, fitness functions should be used to ensure integrity of integration points.

Team coupling characteristics

Most architects don't think about how team structure affects the coupling characteristics of the architecture, but it has a huge impact.

• Culture: well-functioning architects take on leadership roles, creating the technical culture and designing approaches for how developers build systems. Adjusting the behavior of the team often involves adjusting the process around the team, as people respond to what is asked of them to do.

  Tell me how you measure me, and I will tell you how I will behave.

  Dr. Eliyahu M. Goldratt (The Haystack Syndrome)

• Culture of experimentation: successful evolution demands experimentation, but some companies fail to experiment because they are too busy delivering to plans. Successful experimentation is about running small activities on a regular basis to try out new ideas (both from a technical and product perspective) and to integrate successful experiments into existing systems.

CFO and budgeting

In an evolutionary architecture, architects strive to find the sweet spot between the proper quantum size and the corresponding costs. As we face an ecosystem that defies planning, many factors determine the best match between architecture and cost. This reflects our observation that the role of architect has expanded: architectural choices have more impact than ever. Rather than adhere to decades-old "best practices" guides about enterprise architecture, modern architects must understand the benefits of evolvable systems along with the inherent uncertainty that goes with them.

Where do you start?

While appropriate coupling and using modularity are some of the first steps you should take, sometimes there are other priorities. For example, if your data schema is hopelessly coupled, determining how DBAs can achieve modularity might be the first step.

• Low-hanging fruit: if an organization needs an early win to prove the approach, architects may choose the easiest problem that highlights the evolutionary architecture approach. Generally, this will be part of the system that is already decoupled to a large degree and hopefully not on the critical path to any dependencies. If teams use this effort as a proof-of-concept, developers should gather appropriate metrics for both before and after scenarios. Gathering concrete data is the best way to for developers to vet the approach: remember the adage that demonstration defeats discussion. This "easiest first" approach minimizes risk at the possible expense of value, unless a team is lucky enough to have easy and high value align.
• Highest-value: an alternative approach to "easiest first" is "highest value first" — find the most critical part of the system and build evolutionary behavior around it first.
• Testing: if developers find themselves in a code base with anemic or no testing, they may decide to add some critical tests before undertaking the more ambitious move to evolutionary architecture. Testing is a critical component to the incremental change aspect of evolutionary architecture, and fitness functions leverage tests aggressively. Thus, at least some level of testing enables these techniques, and a strong correlation exists between comprehensiveness of testing and ease of implementing an evolutionary architecture.
• Infrastructure: for companies that have a dysfunctional infrastructure, getting those problems solved may be a precursor to building an evolutionary architecture.

Ultimately, the advice parallels the annoying-but-accurate consultant's answer of It Depends! Only architects, developers, DBAs, DevOps, testing, security, and the other host of contributors can ultimately determine the best roadmap toward evolutionary architecture.

Why should a company decide to build an evolutionary architecture?

• Predictable versus evolvable: many companies value long-term planning for resources and other strategic matter; companies obviously value predictability. However, because of the dynamic equilibrium of the software development ecosystem, predictability has expired. Building evolvable architecture takes extra time and effort, but the reward comes when the company can react to substantive shifts in the marketplace without major rework. The highly volatile nature of the development world increasingly pushes all organizations toward incremental change.
• Scale: any coupling point in an architecture eventually prevents scale, and relying on coordination at the database eventually hits a wall. Inappropriate coupling represents the biggest challenge to evolution. Building a scalable system also tends to correspond to an evolvable one.
• Advanced business capabilities: many companies look with envy at Facebook, Netflix, and other cutting-edge technology companies because they have sophisticated features. Incremental change allows well-known practices such as hypotheses and data-driven development.
• Cycle time as a business metric: building continuous deployment takes a fair amount of engineering sophistication — why would a company go quite that far? Because cycle time has become a business differentiator in some markets. Some large conservative organizations view software as overhead and thus try to minimize cost. Innovative companies see software as a competitive advantage. Many companies have made cycle time a first-class business metric, mostly because they live in a highly competitive market. All markets eventually become competitive in this way.
• Isolating architectural characteristics at the quantum level: a common problem in highly coupled architectures is inadvertent overengineering. In a more coupled architecture, developers would have to build scalability, resiliency, and elasticity into every service, complicating the ones that don't need those capabilities. Architects are accustomed to choosing architectures against a spectrum of trade-offs. Building architectures with clearly defined quantum boundaries allows exact specification of the required architectural characteristics.
• Adaptation versus evolution: many organizations fall into the trap of gradually increasing technical debt and reluctance to make needed restructuring modifications, which in turns makes system and integration points increasingly brittle. Companies try to pave over the brittleness with connection tools like service buses, which alleviates some of the technical headaches but doesn't address deeper logical cohesion of business processes. Using a service bus is an example of adapting an existing system to use in another setting. But as we've highlighted previously, a side effect of adaptation is increased technical debt. When developers adapt something, they preserve the original behavior and layer new behavior alongside it. The more adaptation cycles a component endures, the more parallel behavior there is, increasing complexity, hopefully strategically. The use of feature toggles offers a good example of the benefits of adaptation. Often, developers use toggles when trying several alternate alternatives via hypotheses-driven dvelopment, testing their users to see what resonates best. In this case, the technical debt imposed by toggles is purposeful and desirable. Of course, the engineering best practices around these types of toggles is to remove them as soon as the decision is resolved. Alternatively, evolving implies fundamental change. Building an evolvable architecture entails changing the architecture in situ, protected from breakages via fitness functions. The end result is a system that continues to evolve in useful ways without an increasing legacy of outdated solutions lurking within.

Why would a company choose not to build an evolutionary architecture?

We don't believe that evolutionary architecture is the cure for all ailments! Companies have several legitimate reasons to pass on these ideas.

• Can't evolve a ball of mud: one of the key "-ilities" architects neglect is feasibility — should the team undertake this project? If an architecture is a hopelessly coupled Big Ball of Mud, making it possible to evolve it cleanly will take an enormous amount of work — likely more than rewriting it from scratch. Companies loath throwing anything away that has perceived value, but often rework is more costly than rewrite. How can companies tell if they're in this situation? The first step to converting an existing architecture into an evolvable one is modularity. Thus, a developer's first task requires finding whatever modularity exists in the current system and restructuring the architecture around those discoveries. Once the architecture becomes less entagled, it becomes easier for architects to see underlying structures and make reasonable determinations about the effort needed for restructuring.
• Other architectural characteristics dominate: evolvability is only one of many characteristics architects must weigh when choosing a particular architecture style. No architecture can fully support conflicting core goals. For example, building high performance and high scale into the same architecture is difficult. In some cases, other factors may outweigh evolutionary change.
• Sacrificial architecture: building a sacrificial architecture implies that architects aren't going to try to evolve it but rather replace it at the appropriate time with something more permanent. Cloud offerings make this an attractive option for companies experimenting with the viability of a new market or offering.
• Planning on closing the business soon: evolutionary architecture helps businesses adapt to changing ecosystem forces. If a company doesn't plan to be in business in a year, there's no reason to build evolvability into their architecture. Some companies are in this position; they just don't realize it yet.

Convincing others

Architects and developers struggle to make nontechnical managers and coworkes understand the benefits of something like evolutionary architecture. This is especially true of parts of the organization most disrupted by some of the necessary changes. For example, developers who lecture oprations group about doing their job incorrectly will generally find resistance. Rather than try to convince reticent parts of the organization, demonstrate how these ideas improve their practices.

This business case

Business people are often wary of ambitious IT projects, which sound like expensive replumbing exercises. However, many businesses find that many desirable capabilities have their basis in more evolutionary architectures.

• Moving fast without breaking things: most large enterprises complain about the pace of change within the organization. One side effect of building an evolutionary architecture manifests as better engineering efficiency. All the practices we call incremental change improve automation and efficiency. Defining top-level enterprise architecture concerns as fitness functions both unifies a disparate set of concerns under one umbrella and forces developers to think in terms of objective outcomes. Business people fear breaking change. If developers build an architecture that allows incremental change with better confidence than older architectures, both business and engineering win.
• Less risk: with improved engineering practices comes decreased risk. Once developers have confidence that their practices will allow them to make changes in the architecture without breaking things, companies can increase their release cadence.

Building evolutionary architectures

Our ideas about building evolutionary architectures build upon and rely on many existing things: testing, metrics, deployment pipelines, and a host of other supporting infrastructure and innovation. We're creating a new perspective to unify previously diversified concepts using fitness functions. We want architects to start thinking of architectural characteristics as evaluable things rather than ad hoc aspirations, allowing them to build more resilient architectures. The software development ecosystem is going to continue to churn out new ideas from unexpected places. Organizations who can react and thrive in that environment will have a serious advantage.

Neal Ford, Rebecca Parsons & Patrick Kua, "Putting Evolutionary Architecture into Practice", in Building Evolutionary Architectures: Support Constant Change, 141-165.

Evolutionary architecture pitfalls and antipatterns

We identify two kinds of bad engineering practices that manifest in software projects — pitfalls and antipatterns. Many developers use the word antipattern as a jargon for "bad", but the real meaning is more subtle. A software antipattern has two parts. First, an antipattern is a practice that initally looks like a good ideia, but turns out to be a mistake. Second, better alternatives exist for most antipatterns. Architects notice many antipatterns only in hindsight, so they are hard to avoid. A pitfall looks superficially like a good idea but immediately reveals itself to be a bad path.

Technical architecture

Antipattern: vendor king

Some large enterprises buy Enterprise Resource Planning (ERP) software to handle common business tasks like accouting, inventory management, and other common chores. However, many organizations become overambitious with this category of software, leading to the vendor king antipattern, an architecture built entirely around a vendor product that pathologically couples the organization to a tool.

To escape this antipattern, treat all software as just another integration point, even if it initially has broad responsibilities. By assuming integration at the outset, developers can more easily replace behavior that isn't useful with other integration points, dethroning the king.

Pitfall: leaky abstractions

Increased tech stack complexity has made the abstraction distraction problem worse recently. Not only does the ecosystem change, but the constituent parts become more complex and interwined over time as well. Our mechanism for protecting evolutionary change — fitness functions — can protect the fragile join points or architecture. Architects define invariants at key integration points as fitness functions, which run as part of a deployment pipeline, ensuring abstractions don't start to leak in undesirable ways.

Antipattern: last 10% trap

Another kind of reusability trap exists at the other end of the abstraction spectrum, with package software, platforms, and frameworks.

In 4GLs, 80% of what the client wants is quick and easy to build. These environments are modeled as rapid application development tools, with drag-and-drop support for UIs and other niceties. However, the next 10% of what the client wants is, while possible, extemely difficult — because that functionality wasn't built into the tool, framework, or language. So clever developers figure out a way to hack tools to make things work: adding a script to execute where static things are expected, chaining methods, and other hacks. The hack only gets you from 80% to 90%. Ultimately the tool can't solve the problem completely — a phrase we coined as the Last 10% Trap — leaving every project a disappointment. While 4GLs make it easy to build simple things fast, they don't scale to meet the demands of the real world.

Antipattern: code reuse abuse

Ironically, the more effort developers put into making code reusable the harder it is to use. Making code reusable involves adding additional options and decisions points to accomodate the different uses. The more developers add hooks to enable reusability the more they harm the basic usability of the code.

Microservices eschew code reuse, adopting the philosophy of prefer duplication to coupling: reuse implies coupling, and microservices architectures are extremely decoupled. However, the goal in microservices isn't to embrace duplication but rather to isolate entities within domains. Services that share a common class are no longer independent. In a microservices architecture, Checkout and Shipping would each have their own internal representation of Customer. If they need to collaborate on customer-related information, they send the pertinent information to each other. Architects don't try to reconcile and consolidate the disparate versions of Customer in their architecture. The benefits of reuse are illusory and the coupling it introduces comes with its disadvantages. Thus, while architects understand the downsides of duplication, the offset that localized damage to the architectural damage too much coupling introduces.

We're not suggesting teams to avoid building reusable assets, but rather evaluate them continually to ensure they still deliver value. When coupling points impede evolution or other important architectural characteristics, break the coupling by forking or duplication.

All too often architects make a decision that is the correct decision at the time but becomes a bad decision over time because of changing conditions like dynamic equilibrium. Architects must continually evaluate the fitness of the "-ilities" of the architecture to ensure they still add value and haven't become antipatterns.

Pitfall: resume-driven development

Don't build architecture for the sake of architecture — you are trying to solve a problem. Always understand the problem domain before choosing an architecture rather than the other way around.

Incremental change

Antipattern: inappropriate governance

In modern environments, it is inappropriate governance to homogenize on a single technology stack. This leads to the inadvertent overcomplication problem, where governance decisions add useless multipliers to the effort required to implement a solution. When developers build monolith architectures, governance choices affect everyone. Thus, the architect must look at the requirements of every project and make a choice that will serve the most complex case. A small project may have simple needs yet must take on the full complexity for consistency.

With microservices, because none of the services are coupled via technical or data architecture, different teams can choose the right level of complexity and sophistication required to implement their service. This partitioning tends to work best when the team wholly owns their service, including the operational aspects.

Building services in different technology stacks is one way to achieve technical architecture decoupling. Many companies try to avoid this approach because they fear it hurts the ability to move employees across projects. However, Chad Fowler, an architect at Wunderlist, took the opposite approach: he insisted that teams use different technology stacks to avoid inadvertent coupling. His philosophy is that accidental coupling is a bigger problem than developer portability.

From a practical governance standpoint in large organizations, we find the Goldilocks Governance model works well: pick three technology stacks for standardization — simple, intermediate, and complex — and allow individual service requirements to drive stack requirements. This gives teams the flexibility to choose a suitable technology stack while still providing the company some benefits of standards.

Pitfall: lack of speed to release

While the extreme version of Continuous Delivery, continuous deployment, isn't required for an evolutionary architecture, a strong correlation exists between the ability to release software and the ability to evolve that software design.

If companies build an engineering culture around continuous deployment, expecting that all changes will make their way to production only if they pass the gauntlet laid out by the deployment pipeline, developers become accustomed to constant change. On the other hand, if releases are a formal process that require a lot of specialized work, the chances of being able to leverage evolutionary architecture diminishes.

Cycle time is therefore a critical metric in evolutionary architecture projects — faster cycle time implies a faster ability to evolve.

Business concerns

Most of the time, business people aren't nefarious characters trying to make things difficult for developers, but rather have priorities that drive inappropriate decisions from an architectural standpoint, which inadvertently constrain future options.

Pitfall: product customization

Salespeople want options to sell. The caricature of sales people has them selling any requested feature before determining if their product actually contains that feature. Thus, sales people want infinitely customizable software to sell. However, that capability comes at a cost along with a spectrum of implementation techniques, like unique build for each customer, permanent feature toggles, and product-driven customization.

Customization also impedes evolvability, but this shouldn't discourage companies from building customizable software, but rather to realistically asses the associated costs.

Antipattern: reporting

Reporting is a good example of inadvertent coupling in monolithic architectures. Architects and DBAs want to use the same database schema for both system of record and reporting, but encounter problems because a design to support both is optimized for neither. A common pitfall developers and report designers conspire to create in layered architecture illustrates the tension between concerns. Architects build layered architecture to cut down on incidental coupling, creating layers of isolation and separation of concerns. However, reporting doesn't need separate layers to support its function, just data. Additionally, routing requests through layers adds latency. Thus, many organizations with good layered architectures allow report designers to couple reports directly to database schemas, destroying the ability to make changes to the schema without wrecking reports. This is a good example of conflicting business goals subverting the work of architects and making evolutionary change extremely difficult. While no one set out to make the system hard to evolve, it was the cumulative effect of decisions.

Many microservices architectures solve the reporting problem by separating behavior, where the isolation of services benefits separation but not consolidation. Architects commonly build these architectures using event streaming or message queues to populate domain "system of record" databases, each embedded within the architectural quantum of the service, using eventual consistency rather than transactional behavior. A set of reporting services also listens to the event stream, populating a denormalized reporting database optimized for reporting. Using eventual consistency frees architects from coordination — a form of coupling from an architectural standpoint — allowing different abstractions for different uses of the application.

Pitfall: planning horizons

The Sunk Cost Fallacy describes decisions affected by emotional investment. Put simply, the more someone invests time or effort into something, the harder it becomes to abandon it. In software, this is seen in the form of the irrational artifact attachment — the more time and effort you invest in planning or a document, the more likely you will protect what's contained in the plan or document even in the face of evidence that it is inaccurate or outdated.

Beware of long planning cycles that force architects into irreversible decisions and find ways to keep options open. Architects should avoid following technologies that require a significant upfront investment before software is actually built (e.g., large licenses and support contracts) before they have validated through end-user feedback that the technology actually fits the problem they are trying to solve.

Neal Ford, Rebecca Parsons & Patrick Kua, "Evolutionary Architecture Pitfalls and Antipatterns", in Building Evolutionary Architectures: Support Constant Change, 12-139.