Architectural coupling

Make sure your architecture matches the problem domain. Don't try to force fit an unsuitable architecture.

Monoliths

Monolithic architectures, particularly layered architectures, are a common choice when starting a project because developers understand the structure easily. However, many monoliths reach end of life and must be replaced because of decreasing performance, size of code base, and a host of other factors. A current common target for monolith migration is microservices-style architectures, which are more complex than monolithic architectures in areas like service and data granularity, operationalization, coordination, transactions, and so on. If a development team has a hard time building one of the simplest architectures, how will moving to a more complex architecture solve their problems?

If you can't build a monolith, what makes you think microservices are the answer?

Simon Brown

Before embarking on an expensive architecture restructuring exercise, architects may benefit from improved modularization of what's already present. If nothing else, it's an excellent starting point for the more serious restructuring that follows.

Microservices

In microservices architecture, the domain encapsulates technical and other architectures, making evolution across domain dimensions easy. No one perspective on architecture is "correct", but rather a reflection on the goals developers build into their projects. If the focus is entirely on technical architecture, then making changes across that dimension is easier. However, if the domain perspective is ignored, then evolving across that dimension is no better than the Big Ball of Mud.

One of the major factors that impacts the ability to evolve an application at the architectural level is how unintentionally coupled each part of the system is. For example, in a layered architecture, architects specifically couple layers together in an intentional way. However, the domain dimension is unintentionally coupled, making evolution in that dimension difficult, because the architecture is designed around technical architecture layers, not the domain. Thus, one of the important aspects of an evolvable architecture is appropriate coupling across dimensions.

Neal Ford, Rebecca Parsons & Patrick Kua, "Architectural Coupling", in Building Evolutionary Architectures: Support Constant Change, 57-58, 74, 78.

Evitando chamadas custosas em testes automáticos

Ao escrever testes automáticos, é comum utilizarmos Test Doubles para substituir dependências diretas do código sob teste. Além de ter impacto direto na forma de pensar e construir tanto os casos de teste quanto o próprio código sob teste, essa prática também permite eliminar chamadas custosas e/ou não confiáveis durante a execução dos testes, garantindo assim mais eficiência e confiabilidade para a suíte de testes como um todo.

Por exemplo, suponha que precisemos testar uma função ou objeto que efetue uma requisição HTTP para uma API externa à nossa aplicação. Podemos criar um mock para o trecho de código que faz a requisição, de forma a simular o retorno esperado e, assim, não ter que fazer a chamada real de rede. Ganhamos em eficiência, pois, sem a requisição, não é necessário lidar com a latência de rede durante a execução dos testes. Ganhamos também em confiabilidade, pois, como chamadas HTTP podem falhar, qualquer requisição externa tem o potencial de gerar um falso positivo na suíte de testes; removendo a chamada, removemos essa possibilidade.

Pensando nisso, é de vital importância que numa base de código grande — com uma suíte de testes, em geral, igualmente grande — consigamos automaticamente garantir que chamadas custosas sejam apropriadamente mockadas. No caso de requisições de rede, podemos simplesmente bloquear ou restringir chamadas de socket durante a execução dos testes. É o que o pytest-socket, por exemplo, faz. Para quem usa o pytest, basta instalar o plugin e executar pytest --disable-socket. Qualquer teste em que uma chamada de socket for feita (mesmo por pacotes terceiros importados) irá falhar com SocketBlockedError.

Para códigos com custo de computação alto (ex: uma função com um pesado cálculo recursivo) que façam parte da aplicação, uma forma de evitar seu uso sem mock em testes é torna esse requisito uma parte do próprio código, isto é, permitir que a pessoa que o desenvolveu adicione alguma marcação que bloqueie a chamada em ambiente de teste. Não conhecendo uma solução já existente para isso em Python, me aventurei em criar a minha própria e daí surgiu o decorador enforce_mock_in_tests. Esse decorador pode ser aplicado a classes ou funções. A cada chamada do código decorado, ele avalia se o ambiente é de teste ou não e, se for, uma exceção é lançada, dando uma clara indicação ao desenvolvedor de que aquele código não deve ser chamado diretamente nos casos de teste, mas sim mockado. Além do código do decorador em si, o Gist abaixo contém também um arquivo com casos de teste para ele, permitindo ver como o decorador pode ser usado no código da aplicação.

Engineering incremental change

Architecture is abstract until operationalized, when it becomes a living thing.

Conflicting goals

The agile software development process has taught us that the sooner a developer can detect problems, the less effort is required to fix them. One of the side effects of broadly considering all the dimensions in software architecture is the early identification of goals that conflict across dimensions. For example, developers at an organization may want to support the most aggressive pace of change to support new features. Fast change to code implies fast changes to database schemas, but the database administrators are more concerned about stability because they are building a data warehouse. The two evolution goals conflict across the technical and data architecture.

Obviously, some compromise must occur, taking into account the myriad factors that affect the underlying business. Using architecture dimensions as a technique for identifying portions of concern in architecture (plus fitness functions to evaluate them) allows an apples-to-apples comparison, making the prioritization exercise more informed.

Conflicting goals are inevitable. However, discovering and quantifying those conflicts early allows architects to make better informed decisions and create more clearly defined goals and principles.

Hypothesis- and data-driven development

[...] Using hypothesis-driven development, we can incorporate users in an unprecedented way, learning from behavior and building what users really find valuable.

Hypothesis-driven development requires the coordination of many moving parts: evolutionary architecture, modern DevOps, modified requirements gathering, and the ability to run multiple versions of an application simultaneously. Service-based architectures (like microservices) usually achieve side-by-side versions by intelligent routing of services. For example, one user may execute the application using a particular constellation of services while another request may use an entirely different set of instances of the same services. If most services include many running instances (for scalability, for example), it becomes trivial to make some of those instances slightly different with enhanced functionality, and to route some users to those features.

Experiments should run long enough to yield significant results. Generally, it is preferable to find a measurable way to determine better outcomes rather than annoy users with things like pop-up surveys. For example, does one hypothesized workflow allow the user to complete a task with fewer keystrokes and clicks? By silently incorporating users into the development and design feedback loop, you can build much more functional software.

Neal Ford, Rebecca Parsons & Patrick Kua, "Engineering Incremental Change", in Building Evolutionary Architectures: Support Constant Change, 29, 39-40, 44.

Evolutionary architectures and Conway's law

An evolutionary architecture supports guided, incremental change across multiple dimensions.

Neal Ford, Rebecca Parsons & Patrick Kua, "Software Architecture", in Building Evolutionary Architectures: Support Constant Change, 6.

Structure teams to look like your target architecture, and it will be easier to achieve it.

Neal Ford, Rebecca Parsons & Patrick Kua, "Software Architecture", in Building Evolutionary Architectures: Support Constant Change, 13.

How to be managed

Part of being a good manager is figuring out how to be managed. This is not exactly the same as managing up, although it is related. Developing a sense of ownership and authority for your own experiences at work, and not relying on your manager to set the entire tone for your relationship, is an important step in owning your career and workplace happiness.

Spend time thinking about what you want

Your manager can point out opportunities for growth. She can show you projects. She can provide feedback on your areas of learning and development. But she cannot read your mind, and she cannot tell you what will make you happy. Whether you are brand new to the workplace or 20 years into your career, the onus of figuring out what you want to do, what you want to learn, and what will make you happy rests on your shoulders.

As you go through various stages of your career, you'll start to realize how much uncertainty there is in the world. It’s a pretty universal truth that once you get the job you thought you wanted, the enjoyment eventually fades and you find yourself looking for something else. You think you want to work for that cool startup, and you get there only to find it's a mess. You think you want to be a manager, only to discover that the job is hard and not rewarding in the ways you expected.

In all of this uncertainty, the only person you can rely on to pull through it is yourself. Your manager cannot do that for you. Use your manager to discover what's possible where you are, but look to understand yourself in order to figure out where you want to go next.

You are responsible for yourself

Knowing yourself is step one. Step two is going after what you want.

Bring agendas to your 1-1s when you have things you need to talk about. When you want to work on projects, ask. Advocate for yourself. When your manager isn't helpful, look for other places to get help. Seek out feedback, including constructive feedback on areas to improve. When that feedback comes to you, take it graciously, even when you don't agree with it.

When you are persistently unhappy, say something. When you are stuck, ask for help. When you want a raise, ask for it. When you want a promotion, find out what you need to do to get it. Your manager cannot force work–life balance on you. If you want to go home, figure out how to get your work done and go home. Sometimes you will have to go against the cultural grain to set your own boundaries, and that will feel uncomfortable. On the flip side, sometimes if you want a bigger job, you will have to work more hours to get it.

You will not get everything you ask for, and asking is not usually a fun or comfortable experience. However, it's the fastest way forward. If your manager is conscientious, he'll appreciate your candor. He may not be conscientious, or he may like you less for asking, and then you'll know that about your current situation. I can't guarantee you that it'll go well, but if you've set a goal for yourself, you owe it to yourself to do what you can to make it happen.

Give your manager a break

This is a job. Your manager will be stressed out sometimes. She'll be imperfect. She will say dumb things, or do things that feel unfair or harmful to you. She'll give you work that you don't want to do, and get annoyed when you complain about doing it. Her job is to do the best thing for the company and the team. It is not to do whatever it takes to make you happy all the time.

Your relationship with your manager is like any other close interpersonal relationship. The only person you can change is yourself. You should absolutely provide feedback to your manager, but understand that she may not listen or change no matter how much you think she should. If you find yourself starting to actively resent your manager for whatever reason, you probably need to move to a different team or look for a new job. If you find yourself resenting every manager you work for, you may need to think about whether the cause is them or you. Perhaps you'd be happier in a job where you don't have a manager.

Especially as you become more senior, remember that your manager expects you to bring solutions, not problems. Try not to make every 1-1 about how you need something, how something is wrong, or how you want something more. When you have a problem, instead of demanding that your manager solve it for you, try asking her for advice on how she might approach the problem. Asking for advice is always a good way to show respect and trust.

Choose your managers wisely

Your manager can make a huge difference in your career. So, as much as you can, consider not only the job, the company, and the pay, but also the manager when you are evaluating job opportunities.

Strong managers know how to play the game at their company. They can get you promoted; they can get you attention and feedback from important people. Strong managers have strong networks, and they can get you jobs even after you stop working for them.

There's a difference between a strong manager and a manager that you like as a friend, or even one you respect as an engineer. Plenty of great engineers make ineffective managers because they don't know or want to deal with the politics of leadership in their companies. A strong engineer may make a great mentor-manager to someone early in his career, but a terrible advocate-manager for someone who is more senior.

Camille Fournier, "Management 101", in The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change, chapter 1.

Bootstrapping culture

When you are in the role of senior engineering leader, part of your job is to set the culture of your function. A common failing of first-time CTOs is to underestimate the importance of being clear and thoughtful about the culture of the engineering team. Whether you are growing a new team or reforming an existing team, neglecting the team culture is a sure-fire way to make your job harder. As the team grows and evolves, it's important to attend to your culture as you would attend to any other important piece of infrastructure that you rely on.

For many people who are attracted to startup culture, the ideas of "structure" and "process" are seen as pointless at best and harmful at worst. [...] When talking about structure with skeptics, I try to reframe the discussion. Instead of talking about structure, I talk about learning. Instead of talking about process, I talk about transparency. [...] This learning and sharing is how organizations become more stable and more scalable over time.

Even when the overall company grows beyond the small group, the engineering team often pushes itself to stay unstructured. Hiring "full stack" engineers who are exclusively sourced from the professional and social networks of the current team results in low skill specialization and high homogeneity. Forcing the team to be collocated lowers communication barriers. And perhaps most critically, having an engineering team that operates solely as the execution arm of the product or founder makes the team highly task-oriented. [...] Be that as it may, the unstructured organization either displays characteristics that ultimately make it less self-directed than the members might wish to believe, or is run by hidden hierarchies and power dynamics. In many cases both things are true to some extent.

The example of the structureless team also applies to technical decisions and processes. There is a reason that you often find a lot of spaghetti code in early startups. When work is done to satisfy an immediate task, in a unified code base worked on by a team of interchangeables, the result is not usually a larger thoughtful structure, but a tweak here, a hack there — anything to get things done and moving forward. It's no surprise that we usually end up refactoring spaghetti code when we want to make it scalable, because refactoring usually involves identifying and explicitly drawing out structure in order to make the code base easier to read and work in.

That, in short, is the value of structure. Structure is how we scale, diversify, and take on more complex long-term tasks. We do it to our software, we do it to our teams, and we do it to our processes. In the same way that strong technical systems designers are capable of identifying and shaping underlying system structures, strong leaders are capable of identifying and shaping underlying team structures and dynamics, and doing so in a way that supports the long-term goals of the team and equips the individuals to achieve their best.

Nothing is more ridiculous than a small team with a rigid hierarchy. [...] However, it's more common in small companies to see structure come too late. The problems creep up slowly. One person gets used to making all of the decisions and changing his mind frequently. This strategy works fine when it's just him and a couple of others. But when he keeps doing it with a team of 10, a team of 20, a team of 50, what you start to see is a high degree of confusion and wasted effort. The cost to change his mind becomes more and more expensive.

Assessing your role

I don't think there's a huge benefit in overdesigning your team structure or process when your team is small and functioning well. However, at some point you'll start to experience failure, and failure is the best place to investigate and identify where your structure needs to change.

My advice to leaders is simple: when failures occur, examine all aspects of reality that are contributing to those failures. The patterns you see are opportunities to evolve your structure, either by creating more or different structure or removing it. [...] What about examining success? Well, you can learn things from success, but it is often a poor teacher. Ironically, while luck plays a role in both failure and success, we often attribute failure to bad luck and success to our own actions. [...] If you want to learn from success, make sure you can identify the actual improvement you're seeking when applying those lessons more broadly, and that you understand the context required to repeat that success.

Learning rarely comes for free. Analyzing situations and thinking about good takeaways takes time. If the value of your future time is less than the value of your current time, then you're probably not going to worry too much about saving future time. Just because your company is big, old, and stable doesn't mean you can have as much rigid, unchanging structure as you want. [...] But if you don't adopt structure when you need it, things can also go wrong.

When every new hire slows the team down for months because there is no onboarding process, that is a failure due to lack of structure. When people regularly leave the company because they have no path to advancement or career growth, that is a failure due to lack of structure. The third time you have a production outage because someone logged directly into the database and accidentally dropped a critical table, that is a failure due to lack of structure. I said earlier that I prefer to talk about learning and transparency rather than using the word structure, because really what we're talking about here is identifying the causes of failures, especially frequent failures, and trying to figure out what we can change to solve for those failures. This is fundamentally about learning.

Creating your culture

One of the things I have come to believe strongly is that culture is real; it's also incredibly important, and it's something that many people don't understand at all. It's both an easy, natural consequence of your company's evolution and something that can quickly become a problem if you don't tend to it. Consciously guiding the culture of your team is part of a leader's job, and to do this well, you need to understand what it means in the first place.

So what is culture? Culture is the generally unspoken shared rules of a community. [...] Culture doesn't mean that every single person holds exactly the same values, but it tends to guide a general overlap, and it creates a bunch of rules of interaction that you don't have to think much about if you are deeply ingrained in that culture.

People do make decisions using methods other than cultural values. They may adhere to the standards of a formal or informal contract, for example. They may do a pure data-driven analysis and determine the optimal outcome. But in complex environments where the needs of the group must override the needs of the individual, cultural values are the glue that enables us to work as a team and make decisions when faced with uncertainty. This is why figuring out and guiding your culture is such an important part of building a successful company.

Applying core values

Understand what your company's values are, understand what your team's values are, and think about what you personally value. Write the values down if they aren't already written, and try to be explicit. Use this explicit list to evaluate candidates, praise team members, and inform your performance review process.

Structuring cross-functional teams

The implications of the cross-functional structure are subtle. The values of everyone in these teams will start to change. In technology-focused structures where engineers work solely with other engineers, particularly engineers of their same "type" (mobile, backend, middleware, etc.), the focus is on being the best engineer by some measure of engineering excellence. People who design complex systems or who know the details of the latest iOS are the leaders and role models for the teams. In a product-focused structure, the leadership focus changes. Now the engineers who have the best product sense, the engineers who are capable of getting features done quickly and efficiently, and the engineers who communicate the best with the other functions will start to emerge as the leaders of the team.

I mean no value judgment here, but I encourage you to be aware of the product/business versus technology focus and apply it where it makes sense. What is truly important to the success of your company or your organization? If the most important thing is evolving a product that is a function of many different business areas coming together, you probably want leaders who have that business sense. On the other hand, in the areas where the technology must be rock-solid or exceptionally innovative and cutting-edge, you probably want teams that have more of an engineering focus and that are led by people who can design complex systems. You don't have to go entirely one way or the other, but recognize that one of these will lead the company as a whole, and — especially if your role is in senior management — focus your skill set on the one that the company itself most values and hire in for the other.

Camille Fournier, "Bootstrapping Culture", in The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change, chapter 9.

The big leagues

Ruling with fear, guiding with trust

How do you know if you're creating a culture of fear? It can come from placing a high value on being correct and following the rules, and having a strong affinity for hierarchy-based leadership. I also believe that coming from places where conflict was openly tolerated, if not actively encouraged, made me even more likely to create this culture. Engineering culture has a high tolerance for open debate to resolve conflict, so leaders who come from heavily engineering-focused backgrounds may feel particularly comfortable aggressively sparring with others over issues. Unfortunately, when you're the leader, the dynamic changes, and those who may have fought back when you were an individual contributor will feel threatened by you as a leader.

The culture of fear is pretty common in technology, and it survives best in environments where things are otherwise going well. Don't be fooled by external circunstances that enable your bad behavior. If you're feared but respected, the company is growing, and the team is working on interesting problems, you might get along OK for a time. However, if you lose any of these elements, you can expect to see people who have better options leave for greener pastures. I know firsthand that having a team that fears but respects you isn't enough when they're frustated by other things happening around them. So work on softening your rough edges, practice caring about your team as humans, and get curious. Building a culture of trust takes time, but the results are well worth it.

True north

A core role of senior leadership is sometimes overlooked. This role, played by the senior leader of a functional area (the CTO plays it for technology, the CFO plays it for finance, etc.), sets the baseline of what excellence looks like in this function. I call it "True North".

Technology leaders must help set the standard for True North in their organizations for different types of projects and exposures. Another way to think of this is through the lens of risk analysis. Risk analysis doesn't mean that we don't take risks. Some things that are generally considered "bad" can be OK under certain circunstances. These include:

• Having a single point of failure
• Having known bugs and issues
• Being unable to tolerate high load
• Losing data
• Putting out code that is undertested
• Having slow performance

There are situations and companies in which all of those risks are acceptable to take. That being said, True North helps us understand that all these issues must be carefully considered when we put code into production. Just because these rules have exceptions doesn't mean we forget that they exist.

I call this concept True North because it's important to understand it as an underlying pull, as a guiding instinct that we as leaders have developed over time and strive to help our teams as a whole develop as well. When our teams develop this instinct, they can be trusted to independently follow these guidelines without much direction or nudging.

Camille Fournier, "The Big Leagues", in The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change, chapter 8.

Leadership and management

If you're a leader with no power over business strategy and no ability to allocate people to important tasks, you're at best at the mercy of your influence with other executives and managers, and at worst a figurehead. You can't give up the responsibility of management without giving up the power that comes with it.

Camille Fournier, "What's a CTO?", in The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change, chapter 8.

List of books (2nd edition)

• 
  Peopleware: Productive Projects and Teams
  
  This is simply the best book about people and software that I've read to date. The authors use knowledge drawn from their years as software engineers and consultants to explore many questions about productivity. The book shows that leaders and managers should be constantly re-evaluating their practices (especially those that are taken as common sense) in order to maintain people at their balance point and, consequently, obtain the best results in the projects. Although the ideas in the book are focused primarily on software teams, they can also be applied for teams in general.
• 
  Agile Software Development, Principles, Patterns, and Practices
  
  This book by Uncle Bob was published shortly after the publication of the Agile Manifesto. It presents many design patterns and demonstrates how to use them by showing their application in real projects. All case studies are presented using principles of XP, like TDD and pair programming. The examples can become a bit tiresome by the end of the book, but the sections that present the design patterns are worth reading.
• 
  The Effective Engineer: How to Leverage Your Efforts In Software Engineering to Make a Disproportionate and Meaningful Impact
  
  A really practical book with many useful tips. Altough not extensive, the text covers a few different topics, offering advices that can easily be applied to everyday work. A light read that indeed has the potential to increase the effectiveness of any software engineer.

Making implicit concepts explicit

Many transformations of domain models and the corresponding code happen when developers recognize a concept that has been hinted at in discussion or present implicitly in the design, and they then represent it explicitly in the model with one or more objects or relationships.

Digging out concepts

Developers have to sensitize themselves to the hints that reveal lurking implicit concepts, and sometimes they have to proactively search them out. Most such discoveries come from listening to the language of the team, scrutinizing awkwardness in the design and seeming contradictions in the statements of experts, mining the literature of the domain, and doing lots and lots of experimentation.

How to model less obvious kinds of concepts

Explicit constraints

Constraints make up a particularly important category of model concepts. They often emerge implicitly, and expressing them explicitly can greatly improve a design.

Here are some warning signs that a constraint is distorting the design of its host object.

1. Evaluating a constraint requires data that does not otherwise fit the object's definition.
2. Related rules appear in multiple objects, forcing duplication or inheritance between objects that are not otherwise a family.
3. A lot of design and requirements conversation revolves around the constraints, but in the implementation, they are hidden away in procedural code.

When the constraints are obscuring the object's basic responsibility, or when the constraint is prominent in the domain yet not prominent in the model, you can factor it out into an explicit object or even model it as a set of objects and relationships.

Processes as domain objects

What I am talking about here are processes that exist in the domain, which we have to represent in the model. When these emerge, they tend to make for awkward object designs.

A SERVICE is one way of expressing such a process explicitly, while still encapsulating the extremely complex algorithms.

When there is more than one way to carry out a process, another approach is to make the algorithm itself, or some key part of it, an object in its own right. The choice between processes becomes a choice between these objects, each of which represents a different STRATEGY.

The key to distinguishing a process that ought to be made explicit from one that should be hidden is simple: Is this something the domain experts talk about, or is it just part of the mechanism of the computer program?

SPECIFICATION

Business rules often do not fit the responsibility of any of the obvious ENTITIES or VALUE OBJECTS, and their variety and combinations can overwhelm the basic meaning of the domain object. But moving the rules out of the domain layer is even worse, since the domain code no longer expresses the model.

A SPECIFICATION is a predicate that determines if an object does or does not satisfy some criteria.

The SPECIFICATION keeps the rule in the domain layer. Because the rule is a full-fledged object, the design can be a more explicit reflection of the model.

Applying and implementing SPECIFICATION

Much of the value of SPECIFICATION is that it unifies application functionality that may seem quite different. We might need to specify the state of an object for one or more of these three purposes.

1. To validate an object to see if it fulfills some need or is ready for some purpose
2. To select an object from a collection (as in the case of querying for overdue invoices)
3. To specify the creation of a new object to fit some need

These three uses — validation, selection, and building to order — are the same on a conceptual level. Without a pattern such as SPECIFICATION, the same rule may show up in different guises, and possibly contradictory forms. The conceptual unity can be lost. Applying the SPECIFICATION pattern allows a consistent model to be used, even when the implementation may have to diverge.

Eric Evans, "Making Implicit Concepts Explicit", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 205-227.

Refactoring toward deeper insight

Success developing useful models comes down to three points:

1. Sophisticated domain models are achievable and worth the trouble.
2. They are seldom developed except through an iterative process of refactoring, including close involvement of the domain experts with developers interested in learning about the domain.
3. They may call for sophisticated design skills to implement and to use effectively.

Eric Evans, "Refactoring Toward Deeper Insight", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 188.

Managing projects

Doesn't agile software development get rid of the need for project management? No. Agile software development is a great way to think about work because it forces you to focus on breaking tasks down into smaller chuncks, planning those smaller chuncks out, and delivering value incrementally instead of all at once. None of this means that you don't need to understand how to do project management. You'll have projects that for whatever reason can't be completed in a single sprint, or even two small sprints. You'll need to estimate project length for your management team, and give some detail on why you believe things will take that long. There are some projects, usually described by words like infrastructure, platform, or system, that require architecture or significant advanced planning. When faced with this kind of project, which includes many unknowns and relatively hard deadlines, you will find it doesn't fit so well into the standard agile process.

Project management isn't something that needs to happen in detail for every single effort, and it's overused in some organizations. I don't even like hiring project managers because they often act as a crutch for engineers to use instead of learning to think through their future work and ask real questions about what they're doing and why, and their presence means that you have more waterfall-style projects instead of an agile process. Still, project management has to happen, and as tech lead, you should be doing it when it is needed, especially for deeply technical projects.

Ultimately, the value of planning isn't that you execute the plan perfectly, that you catch every detail beforehand, or that you predict the future; it's that you enforce the self-discipline to think about project in some depth before diving in and seeing what happens. A degree of forethought, in places where you can reasonably make predictions and plans, is the goal. The plan itself, however accurate it turns out, is less important than spending time on the act of planning.

Camille Fournier, "Tech Lead", in The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change, chapter 3.

Repositories

The goal of domain-driven design is to create better software by focusing on a model of the domain rather than the technology. By the time a developer has constructed an SQL query, passed it to a query service in the infrastructure layer, obtained a result set of table rows, pulled the necessary information out, and passed it to a constructor or FACTORY, the model focus is gone. It becomes natural to think of objects as containers for the data that the queries provide, and the whole design shifts toward a data-processing style. The details of the technology vary, but the problem remains that the client is dealing with technology, rather than model concepts.

Therefore:

For each type of object that needs global access, create an object that can provide the illusion of an in-memory collection of all objects of that type. Set up access through a well-known global interface. Provide methods to add and remove objects, which will encapsulate the actual insertion or removal of data in the data store. Provide methods that select objects based on some criteria and return fully instantiated objects or collections of objects whose attribute values meet the criteria, thereby encapsulating the actual storage and query technology. Provide REPOSITORIES only for AGGREGATE roots that actually need direct access. Keep the client focused on the model, delegating all object storage and access to the REPOSITORIES.

REPOSITORIES have many advantages, including the following:

• They present clients with a simple model for obtaining persistent objects and managing their life cycle.
• They decouple application and domain design from persistence technology, multiple database strategies, or even multiple data sources.
• They communicate design decisions about object access.
• They allow easy substitution of a dummy implementation, for use in testing (typically using an in-memory collection).

Working within your frameworks

In general, don't fight your frameworks. Seek ways to keep the fundamentals of domain-driven design and let go of the specifics when the framework is antagonistic. Look for affinities between the concepts of domain-driven design and the concepts in the framework. This is assuming that you have no choice but to use the framework. [...] If you have the freedom, choose frameworks, or parts of frameworks, that are harmonious with the style of design you want to use.

The relationship with FACTORIES

A FACTORY handles the beginning of an object's life; a REPOSITORY helps manage the middle and the end.

[...] In this domain-driven view of the design, FACTORIES and REPOSITORIES have distinct responsibilities. The FACTORY makes new objects; the REPOSITORY finds old objects. The client of a REPOSITORY should be given the illusion that the objects are in memory. The object may have to be reconstituted (yes, a new instance may be created), but it is the same conceptual object, still in the middle of its life cycle.

These two views can be reconciled by making the REPOSITORY delegate object creation to a FACTORY, which (in theory, though seldom in practice) could also be used to create objects from scratch.

This clear separation also helps by unloading all responsibility for persistence from the FACTORIES. A FACTORY's job is to instantiate a potentially complex object from data. If the product is a new object, the client will know this and can add it to the REPOSITORY, which will encapsulate the storage of the object in the database.

One other case that drives people to combine FACTORY and REPOSITORY is the desire for "find or create" functionality, in which a client can describe an object it wants and, if no such object is found, will be given a newly created one. This function should be avoided. It is a minor convenience at best. A lot of cases in which it seems useful go away when ENTITIES and VALUE OBJECTS are distinguished. A client that wants a VALUE OBJECT can go straight to a FACTORY and ask for a new one. Usually, the distinction between a new object and an existing object is important in the domain, and a framework that transparently combines them will actually muddle the situation.

Designing objects for relational databases

• When the database is being viewed as an object store, don't let the data model and the object model diverge far, regardless of the powers of the mapping tools. Sacrifice some richness of object relationships to keep close to the relational model. Compromise some formal relational standards, such as normalization, if it helps simplify the object mapping.
• Processes outside the object system should not access such an object store. They could violate the invariants enforced by the objects. Also, their access will lock in the data model so that it is hard to change when the objects are refactored.

Eric Evans, "The Life Cycle of a Domain Object", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 148-160.

Mentoring

Tips for the manager of a mentor

If the mentor does a good job, her productivity may slow down some during the mentoring period. If you've got an engineer involved in a time-sensitive project, you may not want to push him into mentoring at the same time. Because this is an additional responsibility, treat it as you would any other important additional responsibility you might hand out. Look for someone that you believe can succeed in the role, and who wants to distinguish herself beyond her coding ability.

[...] Because the outcome can be hard to quantitatively measure, emotional labor is often dismissed as less important work than writing software. It's assumed to be something that should just be provided without financial recognition. I'm not suggesting that you should pay people extra money to serve as mentors, but they need to be recognized for the work they put in, and the mentor should be treated as a first-class citizen with respect to other responsibilities the person might have. As I said before, plan for it, and provide the mentor the time to do the job right.

Key takeaways for the mentor: listen and speak their language

Senior engineers can develop bad habits, and one of the worst is the tendency to lecture and debate with anyone who does not understand them or who disagrees with what they are saying. To work successfully with a newcomer or a more junior teammate, you must be able to listen and communicate in a way that person can understand, even if you have to try several times to get it right. Software development is a team sport in most companies, and teams have to communicate effectively to get anything done.

Camille Fournier, "Mentoring", in The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change, chapter 2.

Factories

Shift the responsibility of creating instances of complex objects and AGGREGATES to a separate object, which may itself have no responsibility in the domain model but is still part of the domain design. Provide an interface that encapsulates all complex assembly and that does not require the client to reference the concrete classes of the objects being instantiated. Create entire AGGREGATES as a piece, enforcing their invariants.

When a constructor is all you need

The trade-offs favor a bare, public constructor in the following circunstances:

• The class is the type. It is not part of any interesting hierarchy, and it isn't used polymorphically by implementing an interface.
• The client cares about the implementation, perhaps as a way of choosing a STRATEGY.
• All of the attributes of the object are available to the client, so that no object creation gets nested inside the constructor exposed to the client.
• The construction is not complicated.
• A public constructor must follow the same rules as a FACTORY: It must be an atomic operation that satisfies all invariants of the created object.

Avoid calling constructors within constructors of other classes. Constructors should be dead simple. Complex assemblies, especially of AGGREGATES, call for FACTORIES. The threshold for choosing to use a little FACTORY METHOD isn't high.

Eric Evans, "The Life Cycle of a Domain Object", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 129.

Aggregates

Cluster the ENTITIES and VALUE OBJECTS into AGGREGATES and define boundaries around each. Choose one ENTITY to be the root of each AGGREGATE, and control all access to the objects inside the boundary through the root. Allow external objects to hold references to the root only. Transient references to internal members can be passed out for use within a single operation only. Because the root controls access, it cannot be blindsided by changes to the internals. This arrangement makes it practical to enforce all invariants for objects in the AGGREGATE and for the AGGREGATE as whole in any state change.

Eric Evans, "The Life Cycle of a Domain Object", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 129.

Modeling paradigms

The four patterns [ENTITIES, VALUE OBJECTS, SERVICES and MODULES] in this chapter provide the building blocks for an object model. But MODEL-DRIVEN DESIGN does not necessarily mean forcing everything into an object mold. There are also other model paradigms supported by tools, such as rules engines. Projects have to make pragmatic trade-offs between them. These other tools and techniques are means to the end of a MODEL-DRIVEN DESIGN, not alternatives to it.

Here are four rules of thumb for mixing nonobject elements into a predominantly object-oriented system:

• Don't fight the implementation paradigm. There's always another way to think about a domain. Find model concepts that fit the paradigm.
• Lean on the ubiquitous language. Even when there is no rigorous connection between tools, very consistent use of language can keep parts of the design from diverging.
• Don't get hung up on UML. Sometimes the fixation on a tool, such as UML diagramming, lead people to distort the model to make it fit what can easily be drawn. For example, UML does have some features for representing constraints, but they are not always sufficient. Some other style of drawing (perhaps conventional for the other paradigm), or simple English descriptions, are better than tortuous adaptation of a drawing style intended for a certain view of objects.
• Be skeptical. Is the tool really pulling its weight? Just because you have some rules, that doesn't necessarily mean you need the overhead of a rules engine. Rules can be expressed as objects, perhaps a little less neatly; multiple paradigms complicate matters enormously.

Eric Evans, "A Model Expressed in Software", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 116, 122.

The pitfalls of infrastructure-driven packaging

Elaborate technically driven packaging schemes impose two costs.

• If the framework's partitioning conventions pull apart the elements implementing the conceptual objects, the code no longer reveals the model.
• There is only so much partitioning a mind can stitch back togheter, and if the framework uses it all up, the domain developers lose their ability to chunck the model into meaningful pieces.

It is best to keep things simple. Choose a minimum of technical partitioning rules that are essential to the technical environment or actually aid development. For example, decoupling complicated data persistence code from the behavioral aspects of the objects may make refactoring easier.

Unless there is a real intention to distribute code on different servers, keep all the code that implements a single conceptual object in the same MODULE, if not the same object.

Use packaging to separate the domain layer from other code. Otherwise, leave as much freedom as possible to the domain developers to package the domain objects in ways that support their model and design choices.

Eric Evans, "A Model Expressed in Software", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 114-115.

Modules (a.k.a packages)

Everyone uses MODULES, but few treat them as a full-fledged part of the model. Code gets broken down into all sorts of categories, from aspects of the technical architecture to developers' work assignments. Even developers who refactor a lot tend to content themselves with MODULES conceived early in the project.

It is a truism that there should be low coupling between MODULES and high cohesion within them. Explanations of coupling and cohesion tend to make them sound like technical metrics, to be judged mechanically based on the distributions of associations and interactions. Yet it isn't just code being divided into MODULES, but concepts. There is a limit to how many things a person can think about at once (hence low coupling). Incoherent fragments of ideas are as hard to understand as an undifferentiated soup of ideas (hence high cohesion).

Choose MODULES that tell the story of the system and contain a cohesive set of concepts. This often yields low coupling between MODULES, but if it doesn't, look for a way to change the model to disentangle the concepts, or search for an overlooked concept that might be the basis of a MODULE that would bring the elements togheter in a meaningful way. Seek low coupling in the sense of concepts that can be understood and reasoned about independently of each other. Refine the model until it partitions according to high-level domain concepts and the corresponding code is decoupled as well.

Give the MODULES names that become part of the UBIQUITOUS LANGUAGE. MODULES and their names should reflect insight into the domain.

Eric Evans, “A Model Expressed in Software”, in Domain-Driven Design: Tackling Complexity in the Heart of Software, 109-111.

Services

Some concepts from the domain aren't natural to model as objects. Forcing the required domain functionality to be the responsibility of an ENTITY or VALUE either distorts the definition of a model-based object or adds meaningless artificial objects.

When a significant process or transformation in the domain is not a natural responsibility of an ENTITY or VALUE OBJECT, add an operation to the model as an standalone interface declared as a SERVICE. Define the interface in terms of the language of the model and make sure the operation name is part of the UBIQUITOUS LANGUAGE. Make the SERVICE stateless.

Eric Evans, "A Model Expressed in Software", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 105-106.

Value objects

An object that represent a descriptive aspect of the domain with no conceptual identity is called a VALUE OBJECT. VALUE OBJECTS are instantiated to represent elements of the design that we care about only for what they are, not who or which they are.

When you care only about the attributes of an element of the model, classify it as a VALUE OBJECT. Make it express the meaning of the attributes it conveys and give it related functionality. Treat the VALUE OBJECT as immutable. Dont' give it any identity and avoid the design complexities necessary to maintain ENTITIES.

Eric Evans, "A Model Expressed in Software", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 98-99.

Entities (a.k.a. reference objects)

An object defined primarily by its identity is called an ENTITY. ENTITIES have special modeling and design considerations. They have life cycles that can radically change their form and content, but a thread of continuity must be maintained. Their identities must be defined so that they can be effectively tracked. Their class definitions, responsibilities, attributes, and associations should revolve around who they are, rather than the particular attributes they carry. Even for ENTITIES that don't transform so radically or have such complicated life cycles, placing them in the semantic category leads to more lucid models and more robust implementations.

The model must define what it means to be the same thing.

Eric Evans, "A Model Expressed in Software", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 91-92.

Not a software engineer, but a software cook

Even the best teams will have cruft in their software. The difference is that the best teams both create much less cruft but also remove enough of the cruft they do create that they can continue to add features quickly. [...] A common metaphor is that it's like cleaning up work surfaces and equipment in the kitchen. You can't not make things dirty when you cook, but if you don't clean things quickly, muck dries up, is harder to remove, and all the dirty stuff gets in the way of cooking the next dish.

Martin Fowler, in Is High Quality Software Worth the Cost?.

Layered architecture

Partition a complex program into layers. Develop a design within each layer that is cohesive and that depends only on the layers below. Follow standard architectural patterns to provide loose coupling to the layers above. Concentrate all the code related to the domain model in one layer and isolate it from the user interface, application, and infrastructure code. The domain objects, free of the responsibility of displaying themselves, storing themselves, managing application tasks, and so forth, can be focused on expressing the domain model. This allows a model to evolve to be rich enough and clear enough to capture essential business knowledge and put it to work.

The domain layer is where the model lives

The domain model is a set of concepts. The "domain layer" is the manifestation of that model and all directly related design elements. The design and implementation of business logic constitute the domain layer. In a MODEL-DRIVEN DESIGN, the software constructs of the domain layer mirror the model concepts.

It is not practical to achieve that correspondence when the domain logic is mixed with other concerns of the program. Isolating the domain implementation is a prerequisite for domain-driven design.

Eric Evans, "Isolating the Domain", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 70-71, 75.

Model-driven design

Why models matter to users

When a design is based on a model that reflects the basic concerns of the users and domain experts, the bones of the design can be revealed to the user to a greater extent than with other design approaches. Revealing the model gives the user more access to the potential of the software and yields consistent, predictable behavior.

Hands-on modelers

Manufacturing is a popular metaphor for software development. One inference from this metaphor: highly skilled engineers design; less skilled laborers assemble the products. This metaphor has messed up a lot of projects for one simple reason — software development is all design. All teams have specialized roles for members, but overseparation of responsibility for analysis, modeling, design, and programming interferes with MODEL-DRIVEN DESIGN.

Eric Evans, "Binding Model and Implementation", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 59-60.

Written design documents

Documents should complement code and speech

Each Agile process has its own philosophy about documents. Extreme Programming advocates usign no extra design documents at all and letting the code speak for itself. Running code doesn't lie, as any other document might.The behavior of running code is unambiguous.

But the code as design document does have its limits. It can overwhelm the reader with detail. Although its behavior is unambiguous, that doesn't mean its obvious. And the meaning behind a behavior can be hard to convey. In other words, documenting exclusively through code has some of the same basic problems as using comprehensive UML diagrams. Of course, massive spoken communication within the team gives context and guidance around the code, but it is ephemeral and localized. And developers are not the only people who need to understand the model.

A document shouldn't try to do what the code already does well. [...] Written documents should complement the code and the talking.

Documents should work for a living and stay current

Listen to the UBIQUITOUS LANGUAGE and how it is changing. If the terms explained in a design document don't start showing up in conversations and code, the document is not fulfilling its purpose. Maybe the document is too big or complicated. Maybe it is not focused on a sufficiently important topic. People are either not reading it or not finding it compelling. If it is having no impact on the UBIQUITOUS LANGUAGE, something is wrong.

Conversely, you may hear the UBIQUITOUS LANGUAGE changing naturally while a document is being left behind. Evidently the document does not seem relevant to people or does not seem important enough to update. It could safely be archived as history, but left active it could create confusion and hurt the project. And if a document isn't playing an important role, keeping it up to date through sheer will and discipline wastes effort.

Eric Evans, "Communication and the Use of Language", in Domain-Driven Design: Tackling Complexity in the Heart of Software, 37-39.

Design versus development process

When people learn design techniques, they feel excited by the possibilities. Then the messy realities of a real project descend on them. They can't fit the new design ideas with the technology they must use. Or they don't know when to let go of a particular design aspect in the interest of time and when to dig in their heels and find a clean solution. Developers can and do talk with each other abstractly about the application of design principles, but it is more natural to talk about how real things get done. So, although this is a design book, I'm going to barge right across that artificial boundary into process when I need to. This will help put design principles in context.

Eric Evans, "Preface", in Domain-Driven Design: Tackling Complexity in the Heart of Software, xxii.

Domain-driven design

In recent years there has been a rebellion against elaborate development methodologies that burden projects with useless, static documents and obsessive upfront planning and design. Instead, the Agile processes, such as XP, emphasize the ability to cope with change and uncertainty.

Extreme Programming recognizes the importance of design decisions, but it strongly resists upfront design. Instead, it puts an admirable effort into communication and improving the project's ability to change course rapidly. With that ability to react, developers can use the "simplest thing that could work" at any stage of a project and then continuously refactor, making many small design improvements, ultimately arriving at a design that fits the customer's true needs.

This minimalism has been a much-needed antidote to some of the excesses of design enthusiasts. Projects have been bogged down by cumbersome documents that provided little value. They have suffered from "analysis paralysis" with team members so afraid of an imperfect design that they made no progress at all. Something had to change.

Unfortunately, some of these process ideas can be misinterpreted. Each person has a differente definition of "simplest". Continuous refactoring is a series of small redesigns; developers without solid design principles will produce a code base that is hard to understand or change - the opposite of agility. And although fear of unantecipated requirements often leads to overengineering, the attempt to avoid overengineering can develop into another fear: a fear of doing any deep design thinking at all.

In fact, XP works best for developers with a sharp design sense. The XP process assumes that you can improve a design by refactoring, and that you will do this often and rapidly. But past design choices make refactoring itself either easier or harder. The XP process attempts to increase team communication, but model and design choices clarify or confuse communication.

This book intertwines design and development practice and illustrates how domain-drive design and Agile development reinforce each other. A sophisticated approach to domain modeling within the context of an Agile development process will accelerate development. The interrelationship of process with domain development makes this approach more practical than any treatment of "pure" design in a vacuum.

Eric Evans, "Preface", in Domain-Driven Design: Tackling Complexity in the Heart of Software, xxiii.

All the little things

Sandi Metz's presentation on RailsConf 2014 is one of the best talks on OOP and refactoring that I've ever seen:

Main takeways:

• Refactor under green
• Inheritance is not evil
• Prefer duplication over the wrong abstraction
• "make the change easy (warning: this may be hard) then make the easy change" - Kent Beck
• Make small things (functions, methods, classes, etc.)
• Refactor through complexity

Invest in good software abstractions to simplify difficult problems

But like many other aspects of code quality, building an abstraction for a problem comes with tradeoffs. Building a generalized solution takes more time than building one specific to a given problem. To break even, the time saved by the abstraction for future engineers needs to outweigh the time invested. That’s more likely to happen with software the team relies heavily on — such as logging or user authentication libraries — than with peripheral parts of the codebase, so focus more energy on making the core abstractions great.

Edmond Lau, "Manage Complexity through Abstraction", in The Effective Engineer: How to Leverage Your Efforts In Software Engineering to Make a Disproportionate and Meaningful Impact, 113.