Architectural coupling

Make sure your architecture matches the problem domain. Don't try to force fit an unsuitable architecture.

Monoliths

Monolithic architectures, particularly layered architectures, are a common choice when starting a project because developers understand the structure easily. However, many monoliths reach end of life and must be replaced because of decreasing performance, size of code base, and a host of other factors. A current common target for monolith migration is microservices-style architectures, which are more complex than monolithic architectures in areas like service and data granularity, operationalization, coordination, transactions, and so on. If a development team has a hard time building one of the simplest architectures, how will moving to a more complex architecture solve their problems?

If you can't build a monolith, what makes you think microservices are the answer?

Simon Brown

Before embarking on an expensive architecture restructuring exercise, architects may benefit from improved modularization of what's already present. If nothing else, it's an excellent starting point for the more serious restructuring that follows.

Microservices

In microservices architecture, the domain encapsulates technical and other architectures, making evolution across domain dimensions easy. No one perspective on architecture is "correct", but rather a reflection on the goals developers build into their projects. If the focus is entirely on technical architecture, then making changes across that dimension is easier. However, if the domain perspective is ignored, then evolving across that dimension is no better than the Big Ball of Mud.

One of the major factors that impacts the ability to evolve an application at the architectural level is how unintentionally coupled each part of the system is. For example, in a layered architecture, architects specifically couple layers together in an intentional way. However, the domain dimension is unintentionally coupled, making evolution in that dimension difficult, because the architecture is designed around technical architecture layers, not the domain. Thus, one of the important aspects of an evolvable architecture is appropriate coupling across dimensions.

Neal Ford, Rebecca Parsons & Patrick Kua, "Architectural Coupling", in Building Evolutionary Architectures: Support Constant Change, 57-58, 74, 78.

Evitando chamadas custosas em testes automáticos

Ao escrever testes automáticos, é comum utilizarmos Test Doubles para substituir dependências diretas do código sob teste. Além de ter impacto direto na forma de pensar e construir tanto os casos de teste quanto o próprio código sob teste, essa prática também permite eliminar chamadas custosas e/ou não confiáveis durante a execução dos testes, garantindo assim mais eficiência e confiabilidade para a suíte de testes como um todo.

Por exemplo, suponha que precisemos testar uma função ou objeto que efetue uma requisição HTTP para uma API externa à nossa aplicação. Podemos criar um mock para o trecho de código que faz a requisição, de forma a simular o retorno esperado e, assim, não ter que fazer a chamada real de rede. Ganhamos em eficiência, pois, sem a requisição, não é necessário lidar com a latência de rede durante a execução dos testes. Ganhamos também em confiabilidade, pois, como chamadas HTTP podem falhar, qualquer requisição externa tem o potencial de gerar um falso positivo na suíte de testes; removendo a chamada, removemos essa possibilidade.

Pensando nisso, é de vital importância que numa base de código grande — com uma suíte de testes, em geral, igualmente grande — consigamos automaticamente garantir que chamadas custosas sejam apropriadamente mockadas. No caso de requisições de rede, podemos simplesmente bloquear ou restringir chamadas de socket durante a execução dos testes. É o que o pytest-socket, por exemplo, faz. Para quem usa o pytest, basta instalar o plugin e executar pytest --disable-socket. Qualquer teste em que uma chamada de socket for feita (mesmo por pacotes terceiros importados) irá falhar com SocketBlockedError.

Para códigos com custo de computação alto (ex: uma função com um pesado cálculo recursivo) que façam parte da aplicação, uma forma de evitar seu uso sem mock em testes é torna esse requisito uma parte do próprio código, isto é, permitir que a pessoa que o desenvolveu adicione alguma marcação que bloqueie a chamada em ambiente de teste. Não conhecendo uma solução já existente para isso em Python, me aventurei em criar a minha própria e daí surgiu o decorador enforce_mock_in_tests. Esse decorador pode ser aplicado a classes ou funções. A cada chamada do código decorado, ele avalia se o ambiente é de teste ou não e, se for, uma exceção é lançada, dando uma clara indicação ao desenvolvedor de que aquele código não deve ser chamado diretamente nos casos de teste, mas sim mockado. Além do código do decorador em si, o Gist abaixo contém também um arquivo com casos de teste para ele, permitindo ver como o decorador pode ser usado no código da aplicação.

Engineering incremental change

Architecture is abstract until operationalized, when it becomes a living thing.

Conflicting goals

The agile software development process has taught us that the sooner a developer can detect problems, the less effort is required to fix them. One of the side effects of broadly considering all the dimensions in software architecture is the early identification of goals that conflict across dimensions. For example, developers at an organization may want to support the most aggressive pace of change to support new features. Fast change to code implies fast changes to database schemas, but the database administrators are more concerned about stability because they are building a data warehouse. The two evolution goals conflict across the technical and data architecture.

Obviously, some compromise must occur, taking into account the myriad factors that affect the underlying business. Using architecture dimensions as a technique for identifying portions of concern in architecture (plus fitness functions to evaluate them) allows an apples-to-apples comparison, making the prioritization exercise more informed.

Conflicting goals are inevitable. However, discovering and quantifying those conflicts early allows architects to make better informed decisions and create more clearly defined goals and principles.

Hypothesis- and data-driven development

[...] Using hypothesis-driven development, we can incorporate users in an unprecedented way, learning from behavior and building what users really find valuable.

Hypothesis-driven development requires the coordination of many moving parts: evolutionary architecture, modern DevOps, modified requirements gathering, and the ability to run multiple versions of an application simultaneously. Service-based architectures (like microservices) usually achieve side-by-side versions by intelligent routing of services. For example, one user may execute the application using a particular constellation of services while another request may use an entirely different set of instances of the same services. If most services include many running instances (for scalability, for example), it becomes trivial to make some of those instances slightly different with enhanced functionality, and to route some users to those features.

Experiments should run long enough to yield significant results. Generally, it is preferable to find a measurable way to determine better outcomes rather than annoy users with things like pop-up surveys. For example, does one hypothesized workflow allow the user to complete a task with fewer keystrokes and clicks? By silently incorporating users into the development and design feedback loop, you can build much more functional software.

Neal Ford, Rebecca Parsons & Patrick Kua, "Engineering Incremental Change", in Building Evolutionary Architectures: Support Constant Change, 29, 39-40, 44.

Evolutionary architectures and Conway's law

An evolutionary architecture supports guided, incremental change across multiple dimensions.

Neal Ford, Rebecca Parsons & Patrick Kua, "Software Architecture", in Building Evolutionary Architectures: Support Constant Change, 6.

Structure teams to look like your target architecture, and it will be easier to achieve it.

Neal Ford, Rebecca Parsons & Patrick Kua, "Software Architecture", in Building Evolutionary Architectures: Support Constant Change, 13.

How to be managed

Part of being a good manager is figuring out how to be managed. This is not exactly the same as managing up, although it is related. Developing a sense of ownership and authority for your own experiences at work, and not relying on your manager to set the entire tone for your relationship, is an important step in owning your career and workplace happiness.

Spend time thinking about what you want

Your manager can point out opportunities for growth. She can show you projects. She can provide feedback on your areas of learning and development. But she cannot read your mind, and she cannot tell you what will make you happy. Whether you are brand new to the workplace or 20 years into your career, the onus of figuring out what you want to do, what you want to learn, and what will make you happy rests on your shoulders.

As you go through various stages of your career, you'll start to realize how much uncertainty there is in the world. It’s a pretty universal truth that once you get the job you thought you wanted, the enjoyment eventually fades and you find yourself looking for something else. You think you want to work for that cool startup, and you get there only to find it's a mess. You think you want to be a manager, only to discover that the job is hard and not rewarding in the ways you expected.

In all of this uncertainty, the only person you can rely on to pull through it is yourself. Your manager cannot do that for you. Use your manager to discover what's possible where you are, but look to understand yourself in order to figure out where you want to go next.

You are responsible for yourself

Knowing yourself is step one. Step two is going after what you want.

Bring agendas to your 1-1s when you have things you need to talk about. When you want to work on projects, ask. Advocate for yourself. When your manager isn't helpful, look for other places to get help. Seek out feedback, including constructive feedback on areas to improve. When that feedback comes to you, take it graciously, even when you don't agree with it.

When you are persistently unhappy, say something. When you are stuck, ask for help. When you want a raise, ask for it. When you want a promotion, find out what you need to do to get it. Your manager cannot force work–life balance on you. If you want to go home, figure out how to get your work done and go home. Sometimes you will have to go against the cultural grain to set your own boundaries, and that will feel uncomfortable. On the flip side, sometimes if you want a bigger job, you will have to work more hours to get it.

You will not get everything you ask for, and asking is not usually a fun or comfortable experience. However, it's the fastest way forward. If your manager is conscientious, he'll appreciate your candor. He may not be conscientious, or he may like you less for asking, and then you'll know that about your current situation. I can't guarantee you that it'll go well, but if you've set a goal for yourself, you owe it to yourself to do what you can to make it happen.

Give your manager a break

This is a job. Your manager will be stressed out sometimes. She'll be imperfect. She will say dumb things, or do things that feel unfair or harmful to you. She'll give you work that you don't want to do, and get annoyed when you complain about doing it. Her job is to do the best thing for the company and the team. It is not to do whatever it takes to make you happy all the time.

Your relationship with your manager is like any other close interpersonal relationship. The only person you can change is yourself. You should absolutely provide feedback to your manager, but understand that she may not listen or change no matter how much you think she should. If you find yourself starting to actively resent your manager for whatever reason, you probably need to move to a different team or look for a new job. If you find yourself resenting every manager you work for, you may need to think about whether the cause is them or you. Perhaps you'd be happier in a job where you don't have a manager.

Especially as you become more senior, remember that your manager expects you to bring solutions, not problems. Try not to make every 1-1 about how you need something, how something is wrong, or how you want something more. When you have a problem, instead of demanding that your manager solve it for you, try asking her for advice on how she might approach the problem. Asking for advice is always a good way to show respect and trust.

Choose your managers wisely

Your manager can make a huge difference in your career. So, as much as you can, consider not only the job, the company, and the pay, but also the manager when you are evaluating job opportunities.

Strong managers know how to play the game at their company. They can get you promoted; they can get you attention and feedback from important people. Strong managers have strong networks, and they can get you jobs even after you stop working for them.

There's a difference between a strong manager and a manager that you like as a friend, or even one you respect as an engineer. Plenty of great engineers make ineffective managers because they don't know or want to deal with the politics of leadership in their companies. A strong engineer may make a great mentor-manager to someone early in his career, but a terrible advocate-manager for someone who is more senior.

Camille Fournier, "Management 101", in The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change, chapter 1.

Bootstrapping culture

When you are in the role of senior engineering leader, part of your job is to set the culture of your function. A common failing of first-time CTOs is to underestimate the importance of being clear and thoughtful about the culture of the engineering team. Whether you are growing a new team or reforming an existing team, neglecting the team culture is a sure-fire way to make your job harder. As the team grows and evolves, it's important to attend to your culture as you would attend to any other important piece of infrastructure that you rely on.

For many people who are attracted to startup culture, the ideas of "structure" and "process" are seen as pointless at best and harmful at worst. [...] When talking about structure with skeptics, I try to reframe the discussion. Instead of talking about structure, I talk about learning. Instead of talking about process, I talk about transparency. [...] This learning and sharing is how organizations become more stable and more scalable over time.

Even when the overall company grows beyond the small group, the engineering team often pushes itself to stay unstructured. Hiring "full stack" engineers who are exclusively sourced from the professional and social networks of the current team results in low skill specialization and high homogeneity. Forcing the team to be collocated lowers communication barriers. And perhaps most critically, having an engineering team that operates solely as the execution arm of the product or founder makes the team highly task-oriented. [...] Be that as it may, the unstructured organization either displays characteristics that ultimately make it less self-directed than the members might wish to believe, or is run by hidden hierarchies and power dynamics. In many cases both things are true to some extent.

The example of the structureless team also applies to technical decisions and processes. There is a reason that you often find a lot of spaghetti code in early startups. When work is done to satisfy an immediate task, in a unified code base worked on by a team of interchangeables, the result is not usually a larger thoughtful structure, but a tweak here, a hack there — anything to get things done and moving forward. It's no surprise that we usually end up refactoring spaghetti code when we want to make it scalable, because refactoring usually involves identifying and explicitly drawing out structure in order to make the code base easier to read and work in.

That, in short, is the value of structure. Structure is how we scale, diversify, and take on more complex long-term tasks. We do it to our software, we do it to our teams, and we do it to our processes. In the same way that strong technical systems designers are capable of identifying and shaping underlying system structures, strong leaders are capable of identifying and shaping underlying team structures and dynamics, and doing so in a way that supports the long-term goals of the team and equips the individuals to achieve their best.

Nothing is more ridiculous than a small team with a rigid hierarchy. [...] However, it's more common in small companies to see structure come too late. The problems creep up slowly. One person gets used to making all of the decisions and changing his mind frequently. This strategy works fine when it's just him and a couple of others. But when he keeps doing it with a team of 10, a team of 20, a team of 50, what you start to see is a high degree of confusion and wasted effort. The cost to change his mind becomes more and more expensive.

Assessing your role

I don't think there's a huge benefit in overdesigning your team structure or process when your team is small and functioning well. However, at some point you'll start to experience failure, and failure is the best place to investigate and identify where your structure needs to change.

My advice to leaders is simple: when failures occur, examine all aspects of reality that are contributing to those failures. The patterns you see are opportunities to evolve your structure, either by creating more or different structure or removing it. [...] What about examining success? Well, you can learn things from success, but it is often a poor teacher. Ironically, while luck plays a role in both failure and success, we often attribute failure to bad luck and success to our own actions. [...] If you want to learn from success, make sure you can identify the actual improvement you're seeking when applying those lessons more broadly, and that you understand the context required to repeat that success.

Learning rarely comes for free. Analyzing situations and thinking about good takeaways takes time. If the value of your future time is less than the value of your current time, then you're probably not going to worry too much about saving future time. Just because your company is big, old, and stable doesn't mean you can have as much rigid, unchanging structure as you want. [...] But if you don't adopt structure when you need it, things can also go wrong.

When every new hire slows the team down for months because there is no onboarding process, that is a failure due to lack of structure. When people regularly leave the company because they have no path to advancement or career growth, that is a failure due to lack of structure. The third time you have a production outage because someone logged directly into the database and accidentally dropped a critical table, that is a failure due to lack of structure. I said earlier that I prefer to talk about learning and transparency rather than using the word structure, because really what we're talking about here is identifying the causes of failures, especially frequent failures, and trying to figure out what we can change to solve for those failures. This is fundamentally about learning.

Creating your culture

One of the things I have come to believe strongly is that culture is real; it's also incredibly important, and it's something that many people don't understand at all. It's both an easy, natural consequence of your company's evolution and something that can quickly become a problem if you don't tend to it. Consciously guiding the culture of your team is part of a leader's job, and to do this well, you need to understand what it means in the first place.

So what is culture? Culture is the generally unspoken shared rules of a community. [...] Culture doesn't mean that every single person holds exactly the same values, but it tends to guide a general overlap, and it creates a bunch of rules of interaction that you don't have to think much about if you are deeply ingrained in that culture.

People do make decisions using methods other than cultural values. They may adhere to the standards of a formal or informal contract, for example. They may do a pure data-driven analysis and determine the optimal outcome. But in complex environments where the needs of the group must override the needs of the individual, cultural values are the glue that enables us to work as a team and make decisions when faced with uncertainty. This is why figuring out and guiding your culture is such an important part of building a successful company.

Applying core values

Understand what your company's values are, understand what your team's values are, and think about what you personally value. Write the values down if they aren't already written, and try to be explicit. Use this explicit list to evaluate candidates, praise team members, and inform your performance review process.

Structuring cross-functional teams

The implications of the cross-functional structure are subtle. The values of everyone in these teams will start to change. In technology-focused structures where engineers work solely with other engineers, particularly engineers of their same "type" (mobile, backend, middleware, etc.), the focus is on being the best engineer by some measure of engineering excellence. People who design complex systems or who know the details of the latest iOS are the leaders and role models for the teams. In a product-focused structure, the leadership focus changes. Now the engineers who have the best product sense, the engineers who are capable of getting features done quickly and efficiently, and the engineers who communicate the best with the other functions will start to emerge as the leaders of the team.

I mean no value judgment here, but I encourage you to be aware of the product/business versus technology focus and apply it where it makes sense. What is truly important to the success of your company or your organization? If the most important thing is evolving a product that is a function of many different business areas coming together, you probably want leaders who have that business sense. On the other hand, in the areas where the technology must be rock-solid or exceptionally innovative and cutting-edge, you probably want teams that have more of an engineering focus and that are led by people who can design complex systems. You don't have to go entirely one way or the other, but recognize that one of these will lead the company as a whole, and — especially if your role is in senior management — focus your skill set on the one that the company itself most values and hire in for the other.

Camille Fournier, "Bootstrapping Culture", in The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change, chapter 9.

The big leagues

Ruling with fear, guiding with trust

How do you know if you're creating a culture of fear? It can come from placing a high value on being correct and following the rules, and having a strong affinity for hierarchy-based leadership. I also believe that coming from places where conflict was openly tolerated, if not actively encouraged, made me even more likely to create this culture. Engineering culture has a high tolerance for open debate to resolve conflict, so leaders who come from heavily engineering-focused backgrounds may feel particularly comfortable aggressively sparring with others over issues. Unfortunately, when you're the leader, the dynamic changes, and those who may have fought back when you were an individual contributor will feel threatened by you as a leader.

The culture of fear is pretty common in technology, and it survives best in environments where things are otherwise going well. Don't be fooled by external circunstances that enable your bad behavior. If you're feared but respected, the company is growing, and the team is working on interesting problems, you might get along OK for a time. However, if you lose any of these elements, you can expect to see people who have better options leave for greener pastures. I know firsthand that having a team that fears but respects you isn't enough when they're frustated by other things happening around them. So work on softening your rough edges, practice caring about your team as humans, and get curious. Building a culture of trust takes time, but the results are well worth it.

True north

A core role of senior leadership is sometimes overlooked. This role, played by the senior leader of a functional area (the CTO plays it for technology, the CFO plays it for finance, etc.), sets the baseline of what excellence looks like in this function. I call it "True North".

Technology leaders must help set the standard for True North in their organizations for different types of projects and exposures. Another way to think of this is through the lens of risk analysis. Risk analysis doesn't mean that we don't take risks. Some things that are generally considered "bad" can be OK under certain circunstances. These include:

• Having a single point of failure
• Having known bugs and issues
• Being unable to tolerate high load
• Losing data
• Putting out code that is undertested
• Having slow performance

There are situations and companies in which all of those risks are acceptable to take. That being said, True North helps us understand that all these issues must be carefully considered when we put code into production. Just because these rules have exceptions doesn't mean we forget that they exist.

I call this concept True North because it's important to understand it as an underlying pull, as a guiding instinct that we as leaders have developed over time and strive to help our teams as a whole develop as well. When our teams develop this instinct, they can be trusted to independently follow these guidelines without much direction or nudging.

Camille Fournier, "The Big Leagues", in The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change, chapter 8.

Leadership and management

If you're a leader with no power over business strategy and no ability to allocate people to important tasks, you're at best at the mercy of your influence with other executives and managers, and at worst a figurehead. You can't give up the responsibility of management without giving up the power that comes with it.

Camille Fournier, "What's a CTO?", in The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change, chapter 8.