Observations on the practice of software architecture - III

"It is the developer’s assumptions which get shipped to production" - Alberto Brandolini

In parts one and two of this series we explored why software architecture is needed and some of the common pitfalls and failure modes that afflict "architects". This article will go into some of the principles and practices that are part of my toolbox. Very little of this is new and I've tried to link to the original sources so be sure to check the references. If I have misrepresented anything, that's solely on me.

Before we start, let's dive into what "Software Architecture" is, as there are different definitions out there:

• Software architecture is the study of the components of a system and their relationships;

• Software architecture is the collection of important/irreversible design decisions;

Both of these definitions are useful, however I would venture a slightly different one: Software architecture is a method of inquiry, for creating a shared understanding of how systems operate in their organizational context, helping drive better (i.e. fit for purpose, sustainable in the particular context) technical decisions.

A consequence of the definition I posited is that Architecture is not "a thing" per se, it is a process - it is never "done". Artifacts like diagrams or specific techniques to reason about a problem (e.g. DDD) have their place, but ultimately these are instruments to help make sense of the socio-technical environment we are operating in. It is context-sensitive and it should be practiced across the organization. You don't need to be one of the anointed ones to practice architecture.

Enable distributed decision making

Building software in larger organizations is a team sport, therefore centralized decision making, is fragile and very easily becomes counterproductive. Therefore design and decision-making should be decentralized, with teams on the driver's seat, and ultimately responsible for crafting the decisions that directly affect them. And since decision-making has an almost fractal quality to it as happens at all levels, thoughtful, disciplined, hopefully not-too-bad decision-making is a skill that software engineers can and should learn.

In order to support teams, and let people develop their design chops, establishing an ensemble practice for example via an "Architecture Advisory Forum" ¹ where team members are free to come with proposals on the evolution and change of their systems can be an excellent tool. The goal is for the group to engage as peers and help surface tradeoffs, different implementation options, or viability crushing constraints that may affect the proposal and shape the ultimate decision².

This can serve as the connective tissue between people operating in different scopes and abstraction levels, and provides a forum for practices such as:

• "Bytesize Architecture Sessions"³ to surface the assumptions and mental models that people have about the systems under their purview;

• "Let's break it" sessions where the goal is to come up with stressors⁴ and see how the current architecture, that is the individual systems and their relationships, are able to cope. These "random walks" through your system often uncover interesting inconsistencies and other urban legends.

• Encouraging a writing culture. If you work long enough in this industry you will find yourself in the situation where your team inherited a piece of software that has some implementation decisions that are... well questionable. And to make things worse, the reason as to why they came to be has been lost to the sands of time! What do you do then? Do you live with this because you're afraid of unintended consequences of your action - a classic case of Chesterton's fence⁵?

  Capturing why a decision was taken at a particular point in time can be a life-saver in those circumstances, through ADRs⁶ for example. In addition to this, clear writing helps clarify and structure ideas while providing a good basis for communication and knowledge sharing. This is often labeled (dismissively so) as a "soft-skill", but good writing skills are crucial for growth especially in the age of geographically distributed teams and work-from-home.⁷

Advisory

Teams should have the ultimate word about the decisions affecting them. Unless working in the team directly, a "software architect" should behave as an advisor to the team. The reason for this is that the team is usually in the best position to come up with a solution that works in their context, balancing relevant attributes (performance, security, scalability, etc) with the inevitable trade offs in terms of feasibility, cognitive load and operational overhead. As a rule of thumb: If you're not on the on-call rotation, then your default position should be that of an advisor. This will demonstrate trust, and will empower teams to own their decisions and learn from their mistakes.

Air support

If an organization has dedicated architecture roles (can also apply to Staff+ roles) a key part of the job will be to use the organizational power that comes with the title to provide "air support" for teams. This means making their work visible and readable to others when and where it is appropriate to do so. The role of the architect in this context can be crucial in order to allow different stakeholders to "connect the dots" and contextualize why certain technical investments are coherent with the overall direction where the organization wants to go and/or provide more optionality later down the line^{8 9}.

Enabling constraints

We often think of constraints in a negative light, after all constraining the range of permissible options seems like a bad thing, right? It actually depends, this is one of the most powerful tools that senior technical folks have at their disposal. Setting some constraints on the range of permissible technical options can actually be empowering. The classic example is Jeff Bezos mandate that all services in Amazon should communicate via "service interfaces", AKA APIs, that are designed to be externalizable. This certainly constrains how teams integrate their services, for example integrations via sharing a database or memory is off-limits, however teams now have more freedom to choose the right technology for the job, and the organization's resources and know-how can be focused on supporting this style of integration.

Such constraints need to be applied judiciously, but they can enable teams to operate with less friction in a particular socio-technical ecosystem¹⁰. So some constraints on system integration patterns, languages and frameworks usage, or how and where to find system documentation can actually be quite helpful to teams (just remember there may be exceptions!). And if we look at the stability of a system as a function of connectivity and variability - stability = connectivity x variability - then reducing variability, for instance by introducing some constraints on how services integrate with one another, can have a positive impact by creating a more stable system overall¹¹.`

"Every software‐intensive system has an architecture. In some cases that architecture is intentional, while in others it is accidental. Most of the time it is both" - Grady Booch

As we build more ambitious software intensive systems and software permeates more and more of our daily lives, often producing unexpected and surprising results, software should be thoughtfully designed.

Outside of very specific teams and social contexts, a fully emergent, one could say quasi-anarchical approach to software design will fail. The larger question, especially as waterfall-esque thinking seems to be gaining some momentum¹², is how to successfully integrate software architecture with an iterative, empirical software development process. This entails empowering teams to make better decisions, and taking a balanced approach with just the right level of decision making process and constraints imposed on teams¹³. This article explored my current toolbox to do so and hopefully this will be useful for some of you out there.

---

Footnotes

1. The idea of an "Architecture Advisory Forum" is explained in further detail here. ↩

2. I find it important, especially when initially introducing such a practice to come up with some proposed structure in mind - for example focus on discussing ADRs for new features. This should undergo some iterations until it works in your context. If you're a more senior engineer introducing this practice, don't be afraid to start with some ground rules and depending on the maturity of your organization make sure that the ways of working are set in a charter-like document so that everyone knows the rules-of-engagement. ↩

3. Bytesize Architecture Sessions ↩

4. Barry O'Reilly makes a very compelling case for the idea that random simulation can yield resilient architectures. Definitely check this introduction to the topic of [Residuality theory](https://www.youtube.com/ watch?v=0wcUG2EV-7E) ↩

5. Chesterton's Fence ↩

6. ADR: Architecture Decision Record. Love Unrequited: The Story of Architecture, Agile, and How Architecture Decision Records Brought Them Together ↩

7. When trying to reason about how to communicate technical decisions I find that Good Strategy, Bad Strategy: The difference and why it matters provides very good advice on writing documents that have some substance in them, instead of just playing buzzword bingo. ↩

8. If this sounds like playing politics, well yes it is. Politics gets a bad reputation - perhaps deservedly so - however one should keep in mind that it is through politics that societies become more than just a sum of individuals and their choices. This is how humans organize collective, coherent and cohesive action at scale. ↩

9. Gregor Hohpe's book: The Software Architect Elevator: Redefining the Architect's Role in the Digital Enterprise provides a good metaphor for this sort of work. A big part of the role of an architect in a large organization is to ride an elevator from the engine room where technical decisions all the way up to the penthouse where key management decisions are taken. ↩

10. Ruth Malan's free ebook is great and offers additional insight into this. Her courses are also really good, I highly recommend them. ↩

11. This is one of the topics that Residuality Theory expands upon. You can find more about Kauffman Networks and how this can be valuable in the context of sofware engineering, which is essentially what I am getting to in Residues: Time, Change, and Uncertainty in Software Architecture. ↩

12. One could argue that the idea of treating software development as a mechanistic productive process with well defined stages has some appeal to organizations. This is wrong because that is not the game we are playing - we're really playing a design and intrinsically exploratory game - but nevertheless Waterfall keeps raising it's head. Dave Farley and Kent Beck had an excellent conversation that touched on the subject. ↩

13. Eduardo Silva's article on Minimalistic Architecture is quite good: Less is More with Minimalist Architecture, Ruth Malan and Dana Bredemeyer ↩