Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
As you may have seen on Twitter, I've been mulling over an idea for a new book, which I'm pleased to say is going to happen. It's currently titled "The Art of Visualising Software Architecture" and, as the title suggests, it will focus on the visual communication of software architecture through diagrams.
The core of the book is my C4 software architecture model and although this is covered in my existing Software Architecture for Developers book, I want to create a single resource related to this topic because I still see effective communication of software architecture as a huge gap across the software development industry. You'll notice that the title of this book includes the word "art". I've seen a number of debates over the years about whether software development is a craft or an engineering discipline. Although I think it *should* be an engineering discipline, I believe we're a number of years away from this being a reality. So while this book won't present a formalised, standardised method to communicate software architecture, it will provide a collection of ideas and techniques that thousands of people across the world have found useful.
I also want to include a number of other topics and answers to frequently asked questions that I get during my software architecture sketching workshops, including some of the blog posts I've written recently such as Help, my diagram doesn't fit on one page! and Diff'ing software architecture diagrams again, for example. I'm also going to include more discussion about notation, the various uses for diagrams, the value of creating a model and tooling. Structurizr will be in there too.
Thanks very much for all of the support so far on this; the tweets/e-mails I've had are telling me that this is the right decision. Since it's not going to be a long book and initial drafts may include some text copied verbatim from my "Software Architecture for Developers book", I'm going to make this available via Leanpub using their variable pricing model ... with a starting price of free, certainly for a while anyway. It's a work in progress, but please feel free to grab a copy from Leanpub if you're interested. Thanks very much!
This definitely goes into the category of a frequently asked question because it crops up time and time again, both during and after my software architecture sketching workshop.I'm following your C4 approach but my software system is much bigger than the example you use in your workshop. How do you deal with the complexity of these larger systems? And what happens when my diagram doesn't fit on one page?
Even with a relatively small software system, it's tempting to try and include the entire story on a single diagram. For example, if you have a web application, it seems logical to create a single component diagram that shows all of the components that make up that web application. Unless your software system really is that small, you're likely to run out of room on the diagram canvas or find it difficult to discover a layout that isn't cluttered by a myriad of overlapping lines. Using a larger diagram canvas can sometimes help, but large diagrams are usually hard to interpret and comprehend because the cognitive load is too high. And if nobody understands the diagram, nobody is going to look at it.
Instead, don't be afraid to split that single complex diagram into a larger number of simpler diagrams, each with a specific focus around a business area, functional area, functional grouping, bounded context, use case, user interaction, feature set, etc. You can see an example of this in One view or many?, where I create one component diagram per web MVC controller rather than having a single diagram showing all components. You can see this in action in the software architecture diagrams for techtribes.je that are hosted on Structurizr. The key is to ensure that each of the separate diagrams tells a different part of the same overall story, at the same level of abstraction.
While at the Devoxx UK conference recently, I was interviewed by Lucy Carey from Voxxed about software architecture, diagrams, monoliths, microservices, design thinking and modularity. You can watch this short interview (~5 minutes) at Step Away from the Code! ... enjoy!
While at the YOW! conference in Australia during December 2014, I was interviewed by Craig Smith and Tony Ponton for The Agile Revolution podcast. It's a short episode (28 minutes) but we certainly packed a lot in, with the discussion covering software architecture, my C4 model, technical leadership, agility, lightweight documentation and even a little bit about enterprise architecture.
Thanks to Craig and Tony for taking the time to do this and I hope you enjoy The Agile Revolution - Episode 91: Coding The Architecture with Simon Brown.
A printed copy of Talking with Tech Leads, by Patrick Kua (a Tech Lead at Thoughtworks), arrived in the post this morning. We often discuss the technical side of software development and rarely the âsofterâ side. This book is a collection of short interviews with people new to or in technical leadership roles. They share their stories, tips and experiences of making that leap from developer to tech lead. To clarify, by "tech lead", Patrick is referring to somebody in a technical leadership role who still writes code. This is synonymous with what I call the software architecture role, with an expectation that part of the role includes coding.
I actually recommend this book during my software architecture workshops as there are some fascinating âeurekaâ moments presented in the book, especially related to leadership and dealing with people. One of the messages you'll see repeated again and again is that, as software developers, nobody really prepares you for how to deal with people when you make the jump into your first tech lead role. Looking back at my own career and of people I worked with at the time, I'd say the same was true. Hindsight is a wonderful thing, and I wish I had a book like this earlier in my career.
I'd certainly recommend taking a look if you're interested in the softer side of the technical leadership/software architecture role. The book is available on Leanpub and as a printed book via Amazon.com and Amazon.co.uk.
Disclaimer: I'm interviewed in the book, as is Robert Annett and a bunch of other people you may recognise ... I receive no royalties for recommending it though. :-)
I'm delighted to say that I'll be presenting two conference keynotes during October, both about software architecture and a little bit about microservices.1. Software Architect 2015 in London, England
The first is titled Modular Monoliths at the Software Architect 2015 conference taking place in London. Sander Hoogendoorn is delivering the other keynote about microservices, and I hope to bring some balance to the discussion by asking this question: "if you canât build a well-structured monolith, what makes you think microservices is the answer?". I'll also be running a new workshop at the event called Extracting software architecture from code, but more on that later. The Software Architect conference is certainly evolving from year to year, and it's fantastic to see a wide range of topics related to software architecture, design and development.2. The Architecture Gathering in Munich, Germany
The week after, I'll be presenting another keynote titled Software architecture as code at a new conference in Munich called the The Architecture Gathering. It's a predominantly German language event and the content (from what I understand, and I don't speak German!) looks pretty interesting, as does the list of speakers.
I'm very much looking forward to both, especially as I think we've reached a turning point in our industry where people are starting to think about and appreciate the role that software architecture plays. See you there!
I can't read it, but seeing my C4 diagrams in Chinese is fun! Stay tuned for more translations.
If you like the look and feel of the C4 software architecture diagrams in my Software Architecture for Developers book (see examples here), Dennis Laumen has created an OmniGraffle stencil that will save you some time. Just download the stencil, install and it will appear in your stencil library.
The C4 stencil is available from Omni Group's Stenciltown. Thanks Dennis!
This is a quick update on my upcoming trip to Australia ... in conjunction with the lovely folks behind the YOW! conference, we've scheduled two public software architecture sketching workshops as follows.
This workshop addresses one of the major problems I've evidenced during my career in software development; namely that people find it really hard to describe, communicate and visualise the design of their software. Sure, we have UML, but very few people use it and instead resort to an ad hoc "boxes and lines" notation. This is fine, but the resulting diagrams (as illustrated below) need to make sense and they rarely do in my experience.
My workshop explores this problem from a number of different perspectives, not least by giving you an opportunity to practice these skills yourself. My Voxxed article titled Simple Sketches for Diagramming Your Software Architecture provides an introduction to this topic and the C4 model that I use. I've been running this workshop in various formats for nearly 10 years now and the techniques we'll cover have proven invaluable for software development teams around the world in the following situations:
In Diff'ing software architecture diagrams, I showed that creating a software architecture model with a textual format provides you with the ability to version control and diff different versions of the model. As a follow-up, somebody asked me whether Structurizr provides a way to recreate what Robert Annett originally posted in Diagrams for System Evolution. In other words, can the colours of the lines be changed? As you can see from the images below, the answer is yes.
To do this, you simply add some tags to the relationships and add the appropriate styles to the view configuration. structurizr.com will even auto-generate a key for you.
And yes, you can do the same with elements too. As this illustrates, the choice of approach is yours.
Robert Annett wrote a post titled Diagrams for System Evolution where he describes a simple approach to showing how to visually describe changes to a software architecture. In essence, in order to show how a system is to change, he'll draw different versions of the same diagram and use colour-coding to highlight the elements/relationships that will be added, removed or modified.
I've typically used a similar approach for describing as-is and to-be architectures in the past too. It's a technique that works well. Although you can version control diagrams, it's still tricky to diff them using a tool. One solution that addresses this problem is to not create diagrams, but instead create a textual description of your software architecture model that is then subsequently rendered with some tooling. You could do this with an architecture description language (such as Darwin) although I would much rather use my regular programming language instead.Creating a software architecture model as code
And since the diagrams were created by a model described as Java code, that description can be diff'ed using your regular toolchain.
Code provides opportunities
This perhaps isn't as obvious as Robert's visual approach, and I would likely still highlight the actual differences on diagrams using notation as Robert did too. Creating a textual description of a software architecture model does provide some interesting opportunities though.
I've blogged about architecturally-aligned testing before, which essentially says that our automated tests should be aligned with the constructs that we use to describe a software system. I want to expand upon this topic and clarify a few things, but first...Why?
In a nutshell, I think the terms "unit test" and "integration test" are ambiguous and open to interpretation. We tend to all have our own definition, but I've seen those definitions vary wildly, which again highlights that our industry often lacks a shared vocabulary. What is a "unit"? How big is a "unit"? And what does an "integration test" test the integration of? Are we integrating components? Or are we integrating our system with the database? The Wikipedia page for unit testing talks about individual classes or methods ... and that's good, so why don't we use a much more precise terminology instead that explicitly specifies the scope of the test?Align the names of the tests with the things they are testing
I won't pretend to have all of the answers here, but aligning the names of the tests with the things they are testing seems to be a good starting point. And if you have a nicely defined way of describing your software system, the names for those tests fall into place really easily. Again, this comes back to creating that shared vocabulary - a ubiquitous language for describing the structures at different levels of abstraction within a software system.
Here are some example test definitions for a software system written in Java or C#, based upon my C4 model (a software system is made up of containers, containers contain components, and components are made up of classes).
Name What? Test scope? Mocking, stubbing, etc? Lines of production code tested per test case? Speed Class tests Tests focused on individual classes, often by mocking out dependencies. These are typically referred to as âunitâ tests. A single class - one method at a time, with multiple test cases to test each path through the method if needed. Yes; classes that represent service providers (time, random numbers, key generators, non-deterministic behaviour, etc), interactions with âexternalâ services via inter-process communication (e.g. databases, messaging systems, file systems, other infrastructure services, etc ... the adapters in a âports and adaptersâ architecture). Tens Very fast running; test cases typically execute against resources in memory. Component tests Tests focused on components/services through their public interface. These are typically referred to as âintegrationâ tests and include interactions with âexternalâ services (e.g. databases, file systems, etc). See also ComponentTest on Martin Fowler's bliki. A component or service - one operation at a time, with multiple test cases to test each path through the operation if needed. Yes; other components. Hundreds Slightly slower; component tests may incur network hops to span containers (e.g. JVM to database). System tests UI, API, functional and acceptance tests (âend-to-endâ tests; from the outside-in). See also BroadStackTest on Martin Fowler's bliki. A single scenario, use case, user story, feature, etc across a whole software system. Not usually, although perhaps other systems if necessary. Thousands Slower still; a single test case may require an execution path through multiple containers (e.g. web browser to web server to database).Thoughts?
This definition of the tests doesn't say anything about TDD, when you write your tests and how many tests you should write. That's all interesting, but it's independent to the topic we're discussing here. So then, do *you* have a good clear definition of what "unit testing" means? Do your colleagues at work agree? Can you say the same for "integration testing"? What do you think about aligning the names of the tests with the things they are testing? Thoughts?
A quick note to say that my recent interview/chat with Sven Johann for Software Engineering Radio has now been published. As you might expect, it covers software architecture, diagramming, the use of UML, my C4 model, doing "just enough" and why the code should (but doesn't often) reflect the software architecture. You can listen to the podcast here. Enjoy!
Every system has an architecture (whether deliberately designed or not) and most systems change and evolve during their existence - which includes their architecture. Sometimes this happens gradually and sometimes in large jumps. Let's have a look at a simple example.
Basic context diagram for a furniture company
The main elements are:
The data flow is generally one way, with each system pushing out data with little feedback, due to an old batch-style architecture. For example, the designer updates product information but all the updates for the day are pushed downstream at end-of-day. The Warehouse application only accepts a feed from the Sales Orderbook, which includes both orders and product details.
There are multiple issues with a system like this:
The users are unhappy with the system and want the processes to change as follows:
Therefore the system is re-architected as follows:
This is a significant modification to how the systems works:
However, if you place the two diagrams next to each other they look similar and the scale of the change is not obvious.
When we write code, we would use a âdiffâ tool to show us the modifications.
I often do something similar on my architecture diagrams. For example:
System before modification:
System after modification:
When placed side-by-side the differences are much more obvious:
Iâve used similar colours to those used by popular diff tools; with red for a removed relationship and green for a new relationship (I would use the same colours if components were added or removed). Iâve used blue to indicate where a relationship has changed significantly. Note that Iâve included a Key on these diagrams.
You may be thinking; why have I coloured before and after diagrams rather than having a single, coloured diagram? A single diagram can work if you are only adding and removing elements but if they are being modified it becomes very cluttered. As an example, consider the relationship between the Sales Orderbook and the Product Catalogue. Would the description on a combined diagram be "product updates", "exchange sales and product information", or both? How would I show which was before and after? Lastly, what would I do with the arrow on the connection? Colourise it differently from the line?Conclusion
I find that having separate, coloured diagrams for the current and desired system architectures allows the changes to be seen much more clearly. Having spoken to several software architects, it appears that many use similar techniques, however there appears to be no standard way to do this. Therefore we should make sure that we follow standard advice such as using keys and labelling all elements and connections.
I ran my Extracting software architecture from code session at Skills Matter this evening and it was a lot of fun. The basic premise of the short workshop part was simple; "here's some code, now draw some software architecture diagrams to describe it". Some people did this individually and others worked in groups. It sounds easy, but you can see for yourself what happened.
There are certainly some common themes, but each diagram is different. Also, people's perception of what architectural information can be extracted from the code differed slightly too, but more on that topic another day. If you want to have a go yourself, the codebase I used was a slightly cutdown version* of the Spring PetClinic web application. The presentation parts of the session were videoed and I'm creating a 1-day version of this workshop that I'll be running at a conference or two in the Autumn.
This again raises some basic questions about the software development industry though. Why, in 2015, do we still not have a consistent way to do this? Why don't we have a shared vocabulary? When will we be able to genuinely call ourselves "software engineers"?
Here's a question I'm being asked more and more ... how do you diagram a microservices architecture with the C4 software architecture model?
It's actually quite straightforward providing that you have a defined view of what a microservice is. If a typical modular monolithic application is a container with a number of components running inside it, a microservice is simply a container with a much smaller number of components running inside it. The actual number of components will depend on your implementation strategy. It could range from the very simple (i.e. one, where a microservice is a container with a single component running inside) through to something like a mini-layered or hexagonal architecture. And by "container", I mean anything ranging from a traditional web server (e.g. Apache Tomcat, IIS, etc) through to something like a self-contained Spring Boot or Dropwizard application. In concrete terms then...
So there you go, that's how I would approach diagramming a microservices architecture with the C4 model.
I'm running a short "In The Brain" session at Skills Matter in London next Monday evening, focussed around the topic of extracting the software architecture model from code.Itâs often said that the code is the true embodiement of the software architecture, yet my experience suggests that itâs difficult to actually extract this information from the code. Why isnât the architecture in the code? Join me for this "In The Brain" session where weâll look at a simple Java web application to see what information we can extract from the code and how to supplement it with information we can't. This session will be interactive, so bring a laptop.
This will be an interactive session so you will need to bring a laptop, or at least something that you can browse a GitHub repository with. We'll be looking at a Java web application, but the concepts are applicable to other programming languages and platforms. See you next week.
"This is a component of our system", says one developer, pointing to a box on a diagram labelled "Web Application". Next time you're sitting in an conversation about software design, listen out for how people use terms like "component", "module", "sub-system", etc. We can debate whether UML is a good notation to visually communicate the design of a software system, but we have a more fundamental problem in our industry. That problem is our vocabulary, or rather, the lack of it.Notation
I've been running my software architecture sketching exercises in various formats for nearly ten years, and thousands of people have participated. The premise is very simple - here are some requirements, design a software solution and draw some pictures to illustrate the design. The range of diagrams I've seen, and still see, is astounding. The percentage of people who choose to use UML is tiny, with most people choosing to use an informal boxes and lines notation instead. With some simple guidance, the notational aspects of the diagrams are easy to clean up. There's a list of tips in my book that can be summarised with this slide from my workshop.
What's more important though is the set of abstractions used. What exactly are people supposed to be drawing? How should they think about, describe and communicate the design of their software system? The primary aspect I'm interested in is the static structure. And I'm interested in the static structure from different levels of abstraction. Once this static structure is understood and in place, it's easy to supplement it with other views to illustrate runtime/behavioural characteristics, infrastructure, deployment models, etc.
In order to get to this point though, we need to agree upon some vocabulary. And this is the step that is usually missed during my workshops. Teams charge headlong into the exercise without having a shared understanding of the terms they are using. I've witnessed groups of people having design discussions using terms like "component" where they are clearly not talking about the same thing. Yet everybody in the group is oblivious to this. For me, the answer is simple. Each group needs to agree upon the vocabulary, terminology and abstractions they are going to use. The notation can then evolve.
My Simple Sketches for Diagramming Your Software Architecture article explains why I believe that abstractions are more important than notation. Maps are a great example of this. Two maps of the same location will show the same things, but they will often use different notations. The key to understanding these different maps is exactly that - a key tucked away in the corner of each map somewhere. I teach people my C4 model, based upon a simple set of abstractions (software systems, containers, components and classes), which can be used to describe the static structure of a software system from a number of different levels of abstraction. A common set of abstractions allows you to have better conversations and easily compare solutions. In my workshops, the notation people use to represent this static structure is their choice, with the caveat that it must be self-describing and understandable by other people without explanation.
Next time you have a design discussion, especially if it's focussed around some squiggles on a whiteboard, stop for a second and take a step back to make sure that everybody has a shared understanding of the vocabulary, terminology and abstractions that are being used. If this isn't the case, take some time to agree upon it. You might be surprised with the outcome.