Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Now I may be biased, but a quick look at my calendar hints to me that there's a renewed and growing interest in software architecture. Although I really like much of the improvement the agile movement has provided to the software development industry, I still can't help feeling that there are a large number of teams out there who struggle with a lack of process. After all, not every team is staffed with rockstar engineers! Although we've moved away from heavy prescriptive process frameworks, at least they provided a starting point for many of the activities associated with software development ... and this includes software architecture.
Put very simply, software architecture plays a pivotal role in the delivery of successful software yet it's frustratingly neglected by many teams. Whether performed by one person or shared amongst the team, the architecture role exists on even the most agile of teams yet the balance of up front and evolutionary thinking often reflects aspiration rather than reality. The big problem is that software architecture has fallen out of favour over the past decade or so. Here are five things that every software developer should know about it.1. Software architecture isn't about big design up front
Software architecture has traditionally been associated with big design up front and waterfall-style projects, where a team would ensure that every last element of the software design was considered before any code was written. Software architecture is basically about the high-level structure of a software system and how you get to an understanding of it. This is about the significant decisions that influence the shape of a software system rather than understanding how long every column in the database should be.2. Every software team needs to consider software architecture
Regardless of the size and complexity of the resulting product, every software team needs to consider software architecture. Why? Put simply, bad things tend to happen if they don't! If software architecture is about structure and vision, not thinking about this tends to lead to poorly structured, internally inconsistent software systems that are hard to understand, hard to maintain and potentially don't satisfy one or more of the important non-functional requirements such as performance, scalability or security. Explicitly thinking about software architecture provides you with a way to introduce technical leadership and stacks the odds of a successful delivery in your favour.3. The software architecture role is about coding, coaching and collaboration
The image that many people have of software architects is of traditional "ivory tower" software architects dictating instructions to an unsuspecting development team. It doesn't need to be like this though, with modern software architects preferring an approach that favours coding, coaching and collaborative design. The software architecture role doesn't necessarily need to be undertaken by a single person plus coding is a great way to understand whether the resulting architecture is actually going to work.4. You don't need to use UML
Again, traditional views of software architecture often conjure up images of huge UML (Unified Modeling Language) models that attempt to capture every last drop of detail. While creating and communicating a common vision is important, you don't need to use UML. In fact, you could argue that UML isn't a great method for communicating software architecture anyway. If you keep a few simple guidelines in mind, lightweight "boxes and lines" style sketches are an effective way to communicate software architecture.5. A good software architecture enables agility
There's a common misconception that "architecture" and "agile" are competing forces, there being a conflict between them. This simply isn't the case though. On the contrary, a good software architecture enables agility, helping you embrace and implement change. Good software architectures aren't created by themselves though, and some conscious effort is needed.
Since the code doesn't tell the whole story, I believe that all software teams should deliver some form of supplementary documentation. It must be lightweight and useful though. I think of such supplementary documentation as being a software guidebook and I use my C4 approach to create a set of simple software architecture sketches.
To be honest, many typical software architecture document templates aren't actually too bad as a starting point for supplementary documentation, but often the names of the various sections confuse people. If you glance over the list of section headings in my software guidebook, you might be wondering where the typical software architecture "views" are.
If you've not seen these before, there are a number of different ways to look at a software system. Examples include IEEE 1471, ISO/IEC/IEEE 42010, Philippe Kruchten's 4+1 model, etc. What they have in common is that they all provide different "views" onto a software system to describe different aspects of it. For example, there's often a "logical view", a "physical view", a "development view" and so on.
The big problem I've found with many of these approaches is that it starts to get confusing very quickly if people aren't versed in the terminology used. For example, I've heard people argue about what the difference between a "conceptual view" and a "logical view" is. And let's not even start asking questions about whether technology is permitted in a logical view! Perspective is important too. If I'm a software developer, is the "development view" about the code, or is that the "implementation view"? But what about the "physical view"? I mean, code is the physical output, right? But then "physical view" means something different to infrastructure architects. But what if the target deployment environment is virtual rather than physical?
My advice is, however you write documentation, just be clear on what it is you're trying to communicate and name the section accordingly. One option to resolve the terminology issue is to ensure that everybody on the team can point to a clear definition of what the various architectural views are. Software Systems Architecture by Eoin Woods and Nick Rozanski comes highly recommended in this regard. Another approach is to simply rename the sections to remove any ambiguity.
I still struggle to believe that, even in 2014, the role of software architects remains hugely misunderstood by many people in our industry. We generally understand the role of software developers, business analysts, project managers, ScrumMasters, testers, etc but a common definition of the software architecture role still seems to elude us. There's nothing new here, I've spoken and written about this before, as have many others. The thing that irritates me the most is the common knee-jerk reaction along the lines of...There is no such thing as a software architect and we don't need people telling us what to do. Software architects should write code along with the rest of the development team.
I agree 100% with the coding part of this statement. There are a number of reasons why software architects should be involved with the day-to-day coding activities on a software team. This is *my* preferred way of working. The "there's no such thing as a software architect" and "we don't need software architects" is shortsighted though.
Our industry is very hype-driven and it's common to hear people speaking at conferences about their "flat teams where we're all just developers". The implication here is that all of the other disciplines needed by the team (project management, business analysis, testing, etc) are also being accounted for by the team, typically through self-organisation, perhaps by agreeing that everybody is jointly responsible. In the latter case, this implies that the software architecture duties are being shared amongst the team too. In this situation, everybody is a software architect, except when they're not.
While I agree that software architects should be developers, most software developers are not architects. This is an important distinction. Many software teams out there are operating without anybody who understands the value of the software architecture role, let alone have anybody doing it. It's a technical leadership role and a lack of technical leadership leads to a whole raft of problems.
To give a real-world example, Troy Hunt wrote a great blog post last year called Should websites be required to publicly disclose their password storage strategy? that talks about how many websites are much less secure than they would make you believe. Some of the commenters suggest that perhaps the developers simply don't know any better, particularly when it comes to securing user passwords in a database. I agree, and while it would be great if all software developers had a thorough understanding of security, that's not going to happen overnight. Besides, we can't be experts in everything, but then we don't need to be. Such stories point to a lack of technical leadership. Every software team needs at least one person who is able to take a holistic view of the software being built, create a vision for the team to work with and ensure that any high priority risks are identified and mitigated. I'd say that these risks include security breaches.
If you think about software architects as being the equivalent to structural engineers in the building world, then it's not surprising that many software systems crash and burn. This frightening letter on Uncle Bob Martin's blog additionally highlights the need for more professionalism in the industry. I think it's time we started looking at how to tip the scales in balance of software development being more about engineering than craft, especially as we seem to be telling the next generation that coding is easy. Explicitly identifying those responsible for the technical leadership role, and educating others about it, is a step in the right direction.
One of the recurring questions I get asked whenever I talk to teams about software architecture is how it relates to techniques such as TDD, BDD, DDD, RDD, etc. The question really relates to whether xDD is a substitute for "software architecture". The short answer is no. The slightly longer answer is that the process of thinking about software architecture is really about putting some boundaries in place, inside which you can build your software using whatever xDD practices/principles you like.
For me, the "why?" is simple - you need to think about the things that play a huge part in influencing the resulting software architecture that you come up with, including:
Considering these important elements needn't take very long and can provide you with a starting point for the rest of the software design activities. Of course, this doesn't mean that the architecture shouldn't be changed, especially when you start writing code and getting feedback. The point is that you now have a framework and some boundaries to work within, which provide some often needed vision and guidance for the team. My experience suggests that a little direction can go a long way.
Although you won't generally find me talking about IT project management, it is a topic that overlaps with software architecture in a number of areas. For example, it's fairly typical to see software teams where a software architect and project manager are jointly responsible for the successful delivery of a software system. Here, the software architect is usually responsible for the technical risks and the project manager picks up all of the "other stuff" (e.g. risks associated with timescales, budget, scope, etc). Project management is an interesting area, particularly when you start throwing agile approaches into the mix. In the same way that I've discovered a lot of uncertainty about the software architecture role, my wife has discovered the same of project management during her career. And this has inspired her to write a book. To quote from start of it:My background is in software development and, having made the decision to become more involved with managing projects, I noticed that most websites, articles and books discuss project management from a very generic perspective. There seem to be very few books that take a more holistic view of IT project management including what skills you need, problems you may come up against due to technology and general tips on project management principles. After all, technology projects have their own unique activities and challenges.
The book is currently titled "Are you an IT project manager?" and is a work in progress, being published incrementally on Leanpub. I may be biased, but I'm looking forward to it evolving over the coming weeks, particularly as I've heard some of the stories that are yet to be written about.
Are you struggling to communicate/visualise the architecture of your software? If so, regardless of whether your software system is bespoke or based upon a platform (e.g. Microsoft SharePoint, Dynamics CRM, etc), you might be interested in a short article that I wrote for Methods & Tools. It's a summary (with examples) of the "C4" approach that I talk about in my Software Architecture for Developers book.If you're working in an agile software development team at the moment, take a look around at your environment. Whether it's physical or virtual, there's likely to be a story wall or Kanban board visualising the work yet to be started, in progress and done. Visualising your software development process is a fantastic way to introduce transparency because anybody can see, at a glance, a high-level snapshot of the current progress. As an industry, we've become pretty adept at visualising our software development process over the past few years although it seems we've forgotten how to visualise the actual software that we're building. I'm not just referring to post-project documentation, this also includes communication during the software development process. Agile approaches talk about moving fast, and this requires good communication, but it's surprising that many teams struggle to effectively communicate the design of their software.
Maybe software developers are naturally optimistic but in my experience they rarely consider system failure or disaster scenarios when designing software. Failures are varied and range from the likely (local disk failure) to the rare (tsunami) and from low impact to fatal (where fatal may be the death of people or bankruptcy of a business).
Failure planning broadly fits into the following areas:
Avoiding failure is what a software architect is most likely to think about at design time. This may involve a number of High Availability (HA) techniques and tools including; redundant servers, distributed databases or real time replication of data and state. This usually involves removing any single point of failure but you should be careful to not just consider the software and hardware that it immediately runs on - you should also remove any single dependency on infrastructure such as power (battery backup, generators or multiple power supplies) or telecoms (multiple wired connections, satellite or radio backups etc).
Failing safely is a complex topic that I touched on recently and may not apply to your problem domain (although you should always consider if it does).
Failure recovery usually goes hand-in-hand with High Availability and ensures that when single components are lost they can be re-created/started to join the system. There is no point in having redundancy if components cannot be recovered as you will eventually lose enough components for the system to fail!
Disaster Recovery Scenarios and Planning
However, the main topic I want to discuss here is disaster recovery. This is the process that a system and its operators have to execute in order to recreate a fully operational system after a disaster scenario. This differs from a failure in that the entire system (potentially all the components but at least enough to render it inoperable) stops working. As I stated earlier, many software architects don't consider these scenarios but they can include:
These are usually classified into either natural or man-made disasters. Importantly these are likely to cause outright system failure and require some manual intervention - the system will not automatically recover. Therefore an organisation should have a Disaster Recovery (DR) Plan for the operational staff to follow when this occurs.
A disaster recovery plan should consider a range of scenarios and give very clear and precise instructions on what to do for each of them. In the event of a disaster scenario the staff members are likely to be stressed and not thinking as clearly as they would otherwise. Keep any steps required simple and don't worry about stating the obvious or being patronising - remember that the staff executing the plan may not be the usual maintainers of the system.
Please remember that 'cloud hosted' systems still require disaster recovery plans! Your provider could have issues and you are still affected by scenarios that involve corrupt data and disgruntled staff. Can you roll-back your data store to a known point in the past before corruption?
The aims and actions of any recovery will depend on the scenario that occurs. Therefore the scenarios listed should each refer to a strategy which contains some actions.
Before any strategy is executed you need to be able to detect the event has occurred. This may sound obvious but a common mistake is to have insufficient monitoring in place to actually detect it. Once detected there needs to be comprehensive notification in place so that all systems and people are aware that actions are now required.
For each strategy there has to be an aim for the actions. For example, do you wish to try to bring up a complete system with all data (no data loss) or do you just need something up and running? Perhaps missing data can be imported at a later time or maybe some permanent data-loss is tolerated? Does the recovered system have to provide full functionality or is an emergency subset sufficient?
This is hugely dependent on the problem domain and scenario but the key metrics are recovery point objectives (RPO) and recovery time objectives (RTO) along with level of service. Your RPO and RTO are key non-functional (quality) requirements and should be listed in your software architecture document. These metrics should influence your replication, backup strategies and necessary actions.
The disaster recovery plans for the IT systems are actually a subset of the boarder 'business continuity' plans (BCP) that an organisation should have. This covers all the aspects of keeping an organisation running in the event of a disaster. BCP plans also includes manual processes, staff coverage, building access etc. You need to make sure that the IT disaster recovery plan fits into the business continuity plan and you state the dependencies between them.
There are a range of official standards covering Business Continuity Planning such as ISO22301, ISO22313 and ISO27031. Depending on your business and location you might have a legal obligation to comply with these or other local standards. I would strongly recommend that you investigate whether your organisation needs to be compliant - if you fail to do so then there could be legal consequences.
This is a complex topic which I have only touched upon - if it raises concerns then you may have a lot of work to do! If you don't know where to start then I'd suggest getting your team together and running a risk storming workshop.
2013 has been a fantastic year for me and I've had the pleasure of meeting so many people in more than a dozen countries. I just wanted to take this opportunity to say thank you and to wish all of my friends and clients a Merry Christmas. I hope that you have a happy and healthy 2014.
Instead of sending greeting cards, I'll be making a donation to the Channel Islands Air Search and RNLI Jersey. Both are voluntary organisations made up of people who regularly risk their own lives to save others. Jersey has experienced some very stormy weather during recent weeks where the RNLI and the CI AirSearch have both been called out. The CI AirSearch aeroplane unfortunately had to make an emergency landing too. They're incredibly brave to head out in the scary conditions we've had over the past few weeks and are an essential part of island life. "Inspirational" is an understatement. Let's hope they have a quiet holiday season.
I've been working with a number of teams recently, helping them to diagram their software systems using the C4 approach that is described in my Software Architecture for Developers book. To summarise, it's a way to visualise the static structure of a software system using a small number of simple diagrams as follows:
In the real-world, software systems never live in isolation and it's often useful to understand how all of the various software systems fit together within the bounds of an enterprise. To do this, I'll simply add another diagram that sits on top of the C4 diagrams, to show the enterprise context from an IT perspective. This usually includes:
Essentially this becomes a high-level map of the software systems at the enterprise level, with a C4 drill-down for each software system of interest. Caveat time. I do appreciate that enterprise architecture isn't simply about technology but, in my experience, many organisations don't have an enterprise architecture view of their IT landscape. In fact, it shocks me how often I come across organisations of all sizes that lack such a holistic view, especially considering IT is usually a key part of the way they implement business processes and service customers. Sketching out the enterprise context from a technology perspective at least provides a way to think outside of the typical silos that form around IT systems.
The Software Architect 2013 conference was a fantastic few days of learning and talking about everything software architecture related. Here are links to the video, slides and photos from my sessions.
Why are some teams successful while others are less than stellar? Can teams use processes to do their work? How can managers help teams to become better? And do we need incentives to improve the quality of software? These are some of the topics that Simon Brown talked about at the GOTO Amsterdam conference earlier this year in Sustainable competence - the people vs process and technology.
InfoQ did an interview with Simon Brown, in which he talks about sustainable competence for continuous improvement, balancing people and processes, and software quality and architecture.