Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Testing & QA

TDD Against the Clock

Actively Lazy - Thu, 02/19/2015 - 07:42

A couple of weeks ago I ran a “TDD Against the Clock” session. The format is simple: working in pairs following a strict red-green-refactor TDD cycle we complete a simple kata. However we add one key constraint: the navigator starts a five minute timer. As soon as the five minutes is up:

  • If the code compiles and the tests are green, commit!
  • Otherwise, revert!

Either way, the pairs swap roles when the timer goes off. If the driver completes a red-green-refactor cycle faster than five minutes, they can commit ahead of time but the roles still swap.

The kata we chose for this session was the bowling kata. This is a nice, simple problem that I expected we could get a decent way through in each of the two 45 minute sessions.

Hard Time

The five minute time constraint sounds fiendish doesn’t it? How can you possibly get anything done in five minutes? Well, you can, if you tackle something small enough. This exercise is designed to force you to think in small increments of functionality.

It’s amazing how little you can type in five minutes. But if you think typing speed is the barrier, you’re not thinking hard enough about the right way to tackle the problem. There comes a point in the bowling kata where you go from dealing with single frames and simple scores to spares (or strikes) for the first time. This always requires a jump because what you had before won’t suit what you need now. How to tackle this jump incrementally is part of the challenge when working within a five minute deadline. One of our group had an idea but knew it was tough to get it done in five minutes. He typed like a demon trying to force his solution in: he still ran out of time. Typing speed is not the problem (no matter how much it seems like it is). You need a better approach, you need to think more not type more.

Good Behaviour

After a few cycles, we found hardly anybody hit the 5 minute deadline any more. It’s fascinating how quickly everyone realised that it was better to spend a 5 minute cycle discussing than to get lost half-way through a change and end up reverting. Similarly, when you find the change you wanted to make in this cycle is too hard or too time consuming, it’s better to throw away what you have, swap pairs and refactor before you try and write the failing test again.

These are all good behaviours that are useful in day-to-day life, where it’s all too easy to keep chasing down a rat hole. Learning to work in small, independent increments and making that a subconscious part of how you work will make you a better programmer.

Wrong School

The biggest trouble we found is that the bowling kata isn’t well suited to what I consider “normal”, outside-in TDD (London School TDD). Most of the time I use TDD as a design tool, to help me uncover the right roles and responsibilities. However, with the bowling kata the most elegant solution is the one Uncle Bob drives towards, which is just simple types with no object modelling.

This is fine for an algorithm like scoring a game of bowling, which has an ultimate truth and isn’t likely to change. But in the normal day-to-day world we’re designing for flexibility and constant change. This is where a good object model of the domain makes things easier to reason about and simpler to change. This is typically where outside-in TDD will help you.

A couple of the group were determined to implement an OO version of the bowling kata. It isn’t easy as it doesn’t lend itself naturally to being built incrementally towards a good object model. However, with enough stubbornness it can be done. This led to an interesting discussion of whether you can TDD algorithms and whether TDD is better suited to problems where an object model is the desired outcome.

Obviously you can TDD algorithms incrementally, whether it’s worthwhile I’m not so sure.  Typically you’re implementing an algorithm because there is a set of rules to follow. Implementing each rule one at a time might help keep you focussed, but you always need to be aware of the algorithm as a whole.

Using TDD to drive an OO design is different. There can be many, similarly correct object models that vary only by subtle nuances. TDD can help guide your design and choose between the nuances. While you still need to think of the overall system design, TDD done outside-in is very deliberately trying to limit the things you need to worry about at any given stage: focus on one pair of interactions at a time. This is where TDD is strongest: providing a framework for completing a large task in small, manageable increments.

Even if the problem we chose wasn’t ideal, overall I found the TDD against the clock session a great way to practice the discipline of keeping your commits small, with constant refactoring, working incrementally towards a better design.

How do you move a mountain? Simply move it one teaspoonful at a time.


Categories: Programming, Testing & QA

Diamond Kata - Thoughts on Incremental Development

Mistaeks I Hav Made - Nat Pryce - Thu, 02/19/2015 - 00:35
Some more thoughts on my experience doing the Diamond Kata with property-based tests
 When test-driving development with example-based tests, an essential skill to be learned is how to pick each example to be most helpful in driving development forwards in small steps. You want to avoid picking examples that force you to take too big a step (A.K.A. “now draw the rest of the owl”). Conversely, you don’t want to get sidetracked into a boring morass of degenerate cases and error handling when you’ve not yet addressed the core of the problem to be solved. Property-based tests are similar: the skill is in picking the right next property to let you make useful progress in small steps. But the progress from nothing to solution is different. Doing TDD with example-based tests, I’ll start with an arbitrary, specific example (arbitrary but carefully chosen to help me make useful progress), and write a specific implementation to support just that one example. I’ll add more examples to “triangulate” the property I want to implement, and generalise the implementation to pass the tests. I continue adding examples and triangulating, and generalising the implementation until I have extensive coverage and a general implementation that meets the required properties. Example TDD progress Doing TDD with property-based tests, I’ll start with a very general property, and write a specific but arbitrary implementation that meets the property (arbitrary but carefully chosen to help me make useful progress). I’ll add more specific properties, which force me to generalise the the implementation to make it meet all the properties. The properties also double-check one another (testing the tests, so to speak). I continue adding properties and generalising the implementation until I have a general implementation that meets the required properties. Property TDD progress I find that property-based tests let me work more easily in larger steps than when testing individual examples. I am able to go longer without breaking the problem down to avoid duplication and boilerplate in my tests, because the property-based tests have less scope for duplication and boilerplate. For example, if solving the Diamond Kata with example-based tests, my reaction to the “now draw the rest of the owl” problem that Seb identified would be to move the implementation towards a compositional design so that I could define the overall behaviour declaratively and not need to write integration tests for the whole thing. For example, when I reached the test for “C”, I might break the problem down into two parts. Firstly, a higher-order mirroredLines function that is passed a function to generate each line, with the type (in Haskell syntax): mirroredLines :: (Char -> Char -> String) -> Char -> String I would test-drive mirroredLines with a stub function to generate fake lines, such as: let fakeLine ch maxCh = "line for " ++ [ch] Then, I would write a diamondLine function, that calculates the actual lines of the diamond. And declare the diamond function by currying: let diamond = mirroredLines diamondLine I wouldn’t feel the need to write tests for the diamond function given adequate tests of mirroredLines and diamondLine.
Categories: Programming, Testing & QA

Software Development Linkopedia February 2015

From the Editor of Methods & Tools - Tue, 02/17/2015 - 16:09
Here is our monthly selection of interesting knowledge material on programming, software testing and project management. This month you will find some interesting information and opinions about software and system modeling, programmer psychology, managing priorities, improving software architecture, technical user stories, free tools for Scrum, coding culture and integrating UX in Agile approaches. Web site: Fundamental Modeling Concepts The Fundamental Modeling Concepts (FMC) primarily provide a framework for the comprehensive description of software-intensive systems. Blog: The Ten Commandments of Egoless Programming Blog: WIP and Priorities: Hw to Get Fast and Focused! Blog: Sacrificial Architecture Blog: ...

Diamond Kata - Some Thoughts on Tests as Documentation

Mistaeks I Hav Made - Nat Pryce - Sun, 02/15/2015 - 13:13
Comparing example-based tests and property-based tests for the Diamond Kata, I’m struck by how well property-based tests reduce duplication of test code. For example, in the solutions by Sandro Mancuso and George Dinwiddie, not only do multiple tests exercise the same property with different examples but the tests duplicate assertions. Property-based tests avoid the former by defining generators of input data, but I’m not sure why the latter occurs. Perhaps Seb’s “test recycling” approach would avoid this kind of duplication. But compared to example based tests, property based tests do not work so well as as an explanatory overview. Examples convey an overall impression of what the functionality is, but are are not good at describing precise details. When reading example-based tests, you have to infer the properties of the code from multiple examples and informal text in identifiers and comments. The property-based tests I wrote for the Diamond Kata specify precise properties of the diamond function, but nowhere is there a test that describes that the function draws a diamond! There’s a place for both examples and properties. It’s not an either/or decision. However, explanatory examples used for documentation need not be test inputs. If we’re generating inputs for property tests and generating documentation for our software, we can combine the two, and insert generated inputs and calculated ouputs into generated documentation.
Categories: Programming, Testing & QA

Quote of the Month February 2015

From the Editor of Methods & Tools - Tue, 02/10/2015 - 13:47
The worrying thing about writing tests is that there are numerous accounts of people introducing tests into their development process and.. ending up with even more junk code to support than they had to begin with! Why? I guess, mainly because they lacked the knowledge and understanding needed to write good tests: tests, that is, that will be an asset, not a burden. Reference: Bad Tests, Good Tests, Tomek Kaczanowski,

Pairing Patterns

Actively Lazy - Wed, 02/04/2015 - 22:01

Pair programming is hard. When most developers start pairing it feels unnatural. After a lifetime of coding alone, headphones on, no human contact; suddenly talking about every damned line of code can seem weird. Counter-productive, even.

And yet… effective pairing is the cheapest way to improve code quality. Despite what superficially seems like a halving in productivity – after all, your team of eight developers are only working on four things now instead of eight! – it turns out that productivity doesn’t drop at all. If anything, I’ve seen the opposite.

Going it Alone

In my experience most developers are used to, and feel most comfortable, coding on their own. It seems the most natural way to write code. But it introduces all sorts of problems.

If you’re the only person that wrote this code there’s only one person that knows it, that means at 3am in 6 months time guess who’s getting the phone call? And what happens when you decide to leave? No, worse, what happens when that other guy decides to leave and now you’ve got a metric fuckton of code to support. And of course, he couldn’t code for shit. His code stinks. His design stinks. You question his ability, his morals, even his parentage. Because everybody codes to a different style it’s hard to maintain any consistency. This varies from the most trivial of stylistic complaints (braces on new lines, puhleeze, what a loser) to consistency of architectural approach and standardised tools and libraries. This makes picking up other people’s code hard.

When you’re coding on your own, it’s harder to be disciplined: I don’t need to write a unit test for this class, it’s pretty trivial. I don’t need to refactor this mess, I know how it works. With nobody looking over your shoulder it takes a lot more self-discipline to write the high quality code you know you ought to.

Getting Started Pairing

The easiest way to get started is to pair with someone that’s experienced at doing it. It can feel quite strange and quickly become dysfunctional if you’re not used to it, so having an experienced hand show you what effective pairing feels like is really important.

The most important thing to realise is that pairing is incredibly social. You will spend a massive amount of time talking. It turns out that days of coding can save literally minutes of thought up front. When you’re pairing, this thinking happens out loud as you argue about the best way to approach the design, the best way to test this class, the best way to refactor it.

This can feel alien at first and incredibly wasteful. Why don’t you just shut up and let me code? Because then we’ll just have to delete your crap code and you’ll feel bad. Or worse, we’ll push it so you don’t feel bad and then we’ll come back to this mess again and again over the coming months and pay an incredibly high price instead of spending another few minutes discussing it now until we agree.

The Roles

When pairing we traditionally label the two roles “driver” and “navigator”. The driver is the person with their hands on the keyboard, typing. The navigator isn’t. So what the hell’s the navigator doing? The critical thing is that they’re not just sitting there watching. The driver is busy writing good code that compiles; the driver is focused on details. The navigator is looking at the bigger picture: making sure that what we’re doing is consistent with the overall design.

One thing I really struggle with, but as a navigator it’s really important: don’t interrupt the driver’s flow. Resist the temptation to tell the driver there’s a missing bracket or semi-colon. Resist the urge to tell them what order to fix the compile errors in. Keep track of what needs to be done, if the driver misses something small write it down and come back to it.

The navigator should be taking copious notes, letting the driver stay hands-on-keyboard typing. If there’s a test we’ve spotted we’re missing, write it down. If there’s an obvious design smell we need to come back to, write it down. If there’s a refactoring we should do next, write it down. The navigator uses these notes to guide the coding session – ensuring details aren’t missed and that we keep heading in the right direction and come back to every detail we’ve spotted along the way.

The navigator can also keep track of the development “call stack”. You know how it goes: we started writing the shopping basket returns a price in euros test; but to do that we need to change the basket item get price method; this breaks a couple of basket item unit tests, the first of these shows we don’t have a currency conversion available for a basket item; so now we’re changing how currency conversion is constructed so we can pass it into the basket item factory. This call stack of development activities can get very deep if you’re not careful, but a disciplined navigator with a clear navigator’s pad will guide the way.

Changing Roles

Generally the person that knows the domain / code base / problem the best should spend the least time being the driver. If I don’t know this code and you’re driving, I’m just gonna sit here watching you type. I can’t really contribute any design ideas because you know the domain. I can’t ask questions because it stops you typing. But the other way round: I can be busy typing learning the code as I go; while you use your superior knowledge to guide me in the right direction. I can ask lots of questions because when I don’t know, work stops until I’m happy again.

A good approach can be ping-pong pairing: this is where one person writes a failing test, the other makes it pass then writes another failing test, back to the first to make this test pass and then write another failing test, and so on and so on… This can give a good balance to a pairing session as both developers write test and production code and gives a natural rhythm preventing any one developer from dominating the driver role.

Sometimes it’s necessary to impose a time limit, I find 25 minutes is long enough for one person to be driving. This can happen when someone has an idea about a refactoring, especially if it becomes a sprawling change. 25 minutes also puts a good upper limit on a change, if you’ve not been able to commit to source control in 25 minutes it is definitely time to abort and do-over.

At the end of the day, write up your navigator pad and email it your partner. The following day you can swap pairs allowing either of you to carry on from exactly where you left off today.


Pairing can feel strange at first, but with practice it will begin to feel normal. If you can keep pairing day-in day-out you will come to rely on having a second brain alongside you. You’ll realise you can get through complex work faster because you’ve got two people working at different detail levels. Keep pairing long enough and coding on your own will begin to feel strange, almost dangerous. Who’s watching my back?

Categories: Programming, Testing & QA

The First Annual Testing on the Toilet Awards

Google Testing Blog - Tue, 02/03/2015 - 17:51
By Andrew Trenk

The Testing on the Toilet (TotT) series was created in 2006 as a way to spread unit-testing knowledge across Google by posting flyers in bathroom stalls. It quickly became a part of Google culture and is still going strong today, with new episodes published every week and read in hundreds of bathrooms by thousands of engineers in Google offices across the world. Initially focused on content related to testing, TotT now covers a variety of technical topics, such as tips on writing cleaner code and ways to prevent security bugs.

While TotT episodes often have a big impact on many engineers across Google, until now we never did anything to formally thank authors for their contributions. To fix that, we decided to honor the most popular TotT episodes of 2014 by establishing the Testing on the Toilet Awards. The winners were chosen through a vote that was open to all Google engineers. The Google Testing Blog is proud to present the winners that were posted on this blog (there were two additional winners that weren’t posted on this blog since we only post testing-related TotT episodes).

And the winners are ...

Erik Kuefler: Test Behaviors, Not Methods and Don't Put Logic in Tests 
Alex Eagle: Change-Detector Tests Considered Harmful

The authors of these episodes received their very own Flushy trophy, which they can proudly display on their desks.

(The logo on the trophy is the same one we put on the printed version of each TotT episode, which you can see by looking for the “printer-friendly version” link in the TotT blog posts).

Congratulations to the winners!

Categories: Testing & QA

Setting up an OpenVPN server inside an AWS VPC

Agile Testing - Grig Gheorghiu - Sat, 01/31/2015 - 23:56
My goal in this post is to show how to set up an OpenVPN server within an AWS VPC so that clients can connect to EC2 instances via a VPN connection. It's fairly well documented how to configure a site-to-site VPN with an AWS VPC gateway, but articles talking about client-based VPN connections into a VPC are harder to find.

== OpenVPN server setup ==

Let's start with setting up the OpenVPN server. I launched a new Ubuntu 14.04 instance in our VPC and I downloaded the latest openvpn source code via:


In order for the 'configure' step to succeed, I also had to install the following Ubuntu packages:

apt-get install build-essential openssl libssl-dev lzop liblzo2-dev libpam-dev

I then ran the usual commands inside the openvpn-2.3.6 directory:

./configure; make; sudo make install

At this point I proceeded to set up my own Certificate Authority (CA), per the OpenVPN HOWTO guide.  As it turned out, I needed the easy-rsa helper scripts on the server running openvpn. I got them from github:

git clone

To generate the master CA certificate & key, I did the following:

cd ~/easy-rsa/easyrsa3
cp vars.example vars

- edited vars file and set these variables with the proper values for my organization:

./easyrsa init-pki
(this will create the correct directory structure)
./easyrsa build-ca
(this will use the info specified in the vars file above)

To generate the OpenVPN server certificate and key, I ran:

./easyrsa build-server-full server
(I was prompted for a password for the server key)

To generate an OpenVPN client certificate and key for user myuser, I ran:

./easyrsa  build-client-full myuser
(I was prompted for a password for the client key)

The next step was to generate the Diffie Hellman (DH) parameters for the server by running:

./easyrsa gen-dh

I was ready at this point to configure the OpenVPN server.

I created a directory called /etc/openvpn and copied the pki directory under ~/easy-rsa/easyrsa3 to /etc/openvpn. I also copied the sample server configuration file ~/openvpn-2.3.6/sample/sample-config-files/server.conf to /etc/openvpn.

I edited /etc/openvpn/server.conf and specified the following:

ca /etc/openvpn/pki/ca.crt
cert /etc/openvpn/pki/issued/server.crt
key /etc/openvpn/pki/private/server.key  # This file should be kept secret
dh /etc/openvpn/pki/dh.pem

ifconfig-pool-persist /etc/openvpn/ipp.txt

push "route"

The first block specifies the location of the CA certificate, the server key and certificate, and the DH certificate.

The 'server' parameter specifies a new subnet from which both the OpenVPN server and the OpenVPN clients connecting to the server will get their IP addresses. I set it to The client IP allocations will be saved in the ipp.txt file, as specified in the ifconfig-pool-persist parameter.

One of the most important options, which I missed when I initially configured the server, is the 'push route' one. This makes the specified subnet (i.e. the instances in the VPC that you want to get to via the OpenVPN server) available to the clients connecting to the OpenVPN server without the need to create static routes on the clients. In my case, all the EC2 instances in the VPC are on the subnet, so that's what I specified above.

Two more very important steps are needed on the OpenVPN server. It took me quite a while to find them so I hope you will be spared the pain.

The first step was to turn on IP forwarding on the server:

- uncomment the following line in /etc/sysctl.conf:

- run
sysctl -p

The final step in the configuration of the OpenVPN server was to make it do NAT via itpables masquerading (thanks to rbgeek's blog post for these last two critical steps):

- run
iptables -t nat -A POSTROUTING -s -o eth0 -j MASQUERADE

- also add the above line to /etc/rc.local so it gets run on reboot

Now all that's needed on the server is to actually run openvpn. You can run it in the foreground for troubleshooting purposes via:

openvpn /etc/openvpn/server.conf

Once everything works, run it in daemon mode via:

openvpn --daemon --config /etc/openvpn/server.conf

You will be prompted for the server key password when you start up openvpn. Haven't looked yet on how to run the server in a fully automated way.

Almost forgot to specify that you need to allow incoming traffic to UDP port 1194 in the AWS security group where your OpenVPN server belongs. Also allow traffic from that security group to the security groups of the EC2 instances that you actually want to reach over the OpenVPN tunnel.

== OpenVPN client setup ==

This is on a Mac OSX Mavericks client, but I'm sure it's similar for other clients.

Install tuntap
- download tuntap_20150118.tar.gz from
- untar and install tuntap_20150118.pkg

Install lzo
tar xvfz lzo-2.06.tar.gz
cd lzo-2.06
./configure; make; sudo make install

Install openvpn
- download openvpn-2.3.6.tar.gz from xvf openvpn-2.3.6.tar
cd openvpn-2.3.6
./configure; make; sudo make install

At this point ‘openvpn --help’ should work.
The next step for the client setup is to copy the CA certificate ca.crt, and the client key and certificate (myuser.key and myuser.crt) from the OpenVPN server to the local client. I created an openvpn directory under my home directory on my Mac and dropped ca.crt in ~/openvpn/pki, myuser.key in ~/openvpn/pki/private and myuser.crt in ~/openvpn/pki/issued. I also copied the sample file ~/openvpn-2.3.6/sample/sample-config-files/client.conf to ~/openvpn and specified the following parameters in that file:
ca /Users/myuseropenvpn/pki/ca.crtcert /Users/myuser/openvpn/pki/issued/myuser.crtkey /Users/myuser/openvpn/pki/private/myuser.key
Then I started up the OpenVPN client via:
sudo openvpn ~/openvpn/client.conf(at this point I was prompted for the password for myuser.key)
To verify that the OpenVPN tunnel is up and running, I ping-ed the internal IP address of the OpenVPN server (in my case it was on the internal subnet I specified in server.conf), as well as the internal IPs of various EC2 instances behind the OpenVPN server. Finally, I ssh-ed into those internal IP addresses and declared victory.
That's about it. Hope this helps!

UPDATE: I discovered in the mean time a very good Mac OSX GUI tool for managing client OpenVPN connections: Tunnelblick. All it took was importing the client.conf file mentioned above.

Testing on the Toilet: Change-Detector Tests Considered Harmful

Google Testing Blog - Wed, 01/28/2015 - 01:43
by Alex Eagle

This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.

You have just finished refactoring some code without modifying its behavior. Then you run the tests before committing and
 a bunch of unit tests are failing. While fixing the tests, you get a sense that you are wasting time by mechanically applying the same transformation to many tests. Maybe you introduced a parameter in a method, and now must update 100 callers of that method in tests to pass an empty string.

What does it look like to write tests mechanically? Here is an absurd but obvious way:
// Production code:
def abs(i: Int)
return (i < 0) ? i * -1 : i

// Test code:
for (line: String in File(prod_source).read_lines())
switch (line.number)
1: assert line.content equals def abs(i: Int)
2: assert line.content equals return (i < 0) ? i * -1 : i

That test is clearly not useful: it contains an exact copy of the code under test and acts like a checksum. A correct or incorrect program is equally likely to pass a test that is a derivative of the code under test. No one is really writing tests like that, but how different is it from this next example?
// Production code:
def process(w: Work)

// Test code:
part1 = mock(FirstPart)
part2 = mock(SecondPart)
w = Work()
Processor(part1, part2).process(w)
was_called part1.process(w)
was_called part2.process(w)

It is tempting to write a test like this because it requires little thought and will run quickly. This is a change-detector test—it is a transformation of the same information in the code under test—and it breaks in response to any change to the production code, without verifying correct behavior of either the original or modified production code.

Change detectors provide negative value, since the tests do not catch any defects, and the added maintenance cost slows down development. These tests should be re-written or deleted.

Categories: Testing & QA

Software Development Conferences Forecast January 2015

From the Editor of Methods & Tools - Tue, 01/27/2015 - 14:37
Here is a list of software development related conferences and events on Agile ( Scrum, Lean, Kanban), software testing and software quality, software architecture, programming (Java, .NET, JavaScript, Ruby, Python, PHP) and databases (NoSQL, MySQL, etc.) that will take place in the coming weeks and that have media partnerships with the Methods & Tools software development magazine. SPTechCon, February 8-11 2015, Austin, USA Use code SHAREPOINT for a $200 conference discount off the 3 and 4-day pass NorDevCon, February 27 2015, Norwich, UK Early birds tickets until February 13. QCon London, March 2-6 2015, London, ...

Quote of the Month January 2015

From the Editor of Methods & Tools - Mon, 01/26/2015 - 14:50
Principles Trump Diagrams. Most of the problems in using the 1988 spiral model stemmed from users looking at the diagram and constructing processes that had nothing to do with the underlying concepts. This is true of many process models: people just adopt the diagram and neglect the principles that need to be respected. Source: The Incremental Commitment Spiral Model, Barry Boehm,, Jo Ann Lane, Supannika Koolmanojwong & Richard Turner, Addison-Wesley

Software Development Linkopedia January 2015

From the Editor of Methods & Tools - Tue, 01/20/2015 - 14:58
Here is our monthly selection of interesting knowledge material on programming, software testing and project management. This month you will find some interesting information and opinions about managing software developers, software architecture, Agile testing, product owner patterns, mobile testing, continuous improvement, project planning and technical debt. Blog: Killing the Crunch Mode Antipattern Blog: Barry’s Rules of Engineering and Architecture Blog: Lessons Learned Moving From Engineering Into Management Blog: A ScrumMaster experience report on using Feature Teams Article: Using Models to Help Plan Tests in Agile Projects Article: A Mitigation Plan for a Product Owner’s Anti-Pattern Article: Guerrilla Project ...

Software Architecture Articles of 2014

From the Editor of Methods & Tools - Thu, 01/15/2015 - 15:07
When software features are distributed on multiple infrastructures (server, mobile, cloud) that needs to communicate and synchronize, having a sound and reactive software architecture is a key for success and evolution of business functions. Here are seven software architecture articles published in 2014 that can help you understand the basic topics and the current trends in software architecture: Agile, Cloud, SOA, Security
 and even a little bit of data modeling. * Designing Software in a Distributed World This is an overview of what is involved in designing services that use distributed computing ...

Testing on the Toilet: Prefer Testing Public APIs Over Implementation-Detail Classes

Google Testing Blog - Wed, 01/14/2015 - 18:35
by Andrew Trenk

This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.

Does this class need to have tests?
class UserInfoValidator {
public void validate(UserInfo info) {
if (info.getDateOfBirth().isInFuture()) { throw new ValidationException()); }
Its method has some logic, so it may be good idea to test it. But what if its only user looks like this?
public class UserInfoService {
private UserInfoValidator validator;
public void save(UserInfo info) {
validator.validate(info); // Throw an exception if the value is invalid.
The answer is: it probably doesn’t need tests, since all paths can be tested through UserInfoService. The key distinction is that the class is an implementation detail, not a public API.

A public API can be called by any number of users, who can pass in any possible combination of inputs to its methods. You want to make sure these are well-tested, which ensures users won’t see issues when they use the API. Examples of public APIs include classes that are used in a different part of a codebase (e.g., a server-side class that’s used by the client-side) and common utility classes that are used throughout a codebase.

An implementation-detail class exists only to support public APIs and is called by a very limited number of users (often only one). These classes can sometimes be tested indirectly by testing the public APIs that use them.

Testing implementation-detail classes is still useful in many cases, such as if the class is complex or if the tests would be difficult to write for the public API class. When you do test them, they often don’t need to be tested in as much depth as a public API, since some inputs may never be passed into their methods (in the above code sample, if UserInfoService ensured that UserInfo were never null, then it wouldn’t be useful to test what happens when null is passed as an argument to UserInfoValidator.validate, since it would never happen).

Implementation-detail classes can sometimes be thought of as private methods that happen to be in a separate class, since you typically don’t want to test private methods directly either. You should also try to restrict the visibility of implementation-detail classes, such as by making them package-private in Java.

Testing implementation-detail classes too often leads to a couple problems:

- Code is harder to maintain since you need to update tests more often, such as when changing a method signature of an implementation-detail class or even when doing a refactoring. If testing is done only through public APIs, these changes wouldn’t affect the tests at all.

- If you test a behavior only through an implementation-detail class, you may get false confidence in your code, since the same code path may not work properly when exercised through the public API. You also have to be more careful when refactoring, since it can be harder to ensure that all the behavior of the public API will be preserved if not all paths are tested through the public API.
Categories: Testing & QA

MVVM and Threading

Actively Lazy - Sun, 01/11/2015 - 21:01

The Model-View-ViewModel pattern is a very powerful design pattern when building WPF applications, even if I’m not sure everyone interprets it the same way. However, it’s never been clear to me how to easily manage multi-threaded WPF applications: writing multi-threaded code is hard and there seems to be no real support baked into WPF or the idea of MVVM to make multi-threaded code easier to get right.

The Problem

All WPF applications are effectively bound by the same three constraints:

  1. All interaction with UI elements must be done on the UI thread
  2. All long-running tasks (web service calls etc) should be done on a background thread to keep the UI responsive
  3. Repeatedly switching between threads is expensive

Bound by these constraints it means that all WPF code has two types of thread running through it: UI threads and background threads. It is clearly important to know which type of thread will be executing any given line of code: a background thread cannot interact with UI elements and UI threads should not make expensive calls.


A very brief, contrived example might help. The source code for this is available on github.

Here is a viewmodel for a very simple view:

class ViewModel
  public ObservableCollection<string> Items { get; private set; }
  public ICommand FetchCommand { get; private set; }

  public async void Fetch()
    var items = await m_model.Fetch();
    foreach (var item in items)

The ViewModel exposes a command, which calls ViewModel.Fetch() to retrieve some data from the model; once retrieved this data is added to the list of displayed items. Since Fetch is called by an ICommand and interacts with an ObservableCollection (bound to the view) it clearly runs on the UI thread.

Our model is then responsible for fetching the data requested by the viewmodel:

class Model
  public async Task<IList<string>> Fetch()
    return await Task.Run(() => DoFetch());

  private IList<string> DoFetch()
    return new[] { "Hello" };

In a real application DoFetch() would obviously call a database or web service or whatever is required; it is also probable that the Fetch() method might be more complex, e.g. coordinating multiple service calls or managing caching or other application logic.

There is a difference in threading in the model compared to the viewmodel: the Fetch method will, in our example, be called on the UI thread whereas DoFetch will be called on a background thread. Here we have a class through which may run two separate types of thread.

In this very simple example which thread type calls each method is obvious. But scale this up to a real application with dozens of classes and many methods and it becomes less obvious. It suddenly becomes very easy to add a long-running service call to a method which runs on the UI thread; or a switch to a background thread from a method that already runs on a background thread. These errors can be difficult to spot: neither causes an obvious failure. The first will only show if the service call is obviously slow, the observed behaviour may simply be a UI that intermittently pauses for no apparent reason. The second simply slows tasks down, the application will seem slower than it ought to be with no obvious indication of why.

It seems as though WPF and MVVM have carefully guided us into a multi-threaded minefield.

First Approach

The first idea is to simply apply a naming convention, each method is suffixed with _UI or _Worker to indicate which type of thread invoked it. E.g. our model class becomes:

class Model
  public async Task<IList<string>> Fetch_UI()
    return await Task.Run(() => Fetch_Worker());

  private IList<string> Fetch_Worker()
    return new[] { "Hello" };

This at least makes it obvious to my future self which type of thread I think executes each method. Simple code inspection shows that a _UI method calling someWebService.Invoke(…) is a Bad Thing. Similarly, Task.Run(…) in a _Worker method is obviously suspect. However, it looks ugly and isn’t guaranteed to be correct – it is just a naming convention, nothing stops me calling a _UI method from a background thread, or vice versa.

Introducing PostSharp

If you haven’t tried PostSharp, the C# AOP library, it is definitely worth a look. Even the free version is quite powerful and allows us to evolve the first idea into something more powerful.

PostSharp allows you to create an attribute which will introduce “advice” (i.e. code to run) around any method annotated with the attribute. For example, we can annotate the ViewModel constructor with a new UIThreadPolicy attribute:

  public ViewModel()
    Items = new ObservableCollection<string>();
    FetchCommand = new FetchCommand(this);

This attribute is logically similar to using a suffix on the method name, in that it documents our intention. However, by using AOP it also allows us to introduce code to run before the method executes:

class UIThreadPolicy : OnMethodBoundaryAspect
  public override void OnEntry(MethodExecutionArgs args)
    if (Thread.CurrentThread.IsBackground)
      Console.WriteLine("*** Thread policy warning: \n" + Environment.StackTrace);

The OnEntry method will be triggered before the annotated method is invoked. If the method is ever invoked on a background thread, we’ll output a warning to the console. In this rudimentary way, we not only document our intention that the ViewModel should only be created on a UI thread, we also enforce it at runtime to ensure that it remains correct.

We can define another attribute, WorkerThreadPolicy, to enforce the reverse: that a method is only invoked on a background thread. With discipline one of these two attributes can be applied to every method. This makes it trivial when making changes in months to come to know whether a given method runs on the UI thread or a background thread.


Understanding situations where multiple threads access code is hard, so wouldn’t it be great if we could easily identify situations where it’s safe to ignore it?

By using the thread attributes, we can identify situations where code is only accessed by the UI thread. In this case, we have no concurrency to deal with. There is exactly one UI thread so we can ignore any concurrency concerns. For example, our simple Fetch() method above can add a property to keep track of whether we’re already busy:

  public async void Fetch()
    IsFetching = true;
      var items = await m_model.Fetch();
      foreach (var item in items)
      IsFetching = false;

So long as all access to the IsFetching property is on the UI thread, there is no need to worry about locking. We can enforce this by adding attributes to the property itself, too:

  private bool m_isFetching;

  internal bool IsFetching
    get { return m_isFetching; }

      m_isFetching = value;
      IsFetchingChanged(this, new EventArgs());

Here we use a simple, unlocked bool – knowing that the only access is from a single thread. Without these attributes, it is possible that someone in the future writes to IsFetching from a background thread. It will generally work – but access from the UI thread could continue to see a stale value for a short period.


In general we have aimed for a pattern where ViewModels are only accessed on the UI thread. Model classes, however, tend to have a mixture. And since the “model” in any non-trivial application actually includes dozens of classes this mixture of threads permeates the code. By using these attributes it is possible to understand which threads are used where without exhaustively searching up call stacks.

Writing multi-threaded code is hard: but, with a bit of PostSharp magic, knowing which type of thread will execute any given line of code at least makes it a little bit easier.

Categories: Programming, Testing & QA

Decoupling JUnit and Hamcrest

Mistaeks I Hav Made - Nat Pryce - Tue, 01/06/2015 - 23:34
Evolution of Hamcrest and JUnit has been complicated by the dependencies between the two projects. If they can be decoupled, both projects can move forward with less coordination required between the development teams. To that end, we've come up with a plan to allow JUnit to break its dependency on Hamcrest. We've created a new library, hamcrest-junit , that contains a copy of the JUnit code that uses Hamcrest. The classes have been repackaged into org.hamcrest.junit so that the library can be added to existing projects without breaking any code and projects can migrate to the new library at their own pace. The Assert.assertThat function has been copied into the class org.hamcrest.junit.MatcherAssert and Assume.assumeThat to org.hamcrest.junit.MatcherAssume. The ExpectedException and ErrorCollector classes have been copied from package org.junit.rules to package org.hamcrest.junit. The classes in hamcrest-junit only pass strings to JUnit. For example, the assumeThat function does not use the constructor of the AssumptionFailedException that takes a Matcher but generates the description of the matcher as a String and uses that to create the exception instead. This will allow JUnit to deprecate and eventually remove any code that depends on types defined by Hamcrest, finally decoupling the two projects. Version of the hamcrest-junit library has been published to Maven Central (groupId: org.hamcrest, artifactId: hamcrest-junit, version: The source is on GitHub at Feedback about the library, about how easy it is to transition code from using JUnit's classes to using hamcrest-junit and suggestions for improvement are most welcome. Discussion can happen on the hamcrest mailing list or the project's issue tracker on GitHub.
Categories: Programming, Testing & QA

10 technologies that impressed me in 2014

Agile Testing - Grig Gheorghiu - Wed, 12/31/2014 - 07:26
Some of these have been new to me, some are old friends that I came to appreicate more. In alphabetical order:

  1. Ansible
  2. Bitcoin
  3. Consul
  4. Docker
  5. Golang
  6. HAProxy
  7. nginx
  8. OpenStack
  9. Sysdig
  10. Varnish

Agile Analysis, Self-Selecting Teams, TDD & BDD in Methods & Tools Winter 2014 issue

From the Editor of Methods & Tools - Mon, 12/22/2014 - 15:22
Methods & Tools – the free e-magazine for software developers, testers and project managers – has just published its Winter 2014 issue that discusses Agile Analysis, Self-Selecting Teams,Collaborative Development of Domain-specific Languages, TDD with Mock Objects, BDDfire. Methods & Tools Winter 2014 contains the following articles: * Analysis on Analysts in Agile * Self-Selecting Teams – Why You Should Try Self-selection * Collaborative Development of Domain-specific Languages, Models and Generators * TDD with Mock Objects: Design Principles and Emergent Properties * BDDfire: Instant Ruby-Cucumber Framework 55 pages of software development knowledge that you can freely download from ...

Testing on the Toilet: Truth: a fluent assertion framework

Google Testing Blog - Fri, 12/19/2014 - 19:28
by Dori Reuveni and Kurt Alfred Kluever

This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.

As engineers, we spend most of our time reading existing code, rather than writing new code. Therefore, we must make sure we always write clean, readable code. The same goes for our tests; we need a way to clearly express our test assertions.

Truth is an open source, fluent testing framework for Java designed to make your test assertions and failure messages more readable. The fluent API makes reading (and writing) test assertions much more natural, prose-like, and discoverable in your IDE via autocomplete. For example, compare how the following assertion reads with JUnit vs. Truth:
assertEquals("March", monthMap.get(3));          // JUnit
assertThat(monthMap).containsEntry(3, "March"); // Truth
Both statements are asserting the same thing, but the assertion written with Truth can be easily read from left to right, while the JUnit example requires "mental backtracking".

Another benefit of Truth over JUnit is the addition of useful default failure messages. For example:
ImmutableSet<String> colors = ImmutableSet.of("red", "green", "blue", "yellow");
assertTrue(colors.contains("orange")); // JUnit
assertThat(colors).contains("orange"); // Truth
In this example, both assertions will fail, but JUnit will not provide a useful failure message. However, Truth will provide a clear and concise failure message:

AssertionError: <[red, green, blue, yellow]> should have contained <orange>

Truth already supports specialized assertions for most of the common JDK types (Objects, primitives, arrays, Strings, Classes, Comparables, Iterables, Collections, Lists, Sets, Maps, etc.), as well as some Guava types (Optionals). Additional support for other popular types is planned as well (Throwables, Iterators, Multimaps, UnsignedIntegers, UnsignedLongs, etc.).

Truth is also user-extensible: you can easily write a Truth subject to make fluent assertions about your own custom types. By creating your own custom subject, both your assertion API and your failure messages can be domain-specific.

Truth's goal is not to replace JUnit assertions, but to improve the readability of complex assertions and their failure messages. JUnit assertions and Truth assertions can (and often do) live side by side in tests.

To get started with Truth, check out

Categories: Testing & QA

Dynamic DNS updates with nsupdate (new and improved!)

Agile Testing - Grig Gheorghiu - Wed, 12/17/2014 - 23:14
I blogged about this topic before. This post shows a slightly different way of using nsupdate remotely against a DNS server running BIND 9 in order to programatically update DNS records. The scenario I am describing here involves an Ubuntu 12.04 DNS server running BIND 9 and an Ubuntu 12.04 client running nsupdate against the DNS server.

1) Run ddns-confgen and specify /dev/urandom as the source of randomness and the name of the zone file you want to dynamically update via nsupdate:

$ ddns-confgen -r /dev/urandom -z

# To activate this key, place the following in named.conf, and
# in a separate keyfile on the system or systems from which nsupdate
# will be run:
key "" {
algorithm hmac-sha256;
secret "1D1niZqRvT8pNDgyrJcuCiykOQCHUL33k8ZYzmQYe/0=";

# Then, in the "zone" definition statement for "",
# place an "update-policy" statement like this one, adjusted as
# needed for your preferred permissions:
update-policy {
 grant zonesub ANY;

# After the keyfile has been placed, the following command will
# execute nsupdate using this key:
nsupdate -k <keyfile>

2) Follow the instructions in the output of ddns-keygen (above). I actually named the key just ddns-key, since I was going to use it for all the zones on my DNS server. So I added this stanza to /etc/bind/named.conf on the DNS server:

key "ddns-key" {
algorithm hmac-sha256;
secret "1D1niZqRvT8pNDgyrJcuCiykOQCHUL33k8ZYzmQYe/0=";

3) Allow updates when the key ddns-key is used. In my case, I added the allow-update line below to all zones that I wanted to dynamically update, not only to

zone "" {
        type master;
        file "/etc/bind/zones/";
allow-update { key "ddns-key"; };

At this point I also restarted the bind9 service on my DNS server.

4) On the client box, create a text file containing nsupdate commands to be sent to the DNS server. In the example below, I want to dynamically add both an A record and a reverse DNS PTR record:

$ cat update_dns1.txt
debug yes
update add 3600 A
update add 3600 PTR

Still on the client box, create a file containing the stanza with the DDNS key generated in step 1:

$ cat ddns-key.txt
key "ddns-key" {
algorithm hmac-sha256;
secret "Wxp1uJv3SHT+R9rx96o6342KKNnjW8hjJTyxK2HYufg=";

5) Run nsupdate and feed it both the update_dns1.txt file containing the commands, and the ddns-key.txt file:

$ nsupdate -k ddns-key.txt -v update_dns1.txt

You should see some fairly verbose output, since the command file specifies 'debug yes'. At the same time, tail /var/log/syslog on the DNS server and make sure there are no errors.

In my case, there were some hurdles I had to overcome on the DNS server. The first one was that apparmor was installed and it wasn't allowing the creation of the journal files used to keep track of DDNS records. I saw lines like these in /var/log/syslog:

Dec 16 11:22:59 dns1 kernel: [49671335.189689] type=1400 audit(1418757779.712:12): apparmor="DENIED" operation="mknod" parent=1 profile="/usr/sbin/named" name="/etc/bind/zones/" pid=31154 comm="named" requested_mask="c" denied_mask="c" fsuid=107 ouid=107
Dec 16 11:22:59 dns1 kernel: [49671335.306304] type=1400 audit(1418757779.828:13): apparmor="DENIED" operation="mknod" parent=1 profile="/usr/sbin/named" name="/etc/bind/zones/" pid=31153 comm="named" requested_mask="c" denied_mask="c" fsuid=107 ouid=107

To get past this issue, I disabled apparmor for named:

# ln -s /etc/apparmor.d/usr.sbin.named /etc/apparmor.d/disable/
# service apparmor restart

The next issue was an OS permission denied (nothing to do with apparmor) when trying to create the journal files in /etc/bind/zones:

Dec 16 11:30:54 dns1 named[32640]: /etc/bind/zones/ create: permission denied
Dec 16 11:30:54 dns named[32640]: /etc/bind/zones/ create: permission denied

I got past this issue by running

# chown -R bind:bind /etc/bind/zones

At this point everything worked as expected.