Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Xebia Blog
Syndicate content
Updated: 6 hours 1 min ago

CITCON Europe 2014 wrap-up

Mon, 11/24/2014 - 19:33

On the 19th and 20th of September CITCON (pronounced "kit-con") took place in Zagreb, Croatia. CITCON is dedicated to continuous integration and testing. It brings together some of the most interesting people of the European testing and continuous integration community. These people also determine the topics of the conference.

They can do this because CITCON is an Open Space conference. If you're not familiar with the concept of Open Space, check out Wikipedia. On Friday evening, attendees can pitch their proposals. Through dot voting and (constant) shuffling of the schedule, the attendees create their conference program.

In this post we'll wrap up a few topics that were discussed.

Polytesting

Peter Zsoldos (@zsepi) went into his most recent brain-spin: polytesting. If I have a set of requirements, is it feasible to apply those requirements at different levels of my application; say, component, integration and UI level. This sounds very appealing because you can perform ATDD at different levels.
This approach is particularly interesting because it has the potential to keep you focused on the required functionality all the way. You'll need good, concrete requirements for this to work.

Microservices

Microservices are a hot topic nowadays. The promise of small, isolated units with clear interfaces is tempting. There are generally two types of architectures that can be applied. The most common one is where there is no central entity, and services communicate to each other directly.

Douglas Squirrel (@douglasquirrel) explained an alternative architecture by using a central pub-sub "database" to which each service is allowed to publish "facts". Douglas deliberately used the term facts to describe single items that are considered true at a specific point in time ("events"" is too generic a term).

The second model comes closer to mechanisms such as event sourcing (or even ESBs if you take it to the extreme). One of the advantages of this approach is that, because facts are stored, it's possible to construct new functionality based on existing facts. For example, you could use this functionality in a game to create leader boards and, at a later stage, create leaderboards per continent, country, or whatever seems fit.

Unit testing

Arjan Molenaar introduced a flaming hot topic this year: "unit testing is a waste". Inspired by recent discussions of DHH, Martin Fowler, and Kent Beck, Arjan tried to find out the opinions of the CITCON crowd. Most of the people contributing to the discussion must have been working in consultancy, because the main conclusion was "It depends".

Whether unit testing is worth the effort mainly depends on the goals that people try to achieve when writing their unit tests. Some people write them from a TDD perspective. They use tests to guide themselves through development cycles, making sure they haven't made little errors. If this helps you, then please keep doing it! If it does not really help, well ...

Other people write unit tests from a regression perspective, or at least maintain them for regression testing. This part caused the most discussion. How useful are unit tests for regression testing purposes? Are you really catching regression if you isolate a single unit?

The growing interest in microservices also sheds new light on this discussion. In the future, when microservices will be widely adopted, we will be working with much smaller codebases. They might be so small and clear that unit tests are no longer required to guide us through the development process.

CI scaling

Another trending topic was scaling CI systems. It was good to see that the ideas we have at Xebia were consistent with the ideas we heard at CITCON. First off, the solution for everything (and world peace, it seems) is microservices. Unfortunately, some of us, for the time being, must deal with monolithic codebases. Luckily there are still options for your growing CI system, even though for now it remains one big chunk of code.

The staged pipeline: you test the things most likely to break first. Basically, you break your test suite up into multiple test suites and run them at separate stages in the pipeline.

But how do you determine what is most likely to break and what to test where? Tests that are most likely to break are those that are closely linked to the code changes, so run them first. Also, determine high-risk areas for your application. These areas can be identified based on trends (in failing tests) or simply through human analysis. To determine where to run the different test suites is mainly a matter of speed versus confidence. You want fast feedback so you don't want to push all your tests to the end of the pipeline. But you also don't want to wait forever before you know you can move on to the next thing.

Beer brewing for process refinement

Who isn't interested in beer brewing? Tom Denley (@scarytom) proposed a session on home brewing and the analogy. Because Arjan is a homebrewer himself, this seemed like an obvious session for him.

In addition to Tom explaining the process of brewing, we discussed how we got into brewing. In both cases, the first brew was made with a can of hopped malt syrup, adding yeast, and producing a beer from there. For your second beer, you replace the can of syrup with malt extract powder and dark malt (for flavour). At a later stage, you can replace the malt extract with ground malt.

What we basically do is start with the end in mind. If you're starting with continuous delivery, it is considered good practice to do the same: get your application deployed in production as soon as possible and optimise your process from your deployed app back to source code.

Again, it was a good conference with some nice take-aways. Next year's episode will most likely take place in Finland. The year after that... The Netherlands?

Testing cheatsheet

Mon, 11/24/2014 - 19:00

Sometimes it is not clear for everybody how unit tests relates to e2e-test. This cheatsheet, I created, describes in one page:

  1. The different definitions
  2. Different structures of the tests
  3. The importance of unit tests
  4. The importance of e2e tests
  5. External versus internal quality
  6. E2E and unit tests living next to each other

Feel free to download and use it in your project if you feel there is a confusion of tongues between unit and e2e tests.

Download: TestingCheatSheet

 

Ready, Test, Go!

Tue, 11/18/2014 - 10:42

The full potential of many an agile organization is hardly ever reached. Many teams find themselves redefining¬†user stories although they have been committed to as part of the sprint. The ‚Äėready phase‚Äô, meant to get user stories clear and sufficiently detailed so they can be implemented, is missed. How will each user story result in¬†high quality features that deliver business value? The ‚ÄėDefinition of Ready‚Äô is lacking one important entry: ‚ÄúAutomated tests are available.‚ÄĚ Ensuring to have testable and hence automated acceptance criteria before committing to user stories in a sprint, allows you to retain focus during the sprint. We define this as: Ready, Test,¬†Go!

Ready

Behaviour-Driven Development has proven to be a fine technique to write automated acceptance criteria. Using the Gherkin format (given, when, then), examples can be specified that need to be supported by the system once the user story is completed. When a sufficiently detailed list of examples is available, all Scrum stakeholders agree with the specification. Common understanding is achieved that when the story is implemented, we are one step closer to building the right thing.

Test

The specification itself becomes executable: at any moment in time, the gap between the desired and implemented functionality becomes visible. In other words, this automated acceptance test should be run continuously. First time, it happily fails. Next, implementation can start. This, following Test-Driven Development principles, starts with writing (also failing) unit tests. Then, development of the production code starts. When the unit tests are passing and acceptance tests for a story are passing, other user stories can be picked up; stories of which the tests happily fail. Tests thus act as a safeguard to continuously verify that the team is building the thing right. Later, the automated tests (acceptance tests and unit tests) serve as a safety net for regression testing during subsequent sprints.

Go!

That's simple: release your software to production. Ensure that other testing activities (performance tests, chain tests, etc) are as much as possible automated and performed as part of the sprint.

The (Agile) Test Automation Engineer

In order to facilitate or boost this way of working, the role of the test automation engineer is key. The test automation engineer is defining the test architecture and facilitating the necessary infrastructure needed to run tests often and fast. He is interacting with developers to co-develop fixtures, to understand how the production code is built, and to decide upon the structure and granularity of the test code.

Apart from their valuable and unique analytical skills, relevant testers grow their technical skills. If it cannot be automated, it cannot be checked, so one might question whether the user story is ready to be committed to in a sprint. The test automation engineer helps the Scrum teams to identify when they¬†are ‚Äėready to test‚Äô and urges the product owner and business to specify requirements ‚Äď at least for the next sprint ahead.

So: ready, test, go!!

The neverending waveform of the full-stack developer

Tue, 11/11/2014 - 16:50

There was an article on Techcrunch a couple days ago which was linked in our internal mailing list the other day, titled The Rise And Fall Of The Full Stack Developer. I read it, and I just couldn't figure out why the title is about "the fall" of the full-stack developer (and I said as much on the mailing list, after which I was encouraged to write this blog post). In this post I'll try and explain why it's not the end, but just a stage in a recurring cycle

What I read was a waveform - the article describes that first you had low-level assembly, which is specialised but still pretty straightforward since it's just one platform and language. Then things started to get more complicated, with the larger web applications involving lots of experts in various fields (Java, database management, server and JVM management, to name a few from the article).

In or around 2005, there was a bit of a revolution going on. Internet access (and high-speed internet for that matter) became accessible to all, and with it, cheap webhosting - most notably, there was a boom in PHP development. What the article highlights is that that was an era where single or small teams of developers could build a web application from scratch, without needing to lug around all of that expert knowledge - effectively, a full-stack developer (storage, webhosting, a programming language (PHP, Python, Ruby, etc), HTML, CSS, Javascript, the whole stack).

But, as the article states, that isn't the full stack of today - added to that are things like machine learning, mobile development, more advanced and less traditional programming languages, frameworks and persistence solutions, etcetera. The conclusion of the author is that it's too much for one full-stack developer to take, that there's no way a full-stack developer can be an expert in all of those fields - and he basically renames the person that knows a bit of all of those technologies a full-stack integrator, and declares the full-stack developer dead.

But I didn't see that. What I saw was just history repeating - from simple and manageable, to complex and too much for one person to handle and know all about. From assembly and PHP, to Java enterprise software and Docker-contained AWS instances running a MongoDB and Scala REST interface to power your Android, iOS and single-page AngularJS-powered webapps (to name but a few buzzwords).

I'm sure the next 'simple' wave is already around the corner - maybe it's already here, lurking somewhere until some more influential guy goes "Hey guys, let's go back to the simpler, good old times where you wrote code and it Just Worked‚ĄĘ!", where a wave of developer will follow - most likely a combination of younger people, new to the software development world, and older people that have been stuck in an overly complicated software development rut for far too long.

As for not being able to keep up with it all, this is why it's probably better to assemble teams out of T-shaped developers - full-stack developers with a broad knowledge set (and more importantly, broad interests and curiosity), but with at least one specialisation. This was extended within Xebia to pi-shaped developers, then made extreme to comb-shaped developers, but the latter is just a generalist like mentioned in the article - a jack of all trades, master of none. And it's important to realise, as a developer, that it's OK to not know everything, to miss some update, to not learn that fancy new programming language by people disgruntled by Java's slow development - there's simply too much information updates today, and trying to manage it all in the extremest forms can lead to serious problems. But the rapid development is a good thing, I might add, I don't think the software development world has ever moved as fast as it does today, and it's not nearly the end yet as long as more than half of the world's population doesn't have access to the internet and its boundless resources.

I think increasing complexity is just part of software development. I mean look at Facebook and Twitter - they were started using those simple, accessible tools like PHP and Ruby on Rails, which allowed them to get a flying start and adapt quickly to their huge growth (and with the former desperately clinging to their decision, despite lots of people telling them they should use a different technology), but despite that they still turned into some of the most complicated pieces of software ever. The important bit is to be able to manage all that, not so much a decision between a simple or complicated toolset - or having all of your developers know everything. Similarly, the current trend is microservices - again back to simple, one-purpose miniature applications that do one thing and do them right. But those will just end up being the next generation of huge, complicated software systems, given enough time. As a colleague stated, "Microservices are just hipster SOA".

As the saying goes, the more things change, the more they stay the same. The full-stack developer isn't dead and won't be going away any time soon, he'll just go by a different name depending on the times - J2EE Certified Software Engineer, full-stack developer, multidisciplinary expert, T-shaped developer, Xebian, chief of technology evangelisation, or whatever else the HR or marketing departments of the near future will come up with.

Mutation Testing: How Good are your Unit Tests?

Thu, 11/06/2014 - 20:56

You write unit tests for every piece of code you deliver. Your test coverage is close to 100%. So when the point comes when you have to make some small changes to the existing code, you feel safe and confident that your test suite will protect you against possible mistakes.
You make your changes, and all your tests still pass. You should be fairly confident now that you can commit your new code without breaking anything, right?

Well, maybe not. Maybe your unit tests were fooling you. Sure they covered every line of your code, but they could have performed the wrong assertions.
In this post I will introduce mutation testing. Mutation testing can help you find omissions in your unit tests.

Let's begin with a small example:

package com.xebia;

public class NameParser {
  public Person findPersonWithLastName(String[] names, String lastNameToFind) {
    Person result = null;
    for(int i=0; i <= names.length; i++) { // bug 1
      String[] parts = names[i].split(" ");
      String firstName = parts[0];
      String lastName = parts[1];
      if(lastName.equals(lastNameToFind)) {
        result = new Person(firstName, lastName);
        break;
      }
    }
    return result;
  }
}

NameParser takes a list of strings which consist of a first name and a last name, searches for the entry with a given last name, instantiates a Person object out of it and returns it.
Here is the Person class:

package com.xebia;

public class Person {
  private final String firstName;
  private final String lastName;

  public Person(String firstName, String lastName) {
    this.firstName = firstName;
    this.lastName = lastName;
  }

  public String getFirstName() {
    return firstName;
  }

  public String getLastName() {
    return firstName; // bug 2
  }
}

You can see that there are two bugs in the code. The first one is in the loop in NameParser, which loops past the end of the names array. The second on is in Person, which mistakenly returns firstName in its getLastName method.

NameParser has a unit test:

package com.xebia;

import org.junit.Before;
import org.junit.Test;
import static org.junit.Assert.assertEquals;

public class NameParserTest {
  private NameParser nameParser;
  private String[] names;

  @Before
  public void setUp() {
    nameParser = new NameParser();
    names = new String[]{"Mike Jones", "John Doe"};
  }

  @Test
  public void shouldFindPersonByLastName() {
    Person person = nameParser.findPersonWithLastName(names, "Doe");
    String firstName = person.getFirstName();
    String lastName = person.getLastName();
    assertEquals("John", firstName);
  }
}

The unit tests covers the Person and NameParser code for 100% and succeeds!
It doesn't find the bug in Person.getLastName because it simply forgets to do an assertion on it. And it doesn't find the bug in the loop in NameParser because it doesn't test the case where the names list does not contain the given last name; so the loop always terminates before it has a chance to throw an IndexOutOfBoundsException.

Especially the last case is easy to overlook, so it would be nice if there existed a tool which could detect these kinds of problems.
And there is one: actually there are a couple. For this post I have chosen PIT, down at the end are links to some alternatives.

But first: what is mutation testing?

A mutation test will make a small change to your code and then run the unit test(s). Such a change is called a 'mutant'. If a change can be made and the unit tests still succeed, it will generate a warning saying that the mutant 'survived'.
The test framework will try to apply a number of predefined mutants at every point in your code where they are applicable. The higher the percentage of the mutants that get killed by your unit tests, the better the quality of your test suite.
Examples of mutants are: negating a condition in an If statement, changing a conditional boundary in a For loop, or throwing an exception at the end of a method.

Putting NameParser's testcase to the test with PIT

PIT stands for Parallel Isolated Test, which is what the project originally was meant for. But it turned out to be a much more interesting goal to do mutation testing, which required much of the same plumbing.

PIT integrates with JUnit or TestNG and can be configured with Maven, Gradle and others. Or it can be used directly as a plugin in Eclipse or IntelliJ.
I'm choosing for the last option: the IntelliJ plugin. The setup is easy: just install PITest from the plugin manager and you are ready to go. Once you're done, you'll find a new launch configuration option in the 'edit configurations' menu called PIT.

PIT launch configuration

You have to specify the classes where PIT will makes its mutations under 'Target classes'.
When we run the mutation test, PIT creates a Html report with the results for every class.
Here are the results for the NameParser class:

NameParser mutation testing results

As you can read under 'Mutations', PIT has been able to apply five code mutations to the NameParser class. Four of them resulted in a failing NameParserTest, which is exactly what we'd like to see.
But one of them did not: when the condition boundary in line 6, the loop constraint, was changed, NameParserTest still succeeded!
PIT changes loop constraints with a predefined algorithm; in this case, when the loop constraint was i <= names.length, it changed the '<=' into a '<'. Actually this accidentally corrected the bug in NameParser, and of course that didn't break the unit test.
So PIT found an omission in our unit test here, and it turned out that this omission even left a bug undetected!

Note that this last point doesn't always need to be the case. It could be that for the correct behavior of your class, there is some room for some conditions to change.
In the case of NameParser for instance, it could have been a requirement that the names list always contained an entry with the last name that was to be found. In that case the behavior for a missing last name would be unspecified and an IndexOutOfBoundsException would have been as good a result as anything else.
So PIT can only find strong indications of omissions in your unit tests, but they don't necessarily have to be.

And here are the results for the Person class:

Person mutation test results

PIT was able to do two mutations in the Person class; one in every getter method. Both times it replaced the return value with null. And as expected, the mutation in the getLastName method went undetected by our unit test.
So PIT found the second omission as well.

Conclusion

In this case, mutation testing would have helped us a lot. But there can still be cases where possible bugs can go unnoticed. In our code for example, there is no test in NameParser test that verifies the behavior when an entry in the names list does not contain both a first name and a last name. But PIT didn't find this omission.
Still it might make good sense to integrate mutation testing in your build process. PIT can be configured to break your Maven build if too many warnings are found.
And there's a lot more that can be configured as well, but for that I recommend to bring a visit to the website of PIT at www.pitest.org.

Alternatives

PIT is not the only mutation testing framework for Java, but it is the most popular and the one most actively maintained. Others are ¬ĶJava and Jumble.
Although most mutation testing frameworks are written for Java, probably because it's so easy to dynamically change its bytecode, mutation testing is possible in some other languages as well: notable are grunt-mutation-testing for Javascript and Mutator, a commercial framework which is available for a couple of languages.

Swift Function Currying

Thu, 11/06/2014 - 12:55

One of the lesser known features of Swift is Function Currying. When you read the Swift Language Guide you won't find anything about curried functions. Apple only describes it in their Swift Language Reference. And that's a pity, since it's a very powerful and useful feature that deserves more attention. This post will cover the basics and some scenarios in which it might be useful to use curried functions.

I assume you're already somewhat familiar with function currying since it exists in many other languages. If not, there are many articles on the Internet that explain what it is and how it works. In short: you have a function that receives one or more parameters. You then apply one or more known parameters to that function without already executing it. After that you get a function reference to a new function that will call the original function with the applied parameters.

One situation in which I find it useful to use curried functions is with completion handlers. Imagine you have a function that makes a http request and looks something like this:

func doGET(url: String, completionHandler: ([String]?, NSError?) -> ()) {
    // do a GET HTTP request and call the completion handler when receiving the response
}

This is a pretty common pattern that you see with most networking libraries as well. We can call it with some url and do a bunch of things in the completion handler:

doGET("http://someurl.com/items?all=true", completionHandler: { results, error in
    self.results = results
    self.resultLabel.text = "Got all items"
    self.tableView.reloadData()
})

The completion handler can become a lot more complex and you might want to reuse it in different places. Therefore you can extract that logic into a separate function. Luckily with Swift, functions are just closures so we can immediately pass a completion handler function to the doGET function:

func completionHandler(results: [String]?, error: NSError?) {
    self.results = results
    self.resultLabel.text = "Got all items"
    self.tableView.reloadData()
}

func getAll() {
    doGET("http://someurl.com/items?all=true", completionHandler)
}

func search(search: String) {
    doGET("http://someurl.com/items?q=" + search, completionHandler)
}

This works well, as long as the completion handler should always do exactly the same thing. But in reality, that's usually not the case. In the example above, the resultLabel will always display "Got all items". Lets change that into "Got searched items" for the search request:

func search(search: String) {
    doGET("http://someurl.com/items?q=" + search, {results, error in
        self.completionHandler(results, error: error)
        self.resultLabel.text = "Got searched items"
    })
}

This will work, but it doesn't look very nice. What we actually want is to have this dynamic behaviour in the completionHandler function. We can change the completionHandler in such a way that it accepts the text for the resultLabel as a parameter and then returns the actual completion handler as a closure.

func completionHandler(text: String) -> ([String]?, NSError?) -> () {
    return {results, error in
        self.results = results
        self.resultLabel.text = text
        self.tableView.reloadData()
    }
}

func getAll() {
    doGET("http://someurl.com/items?all=true", completionHandler("Got all items"))
}

func search(search: String) {
    doGET("http://someurl.com/items?q=" + search, completionHandler("Got searched items"))
}

And as it turns out, this is exactly what we can also do using currying. We just need to add the parameters of the actual completion handler as a second parameters group to our function:

func completionHandler(text: String)(results: [String]?, error: NSError?) {
    self.results = results
    self.resultLabel.text = text
    self.tableView.reloadData()
}

Calling this with the first text parameter will not yet execute the function. Instead it returns a new function with the [String]?, NSError? as parameters. Once that function is called the completionHandler function is finally executed.

You can create as many levels of this currying as you want. And you can also leave the last parameter group empty just to get a reference to the fully applied function. Let's look at another example. We have a simple function that sets the text of the resultLabel:

func setResultLabelText(text: String) {
    resultLabel.text = text
}

And for some reason, we need to call this method asynchronously. We can do that using the Grand Central Dispatch functions:

dispatch_async(dispatch_get_main_queue(), {
    self.setResultLabelText("Some text")
})

Since the dispatch_async function only accepts a closure without any parameters, we need to create an inner closure here. If the setResultLabelText was a curried function, we could fully apply it with the parameter and get a reference to a function without parameters:

func setResultLabelText(text: String)() { // now curried
    resultLabel.text = text
}

dispatch_async(dispatch_get_main_queue(), setResultLabelText("Some text"))

But you might not always have control over such functions, for example when you're using third party libraries. In that case you cannot change the original function into a curried function. Or you might not want to change it since you already using it at many other places and you don't want to break anything. In that case we can achieve something similar by creating a function that creates the curried function for us:

// defined in global scope
func curry<T>(f: (T) -> (), arg: T)() {
    f(arg)
}

We can now use it as following:

func setResultLabelText(text: String) {
    resultLabel.text = text
}

dispatch_async(dispatch_get_main_queue(), curry(setResultLabelText, "Some text"))

Probably in this example it might be just as easy to go with the inner closure, but being able to pass around partial applied functions is very powerful and used in many programming languages already.

Unfortunately the last example also show a great drawback about the way currying is implemented in Swift: you cannot simply curry normal functions. It would be great to be able to curry any function that takes multiple parameters instead of having to explicitly create curried functions. Another drawback is that you can only curry in the defined order of parameters. That doesn't allow you to do reverse currying (e.g. apply only the last parameter) or even apply just any parameter you want, regardless of its position. Hopefully the Swift language will evolve in this and get more powerful currying features.

iOS localization tricks for Storyboard and NIB files

Fri, 10/31/2014 - 23:36

Localization in iOS from Interface Builder designed UI has never been without any problems. The right way of doing localization is by having multiple Strings files. Duplicating Nib or Storyboard files and then changing the language is not an acceptable method. Luckily Xcode 5 has improved this for Storyboards by introducing Base Localization, but I've personally come across several situations where this didn't work at all or when it seemed buggy. Also Nib (Xib) files without ViewController don't support it.

In this post I'll show a couple of tricks that can help with the Localization of Storyboard and Nib files.

Localized subclasses

When you use this method, you create specialized subclasses of view classes that handle the localization in the awakeFromNib() method. This method is called for each view that is loaded from a Storyboard or Nib and all properties that you've set in Interface Builder will be set already.

For UILabels, this means getting the text property, localizing it and setting the text property again.

Using Swift, you can create a single file (e.g. LocalizationView.swift) in your project and put all your subclasses there. Then add the following code for the UILabel subclass:

class LocalizedLabel : UILabel {
    override func awakeFromNib() {
        if let text = text {
            self.text = NSLocalizedString(text, comment: "")
        }
    }
}

Now you can drag a label onto your Storyboard and fill in the text in your base language as you would normally. Then change the Class to LocalizedLabel and it will get the actual label from you Localizable.strings file.

Screen Shot 2014-10-31 at 22.45.46

Screen Shot 2014-10-31 at 22.46.56

No need to make any outlets or write any code to change it!

You can do something similar for UIButtons, even though they don't have a single property for the text on a button.

class LocalizedButton : UIButton {
    override func awakeFromNib() {
        for state in [UIControlState.Normal, UIControlState.Highlighted, UIControlState.Selected, UIControlState.Disabled] {
            if let title = titleForState(state) {
                setTitle(NSLocalizedString(title, comment: ""), forState: state)
            }
        }
    }
}

This will even allow you to set different labels for the different states like Normal and Highlighted.

User Defined Runtime Attributes

Another way is to use the User Defined Runtime Attributes. This method requires slightly more work, but has two small advantages:

  1. You don't need to use subclasses. This is nice when you already use another custom subclass for your labels, buttons and other view classes.
  2. Your keys in the Strings file and texts that show up in the Storyboard don't need to be the same. This works well when you use localization keys such as myCoolTableViewController.header.subtitle. It doesn't look very nice to see those everywhere in your Interface Builder labels and buttons.

So how does this work? Instead of creating a subclass, you instead add a computed property to an existing view class. For UILabels you use the following code:

extension UILabel {

    var localizedText: String {
        set (key) {
            text = NSLocalizedString(key, comment: "")
        }
        get {
            return text!
        }
    }

}

Now you can add a User Defined Runtime Attribute with the key localizedText to your UILabel and have the Localization key as its value.

Screen Shot 2014-10-31 at 23.05.18

Screen Shot 2014-10-31 at 23.06.16

Also here if you want to make this work for buttons, it becomes slightly more complicated. You will have to add a property for each state that needs a label.

extension UIButton {
    var localizedTitleForNormal: String {
        set (key) {
            setTitle(NSLocalizedString(key, comment: ""), forState: .Normal)
        }
        get {
            return titleForState(.Normal)!
        }
    }

    var localizedTitleForHighlighted: String {
        set (key) {
            setTitle(NSLocalizedString(key, comment: ""), forState: .Highlighted)
        }
        get {
            return titleForState(.Highlighted)!
        }
    }
}
Conclusion

Always try and pick the best solution for your problem. Use Storyboard Base Localization if that works well for you. If it doesn't, use the approach with subclasses if you don't need to use another subclass and if you don't care about using your base location strings as localization keys. Else, use the last approach with User Defined Runtime Attributes.

How to Dockerize your Dropwizard Application

Wed, 10/29/2014 - 10:47

If you want to deploy your Dropwizard Application on a Docker server, you can Dockerize your Dropwizard Application. Since a Dropwizard Application is already packaged as an executable Java ARchive file, creating a Docker image for such an application should be easy.

 

In this blog, you will learn how to Dockerize a Dropwizard Application using 4 easy steps.

Before you start

  • You are going to use the Dropwizard-example application, which can be found at the Dropwizard GitHub repository.
  • Additionally you need Docker. I used Boot2Docker to run the Dockerized Dropwizard Application on my laptop. If you use boot2Docker, you may need this Boot2Docker workaround to access your Dockerized Dropwizard application.
  • This blog does not describe how to create Dropwizard applications. The Dropwizard getting started guide provides an excellent starting point if you like to know more about building your own Dropwizard applications.

 

Step 1: create a Dockerfile

You can start with creating a Dockerfile. Docker can automatically build images by reading the instructions described in this file. Your Dockerfile could look like this:

FROM dockerfile/java:openjdk-7-jdk

ADD dropwizard-example-1.0.0.jar /data/dropwizard-example-1.0.0.jar

ADD example.keystore /data/example.keystore

ADD example.yml /data/example.yml

RUN java -jar dropwizard-example-1.0.0.jar db migrate /data/example.yml

CMD java -jar dropwizard-example-1.0.0.jar server /data/example.yml

EXPOSE 8080

 

The Dropwizard Application needs a Java Runtime, so you can start from an base image already available at Docker Hub, for example: dockerfile/java:openjdk-7-jdk.

You must add the Dropwizard Application files to the image, using the ADD instruction in your Dockerfile.

Next, simply specify the commands of your Dropwizard Application, which you want to execute during image build and container runtime. In the example above, the db migrate command is executed when the Docker image is build and the server command is executed when you issue a Docker run command to create a running container.

Finally, the EXPOSE instruction tells Docker that your container will listen on the specified port(s) at runtime.

 

Step 2: build the Docker image

Place the Dockerfile and your application files in a directory and execute the Docker build command to build an Docker image.

docker@boot2docker:~$ docker build -t dropwizard/dropwizard-example ~/dropwizard/

 

In the console output you should be able to that the Dropwizard Application db migrate command is executed. If everything is ok, the last line reported informs you that the image is successfully build.

Successfully built dd547483b57b

 

Step 3: run the Docker image

Use the Docker run command to create a container based on the image you have created. If you need to find your image id use the Docker images command to list your images. It should take around 3 seconds to start the Dockerized Dropwizard example application.

Docker run ‚Äďp 8080:8080 dd547483b57b

Notice that I included the ‚Äďp option to include a network port binding, which maps 8080 inside the container to port 8080 on the Docker host. ¬†You can verify whether your container is running using the docker ps command.

docker@boot2docker:~$ docker ps

CONTAINER ID        IMAGE                                  COMMAND                CREATED             STATUS              PORTS                    NAMES

3b6fb75adad6        dropwizard/dropwizard-example:latest   "/bin/sh -c 'java -j   3 minutes ago       Up 3 minutes        0.0.0.0:8080->8080/tcp   high_turing

 

  1. Test the application

Now the application is ready for use. You can access the application using your Docker host ip address and the forward port 8080. For example, use the Google Advanced Rest Client App to register ‚ÄúJohn Doe‚ÄĚ.

GoogleRestClient

How to create Java microservices with Dropwizard

Mon, 10/27/2014 - 15:10

On Tuesday October 14th the Amsterdam Middleware Meetup experimented with Dropwizard. The idea was to find out what this technology is about, where it could be useful and what the alternatives are. So below I’ll give you an overview of Dropwizard and compare it to Spring Boot.
The Dropwizard website claims:

Dropwizard pulls together stable, mature libraries from the Java ecosystem into a simple, light-weight package that lets you focus on getting things done.

I’ll discuss each of these claims below.

Stable and mature
Dropwizard uses Jetty, Jersey, Jackson and Metrics as its most important frameworks, but also a host of other stuff like Guava, Liquibase and Joda Time. The latest Dropwizard release is version 0.7.1, released on June 20th 2014. It depends on these versions of some core libraries:
Jetty - 9.2.3.v20140905 - May 2014
Jackson - 2.4.1 - June 2014
Jersey - 2.11 - July 2014

The table shows that stable != out-of-date which is fine of course. The versions of core libraries used are recent though. I guess ‚Äėstable‚Äô means libraries with a long history.

Simple
The components of a Dropwizard application are shown below (taken from the tutorial
http://dropwizard.io/getting-started.html):
Dropview components overview

  1. Application (HelloWorldApplication.java): the applications main method, responsible for startup.
  2. Configuration (HelloWorldConfiguration.java) sets configuration for an environment, this is where you may set hostnames for systems the application depends on or set usernames.
  3. Data object (Saying.java).
  4. Resource (HelloWordResource.java): service implementation entry point
  5. Health Check (TemplateHealthCheck.java): runtime tests that show if the application still works.

Light weight
We did some experiments trying to answer the question whether Dropwizard applications are light weight. The table below summarizes some of the sizes of deployments and tools.
Tomcat size 14 mb
Tomcat lib folder size 7 MB
Jetty size 14,6 MB
Jetty in Dropwizard jar: 5,4 MB
Dropwizard tutorial example 10 mb
Dropwizard extended example 20 MB
Dropwizard Hibernate classes in package: 5 MB

A Tomcat or Jetty installation takes about 14 MB, but if you count only the lib folder the size goes down to about 7 MB. The Jetty folder in Dropwizard however is only 5.5 MB. Apparently Dropwizard managed to strip away some code you don’t really need (or is packaged somewhere else, we didn’t look into that).
Building the tutorial results in a 10 MB jar, so if you would run a webapp in its own Tomcat container, switching to Dropwizard saves quite a bit. On the other hand, deployment size isn’t all that important if we’re still talking < 50 MB.
Compared to your default Weblogic install (513 MB, Weblogic-only on OSX) however, savings are humongous (but this is also true when you compare Weblogic to Tomcat or Jetty).

Productivity
We tried to run the build for the tutorial application (dropwizard-example in the dropwizard project on Github). This works fine and takes about 8 seconds using mocks for external connections. One option to explore would be to run tests against a deployed application. What we’re used to is that deploying an application for test takes lots of time and resources, but starting a Dropwizard app is quite cheap. Therefore it would be possible to run an integration test of services at the end of a build. This would be quite hard to do with e.g. Weblogic or Websphere.

Spring boot
Spring boot is interesting, as well as the discussion around the differences between Spring boot and Dropwizard. See https://groups.google.com/forum/#!topic/dropwizard-user/vH1h2PgC8bU

The official Spring boot website says: Spring Boot makes it easy to create stand-alone, production-grade Spring based Applications that can you can "just run". We take an opinionated view of the Spring platform and third-party libraries so you can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration.
It’s good to see a platform change according to new insights, but still, I remember Rod Johnson saying some ten years ago that J2EE was bloated and complex and Spring was the answer. Now it seems we need Spring boot to make Spring simple? Or is it just that we don’t need application servers anymore to divide resources among processes?

Dropwizard and Docker
Finally we experimented with running Dropwizard in a Docker container. This can be done with limited effort because Dropwizard applications have such a small number of dependencies. Thomas Kruitbosch will report on this later.

References
Spring boot: http://projects.spring.io/spring-boot/
Dropwizard: http://dropwizard.io/

How to deploy a Docker application into production on Amazon AWS

Fri, 10/17/2014 - 17:00

Docker reached production status a few months ago. But having the container technology alone is not enough. You need a complete platform infrastructure before you can deploy your docker application in production. Amazon AWS offers exactly that: a production quality platform that offers capacity provisioning, load balancing, scaling, and application health monitoring for Docker applications.

In this blog, you will learn how to deploy a Docker application to production in five easy steps.

For demonstration purposes, you are going to use the node.js application that was build for CloudFoundry and used to demonstrate Deis in a previous post. A truly useful app of which the sources are available on github.

1. Create a Dockerfile

First thing you need to do is to create a Dockerfile to create an image. This is quite simple: you install the node.js and npm packages, copy the source files and install the javascript modules.

# DOCKER-VERSION 1.0
FROM    ubuntu:latest
#
# Install nodejs npm
#
RUN apt-get update
RUN apt-get install -y nodejs npm
#
# add application sources
#
COPY . /app
RUN cd /app; npm install
#
# Expose the default port
#
EXPOSE  5000
#
# Start command
#
CMD ["nodejs", "/app/web.js"]
2. Test your docker application

Now you can create the Docker image and test it.

$ docker build -t sample-nodejs-cf .
$ docker run -d -p 5000:5000 sample-nodejs-cf

Point your browser at http://localhost:5000, click the 'start' button and Presto!

3. Zip the sources

Now you know that the instance works, you zip the source files. The image will be build on Amazon AWS based on your Dockerfile.

$ zip -r /tmp/sample-nodejs-cf-srcs.zip .
4. Deploy Docker application to Amazon AWS

Now you install and configure the amazon AWS command line interface (CLI) and deploy the docker source files to elastic beanstalk.  You can do this all manually, but here you use the script deploy-to-aws.sh that I created.

$ deploy-to-aws.sh \
         sample-nodejs-cf \
         /tmp/sample-nodejs-cf-srcs.zip \
         demo-env

After about 8-10 minutes your application is running. The output should look like this..

INFO: creating application sample-nodejs-cf
INFO: Creating environment demo-env for sample-nodejs-cf
INFO: Uploading sample-nodejs-cf-srcs.zip for sample-nodejs-cf, version 1412948762.
upload: ./sample-nodejs-cf-srcs.zip to s3://elasticbeanstalk-us-east-1-233211978703/1412948762-sample-nodejs-cf-srcs.zip
INFO: Creating version 1412948762 of application sample-nodejs-cf
INFO: demo-env in status Launching, waiting to get to Ready..
...
INFO: demo-env in status Launching, waiting to get to Ready..
INFO: Updating environment demo-env with version 1412948762 of sample-nodejs-cf
INFO: demo-env in status Updating, waiting to get to Ready..
...
INFO: demo-env in status Updating, waiting to get to Ready..
INFO: Version 1412948762 of sample-nodejs-cf deployed in environment
INFO: current status is Ready, goto http://demo-env-vm2tqi3qk4.elasticbeanstalk.com
5. Test your Docker application on the internet!

Your application is now available on the Internet. Browse to the designated URL and click on start. When you increase the number of instances at Amazon, they will appear in the application. When you deploy a new version of the application, you can observe how new versions of the application  appear without any errors on the client application.

For more information, goto Amazon Elastic Beanstalk adds Docker support. and Dockerizing a Node.js Web App.

Then When Given

Fri, 10/17/2014 - 14:50

People who practice ATDD all know how frustrating it can be to write automated examples. Especially when you get stuck overthinking the preconditions of examples.

This post describes an alternative approach to writing acceptance tests: write them backwards!

Imagine that you are building the very first online phone book. We need to define an acceptance tests for viewing the location of a florist. Using the Given-When-Then formula you would probably describe the behaviour like this.


Given I am on the online phone book homepage
When I type ‚ÄúFlorist‚ÄĚ in the business type field
And I click …
...

Most of the time you will be discussing and describing details that have nothing to do with viewing the location of a florist. To avoid this, write down the Then clause of the formula first.
Make sure the Then clause contains an observable result.


Then I see the location ‚ÄúFloriststreet 123‚ÄĚ

Next, we will try to answer the following question: What caused the Then clause?
Make sure the When clause contains an actor and an action.


When I click ‚ÄúView map‚ÄĚ of the search result
Then I see the location ‚ÄúFloriststreet 123‚ÄĚ

The last thing we will need to do is answer the following question: Why can I perform that action?
Make sure the Given clause contains a simple precondition.


Given I see a search result for florist ‚ÄúFloral Designs‚ÄĚ
When I click ‚ÄúView map‚ÄĚ of the search result
Then I see the location ‚ÄúFloriststreet 123‚ÄĚ

You might have noticed that I left out certain parts where the user goes to the homepage and selects UI objects in the search area. It was not worth mentioning in the Given-When-Then formula. Too much details make us lose focus of what we really want to check. The essence of this acceptance test is clicking on the link "View map" and exposing the location to the user.

Try it a couple of times and let me know how it went.

AngularJS Training Week

Tue, 10/14/2014 - 07:00

Just a few more weeks and it's the AngularJS Training Week at Xebia in Hilversum (The Netherlands). 4 days full with AngularJS content, from 17 to 20 October, 2014. In these different days we will cover the AngularJS basics, AngularJS advanced topics, Tooling & Scaffolding and Testing with Jasmine, Karma and Protractor.

If you already have some experience or if you are only interested in one or two of the topics, then you can sign up for just the days that are of interest to you.

Visit www.angular-training.com for a full overview of the days and topics or sign up on the Xebia Training website using the links below.

Fast and Easy integration testing with Docker and Overcast

Mon, 10/13/2014 - 18:40
Challenges with integration testing

Suppose that you are writing a MongoDB driver for java. To verify if all the implemented functionality works correctly, you ideally want to test it against a REAL MongoDB server. This brings a couple of challenges:

  • Mongo is not written in java, so we can not embed it easily in our java application
  • We need to install and configure MongoDB somewhere, and maintain the installation, or write scripts to set it up as part of our test run.
  • Every test we run against the mongo server, will change the state, and tests might influence each other. We want to isolate our tests as much as possible.
  • We want to test our driver against multiple versions of MongoDB.
  • We want to run the tests as fast as possible. If we want to run tests in parallel, we need multiple servers. How do we manage them?

Let's try to address these challenges.

First of all, we do not really want to implement our own MonogDB driver. Many implementations exist and we will be reusing the mongo java driver to focus on how one would write the integration test code.

Overcast and Docker

logoWe are going to use Docker and Overcast. Probably you already know Docker. It's a technology to run applications inside software containers. Overcast is the library we will use to manage docker for us. Overcast is a open source java library
developed by XebiaLabs to help you to write test that connect to cloud hosts. Overcast has support for various cloud platforms, including EC2, VirtualBox, Vagrant, Libvirt (KVM). Recently support for Docker has been added by me in Overcast version 2.4.0.

Overcast helps you to decouple your test code from the cloud host setup. You can define a cloud host with all its configuration separately from your tests. In your test code you will only refer to a specific overcast configuration. Overcast will take care of creating, starting, provisioning that host. When the tests are finished it will also tear down the host. In your tests you will use Overcast to get the hostname and ports for this cloud host to be able to connect to them, because usually these are dynamically determined.

We will use Overcast to create Docker containers running a MongoDB server. Overcast will help us to retrieve the dynamically exposed port by the Docker host. The host in our case will always be the docker host. Docker in our case runs on an external Linux host. Overcast will use a TCP connection to communicate with Docker. We map the internal ports to a port on the docker host to make it externally available. MongoDB will internally run on port 27017, but docker will map this port to a local port in the range 49153 to 65535 (defined by docker).

Setting up our tests

Lets get started. First, we need a Docker image with MongoDB installed. Thanks to the Docker community, this is as easy as reusing one of the already existing images from the Docker Hub. All the hard work of creating such an image is already done for us, and thanks to containers we can run it on any host capable of running docker containers. How do we configure Overcast to run the MongoDB container? This is our minimal configuration we put in a file called overcast.conf:

mongodb {
    dockerHost="http://localhost:2375"
    dockerImage="mongo:2.7"
    exposeAllPorts=true
    remove=true
    command=["mongod", "--smallfiles"]
}

That's all! The dockerHost is configured to be localhost with the default port. This is the default value and you can omit this. The docker image called mongo version 2.7 will be automatically pulled from the central docker registry. We set exposeAllPorts to true to inform docker it needs to dynamically map all exposed ports by the docker image. We set remove to true to make sure the container is automatically removed when stopped. Notice we override the default container startup command by passing in an extra parameter "--smallfiles" to improve testing performance. For our setup this is all we need, but overcast also has support for defining static port mappings, setting environment variables, etc. Have a look at the Overcast documentation for more details.

How do we use this overcast host in our test code? Let's have a look at the test code that sets up the Overcast host and instantiates the mongodb client that is used by every test. The code uses the TestNG @BeforeMethod and @AfterMethod annotations.

private CloudHost itestHost;
private Mongo mongoClient;

@BeforeMethod
public void before() throws UnknownHostException {
    itestHost = CloudHostFactory.getCloudHost("mongodb");
    itestHost.setup();

    String host = itestHost.getHostName();
    int port = itestHost.getPort(27017);

    MongoClientOptions options = MongoClientOptions.builder()
        .connectTimeout(300 * 1000)
        .build();

    mongoClient = new MongoClient(new ServerAddress(host, port), options);
    logger.info("Mongo connection: " + mongoClient.toString());
}

@AfterMethod
public void after(){
    mongoClient.close();
    itestHost.teardown();
}

It is important to understand that the mongoClient is the object under test. Like mentioned before, we borrowed this library to demonstrate how one would integration test such a library. The itestHost is the Overcast CloudHost. In before(), we instantiate the cloud host by using the CloudHostFactory. The setup() will pull the required images from the docker registry, create a docker container, and start this container. We get the host and port from the itestHost and use them to build our mongo client. Notice that we put a high connection timeout on the connection options, to make sure the mongodb server is started in time. Especially the first run it can take some time to pull images. You can of course always pull the images beforehand. In the @AfterMethod, we simply close the connection with mongoDB and tear down the docker container.

Writing a test

The before and after are executed for every test, so we will get a completely clean mongodb server for every test, running on a different port. This completely isolates our test cases so that no tests can influence each other. You are free to choose your own testing strategy, sharing a cloud host by multiple tests is also possible. Lets have a look at one of the tests we wrote for mongo client:

@Test
public void shouldCountDocuments() throws DockerException, InterruptedException, UnknownHostException {

    DB db = mongoClient.getDB("mydb");
    DBCollection coll = db.getCollection("testCollection");
    BasicDBObject doc = new BasicDBObject("name", "MongoDB");

    for (int i=0; i &lt; 100; i++) {
        WriteResult writeResult = coll.insert(new BasicDBObject("i", i));
        logger.info("writing document " + writeResult);
    }

    int count = (int) coll.getCount();
    assertThat(count, equalTo(100));
}

Even without knowledge of MongoDB this test should not be that hard to understand. It creates a database, a new collection and inserts 100 documents in the database. Finally the test asserts if the getCount method returns the correct amount of documents in the collection. Many more aspects of the mongodb client can be tested in additional tests in this way. In our example setup, we have implemented two more tests to demonstrate this. Our example project contains 3 tests. When you run the 3 example tests sequentially (assuming the mongo docker image has been pulled), you will see that it takes only a few seconds to run them all. This is extremely fast.

Testing against multiple MongoDB versions

We also want to run all our integration tests against different versions of the mongoDB server to ensure there are no regressions. Overcast allows you to define multiple configurations. Lets add configuration for two more versions of MongoDB:

defaultConfig {
    dockerHost="http://localhost:2375"
    exposeAllPorts=true
    remove=true
    command=["mongod", "--smallfiles"]
}

mongodb27=${defaultConfig}
mongodb27.dockerImage="mongo:2.7"

mongodb26=${defaultConfig}
mongodb26.dockerImage="mongo:2.6"

mongodb24=${defaultConfig}
mongodb24.dockerImage="mongo:2.4"

The default configuration contains the configuration we have already seen. The other three configurations extend from the defaultConfig, and define a specific mongoDB image version. Lets also change our test code a little bit to make the overcast configuration we use in the test setup depend on a parameter:

@Parameters("overcastConfig")
@BeforeMethod
public void before(String overcastConfig) throws UnknownHostException {
    itestHost = CloudHostFactory.getCloudHost(overcastConfig);

Here we used the paramaterized tests feature from TestNG. We can now define a TestNG suite to define our test cases and how to pass in the different overcast configurations. Lets have a look at our TestNG suite definition:

<suite name="MongoSuite" verbose="1">
    <test name="MongoDB27tests">
        <parameter name="overcastConfig" value="mongodb27"/>
        <classes>
            <class name="mongo.MongoTest" />
        </classes>
    </test>
    <test name="MongoDB26tests">
        <parameter name="overcastConfig" value="mongodb26"/>
        <classes>
            <class name="mongo.MongoTest" />
        </classes>
    </test>
    <test name="MongoDB24tests">
        <parameter name="overcastConfig" value="mongodb24"/>
        <classes>
            <class name="mongo.MongoTest" />
        </classes>
    </test>
</suite>

With this test suite definition we define 3 test cases that will pass a different overcast configuration to the tests. The overcast configuration plus the TestNG configuration enables us to externally configure against which mongodb versions we want to run our test cases.

Parallel test execution

Until this point, all tests will be executed sequentially. Due to the dynamic nature of cloud hosts and docker, nothing limits us to run multiple containers at once. Lets change the TestNG configuration a little bit to enable parallel testing:

<suite name="MongoSuite" verbose="1" parallel="tests" thread-count="3">

This configuration will cause all 3 test cases from our test suite definition to run in parallel (in other words our 3 overcast configurations with different MongoDB versions). Lets run the tests now from IntelliJ and see if all tests will pass:

Screen Shot 2014-10-08 at 8.32.38 PM
We see 9 executed test, because we have 3 tests and 3 configurations. All 9 tests have passed. The total execution time turned out to be under 9 seconds. That's pretty impressive!

During test execution we can see docker starting up multiple containers (see next screenshot). As expected it shows 3 containers with a different image version running simultaneously. It also shows the dynamic port mappings in the "PORTS" column:

Screen Shot 2014-10-08 at 8.50.07 PM

That's it!

Summary

To summarise, the advantages of using Docker with Overcast for integration testing are:

  1. Minimal setup. Only a docker capable host is required to run the tests.
  2. Save time. Minimal amount of configuration and infrastructure setup required to run the integration tests thanks to the docker community.
  3. Isolation. All test can run in their isolated environment so the tests will not affect each other.
  4. Flexibility. Use multiple overcast configuration and parameterized tests for testing against multiple versions.
  5. Speed. The docker container starts up very quickly, and overcast and testng allow you to even parallelize the testing by running multiple containers at once.

The example code for our integration test project is available here. You can use Boot2Docker to setup a docker host on Mac or Windows.

Happy testing!

Paul van der Ende 

Note: Due to a bug in the gradle parallel test runner you might run into this random failure when you run the example test code yourself. The work around is to disable parallelism or use a different test runner like IntelliJ or maven.

 

Xebia KnowledgeCast Episode 5: Madhur Kathuria and Scrum Day Europe 2014

Mon, 10/13/2014 - 10:48

xebia_xkc_podcast
The Xebia KnowledgeCast is a bi-weekly podcast about software architecture, software development, lean/agile, continuous delivery, and big data. Also, we'll have some fun with stickies!

In this 5th episode, we share key insights of Madhur Kathuria, Xebia India’s Director of Agile Consulting and Transformation, as well as some impressions of our Knowledge Exchange and Scrum Day Europe 2014. And of course, Serge Beaumont will have Fun With Stickies!

First, Madhur Kathuria shares his vision on Agile and we interview Guido Schoonheim at Scrum Day Europe 2014.

In this episode's Fun With Stickies Serge Beaumont talks about wide versus deep retrospectives.

Then, we interview Martin Olesen and Patricia Kong at Scrum Day Europe 2014.

Want to subscribe to the Xebia KnowledgeCast? Subscribe via iTunes, or use our direct rss feed.

Your feedback is appreciated. Please leave your comments in the shownotes. Better yet, send in a voice message so we can put you ON the show!

Credits

New daily stand up questions

Fri, 10/10/2014 - 15:51

This post provides some alternate standup questions to let your standup be: aimed forward, goal focused, team focused.

The questions are:

  1. What have I achieved since our last SUM?
  2. What is my goal for today?
  3. What things keep me from reaching my goal?
  4. What is our team goal for the end of our sprint day?

The daily stand up runs on a few standard questions. The traditional questions are:

  • What did I accomplish yesterday?
  • What will I be doing today?
  • What obstacles are impeding my progress?

A couple of effects I see when using the above list are:

  • A lot of emphasis is placed on past activities rather than getting the most out of the day at hand.
  • Team members tell what they will be busy with, but not what they aim to complete.
  • Impediments are not related to daily goals.
  • There is no summary for the team relating to the sprint goal.

If you are experiencing the same issues you could try the alternate questions. They worked for me, but any feedback is appreciated of course. Are you using other questions? Let me know your experience. You could use the PDF below to print out the questions for your scrum board.

STAND_EN

STAND_EN

 

The LGPL on Android

Fri, 10/10/2014 - 08:11

My client had me code review an Android app built for them by a third party. As part of my review, I checked the licensing terms of the open source libraries that it used. Most were using Apache 2.0 without a NOTICE file. One was using the GNU Lesser General Public License (LGPL).

My client has commercial reasons to avoid Copyleft-style licenses and so I flagged the library as unusable. The supplier understandably was not thrilled about the rework that implied and asked for an explanation and ideally some way to make it work within the license. Looking into it in more detail, I'm convinced that if you share my client's concerns, then there is no way to use LGPL licensed code on Android. Here's why I believe this to be the case.

The GNU LGPL

When I first encountered the LGPL years ago, it was explained to me as ‚Äúthe GPL, without the requirement to publish your source code‚ÄĚ. The actual license terms turn out to be a bit more restrictive. The LGPL is an add-on to the full GPL that weakens (only) the restrictions to how you license and distribute your work. These weaker restrictions are in section 4.

Here's how I read that section:

You may convey a Combined Work under terms of your choice that […] if you also
do each of the following:
  a) [full attribution]
  b) [include a copy of the license]
  c) [if you display any copyright notices, you must mention the licensed Library]
  d) Do one of the following:
    0) [provide means for the user to rebuild or re-link your application against
       a modified version of the Library]
    1) [use runtime linking against a copy already present on the system, and allow
       the user to replace that copy]
  e) [provide clear instructions how to rebuild or re-link your application in light
     of the previous point]

The LGPL on Android

An Android app can use two kinds of libraries: Java libraries and native libraries. Both run into the same problem with the LGPL.

The APK file format for Android apps is a single, digitally signed package. It contains native libraries directly, while Java libraries are packaged along with your own bytecode into the dex file. Android has no means of installing shared libraries into the system outside of your APK, ruling out out (d)(1) as an option. That leaves (d)(0). Making the library replaceable is not the issue. It may not be the simplest thing, but I'm sure there is some way to make it work for both kinds of libraries.

That leaves the digital signature, and here's where it breaks down. Any user who replaces the LGPL licensed library in your app will have to digitally sign their modified APK file. You can't publish your code signing key, so they have to sign with a different key. This breaks signature compatibility, which breaks updates and custom permissions and makes shared preferences and expansion files inaccessible. It can therefore be argued that such an APK file is not usable in lieu of the original app, thus violating the license.

In short

The GNU Lesser General Public License ensures that a user has freedom to modify a so licensed library used by your application, even if your application is itself closed source. Android's app packaging and signature requirements are such that I believe it is impossible to comply with the license when using an LGPL licensed library in a closed source Android app.

Function references in Swift and retain cycles

Thu, 10/09/2014 - 14:49

The Swift programming language comes with some nice features. One of those features are closures, which are similar to blocks in objective-c. As mentioned in the Apple guides, functions are special types of closures and they too can be passed around to other functions and set as property values. In this post I will go through some sample uses and especially explain the dangers of retain cycles that you can quickly run into when retaining function pointers.

Let's first have a look at a fairly simple objective-c sample before we write something similar in Swift.

Objective-c

We will create a button that executes a block statement when tapped.

In the header file we define a property for the block:

@interface BlockButton : UIButton

@property (nonatomic, strong) void (^action)();

@end

Keep in mind that this is a strong reference and the block and references in the block will be retained.

And then the implementation will execute the block when tapped:

#import "BlockButton.h"

@implementation BlockButton

-(void)setAction:(void (^)())action
{
    _action = action;
    [self addTarget:self action:@selector(performAction) forControlEvents:UIControlEventTouchUpInside];
}

-(void)performAction {
    self.action();
}

@end

We can now use this button in one of our view controllers as following:

self.button.action = ^{
    NSLog(@"Button Tapped");
};

We will now see the message "Button Tapped" logged to the console each time we tap the button. And since we don't reference self within our block, we won't get into trouble with retain cycles.

In many cases however it's likely that you will reference self because you might want to call a function that you also need to call from other places. Let's look as such an example:

-(void)viewDidLoad {
    self.button.action = ^{
        [self buttonTapped];
    };
}

-(void)buttonTapped {
    NSLog(@"Button Tapped");
}

Because our view controller (or it's view) retains our button, and the button retains the block, we're creating a retain cycle here because the block will create a strong reference to self. That means that our view controller will never be deallocated and we'll have a memory leak.

This can easily be solved by using a weak reference to self:

__weak typeof(self) weakSelf = self;
self.button.action = ^{
    [weakSelf buttonTapped];
};

Nothing new so far, so let's continue with creating something similar in Swift.

Swift

In Swift we can create a similar Button that executes a closure instead of a block:

class ClosureButton: UIButton {

    var action: (() -> ())? {
        didSet {
            addTarget(self, action: "callClosure", forControlEvents: .TouchUpInside)
        }
    }

    func callClosure() {
        if let action = action {
            action()
        }
    }
}

It doing the same as the objective-c version (and in fact you could use it from objective-c with the same block as before). We can assign it an action from our view controller as following:

button.action = {
    println("Button Tapped")
}

Since this closure doesn't capture self, we won't be running into problems with retain cycles here.

As mentioned earlier, functions are just a special type of closures. Which is pretty nice, because it lets us reference functions immediately like this:

override func viewDidLoad() {
    button.action = buttonTapped
}

func buttonTapped() {
    println("Button Tapped")
}

Nice and easy syntax and good for functional programming. If only it wouldn't give us problems. Without it being immediately obvious, the above sample does create a retain cycle. Why? We're not referencing self anywhere? Or are we? The problem is that the buttonTapped function is part of our view controller instance. So when the button.action references to that function, it creates a strong reference to the view controller as well. In this case we could fix it by making buttonTapped a class function. But since in most cases you'll want to do something with self in such a function, for example accessing variables, this is not an option.

The only thing we can do to fix this is to make sure that the button won't get a strong reference to the view controller. Just like in our last objective-c sample, we need to create a weak reference to self. Unfortunately there is no easy way to simply get a weak reference to our function. So we need a work around here.

Work around 1: wrapping in closure

We can create a weak reference by wrapping the function in a closure:

button.action = { [weak self] in
    self!.buttonTapped()
}

Here we first create a weak reference of self. And in Swift, weak references are always optional. That means self within this closure is now an optional and need to unwrap it first, which is what the exclamation mark is for. Since we know this code cannot be called when self is deallocated we can safely use the ! instead of ?.

A lot less elegant than immediately referencing our function immediately.

In theory, using an unowned reference to self should also work as following:

button.action = { [unowned self] in
    self.buttonTapped()
}

Unfortunately (for reasons unknown to me) this crashes with a EXC_BAD_ACCESS upon deallocation of the ClosureButton. Probably a bug.

Work around 2: method pointer function

Thanks to a question on StackOverflow about this same problem and an answer provided by Rob Napier, there is a way to make the code a bit more elegant again. We can define a function that does the wrapping in a closure for us:

func methodPointer<T: AnyObject>(obj: T, method: (T) -> () -> Void) -> (() -> Void) {
    return { [weak obj] in
        method(obj!)()
    }
}

Now we can get a weak reference to our function a bit easier.

button.action = methodPointer(self, ViewController.buttonTapped)

The reason this works is because you can get a reference to any instance function by calling it as a class function with the instance (in this case self) as argument. For example, the following all does the same thing:

// normal call
self.buttonTapped()

// get reference through class
let myFunction = MyViewController.buttonTapped(self)
myFunction()

// directly through class
MyViewController.buttonTapped(self)()

However, the downside of this is that it only works with functions that take no arguments and return Void. i.e. methods with a () -> () signature, like our buttonTapped.

For each signature we would have to create a separate function. For example for a function that takes a String parameter and returns an Int:

func methodPointer<T: AnyObject>(obj: T, method: (T) -> (String) -> Int) -> ((String) -> Int) {
    return { [weak obj] string in
        method(obj!)(string)
    }
}

We can then use it the same way:

func someFunction() {
    let myFunction = methodPointer(self, MyViewController.stringToInt)
    let myInt = myFunction("123")
}

func stringToInt(string: String) -> Int {
    return string.toInt()
}
Retain cycles within a single class instance

Retain cycles do not only happen when strong references are made between two instances of a class. It's also possible, and probably less obvious, to create a strong reference within the same instance. Let look an an example:

var print: ((String) -> ())?

override func viewDidLoad() {
    print = printToConsole
}

func printToConsole(message: String) {
    println(message)
}

Here we do pretty much the same as in our button examples. We define an optional closure variable and then assign a function reference to it. This creates a strong reference from the print variable to self and thus creating a retain cycle. We need to solve it by using the same tricks we used earlier.

Another example is when we define a lazy variable. Since lazy variables are assigned after initialisation, they are allowed to reference self directly. That means we can set them to a function reference as following:

lazy var print: ((String) -> ()) = self.printToConsole

Of course this also creates a retain cycle.

Conclusion

To avoid creating retain cycles in Swift you should always remember that a reference to an instance function means that you're referencing the instance as well. And thus when assigning to a variable, you're creating a strong reference. Always make sure to wrap such references in a closure with a weak reference to the instance or make sure to manually set the variables to nil once you're done with them.

Unfortunately Swift does not support weak closure variables, which is something that would solve the problem. Hopefully they will support it in the future or come up with a way to create a weak reference to a function much like we can use [weak self] now in closures.

Why 'Why' Is Everything

Mon, 10/06/2014 - 20:46

The 'Why' part is perhaps the most important aspect of a user story. This links to the sprint goal which links ultimately to the product vision and organisation's vision.

Lately, I got reminded of the very truth of this statement. My youngest son is part of a soccer team and they have training every week. Part of the training are exercises that use a so-called speedladder.

foto-1

After the training while driving home I asked him what he especially liked about the training and what he wants to do differently next time. This time he answered that he didn't like the training at all. So I asked him what part he disliked: "The speedladder. It is such a stupid thing to do.". Although I realised it to be a poor mans answer I told him that some parts are not that nice and he needs to accept that: practising is not always fun. I wasn't happy with the answer but couldn't think of a better one.

Some time passed when I overheard the trainers explaining to each other that the speedladder is for improving the 'footwork', coordination, and sensory development. Then I got an idea!
I knew that his ambition is to become as good as Messi :-) so when at home I explained this to my son and that it helps him to improve feints and unparalleled actions. I noticed his twinkling eyes and he enthusiastically replied: "Dad, can we buy a speedladder so I can practise at home?".  Of course I did buy one! Since then the 'speedladder' is the most favourable part of the soccer training!

Summary

The goal, purpose and the 'Why' is the most important thing for persons and teams. Communicating this clearly to the team is one of the most important things a product owner and organisation need to do in order to get high performant teams.

How to create a Value Stream Map

Mon, 10/06/2014 - 08:05

Value Stream Mapping (VSM) is a -very- useful tool to gain insight in the workflow of a process and can be used to identify both Value Adding Activities and Non Value Adding Activities in a process stream while providing handles for optimizing the process chain. The results of a VSM can be used for many occasions: from writing out a business case, to defining a prioritized list to optimize processes within your organization, to pinpointing bottlenecks in your existing processes and gain a common understanding of process related issues.

When creating a VSM of your current software delivery process you quite possibly will be amazed by the amount of waste and therefor the room for improvement you might find. I challenge you to try this out within your own organization. It will leave you with a very powerful tool to explain to your management the steps that need to change, as it will leave you with facts.

To quickly get you started, I wrote out some handles on how to write out a proper Value Stream Map.

In many organizations there is the tendency to ‚Äėsolely‚Äô perform local optimizations to steps in the process (i.e. per Business Unit), while in reality the largest process optimizations can be gained by optimizing the area‚Äôs which are in between the process steps and do not add any value to the customer at all; the Non Value Adding activities. Value Stream Mapping is a great tool for optimizing the complete process chain, not just the local steps.

Local_vs_complete

The Example - Mapping a Software Delivery Process
Many example value streams found on the internet focus on selling a mortgage, packaging objects in a factory or some logistic process. The example I will be using focuses on a typical Software Delivery Processes as we still see them today: the 'traditional' Software Delivery Process containing many manual steps.

You first need to map the 'as-is' process as you need this to form the baseline. This baseline provides you the required insight to remove steps from the process that do not add any value to your customer and therefor can be seen as pure waste to your organization.

It is important to write out the Value Stream as a group process (a workshop), where group-members represent people that are part of the value chain as it is today*. This is the only way to spot (hidden) activities and will provide a common understanding of the situation today. Apart from that, failure to execute the Value Stream Mapping activity as a group process will very likely reduce the acceptance rate at the end of the day. Never write out a VSM in isolation.

Value Stream mapping is 'a paper and pencil tool’ where you should ask participants to write out the stickies and help you form the map. You yourself will not write on stickies (okay, okay, maybe sometimes … but be careful not to do the work for the group). Writing out a process should take you about 4 to 6 hours, including discussions and the coffee breaks of course. So, now for the steps!

* Note that the example value stream is a simplified and fictional process based on the experience at several customers.

Step 0 Prepare
Make sure you have all materials available.

Here is a list:
- two 4 meter strokes of brown paper.
- Plastic tape to attach paper to the wall
- stickies square multiple colors
- stickies rectangle multiple colors
- small stickies one or two colors
- lot’s of sharpies (people need to be able to pick up the pens)
- colored ‚Äėdot' stickies.

What do you need? (the helpful colleague not depicted)

What do you need? (the helpful colleague not depicted)

Step 1 & 2 define objectives and process steps
Make sure to work one process at a time and start off with defining customer objectives (the Voice Of Customer). A common understanding of the VoC is important because in later stage you will determine with the team which activities are really adding to this VoC and which steps are not. Quite often these objectives are defined in Time, Cost and Quality. For our example, let’s say the customer would like to be able to deliver a new feature every hour, with a max cost of $1000 a feature and with zero defects.

First, write down the Voice of the Customer in the top right corner. Now, together with the group, determine all the actors (organizations / persons / teams) that are part of the current process and glue these actors as orange stickies to the brown paper.

Defining Voice of Customer and Process Steps

Defining Voice of Customer and Process Steps

Step 3 Define activities performed within each process step
With the group, per determine the activities that take place. Underneath the orange stickies, add green stickies that describe the activities that take place in a given step.

Defining activities performed in each step

Defining activities performed in each step

Step 4 Define Work in Progress (WiP)
Now, add pink stickies in between the steps, describing the number of features / requirements / objects / activities that is currently in process in between actors. This is referred to as WiP - Work in Progress. Whenever there is a high WiP limit in between steps, you have identified a bottleneck causing the process 'flow' to stop.

On top of the pink WiP stickies containing particular high WiP levels, add a small sticky indicating what the group thinks is causing the high WiP. For instance, a document has to be distributed via internal mail, or a wait is introduced for a bi-weekly meeting or travel to another location is required. This information can later be used to optimize the process.

Note that in the workshop you should also take some time to finding WiP within the activities itself (this is not depicted in this example). Spend time on finding information for causes of high WiP and add this as stickies to each activity.

Define work in process

Define work in process

Step 5 Identify rework
Rework is waste. Still, many times you'll see that a deliverable is to be returned to a previous step for reprocessing. Together with the group, determine where this happens and what is the cause of this rework. A nice additional is to also write out first-time-right levels.

Identify rework

Identify rework

Step 6 Add additional information
Spend some time in adding additional comments for activities on the green stickies. Some activities might for instance not be optimized, are not easy to handle or from a group perspective considered obsolete. Mark these comments with blue stickies next to the activity at hand.

Add additional information

Add additional information

Step 7 Add Process time, Wait time and Lead time and determining Process Cycle Efficiency

Now, as we have the process more or less complete, we can start adding information related to timing. In this step you would like to determine the following information:

  • process time: the real amount of time that is required to perform a task without interruptions
  • lead time: the actual time that it takes for the activity to be completed (also known as elapse time)
  • wait time: time when no processing is done at all, for example when for waiting on a 'event' like a bi-weekly meeting.

(Not in picture): for every activity on the green sticky, write down a small sticky with two numbers vertically aligned. The top-number reflects the process-time, (i.e. 40 hours). The bottom-number reflects the lead time (i.e. 120 hours).

(In picture): add a block diagram underneath the process, where timing information in the upper section represents total processing time for all activities and timing information the lower section represents total lead time for all activities. (just add up the timing information for the individual activities I described in previous paragraph). Also add noticeable wait time in-between process steps. As a final step, to the right of this block diagram, add the totals.

Now that you have all information on the paper, the following  can be calculated:

  • Total Process Time - The total time required to actually work on activities if one could focus on the activity at hand.
  • Total Lead Time - The total time this process actually needs.
  • Project Cycle Efficiency (PCE): -> Total Process Time / Total Lead Time *100%.

Add this information to the lower right corner of your brown paper. The numbers for this example are:

Total Process Time: add all numbers in top section of stickies: 424 hours
Process Lead Time (PLT): add all numbers in lower section of stickies + wait time in between steps: 1740 hours
Project Cycle Efficiency (PCE) now is: -> Total Process  Time / Total Process Lead Time: 24%.
Note that 24% is -very- high which is caused by using an example. Usually you’ll see a PCE at about 4 - 8% for a traditional process.

Add process, wait and lead times

Add process, wait and lead times

Step 8 Identify Customer Value Add and Non Value Add activities
Now, categorize tasks into 2 types: tasks that add value to the customer (Customer Value Add, CVA) and tasks that do not add value to the customer (Non Value Add, NVA). The NVA you can again split into two categories: tasks that add Business Value (Business Value Add, BVA) and ‚ÄėWaste‚Äô. When optimizing a process, waste is to be eliminated completely as it does not add value to the customer nor the business as a whole. But also for the activities categorized as 'BVA', you have to ask yourself whether these activities add to the chain.

Mark CVA tasks with a green dot, BVA tasks with a blue dot and Waste with a red dot. Put the legend on the map for later reference.

When identifying CVA, NVA and BVA … force yourself to refer back to the Voice of Customer you jotted down in step 1 and think about who is your customer here. In this example, the customer is not the end user using the system, but the business. And it was the business that wanted Faster, Cheaper & Better. Now when you start to tag each individual task, give yourself some time in figuring out which tasks actually add to these goals.

Determine Customer Value Add & Non Value Add

Determine Customer Value Add & Non Value Add

To give you some guidance on how you can approach tagging each task, I’ll elaborate a bit on how I tagged the activities. Note again, this is just an example, within the workshop your team might tag differently.

Items I tagged as CVA: coding, testing (unit, static, acceptance), execution of tests and configuration of monitoring are adding value to the customer (business). Why? Because all these items relate to a faster, better (high quality through test + monitoring) and cheaper (less errors through higher quality of code) delivery of code.

Items I tagged as BVA: documentation, configuration of environments, deployments of VMs, installation of MW are required to be able to deliver to the customer when using this (typical waterfall) Software Delivery Processes. (Note: I do not necessarily concur with this process.) :)

Items I tagged as pure Waste, not adding any value to the customer: items like getting approval, the process to get funding (although probably required), discussing details and documenting results for later reference or waiting for the Quarterly release cycle. Non of these items are required to either deliver faster, cheaper or better so in that respect these items can be considered waste.

That's it (and step 9) - you've mapped your current process
So, that’s about it! The Value Stream Map is now more or less complete an contains all relevant information required to optimize the process in a next step. Step 9 here would be: Take some time to write out items/bottlenecks that are most important or easy to address and discuss internally with your team about a solution. Focus on items that you either tagged as BVA or pure waste and think of alternatives to eliminate these steps. Put your customer central, not your process! Just dropping an activity as a whole seems somewhat radical, but sometimes good ideas just are! Note by the way that when addressing a bottleneck, another bottleneck will pop up. There always will be a bottleneck somewhere in the process and therefor process optimization must be seen as a continuous process.

A final tip: to be able to perform a Value Stream Mapping workshop at the customer, it might be a good idea to join a more experienced colleague first, just to get a grasp of what the dynamics in such a workshop are like. The fact that all participants are at the same table, outlining the delivery process together and talk about it, will allow you to come up with an optimized process on which each person will buy in. But still, it takes some effort to get the workshop going. Take your time, do not rush it.

For now, I hope you can use the steps above the identify the current largest bottlenecks within your own organization and get going. In a next blog, if there is sufficient interest, I will write about what would be possible solutions in solving the bottlenecks in my example. If you have any ideas, just drop a line below so we can discuss! The aim for me would be to work towards a solution that caters for Continuous Delivery of Software.

Michiel Sens.

Integrating Geb with FitNesse using the Groovy ConfigSlurper

Fri, 10/03/2014 - 18:01

We've been playing around with Geb for a while now and writing tests using WebDriver and Groovy has been a delight! Geb integrates well with JUnit, TestNG, Spock, and Cucumber. All there is left to do is integrating it with FitNesse ... or not :-).

Setup Gradle and Dependencies

First we start with grabbing the gradle fitnesse classpath builder from Arjan Molenaar.
Add the following dependencies to the gradle build file:

compile 'org.codehaus.groovy:groovy-all:2.3.7'
compile 'org.gebish:geb-core:0.9.3'
compile 'org.seleniumhq.selenium:selenium-java:2.43.1
Configure different drivers with the ConfigSlurper

Geb provides a configuration mechanism using the Groovy ConfigSlurper. It's perfect for environment sensitive configuration. Geb uses the geb.env system property to determine the environment to use. So we use the ConfigSlurper to configure different drivers.

import org.openqa.selenium.chrome.ChromeDriver
import org.openqa.selenium.firefox.FirefoxDriver

driver = { new FirefoxDriver() }

environments {
  chrome {
    driver = { new ChromeDriver() }
  }
}
FitNesse using the ConfigSlurper

We need to tweak the gradle build script to let FitNesse play nice with the ConfigSlurper. So we pass the geb.env system property as a JVM argument. Look for the gradle task "wiki" in the gradle build script and add the following lines.

def gebEnv = (System.getProperty("geb.env")) ? (System.getProperty("geb.env")) : "firefox"
jvmArgs "-Dgeb.env=${gebEnv}"

Since FitNesse spins up a separate 'service' process when you execute a test, we need to pass the geb.env system property into the COMMAND_PATTERN of FitNesse. That service needs the geb.env system property to let Geb know which environment to use. Put the following lines in the FitNesse page.

!define COMMAND_PATTERN {java -Dgeb.env=${geb.env} -cp %p %m}

Now you can control the Geb environment by specifying it on the following command line.

gradle wiki -Dgeb.env=chrome

The gradle build script will pass the geb.env system property as JVM argument when FitNesse starts up. And the COMMAND_PATTERN will pass it to the test runner service.

Want to see it in action? Sources can be found here.