Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Xebia Blog
Syndicate content
Updated: 19 hours 22 min ago

Quickly build a XL Deploy plugin for deploying container applications to CoreOS

Mon, 05/25/2015 - 21:57

You can use fleetctl and script files to deploy your container applications to CoreOS. However, using XL Deploy for deployment automation is a great solution when you need to deploy and track versions of many applications. What does it take to create a XL Deploy plugin to deploy these container applications to your CoreOS clusters?

XL Deploy can be extended with custom plugins which add deployment capabilities. Using XL Rules custom plugins can be created quickly with limited effort. In this blog you can read how a plugin can be created in a matter of hours.

In a number of blog posts, Mark van Holsteijn explained how to create a high available Docker container platform using CoreOS and Consul. In these posts shell scripts (with fleetctl commands) are used deploy container applications. Based on these scripts I have built a XL Deploy plugin which deploys fleet unit configuration files to a CoreOS cluster.


Deploying these container applications using XL Deploy has a number of advantages:

  • Docker containers can be deployed without creating, adjusting and maintaining scripts for individual¬†applications.
  • XL Deploy will track and report the applications deployed to the CoreOS clusters.
  • Additional deployment scenarios can be added with limited effort.
  • Deployments will be consistent and configuration is managed across environments.
  • XL Deploy permissions can be used to control (direct) access to the CoreOS cluster(s).


Building an XL Deploy plugin is fast, since you can:

  • Reuse existing XL Deploy capabilities, like the Overthere plugin.
  • Utilize XL Deploy template processing to inject property values in rules and deploy scripts.
  • Exploit the XL Deploy unified deployment model to get the deltas which drive the required fleetctl deployment commands for any type of deployment (new, update, undeploy and rollback deployments).
  • Use xml and script based rules to build deployment tasks.

Getting started

  • Install XL Deploy, you can download a free edition¬†here.¬†If you are not familiar with XL Deploy, read the¬†getting started¬†documentation.
  • Next add the plugin resources to the ext directory of your XL Deploy installation. You can find the plugin resources in¬†this Github repository. Add the synthetic.xml, xl-rules.xml file from the repository root. In addition, add the scripts directory and its contents. Restart XL Deploy.
  • Next, setup a¬†CoreOS cluster. This¬†blog post¬†explains how you can setup such a platform locally.
  • Now you can connect to XL deploy using your browser. On the deployment¬†tab¬†you can import the sample application, located in the¬†sample-app folder¬†of the¬†plugin resources Github repository
  • You can now setup the target deployment container based on the Overthere.SshHost configuration item type. Verfiy that you can connect to your CoreOS cluster using this XL Deploy container.
  • Next, you can setup a XL Deploy environment, which contains your target deployment container.
  • Now you can use the¬†deployment tab to deploy and undeploy your fleet configuration file applications.


Building the plugin

The plugin consists of two xml files and a number of script files. Below you find a description of the plugin implementation steps.

The CoreOS container application deployments are based on fleet unit configuration files. So, first we create a XL Deploy configuration Item type definition which represents such a file. This XL Deploy deployed type is defined in the XL Deploy synthetic.xml file. The snippet below shows the contents of this file. I have assigned the name ‚Äúfleet.DeployedUnit‚ÄĚ.


The definition contains  a container-type attribute. The Overthere.SshHost container is referenced. The plugin can simple use the Overthere.SshHost container type to connect to the CoreOS cluster and to execute fleet commands.

Furthermore, I have added two properties. One property which can be used to specify the number of instances. Note that XL Deploy dictionaries can be utilized to define the number of instances for each environment separately. The second property is a flag which controls whether instances will be started (or only submitted and loaded).

If you want to deploy a fleet configuration file using fleetctl, you can issue the following three commands: submit, load and start. In the plugin, I have created a separate script file for each of these fleetctl commands.¬† The caption below shows the script file to load a fleet configuration file. This load script uses the file name property and numberOfInstances property of the ‚Äúfleet.DeployedUnit‚ÄĚ configuration item.


Finally, the plugin can be completed with XML-based rules which create the deployment steps. The caption below shows the rule which adds steps to [1] submit the unit configuration and [2] load the unit when (a version of ) the application is deployed.


Using rules, you can easily define logic to add deployment steps. These steps can closely resemble the commands you perform when you are using fleetctl. For this plugin I have utized xml-based rules only. Using script rules, you can add more intelligence to your plugin. For example, the logic of the restart script can be converted to rules and more fine grained deployment steps.

More information

If you are interested in building your own XL Deploy plugin, the XL Deploy product documentation contains tutorials which will get you started.

If you want to know how you can create a High Available Docker Container Platform Using CoreOS And Consul, the following blogs are great starting point:

Microservices architecture principle #2:Autonomy over coordination

Mon, 05/25/2015 - 16:13

Microservices are a hot topic. Because of that a lot of people are saying a lot of things. To help organizations make the best of this new architectural style Xebia has defined a set of principles that we feel should be applied when implementing a Microservice Architecture. Over the next couple of days we will cover each of these principles in more detail in a series of blog posts.
This blog explains why we prefer autonomy of services over coordination between services.

Our Xebia colleague Serge Beaumont posted "Autonomy over Coordination" in a tweet earlier this year and for me it summarised one of the crucial aspects for creating an agile, scalable and robust IT system or organisational structure. Autonomy over coordination is closely related to the business capabilities described in the previous post in this series, each capability should be implemented in one microservice. Once you have defined your business capabilities correctly the dependencies between those capabilities are minimised. Therefore minimal coordination between capabilities is required, leading to optimal autonomy. Increased autonomy for a microservice gives it freedom to evolve without impacting other services: the optimal technology can be used, it can scale without having to scale others, etc. For the team responsible for the service the advantages are similar, the autonomy enables them to make optimal choices that make their team function at its best.

The drawbacks of less autonomy and more coordination are evident and we all have experienced these. For example, a change leads to a snowball of dependent changes that must be deployed at the same moment, making changes to a module requires approval of other teams,  not being able to scale up a compute intensive function without scaling the whole system, ... the list is endless.

So in summary, pay attention to defining you business capabilities (microservices) in such a manner that autonomy is maximised, it will give you both organisational and technical advantages.

Microservices architecture principle #1: Each Microservice delivers a single complete business capability

Sat, 05/23/2015 - 21:13

Microservices are a hot topic. Because of that a lot of people are saying a lot of things. To help organizations make the best of this new architectural style Xebia has defined a set of principles that we feel should be applied when implementing a Microservice Architecture. Over the next couple of days we will cover each of these principles in more detail in a series of blog posts.
This blog explains why a Microservice should deliver a complete business capability.

A complete business capability is a process that can be finished consecutively without interruptions or excursions to other services. This means that a business capability should not depend on other services to complete its work.
If a process in a microservice depends on other microservices we would end up in the dependency hell ESBs introduced: in order to service a customer request we need many other services and therefore if one of them fails everything stops. A more robust solution would be to define a service that handles a process that makes sense to a user. Examples include ordering a book in a web shop. This process would start with the selection of a book and end with creating an order. Actually fulfilling the order is a different process that lives in its own service. The fulfillment process might run right after the order process but it doesn’t have to. If the customer orders a PDF version of a book order fulfillment may be completed right away. If the order was for the print version, all the order service can promise is to ask shipping to send the book. Separating these two processes in different services allows us to make choices about the way the process is completed, making sure that a problem or delay in one service has no impact on other services.

So, building a microservice such that it does a single thing well without interruptions or waiting time is at the foundation of a robust architecture.

Agile goes beyond Epic Levels

Fri, 05/15/2015 - 17:10

IMG_5514A snapshot from my personal backlog last week:

  • The Agile transformation¬†at ING¬†was¬†frontpage news¬†in the Netherlands. This made us even more realize how epic this transformation and assignment actually is.
  • The Agile-built hydrogen race¬†car from the TU Delft set¬†an official track record¬†on the Nurburgring. We're proud on our guys in Delft!
  • Hanging out with Boeings‚Äô¬†Agile champs at their facilities in Seattle exchanging knowledge.¬†Impressive and extremely fruitful!
  • Coaching¬†the State of Washington on their ground breaking Agile initiatives¬†together with my friend and fellow consultant from ScrumInc,¬†Joe Justice.

One thing became clear for me after a week like this: Something Agile is cookin’.  And it’s BIG!

In this blog I will be explaining why and how Agile will develop in the near future.

Introduction; what’s happening?

Human kind¬†is currently facing the¬†biggest era change¬†since the¬†19th Century. Our industries, education, technologies¬†and¬†society¬†are simply¬†not compliant anymore with today‚Äôs and tomorrows needs. ¬†Some systems like healthcare and the economy are that broken they actually should be to be reinvented again. Everything has just become too vulnerable and complex. Just Quantitative-easing‚Äúlubricating the engine‚ÄĚ like quantitive easing the economy, are no sustainable solutions anymore.¬†Like¬†Russell Ackoff¬†already said, you can only fix a system as a¬†whole not only by separate parts.

This reinvention will only succeed when we are able to learn and adjust our systems very rapidly.  Agile, Lean and a different way of organizing our selfs, can make this reality.  Lean will provide us with the right tools to do exactly what’s needed, nothing more, nothing less.  But applying Lean for only efficiency purposes will not bring the innovations and creativity we need.  We also need an additional catalyst and engine: Agile. It will provide us with the right mindset and tools to innovate, inspect and adapt very fast.  And finally, we must reorganize ourself more on cooperation not on directive command and control. This was useful in the industrial revolution, not in our modern complex times.

Agile’s for everything

Unlike most people think, Agile is not only a software development tool. You can apply it to almost everything. slider_Forze_VI_ValkenburgFor example, as Xebia consultants we've successfully coached Agile and Lean non-IT initiatives in Marketing, Innovation, Education, Automotive, Aviation and non-Profit organizations. It just simply works! And how. A productivity increase of 500% is no exception. But above all, team members and customers are much happier.

Agile's for everybody

At this moment, a lot of people are still unconsciously addicted to their patterns and unaware about the unlimited possibilities out there.  It’s like having a walk in the forrest.  You can bring your own lunch like you always do, but there are some delicious fruits out there for free!  Technology like 3D printing offer unlimited possibilities straight on your desk, where only a few years a go, you needed a complicated, million dollar machine for this. The same goes for Agile. It’s open source and out there waiting for you. It will also help you getting more out of all these new awesome developments!

The maturity of Agile explained

Until recently, most agile initiatives emerged bottom up, but stalled on a misfit between Agile and (conventional) organizations.  Loads of software was produced, but could not be brought to production, simply because the whole development chain was not Agile yet. Tools like TDD and Continuous Integration improved the situation significantly, but dependencies were still not managed properly most of the time.

Screen Shot 2015-05-13 at 7.53.15 PM

The last couple of years, some good scaled agile frameworks like LeSS and SAFe emerged. Managing the dependencies better, but not directly encouraging the Agile Mindset and motivation of people.  In parallel, departments like HR, Control and Finance were struggling with Agile. There was a scaled agile framework implemented, but the hierarchical organization structure was not adjusted creating a gap between fast moving Agile teams and departments still hindered by non-Agile procedures, proceses and systems.

Therefor, we see conventional organizations moving towards a more Agile, community based model like Spotify, Google or Zappos.  ING is now transforming towards to a similar organization model.

Predictions for the near future

My expectation is that we will see Agile transformations continue on a much wider scale.  For example, people developing their own products in an agile fashion while using 3D printing.  Governments will use Agile and Holacracy for solving issues like the reshaping the economic system together with society. Or like I have observed last week, the State of Washington government using these techniques successfully in solving the challenges they're facing.

For me, it currently feels like the early Nineties again when the Internet emerged.  In that time I explained to many people the Internet would be like electricity for them in the near future.  Most people laughed and stated it was just a new way of communication.  The same applies now for the Agile Mindset.   It's not just a hype or a corporate tool. It will reshape the world as we know it today.





How to improve Scrum team performance with Kanban

Mon, 05/11/2015 - 21:22
This blogpost has been a collaborative effort between Jeroen Willemsen and Jasper Sonnevelt

In this post we will take a look at a real life example of a Scrum team transitioning to Kanban. We will address a few pitfalls and discuss how to circumvent those. With this we provide additional insights to advance your agile way of working.

The example

At the time we joined this project, three Scrum-teams worked together to develop a large software product. The teams shared a Product Owner (PO) and backlog and just had their MVP released. Now, bug-reports started coming in and the Scrum-teams faced several problems:

  • The developers were making a lot of progress per sprint, so the PO had to put a lot of effort in the backlog to create a sufficiently sized work inventory.New bugs came to light as the product started to become adopted. Filing and fixing these bugs and monitoring the progress was taking a lot of effort from the PO. Solving each bug in the next sprint¬Ě reduced the team's response time.
  • Prioritizing the bug-fixing and the newly requested features became hard as the PO had to many stakeholders to manage.
  • The Product Owner saw Kanban as a possible solution to these problems. He convinced the team to implement it in an effort to deal with these problems and to provide a better way of working.

The practices that were implemented included:

  • No more sprint replenishment rhythm (planning & backlog refinement);
  • A Work in Progress (WIP) limit.

As a result the backlog quality started to deprecate. Less information was written down about a story and developers did not take the time to ask the right questions towards the PO. The WIP limit maximised the amount of work the team could take on. This made them focus on the current, sometimes unclear and too complex stories. Because of this developers would keep working on their current tasks, making assumptions along the way. These assumptions could sometimes lead to conflicting insights as developers would collaborate on stories, while at the same time being influenced by different stakeholders. This resulted in misalignment.

All of these actions resulted in a lower burn down rate, less predictability and therefore nervous stakeholders. And after a few weeks, the PO decided to stop the migration to Kanban, and go back to Scrum.

However, the latter might not have been necessary for the team to be successful. After all, it is about how you implement Kanban. There were a few key elements missing for instance. In our experience, teams that are successful (would) have also implemented:

  • Forecasting based on historical data and simulations: Scrum teams use Planning Poker and Velocity to make predictions about the amount of work they will be able to do in a sprint. When sprint cadence is let go this will become more difficult. Practices in the Kanban Method tell us that by using historical data in simulations, the same and probably more predictability can be achieved.
  • Policies for prioritizing, WIP monitoring and handover criteria: Kanban demands a very high amount of discipline from all participants. Policies will help here. In the case of this team, it would have benefited greatly from having clear defined policies about priorities and prioritization. For instance: Who can (and is allowed to) prioritize? How can the team see what has most priority? Are there different classes of service we can identify? And how do we treat those? The same holds for WIP-limits and handover criteria. We always use a Definition of Ready and Definition of Done in Scrum teams. Keeping these in a Kanban team is a very good practice.
  • A feedback cycle to review data and process on a regular basis: Where Scrum demands a retrospective after every sprint, Kanban does not explicitly define such a thing. Kanban only dictates that you should implement feedback loops. When an organization starts implementing Kanban it is key that they do review the implementation, data and process at a regular basis. This is crucial during the first weeks: all participants will have to get used to the new way of working. Policies should be reviewed as well, as they might need adjustments to facilitate the team in having an optimized workflow.

The hard thing about Kanban

What most people think and see when talking about Kanban is introducing WIP limits to their board, adding columns (other than To do, In Progress, Done) and stop using sprints. And herein lies one of the problems. For organizations that are used to working with Scrum letting go of sprints is a very unnatural thing to do. It feels like letting go of predictability and regular feedback. And instead of making the organization a bit better people feel like they have just taken a step back.

The hard thing about Kanban is that it doesn't provide you with a clear cut solution to your problems like Scrum does. Kanban is a method that helps you fix the problems in your organization in a way that best fits your context. For instance: it tells you to visualize a lot of things but only provides examples of what you could visualize. Typically teams and organizations visualize their process and work(types).

To summarize:

  • Kanban uses the current process and doesn't enforce a process of it's own. Therefore it demands a higher degree of discipline, both from the team-members and the the rest of the organization. If you are not aware of this the introduction of Kanban will only lead to chaos.
  • Kanban doesn't say anything about (To Do column) replenishment frequency or demo¬Ě frequency.


  • Implement forecasting based on historical data and simulations, policies for prioritizing, WIP limits and handover criteria and a feedback cycle to review data and process on a regular basis.
  • Define clear policies about how to collaborate. This will help create transparency and predictability.
  • The Scrum ceremonies happen in a rhythm that is very easy to understand and learn for stakeholders. Keep that mind when designing Kanban policies.


Understanding the 'sender' in segues and use it to pass on data to another view controller

Fri, 05/08/2015 - 22:59

One of the downsides of using segues in storyboards is that you often still need to write code to pass on data from the source view controller to the destination view controller. The prepareForSegue(_:sender:) method is the right place to do this. Sometimes you need to manually trigger a segue by calling performSegueWithIdentifier(_:sender:), and it's there you usually know what data you need to pass on. How can we avoid adding extra state variables in our source view controller just for passing on data? A simple trick is to use the sender parameter that both methods have.

The sender parameter is normally used by storyboards to indicate the UI element that triggered the segue, for example an UIButton when pressed or an UITableViewCell that triggers the segue by selecting it. This allows you to determine what triggered the segue in prepareForSegue:sender:, and based on that (and of course the segue identifier) take some actions and configure the destination view controller, or even determine that it shouldn't perform the segue at all by returning false in shouldPerformSegueWithIdentifier(_:sender:).

When it's not possible to trigger the segue from a UI element in the Storyboard, you need to use performSegueWithIdentifier(_:sender:) instead to manually trigger it. This might happen when no direct user interaction should trigger the action of some control that was created in code. Maybe you want to execute some additional logic when pressing a button and after that perform the segue. Whatever the situation is, you can use the sender argument to your benefit. You can pass in whatever you may need in prepareForSegue(_:sender:) or shouldPerformSegueWithIdentifier(_:sender:).

Let's have a look at some examples.

Screen Shot 2015-05-08 at 23.25.37

Here we have two very simple view controllers. The first has three buttons for different colors. When tapping on any of the buttons, the name of the selected color will be put on a label and it will push the second view controller. The pushed view controller will set its background color to the color represented by the tapped button. To do that, we need to pass on a UIColor object to the target view controller.

Even though this could be handled by creating 3 distinct segues from the buttons directly to the destination view controller, we chose to handle the button tap ourselves and the trigger the segue manually.

You might come up with something like the following code to accomplish this:

class ViewController: UIViewController {

    @IBOutlet weak var label: UILabel!

    var tappedColor: UIColor?

    @IBAction func tappedRed(sender: AnyObject) {
        label.text = "Tapped Red"
        tappedColor = UIColor.redColor()
        performSegueWithIdentifier("ShowColor", sender: sender)

    @IBAction func tappedGreen(sender: AnyObject) {
        label.text = "Tapped Green"
        tappedColor = UIColor.greenColor()
        performSegueWithIdentifier("ShowColor", sender: sender)

    @IBAction func tappedBlue(sender: AnyObject) {
        label.text = "Tapped Blue"
        tappedColor = UIColor.blueColor()
        performSegueWithIdentifier("ShowColor", sender: sender)

    override func prepareForSegue(segue: UIStoryboardSegue, sender: AnyObject?) {
        if segue.identifier == "ShowColor" {
            if let colorViewController = segue.destinationViewController as? ColorViewController {
                colorViewController.color = tappedColor


class ColorViewController: UIViewController {

    var color: UIColor?

    override func viewDidLoad() {

        view.backgroundColor = color


We created a state variable called tappedColor to keep track of the color that needs to be passed on. It is set in each of the action methods before calling performSegueWithIdentifier("ShowColor", sender: sender) and then read again in prepareForSegue(_:sender:) so we can pass it on to the destination view controller.

The action methods will have the tapped UIButtons set as the sender argument, and since that's the actual element that initiated the action, it makes sense to set that as the sender when performing the segue. So that's what we do in the above code. But since we don't actually use the sender when preparing the segue, we might as well pass on the color directly instead. Here is a new version of the ViewController that does exactly that:

class ViewController: UIViewController {

    @IBOutlet weak var label: UILabel!

    @IBAction func tappedRed(sender: AnyObject) {
        label.text = "Tapped Red"
        performSegueWithIdentifier("ShowColor", sender: UIColor.redColor())

    @IBAction func tappedGreen(sender: AnyObject) {
        label.text = "Tapped Green"
        performSegueWithIdentifier("ShowColor", sender: UIColor.greenColor())

    @IBAction func tappedBlue(sender: AnyObject) {
        label.text = "Tapped Blue"
        performSegueWithIdentifier("ShowColor", sender: UIColor.blueColor())

    override func prepareForSegue(segue: UIStoryboardSegue, sender: AnyObject?) {
        if segue.identifier == "ShowColor" {
            if let colorViewController = segue.destinationViewController as? ColorViewController {
                colorViewController.color = sender as? UIColor


This allows us to get rid of our extra tappedColor variable.

It might seem to (and perhaps it does) abuse the sender parameter though, so use it with care and only where appropriate. Be aware of the consequences; if some other code or some element in a Storyboard triggers the same segue (i.e. with the same identifier), then the sender might just be an UI element instead of the object you expected, which will lead to unexpected results and perhaps even crashes when you force cast the sender to something it's not.

You can find the sample code in the form of an Xcode project on

Minimal Viable UX

Wed, 05/06/2015 - 20:38

An approach to incorporate UX into the LEAN principle.

User Experience is often interpreted as a process where the ‚ÄėUX guru‚Äô holds the ultimate truth in designing for an experience. The guru likes to keep control of his design and doesn‚Äôt want to feel less valuable when adopting advice from non-designers, where his concern is becoming a pixel pusher.

Adopting UX in a LEAN way, the feedback from team members minimizes the team going down the wrong path. This prevents the guru from perfecting a design where constraints over time will become clearer and less aligned with the customer needs. Interaction with the team speeds up development time by giving early insight.

Design for User Experience

UX has many different definitions, in the end it enables the user to perform a task with the help of an interface. All disciplines in a software development team should be aware of the user they are designing or developing for, starting in Sprint Zero. UX is not about setting up mockups, wireframes, prototypes and providing designs, it has to be part of the team culture where every team member can attribute to. We are trying to solve problems and problems are not being solved with design documentation but solved with efficient, elegant and sophisticated software.

How to get there

Create user awareness

Being aware of the user helps reduce waste and keeps you focused on things you should care about, functionality that adds value in the perception of the customer.

First, use a set of personas, put them on a wall and let your team members align those users with the functionality they are building. Developers can reflect functionality, interaction designers can optimize interface elements and visual designers can align styling with the user.

Second, use a customer journey map. This is a powerful tool. It helps in creating context, gives an overview of the user experience and helps to find gaps.

Prototype quickly

Prototyping becomes easier by the day, thanks to the amount and quality of tools out there. Prototyping can be performed by using paper, mockups (Balsamiq) or a web framework, such as FramerJS. Pick the type you prefer and which is suitable for the situation and has the appropriate depth.

Diagram of the iterative design and critique process. Warfel, Todd Zaki. 2009. Prototyping: A Practitioner’s Guide. New York: Rosenfeld Media.

Use small portions of prototypes and validate those with a minimal set of users. This helps you to deliver faster, therefore again eliminate waste and improves built-in quality. Iterative design helps you to amplify learning. KISS!


Involved parties need to be convinced that what you are saying is based on business needs, the product and the people. You need to befriend and understand all involved parties in order to make it work across the board. Besides that, don’t forget your best friends, the users.

If you don't talk to your customers, how will you know how to talk to your customers? - Will Evans

How to deploy an ElasticSearch cluster using CoreOS and Consul

Sun, 05/03/2015 - 13:39

The hot potato in the room of Containerized solutions is persistent services. Stateless applications are easy and trivial, but to deploy a persistent services like ElasticSearch is a totally different ball game. In this blog post we will show you how easy it is on this platform to create ElasticSearch clusters. The key to the easiness is the ability to lookup external ip addresses and port numbers of all cluster members in Consul and the reusable power of the CoreOS unit file templates. The presented solution is a ready-to-use ElasticSearch component for your application.

This solution:

  • uses empheral ports so that we can actually run multiple ElasticSearch nodes on the same host
  • mounts persistent storage under each node to prevent data loss on server crashes
  • uses the power of the CoreOS unit template files to deploy new ElasticSearch clusters.

In the previous blog posts we defined our A High Available Docker Container Platform using CoreOS and Consul and showed how we can add persistent storage to a Docker container. 

Once this platform is booted the only thing you need to do to deploy an ElasticSearch Cluster,  is to submit the following fleet unit system template file elasticsearch@.service  and start 3 or more instances.

Booting the platform

To see the ElasticSearch cluster in action, first boot up our CoreOS platform.

git clone
cd coreos-container-platform-as-a-service/vagrant
vagrant up
Starting an ElasticSearch cluster

Once the platform is started, submit the elasticsearch unit file and start three instances:

cd ../fleet-units/elasticsearch
fleetctl submit elasticsearch@.service
fleetctl start elasticsearch@{1..3}

Now wait until all elasticsearch instances are running by checking the unit status.

fleetctl list-units
UNIT            MACHINE             ACTIVE  SUB
elasticsearch@1.service f3337760.../    active  running
elasticsearch@2.service ed181b87.../    active  running
elasticsearch@3.service 9e37b320.../    active  running
mnt-data.mount      9e37b320.../    active  mounted
mnt-data.mount      ed181b87.../    active  mounted
mnt-data.mount      f3337760.../    active  mounted
Create an ElasticSearch index

Now that the ElasticSearch cluster is running, you can create an index to store data.

curl -XPUT -d \
     '{ "settings" : { "index" : { "number_of_shards" : 3, "number_of_replicas" : 2 } } }'
Insert a few documents
curl -XPUT -d@- <<!
    "first_name" : "John",
    "last_name" :  "Smith",
    "age" :        25,
    "about" :      "I love to go rock climbing",
    "interests": [ "sports", "music" ]

curl -XPUT -d@- <<!
    "first_name" :  "Jane",
    "last_name" :   "Smith",
    "age" :         32,
    "about" :       "I like to collect rock albums",
    "interests":  [ "music" ]

curl -XPUT -d@- <<!
    "first_name" :  "Douglas",
    "last_name" :   "Fir",
    "age" :         35,
    "about":        "I like to build cabinets",
    "interests":  [ "forestry" ]
And query the index
curl -XGET
  "took": 50,
  "timed_out": false,
  "_shards": {
    "total": 3,
    "successful": 3,
    "failed": 0
  "hits": {
    "total": 2,

restarting the cluster

Even when you restart the entire cluster, your data is persisted.

fleetctl stop elasticsearch@{1..3}
fleetctl list-units

fleetctl start elasticsearch@{1..3}
fleetctl list-units

curl -XGET
  "took": 50,
  "timed_out": false,
  "_shards": {
    "total": 3,
    "successful": 3,
    "failed": 0
  "hits": {
    "total": 2,

Open the console

Finally you can see the servers and the distribution of the index in the cluster by opening the console

elasticsearch head

Deploy other ElasticSearch clusters

Changing the name of the template file is the only thing you need to deploy another ElasticSearch cluster.

cp elasticsearch\@.service my-cluster\@.service
fleetctl submit my-cluster\@.service
fleetctl start my-cluster\@{1..3}
How does it work?

Starting a node in an ElasticSearch cluster is quite trivial, as shown in by the command line below:

exec gosu elasticsearch elasticsearch \ \$HOST_LIST \
    --transport.publish_host=$PUBLISH_HOST \
    --transport.publish_port=$PUBLISH_PORT \

We use the unicast protocol and specify our own publish host and port and list of ip address and port numbers of all the other nodes in the cluster.

Finding the other nodes in the cluster

But how do we find the other nodes in the cluster? That is quite easy. We query the Consul REST API for all entries with the same service name that are tagged as the "es-transport". This is the service exposed by ElasticSearch on port 9300.

curl -s http://consul:8500/v1/catalog/service/$SERVICE_NAME?tag=es-transport

        "Node": "core-03",
        "Address": "",
        "ServiceID": "elasticsearch-1",
        "ServiceName": "elasticsearch",
        "ServiceTags": [
        "ServiceAddress": "",
        "ServicePort": 49170
        "Node": "core-01",
        "Address": "",
        "ServiceID": "elasticsearch-2",
        "ServiceName": "elasticsearch",
        "ServiceTags": [
        "ServiceAddress": "",
        "ServicePort": 49169
        "Node": "core-02",
        "Address": "",
        "ServiceID": "elasticsearch-3",
        "ServiceName": "elasticsearch",
        "ServiceTags": [
        "ServiceAddress": "",
        "ServicePort": 49169

Turning this into a comma seperated list of network endpoints is done using the following jq command:

curl -s http://consul:8500/v1/catalog/service/$SERVICE_NAME?tag=es-transport |\
     jq -r '[ .[] | [ .Address, .ServicePort | tostring ] | join(":")  ] | join(",")'
Finding your own network endpoint

As you can see in the above JSON output, each service entry has a unique ServiceID. To obtain our own endpoint, we use the following jq command:

curl -s http://consul:8500/v1/catalog/service/$SERVICE_NAME?tag=es-transport |\
     jq -r ".[] | select(.ServiceID==\"$SERVICE_9300_ID\") | .Address, .ServicePort" 
Finding the number of node in the cluster

Finding the intended number of nodes in the cluster is determined by counting the number of fleet unit instance files in CoreOS on startup and passing this number as an environment variable.

TOTAL_NR_OF_SERVERS=$(fleetctl list-unit-files | grep '%p@[^\.][^\.]*.service' | wc -l)

The %p refers to the part of the fleet unit file before the @ sign.

The Docker run command

The Docker run command is shown below. ElasticSearch exposes two ports: port 9200 exposes a REST api to the clients and port 9300 is used as the transport protocol between nodes in the cluster. Each port is a service and tagged appropriately.

ExecStart=/bin/sh -c "/usr/bin/docker run --rm \
    --name %p-%i \
    --env SERVICE_NAME=%p \
    --env SERVICE_9200_TAGS=http \
    --env SERVICE_9300_ID=%p-%i \
    --env SERVICE_9300_TAGS=es-transport \
    --env TOTAL_NR_OF_SERVERS=$(fleetctl list-unit-files | grep '%p@[^\.][^\.]*.service' | wc -l) \
    -P \
    --dns $(ifconfig docker0 | grep 'inet ' | awk '{print $2}') \
    --dns-search=service.consul \

The options are explained in the table below:

option description --env SERVICE_NAME=%p The name of this service to be advertised in Consul, resulting in a FQDN of %p.service.consul and will be used as the cluster name. %p refers to the first part of the fleet unit template file up to the @. --env SERVICE_9200_TAGS=www The tag assigned to the service at port 9200. This is picked up by the http-router, so that any http traffic to the host elasticsearch is direct to this port. --env SERVICE_9300_ID=%p-%i The unique id of this service in Consul. This is used by the startup script to find it's external port and ip address in Consul and will be used as the node name for the ES server. %p refers to the first part of the fleet unit template file up to the @ %i refers to the second part of the fleet unit file upto the .service. --env SERVICE_9300_TAGS=es-transport The tag assigned to the service at port 9300. This is used by the startup script to find the other servers in the cluster. --env TOTAL_NR_OF_SERVERS=$(...) The number of submitted unit files is counted and passed in as the environment variable 'TOTAL_NR_OF_SERVERS'. The start script will wait until this number of servers is actually registered in Consul before starting the ElasticSearch Instance. --dns $(...) Set DNS to query on the docker0 interface, where Consul is bound on port 53. (The docker0 interface ip address is chosen at random from a specific range). -dns-search=service.consul The default DNS search domain. Sources

The sources for the ElasticSearch repository can be found on github.

source description   complete startup script of elasticsearch elasticsearch CoreOS fleet unit files for elasticsearch cluster consul-elasticsearch Sources for the Consul ElasticSearch repository Conclusion

CoreOS fleet template unit files are a powerful way of deploying ready to use components for your platform. If you want to deploy cluster aware applications, a service registry like Consul is essential.

Next-gen Web Apps with Isomorphic JavaScript

Fri, 05/01/2015 - 20:54

The web application landscape has recently seen a big shift in application architecture. Nowadays we build so-called Single Page Applications. These are web applications which render and run in the browser, powered by JavaScript. They are called ‚ÄúSingle Page‚ÄĚ because in such an application the browser never actually switches between pages. All interaction takes place within a single HTML document. This is great because users will not see a ‚ÄĚflash of white‚ÄĚ whenever they perform an action, so all interaction feels much more fluid and natural. The application seems to respond much quicker which has a positive effect on user experience and conversion of the site. Unfortunately Single Page Applications also have several big drawbacks, mostly concerning the initial loading time and poor rankings in search engines.

Continue reading on Medium ¬Ľ

How to deploy High Available persistent Docker services using CoreOS and Consul

Thu, 04/23/2015 - 15:45

Providing High Availability to stateless applications is pretty trivial as was shown in the previous blog posts A High Available Docker Container Platform and Rolling upgrade of Docker applications using CoreOS and Consul. But how does this work when you have a persistent service like Redis?

In this blog post we will show you how a persistent service like Redis can be moved around on machines in the cluster, whilst preserving the state. The key is to deploy a fleet mount configuration into the cluster and mount the storage in the Docker container that has persistent data.

To support persistency we have added a NAS to our platform architecture in the form of three independent NFS servers which act as our NAS storage, as shown in the picture below.

CoreOS platform architecture with fake NASThe applications are still deployed in the CoreOS cluster as docker containers.  Even our Redis instance is running in a Docker container. Our application is configured using the following three Fleet unit files:

The unit file of the Redis server is the most interesting one because it is our persistence service. In the unit section of the file, it first declares that it requires a mount for '/mnt/data' on which it will persist its data.


In the start clause of the redis service, a specific subdirectory of /mnt/data is mounted into the container.

ExecStart=/usr/bin/docker run --rm \
    --name app-redis \
    -v /mnt/data/app-redis-data:/data \
    -p 6379:6379 \

The mnt-data.mount unit file is quite simple: It defines an NFS mount with the option 'noauto' indicating  that device should be automatically mounted on boot time.  The unit file has the option 'Global=true' so that the mount is distributed to  all the nodes in the cluster. The mount is only activated when another unit requests it.



Please note that the NFS mount specifies system security (sec=sys) and uses NFS version 3 protocol, to avoid all sorts of errors surrounding mismatches in user- and group ids between the client and the server.

Preparing the application

To see the failover in action, you need to start the platform and deploy the application:

git clone
cd coreos-container-platform-as-a-service/vagrant
vagrant up

This will start 3 NFS servers and our 3 node CoreOS cluster. After that is done, you can deploy the application, by first submitting the mount unit file:

cd ../fleet-units/app
fleetctl load mnt-data.mount

starting the redis service:

fleetctl start app-redis.service

and finally starting a number of instances of the application:

fleetctl submit app-hellodb@.service
fleetctl load app-hellodb@{1..3}.service
fleetctl start app-hellodb@{1..3}.service

You can check that everything is running by issuing the fleetctl list-units command. It should show something like this:

fleetctl list-units
app-hellodb@1.service	8f7472a6.../	active		running
app-hellodb@2.service	b44a7261.../	active		running
app-hellodb@3.service	2c19d884.../	active		running
app-redis.service	2c19d884.../	active		running
mnt-data.mount		2c19d884.../	active		mounted
mnt-data.mount		8f7472a6.../	inactive	dead
mnt-data.mount		b44a7261.../	inactive	dead

As you can see three app-hellodb instances are running and the redis service is running on, which is the only host that as /mnt/data mounted. The other two machines have this mount in the status 'dead', which is an unfriendly name for stopped.

Now you can access the app..

yes 'curl; echo ' | head -10 | bash
Hello World! I have been seen 20 times.
Hello World! I have been seen 21 times.
Hello World! I have been seen 22 times.
Hello World! I have been seen 23 times.
Hello World! I have been seen 24 times.
Hello World! I have been seen 25 times.
Hello World! I have been seen 26 times.
Hello World! I have been seen 27 times.
Hello World! I have been seen 28 times.
Hello World! I have been seen 29 times.
Redis Fail-over in Action

To see the fail-over in action, you start a monitor on a machine not running Redis. In our case the machine running app-hellodb@1.

vagrant ssh -c \
   "yes 'curl --max-time 2; sleep 1 ' | \
    bash" \

Now restart the redis machine:

vagrant ssh -c "sudo shutdown -r now" app-redis.service

After you restarted the machine running Redis, the  output should look something like this:

Hello World! I have been seen 1442 times.
Hello World! I have been seen 1443 times.
Hello World! I have been seen 1444 times.
Hello World! Cannot tell you how many times I have been seen.
	(Error 111 connecting to redis:6379. Connection refused.)
curl: (28) Operation timed out after 2004 milliseconds with 0 out of -1 bytes received
curl: (28) Operation timed out after 2007 milliseconds with 0 out of -1 bytes received
Hello World! I have been seen 1445 times.
Hello World! I have been seen 1446 times.
curl: (28) Operation timed out after 2004 milliseconds with 0 out of -1 bytes received
curl: (28) Operation timed out after 2004 milliseconds with 0 out of -1 bytes received
Hello World! I have been seen 1447 times.
Hello World! I have been seen 1448 times.

Notice that the distribution of your units has changed after the reboot.

fleetctl list-units
app-hellodb@1.service	3376bf5c.../	active		running
app-hellodb@2.service	ff0e7fd5.../	active		running
app-hellodb@3.service	3376bf5c.../	active		running
app-redis.service	ff0e7fd5.../	active		running
mnt-data.mount		309daa5a.../	inactive	dead
mnt-data.mount		3376bf5c.../	inactive	dead
mnt-data.mount		ff0e7fd5.../	active		mounted

We now have the basis for a truly immutable infrastructure setup: the entire CoreOS cluster including the application can be destroyed and a completely identical environment can be resurrected within a few minutes!

  • Once you have an reliable external persistent store, CoreOS can help you migrate persistent services just as easy as stateless services. We chose a NFS server for ease of use on this setup, but nothing prevents you from mounting¬†other kinds of storage systems for your application.
  • Consul excels in providing fast and dynamic service discovery for ¬†services, allowing the¬†Redis service to migrate to a different machine and the application instances to find the new address of the Redis service through as simple DNS lookup!


Scaling Agile? Keep it simple, scaler!

Wed, 04/22/2015 - 08:59

The promise of Agile is short cycled value delivery, with the ability to adapt. This is achieved by focusing on the people that create value and optimising the way they work.

Scrum provides a framework that provides a limited set of roles and artefacts and offers a simple process framework that helps to implement the Agile values and to adhere to the Agile principles.

I have supported many organisations in adopting Agile as their mindset and culture. What puzzles me is that many larger organisations seem to think that Scrum is not enough in their context and they feel the need for something bigger and more complicated. As a result of this, more and more Agile transformations start with scaling Agile to fit their context and then try to make things less complex.

While the various scaling frameworks for Agile contain many useful and powerful tools to apply in situations that require them, applying a complete Agile scaling framework to an organisation from the get-go often prevents the really needed culture and mindset change.

When applying a little bit of creativity, already present organisational structure can be mapped easily on the structure suggested by many scaling frameworks. Most frameworks explain the needed behaviour in an Agile environment, but these explanations are often ignored or misinterpreted. Due to (lengthy) descriptions of roles and responsibilities, people tend to stop thinking for themselves about what would work best and start to focus on who plays which role and what is someone else’s responsibility. There is a tendency to focus on the ceremonies rather than on the value that should be delivered by the team(s) with regards to product or service.

My take on adopting Agile would be to start simple. Use an Agile framework that prescribes very little, like Scrum or Kanban, in order to provoke learning and experiencing. From this learning and experiencing will come changes in the organisational structure to best support the Agile Values and Principles. People will find or create positions where their added value has most impact on the value that the organisation creates and, when needed, will dismantle positions and structure that prevent this value to be created.

Another effect of starting simple is that people will not feel limited by rules and regulations, and that way use their creativity, experience and capabilities easier. Oftentimes, more energy is create by less rules.

As said by others as well, some products or value are difficult to create with simple systems. As observed by Dave Snowden and captured in his Cynefin framework, too much simplicity could result in chaos when this simplicity is applied to complex systems. To create value in more complex systems, use the least amount of tools provided by the scaling frameworks to prevent chaos and leverage the benefits that simpler systems provide. Solutions to fix problems in complex systems are best found when experiencing the complexity and discovering what works best to cope with that. Trying to prevent problems to pop up might paralyse an organisation too much to put out the most possible value.

So: Focus on delivering value in short cycles, adapt when needed and add the least amount of tools and/or process to optimise communication and value delivery.

Swift optional chaining and method argument evaluation

Tue, 04/21/2015 - 08:21

Everyone that has been programming in Swift knows that you can call a method on an optional object using a question mark (?). This is called optional chaining. But what if that method takes any arguments whose value you need to get from the same optional? Can you safely force unwrap those values?

A common use case of this is a UIViewController that runs some code within a closure after some delay or after a network call. We want to keep a weak reference to self within that closure because we want to be sure that we don't create reference cycles in case the closure would be retained. Besides, we (usually) don't need to run that piece of code within the closure in case the view controller got dismissed before that closure got executed.

Here is a simplified example:

class ViewController: UIViewController {

    let finishedMessage = "Network call has finished"
    let messageLabel = UILabel()

    override func viewDidLoad() {

        someNetworkCall { [weak self] in

    func finished(message: String) {
        messageLabel.text = message

Here we call the someNetworkCall function that takes a () -> () closure as argument. Once the network call is finished it will call that closure. Inside the closure, we would like to change the text of our label to a finished message. Unfortunately, the code above will not compile. That's because the finished method takes a non-optional String as parameter and not an optional, which is returned by self?.finishedMessage.

I used to fix such problem by wrapping the code in a if let statement:

if let this = self {

This works quite well, especially when there are multiple lines of code that you want to skip if self became nil (e.g. the view controller got dismissed and deallocated). But I always wondered if it was safe to force unwrap the method arguments even when self would be nil:


The question here is: does Swift evaluate method argument even if it does not call the method?

I went through the Swift Programming Guide to find any information on this but couldn't find an answer. Luckily it's not hard to find out.

Let's add a method that will return the finishedMessage and print a message and then call the finished method on an object that we know for sure is nil.

override func viewDidLoad() {

    let vc: ViewController? = nil

func printAndGetFinishedMessage() -> String {
    println("Getting message")
    return finishedMessage

When we run this, we see that nothing gets printed to the console. So now we know that Swift will not evaluate the method arguments when the method is not invoked. Therefore we can change our original code to the following:

someNetworkCall { [weak self] in

Is LeSS meer dan SAFe?

Fri, 04/17/2015 - 14:48

(Grote) Nederlandse bedrijven die op zoek zijn naar een oplossing om de voordelen die hun Agile teams brengen op te schalen, gebruiken vooral het Scaled Agile Framework (SAFe) als referentiemodel. Dit model is -ook voor managers- zeer toegankelijke opgezet en trainingen en gecertificeerde consultants zijn beschikbaar. Al in 2009 beschreven Craig Larman en Bas Vodde hun ervaringen met de toepassing van Scrum in grote organisaties (onder andere Nokia) in hun boeken 'Scaling Lean & Agile Development' en 'Practices for Scaling Lean & Agile Development'. De methode noemden ze Large Scale Scrum, afgekort LeSS.
LeSS heeft de afgelopen jaren een onopvallend bestaan geleid. Onlangs is besloten dit waardevolle gedachtengoed meer in de spotlights te zetten. Er komt in de zomer een derde boek, de site is gelanceerd, er is een trainingen tournee gestart en Craig en Bas geven acte de presence op de toonaangevende conferenties. Zo zal Bas 4 juni als keynote optreden tijdens Xebicon 2015, in Amsterdam. Is LeSS meer of minder dan SAFe? Of min of meer SAFe?

Wat is LeSS?
Less is dus een methode om een grote(re) organisatie met Agile teams als basis in te richten. Zoals de naam verraadt, Scrum is daarbij het uitgangspunt. Er zijn 2 smaken: ‚Äėgewoon‚Äô LeSS, tot 8 teams en Less Huge, vanaf 8 teams. LeSS bouwt op verplichte regels (rules), bijvoorbeeld "An Overall Retrospective is held after the Team Retrospectives to discuss cross-team and system-wide issues, and create improvement experiments. This is attended by Product Owner, ScrumMasters, Team Representatives, and managers (if there are any).‚ÄĚ Daarnaast kent LeSS principles (ontwerp criteria). De principes vormen het referentie raamwerk op basis waarvan je de juiste ontwerp besluiten neemt. Tenslotte zijn er de Guidelines en Experiments, de dingen die in de praktijk bij organisaties succesvol of juist niet zijn gebleken. LeSS gaat naast het basis framework verder dieper in op:

  • Structure (de organisatie structuur)
  • Management (de -veranderende- rol van management)
  • Technical Excellence (sterk gebaseerd op XP en Continuous Delivery)
  • Adoption (de transformatie naar de LeSS organisatie).

LeSS in een notendop
De basis van LeSS is dat Large Scale Scrum = Scrum! Net als SAFe wordt in LeSS gezocht naar hoe Scrum toegepast kan worden op een groep van zeg 100 man. LeSS blijft het dichtst bij Scrum: er is 1 sprint, met 1 Product Owner, 1 product backlog, 1 planning en 1 sprint review, waarin 1 product wordt gerealiseerd. Dit is dus anders dan in SAFe, waarin een opgeblazen sprint is gedefinieerd (de Product Increment). Om deze 1 sprint implementatie te kunnen waarmaken is naast een hele sterke whole product focus, bijvoorbeeld ook een technisch platform nodig, dat dit ondersteunt. Waar SAFe pragmatisch een geleidelijke invoering van Agile at Scale toestaat, is LeSS strenger in de klaar-voor-de-start eisen. Er moet een structuur worden neergezet die de cultuur van de 'contract game’ doorbreekt. De cultuur van overvragen, druk, onduidelijkheid, verrassingen, en afrekenende verantwoordelijkheid.

LeSS is meer en minder SAFe
De recente inspanning om LeSS toegankelijk te maken gaan ongetwijfeld leiden tot een sterk toenemende aandacht voor deze aansprekende benadering voor de inrichting van Agile at Scale. LeSS is anders dan SAFe, al hebben beide modellen vooral in hun inspiratiebronnen ook veel gemeen.
Beide modellen kiezen een andere insteek, bijvoorbeeld mbt:

  • hoe Scrum toe te passen op een cluster van teams
  • de benadering van de transformatie naar Agile at Scale
  • hoe oplossingen worden gebracht: SAFe geeft de oplossing, LeSS de voors en tegens van keuzes

Opvallend is verder dat SAFe (met het portfolioniveau) uitlegt hoe de verbinding tussen strategie en backlogs gelegd moet worden. LeSS besteedt daarentegen meer aandacht aan de transformatie (Adoption) en Agile op hele grote schaal (LeSS Huge).

Of een organisatie kiest voor LeSS of SAFe, zal afhangen wat het best past bij de organisatie. Past bij de veranderambitie en bij de ‚Äėagility‚Äô op moment van starten. Sterk ‚Äėblauwe‚Äô organisaties zullen kiezen voor SAFe, organisaties die een overtuigende stap richting een Agile organisatie durven te zetten, zullen eerder kiezen voor LeSS. In beide gevallen loont het om kennis te nemen van de oplossingen die de andere methode biedt.

Experimenting with Swift and UIStoryboardSegues

Wed, 04/15/2015 - 21:58

Lately I've been experimenting a lot with doing things differently in Swift. I'm still trying to find best practices and discover completely new ways of doing things. One example of this is passing objects from one view controller to another through a segue in a single line of code, which I will cover in this post.

Imagine two view controllers, a BookViewController and an AuthorViewController. Both are in the same storyboard and the BookViewController has a button that pushes the AuthorViewController on the navigation controller through a segue. To know which author we need to show on the AuthorViewController we need to pass an author object from the BookViewController to the AuthorViewController. The traditional way of doing this is giving the segue an identifier and then setting the object:

class BookViewController: UIViewController {

    var book: Book!

    override func prepareForSegue(segue: UIStoryboardSegue, sender: AnyObject?) {
        if segue.identifier == "ShowAuthor" {
            let authorViewController = segue.destinationViewController as! AuthorViewController

class AuthorViewController: UIViewController {

    var author: Author!

And in case we would use a modal segue that shows a AuthorViewController embedded in a navigation controller, the code would be slightly more complex:

if segue.identifier == "ShowAuthor" {
  let authorViewController = (segue.destinationViewController as! UINavigationController).viewControllers[0] as! AuthorViewController =

Now let's see how we can add an extension to UIStoryboardSegue that makes this a bit easier and works the same for both scenarios. Instead of checking the segue identifier we will just check on the type of the destination view controller. We assume that based on the type the same object is passed on, even when there are multiple segues going to that type.

extension UIStoryboardSegue {

    func destinationViewControllerAs<T>(cl: T.Type) -> T? {
        return destinationViewController as? T ?? (destinationViewController as? UINavigationController)?.viewControllers[0] as? T

What we've done here is add the method destinationViewControllerAs to UIStoryboardSegue that checks if the destinationViewController is of the generic type T. If it's not, it will check if the destinationViewController is a navigation controller and if it's first view controller is of type T. If it finds either one, it will return that instance of T. Since it can also be nil, the return type is an optional T.

It's now incredibly simple to pass on our author object to the AuthorViewController:

override func prepareForSegue(segue: UIStoryboardSegue, sender: AnyObject?) {
  segue.destinationViewControllerAs(AuthorViewController.self)?.author =

No need to check any identifiers anymore and less code. Now I'm not saying that this is the best way to do it or that it's even better than the traditional way of doing things. But it does show that Swift offers us new ways of doing things and it's worth to experiment to find best practices.

The source code of the samples and extension is available on

Questions with a license to kill in the Sprint Review

Tue, 04/14/2015 - 09:19

A team I had been coaching held a sprint review to show what they had achieved and to get feedback from stakeholders. Among these were managers, other teams, enterprise architects, and other interested colleagues.

In the past sprint they had built and realized the automation of part of the Continuous Delivery pipeline. This was quite a big achievement for the team. The organization had been struggling for quite some time to get this working, and the team had realized this in a couple of sprints!

Team - "Anyone has questions or wants to know more?"
Stakeholder - "Thanks for the demo. How does the shown solution deal with 'X'?"

The team replied with a straightforward answer to this relatively simple question.

Stakeholder - "I have more questions related to the presented solution and concerns on corporate level, but this is probably not the good time to go into details."

What just happened and how did the team respond?

First, let me describe how the dialogue continued.

Team - "Let's make time now because it is important. What do you have on your mind?"
Stakeholder - "On corporate level solution Y is defined to deal with the company's concern related to Z compliance. I am concerned with how your solution will deal with this. Please elaborate on this."
[Everybody in the organization knows that Z compliance has been a hot topic during the past year.]

Team - "We have thought of several alternatives to deal with this issue. One of these is to have a coupling with another system that will provide the compliance. Also we see possibilities in altering the ....."
Stakeholder - "The other system has issues coping with this and is not suited for what you want it to do. What are your plans for dealing with this?"

The team replied with more details after which the stakeholder asked even more detailed questions....

How dit the team get itself out of this situation?

After a couple of questions and answers the team responded with "Look, the organisation has been struggling to find a working solution for quite some time now and has't succeeded. Therefor, we are trying a new and different approach. Since this is new we don't have all the answers yet. Next steps will deal with your concerns."

Team - "Thanks for your feedback and see you all at the next demo!"

Killing a good idea

In the above dialogue between the team and one stakeholder during the sprint review the stakeholder kept asking details questions about specific aspects of the solution. He also related these to well-known corporate issues of which the importance was very clear to everyone. Thereby, consciously or unconsciously, intimidating the audience whether the approach chosen by the team is a good one and perhaps should be abandoned.

This could be especially dangerous if not appropriately dealt with. For instance, managers at first being supportive of the (good) idea might turn against the approach, even though the idea is a good one.

Dealing with these and other difficult questions

In his book 'Buy-in - saving your good idea from getting shot down' John Kotter describes 4 basic types of attack:

  1. Fear mongering
  2. Death by delay
  3. Confusion
  4. Ridicule

Attacks can be one of these four or any combination of these. The above attack is an example of a combination of 'Fear mongering' (relating to the fear that important organisational concerns are not properly addressed) and 'Confusion' (asking about many details that are not yet worked out).

In addition, Kotter describes 24 basic attacks. The attack as described above is an example of attack no. 6.

Don't worry. No need to remember all 24 responses; they all follow a very simple strategy that can be applied:

Step 1: Invite the stakeholder(s) to ask their questions,

Step 2: Respect the person asking the question by taking his point seriously,

Step 3: Respond in a reasonable and concise way.

The team did well by inviting the stakeholder to come forward with his questions. This is good marketing to the rest of the stakeholders: this shows the team believes in the idea (their solution) and is confident to respond to any (critical) question.

Second, the team responded in a respectful way taking the question serious as a valid concern. The team did so by responding in a concise and reasonable way.

As Kotter explains, it is not about convincing that one critical  stakeholder, but it's about not losing the rest of the stakeholders!


"Buy-in - saving your good idea from getting shot down" - John P. Kotter & Lorne A. Whitehead,

"24 attacks & responses" - John P. Kotter & Lorne Whitehead,

Rolling upgrade of Docker applications using CoreOS and Consul

Mon, 04/06/2015 - 21:17

In the previous blog post, we showed you how to setup a High Available Docker Container Application platform using CoreOS and Consul. In this short blog post, we will show you how easy it is to perform a rolling upgrade of a deployed application.


In this blog post we will use the same architecture but for the deployment we used Vagrant instead of Amazon AWS which is a little bit snappier to use  :-).


In the Vagrant architecture we have replaced the AWS Elastic Load Balancer with our own consul-http-router Load Balancer and moved it inside the cluster. This is a HA proxy which dynamicly routes traffic to any of the http routers in the cluster.

Getting Started

In order to get your own container platform as a service running on vagrant, clone the repository and start your cluster.

git clone
cd coreos-container-platform-as-a-service/vagrant
vagrant up
vagrant up
Bringing machine 'core-01' up with 'virtualbox' provider...
Bringing machine 'core-02' up with 'virtualbox' provider...
Bringing machine 'core-03' up with 'virtualbox' provider...
Checkout the cluster

After the cluster has started, you can use the following command to checkout whether your cluster is fully operational. You should see 5 units running on each machine.

for node in 1 2 3 ; do
  vagrant ssh -c "systemctl | grep consul" core-0$node
consul-http-router-lb.service loaded active running consul-http-router-lb
consul-http-router.service loaded active running consul-http-router
consul-server-announcer.service loaded active running Consul Server Announcer
consul-server-registrator.service loaded active running Registrator
consul-server.service loaded active running Consul Server Agent


Deploying the application

Once the cluster is running, you can deploy or paas-monitor application. This happens in two stages. First you submit the template.

fleetctl submit paas-monitor\@.service

Then, you load and start the new instances.

fleetctl load paas-monitor\@{1..6}.service
fleetctl start paas-monitor\@{1..6}.service
fleetctl list-units | grep paas-monitor

You can now see the paas-monitor in operation on your machine by opening and clicking on start.  You should see something like shown in the table below. Leave this running while we are going to update the application!

host release message # of calls avg response time last response time 47ea72be3817:1337 v1 Hello World from v1 53 8 11 cc4227a493d7:1337 v1 Hello World from v1 59 11 8 04a58012910c:1337 v1 Hello World from v1 54 7 6 090caf269f6a:1337 v1 Hello World from v1 58 7 7 096d01a63714:1337 v1 Hello World from v1 53 7 9 d43c0744622b:1337 v1 Hello World from v1 55 7 6 Updating the application

Now we are going to update your application. Normally, we would expect that you specify a higher version of the Docker image into unit file. But instead of changing the version of the image to be executed change the value of the environment variable RELEASE in the unit template file  paas-monitor\@.service.

sed -i -e 's/--env RELEASE=[^ ]*/--env RELEASE=v2/' paas-monitor\@.service

Now we have changed the unit template file, you should destroy the old unit file and submit the new one.

fleetctl destroy paas-monitor\@.service
fleetctl submit paas-monitor\@.service

Now you have two options, a slow one and a fast one.

Slow Option

The slow option is to  iterate over the running instances, stop them one by one, destroy them and start a new instance based on the newly submitted template. Because this is boring, repetitive work we have created a small script that does this :-)

./ paas-monitor\@.service

Fast Option

The fast option is to start 6 new ones and stop all 6 old ones after the new ones are running.

fleetctl load paas-monitor\@1{1..6}.service
fleetctl start paas-monitor\@1{1..6}.service
fleetctl list-units | grep 'paas-monitor\@1[1-6].service' | grep running
fleetctl stop paas-monitor@{1..6}.service
fleetctl destroy paas-monitor@{1..6}.service

When you watch your monitor, you should see the new instance appear one by one.

host release message # of calls avg response time last response time 47ea72be3817:1337 v1 Hello World from v1 53 8 11 cc4227a493d7:1337 v1 Hello World from v1 59 11 8 04a58012910c:1337 v1 Hello World from v1 54 7 6 090caf269f6a:1337 v1 Hello World from v1 58 7 7 096d01a63714:1337 v1 Hello World from v1 53 7 9 d43c0744622b:1337 v1 Hello World from v1 55 7 6 fee39f857479:1337 v2 Hello World from v2 18 7 9 99c1a5aa3b8b:1337 v2 Hello World from v2 2 7 9 Conclusion

CoreOS and Consul provides all the basic functionality to manage your Docker containers and perform rolling upgrade of your application.


Increasing the tap area of UIButtons made with PaintCode

Sun, 04/05/2015 - 21:04

Whether you're a developer or not, every iPhone user has experienced the case where he or she tries to tap a button (with an image and nothing happens. Most likely because the user missed the button and pressed next to it. And that's usually not the fault of the user, but the fault of the developer/designer because the button is too small. The best solution is to have bigger icons (we're only talking about image buttons, no text only buttons) so it's easier for the user to tap. But sometimes you (or your designer) just wants to use a small icon, because it simply looks better. What do you do then?

For buttons with normal images this is very easy. Just make the button bigger. However, for buttons that draw themselves using PaintCode, this is slightly harder. In this blogpost I'll explain why and show two different ways to tackle this problem.

I will not go into the basics of PaintCode. If you're new to PaintCode have a look at their website or go to Freek Wielstra's blogpost Working With PaintCode And Interface Builder In XCode. I will be using the examples of his post as basis for the examples here so it's good to read his post first (though not strictly necessary).

To get a better understanding of how a UIButton with an image differs from a UIButton drawn with PaintCode, I will first show what a non-PaintCode button would look like. Below we have a PNG image of 25 by 19 pixels (for non-Retina). Apple recommends buttons to be 44 by 44 points so we increase the size of the button. The button has a gray background and we can see that the image stays the original size and is nicely in the center while the touch target becomes big enough for the user to easily tap it.


The reason it behaves like this is because of the Control settings in the Attribute Inspector. We could let the content align different or even make it stretch. But stretching this would be a bad idea since it's not a vector image and it would look pixelated.

That's why we love PaintCode graphics. They can scale to any size so we can use the same graphic in different places and at different screen scales (non-retina, retina @2x and iPhone 6 plus which uses @3x). But what if we don't want the graphic to scale of to the entire size of the button like above?

A bad solution would be not to use a frame in PaintCode. Yes, the image will stay the size you gave it when you put it in a button, but it will be left-top aligned. Also you won't be able to use it anywhere else.

If you are really sure that you will never use the image anywhere else than on your button you can group your vector graphic and tell it to have flexible margins within the frame and have a fixed width and height. Using the email icon from Freek Wielstra this will look something like this:

Screen Shot 2015-04-05 at 20.39.36

You can verify that this works by changing the size of the frame. The email icon should always stay in the center both horizontal and vertical and it should stay the same size. The same will happen if you use this in a custom UIButton.

Now let's have a look at a better solution that will allow us to reuse the graphic at different sizes and have some padding around it in a button. It's easier than you might expect and it follows the actual rules of a UIButton. The UIButton class has a property named contentEdgeInsets with the following description: "The inset or outset margins for the rectangle surrounding all of the button’s content." That sounds exactly like what we need. Our PaintCode graphics are the content of the buttons right? Unfortunately it's not treated as such since we are doing the drawing ourselves which does not adhere to that property. However we can very easily make it adhere to it:

class EmailButton: UIButton {

    override func drawRect(rect: CGRect) {
        StyleKit.drawEmail(frame: UIEdgeInsetsInsetRect(rect, contentEdgeInsets), color: tintColor!)

And that's really all there is to it. Just make sure to put a frame around your graphics in PaintCode and make it stretch to all sides of the frame. Now you can specify the content insets of your buttons in Interface Builder and you icon will have padding around it.

Screen Shot 2015-04-05 at 20.59.42

Working with PaintCode and Interface Builder in XCode

Sat, 04/04/2015 - 08:00

Every self-respecting iOS developer should know about PaintCode by now, an OSX app for drawing graphics that don't save as images, but as lengths of code that draw graphics. The benefits of this are vastly reduced app installation size - no need to include three resolutions of the same image for every image - and seamlessly scalable graphics.

One thing that I personally struggled with for a while now was how to use them effectively in combination with Interface Builder, the UI development tool for iOS and OSX apps. In this blog I will explain an effective and simple method to draw PaintCode graphics in a way where you can see what you're doing in Interface Builder, using the relatively new @IBDesignable annotation. I will also go into setting colors, and finally about how to deal with views that depend on dynamic runtime data to draw themselves.

First, let's create a simple graphic in PaintCode. Let's draw a nice envelope or email icon. Add a frame around it, and set the constraints to variable so it will adjust to the size of the frame (drag the frame corners around within PaintCode to confirm). This gives us a nice, scalable icon. Save the file, then export it to your xcode project in the language of your choice.

Email icon in PaintCode

Squiggly lines are the best


Now, the easiest and most straightforward way to get a PaintCode image into your Interface Builder is to simply create a UIView subclass, and call the relevant StyleKit method in its drawRect method. You can do this too in either Swift or Objective-C; this example will be in Swift, but if you're stuck with the slow Swift compiler in XCode 6.x, you might prefer Objective-C for this kind of simple and straightforward code.

To use it in Interface Builder, simply create a storyboard or xib, drag an UIView onto it, and change its Custom Class to your UIView subclass. Make sure to have your constraints set properly, and run - you should be able to see your custom icon in your working app.

class EmailIcon: UIView {
  override func drawRect(rect: CGRect) {
    StyleKit.drawEmail(frame: rect)
Screenshot 2015-04-03 21.36.42

Ghost views, the horror of Interface Builder


As you probably noticed though, it's far from practical as it is in Interface Builder right now - all you see (or don't see really) is an empty UIView. I guess you could give it a background color in IB and unset that again in your code, but that's far from practical. Instead, we'll just slap an @IBDesignable annotation onto the UIView subclass. Going back to IB, it should start to compile code, and a moment later, your PaintCode image shows up, resizable and all.

class EmailIcon: UIView {
  override func drawRect(rect: CGRect) {
    StyleKit.drawEmail(frame: rect)
@IBDesignable view in Interface Builder

like magic


Adding color

We can configure our graphic in PaintCode to have a custom color. Just go back to PaintCode and change the Color to 'Parameter' - see image.

Set color to Parameter

Export the file again, and change the call to the StyleKit method to include the color. It's easiest in this case to just pass the UIView's own tintColor property. Going back to Interface Builder, we can now set the default tintColor property, and it should update in the IB preview instantly.

As an alternative, you can add an @IBInspectable color property to your UIView, but I would only recommend this for more complicated UIViews - if they include multiple StyleKit graphics, for example.

class EmailIcon: UIView {
  override func drawRect(rect: CGRect) {
    StyleKit.drawEmail(frame: rect, color: tintColor)

let's make it Xebia purple, to please the boss


Dealing with dynamic properties

One case I had to deal with is an UIView that can draw a selection of StyleKit images, depending on the value of a dynamic model value. I considered a few options to deal with that:

  1. Set a default value, to be overridden at runtime. This is a bit dangerous though; forget to reset or unset this property at runtime and your user will be stuck with some default placeholder, or worse, flat-out wrong information.
  2. Make the property @IBInspectable. This only works with relatively simple values though (strings, numbers, colors), and it has the same problem as the above.

What I needed was a way to set a property, but only from within Interface Builder. Luckily, that exists, as I found out later. In UIView, there is a method called prepareForInterfaceBuilder, which does exactly what it says on the tin - override the method to set properties only relevant in Interface Builder. So in our case:

class EmailIcon: UIView {
  // none set by default
  var iconType: String? = nil

  override func drawRect(rect: CGRect) {
    if iconType == "email" {
      StyleKit.drawEmail(frame: rect, color: tintColor)

    if iconType = "email-read" {
      StyleKit.drawEmailRead(frame: rect, color: tintColor)

    // if none of the above, draw nothing

  // draw the 'email' icon by default in Interface Builder
  override func prepareForInterfaceBuilder() {
    iconType = "email"

And that's all there is to it. You can use this method to create all of your graphics, keep them dynamically sized and colored, and to use the full power of Interface Builder and PaintCode combined.

Animate constraints with a simple UIView extension in Swift

Wed, 04/01/2015 - 22:01

iOS Developers that are getting started with Auto Layout for the first time often run in trouble once they try to animate their views. The solution is quite simple, but in this post I'll create a simple Swift extension that makes it even easier.

When you don't use Auto Layout, you need to make the changes to your UIView frames inside an animations block. Those changes will then be animated. So without knowing any better, you would do the same for your Auto Layout constraints, which means changing the constraint constants inside the animations block. This doesn't work however, and a quick google search on "how to animate constraints" shows that your need only need to call view.layoutIfNeeded within the animations block after you've made the changes to the constraints.

A simple example of this:

heightConstraint.constant = 100
UIView.animateWithDuration(0.5, animations: { [weak self] in
  self?.view.layoutIfNeeded() ?? ()

it's necessary to call it this wat since updating constants of constraints doesn't immediately change the frames of the views affected by those constraints. That's why we need to call layoutIfNeeded, to update all the frames of our views.

Since we just want to be always safe with memory management we'll use a weak reference of self within the closure, even though we know the closure is not retained after the animation runs and the view will not be deallocated before the animation finishes (it's good practice to use weak whenever you can to avoid memory related problems).

The above sample is not a lot of code, but each time you want to animate something with constraints you need to write the same thing. Wouldn't it be much easier to write the following instead?

heightConstraint.constant = 100

We can achieve that by adding an extension to UIView:

extension UIView {
  func animateConstraintWithDuration(duration: NSTimeInterval = 0.5) {
    UIView.animateWithDuration(duration, animations: { [weak self] in
      self?.layoutIfNeeded() ?? ()

Default values for the method parameters makes this even nicer. Above we used 0.5 as default duration but of course you can fill in duration you want when calling the method. Also we can add parameters with default values for the other parameters that animateWithDuration has.

The final extension
extension UIView {
  func animateConstraintWithDuration(duration: NSTimeInterval = 0.5, delay: NSTimeInterval = 0.0, options: UIViewAnimationOptions = nil, completion: ((Bool) -> Void)? = nil) {
    UIView.animateWithDuration(duration, delay:delay, options:options, animations: { [weak self] in
      self?.layoutIfNeeded() ?? ()
    }, completion: completion)

There we have it. An extremely simple extension method that not only saves us a little bit of code each time we need to animate constraints but is also makes the code a lot cleaner since the name of the method clearly indicates what it does, as apposed to the layoutIfNeeded method within an animations block.

Internet of things

Sat, 03/28/2015 - 11:26

Do you remember the time that you were not connected to the internet? When you had to call your friend instead of sending a message via Whatsapp to make an appointment. The times you had to apply for a job by writing a letter with  a pen instead of sending your Linkedin page? We all have gone through a major revolution in which we've all got a digital identity. The same revolution has started for things, right now!

Internet of Things


Things we use all day, like your watch or thermostat, get suddenly connected to the internet. These things are equipped with electronics, sensors and connectivity. Whether you like it or not, you will not escape this movement. According to Gartner there will be nearly 26 billion devices on the Internet of Things by 2020.
The potential of the Internet of Things isn’t just linking millions of devices. It is more like the potential of Google or Amazon had when building massive IT infrastructure. The IoT is about transforming business models and enabling companies to sell products in entirely new and better ways.


The definition of IoT is fairly simple. The internet of things involves `every thing that is connected to the internet`. You should exclude mobiles and computers, because they've already become mainstream.

In fact, the Internet of things is a name for a set of trends. First of all we have the trend that objects, we use all day,  are suddenly connected to the internet. For example bathroom scales that compares your weight with others.
Another trend is that devices that were already shipped with CPUs have become much more powerful when they’re connected to a network. Functionality of the hardware depends on local computing as well as powerful cloud computing. Fitness tracker, can carry out some functionality, like basic data cleaning and tabulation, on local hardware, but it depends on sophisticated machine-learning algorithms on an ever improving cloud service to offer prescription and insight.
Besides that we have the makers movement. Creating hardware isn’t for the big companies like Sony or Samsung anymore, just like creating software in the 80’s and 90’s was a privilege for IBM and Microsoft.

What’s new?

Devices that are connected to the internet aren’t new. For many years, we've connected devices to the internet, although not at that size. I think we have to compare it concepts like 'the cloud'. Before I heard the name 'the cloud' I was already using services that run on the internet, like Hotmail. But one swallow doesn't make a summer. When suddenly lots of services are running on the web, we speak of a trend and the market will give it a name to it. The same is happening with the internet of things. Suddenly all kinds of 'normal' devices are connected to the internet and we have a movement towards that direction that is unstoppable. And the possibilities it will bring are endless.

Splitting up in pieces

We can divide the internet of things into five smart categories. First of all we have the smart wearable. It is designed for a variety of purposes as well as for wear on a variety of parts of the body. Think of a bicycle helmet that detects a crash. Next to it we have the smart home. It’s goal is to make the experience of living at home more convenient and pleasant. A thermostat that learns what temperatures users like and builds a context-aware personalized schedule. A third and fourth category are the smart city and smart environment. It focuses on finding sustainable solutions to the growing problems. The last category is very interesting. It is the smart enterprise. One can think of real-time shipment tracking or smart metering solution that manages energy consumption at the individual appliance and machine level.

Changes of business models

There are plenty of new business models due to the internet of things. An example is a system that provides a sensor-embedded trash, so it’s capable of real-time context analysis and alerting the authorities when it is full and needs to be emptied. Another example is the insurance world, which will adopt the internet of things. There's already behaviour-based car insurance pricing available. Customers who agree to install a monitoring device on their car's diagnostic port, get a break on their insurance rates and pricing that varies according to usage and driving habits.

Big data

Other disciplines in software development also have interest in the new movement, like big data. Internet of things will not generate big data, but obesitas data. Did you know that within every flight, there are  3 terabytes of data generated with all the sensors in an airplane? Think about how much data will be generated, that has to be analyzed, when all buildings in a town are equipped with sensors to measure the quality of the air?


The intersection between software and the physical world becomes smaller every day and that is the result of the unstoppable movement of internet of things. It is still at the beginning. Business models are changing and there are lots of opportunities. You just have to find them...