Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator/categories/7&page=2' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

Keeping an eye on your Amazon EC2 firewall rules

Xebia Blog - Sun, 10/25/2015 - 14:56

Amazon AWS makes it really easy for anybody to create and update firewall rules that provide access to the virtual machines inside AWS. Within seconds you can add your own IP address so you can work from home or the office. However, it is also very easy to forget to remove them once your are finished. The utility aws-sg-revoker , will help you maintain your firewall rules.

aws-sg-revoker inspects all your inbound access permission and compares them with the public IP addresses of the machines in your AWS account. For grants to IP addresses not found in your account, it will generate a aws CLI revoke command. But do not be afraid: it only generates, it does not execute it directly. You may want to investigate before removal. Follow the following 4 steps to safeguard your account!

step 1. Investigate

First run the following command to generate a list of all the IP address ranges that are referenced but not in your account.

aws-sg-revoker -l x.y.z. hostname.com. a.b.c.

You may find that you have to install jq and the aws CLI :-)

step 2. Exclude known addresses

Exclude the ip addresses that are ok. These addresses are added as regular expressions.

aws-sg-revoker -l -w 1\.2\.\3\.4 -w 8\.9\.10\.11/16
step 3. generate revoke commands

Once you are happy, you can generate the revoke commands:

aws-sg-revoker -w 1\.2\.\3\.4 -w 4\.5\.6\.7 -w 8\.9\.10\.11/16

aws ec2 revoke-security-group-ingress --group-id sg-aaaaaaaa --port 22-22 --protocol tcp --cidr # revoke from sg blablbsdf
aws ec2 revoke-security-group-ingress --group-id sg-aaaaaaaa --port 9200-9200 --protocol tcp --cidr # revoke from sg blablbsdf
aws ec2 revoke-security-group-ingress --group-id sg-aaaaaaaa --port 9080-9080 --protocol tcp --cidr # revoke from sg blablbsdf
aws ec2 revoke-security-group-ingress --group-id sg-bbbbbbbb --protocol -1 --cidr # revoke from sg sg-1
aws ec2 revoke-security-group-ingress --group-id sg-bbbbbbbb --protocol -1 -cidr # revoke from sg sg-3
step 4. Execute!

If the revokes look ok, you can execute them by piping them to a shell:

aws-sg-revoker -w 1\.2\.\3\.4 -w 8\.9\.10\.11/16 | tee revoked.log | bash

This utility makes it easy to for you to regularly inspect and maintain your firewall rules and keep your AWS resources safe!

Stuff The Internet Says On Scalability For October 23rd, 2015

Hey, it's HighScalability time:

The amazing story of Voyager's walkabout and the three body problem.
If you like Stuff The Internet Says On Scalability then please consider supporting me on Patreon.
  • $18 billion: wasted on US Army Future Combat system; 70%: Americans who support an Internet sales tax;  $1.3 billion: wasted on an interoperable health record system; trillions: NSA breaking Web and VPN connections; 615: human data teams beat by a computer; $900,000: cost of apps on your smartphone 30 years ago.

  • Quotable Quotes:
    • @PatrickMcFadin: 'Sup 10x coder. Grace Hopper invented the compiler and has a US Navy destroyer named after her. Just how badass are you again?
    • @benwerd: I love Marty McFly too, but more importantly, the first transatlantic voice transmission was sent 100 years ago today. What a century.
    • Martin Goodwell: The nearly two-billion requests that Netflix receives each day result in roughly 20 billion internal API calls.
    • sigma914: It's great to see people implementing distributed services using a vertically scalable technology stack again. The past ~decade has seen a lot of "We can scale sideways so constant overheads are irrelevant! We'll just use Java and add more machines!" which, in real life, seems to leave a lot of performance on the table.
    • Eric Schmidt: The way you build great products is small teams with strong leaders who make tradeoffs and work all night to build a product that just barely works.
    • @boulderDanH: We adopted stateful services early on @VictorOps and I always worried we were crazy. Maybe not
    • @jamesallworth: "The pressure for conformity isn’t limited to car design, it affects *everything*."
    • Eric Schmidt: Hindsight is always that you make the important decisions more quickly.
    • @fromroots: Facebook bought Instagram and WhatsApp to block Chinese competitors like Tencent and Alibaba from scaling globally quickly
    • Eric Schmidt: You’ve got to have products that can scale. What’s new is that once you have that product, you can scale very quickly. Look at Uber.
    • David Ehrenberg: So, before scaling, build your plan, get your systems in place, control your cash burn, create meaningful milestones and plan for cash-flow positive. That’s the foundation to successfully scale.
    • Francis Fukuyama: Hence patrimonialism has evolved into what is called “neopatrimonialism,” in which political leaders adopt the outward forms of modern states—with bureaucracies, legal systems, elections, and the like—and yet in reality rule for private gain. 
    • @sandromancuso: I hate all these bloody Java frameworks. Why devs keep using them? No, you won’t die if you write some code yourself. 
    • Eric Schmidt: Their point was that the industry overvalues experience, and undervalues strategic and tactical flexibility.
    • @AWSUserGroupUK: Daily load fluctuates by two orders of magnitude - auto scaling architecture is essential #BMW #reinvent
    • @tpechacek: “The greatest shortcoming of the human race,” he said, “is our inability to understand the exponential function”
    • @mjpt777: +1000 "Programming with Java concurrency is like working with the inlaws. You never know will happen." - @venkat_s #jokerconf
    • Eric Schmidt: The teams are far larger than they should be. It’s a failure of architecture — the programmers don’t have the right libraries. I hope that machine learning will fix that problem.
    • Marcus Zetterquist: Start using writing your C++, Java and Javascript code using pure functions and immutability NOW. It gets super powers
    • Eric Schmidt: Companies like ours have so much cash that the main limit is opportunities to deploy it.
    • @berkson0: Loving and hating the Scaling keynote at #AWS #reinvent. All my painfully earned infrastructure experience rendered superfluous <sigh>
    • Eric Schmidt: The day we turned on the auctions, revenue tripled.
    • James S.A. Corey: Awareness is a function of the brain just like vision or motor control or language. It isn’t exempt from being broken
    • @mitchellh: Sure, but scaling linearly from millions to trillions of requests won’t scale financially. I’m talking about financial efficiency
    • Anil Ananthaswamy: It turns out that in order to anchor the self to the body, the brain has to integrate signals from within the body with external sensations, and with sensations of position and balance. When something goes wrong with brain regions that integrate all these signals, the results are even more dramatic than out-of-body experiences
    • @alemacgo: “The whole point of science is to penetrate the fog of human senses, including common sense.”
    • @themadstone: Why is life special? Bc a billionth of a billionth of a fraction of all matter in the universe is living matter.

  • Oh how the world has changed. Here's an email from 1996: Alta Vista is a very large project, requiring the cooperation of at least 5 servers, configured for searching huge indices and handling a huge Internet traffic load. The initial hardware configuration for Alta Vista is as follows... 

  • AWS has helped change the VC industry. AWS and Venture Capital. Getting a new company off the ground takes less than a few hundred thousand dollars these days. With AWS all you have are the variable costs of what you use. Gone are the days of needing to buy a bunch of servers and the people to maintain them. Old news. More interesting is because less money is now needed to start a venture more people can help ventures get started. VC incentives are aligned with companies that need to grow quickly, like an Uber. Given that a VC only needs one in ten investments or so to be a huge hit, it's best for a VC if those other nine die as fast as possible to minimize costs. This may not align with your interests if you would like grow more organically. It's unprofitable for VCs to play in the seed funding realm. VCs used to win because they had access to capital and superior information, both of which have been commoditized at today's lower funding levels and higher availability of expertise. So if you want get to heaven, you may need an Angel.

  • Here's the IPv6 carrot. Accessing Facebook can be 10-15 percent faster over IPv6. IPv6: It's time to get on board.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Agile, but still really not Agile? What Pipeline Automation can do for you. Part 2.

Xebia Blog - Thu, 10/22/2015 - 13:51

Organizations adopting Agile and teams delivering on a feature-by-feature basis producing business value at the end of every sprint. Quite possibly this is also the case in your organization. But do these features actually reach your customer at the same pace and generate business value straight away? And while we are at it: are you able to actually use feedback from your customer and apply it for use in the very next sprint?

Possibly your answer is “No”, which I see very often. Many companies have adopted the Agile way of working in their lines of business, but for some reason ‘old problems’ just do not seem to go away...

Hence the question:

“Do you fully capitalize on the benefits provided by working in an Agile manner?”

Straight forward Software Delivery Pipeline Automation might help you with that.

In this post I hope to inspire you to think about how Software Development Pipeline automation can help your company to move forward and take the next steps towards becoming a truly Agile company. Not just a company adopting Agile principles, but a company that is really positioned to respond to the ever changing environment that is our marketplace today. To explain this, I take the Agile Manifesto as a starting point and work from there.

In my previous post, I addressed Agile Principles 1 to 4, please read below where I'll explain about how automation can help you for Agile Principles 5 to 8.


Agile Principle 5: Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

This is an important aspect in Agile. People get motivated by acknowledged empowerment, responsibility, ownership and trusted support when performing a job. This is one of the reasons Agile teams often feel so vibrant and dynamic. Still, in many organizations development-teams work Agile but “subsequent teams” do not. Resulting in mini-waterfalls slowing down your delivery cycle as a whole.trust-in-hands

“Environment and the support needed” means that the Agile team should work in a creative and innovative environment where team-members can quickly test new features. Where the team can experiment, systems “just work” and “waiting” is not required. The team should be enabled, so to speak .. in terms of automation and in terms of innovation. This means that a build should not take hours, a deployment should not take days and the delivery of new infrastructure should not take weeks.

Applying rigorous automation will help you to achieve the fifth objective of the Agile manifesto. There is a bit of a chicken and egg situation here, but I feel it is safe to say that a sloppy, broken, quirky development environment will not help in raising the bar in terms of motivating individuals. Hence "give them the environment and support they need, and trust them to get the job done".


Agile Principle 6: The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

When working Agile, individuals and interactions are valued over the use of processes and tools. When starting a new project, teams should not be hindered with ticketing systems, extensive documentation to explain themselves and long service times. These type of “services” often exist on boundaries of business-units of bringing different ‘disciplines’ to the solution.500px-People_together.svg

Although working Agile, many companies still have these boundaries in place. An important aspect of Continuous Delivery is executing work in Product teams dedicated to delivery and/or maintenance of an end-product. These product teams have all required disciplines working together in one and the same team. Operating in this manner alleviates the need for slow tooling & ticketing systems and inspires people to work together and get the job done.

Organizing people as a team working on a product instead of individuals performing a task, which in itself has no meaning, will help you to achieve the sixth objective of the Agile Manifesto. There is not a lot automation can do for you here.


Agile Principle 7: Working software is the primary measure of progress.

Agile aims towards delivering working software at the end of each sprint. For the customer that is basically what counts: working software, which can actually be used. Working software means software without defects. There is no point in delivering broken software at the end of every sprint.Working-Software

When sending a continuous flow of new functions to the customer, each function should adhere to the required quality level straight away. In terms of quality, new functions might need to be ‘reliable’, ‘secure’, maintainable’, ‘fast’, etc, which all are fully testable properties. Testing these type of properties should be integral part of team activities. One of the principles related to Continuous Delivery addresses this topic through Test automation. Without it, it is not possible to deliver working production-ready software by the end of each sprint.

Proper implementation of test-disciplines, fostering a culture of delivering high quality software, testing every single feature, adhering to testing disciplines and applying matching & automated test tooling addresses topics related to the seventh object of the Agile Manifesto. Supply a test for every function you add to the product and automate this test.


Agile Principle 8: Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

As software complexity grows exponentially, it will become more difficult overtime to maintain a constant pace of delivering new features, when assembling, deploying, testing or provision manually. Humans are simply not made to perform multiple tasks fast, repetitively and consistently over a longer period of time, that is what machines are for!

The eighth Agile principle typically comes down to a concept called ‘flow’. You might have an Agile team in place for creating new softflow-clipart-9183-flow-designware, but what about the flow in the rest of your organization? Should the team wait for requirements to drip through, should it wait for the testers to manually test software, or is it the Operations team that needs to free resources in order to deploy software? To address this, from idea to product, handover moments should be minimized as much as possible and where possible principles of automation should be applied. This brings us back to build automation, test automation, deployment automation and the automation of infrastructure.


Stay tuned for the next post, where I'll address the final four Agile principles of the bunch.


Michiel Sens.

5 Lessons from 5 Years of Building Instagram

Instagram has always been generous in sharing their accumulated wisdom. Just take a look at the Related Articles section of this post to see how generous.

The tradition continues. Mike Krieger, Instagram co-founder, wrote a really good article on lessons learned from milestones achieved during Five Years of Building Instagram. Here's a summary of the lessons, but the article goes into much more of the connective tissue and is well worth reading.

  1. Do the simple thing first. This is the secret of supporting exponential growth. There's no need to future proof everything you do. That leads to paralysis. For each new challenge find the fastest, simplest fix for each. 
  2. Do fewer things better. Focus on a single platform. This allows you to iterate faster because not everything has to be done twice. When you have to expand create a team explicitly for each platform.
  3. Upfront work but can pay huge dividends. Create an automated scriptable infrastructure implementing a repeatable server provisioning process. This makes it easier to bring on new hires and handle disasters. Hire engineers with the right stuff who aren't afraid to work through a disaster. 
  4. Don’t reinvent the wheel. Instagram moved to Facebook's infrastructure because it allowed them to stay small and leverage a treasure trove of capabilities.
  5. Nothing lasts forever. Be open to evolve your product. Don't be afraid of creating special teams to tackle features and adapt to a rapidly scaling community.
Related Articles
Categories: Architecture

Preparing hands-on conferences: to USB or not to USB

Xebia Blog - Tue, 10/20/2015 - 10:00

On Friday, October 2nd, Xebia organized the inaugural edition of TestWorks Conf. The conference was born out of the apparent need for a hands-on test automation conference in the Netherlands. Early on, we decided that having a high level of engagement from the participants was key in achieving this. Thus, the idea of making everything hands-on was born. Not only did we have workshops throughout the day, people also should be enabled to code along with the speakers during talks.This however posed a challenge on the logistical side of things: How to make sure that everyone has the right tooling and code available on their laptops?

Constraints of a possible solution

Just getting all code on people's machines is not sufficient. As we already learned during our open kitchen events, there is always some edge case causing problems on a particular machine. In order to let participants jump straight into the essentials of the workshop, it would need to meet the following requirements:

  • Take at most 10 minutes to be up and running for everyone
  • Require no internet connection
  • Be identical on every machine
  • Not be intrusive on people's machines
Virtual machines vs local installations?

The first decision we had to make was opting for local installations or installing a virtual machine. Based on the requirements, a local installation would require participants to atleast install all the software beforehand, as the list of required software is quite large (could easily take 60+ minutes). If we were to go this route, we would have to build a custom installer to make sure everyone has all the contents. Having built deployment packages for a Windows domain in the past, this takes a lot of time to get right. Especially if we would need to support multiple platforms. Going down this route, it's questionable if we could satisfy the final requirement. What happens if software we install overrides specific custom settings a user has done? Will the uninstallers revert this properly? This convinced us that using a VM was the way to go.

Provisioning the virtual machine

In order to have all the contents and configuration of the VM under version control, we decided to provision it using Vagrant. This way we could easily synchronize changes in the VM between the speakers while preparing for the workshops and talks. It also posed a nice dillemma. How far will you go in automating the provisioning? Should you provision application specific settings? Or just set these by hand before exporting the VM? In the end, we decided to have a small list of manual actions:

  • Importing all projects in IntelliJ, so all the required indexing is done
  • Putting the relevant shortcuts on the Ubuntu launcher
  • Installing required IntelliJ plugins
  • Setting the desired Atom theme
Distributing the virtual machine

So, now we have a VM, but how do we get it into everybody's hands? We could ask everyone to provision their own beforehand using Vagrant. However, this would require additional work on the Vagrant scripts (so they're robust enough to be sent out into the wild), and we would need to automate all the manual steps. Secondly, it would require everybody to actually do these preparations. What if 30 people didn't and start downloading 5ish GB simultaneously at the start of the conference? This would probably grind the Internet to a halt at the venue.

Because of this, we decided to make an export of the VM image and copy this to a USB thumbdrive, together with installers for VirtualBox for multiple platforms. Every participant would be asked as preparation to install VirtualBox, and would receive the thumbdrive when registering at the conference. The only step left would be to copy all the contents to 180 thumbdrives. No problem right?

Flashing large batches of USB drives

The theory of flashing the USB drives was easy. Get some USB hubs with a lot of ports, plug all the USB drives and flash an image to all the drives. However, practice has proved different.

First of all, what filesystem should we use? Since we're striving for maximal compatibility, FAT32 would be preferred. This however was not feasible, since FAT32 has a file size limit of 4GB, and our VM grew to well over 5GB. This leaves two options: ExFAT or NTFS. ExFAT works by default on OSX and Windows, but requires an additional package to be installed under Linux. NTFS works by default under Windows and Linux, but is readonly under OSX. Since users would not have to write to the drives, NTFS seemed the best choice.

Having to format the initial drive as NTFS, we opted for using a Windows desktop. After creating the first drive, we created an image from this drive which was to be copied to all the remaining drives.

This got us to plug in 26 drives in total (13 per hub), all ready to start copying data. Only to find out that the drive letter assignment that windows does is a bit outdated :)


When you run out of available drive letters, you have to use a NTFS volume mount point. The software we used for cloning the USB drives (imageUSB) would not recognize these mount points as drives however, so this put a limit on the amount of drives to flash at once. When we actually flashed the drives, both hubs turned out to be faulty, disconnecting the drives at random causing the writes to fail. This lead us to spread the effort of flashing the drives over multiple people (thanks Kishen & Viktor!), as we could do less per machine.

Just copying the data is not sufficient however. We have to verify that the data which was written can be read back as such. During this verification, several USB drives turned out to be faulty. After plugging in and out a lot of drives, this was the result:


What did we learn?

During the conference, it turned out that using the USB drives and virtual machine image (mostly) worked out as planned. There were issues, but they were manageable and usually easy to resolve. To sum up the most important points:

  • Some vendors disable the CPU virtualization extensions by default in the BIOS/UEFI. These need to be enabled for the VM to work
  • The USB drives were fairly slow. Using faster drives would've smoothed things out more
  • A 64-bit VM does not work on a 32-bit host :)
  • Some machines ran out of memory. The machine was configured to have 2GB, this was probably slightly too low.
  • Setting up the machines was generally pain-free, but it would still be good to reserve some time at the beginning of the conference for this.
  • A combination of using USB drives together with providing the entire image beforehand is probably the sweet spot

Next year we will host TestWorks Conf again. What we learned here will help us to deliver an even better hands-on experience for all participants.


Segment: Rebuilding Our Infrastructure with Docker, ECS, and Terraform

This is a guest repost from Calvin French-Owen, CTO/Co-Founder of Segment

In Segment’s early days, our infrastructure was pretty hacked together. We provisioned instances through the AWS UI, had a graveyard of unused AMIs, and configuration was implemented three different ways.

As the business started taking off, we grew the size of the eng team and the complexity of our architecture. But working with production was still limited to a handful of folks who knew the arcane gotchas. We’d been improving the process incrementally, but we needed to give our infrastructure a deeper overhaul to keep moving quickly.

So a few months ago, we sat down and asked ourselves: “What would an infrastructure setup look like if we designed it today?”

Over the course of 10 weeks, we completely re-worked our infrastructure. We retired nearly every single instance and old config, moved our services to run in Docker containers, and switched over to use fresh AWS accounts.

We spent a lot of time thinking about how we could make a production setup that’s auditable, simple, and easy to use–while still allowing for the flexibility to scale and grow.

Here’s our solution.

Separate AWS Accounts
Categories: Architecture

Stuff The Internet Says On Scalability For October 16th, 2015

Hey, it's HighScalability time:

The other world beauty of the world's largest underground Neutrino Detector. Yes, this is a real thing.
If you like Stuff The Internet Says On Scalability then please consider supporting me on Patreon.
  • 170,000: depression era photos; $465m: amount lost due to a software bug; 368,778: likes in 4 hours as a reaction to Mark Zuckerberg's post on Reactions; 1.8 billion: pictures uploaded every day; 158: # of families generously volunteering to privately fund US elections.

  • Quotable Quotes:
    • @PreetamJinka: I want to run a 2 TB #golang program with 100 vCPUs on an AWS X1 instance.
    • Richard Stallman: The computer industry is the only industry that is more fashion-driven than women's fashion.
    • The evolution of bottlenecks in the Big Data ecosystem: Seeing all these efforts to bypass the garbage collector, we are entitled to wonder why we use a platform whose main asset is to offer a managed memory, if it is to avoid using it?
    • James Hamilton: Services like Lambda that abstract away servers entirely make it even easier to run alternative instruction set architectures.
    • @adrianfcole: Q: Are we losing money? A: Can't answer that, but I can tell you what average CPU usage was 5ish mins ago..
    • h4waii: Because you can't buy trust through an acquisition. You build trust, you don't transfer it through a merger.
    • @mathiasverraes: TIL Ada Lovelace was not only the world's first programmer, she was also the first debugger, fixing a flaw in an algorithm by Babbage.
    • @BenedictEvans: Ways to think about scale: iOS is as big  as BMW, Mercedes, Lexus & Audi combined
    • @caitie: Really enjoyed The Martian, also began thinking about how space is the true test of any distributed system
    • Bits or Pieces?: This is the point, there are two very distinct forms of disruption. It's not all the same despite everyone treating is as such. Alas, people ignore this.
    • Julien CROUZET: So when you have a function or callback that’ll be called repeatedly, try to make it under 600 characters (or your tweaked value), you’ll have a quick win !
    • exelius: They're the walking dead because they pursued scale over innovation. Once they had achieved scale, they found themselves with too much momentum to innovate. So because they couldn't innovate, they built an army of consultants to hawk their wares to customers who also valued scale. 

  • Will Amazon automatically win the IoT space with their recent announcement? Not so fast says Greg Ferro: AWS IoT vs Cisco Fog Computing – Cloud vs Network IoT: AWS is popular with capital poor, low ARPU and fast moving companies in the consumer market. Cisco et al is popular with high net worth conglomerates who build high value, high profit solutions that are slow moving and built on incumbent positions with known and trustable technology partners. There is a market for both types of approaches. One does not “kill” the other, nor it one better or worse, but does limit possible growth and ability to dominate the market.

  • Conway's Law is being used less descriptively these days and more prescriptively. Projects are choosing the organizational structure that creates the software they want to make. From disutopia to utopia. 

  • In a single day Riot chat servers can route a billion events (presences, messages, and IQ stanzas) and process millions of REST queries. Here are lots of lovely details on the League of Legends chat service architecture and how it works. It's based on Erlang  and XMPP, leveraging the OTP framework, concurrency model, and fault-tolerance semantics. For the heaviest string manipulation parts they dropped into C, which save 60% on CPU and lots of per session memory. Chat clusters are independent, but they do share a few tables that are replicated and reside in memory. Riak is used for the database. Also, JENKINS, DOCKER, PROXIES, AND COMPOSE.

  • Brother, can you spare a dime so I can scale my website? That's all it took to handle 60K unique visitors on Amazon's Lambda + S3, less than a dime. No doubt the original architecture could have worked with a few tweaks, but the point here is using JAWS did work. Though a problem I've had with Lambda is the lack of the idea of a session when you have a single page app.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Save some bandwidth by turning off TCP Timestamps

This is a guest post by Donatas Abraitis, System Engineer at Vinted, with an unusual approach for saving a little bandwidth.

Looking at https://tools.ietf.org/html/rfc1323 there is a nice title: 'TCP Extensions for High Performance'. It's worth to take a look at date May 1992. Timestamps option may appear in any data or ACK segment, adding 12 bytes to the 20-byte TCP header. 

Using TCP options, the sender places a timestamp in each data segment, and the receiver reflects these timestamps back in ACK segments. Then a single subtract gives the sender an accurate RTT measurement for every ACK segment.

To prove this let's dig into kernel source:

./include/net/tcp.h:#define TCPOLEN_TSTAMP_ALIGNED    12
./net/ipv4/tcp_output.c:static void tcp_connect_init(struct sock *sk)
  tp->tcp_header_len = sizeof(struct tcphdr) +
    (sysctl_tcp_timestamps ? TCPOLEN_TSTAMP_ALIGNED : 0);
Some visualizations:
Categories: Architecture

Using golang to integrate Pingdom checks into a Cachet status page

Agile Testing - Grig Gheorghiu - Wed, 10/14/2015 - 00:29
Recently I've been looking at a decent open source status page system for our API endpoints. After looking around, I decided to give Cachet a try. Installing Cachet on an Ubuntu 14.04 box is not hard, but not trivial either; you need to spend a bit of a time on learning how to deploy a php/laravel/composer system. I used the official Cachet installation doc, as well as this top-notch Digital Ocean tutorial on installing nginx and php5-fpm on Ubuntu.

We already use Pingdom as an external monitoring system for our API endpoints, so it made sense to try integrating our existing Pingdom checks into Cachet. Pingdom support for 3rd party services is very limited though. They do offer a way to call a webhook as part of an alerting policy, but AFAICT it's not possible to customize that call with authentication information, which is what 3rd party APIs usually require.

I ended up rolling my own short and sweet golang program that combines russellcardullo's go-pingdom package with making Cachet API calls (it would be cool if somebody came up with a Cachet golang binding, and not hard, but I am punting on that for now.)

To start with, you need to create a Pingdom Application Key. It's not obvious how this is done. You need to go to your my.pingdom.com, click the Sharing icon in the left vertical menu, then click the Pingdom API link, then click the Register Application button. At that point you give your application a name and you get a key. Not sure why they make it so hard.

You also need a Cachet API Token. This is available in the admin user's profile page.

Two main entities in Cachet are Components and Incidents. Components correspond roughly to the Web pages or API endpoints whose status you want to display. Incidents correspond to the results of Pingdom checks for a particular Web page or API endpoint.

I defined some constants in my golang code to make things easier to read:

const PINGDOM_PASS="xxxx"
const PINGDOM_API_KEY="xxxx"

const CACHET_API_KEY="xxxx"
const CACHET_INCIDENT_URL="https://status.mycompany.com/api/v1/incidents"
const CACHET_COMPONENT_URL="https://status.mycompany.com/api/v1/components"


const INCIDENT_FIXED = "4"

The component and incident status values above are described in the Cachet documentation.

Here is a code snippet that determines the status of a Cachet component:

func get_cachet_component_status(component_id string) string {
url := fmt.Sprintf("%s/%s", CACHET_COMPONENT_URL, component_id)
        fmt.Printf("Sending request to %s\n", url)
status_code, json_data := send_http_req_query_string("GET", url, token, nil)
if status_code != 200 {
return ""
component_status := get_nested_item_property(json_data, "data", "status")
        return component_status

I use a couple of helper functions that I found useful, swiped from an integration test suite I wrote. For making HTTP calls, I found the grequests package, which is a golang port of the Python requests library, extremely useful. Here is send_http_req_query_string which in this case makes a GET call to the Cachet API and also sets the X-Cachet-Token header for authentication purposes.

func send_http_req_query_string(req_type, url, token string, query_string map[string]string) (int, map[string]interface{}) {
    ro := &grequests.RequestOptions{}
    ro.Headers = map[string]string{"X-Cachet-Token":token}
    if query_string != nil {
        ro.Params = query_string
    var resp *grequests.Response
    return _http_response(req_type, url, ro, resp)

This in turn calls _http_response:

func _http_response(req_type, url string, ro *grequests.RequestOptions, resp *grequests.Response) (int, map[string]interface{}) {
switch req_type {
case "POST":
resp, _ = grequests.Post(url, ro)
case "PUT":
resp, _ = grequests.Put(url, ro)
case "PATCH":
resp, _ = grequests.Patch(url, ro)
case "DELETE":
resp, _ = grequests.Delete(url, ro)
case "GET":
resp, _ = grequests.Get(url, ro)
fmt.Printf("HTTP method %s not recognized\n", req_type)
return 0, nil

fmt.Printf("Sending HTTP %s request to: %s\n", req_type, url)
var json_data map[string]interface{}
status_code := resp.StatusCode
err := resp.JSON(&json_data)
if err != nil {
fmt.Println("Unable to coerce to JSON", err)
return 0, nil
return status_code, json_data

One other function highlighted in get_cachet_component_status is get_nested_item_property which is useful for getting a value out of a JSON nested dictionary such as 'data' below:

"data": {
"id": 1,
"name": "API",
"description": "This is the Cachet API.",
"link": "",
"status": 1,
"order": 0,
"group_id": 0,
"created_at": "2015-07-24 14:42:10",
"updated_at": "2015-07-24 14:42:10",
"deleted_at": null,
"status_name": "Operational"

Here is the get_nested_item_property function:

func get_nested_item_property(json_data map[string]interface{}, item_name, item_property string) string {
item := json_data[item_name]
prop := item.(map[string]interface{})[item_property]
return prop.(string)

This is a useful example of golang code which converts a JSON value to a map[string]interface{}, grabs another value out of that map and converts that value to string.

Here is an example of a PUT call which updates the status of a Cachet component:

func update_cachet_component_status(component_id, status string ) (int, map[string]interface{}) {
url := fmt.Sprintf("%s/%s", CACHET_COMPONENT_URL, component_id)
        token := CACHET_API_KEY
        fmt.Printf("Sending request to %s\n", url)

        payload := map[string]string{
                "id":  component_id,
                "status": status,

        status_code, json_data := send_http_req_json_body("PUT", url, token, payload)
        return status_code, json_data

This function calls another helper function useful for POST-ing or PUT-ing JSON payloads to an API endpoint. Here is that function, which also uses the grequests package:

func send_http_req_json_body(req_type, url, token string, json_body interface{}) (int, map[string]interface{}) {
    ro := &grequests.RequestOptions{}
    if json_body != nil {
        ro.JSON = json_body
    ro.Headers = map[string]string{"X-Cachet-Token":token}
    var resp *grequests.Response
    return _http_response(req_type, url, ro, resp)

In order to get the details of a given Pingdom check, I used the go-pingdom package:

func get_pingdom_check_details(client *pingdom.Client, check_id int) (string, string, string) {
check_details, err := client.Checks.Read(check_id)
if err != nil {
return "", "", ""
fmt.Println("ID:", check_details.ID)
fmt.Println("Name:", check_details.Name)
fmt.Println("Resolution:", check_details.Resolution)
fmt.Println("Created:", time.Unix(check_details.Created, 0))
fmt.Println("Hostname:", check_details.Hostname)
fmt.Println("Status:", check_details.Status)
fmt.Println("LastErrorTime:", time.Unix(check_details.LastErrorTime, 0))
fmt.Println("LastTestTime:", time.Unix(check_details.LastTestTime, 0))
fmt.Println("LastResponseTime:", check_details.LastResponseTime)
fmt.Println("Paused:", check_details.Paused)
fmt.Println("ContactIds:", check_details.ContactIds)
hostname := check_details.Hostname
status := check_details.Status
message := fmt.Sprintf("LastErrorTime: %v\nLastTestTime: %v\n",
time.Unix(check_details.LastErrorTime, 0),
time.Unix(check_details.LastTestTime, 0))
return hostname, status, message

Note that I only return a few of the fields available in the go-pingdom CheckResponse struct (which is defined here).

The main function in my check_pingdom_and_post_cachet_status.go program does the following for a given mapping of Cachet components to Pingdom checks:

  • check status of Cachet component
  • check status of Pingdom check
  • if Pingdom check is down, create new incident for the given component and set the status of the incident to INCIDENT_IDENTIFIED; also update the given components and set its status to COMPONENT_PARTIAL_OUTAGE
  • if Pingdom check is up
    • if the component status is already COMPONENT_OPERATIONAL, do nothing (we don't want consecutive 'healthy' status updates)
    • otherwise, create new incident for the given component and set the status of the incident to INCIDENT_FIXED; also update the given components and set its status to COMPONENT_OPERATIONAL
Finally, I built a go binary (via 'go build') out of 2 files I have: check_pingdom_and_post_cachet_status.go and utils.go.

I scheduled the resulting binary to run out of cron every 5 minutes.

For the curious, here is a gist with the code for check_pingdom_and_post_cachet_status.go and utils.go.

More concurrency: Improved locking in PostgreSQL

If you want to build a large scale website, scaling out the webserver is not enough. It is also necessary to cleverly manage the database side. a key to high scalability is locking.

In PostgreSQL we got a couple of new cool features to reduce locking and to speed up things due to improved concurrency.

General recommendations: Before attacking locking, however, it makes sense to check what is really going on on your PostgreSQL database server. To do so I recommend to take a look at pg_stat_statements and to carefully track down bottlenecks. Here is how it works:

Categories: Architecture

Sponsored Post: IStreamPlanet, Close.Io, Instrumental, Location Labs, Surge, Redis Labs, Jut.Io, VoltDB, Datadog, SignalFx, InMemory.Net, VividCortex, MemSQL, Scalyr, AiScaler, AppDynamics, ManageEngine, Site24x7

Who's Hiring?
  • As a Networking & Systems Software Engineer at iStreamPlanet you’ll be driving the design and implementation of a high-throughput video distribution system. Our cloud-based approach to video streaming requires terabytes of high-definition video routed throughout the world. You will work in a highly-collaborative, agile environment that thrives on success and eats big challenges for lunch. Please apply here.

  • As a Scalable Storage Software Engineer at iStreamPlanet you’ll be driving the design and implementation of numerous storage systems including software services, analytics and video archival. Our cloud-based approach to world-wide video streaming requires performant, scalable, and reliable storage and processing of data. You will work on small, collaborative teams to solve big problems, where you can see the impact of your work on the business. Please apply here.

  • Close.io is a *profitable* fast-growing SaaS startup looking for a Lead DevOps/Infrastructure engineer to join our ~10 person team in Palo Alto or *remotely*. Come help us improve API performance, tune our databases, tighten up security, setup autoscaling, make deployments faster and safer, scale our MongoDB/Elasticsearch/MySQL/Redis data stores, setup centralized logging, instrument our app with metric collection, set up better monitoring, etc. Learn more and apply here.

  • Location Labs is the global pioneer in mobile security for humans. Our services are used by millions of monthly paying subscribers worldwide. We were named one of Entrepreneur magazine’s “most brilliant” companies and TechCrunch said we’ve “cracked the code” for mobile monetization. If you are someone who enjoys the scrappy, get your hands dirty atmosphere of a startup, but has the measured patience and practices to keep things robust, well documented, and repeatable, Location Labs is the place for you. Please apply here.

  • VoltDB's in-memory SQL database combines streaming analytics with transaction processing in a single, horizontal scale-out platform. Customers use VoltDB to build applications that process streaming data the instant it arrives to make immediate, per-event, context-aware decisions. If you want to join our ground-breaking engineering team and make a real impact, apply here.  

  • At Scalyr, we're analyzing multi-gigabyte server logs in a fraction of a second. That requires serious innovation in every part of the technology stack, from frontend to backend. Help us push the envelope on low-latency browser applications, high-speed data processing, and reliable distributed systems. Help extract meaningful data from live servers and present it to users in meaningful ways. At Scalyr, you’ll learn new things, and invent a few of your own. Learn more and apply.

  • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here.

  • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here.
Fun and Informative Events
  • Surge 2015. Want to mingle with some of the leading practitioners in the scalability, performance, and web operations space? Looking for a conference that isn't just about pitching you highly polished success stories, but that actually puts an emphasis on learning from real world experiences, including failures? Surge is the conference for you.

  • Your event could be here. How cool is that?
Cool Products and Services
  • Instrumental is a hosted real-time application monitoring platform. In the words of one of our customers: "Instrumental is the first place we look when an issue occurs. Graphite was always the last place we looked."

  • Real-time correlation across your logs, metrics and events.  Jut.io just released its operations data hub into beta and we are already streaming in billions of log, metric and event data points each day. Using our streaming analytics platform, you can get real-time monitoring of your application performance, deep troubleshooting, and even product analytics. We allow you to easily aggregate logs and metrics by micro-service, calculate percentiles and moving window averages, forecast anomalies, and create interactive views for your whole organization. Try it for free, at any scale.

  • Datadog is a monitoring service for scaling cloud infrastructures that bridges together data from servers, databases, apps and other tools. Datadog provides Dev and Ops teams with insights from their cloud environments that keep applications running smoothly. Datadog is available for a 14 day free trial at datadoghq.com.

  • Turn chaotic logs and metrics into actionable data. Scalyr replaces all your tools for monitoring and analyzing logs and system metrics. Imagine being able to pinpoint and resolve operations issues without juggling multiple tools and tabs. Get visibility into your production systems: log aggregation, server metrics, monitoring, intelligent alerting, dashboards, and more. Trusted by companies like Codecademy and InsideSales. Learn more and get started with an easy 2-minute setup. Or see how Scalyr is different if you're looking for a Splunk alternative or Sumo Logic alternative.

  • SignalFx: just launched an advanced monitoring platform for modern applications that's already processing 10s of billions of data points per day. SignalFx lets you create custom analytics pipelines on metrics data collected from thousands or more sources to create meaningful aggregations--such as percentiles, moving averages and growth rates--within seconds of receiving data. Start a free 30-day trial!

  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net

  • VividCortex goes beyond monitoring and measures the system's work on your servers, providing unparalleled insight and query-level analysis. This unique approach ultimately enables your team to work more effectively, ship more often, and delight more customers.

  • MemSQL provides a distributed in-memory database for high value data. It's designed to handle extreme data ingest and store the data for real-time, streaming and historical analysis using SQL. MemSQL also cost effectively supports both application and ad-hoc queries concurrently across all data. Start a free 30 day trial here: http://www.memsql.com/

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Also available on Amazon Web Services. Free instant trial, 2 hours of FREE deployment support, no sign-up required. http://aiscaler.com

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Categories: Architecture

Security is dead, long live security

Xebia Blog - Mon, 10/12/2015 - 19:30

Last week the 7th edition of BruCON was held. For those unfamiliar with it, BruCON is a security conference where everybody with an interest in security can share their views and findings. As always it was a great mixture of technology, philosophy, personal opinions and hands-on workshops.

This year however I noticed a certain pattern in some of the talks. Chris Nickerson gave a presentation about "how to make a pentester's life hell" based on experience, Shyma Rose shared her views on risk management, Mark Hillick showed us how the security was improved at Riot Games and David Kennedy provided his opinion on the state of the information security industry nowadays. All four of them basically told pieces of the same tale from a different perspective and I will try to provide my viewpoint on the matter in this blog.

The security bubble

Both Shyma and Dave said the term 'Risk' is inflated and is nowadays used as a buzz word that no longer has a connection with actual threats. I couldn't agree more on this. Nowadays it is almost normal, when someone identifies a new vulnerability, to launch a complete marketing campaign including fancy names and logos, dedicated websites and huge social media presence. Risk is no longer used as an indicator of 'badness', but instead used as a catalyst for pushing 'money making silver bullets' to customers. And, since most clients don't know any better, they get away with it. And, as Chris showed, even customers who do know better, still enable them to push their crappy services by providing them ideal conditions to prove their effectiveness.

Hackers != unstoppable evil geniuses

Hackers are looked upon as the extremely smart guys with elite skills, where reality is that most breaches happen due to stupid stuff and decade old problems. The infosec industry's solution is products and services that no longer qualify for the fast changing world we now live in. Most services rely on stopping or identifying known attacks. In a world that is changing almost every heartbeat and especially in a world of mobile devices and cloud solutions the 'castle and archers' approach no longer works. Facts show that in many hacks exploits weren't even necessary due to the possibilities of modern platforms. If an attacker has the possibility to access some maintenance or configuration part of your system it's game over. If an attacker can access some scripting environment, it's game over. If an attacker can lure one of your employers into going to a website or installing something, it's game over.

Ivory Towers

Another problem is the huge gap between security operations inside a company and the business and development departments. Many companies have internal security guidelines that are hardly aligned with the rest of the organization and therefor bypassed or ignored. The natural response to this is that the security departments push the guidelines ever harder, only causing the gap to increase even more. Based on experience Mark stated that security departments should get out of their ivory tower and start to understand what is really important. It's more effective to achieve 80% security with 100% alignment, than try to reach 100% security with 0% alignment.


Both the infosec industry and clients nowadays have enough money and attention to change things, so we should get rid of the technology driven approach and start focusing on talent and smartness. When you look at the root causes of many hacks it's not the technology that is to blame, but instead ego, culture, miscommunication and the working environment. As long as security is considered as something you can bolt on or use external expertise for it will fail. We, both the suppliers and clients, should consider security as a standard quality attribute where everybody is responsible for.

Telling instead of training

In most companies the ratio between security and non-security minded people is way off. Security teams should therefor start acting as supporters and trainers. By becoming more visible in the organization and start aligning with it, the security awareness will rise within everyone. Every single person in the company should get a basic understanding of what security is about. And it isn't that hard to achieve. Developers should know secure coding, testers should learn to use security tooling, operations should know how hacking tools work and can be identified and taught the basics of forensic research. People also need to be trained to how to handle in case of an issue: build a good incident response program with flowcharts that everybody can use and apply. It's not rocket science, you can achieve a lot with good old common sense.


Another key item is visibility. Often incidents, breaches and other security related issues are 'kept under the radar' and only 'the chosen few' will know the details. By being open and transparent about these to the whole organization, people will start to understand the importance and challenge each other to prevent these in the future. By creating internal security challenges and promoting good ideas a community will form on itself. Use leaderboards and reward with goodies to stimulate people to improve themselves and get accustomed with the matter. Make sure successes are acknowledged. To quote Mark (who also quoted someone else) "If Tetris has taught me anything, it’s that errors pile up and accomplishments disappear."

Hackers don't only knock on the front door

Lastly start to implement defense by default and assess every situation as if a breach had occurred. Assume bad stuff will happen at some point and see how you can minimize the damage from each point. Do this on all levels; disable local admin accounts, use application protection like EMET and Applocker, implement strict password policies, apply network segmentation between byod, office automation and backends, using coding frameworks, patch all the time, test everything, monitor everything, and start analyzing your external and internal network traffic. The ultimate goal is to make it pentesters (and therefor hackers) as difficult as possible. Pentesters should cry and require weeks, months or even years to get somewhere.

There is no I in team!

We, as an infosec industry, are facing a future where change is the constant factor and we have find a way to deal with that. In order to be successful, we have to understand and acknowledge that we can no longer do it on our own. Unless we start to behave as a member of the team, we will fail horribly and become sitting ducks.


Making the Case for Building Scalable Stateful Services in the Modern Era

For a long time now stateless services have been the royal road to scalability. Nearly every treatise on scalability declares statelessness as the best practices approved method for building scalable systems. A stateless architecture is easy to scale horizontally and only requires simple round-robin load balancing.

What’s not to love? Perhaps the increased latency from the roundtrips to the database. Or maybe the complexity of the caching layer required to hide database latency problems. Or even the troublesome consistency issues.

But what of stateful services? Isn’t preserving identity by shipping functions to data instead of shipping data to functions a better approach? It often is, but we don’t hear much about how to build stateful services. In fact, do a search and there’s very little in the way of a systematic approach to building stateful services. Wikipedia doesn’t even have an entry for stateful service.

Caitie McCaffrey, Tech Lead for Observability at Twitter, is fixing all that with a refreshing talk she gave at the Strange Loop conference on Building Scalable Stateful Services (slides).

Refreshing because I’ve never quite heard of building stateful services in the way Caitie talks about building them. You’ll recognize most of the ideas--Sticky Sessions, Data Shipping Paradigm, Function Shipping Paradigm, Data Locality, CAP, Cluster Membership, Gossip Protocols, Consistent Hashing, DHT---but she weaves them around the theme of building stateful services in a most compelling way.

The highlight of the talk for me is when Caitie ties the whole talk together around the discussion of her experiences developing Halo 4 using Microsoft’s Orleans on top of Azure. Orleans doesn’t get enough coverage. It’s based on an inherently stateful distributed virtual Actor model; a highly available Gossip Protocol is used for cluster membership; and a two tier system of Consistent Hashing plus a Distributed Hash Table is used for work distribution. With this approach Orleans can rebalance a cluster when a node fails, or capacity is added/contracted, or a node becomes hot. The result is Halo was able to run a stateful Orleans cluster in production at 90-95% CPU utilization across the cluster.

Orleans isn't the only example system covered. Facebook's Scuba and Uber's Ringpop are also analyzed using Caitie's stateful architecture framework. There's also a very interesting section on how Facebook cleverly implements fast database restarts for large in-memory databases by decoupling the memory lifetime from the process lifetime.

So let’s jump in and learn how to build stateful services...

Stateless Services are Wasteful
Categories: Architecture

Refactoring a monolith to Microservices

Xebia Blog - Mon, 10/12/2015 - 16:37

For a training on Microservices that is currently under development at Xebia, we've created implementations of a web shop in both a monolithic and Microservices architecture. We then used these examples in a couple of workshops to explain a number of Microservices concepts (see here and here). In this post we will describe the process we followed to move from a monolith to services, and what we learned along the way.

First we built ourselves a monolithic web shop. It's a simple application that offers a set of HTTP interfaces that accept and produce JSON documents. You can find the result here on Github. The class com.xebia.msa.rest.ScenarioTest shows how the REST calls can be used to support a user interface (we didn't actually build a user interface and hope you can imagine one based on the ScenarioTest code...).

So this was our starting point. How should we split this monolith into smaller pieces? To answer this question we started out defining our services in a Post It-saturated design session. We used event storming to find out what the boundaries of our services should be. A good place to start reading on event storming is Ziobrando's blog. The result is shown in the picture below.

Results of event storming session

We came up with seven aggregates that were translated into software in four services: Catalog, Shop, Payment and Fulfillment. We decided those four services would be enough to start with. One service calls another using REST, exchanging JSON documents over HTTP. The code can be found here. Our process to move from a monolith to services was to copy the monolith code four times, followed by stripping out unnecessary code from each service. This results in some initial duplication of classes, but gradually the domain models of the services started drifting apart. An Order in the context of Payment is really very simple compared to an Order in the context of Shop, so lots of detail can be removed. Spring JSON support helps a great deal because it allows you to just ignore JSON that doesn't fit the target class model: the complex Order in Shop can be parsed by the bare-bones Order in Payment.

Though the new solution will probably work well, we weren't quite satisfied. What would happen for instance if one of the services became unavailable? In our implementation this would mean that the site would be down; no Payment -> no Order. This is a pity because in practice a shop may want to accept an Order and send an invoice to be paid later. For inspiration on integration patterns refer to Gero's blog.. Our solution was still tightly coupled, not at design time, but at runtime.

To fix this problem we decided to place queues between services: one service may not call another service but only send out messages on queues. The result can be found here. This solution looks more like the diagram below.
Shop services interacting with events
Events in the diagram correspond to events in the code: a service informs the world that something interesting has happened, like a new Order is completed or a Payment is received. Other services register their interest in certain types of event and pick up processing when needed, corresponding to pattern #3 in Gero's blog.

This architecture is more robust in the sense that it can handle delays and un-availability of parts of the infrastructure. This comes at a price though:

  1. A user interface becomes more complex. Customers will expect a complete view of their order including payment information, but this is based on UI parts under control of different services. The question now becomes how to create a UI that looks consistent to end users while it is still robust and respects service boundaries.
  2. What happens to data about orders that is stored by Shop if something goes wrong later in the process? Imagine that a customer completes an order, is presented with a payment interface and then fails to actually pay? This means Shop could be left with stale Orders, so we may need some reporting on that to allow a sales rep to follow up with the customer or a batch job to just archive old orders.
  3. We often got lost while refactoring. The picture showing the main events really helped us stay on track. While this was hard enough in baby systems like our example, it seems really complex in real-life software. Having a monolith makes it easier to see what happens because you can use your IDE to follow the path through code. How to stay on track in larger systems is an open question still.

We plan to explore both issues later and hope to report our findings.

Stuff to think about when you're going to a conference.

Xebia Blog - Mon, 10/12/2015 - 15:48

I go to conferences and meetings a lot and many of them are security related. Most security conferences provide tips and tricks how to make the experience as safe as possible. On non-security related conferences however this is not custom so I was asked to provide some generic guidelines what to think about when you are attending a conference or other event.

Wireless networks

Nowadays every event provides for a wireless network, some of them are completely open, some of them need a password. However, none of these networks should be trusted. Without getting into too much 'tech-talk', you should know it is very easy for an attacker on the same network to attack any device. Telltale signs that indicate your device is being attacked are error messages warning about certificate problems or mobile apps like facebook or twitter complaining about network errors. When this happens it is advised to disconnect from the network immediately. If you are on a laptop, and have some experience with configuring network settings, changing the dynamic gateway route to a static one is also a good countermeasure against many attacks.

Even when your device is not being attacked all traffic from your device can be sniffed. If you are using unencrypted communication like http, ftp, pop3, telnet, etc. an attacker can see your usernames, passwords and all information being transmitted so be aware of what information you are requesting and sending. When you really need to use an unencrypted protocol you should first set up a VPN tunnel to a trusted network as this will encrypt all traffic, making sniffing impossible. If you have the luxury of an unlimited (or big) 3G/4G data plan it is also a viable option as these protocols have built-in mechanisms for authentication and encryption and are hard (or at least expensive) to sniff or eavesdrop.

Never leave your computer unattended

Most of us are already used to locking a laptop when you leave for a cup of coffee or something (if you're not; do it!) but at conferences you should never leave your laptop unattended. Even when it's locked an attacker can place malware on it with special USB devices. So although you might only be leaving your place for a second, take your laptop with you or give it to someone you know you can trust.

USB devices

Talking about USB devices; always be suspicious about placing unknown USB devices in your laptop as it one of the most popular ways to infect devices with or spread malware. Even things like USB hubs, keyboards and mice can contain malicious code and plant backdoors in your laptop. Often virus scanners will not detect this so don't rely on that. Always treat USB as if it stands for Unidentified Security Breach.


Besides the technical threats there is a more obvious one; people. If you are looking at sensitive information or entering passwords always be aware that other people can look at your screen and laptop. For screens you can use screen protectors to make this more difficult, but these don't prevent people directly behind you from looking at your screen. With respect to eavesdropping what you type; there is little you can do to prevent this so be aware of your surroundings.

Use a conference laptop

If you have become really paranoid after reading this it might be a good idea to start using a conference laptop that you freshly install before and after attending a conference.

Agile, but still really not Agile? What Pipeline Automation can do for you. Part 1.

Xebia Blog - Mon, 10/12/2015 - 12:06

Organizations adopting Agile and teams delivering on a feature-by-feature basis producing business value at the end of every sprint. Quite possibly this is also the case in your organization. But do these features actually reach your customer at the same pace and generate business value straight away? And while we are at it: are you able to actually use feedback from your customer and apply it for use in the very next sprint?

Possibly your answer is “No”, which I see very often. Many companies have adopted the Agile way of working in their lines of business, but for some reason ‘old problems’ just do not seem to go away...

Hence the question:

“Do you fully capitalize on the benefits provided by working in an Agile manner?”

Straight forward Software Delivery Pipeline Automation might help you with that.

In this first post of three, I hope to inspire you to think about how Software Development Pipeline automation can help your company to move forward and take the next steps towards becoming a truly Agile company. Not just a company adopting Agile principles, but a company that is really positioned to respond to the ever changing environment that is our marketplace today. To explain this, I take the Agile Manifesto as a starting point and work from there.


Agile Principle 1: Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

This is the main objective what Agile teams aim for. Delivering a fully functionmoney_time_value_blue_dice_400_clr1-300x300al product by the end of every sprint. But far too often this product gets ‘stuck’ somewhere in between the Development environment and the subsequent Test, Acceptance and Production environments, basically leaving you with nothing new to demo to your Product Owner and other stakeholders.

For a large part, if not the largest part, it is about culture: do (product-based) teams literally thrive towards producing high quality functional components in the shortest possible time-frame? If the mindset is correct, you have already come a long way, but often teams work with tooling which is well, errr .. just quirky. Builds need to be started manually, unit tests are performed in a couple of areas and there is a real low coverage for functional testing. In many cases, I see that deployments are performed manually by senior technical developer or the Operations department.

Automation of every manual step in the pipeline that can be automated is key here. Be it the automation of packaging, the automation of tests, the automation of deployments or automatically instantiating new infrastructure. Shaping your company into a product-centric environment, aligning your delivery teams and automating every manual step in your Software Delivery process that can be automated will help you to achieve the first objective of the Agile manifesto.


Agile Principle 2: Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

Agile teams aim to be as transparent as possible. This also fits the necessity for delivering a fully functional product by the end of every sprint. By delivering software on a regular basis, requirements may be updated at any time as well. The best feedback a team could wish for is customer feedback and nothing more dynamic than the voice of customer.Lets-Talk-Change

Being capable of dealing with the ever-changing environment requires a different mindset than the one used for working with rigid processes some Enterprises still work with today. It requires a culture of trust, continuous improvement on organization, product and personal level and fosters experimentation. A company incorporating concepts of Continuous Delivery focuses on shaping products together with its customer, resulting in a product that will optimally fit customer needs. For instance, this can be done setting up a Minimal Viable Product, making it accessible to the customer while monitoring both user -and system behaviors where feedback is channeled back into your end-product. This feedback-driven change is continuous in nature and requires a level of rigorous automation where manual interventions for deploying a new product are no longer welcome.

In order to foster ever-changing requirements, make sure your software development pipeline is able to provide the right amount of flexibility so your team can deal with this. It is of upmost importance you can react instantly and this is where automation comes into play.


Agile Principle 3: Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Deliver working software frequently is what an Agile team aims for, giving it instant ROI on a sprint-by-sprint basis. Agile teams deliver fully functional software by the end of each sprint, but sometimes this software remains stuck in subsequent parts of the software delivery process. As long as software is stuck within the pipeline, in any other environment than Production, it does not generate any value.Time-m

When your aim is to move towards the shortest possible release cycle, like delivering software multiple times a day, this can only be done when applying rigorous automation. These topics are addressed in areas like build automation, test automation, release automation, deployment automation and automating the provisioning of your infrastructure. Automating every manual step in your Software Delivery process that can be automated, will help you to achieve the third objective of the Agile manifesto.


Agile Principle 4: Business people and developers must work together daily throughout the project.

Working in end-to-end responsible multidisciplinary teams is an important aspect of Agile. People with all types of knowledge related to the end product need to be assembled within one and the same team. “You build it you run it” is what we hear often and for this, we need to make sure we are working in teams which objective is to not only deliver new functionality, but also to take care of the product’s operationGrowingTogetherLogo-web_sizeal aspects.

An important part related to Continuous Delivery, if not the most important part, is culture. When the objective is to push product delivery cycle times to an absolute minimum, chances are your workforce needs to be organized in a different manner than it currently is. First of all, you need to minimize the amount of handover moments, which can be achieved by organizing people centrically around (a part of) your product, i.e. put business, developers and ops representatives into one team where each member of the team is to explicitly bring skills to the table that add to the product.

Now, there is not much that automation can do in terms of 'teaming', but automation can help you in pulling product information to an abstract level everybody can understand. A failing build (red screen), a failing test (red screen with clear test name), a failing deployment (red screen with failing component name), a statistic report (6 / 10 deployments successful or 'spikes after deployment of version n of the product) can help the whole team to understand what product status is. And this is what teaming is all about: that every discipline within the team understands that it is not just about their isolated part, but it is about the complete team's product.


Stay tuned for the next post, where I'll address Agile principles 5 to 8.


Michiel Sens.

Simplification of iOS View Controllers: MVVM or Presentation Controls?

Xebia Blog - Fri, 10/09/2015 - 22:34

We've all seen View Controllers consisting of hundred or even thousands of lines of code. One popular strategy of reducing view controller complexity is using the Model-View-ViewModel (MVVM) design pattern. But that's not the only way to separate the view controllers into smaller, easier to understand components. This post will explore using small Presentation Control classes to achieve a similar effect. They can even be used together with MVVM components.

This post will not cover the MVVM pattern in great detail. If you'd like to read more about that please follow one of the links at the end of this post. However, you should be able to understand most of this post even without prior knowledge of MVVM. First we'll show a traditional example of a view controller, followed by an example using MVVM and then another example using Presentation Controls. At the end we'll compare them and see how we can use them together.

Problem: Complex view controller

The problem we're trying to solve here is that view controllers often contain too much functionality. Besides updating the view and handling user interaction, they're often full of state, network requests, error handling, validation, you name it.

Let's have a look at an example of a traditional view controller. Imagine we have a travel planner app where we show the departure and arrival time of a trip that we're about to make. This could look something like this:


All this information is taken from a pretty simple model:

class Trip {

    let departure: NSDate
    let arrival: NSDate
    let actualDeparture: NSDate

    init(departure: NSDate, arrival: NSDate, actualDeparture: NSDate? = nil) {
        self.departure = departure
        self.arrival = arrival
        self.actualDeparture = actualDeparture ?? departure

The model has a departure time, arrival time and an actual departure time. The view shows the date at the right top (we assume the date of departure and arrival is always the same for the sake of keeping things simple), the departure time on the left and arrival time on the right. In between the departure and arrival times you see the duration. If the actual departure time is later than the departure time it means there is a delay. In that case the departure time is displayed in red and the delay displayed at the bottom.

Without MVVM or Presentation Controls the code in our view controller would look as follows:

class ViewController: UIViewController {

    @IBOutlet weak var dateLabel: UILabel!
    @IBOutlet weak var departureTimeLabel: UILabel!
    @IBOutlet weak var arrivalTimeLabel: UILabel!
    @IBOutlet weak var durationLabel: UILabel!
    @IBOutlet weak var delayLabel: UILabel!

    // in a real app the trip would be set from another view controller or loaded from a server or something...
    let trip = Trip(departure: NSDate(timeIntervalSince1970: 1444396193), arrival: NSDate(timeIntervalSince1970: 1444397193), actualDeparture: NSDate(timeIntervalSince1970: 1444396493))

    override func viewDidLoad() {

        dateLabel.text = NSDateFormatter.localizedStringFromDate(trip.departure, dateStyle: .ShortStyle, timeStyle: .NoStyle)
        departureTimeLabel.text = NSDateFormatter.localizedStringFromDate(trip.departure, dateStyle: .NoStyle, timeStyle: .ShortStyle)
        arrivalTimeLabel.text = NSDateFormatter.localizedStringFromDate(trip.arrival, dateStyle: .NoStyle, timeStyle: .ShortStyle)

        let durationFormatter = NSDateComponentsFormatter()
        durationFormatter.allowedUnits = [.Hour, .Minute]
        durationFormatter.unitsStyle = .Short
        durationLabel.text = durationFormatter.stringFromDate(trip.departure, toDate: trip.arrival)

        let delay = trip.actualDeparture.timeIntervalSinceDate(trip.departure)
        if delay > 0 {
            durationFormatter.unitsStyle = .Full
            delayLabel.text = String.localizedStringWithFormat(NSLocalizedString("%@ delay", comment: "Show the delay"), durationFormatter.stringFromTimeInterval(delay)!)
            departureTimeLabel.textColor = .redColor()
        } else {
            delayLabel.hidden = true


Luckily we're using a Storyboard with constraints and don't need any code to position any of the views. And if this would be all, we probably wouldn't have to change anything as this is still small enough to maintain. But in real apps, view controllers do a lot more than this, in which case it's good to simplify things and separate different concerns. That will also improve testability a lot.

Solution 1: Use MVVM

We can improve our code by using the MVVM pattern. This would look something like this:

class TripViewViewModel {

    let date: String
    let departure: String
    let arrival: String
    let duration: String
    let delay: String?
    let delayHidden: Bool
    let departureTimeColor: UIColor

    init(_ trip: Trip) {
        date = NSDateFormatter.localizedStringFromDate(trip.departure, dateStyle: .ShortStyle, timeStyle: .NoStyle)
        departure = NSDateFormatter.localizedStringFromDate(trip.departure, dateStyle: .NoStyle, timeStyle: .ShortStyle)
        arrival = NSDateFormatter.localizedStringFromDate(trip.arrival, dateStyle: .NoStyle, timeStyle: .ShortStyle)

        let durationFormatter = NSDateComponentsFormatter()
        durationFormatter.allowedUnits = [.Hour, .Minute]
        durationFormatter.unitsStyle = .Short
        duration = durationFormatter.stringFromDate(trip.departure, toDate: trip.arrival)!

        let delay = trip.actualDeparture.timeIntervalSinceDate(trip.departure)
        if delay > 0 {
            durationFormatter.unitsStyle = .Full
            self.delay = String.localizedStringWithFormat(NSLocalizedString("%@ delay", comment: "Show the delay"), durationFormatter.stringFromTimeInterval(delay)!)
            departureTimeColor = .redColor()
            delayHidden = false
        } else {
            self.delay = nil
            departureTimeColor = UIColor(red: 0, green: 0, blue: 0.4, alpha: 1)
            delayHidden = true

We've created a separate class TripViewViewModel that translates everything from our Trip model to things we can use in our view. Note that this new class doesn't know anything about UIView classes like UILabels etc. This binding is still handled in our View Controller, which now looks like this:

class ViewController: UIViewController {

    @IBOutlet weak var dateLabel: UILabel!
    @IBOutlet weak var departureTimeLabel: UILabel!
    @IBOutlet weak var arrivalTimeLabel: UILabel!
    @IBOutlet weak var durationLabel: UILabel!
    @IBOutlet weak var delayLabel: UILabel!

    let tripModel = TripViewViewModel(Trip(departure: NSDate(timeIntervalSince1970: 1444396193), arrival: NSDate(timeIntervalSince1970: 1444397193), actualDeparture: NSDate(timeIntervalSince1970: 1444396493)))

    override func viewDidLoad() {

        dateLabel.text = tripModel.date
        departureTimeLabel.text = tripModel.departure
        arrivalTimeLabel.text = tripModel.arrival
        durationLabel.text = tripModel.duration
        delayLabel.text = tripModel.delay
        delayLabel.hidden = tripModel.delayHidden
        departureTimeLabel.textColor = tripModel.departureTimeColor


Since this is now a one-to-one binding from MVVM properties to UIView (in this case UILabel) properties, our view controller's complexity has been greatly reduced. And our TripViewViewModel is really easy to unit test.

Move logic to the model

Our current TripViewViewModel class contains some logic that might fit better in our Trip model. Especially all the calculations that are not directly related to the presentation.

class Trip {

    let departure: NSDate
    let arrival: NSDate
    let actualDeparture: NSDate
    let delay: NSTimeInterval
    let delayed: Bool
    let duration: NSTimeInterval

    init(departure: NSDate, arrival: NSDate, actualDeparture: NSDate? = nil) {
        self.departure = departure
        self.arrival = arrival
        self.actualDeparture = actualDeparture ?? departure

        // calculations
        duration = self.arrival.timeIntervalSinceDate(self.departure)
        delay = self.actualDeparture.timeIntervalSinceDate(self.departure)
        delayed = delay > 0

Here we moved the logic of calculating the duration and delay to the Trip model.

The presentation logic remains in our TripViewViewModel, which is now using the calculated properties of the model:

class TripViewViewModel {

    let date: String
    let departure: String
    let arrival: String
    let duration: String
    let delay: String?
    let delayHidden: Bool
    let departureTimeColor: UIColor

    init(_ trip: Trip) {
        date = NSDateFormatter.localizedStringFromDate(trip.departure, dateStyle: .ShortStyle, timeStyle: .NoStyle)
        departure = NSDateFormatter.localizedStringFromDate(trip.departure, dateStyle: .NoStyle, timeStyle: .ShortStyle)
        arrival = NSDateFormatter.localizedStringFromDate(trip.arrival, dateStyle: .NoStyle, timeStyle: .ShortStyle)

        let durationFormatter = NSDateComponentsFormatter()
        durationFormatter.allowedUnits = [.Hour, .Minute]
        durationFormatter.unitsStyle = .Short
        duration = durationFormatter.stringFromTimeInterval(trip.duration)!

        delayHidden = !trip.delayed
        if trip.delayed {
            durationFormatter.unitsStyle = .Full
            delay = String.localizedStringWithFormat(NSLocalizedString("%@ delay", comment: "Show the delay"), durationFormatter.stringFromTimeInterval(trip.delay)!)
            departureTimeColor = .redColor()
        } else {
            self.delay = nil
            departureTimeColor = UIColor(red: 0, green: 0, blue: 0.4, alpha: 1)

Since the properties of the TripViewViewModel didn't change, our View Controller remains exactly the same.

Solution 2: Presentation Controls

If you're like me, you might feel that setting certain properties like our delayHidden and departureTimeColor in TripViewViewModel is a bit odd. These clearly just translate to the hidden and textColor properties of UILabels. And we also need a lot of extra code just for copying property values.

So what can we do to bring the presentation logic closer to our UIView classes?

Small classes that control the binding between the model and the views is the answer: Presentation Controls. The main difference between MVVM classes and Presentation Controls is that the latter does know about UIView classes. Presentation Controls are like mini View Controllers that are only responsible for the presentation.

You could achieve a similar effect by creating custom UIView subclasses that are a combination of views. This worked quite well with views that were completely written in code or views created from a nib file, but it doesn't work well at all in combination with Storyboards. While we want to remove some of the complexity from our view controller, we don't want to move our individual views in a storyboard scene to a separate nib.

Something you might not know is that you can create outlets from views in your scene to objects other than the View Controller of that scene. You can connect them to any object that you add to your scene. Lets first create our presentation control, which is a new class that we call TripPresentationControl:

class TripPresentationControl: NSObject {


Make sure it's a subclass of NSObject, otherwise you'll have problems creating it from Interface Builder.

Now go to the Object Library in Interface Builder and drag a new Object to your scene.

Screen Shot 2015-10-09 at 17.09.22

Change its class in the Identity Inspector to the presentation control class: TripPresentationControl.

Screen Shot 2015-10-09 at 17.13.21

You can now use it pretty much the same as you would a view controller. Just drag outlets from your scene to this class. By doing so, we'll move all of our labels to the presentation control and connect them to the scene.

class TripPresentationControl: NSObject {

    @IBOutlet weak var dateLabel: UILabel!
    @IBOutlet weak var departureTimeLabel: UILabel!
    @IBOutlet weak var arrivalTimeLabel: UILabel!
    @IBOutlet weak var durationLabel: UILabel!
    @IBOutlet weak var delayLabel: UILabel!

    var trip: Trip! {
        didSet {
            dateLabel.text = NSDateFormatter.localizedStringFromDate(trip.departure, dateStyle: .ShortStyle, timeStyle: .NoStyle)
            departureTimeLabel.text = NSDateFormatter.localizedStringFromDate(trip.departure, dateStyle: .NoStyle, timeStyle: .ShortStyle)
            arrivalTimeLabel.text  = NSDateFormatter.localizedStringFromDate(trip.arrival, dateStyle: .NoStyle, timeStyle: .ShortStyle)

            let durationFormatter = NSDateComponentsFormatter()
            durationFormatter.allowedUnits = [.Hour, .Minute]
            durationFormatter.unitsStyle = .Short
            durationLabel.text = durationFormatter.stringFromTimeInterval(trip.duration)!

            delayLabel.hidden = !trip.delayed
            if trip.delayed {
                durationFormatter.unitsStyle = .Full
                delayLabel.text = String.localizedStringWithFormat(NSLocalizedString("%@ delay", comment: "Show the delay"), durationFormatter.stringFromTimeInterval(trip.delay)!)
                departureTimeLabel.textColor = .redColor()

Of course you can create multiple presentation controls to split complex view controllers into manageable smaller controls.

As you can see, we also created a trip property that contains all our presentation logic, pretty much the same we had in our TripViewViewModel. But now we directly set values on the labels.

The next step is to create an outlet from our Presentation Control from the scene to the view controller.


Upon loading our view we will pass the trip to the presentation control. Our entire view controller now looks like this:

class ViewController: UIViewController {

    @IBOutlet var tripPresentationControl: TripPresentationControl!

    let trip = Trip(departure: NSDate(timeIntervalSince1970: 1444396193), arrival: NSDate(timeIntervalSince1970: 1444397193), actualDeparture: NSDate(timeIntervalSince1970: 1444396493))

    override func viewDidLoad() {

        tripPresentationControl.trip = trip

Pretty clean right?

So what about MVVM?

So should you stop using MVVM? Not necessarily. MVVM and Presentation control both have their own strengths and you can even use them together. With MVVM alone, you probably will write slightly more code than with Presentation Controls alone. This does make MVVM a bit easier to test since you don't need to instantiate any UIViews. If you want to test Presentation Controls, you will have to create the views, but the great thing is that these can be simple dummy views. You don't need to replicate the entire view hierarchy or instantiate your view controllers. And most properties you set on the UIViews you can easily test, like the text of a UILabel.

In many large projects the MVVM classes and Presentation Controls can work perfectly together. In that case you would pass on the MVVM object to your Presentation Control. The Presentation Control would then be the layer in between your views and the MVVM and its Model. Things like network requests are also much better handled in an MVVM object since you probably don't want to have the complexity of networks requests and views together in one component.


It's completely up to you and the size of your project whether you use MVVM, Presentation Controls or both. If you currently have view controllers with over a 1000 lines of code, then using both might be a good idea. But if you only have very simple view controllers than you might just want to stick with the traditional View Controller patterns.

In the near future I will write a follow-up post that will go into more detail about how to deal with model changes and user interaction in Presentation Controls.

To read more about MVVM in iOS, I recommend you to read Introduction to MVVM by Ash Furrow and From MVC to MVVM in Swift by Srdan Rasic.

Also make sure to join the do {iOS} Conference in Amsterdam the 9th of November, 2015. Here Natasha "the Robot" Murashev will be giving a talk about Protocol-oriented MVVM. If you enjoyed reading this post you'll definitely want to see her session about MVVM.

Stuff The Internet Says On Scalability For October 9th, 2015

Hey, it's HighScalability time:

Best selfie ever? All vacation photos taken by Apollo astronauts are now online. Fakes, obvi.
If you like Stuff The Internet Says On Scalability then please consider supporting me on Patreon.
  • millions: # of Facebook users have no idea they’re using the internet; 8%: total of wealth in tax havens; $7.3B: AWS revenues; 11X: YouTube bigger than Facebook; 10: days 6s would last on diesel; 65: years ago the transistor was patented; 80X: reduction in # of new drugs approved per billion US dollars spent since 1950; 37 trillion: cells in the human body; 83%: accuracy of predicting activities from pictures.

  • Quotable Quotes:
    • @Nick_Craver: Stack Overflow HTTP, last 30 days: Bytes 128,095,601,184,645 Hits 5,795,253,218 Pages 1,921,499,030 SQL 19,229,946,858 Redis 11,752,754,019
    • @merv: #reinvent Amazon process for creating new offerings: once decision is made "write the press release and the FAQ you’ll use - then build it."
    • @PaulMiller: @monkchips to @ajassy, “One of your biggest competitors is stupidity.” Quite. Or inertia. #reInvent
    • @DanHarper7: If SpaceX can publish their pricing for going to space, your little SaaS does NOT need "Contact us for pricing" 
    • @etherealmind: If you haven't implemented 10GbE yet, start thinking about 25GbE instead. Cost per port is roughly 1.4x for 2.5x performance.
    • @g2techgroup: Some of the most expensive real estate in the world was being used for data storage...We should not be in the data center business #reinvent
    • The microservices cargo cult: the biggest advantage a microservice architecture brings to the table that is hard to get with other approaches is scalability. Every other benefit can be had by a bit of discipline and a good development process.
    • findjashua: the new 'best practice' is to have a universal app - that renders on the server on first load, and runs as a js app subsequently. This way crawlers and browsers w js disabled still get raw markup.
    • Instagram: Do the simple thing first.
    • erikpukinskis: Generic containers are an awkward mid-way point between special-purpose containers (a Wordpress instance or a rails app on heroku) and an actual machine. You get the hassle of maintaining your own instances, without the flexibility or well-defined performance characteristics of an actual box.
    • @AWSreInvent: Showing off the Amazon Snowball - a 47lb, 50TB device for transporting data to the AWS cloud #reInvent 
    • @merv: #reinvent “There is no compression algorithm for experience” - Andy Jassy. Well said.
    • Alexander von Zitzewitz: I know that about 90% of software systems are suffering from severe architectural erosion, i.e. there is not a lot of the original architectural structure left in them, and coupling and dependencies are totally out of control.
    • Haunted By Data: But information about people retains its power as long as those people are alive, and sometimes as long as their children are alive. No one knows what will become of sites like Twitter in five years or ten. But the data those sites own will retain the power to hurt for decades.

  • Data is valuable, especially if you can turn it into your own private wire. Scandal Erupts in Unregulated World of Fantasy Sports. How many other data archipelagos are being used as private opaque oracles?

  • Cool idea, using drones as an exponential technology to spread seeds, countering deforestation with industrial scale reforestation. BioCarbon Engineering. It's precision forestry. A mapping drone system is used to generate high quality 3D maps of an area. Then drones follow a predetermined planting pattern derived from the mapping phase to air fire biodegradable seed pods onto the ground from a height of 1-2 meters. A problem not unlike dropping a mars rover. Clever pod design shields the seeds from impact while giving them the best chance at germination. This approach recapitulates the batch to real-time transformation that we are seeing everywhere. The current version uses a batch approach with distinct pipelined phases. One can imagine the next version using a swarm of communicating drones to coordinate both the mapping and planting in real-time; perhaps even target selection can be automated to form a continuous reactive system.

  • Birmingham Hippodrome shows how they use Heroku and Facebook's HHVM (HipHop Virtual Machine) to scale their WordPress system and keep it running on a modest budget. Maximum of 4 Standard-1X dynos; Peak requests: ~800/minute; Average memory use per dyno: 130MB; no downtime; Median response time : 5ms; Peak dyno load (so far): ~3.0.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Future of Testing and Automation: The role of the tester in 2020

Xebia Blog - Thu, 10/08/2015 - 19:28

Last week, the first TestWorks Conf was held in Amsterdam. This hands-on conference featured a plethora of test automation tools and allowed the participants to gain practical experience with them. Since we feel and expect that all participants will take next steps towards improving their test automation practices, we decided to take a glance into the future and discuss the future of testing and automation together with Alan Richardson.

In a series of blogposts, we would like to share parts of our vision on testing and automation in the near future. First stop will be: the role of the tester in 2020.

What is the role of the tester in 2020?

You can feel the change in the air. Many companies are starting with Agile, Scrum, DevOps and Continuous Delivery. Everyone is focused on delivering new business value faster. Software can be built and released a lot quicker than before. This has huge impact on old school testing strategies and professions. I often hear testing professionals asking their managers: Is my role as a tester still relevant? Do I need to become more technical? Is my job going to disappear?

The Xebia team took some time to collect thoughts and started several open discussions with Alan Richardson. One question we tried to answer was: "What the role of the tester is in 2020?".

Quality comes from quality minded people

We came to the conclusion that testers will stay around in 2020. Yet, testers should become aware that testing is a task that can be picked up by anyone. Not only by designated testers with quality informing or gatekeeping power. If you really care about your work, company, end product and end users, well... you test! Quality comes from quality minded people. Having a shared responsibility in the development team towards quality and doing testing well (together) will make a difference in 2020. We haven't been able to define the tester role (or resource) since the sixties; why would we do that now? And, we don't particularly like the word "tester", because it is treated as a resource type. We should think about people that have skillsets / qualities instead of being a mere (tester) resource.

Quality is in the eye of the beholder, so it is important that testing is done from different point of views. Development is getting more specialised in new technologies and frameworks . So, if we were to pick one skill testers should develop or possess, it would be being able to catch up and learn how to test these new technologies as soon as possible

We also discussed that the developers will never become good testers and testers will never become good coders. The truth lies somewhere in the middle. Besides delving into requirements and ensuring common understanding between stakeholders and development teams, testers will need to become more technical to understand the risks sooner rather than later. Otherwise, they will have a hard time in 2020.
The TestWorks Conf was a perfect example of (200) testers getting their hands dirty with test automation frameworks like Cucumber, FitNesse, Robot Framework, Serenity, Protractor, Mox, Gatling, Axini and Galen Framework.

Testing can be taught and can result in a cultural change. So developers too can learn to understand the risk in technical implementations. Being critical about your own code and issues that can come out of it is crucial for effective test automation. We also expect developers to reach out to stakeholders and validate that their implementation actually solves the problem.

Fast feedback

Testing should really add value and deliver fast feedback to the team. Everything else delaying the feedback loop needs to be taken seriously and improved; delay holds back the required speed in the continuous delivery pipeline. We think that testing should occur alongside development to keep the feedback loops in testing as short as possible. Testing should not become the bottleneck for continuous delivery teams. It should help the teams decrease bugs in production and increase releases to production.

The activity of the tester in 2020 will focus more on continuous monitoring and reporting deviations in production. Monitoring the impact a code change has made is one of the most important activities in testing. It will help the organisation in visualising the real value of the solution. Like more users or sales, all depending on what quality attribute you want to measure and is valuable to measure.

Fix the root cause

Besides visualising the quality, testers should steer quality by actually solving the root cause of problems instead of informing about the symptoms. That can be either solving ambiguity in specifications or technical implementations like code complexity or redundancy. Making sure those problems will not happen again.


So what is the role of the tester 2020? The tester role has not been defined yet. Perhaps that would be a waste of effort. Yet, here are some guidelines for the tester to remain relevant and to gear up for the world in 2020:

  • Steering continuous delivery teams towards quality instead of being a quality gatekeeper
  • Delving into requirements and creating common understanding between stakeholders and development teams
  • Monitoring impact of code changes in production
  • Catching up and learning about new technology being used to understand technical risk
  • Eliminating testing bottlenecks, improve slow feedback loops in continuous delivery pipelines
  • Steering the team in solving root causes instead of fixing symptoms

And, perhaps most importantly, testing should become a social activity which involves each and everyone in the development team.

Stay tuned for another blogpost about the future of testing and automation. Next topic will be: The biggest problems in test automation.

Babel-free backend prototype - challenge accepted

Xebia Blog - Thu, 10/08/2015 - 12:45
code { display: inline !important; }

This post is part of a series of ES2015 posts. We'll be covering new JavaScript functionality every week!

In several previous posts we've been using Babel to actually run our (transpiled) code. But how far can we get if we only use natively supported features? Let's dig in the stable features of Node.js 4 and see how much of our code can be written without a transpiler dependency.

The setup for the prototype is a really simple HTTP server that serves different content under 3 paths:

  • The root path / serves a simple message as body
  • The async path /async serves a different message and simulates an asynchronous call to another system
  • Other paths give a 404 code.

We're not going to add third party http frameworks to our code and rely on the Node.js provided http server implementation for clarity.

Let's write a server implementation using the current Node.js. Based on the list of supported features, we can safely use the following native ES2015 constructs:

  • const and let
  • arrow functions
  • generator functions
  • Promise
  • Map
  • template strings
  • ... and a 'few' others, accounting for an implementation of 53% of the official ECMA spec

We'll start with the initialisation of our server implementation.

'use strict';

const http = require('http'),
  co = require('co'),
  sleep = require('./sleep');

The first gotcha: to be able to use let and const in our code, we need to add the 'use strict' string "prolog" on top of our files. Else we're treated with a SyntaxError if we use them.

We have to fall back on require for module imports. Node.js 4 doesn't natively support the standardised import/export definitions yet.

Here we require the co lib to write nicer async code with generators, we'll come back to that. Also, for this demonstration, we've written an asynchronous sleep function which returns a Promise that sleeps for a specified time and then resolves with the specified result:

'use strict';

module.exports = (ms, result) => {
  return new Promise((resolve) => {
    setTimeout(() => {
    }, ms);

With this sleep function, we'll simulate an asynchronous call to for instance a database and its response. We can provide arrow functions as an export as expected. Also, we don't have to require a promise library but immediately use the native Promise class.

Let's return to our server implementation.

const routeHandlers = new Map();

routeHandlers.set('/', (req, res) => {

For our route handling, we're using a Map. The advantage over using an object literal is that there are no inherited properties that may influence our routing.

Setting up route handling is again really basic. This particular handler simply responds with a 200 response with body 'okay'. The arrow function gives us improved readability. Of course, you should not build your own but use one of the many http frameworks with feature-packed routers if the need arises.

Now for the async call handling. We imagine a fake underlying system that requires us to do two calls: one to retrieve a certain value and a second call that uses the first output and provides the final output. Both of these calls return a Promise object. We could write the code with then and catch as follows:

routeHandlers.set('/async', (req, res) => {
  function errorHandler() {
    res.end('internal error');

  sleep(1000, 123).then((data) => {
    const a = data;
    sleep(2000, a + 456).then((data) => {
      const b = data;
      res.end(`asynchronous result: ${a} ${b}`);

We need to provide a generic error function to both calls to catch errors which is boilerplate-ish. And especially the nesting of the two then calls makes the flow of code harder to understand.

If we use the co helper function, we can use a generator function and yield as a 'waiting' construct and the code is more clear:

routeHandlers.set('/async', (req, res) => {
  co(function *() {
    const a = yield sleep(1000, 123);
    const b = yield sleep(2000, a + 456);
    res.end(`asynchronous result: ${a} ${b}`);

  }).catch(() => {
    res.end('internal error');

An extra benefit is that we only have to write one catch function to catch both promise failure paths. This idea of using generators and yield is used extensively in the koa http framework. It certainly seems a good combination with Node.js 4.

(Did you see how we used a template string to render our page result? Such wow.)

We continue with the creation of the server:

const server = http.createServer((req, res) => {
  if (routeHandlers.has(req.url)) {
    return routeHandlers.get(req.url)(req, res);

  res.end('not found');

This basically checks whether a route exactly matches one of the routes defined in the Map and delegates the handling to the specified function.

We want to expose the server's listen and close but no other functions:

exports.listen = function () {
  server.listen.apply(server, arguments);

exports.close = function () {
  server.close.apply(server, arguments);

Another gotcha: we can't use the arrow function syntax in this case. We want to apply the arguments array, and that is not provided inside arrow functions.

So, when we start our server we can see that this all works as expected. Great! But what about a test?


For testing, we use the Mocha test framework and the Chai assertion library. Let's initialise the server test:

'use strict';

const server = require('../src/server'),
  getBody = require('./getBody'),
  co = require('co'),
  assert = require('assert'),
  port = 6000;

describe('server', () => {
  before(() => {

We can use the arrow function syntax for our describe and before calls, which is nice. The getBody function returns a Promise that resolves to the body contents (and the original response) of a http get call:

'use strict';

const http = require('http');

module.exports = (uri) => {
  return new Promise((resolve) => {
    const bodyChunks = [];
    http.get(uri, (res) => {
      res.on('data', (chunk) => {
      }).on('end', () => {
        resolve({ res, body: bodyChunks.join() });

We can define a const bodyChunks because the array itself is constant, but we can still manipulate the contents. We can add res in the object literal by using the shorthand property syntax to write a bit less code.

Let's return to our test:

  describe('/', () => {
    it('should return 200 and respond with okay', (done) => {
      co(function *() {
        const ret = yield getBody(`http://localhost:${port}`);
        assert.equal(200, ret.res.statusCode);
        assert.equal('okay', ret.body);

Here we use co again to write easy to follow asynchronous code. We can use a library like co-mocha to improve the readability of our tests even more, but for clarity we'll keep it like this. It would be nice if Mocha supported generator functions out-of-the-box!

Node.js 4 doesn't support destructuring yet, else we could destructure the returned object immediately in res and constants.

Now to test our longer-running asynchronous call:

  describe('/async', () => {
    it('should return 200 and respond with message', function (done) {
      co(function *() {
        const ret = yield getBody(`http://localhost:${port}/async`);
        assert.equal(200, ret.res.statusCode);
        assert.equal('asynchronous result: 123 579', ret.body);

Here, we have to change the default timeout (2000ms) for tests. This can be done for a single it using this.timeout. But in an arrow function, this is not available, so we need to write the full function syntax.

The rest of the test is left out as it is trivial.


We've seen that it's possible to already write quite some sophisticated ES2015 without the use of any transpiler when we upgrade Node.js to 4+, give or take a few gotchas and some integration issues. No transpiler also means improved debugging and no more source maps, so that can be convincing argument for your next project.

Keep an eye on Chrome Status to track development of new EcmaScript features, as they eventually will pop up in Node.

For reference, you can find the working project code at this repo. You can start the server with npm start and run the Mocha test suite with npm test. Of course, make sure you have Node.js 4 set as your default.