Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator/categories/2&page=1' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Testing & QA
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

GTAC 2015: Call for Proposals & Attendance

Google Testing Blog - Sun, 08/21/2016 - 17:32
Posted by Anthony Vallone on behalf of the GTAC Committee

The GTAC (Google Test Automation Conference) 2015 application process is now open for presentation proposals and attendance. GTAC will be held at the Google Cambridge office (near Boston, Massachusetts, USA) on November 10th - 11th, 2015.

GTAC will be streamed live on YouTube again this year, so even if you can’t attend in person, you’ll be able to watch the conference remotely. We will post the live stream information as we get closer to the event, and recordings will be posted afterward.

Presentations are targeted at student, academic, and experienced engineers working on test automation. Full presentations are 30 minutes and lightning talks are 10 minutes. Speakers should be prepared for a question and answer session following their presentation.

For presentation proposals and/or attendance, complete this form. We will be selecting about 25 talks and 200 attendees for the event. The selection process is not first come first serve (no need to rush your application), and we select a diverse group of engineers from various locations, company sizes, and technical backgrounds (academic, industry expert, junior engineer, etc).

The due date for both presentation and attendance applications is August 10th, 2015.

There are no registration fees, but speakers and attendees must arrange and pay for their own travel and accommodations.

More information
You can find more details at developers.google.com/gtac.

Categories: Testing & QA

The Deadline to Apply for GTAC 2015 is Monday Aug 10

Google Testing Blog - Sun, 08/21/2016 - 17:32
Posted by Anthony Vallone on behalf of the GTAC Committee

The deadline to apply for GTAC 2015 is this Monday, August 10th, 2015. There is a great deal of interest to both attend and speak, and we’ve received many outstanding proposals. However, it’s not too late to submit your proposal for consideration. If you would like to speak or attend, be sure to complete the form by Monday.

We will be making regular updates to the GTAC site (developers.google.com/gtac/2015/) over the next several weeks, and you can find conference details there.

For those that have already signed up to attend or speak, we will contact you directly by mid-September.

Categories: Testing & QA

Announcing the GTAC 2015 Agenda

Google Testing Blog - Sun, 08/21/2016 - 17:31
by Anthony Vallone on behalf of the GTAC Committee 

We have completed the selection and confirmation of all speakers and attendees for GTAC 2015. You can find the detailed agenda at: developers.google.com/gtac/2015/schedule.

Thank you to all who submitted proposals!

There is a lot of interest in GTAC once again this year with about 1400 applicants and about 200 of those for speaking. Unfortunately, our venue only seats 250. We will livestream the event as usual, so fret not if you were not selected to attend. Information about the livestream and other details will be posted on the GTAC site soon and announced here.

Categories: Testing & QA

GTAC 2015 is Next Week!

Google Testing Blog - Sun, 08/21/2016 - 17:31
by Anthony Vallone on behalf of the GTAC Committee

The ninth GTAC (Google Test Automation Conference) commences on Tuesday, November 10th, at the Google Cambridge office. You can find the latest details on the conference site, including schedule, speaker profiles, and travel tips.

If you have not been invited to attend in person, you can watch the event live. And if you miss the livestream, we will post slides and videos later.

We have an outstanding speaker lineup this year, and we look forward to seeing you all there or online!

Categories: Testing & QA

The Inquiry Method for Test Planning

Google Testing Blog - Sun, 08/21/2016 - 17:30
by Anthony Vallone
updated: July 2016

Creating a test plan is often a complex undertaking. An ideal test plan is accomplished by applying basic principles of cost-benefit analysis and risk analysis, optimally balancing these software development factors:
  • Implementation cost: The time and complexity of implementing testable features and automated tests for specific scenarios will vary, and this affects short-term development cost.
  • Maintenance cost: Some tests or test plans may vary from easy to difficult to maintain, and this affects long-term development cost. When manual testing is chosen, this also adds to long-term cost.
  • Monetary cost: Some test approaches may require billed resources.
  • Benefit: Tests are capable of preventing issues and aiding productivity by varying degrees. Also, the earlier they can catch problems in the development life-cycle, the greater the benefit.
  • Risk: The probability of failure scenarios may vary from rare to likely, and their consequences may vary from minor nuisance to catastrophic.
Effectively balancing these factors in a plan depends heavily on project criticality, implementation details, resources available, and team opinions. Many projects can achieve outstanding coverage with high-benefit, low-cost unit tests, but they may need to weigh options for larger tests and complex corner cases. Mission critical projects must minimize risk as much as possible, so they will accept higher costs and invest heavily in rigorous testing at all levels.
This guide puts the onus on the reader to find the right balance for their project. Also, it does not provide a test plan template, because templates are often too generic or too specific and quickly become outdated. Instead, it focuses on selecting the best content when writing a test plan.

Test plan vs. strategy
Before proceeding, two common methods for defining test plans need to be clarified:
  • Single test plan: Some projects have a single "test plan" that describes all implemented and planned testing for the project.
  • Single test strategy and many plans: Some projects have a "test strategy" document as well as many smaller "test plan" documents. Strategies typically cover the overall test approach and goals, while plans cover specific features or project updates.
Either of these may be embedded in and integrated with project design documents. Both of these methods work well, so choose whichever makes sense for your project. Generally speaking, stable projects benefit from a single plan, whereas rapidly changing projects are best served by infrequently changed strategies and frequently added plans.
For the purpose of this guide, I will refer to both test document types simply as "test plans”. If you have multiple documents, just apply the advice below to your document aggregation.

Content selection
A good approach to creating content for your test plan is to start by listing all questions that need answers. The lists below provide a comprehensive collection of important questions that may or may not apply to your project. Go through the lists and select all that apply. By answering these questions, you will form the contents for your test plan, and you should structure your plan around the chosen content in any format your team prefers. Be sure to balance the factors as mentioned above when making decisions.

  • Do you need a test plan? If there is no project design document or a clear vision for the product, it may be too early to write a test plan.
  • Has testability been considered in the project design? Before a project gets too far into implementation, all scenarios must be designed as testable, preferably via automation. Both project design documents and test plans should comment on testability as needed.
  • Will you keep the plan up-to-date? If so, be careful about adding too much detail, otherwise it may be difficult to maintain the plan.
  • Does this quality effort overlap with other teams? If so, how have you deduplicated the work?

  • Are there any significant project risks, and how will you mitigate them? Consider:
    • Injury to people or animals
    • Security and integrity of user data
    • User privacy
    • Security of company systems
    • Hardware or property damage
    • Legal and compliance issues
    • Exposure of confidential or sensitive data
    • Data loss or corruption
    • Revenue loss
    • Unrecoverable scenarios
    • SLAs
    • Performance requirements
    • Misinforming users
    • Impact to other projects
    • Impact from other projects
    • Impact to company’s public image
    • Loss of productivity
  • What are the project’s technical vulnerabilities? Consider:
    • Features or components known to be hacky, fragile, or in great need of refactoring
    • Dependencies or platforms that frequently cause issues
    • Possibility for users to cause harm to the system
    • Trends seen in past issues

  • What does the test surface look like? Is it a simple library with one method, or a multi-platform client-server stateful system with a combinatorial explosion of use cases? Describe the design and architecture of the system in a way that highlights possible points of failure.
  • What platforms are supported? Consider listing supported operating systems, hardware, devices, etc. Also describe how testing will be performed and reported for each platform.
  • What are the features? Consider making a summary list of all features and describe how certain categories of features will be tested.
  • What will not be tested? No test suite covers every possibility. It’s best to be up-front about this and provide rationale for not testing certain cases. Examples: low risk areas that are a low priority, complex cases that are a low priority, areas covered by other teams, features not ready for testing, etc. 
  • What is covered by unit (small), integration (medium), and system (large) tests? Always test as much as possible in smaller tests, leaving fewer cases for larger tests. Describe how certain categories of test cases are best tested by each test size and provide rationale.
  • What will be tested manually vs. automated? When feasible and cost-effective, automation is usually best. Many projects can automate all testing. However, there may be good reasons to choose manual testing. Describe the types of cases that will be tested manually and provide rationale.
  • How are you covering each test category? Consider:
  • Will you use static and/or dynamic analysis tools? Both static analysis tools and dynamic analysis tools can find problems that are hard to catch in reviews and testing, so consider using them.
  • How will system components and dependencies be stubbed, mocked, faked, staged, or used normally during testing? There are good reasons to do each of these, and they each have a unique impact on coverage.
  • What builds are your tests running against? Are tests running against a build from HEAD (aka tip), a staged build, and/or a release candidate? If only from HEAD, how will you test release build cherry picks (selection of individual changelists for a release) and system configuration changes not normally seen by builds from HEAD?
  • What kind of testing will be done outside of your team? Examples:
    • Dogfooding
    • External crowdsource testing
    • Public alpha/beta versions (how will they be tested before releasing?)
    • External trusted testers
  • How are data migrations tested? You may need special testing to compare before and after migration results.
  • Do you need to be concerned with backward compatibility? You may own previously distributed clients or there may be other systems that depend on your system’s protocol, configuration, features, and behavior.
  • Do you need to test upgrade scenarios for server/client/device software or dependencies/platforms/APIs that the software utilizes?
  • Do you have line coverage goals?

Tooling and Infrastructure
  • Do you need new test frameworks? If so, describe these or add design links in the plan.
  • Do you need a new test lab setup? If so, describe these or add design links in the plan.
  • If your project offers a service to other projects, are you providing test tools to those users? Consider providing mocks, fakes, and/or reliable staged servers for users trying to test their integration with your system.
  • For end-to-end testing, how will test infrastructure, systems under test, and other dependencies be managed? How will they be deployed? How will persistence be set-up/torn-down? How will you handle required migrations from one datacenter to another?
  • Do you need tools to help debug system or test failures? You may be able to use existing tools, or you may need to develop new ones.

  • Are there test schedule requirements? What time commitments have been made, which tests will be in place (or test feedback provided) by what dates? Are some tests important to deliver before others?
  • How are builds and tests run continuously? Most small tests will be run by continuous integration tools, but large tests may need a different approach. Alternatively, you may opt for running large tests as-needed. 
  • How will build and test results be reported and monitored?
    • Do you have a team rotation to monitor continuous integration?
    • Large tests might require monitoring by someone with expertise.
    • Do you need a dashboard for test results and other project health indicators?
    • Who will get email alerts and how?
    • Will the person monitoring tests simply use verbal communication to the team?
  • How are tests used when releasing?
    • Are they run explicitly against the release candidate, or does the release process depend only on continuous test results? 
    • If system components and dependencies are released independently, are tests run for each type of release? 
    • Will a "release blocker" bug stop the release manager(s) from actually releasing? Is there an agreement on what are the release blocking criteria?
    • When performing canary releases (aka % rollouts), how will progress be monitored and tested?
  • How will external users report bugs? Consider feedback links or other similar tools to collect and cluster reports.
  • How does bug triage work? Consider labels or categories for bugs in order for them to land in a triage bucket. Also make sure the teams responsible for filing and or creating the bug report template are aware of this. Are you using one bug tracker or do you need to setup some automatic or manual import routine?
  • Do you have a policy for submitting new tests before closing bugs that could have been caught?
  • How are tests used for unsubmitted changes? If anyone can run all tests against any experimental build (a good thing), consider providing a howto.
  • How can team members create and/or debug tests? Consider providing a howto.

  • Who are the test plan readers? Some test plans are only read by a few people, while others are read by many. At a minimum, you should consider getting a review from all stakeholders (project managers, tech leads, feature owners). When writing the plan, be sure to understand the expected readers, provide them with enough background to understand the plan, and answer all questions you think they will have - even if your answer is that you don’t have an answer yet. Also consider adding contacts for the test plan, so any reader can get more information.
  • How can readers review the actual test cases? Manual cases might be in a test case management tool, in a separate document, or included in the test plan. Consider providing links to directories containing automated test cases.
  • Do you need traceability between requirements, features, and tests?
  • Do you have any general product health or quality goals and how will you measure success? Consider:
    • Release cadence
    • Number of bugs caught by users in production
    • Number of bugs caught in release testing
    • Number of open bugs over time
    • Code coverage
    • Cost of manual testing
    • Difficulty of creating new tests

Categories: Testing & QA

Software Development Linkopedia August 2016

From the Editor of Methods & Tools - Wed, 08/17/2016 - 15:25
Here is our monthly selection of knowledge on programming, software testing and project management. This month you will find some interesting information and opinions about team management, the (new) software crisis, saying no, software testing, user experience, data modeling, Scrum retrospectives, java microservices, Selenium tests and product backlog refinement. Blog: The Principles of Quantum Team […]

Quote of the Month August 2016

From the Editor of Methods & Tools - Wed, 08/10/2016 - 09:29
Are there unbreakable laws ruling the process of software development? I asked myself this question while reflecting on a recent project, and the answer leads to many conclusions, some already known and some more revealing. Scientific laws reflect reality and cannot be broken. They have strong implications onto how we build things. For instance, there […]

Exposing a private Amazon RDS instance with iptables NAT rules

Agile Testing - Grig Gheorghiu - Thu, 07/28/2016 - 17:43
I needed to expose a private Amazon MySQL RDS instance to a 3rd party SaaS tool. I tried several approaches and finally found one that seemed to work pretty well.

I ended up creating a small EC2 instance in the same VPC as the RDS instance, and applied these iptables NAT/masquerading rules to it, mapping local port 3307 to port 3306 on the RDS instance, whose internal IP address is in this case

# cat iptables_tunnel_port_3307.sh

iptables -F
iptables -F -t nat
iptables -X
iptables -t nat -A PREROUTING -p tcp --dport 3307 -j DNAT --to
iptables -A FORWARD -p tcp -d --dport 3306 -j ACCEPT
iptables -t nat -A OUTPUT -p tcp -o lo --dport 3307 -j DNAT --to
iptables -t nat -A POSTROUTING  -j MASQUERADE

I also had to enable IP forwarding on the EC2 instance:

# sysctl net.ipv4.ip_forward
# sysctl -p

At this point, I was able to hit the external IP of the EC2 instance on port 3307, and get to the private RDS instance on port 3306. I was also able to attach the EC2 instance to an EC2 Security Group allowing the 3rd party SaaS tool IP addresses to access port 3307 on the EC2 instance.

My thanks to the people discussing a similar issue on this thread of LinuxQuestions. Without their discussion, I don't think I'd have been able to figure out a solution.

Code awareness levels

Actively Lazy - Thu, 07/28/2016 - 09:23

Writing code is all about working at multiple levels of abstraction concurrently. But as well as working at multiple levels of abstraction there are also multiple levels of awareness of the code.

The most basic level of code awareness is just making the code work. Does this line of code compile, run and do what I intended it to do? While learning to code this lowest level of awareness consumes almost all our capacity: it is difficult to focus on anything else other than just typing in the correct syntax.

Once we’ve mastered typing in code that works there is another level of awareness: is the intent of this line of code clear? Will another human being be able to understand this? Hell, will I be able to understand it in a month’s time? This is where code readability begins. While writing code we expend some of our mental capacity on making sure that the code will be readable.

Code legibility though is about more than just each single line of code in isolation. We also need to consider whether this line makes sense in the context of the rest of this method, maybe we should extract a new method? Does this line of code make sense in the context of the rest of this class? Does the behaviour belong somewhere else? Does the design that I’m discovering actually make sense? Is it consistent? Is the pattern I’m implementing consistent with the rest of the application? Besides making each line of code compile there are numerous higher levels – rising through each of the larger structures in the application: the method, the class, the module, the component, the system.

Besides code legibility there are other factors to be aware of: will failures in this code be easy to understand, will exceptions help me get to the right part of the code quickly, what happens if things I depend on fail? Will this code perform well, in what scenarios and with what frequency will it be called? Is this code secure, are there ways for an attacker to take advantage of this code?

Writing good code requires developers to be aware of these levels all the time. However, if we’re tired, hungover or fed up we probably won’t give much time to these concerns. We’ll just make the code work that’s in front of us and ignore the bigger picture. When we’re lacking mental capacity, we tend to stop worrying about the higher levels and focus on the lower level, easier problems. “I’ll come back later and fix the design”, we tell ourselves.

This is where pairing can help. I don’t think it’s true that the pairing partner is only thinking about the bigger picture; but the non-driving partner certainly has spare capacity to think about these things and act as a conscience against the driver missing something or getting tired.

I like to use hands-on coding exercises in interview situations because it helps highlight how much awareness a developer has. Generally I’m looking for developers that, for a given problem, have enough capacity to solve the problem quickly and well. Less capable developers will generally take longer and/or solve the problem badly. In an interview situation though, there are extra levels of awareness, besides just solving the problem: is what I’m doing creating a good impression? Am I solving the problem in the right way (e.g. using TDD)? Is the way I’m solving the problem showing off the right skills?

For example a while back I interviewed a candidate who used “yield return”. This is an unusual thing to see, so I asked why use it? “To start a discussion about whether or not you should ever use it”. This I found quite impressive: a candidate with enough mental capacity to not only solve the problem at hand, in a good way; but still with enough spare capacity left to realise that there was an opportunity to use an unusual construct purely to provoke a discussion of it – effectively driving the interview through code, to show off the depth of their knowledge and experience. This is the kind of person you want on your team: someone with enough spare mental capacity to make sure the code is clean and the design correct.

I think this lack of capacity is why the code we write when we’re tired or when we’re learning a new language or learning a new framework is bad: all our capacity is focused on just making the code work, we don’t have spare capacity to also think about whether the code is readable or whether the design makes sense. When we’re struggling to just make the code do what we want, to just keep the compiler happy, is it any wonder the outcome is a mess?

If you want to see how much mental capacity I can bring to your company, why not hire me? I’m looking for new roles right now – so drop me a line dave@activelylazy.co.uk.

Categories: Programming, Testing & QA

CC to Everyone

James Bach’s Blog - Tue, 07/26/2016 - 18:23
I sent this to someone who’s angry with me due to some professional matter we debated. A colleague thought it would be worth showing you, too. So, for whatever it’s worth:

I will say this. I don’t want anyone to feel bad about me, or about my behavior, or about themselves. I can live with that, but I don’t want it.

So, if there is something simple I can do to help people feel better, and it does not require me to tell a lie, then I am willing to do so.

I want people to excel at their craft and be happy. That’s actually what is motivating me, underneath all my arguing.

Categories: Testing & QA

Software Development Conferences Forecast July 2016

From the Editor of Methods & Tools - Mon, 07/18/2016 - 14:12
Here is a list of software development related conferences and events on Agile project management ( Scrum, Lean, Kanban), software testing and software quality, software architecture, programming (Java, .NET, JavaScript, Ruby, Python, PHP), DevOps and databases (NoSQL, MySQL, etc.) that will take place in the coming weeks and that have media partnerships with the Methods […]

Using JMESPath queries with the AWS CLI

Agile Testing - Grig Gheorghiu - Wed, 07/13/2016 - 22:05
The AWS CLI, based on the boto3 Python library, is the recommended way of automating interactions with AWS. In this post I'll show some examples of more advanced AWS CLI usage using the query mechanism based on the JMESPath JSON query language.

Installing the AWS CLI tools is straightforward. On Ubuntu via apt-get:

# apt-get install awscli

Or via pip:

# apt-get install python-pip
# pip install awscli

The next step is to configure awscli by specifying the AWS Access Key ID and AWS Secret Access Key, as well as the default region and output format:

# aws configureAWS Access Key ID: your-aws-access-key-idAWS Secret Access Key: your-aws-secret-access-keyDefault region name [us-west-2]: us-west-2
Default output format [None]: json
The configure command creates a ~/.aws directory containing two files: config and credentials.
You can specify more than one pair of AWS keys by creating profiles in these files. For example, in ~/.aws/credentials you can have:
[profile profile1]AWS_ACCESS_KEY_ID=key1AWS_SECRET_ACCESS_KEY=secretkey1
[profile profile2]AWS_ACCESS_KEY_ID=key2AWS_SECRET_ACCESS_KEY=secretkey2
In ~/.aws/config you can have:[profile profile1]region = us-west-2
[profile profile2]region = us-east-1
You can specify a given profile when you run awscli:
# awscli --profile profile1
Let's assume you want to write a script using awscli that deletes EBS snapshots older than N days. Let's go through this one step at a time.
Here's how you can list all snapshots owned by you:
# aws ec2 describe-snapshots --profile profile1 --owner-id YOUR_AWS_ACCT_NUMBER --query "Snapshots[]"
Note the use of the --query option. It takes a parameter representing a JMESPath JSON query string. It's not trivial to figure out how to build these query strings, and I advise you to spend some time reading over the JMESPath tutorial and JMESPath examples.
In the example above, the query string is simply "Snapshots[]", which represents all the snapshots that are present in the AWS account associated with the profile profile1. The default output in our case is JSON, but you can specify --output text at the aws command line if you want to see each snapshot on its own line of text.
Let's assume that when you create the snapshots, you specify a description with contains PROD or STAGE for EBS volumes attached to production and stage EC2 instances respectively. If you want to only display snapshots containing the string PROD, you would do:
# aws ec2 describe-snapshots --profile profile1 --owner-id YOUR_AWS_ACCT_NUMBER--query "Snapshots[?contains(Description, \`PROD\`) == \`true\`]" --output text
The Snapshots[] array now contains a condition represented by the question mark ?. The condition uses the contains() function included in the JMESPath specification, and is applied against the Description field of each object in the Snapshots[] array, verifying that it contains the string PROD.  Note the use of backquotes surrounding the strings PROD and true in the condition. I spent some quality time troubleshooting my queries when I used single or double quotes with no avail. The backquotes also need to be escaped so that the shell doesn't interpret them as commands to be executed.
To restrict the PROD snapshots even further, to the ones older than say 7 days ago, you can do something like this:
DAYS=7TARGET_DATE=`date --date="$DAYS day ago" +%Y-%m-%d`
# aws ec2 describe-snapshots --profile profile1 --owner-id YOUR_AWS_ACCT_NUMBER--query "Snapshots[?contains(Description, \`PROD\`) == \`true\`]|[?StartTime < \`$TARGET_DATE\`]" --output text
Here I used the StartTime field of the objects in the Snapshots[] array and compared it against the target date. In this case, string comparison is good enough for the query to work.
In all the examples above, the aws command returned a subset of the Snapshots[] array and displayed all fields for each object in the array. If you wanted to display specific fields, let's say the ID, the start time and the description of each snapshot, you would run:
# aws ec2 describe-snapshots --profile profile1 --owner-id YOUR_AWS_ACCT_NUMBER--query "Snapshots[?contains(Description, \`PROD\`) == \`true\`]|[?StartTime < \`$TARGET_DATE\`].[SnapshotId,StartTime,Description]" --output text
To delete old snapshots, you can use the aws ec2 delete-snapshot command, which needs a snapshot ID as a parameter. You could use the command above to list only the SnapshotId for snapshots older than N days, then for each of these IDs, run something like this:
# aws ec2 delete-snapshot --profile profile1 --snapshot-id $id
All this is well and good when you run these commands interactively at the shell. However, I had no luck running them out of cron. The backquotes resulted in boto3 syntax errors. I had to do it the hard way, by listing all snapshots first, then going all in with sed and awk:
aws ec2 describe-snapshots --profile profile1 --owner-id YOUR_AWS_ACCT_NUMBER --output=text --query "Snapshots[].[SnapshotId,StartTime,Description]"  > $TMP_SNAPS
DAYS=7TARGET_DATE=`date --date="$DAYS day ago" +%Y-%m-%d`
cat $TMP_SNAPS | grep PROD | sed 's/T[0-9][0-9]:[0-9][0-9]:[0-9][0-9].000Z//' | awk -v target_date="$TARGET_DATE" '{if ($2 < target_date){print}}' > $TMP_PROD_SNAPS
for sid in `awk '{print $1}' $TMP_PROD_SNAPS` ; do echo Deleting PROD snapshot $sid aws ec2 delete-snapshot --profile $PROFILE --region $REGION --snapshot-id $siddone
Ugly, but it works out of cron. Hope it helps somebody out there.

July 14th 2016: I initially forgot to include this very good blog post from Joseph Lawson on advanced JMESPath usage with the AWS CLI.

Cracking the Code Interview

From the Editor of Methods & Tools - Wed, 07/13/2016 - 14:03
Software development interviews are a different breed from other interviews and, as such, require specialized skills and techniques. This talk will teach you how to prepare for coding interviews, what top companies like Google, Amazon, and Microsoft really look for, and how to tackle the toughest programming and algorithm problems. This is not a fluffy […]

The Bestselling Software Intended for People Who Couldn’t Use It.

James Bach’s Blog - Mon, 06/27/2016 - 16:26

In 1983, my boss, Dale Disharoon, designed a little game called Alphabet Zoo. My job was to write the Commodore 64 and Apple II versions of that game. Alphabet Zoo is a game for kids who are learning to read. The child uses a joystick to move a little character through a maze, collecting letters to spell words.

We did no user testing on this game until the very day we sent the final software to the publisher. On that day, we discovered that our target users (five year-olds) did not have the ability to use a joystick well enough to play the game. They just got frustrated and gave up.

We shipped anyway.

This game became a bestseller.


It was placed on a list of educational games recommended by the National Education Association.


Source: Information Please Almanac, 1986

So, how to explain this?

Some years later, when I became a father, I understood. My son was able to play a lot of games that were too hard for him because I operated the controls. I spoke to at least one dad who did exactly that with Alphabet Zoo.

I guess the moral of the story is: we don’t necessarily know the value of our own creations.

Categories: Testing & QA

Continuum of Design

Actively Lazy - Tue, 05/17/2016 - 06:45

How best to organise your code? At one end of the scale there is a god class – a single, massive entity that stores all possible functions; at the other end of the scale are hundreds of static methods, each in their own class. In general, these two extremes are both terrible designs. But there is a continuum of alternatives between these two extremes – choosing where along this scale your code should be is often difficult to judge and often changes over time.

Why structure code at all?

It’s a fair enough question – what is so bad about the two extremes? They are, really, more similar to each other than any of the points between. In one there is a single class with every method in it; in the other, I have static methods one-per class or maybe I’m using a language which doesn’t require me to even group by classes. In both cases, there is no organisation. No structure larger than methods with which to help organise or group related code.

Organising code is about making it easier for human beings to understand. The compiler or runtime doesn’t care how your code is organised, it will run it just the same. It will work the same and look the same – from the outside there is no difference. But for a developer making changes, the difference is critical. How do I find what I need to change? How can I be sure what my change will impact? How can I find other similar things that might be affected? These are all questions we have to answer when making changes to code and they require us to be able to reason about the code.

Bounded contexts help contain understanding – by limiting the size of the problem I need to think about at any one time I can focus on a smaller portion of it, but even within that bounded context organisation helps. It is much easier for me to understand how 100 methods work when they are grouped into 40 classes, with relationships between the classes that make sense in the domain – than it is for me to understand a flat list of 100 methods.

An Example

Let’s imagine we’re writing software to manage a small library (for you kids too young to remember: a library is a place where you can borrow books and return them when you’re done reading them; a book is like a physically printed blog but without the comments). We can imagine the kinds of things (methods) this system might support:

  • Add new title to the catalog
  • Add new copy of title
  • Remove copy of a title
  • User borrows a copy
  • User returns a copy
  • Register a new user
  • Print barcode for new copy
  • Fine a user for a late return
  • Pay outstanding fines for a user
  • Find title by ISBN
  • Find title by name, author
  • Find copy by scanned id
  • Change user’s address
  • List copies user has already borrowed
  • Print overdue book letter for user

This toy example is small enough that written in one-class it would probably be manageable; pretty horrible, but manageable. How else could we go about organising this?

Horizontal Split

We could split functionality horizontally: by technical concern. For example, by the database that data is stored in; or by the messaging system that’s used. This can work well in some cases, but can often lead to more god classes because your class will be as large as the surface area of the boundary layer. If this is small and likely to remain small it can be a good choice, but all too often the boundary is large or grows over time and you have another god class.

For example, functionality related to payments might be grouped simply into a PaymentsProvider – with only one or two methods we might decide it is unlikely to grow larger. Less obviously, we might group printing related functionality into a PrinterManager – while it might only have two methods now, if we later start printing marketing material or management reports the methods become less closely related and we have the beginnings of a god class.

Vertical Split

The other obvious way to organise functionality is vertically – group methods that relate to the same domain concept together. For example, we could group some of our methods into a LendingManager:

  • User borrows a copy
  • User returns a copy
  • Register a new user
  • Fine a user for a late return
  • Find copy by scanned id
  • List copies user has already borrowed

Even in our toy example this class already has six public methods. A coarse grained grouping like this often ends up being called a SomethingManager or TheOtherService. While this is sometimes a good way to group methods, the lack of clear boundary means new functionality is easily added over time and we grow ourselves a new god class.

A more fine-grained vertical grouping would organise methods into the domain objects they relate to – the recognisable nouns in the domain, where the methods are operations on those nouns. The nouns in our library example are obvious: Catalog, Title, Copy, User. Each of these has two or three public methods – but to understand the system at the outer level I only need to understand the four main domain objects and how they relate to each other, not all the individual methods they contain.

The fine-grained structure allows us to reason about a larger system than if it was unstructured. The extents of the domain objects should be relatively clear, if we add new methods to Title it is because there is some new behaviour in the domain that relates to Title – functionality should only grow slowly, we should avoid new god classes; instead new functionality will tend to add new classes, with new relationships to the existing ones.

What’s the Right Way?

Obviously there’s no right answer in all situations. Even in our toy example it’s clear to see that different structures make sense for different areas of functionality. There are a range of design choices and no right or wrong answers. Different designs will ultimately all solve the same problem, but humans will find some designs easier to understand and change than others. This is what makes design such a hard problem: the right answer isn’t always obvious and might not be known for some time. Even worse, the right design today can look like the wrong design tomorrow.

Categories: Programming, Testing & QA

Message Obsession

Mistaeks I Hav Made - Nat Pryce - Wed, 03/23/2016 - 23:19
I’ve noticed a “code smell” in object-oriented code that I call “Message Obsession”. I find Message Obsession causes similar difficulties to Primitive Obsession. However, Message Obsession appears to be the complete opposite of Primitive Obsession. Refactoring to address the difficulties caused by either Primitive Obsession or Message Obsession leads to the same design. To demonstrate the Message Obsession, and to compare it to Primitive Obsession, imagine a game in which a robot moves around a grid of tiles. Primitive Obsession The code below, which moves the robot east and then north, suffers from Primitive Obsession. Domain concepts – direction of movement, in this case – are held as multiple primitive data types instead of being modelled explicitly. robot.move(0,1) robot.move(-1,0) It’s awkward to work with this interface because you have to pass the deltaX and deltaY coordinates around as a pair of values instead of as a single Direction value. This can lead to code duplication. Any code that has to store moves will have to define a structure to do so. Code that has to perform calculations on directions – to add, invert or rotate directions, for example – will do so via helper functions or inline calculations. This logic ends up duplicated all over the code because programmers working in one area do not know that programmers working in other areas are writing the the same calculations. Primitive Obsession can also lead to errors. Are the coordinates passed to the move function as x and y or row and column? The type system won’t catch the error because both coordinates are integers. Named parameters help, of course, but not all languages support them. Message Obsession Programmers know about the drawbacks of Primitive Obsession, but sometimes create designs that go too far the other way, to the smell I call “Message Obsession”. A design that suffers from Message Obsession does not pass around primitive values – it does not pass around values at all. Instead, objects implement several similar methods, each of which manipulates the same fields used to store primitive values inside the object. Message Obsession leads to code like: robot.moveEast() robot.moveNorth() It’s awkward to work with this kind of interface because you can’t pass around, store or perform calculations on the direction at all. Symptoms Symptoms caused by this smell include… …class hierarchies that mirror the direction methods. For example, a hierarchy of Command classes with a concrete class for each direction the robot can move: interface MoveCommand { fun apply(r: robot) } object MoveNorth : MoveCommand { fun applyTo(r: robot) = r.moveNorth() } object MoveSouth : MoveCommand { fun applyTo(r: robot) = r.moveSouth() } ... etc. ... …dispatching with conditional statements: when (keyCode) { UP_ARROW -> robot.moveNorth() DOWN_ARROW -> robot.moveSouth() ... etc. ... } …using anonymous functions to treat directions as first-class values. val movesByKeyCode : MapUnit> = mapOf( UP_ARROW to {robot -> robot.moveNorth()}, DOWN_ARROW to {robot -> robot.moveSouth()}, ... etc. ... ) Cure The duplication in the method names shows that, as with primitive obsession, there’s a value type to be factored out. Let’s call it “Direction”. The names of the methods show that we’ll need some constant Direction values and what those constants should be called. We really want code that moves the robot to look like: robot.move(east) robot.move(north) We could define the Direction type as: data class Direction(val deltaX: Int, val deltaY: Int) And direction constants as: val north = Direction( 0, -1) val east = Direction( 1, 0) val south = Direction( 0, 1) val west = Direction(-1, 0) Now directions are first class values, they can be stored in collections, mapped to and from user input events, rotated, inverted, etc. For example: val movementKeyBindings: Map The value type acts as an “attractor” for behaviour. We can add operations to it to rotate or invert directions, for example. fun Direction.inverse() = Direction(-deltaX, -deltaY) fun Direction.rotatedClockwise() = Direction(deltaY, -deltaX) fun Direction.rotatedAnticlockwise() = Direction(-deltaY, deltaX) ... etc. ... Those rotatedClockwise and rotatedAnticlockwise methods also exhibit Message Obsession! Maybe we will need to introduce a RightAngleTurn type as the system evolves… Implicit Duplication Sometimes the Message Obsession smell is not obvious from the names. The duplication in naming is missing or implicit. For example, in the term “move” is not used in following code, but is implied by the behaviour of the east and north methods. robot.east() robot.north() The missing concept of “move” has to be named before the duplication stands out. Refactoring too Far, and Back Again I’m struck by how Message Obsession acts as an opposite of primitive obsession. I have often seen Message Obsession appearing as a reaction to Primitive Obsession, pushing the design too far in the opposite direction. The design eventually settles on a compositional style as the need for first-class values arises and the drawbacks of Message Obsession become clear.
Categories: Programming, Testing & QA

Early Impressions of Kotlin

Mistaeks I Hav Made - Nat Pryce - Thu, 12/31/2015 - 00:07
We’ve been using the Kotlin programming language for a few weeks on our latest project to perform technical experiments, explore the problem space, and write a few HTTP services. I’ve also ported Hamcrest to Kotlin, as HamKrest, to help us write tests, and written a small library for type safe configuration of our services. Why Kotlin? The organisation I’m working with has mature infrastructure for deploying JVM services in their internal PaaS cloud. They use a mix of Java and Scala but have found Scala builds too slow. Watching my colleague struggle to use Java 8 streams to write what should have been basic functional map-and-fold code, I decided to have a look at some other “post-Java” JVM languages. We wanted a typed language. We wanted language aware editing. And we wanted a language that had an organisation behind its development and enough people using it that we could get questions answered, even the stupid ones we’d be likely to ask while learning. That eliminated dynamically typed languages (Groovy, JavaScript, Clojure) and languages that are less popular or have small, informal development teams behind them (Xtend, Gosu, Fantom, Frege, etc.). In the end it came down to Red Hat’s Ceylon and JetBrains’ Kotlin. Of the two, Ceylon is the most innovative, and therefore (to me, anyway) the most interesting, but Kotlin met more of our criteria. Kotlin has a more active community, is being used for commercial development by JetBrains and a number of other companies1, has good editing support within IntelliJ, has an active community on social media, and promises easy interop with existing Java libraries. Good points Kotlin was very quick to learn. Kotlin is a conservative increment to Java that smooths off a lot of Java’s rough edges. Kotlin is small and regular, with few special cases and gotchas to learn. In many ways it feels a bit like a compiled, typed Python with curly brace syntax. The type system is a breath of fresh air compared to Java. Type inferencing makes code less cluttered. There is no distinction between primitive & reference types. Generics and subtyping work together far better than Java: the type system uses declaration site variance, not use site variance, and variance does not usually have to be specified at all for functions. You never have unavoidable compiler warnings, as you do with Java’s type system. Functional programming is more convenient in Kotlin than Java. You can define free-standing functions and constants at the top level of a module, instead of having to define them in a “Utils” class. Function definitions can be nested. There is language support for immutable value types (aka “data classes”) and algebraic data types (“sealed classes”). Null safety is enforced by the type system and variables and fields are non-nullable by default. The language defines standard function types and a lambda syntax for anonymous functions. You can define extension methods on existing types. They are only syntactic sugar for free-standing functions, but nevertheless can lead to more concise, expressive code. The Kotlin standard library defines a number of useful extension methods, especially on the Iterable and String types. Kotlin supports, and carefully controls, operator overloading. You can overload operators that act as functions (e.g. arithmetic, comparison, function call) but not those that perform flow control (e.g. short-cut logical operators). You cannot define your own operators, which will stop me going down the rabbit hole of “ascii-art programming”, which I found hard to resist when writing Scala. Class definitions require a lot less boilerplate than in Java. Using old fashioned getter-and-setter style Java code is made more convenient by language support for bean properties. Anonymous extension methods (borrowed from Groovy, I believe) make domain specific embedded languages easier to write. Method chaining builder APIs are unnecessary in Kotlin and more awkward than using the apply function to set properties of an object. Kotlin has a philosophy of preferring behaviour to be explicitly specified. For example, there is no automatic coercion between numeric types, even from low to high precision. That means you have to explicitly convert from Int to Long, for example. I like that, but I expect some people will find it annoying. Surprises I didn’t miss destructuring pattern match Kotlin doesn’t have destructuring pattern match & language support for tuples. A limited form of destructuring can be used in for loops and assignments. The when expression is an alternative conditional expression that doesn’t do destructuring. I was surprised to find that I didn’t really miss destructuring. Without tuples you have to use data classes with named fields. The flow-sensitive typing then works rather well where destructuring would be necessary in a language where the type system is not flow-sensitive. For example: sealed class Result { class Success(val value: T) : Result() class Failure(val exception: Throwable): Result() } ... val e : Result = doSomething() if (e is Result.Success) { // e is known to be a Result.Success and // can be used as such without a downcast println(e.value) } Sealed classes cannot be data classes An algebraic data type cannot be be a data class, so you have to write equals, hashCode, etc. yourself. This is a surprising omission, since I always want an algebraic data type to be a value type. The Kotlin developers say that this will be fixed after version 1.0 is released. Functions are not quite first-class objects There’s a difference between f1 and f2 below: val f1 = {i:Int -> i+1} fun f2(i:Int) = i + 1 Code can refer to the value of f1 directly, but must use the “::” operator to obtain a reference to f2: val f = ::f2 Null safe operators push me to write more extension functions The null safe operators (?. and ?:) only help when dereferencing a nullable reference. When you have a nullable reference and want to pass a value (if it exists) to a function as a parameter, you have to use the awkward construct: nullableThing?.let{ thing -> fn(thing, other, parameters) } I find myself refactoring my functions to extension functions to reduce the syntactic noise, letting me rewrite the code above as: nullableThing?.fn(other, parameters) Is that a good thing? I’m not sure. Sometimes it feels a little forced. Companion objects Kotlin borrows the concept of companion objects from Scala. Why can’t classes be objects? Perhaps it’s a limitation of the JVM but compared to, say, Python, it feels clunky. Frustrations I only have a couple of frustrations with Kotlin. Lack of polymorphism Operators and the for loop are syntactic sugar that desugar to method calls. The methods are invoked statically and structurally — the target does not need to implement an interface that defines the method. For example, a + b desugars to a.add(b). However, there’s no way to write a generic function that sums its parameters, because add is not defined on an interface that can be used as an upper bound. There’s no way to define the following function for any type that has an add operator method. fun sum(T first, T second) : T = first + second You would have to define overloads for different types, but then wouldn’t be able to call sum within a generic function. fun sum(first: Int, second: Int) = first + second fun sum(first: Long, second: Long) = first + second fun sum(first: Double, second: Double)= first + second fun sub(first: Matrix, second: Matrix)= first + second fun sum(first: Money, second: Money)= first + second ... That kind of duplication is what generics are meant to avoid! Extension methods are also statically bound. That means you cannot write a generic function that calls an extension method on the value of a type parameter. For example, the standard library defines the same extension methods on unrelated types — forEach, map, flatMap, fold, etc. But there is no concept of, for example, “mappable” or “foldable”. Nor is there a way of defining such a concept and applying it to existing types to allow you to write generic functions over unrelated types. Kotlin doesn’t have higher kinded types that would let you express this kind of generic function or type classes that would let you add polymorphism without modifying existing type definitions. Compared to Rust’s traits, which support extension methods, operator overloading, type classes for parametric polymorphism, and interfaces for subtype polymorphism, Kotlin’s monomorphic extension methods and operator overloading are quite limited and do not help refactor duplicated logic. However, for typical monomorphic, procedural Java code this is probably not an issue. Optionality is a Special Case Kotlin’s support for nullable types is implemented by a special case in the type system and special case operators that only apply to nullable references. The operators do not desugar to methods that can be implemented for other types. For example, optionalThing?.foo can be considered a map of the function {thing -> thing.foo} over the option optionalThing. If foo itself is nullable, then it can be considered a flatMap. But the expressions do not desugar to map and flatMap And if you want to map or flatMap a function, you have to use a different syntax: optionalThing?.let(theFunction). For typical Java code, which is monomorphic and uses null references with wild abandon, language support for nullability is invaluable. But I would find it more convenient if it could be used polymorphically, or if Kotlin used a common naming convention for optional and other functor types. Null safety is not enforced when you interop with Java code. And you do that a lot. Kotlin doesn’t have many libraries or much of a runtime and makes it easy to call existing Java libraries directly. Kotlin expects you to know what you’re doing with respect to null references when calling Java code, and doesn’t force you to treat every value returned from Java as nullable. As a result, null safety is a bit of an illusion in a lot of our Kotlin code and it has come back to bite us. Conclusion The code we’ve been writing has been a mix of coordinating and piping data between existing Java libraries – Apache HTTP client, Undertow HTTP server, JDBC, Sesame, JSON and XML parsers, and so on – and algorithmic code that analyses human readable text. For that kind of work, Kotlin has been very useful. The Kotlin code is far more concise than the equivalent Java. In our domain models and algorithmic code, Kotlin’s type safety and especially null safety, has been a big help. Exactly how happy we’ve been with Kotlin has depended on the design style of the code we’re writing: For functional programming, Kotlin has occasionally been frustrating, because we cannot use parametric polymorphism to factor out duplicated logic as much as we’d like. For object-oriented programming, Kotlin’s concise syntax for class definitions and language support for delegation avoids a lot of Java’s boilerplate. However, most Java out there is monomorphic, procedural code moving data between “NOJOs” and APIs that expect objects to have “bean” getters and setters. Kotlin has made working with that kind of API much easier and far more concise than doing so in Java. As far as I can tell, RedHat sponsor development of Ceylon but do not actually use it to develop their own products. If I’m wrong, please let me know in the comments.↩
Categories: Programming, Testing & QA

Thought after Test Automation Day 2013

Henry Ford said “Obstacles are those frightful things you see when take your eyes off the goal.” After I’ve been to Test Automation Day last month I’m figuring out why industrializing testing doesn’t work. I try to put it in this negative perspective, because I think it works! But also when is it successful? A lot of times the remark from Ford is indeed the problem. People tend to see obstacles. Obstacles because of the thought that it’s not feasible to change something. They need to change. But that’s not an easy change.

After attending the #TAD2013 as it was on Twitter I saw a huge interest in better testing, faster testing, even cheaper testing by using tools to industrialize. Test automation has long been seen as an interesting option that can enable faster testing. it wasn’t always cheaper, especially the first time, but at least faster. As I see it it’ll enable better testing. “Better?” you may ask. Test automation itself doesn’t enable better testing, but by automating regression tests and simple work the tester can focus on other areas of the quality.


And isn’t that the goal? In the end all people involved in a project want to deliver a high quality product, not full of bugs. But they also tend to see the obstacles. I see them less and less. New tools are so well advanced and automation testers are becoming smarter and smarter that they enable us to look beyond the obstacles. I would like to say look over the obstacles.

At the Test Automation Day I learned some new things, but it also proved something I already know; test automation is here to stay. We don’t need to focus on the obstacles, but should focus on the goal.

Categories: Testing & QA

The Toyota Way: The need for doing it right the first time

After WWII Toyota started developing its Toyota Production System (TPS); which was identified as ‘Lean’ in the 1990s. Taiichi Ohno, Shigeo Shingo and Eiji Toyoda developed the system between 1948 and 1975. In the myth surrounding the system it was not inspired by the American automotive industry, but from a visit to American supermarkets, Ohno saw the supermarket as model for what he was trying to accomplish in the factor and perfect the Just-in-Time (JIT) production system. While accomplishing this low inventory levels were a key outcome of the TPS, and an important element of the philosophy behind its system is to work intelligently and eliminate waste so that only minimal inventory is needed.

As TPS and Lean have their own principles as outlined by Toyota:

  • Long-term Philosophy
  • Right process will produce the right results
  • Value to organization by developing people
  • Solving root problems drives organizational learning

As these principles were summed up and published by Toyota in 2001, by naming it “The Toyota Way 2001”. It consists the above named principles in two key areas: Continuous Improvement, and Respect for People.


The principles for a continuous improvement include establishing a long-term vision, working on challenges, continual innovation, and going to the source of the issue or problem. The principles relating to respect for people include ways of building respect and teamwork. When looking at the ALM all these principles come together in the ‘first time right’ approach already mentioned. And from Toyota’s view they were outlined as followed:

  • The right process will produce the right results
    • Create continuous process flow to bring problems to the surface.
    • Use the ‘pull’ system to avoid overproduction (kanban)
    • Level out the workload (heijunka).
    • Build a culture of stopping to fix problems, to get quality right from the first (jidoka)
  • Continuously solving root problems drives organizational learning
    • Go and see for yourself to thoroughly understand the situation (Genchi Genbutsu);
    • Make decisions slowly by consensus, thoroughly considering all options (nemawashi); implement decisions rapidly;
    • Become a learning organization through relentless reflection (hansei) and continuous improvement (kaizen).
Let’s do it right now!

As the economy is changing and IT is more common sense throughout ore everyday life the need for good quality software products has never been this high. Software issues create bigger and bigger issues in our lives. Think about trains that cannot ride due to software issues, bank clients that have no access to their bank accounts, and people oversleeping because their alarm app didn’t work on their iPhone. As Capers Jones [Jones, 2011] states in his 2011 study that “software is blamed for more major business problems than any other man-made product” and that “poor quality has become one of the most expensive topics in human history”. The improvement of software quality is a key topic for all industries.

 Right the first time vs jidoka

In both TPS and Lean autonomation or jidoka are used. Autonomation can be described as ‘intelligent autonomation’, it means that when an abnormal situation arises the ‘machine’ stops and fix the abnormality. Autonomation prevents the production of defective products, eliminates overproduction, and focuses attention on understanding the problem and ensuring that it never recurs; a quality control process that applies the following four principles:

  • Detect the abnormality.
  • Stop.
  • Fix or correct the immediate condition.
  • Investigate the root cause and install a countermeasure.
Find defects as early as possible

In other words autonomation helps to get quality right the first time perfectly. With IT projects being different from the Toyota car production line, ‘perfectly’ may be a bit too much, but the process around quality assurance should be the same:

  • Find the defect.
  • Stop.
  • Fix or correct the error.
  • Investigate the root cause and take countermeasures.

The defect should be found as early as possible to be fixed as early as possible. And as with Lean and TPS the reason behind this is to make it possible to address the identification and correction of defects immediately in the process.

Categories: Testing & QA

When will you start with test automation?

I just came back from vacation and when I started again I noticed a slight change in resource requests I now see coming by; as almost all requests are with a statement around test automation. In the last two days I had two separate sessions around test automation tools. Has test automation all of a sudden become more important? Did people follow up on my last post, where I state that tools are a prerequisite in testing today, or actually: yesterday!

If you missed the latest cycle in new tools for test automation you’re either an ostrich with your head in the ground (“sorry vacation was in Southern Africa”), or you just simply still afraid. Afraid of change that test automation would cannibalize your manual test execution.

Test automation is not anymore that it takes over test execution in a very complex and unmanageable way. No it offers higher efficiency on test design, test execution, but also more options to test certain non-functional parts of applications – that could not be done without those tools – like security and performance, and virtual environments to do end-2-end tests without test environments that are down all the time. Tools are now also offering more support for testing mobile solutions. Tools are everywhere!


Test automation offers us testers the opportunity to do more, faster, les risky, and cheaper. I set these words specifically in that order. Test automation is often seen as a way to do test cheaper. You can, but you also can do more, for instance:

  • Let the tool do the checks and you explore the application further;
  • Setup a virtual test environment that doesn’t go down after 1 hour of use and test more in the same time;
  • Create and execute more test cases by generating and executing them automatically.
  • Get higher quality by really doing a thorough regression test, instead of a simple check, to find integration errors.

There are enough reasons to work on test automation and I don’t see why not. I think now it is even time for the next step in test automation. What that is time will tell, but I look forward to hearing that at the Test Automation Day in June. Where Bryan Bakker will tell more on this next step in his presentation “Design for Testability – the next step in Test Automation”. After the congress I’ll post my ideas here.

Categories: Testing & QA