Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator/categories/2&page=1' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Testing & QA
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

The Bestselling Software Intended for People Who Couldn’t Use It.

James Bach’s Blog - Mon, 06/27/2016 - 16:26

In 1983, my boss, Dale Disharoon, designed a little game called Alphabet Zoo. My job was to write the Commodore 64 and Apple II versions of that game. Alphabet Zoo is a game for kids who are learning to read. The child uses a joystick to move a little character through a maze, collecting letters to spell words.

We did no user testing on this game until the very day we sent the final software to the publisher. On that day, we discovered that our target users (five year-olds) did not have the ability to use a joystick well enough to play the game. They just got frustrated and gave up.

We shipped anyway.

This game became a bestseller.

alphabet-zoo

It was placed on a list of educational games recommended by the National Education Association.

alphabet-zoo-list

Source: Information Please Almanac, 1986

So, how to explain this?

Some years later, when I became a father, I understood. My son was able to play a lot of games that were too hard for him because I operated the controls. I spoke to at least one dad who did exactly that with Alphabet Zoo.

I guess the moral of the story is: we don’t necessarily know the value of our own creations.

Categories: Testing & QA

Agile Hiring, Load Testing & Goal Management in Methods & Tools Summer 2016 issue

From the Editor of Methods & Tools - Tue, 06/21/2016 - 08:02
Methods & Tools has published its Summer 2016 issue that discusses hiring for agility, load testing scripts errors, managing with goals on every level and Behavior-Driven Development (BDD) with the open source Turnip tool. Methods & Tools is a free e-magazine for software developers, testers and project managers. * Hiring for Agility – Mindset Matters […]

Running Jenkins jobs in Docker containers

Agile Testing - Grig Gheorghiu - Fri, 06/17/2016 - 00:08
One of my main tasks at work is to configure Jenkins to act as a hub for all the deployment and automated testing jobs we run. We use CloudBees Jenkins Enterprise, mostly for its Role-Based Access Control plugin, which allows us to create one Jenkins folder per project/application and establish fine grained access control to that folder for groups of users. We also make heavy use of the Jenkins Enterprise Pipeline features (which I think are also available these days in the open source version).

Our Jenkins infrastructure is composed of a master node and several executor nodes which can run jobs in parallel if needed.

One pattern that my colleague Will Wright and I have decided upon is to run all Jenkins jobs as Docker containers. This way, we only need to install Docker Engine on the master node and the executor nodes. No need to install any project-specific pre-requisites or dependencies on every Jenkins node. All of these dependencies and pre-reqs are instead packaged in the Docker containers. It's a simple but powerful idea, that has worked very well for us. One of the nice things about this pattern is that you can keep adding various types of automated tests. If it can run from the command line, then it can run in a Docker container, which means you can run it from Jenkins!

I have seen this pattern discussed in multiple places recently, for example in this blog post about "Using Docker for a more flexible Jenkins".

Here are some examples of Jenkins jobs that we create for a given project/application:
  • a deployment job that runs Capistrano in its own Docker container, against targets in various environments (development, staging, production); this is a Pipeline script written in Groovy, which can call other jobs below
  • a Web UI testing job that runs the Selenium Python WebDriver and drives Firefox in headless mode (see my previous post on how to do this with Docker)
  • a JavaScript syntax checking job that runs JSHint against the application's JS files
  • an SSL scanner/checker that runs SSLyze against the application endpoints
We also run other types of tasks, such as running an AWS CLI command to perform certain actions, for example to invalidate a CloudFront resource. I am going to show here how we create a Docker image for one of these jobs, how we test it locally, and how we then integrate it in Jenkins.

I'll use as an example a simple Docker image that installs the AWS CLI package and runs a command when the container is invoked via 'docker run'.
I assume you have a local version of Docker installed. If you are on a Mac, you can use Docker Toolbox, or, if you are lucky and got access to it, you can use the native Docker for Mac. In any case,  I will assume that you have a local directory called awscli with the following Dockerfile in it:
FROM ubuntu:14.04
MAINTAINER You Yourself <you@example.com>
# disable interactive functionsARG DEBIAN_FRONTEND=noninteractiveENV AWS_ACCESS_KEY_ID=""ENV AWS_SECRET_ACCESS_KEY=""ENV AWS_COMMAND=""
RUN apt-get update && \    apt-get install -y python-pip && \    pip install awscli
WORKDIR /root
CMD (export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID; export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY; $AWS_COMMAND)
As I mentioned, this simply installs the awscli Python package via pip, then runs a command given as an environment variable when you invoke 'docker run'. It also uses two other environment variables that contain the AWS access key ID and secret access key. You don't want to hardcode these secrets in the Dockerfile and have them end up on GitHub.
The next step is to build an image based on this Dockerfile. I'll call the image awscli and I'll tag it as local:

$ docker build -t awscli:local .

Then you can run a container based on this image. The command line looks a bit complicated because I am passing (via the -e switch) the 3 environment variables discussed above:


$ docker run --rm -e AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=YOUR_SECRET_ACCESS_KEY -e AWS_COMMAND='aws cloudfront create-invalidation --distribution-id=abcdef --invalidation-batch Paths={Quantity=1,Items=[/media/*]},CallerReference=my-invalidation-123456' awscli:local
(where distribution-id needs to be the actual ID of your CloudFront distribution, and CallerReference needs to be unique per invalidation)

If all goes well, you should see the output of the 'aws cloudfront create-invalidation' command.

In our infrastructure, we have a special GitHub repository where we check in the various folders containing the Dockerfiles and any static files that need to be copied over to the Docker images. When we push the awscli directory to GitHub for example, we have a Jenkins job that will be notified of that commit and that will build the Docker image (similarly to how we did it locally with 'docker build'), then it will 'docker push' the image to a private AWS ECR repository we have.

Now let's assume we want to create a Jenkins job that will run this image as a container. First we define 2 secret credentials, specific to the Jenkins folder where we want to create the job (there are also global Jenkins credentials that can apply to all folders). These credentials are of type "Secret text" and contain the AWS access key ID and the AWS secret access key.

Then we create a new Jenkins job of type "Freestyle project" and call it cloudfront.invalidate. The build for this job is parameterized and contains 2 parameters: CF_ENVIRONMENT which is a drop-down containing the values "Staging" and "Production" referring to the CloudFront distribution we want to invalidate; and CF_RESOURCE, which is a text variable that needs to be set to the resource that needs to be invalidated (e.g. /media/*).

In the Build Environment section of the Jenkins job, we check "Use secret text(s) or file(s)" and add 2 Bindings, one for the first secret text credential containing the AWS access key ID, which we save in a variable called AWS_ACCESS_KEY_ID, and the other one for the second secret text credential containing the AWS secret access key, which we save in a variable called AWS_SECRET_ACCESS_KEY.

The Build section for this Jenkins job has a step of type "Execute shell" which uses the parameters and variables defined above and invokes 'docker run' using the path to the Docker image from our private ECR repository:

DISTRIBUTION_ID=MY_CF_DISTRIBUTION_ID_FOR_STAGING
if [ $CF_ENVIRONMENT == "PRODUCTION" ]; then
    DISTRIBUTION_ID=MY_CF_DISTRIBUTION_ID_FOR_PRODUCTION
fi

INVALIDATION_ID=jenkins-invalidation-`date +%Y%m%d%H%M%S`

COMMAND="aws cloudfront create-invalidation --distribution-id=$DISTRIBUTION_ID --invalidation-batch Paths={Quantity=1,Items=[$CF_RESOURCE]},CallerReference=$INVALIDATION_ID"

docker run --rm -e AWS_ACCESS_KEY_ID=$ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=$SECRET_ACCESS_KEY -e AWS_COMMAND="$COMMAND" MY_PRIVATE_ECR_ID.dkr.ecr.us-west-2.amazonaws.com/awscli


When this job is run, the Docker image gets pulled down from AWS ECR, then a container based on the image is run and then removed upon completion (that's what --rm does, so that no old containers are left around).

I'll write another post soon with some more examples of Jenkins jobs that we run as Docker containers to do Selenium test, JSHint testing and SSLyze scanning.






Software Development Linkopedia June 2016

From the Editor of Methods & Tools - Mon, 06/13/2016 - 14:35
Here is our monthly selection of knowledge on programming, software testing and project management. This month you will find some interesting information and opinions about software bugs, rewriting code, project routines, improving retrospectives, acceptance testing problems, #noprojects, DevOps and Agile (or not) software architecture. Blog: Software has bugs. This is normal. Blog: When to Rewrite […]

Quote of the Month June 2016

From the Editor of Methods & Tools - Mon, 06/06/2016 - 12:56
The only software which doesn’t change is dead software. Stopping software from changing is a fast way to kill it. Treating software development as a temporary endeavour kills it. Regarding software as temporary also means the teams which create it get regarded as temporary. Such teams get disbanded at “the end”. The developers scatter to […]

GTAC Diversity Scholarship

Google Testing Blog - Fri, 06/03/2016 - 14:35
by Lesley Katzen on behalf of the GTAC Diversity Committee

We are committed to increasing diversity at GTAC, and we believe the best way to do that is by making sure we have a diverse set of applicants to speak and attend. As part of that commitment, we are excited to announce that we will be offering travel scholarships this year.
Travel scholarships will be available for selected applicants from traditionally underrepresented groups in technology.

To be eligible for a grant to attend GTAC, applicants must:

  • Be 18 years of age or older.
  • Be from a traditionally underrepresented group in technology.
  • Work or study in Computer Science, Computer Engineering, Information Technology, or a technical field related to software testing.
  • Be able to attend core dates of GTAC, November 15th - 16th 2016 in Sunnyvale, CA.


To apply:
Please fill out the following form to be considered for a travel scholarship.
The deadline for submission is June 1st June 15th.  Scholarship recipients will be announced on June 30th July 15th. If you are selected, we will contact you with information on how to proceed with booking travel.


What the scholarship covers:
Google will pay for standard coach class airfare for selected scholarship recipients to San Francisco or San Jose, and 3 nights of accommodations in a hotel near the Sunnyvale campus. Breakfast and lunch will be provided for GTAC attendees and speakers on both days of the conference. We will also provide a $50.00 gift card for other incidentals such as airport transportation or meals. You will need to provide your own credit card to cover any hotel incidentals.


Google is dedicated to providing a harassment-free and inclusive conference experience for everyone. Our anti-harassment policy can be found at:
https://www.google.com/events/policy/anti-harassmentpolicy.html
Categories: Testing & QA

GTAC 2016 - Save the Date

Google Testing Blog - Fri, 06/03/2016 - 05:41
by Sonal Shah on behalf of the GTAC Committee


We are pleased to announce that the tenth GTAC (Google Test Automation Conference) will be held on Google’s campus in Sunnyvale (California, USA) on Tuesday and Wednesday, November 15th and 16th, 2016.  

Based on feedback from the last GTAC (2015) and the increasing demand every year, we have decided to keep GTAC on a fall schedule. This schedule is a change from what we previously announced.

The schedule for the next few months is:
May 1, 2016  - Registration opens for speakers and attendees.June 1, 2016 June 15, 2016 - Registration closes for speaker and attendee submissions.June 30, 2016 July 15, 2016 - Selected attendees will be notified.August 15, 2016 August 29, 2016 - Selected speakers will be notified.November 14, 2016 - Rehearsal day for speakers.November 15-16, 2016 - GTAC 2016!

As part of our efforts to increase diversity of speakers and attendees at GTAC,  we will be offering travel scholarships for selected applicants from traditionally underrepresented groups in technology.

Stay tuned to this blog and the GTAC website for information about attending or presenting at GTAC. Please do not hesitate to contact gtac2016@google.com if you have any questions. We look forward to seeing you there!
Categories: Testing & QA

GTAC 2016 Registration is now open!

Google Testing Blog - Fri, 06/03/2016 - 05:36
by Sonal Shah on behalf of the GTAC Committee

The GTAC (Google Test Automation Conference) 2016 application process is now open for presentation proposals and attendance. GTAC will be held at the Google Sunnyvale office on November 15th - 16th, 2016.

GTAC will be streamed live on YouTube again this year, so even if you cannot attend in person, you will be able to watch the conference remotely. We will post the livestream information as we get closer to the event, and recordings will be posted afterwards.

Speakers
Presentations are targeted at students, academics, and experienced engineers working on test automation. Full presentations are 30 minutes and lightning talks are 10 minutes. Speakers should be prepared for a question and answer session following their presentation.

Application
For presentation proposals and/or attendance, complete this form. We will be selecting about 25 talks and 300 attendees for the event. The selection process is not first come first serve (no need to rush your application), and we select a diverse group of engineers from various locations, company sizes, and technical backgrounds.

Deadline
The due date for both presentation and attendance applications is June 1st, 2016 June 15, 2016.

Cost
There are no registration fees, but speakers and attendees must arrange and pay for their own travel and accommodations.

More information
Please read our FAQ for most common questions
https://developers.google.com/google-test-automation-conference/2016/faq.
Categories: Testing & QA

GTAC 2016 Registration Deadline Extended

Google Testing Blog - Fri, 06/03/2016 - 04:58
by Sonal Shah on behalf of the GTAC Committee

Our goal in organizing GTAC each year is to make it a first-class conference, dedicated to presenting leading edge industry practices. The quality of submissions we've received for GTAC 2016 so far has been overwhelming. In order to include the best talks possible, we are extending the deadline for speaker and attendee submissions by 15 days. The new timelines are as follows:

June 1, 2016 June 15, 2016 - Last day for speaker, attendee and diversity scholarship submissions.
June 15, 2016 July 15, 2016 - Attendees and scholarship awardees will be notified of selection/rejection/waitlist status. Those on the waitlist will be notified as space becomes available.
August 15, 2016 August 29, 2016 - Selected speakers will be notified.

To register, please fill out this form.
To apply for diversity scholarship, please fill out this form.

The GTAC website has a list of frequently asked questions. Please do not hesitate to contact gtac2016@google.com if you still have any questions.

Categories: Testing & QA

Software Development Conferences Forecast May 2016

From the Editor of Methods & Tools - Mon, 05/30/2016 - 13:29
Here is a list of software development related conferences and events on Agile project management ( Scrum, Lean, Kanban), software testing and software quality, software architecture, programming (Java, .NET, JavaScript, Ruby, Python, PHP), DevOps and databases (NoSQL, MySQL, etc.) that will take place in the coming weeks and that have media partnerships with the Methods […]

Flaky Tests at Google and How We Mitigate Them

Google Testing Blog - Sat, 05/28/2016 - 01:34
by John Micco

At Google, we run a very large corpus of tests continuously to validate our code submissions. Everyone from developers to project managers rely on the results of these tests to make decisions about whether the system is ready for deployment or whether code changes are OK to submit. Productivity for developers at Google relies on the ability of the tests to find real problems with the code being changed or developed in a timely and reliable fashion.

Tests are run before submission (pre-submit testing) which gates submission and verifies that changes are acceptable, and again after submission (post-submit testing) to decide whether the project is ready to be released. In both cases, all of the tests for a particular project must report a passing result before submitting code or releasing a project.

Unfortunately, across our entire corpus of tests, we see a continual rate of about 1.5% of all test runs reporting a "flaky" result. We define a "flaky" test result as a test that exhibits both a passing and a failing result with the same code. There are many root causes why tests return flaky results, including concurrency, relying on non-deterministic or undefined behaviors, flaky third party code, infrastructure problems, etc. We have invested a lot of effort in removing flakiness from tests, but overall the insertion rate is about the same as the fix rate, meaning we are stuck with a certain rate of tests that provide value, but occasionally produce a flaky result. Almost 16% of our tests have some level of flakiness associated with them! This is a staggering number; it means that more than 1 in 7 of the tests written by our world-class engineers occasionally fail in a way not caused by changes to the code or tests.

When doing post-submit testing, our Continuous Integration (CI) system identifies when a passing test transitions to failing, so that we can investigate the code submission that caused the failure. What we find in practice is that about 84% of the transitions we observe from pass to fail involve a flaky test! This causes extra repetitive work to determine whether a new failure is a flaky result or a legitimate failure. It is quite common to ignore legitimate failures in flaky tests due to the high number of false-positives. At the very least, build monitors typically wait for additional CI cycles to run this test again to determine whether or not the test has been broken by a submission adding to the delay of identifying real problems and increasing the pool of changes that could contribute.

In addition to the cost of build monitoring, consider that the average project contains 1000 or so individual tests. To release a project, we require that all these tests pass with the latest code changes. If 1.5% of test results are flaky, 15 tests will likely fail, requiring expensive investigation by a build cop or developer. In some cases, developers dismiss a failing result as flaky only to later realize that it was a legitimate failure caused by the code. It is human nature to ignore alarms when there is a history of false signals coming from a system. For example, see this article about airline pilots ignoring an alarm on 737s. The same phenomenon occurs with pre-submit testing. The same 15 or so failing tests block submission and introduce costly delays into the core development process. Ignoring legitimate failures at this stage results in the submission of broken code.

We have several mitigation strategies for flaky tests during presubmit testing, including the ability to re-run only failing tests, and an option to re-run tests automatically when they fail. We even have a way to denote a test as flaky - causing it to report a failure only if it fails 3 times in a row. This reduces false positives, but encourages developers to ignore flakiness in their own tests unless their tests start failing 3 times in a row, which is hardly a perfect solution.
Imagine a 15 minute integration test marked as flaky that is broken by my code submission. The breakage will not be discovered until 3 executions of the test complete, or 45 minutes, after which it will need to be determined if the test is broken (and needs to be fixed) or if the test just flaked three times in a row.

Other mitigation strategies include:
  • A tool that monitors the flakiness of tests and if the flakiness is too high, it automatically quarantines the test. Quarantining removes the test from the critical path and files a bug for developers to reduce the flakiness. This prevents it from becoming a problem for developers, but could easily mask a real race condition or some other bug in the code being tested.
  • Another tool detects changes in the flakiness level of tests and works to identify the change that caused the test to change the level of flakiness.

In summary, test flakiness is an important problem, and Google is continuing to invest in detecting, mitigating, tracking, and fixing test flakiness throughout our code base. For example:
  • We have a new team dedicated to providing accurate and timely information about test flakiness to help developers and build monitors so that they know whether they are being harmed by test flakiness.
  • As we analyze the data from flaky test executions, we are seeing promising correlations with features that should enable us to identify a flaky result accurately without re-running the test.


By continually advancing the state of the art for teams at Google, we aim to remove the friction caused by test flakiness from the core developer workflows.

Categories: Testing & QA

Setting up AWS CloudFront for Magento

Agile Testing - Grig Gheorghiu - Thu, 05/26/2016 - 19:00
Here are some steps I jotted down for setting up AWS CloudFront as a CDN for the 3 asset directories that are used by Magento installations. I am assuming your Magento application servers are behind an ELB.


SSL certificate upload to AWS
Install aws command line utilities.
$ pip install awscli
Configure AWS credentials
Create IAM user and associate it with the IAMFullAccess policy. Run ‘aws configure’ and specify the user’s keys and the region.

Bring SSL key, certificate and intermediate certificate in current directory:
-rw-r--r-- 1 root root 4795 Apr 11 20:34 gd_bundle-g2-g1.crt-rw-r--r-- 1 root root 1830 Apr 11 20:34 wildcard.mydomain.com.crt-rw------- 1 root root 1675 Apr 11 20:34 wildcard.mydomain.com.key
Run following script for installing wildcard SSL certificate to be used in staging CloudFront setup:
$ cat add_ssl_cert_to_iam_for_prod_cloudfront.sh#!/bin/bash
aws iam upload-server-certificate --server-certificate-name WILDCARD_MYDOMAIN_COM_FOR_PROD_CF --certificate-body file://wildcard.mydomain.com.crt --private-key file://wildcard.mydomain.com.key --certificate-chain file://gd_bundle-g2-g1.crt --path /cloudfront/prod/

After uploading the SSL certificates, they will be available in drop-downs when configuring CloudFront for SSL.
Apache Cache-Control headers setup
  • Add these directives (modifying max-age accordingly) in all Apache vhosts, both for port 80 and for port 443
 <FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|js|css|swf)$">        Header set Cache-Control "max-age=604800, public" </FilesMatch>
CloudFront setup
  • Origin: prod ELB (mydomain-production-lb-9321962155.us-west-2.elb.amazonaws.com)
  • Alternate domain name: cdn.mydomain.com\
  • SSL certificate: ID_OF_CERTIFICATE_UPLOADED_ABOVE
  • Custom SSL client support: Only Clients that Support Server Name Indication (SNI)
  • Domain name: eg7ac9k0fa3qwc.cloudfront.net
  • Behaviors
    • /media/* /skin/* /js/*
    • Viewer protocol policy: HTTP and HTTPS
    • Allowed HTTP methods: GET, HEAD
    • Forward headers: None
    • Object caching: Use origin cache headers
    • Forward cookies: None
    • Forward query strings: Yes
    • Smooth streaming: No
    • Restrict viewer access: No
    • Compress objects automatically: No

DNS setup
  • cdn.mydomain.com is a CNAME pointing to the CloudFront domain name above eg7ac9k0fa3qwc.cloudfront.net

Magento setup
This depends on the version of Magento you are running (1.x or 2.x), but you want to look for settings for the Base Skin URL, Base Media URL and Base Javascript URL, which are usually under System->Configuration->General-Web. You need to set them to point to the domain name you set up as a CNAME to CloudFront.

Base Skin URL: http://cdn.mydomain.com/skin
Base Media URL: http://cdn.mydomain.com/media
Base Javascript URL: http://cdn.mydomain.com/js
More in-depth Magento-specific instructions for integrating with CloudFront are available here.

Quote of the Month May 2016

From the Editor of Methods & Tools - Mon, 05/23/2016 - 08:59
In other words, estimating is the key to unlocking the ability to commit. The ability to commit to near-term deliverables is the key to building a reliable, agile enterprise. So, there are good business reason why we need to take a harder look at this funny agile estimating thing. Source: Agile Software Requirements, Dean Leffingwell, […]

Software Development Linkopedia May 2016

From the Editor of Methods & Tools - Tue, 05/17/2016 - 13:44
Here is our monthly selection of knowledge on programming, software testing and project management. This month you will find some interesting information and opinions about software engineering myths, programming career, fixing things, optimization projects, mobile testing, minimum viable products, scaling Agile, lean business analysis and javaScript testing at Facebook. Blog: Exploding SoftwareEngineering Myths Blog: Single […]

Continuum of Design

Actively Lazy - Tue, 05/17/2016 - 06:45

How best to organise your code? At one end of the scale there is a god class – a single, massive entity that stores all possible functions; at the other end of the scale are hundreds of static methods, each in their own class. In general, these two extremes are both terrible designs. But there is a continuum of alternatives between these two extremes – choosing where along this scale your code should be is often difficult to judge and often changes over time.

Why structure code at all?

It’s a fair enough question – what is so bad about the two extremes? They are, really, more similar to each other than any of the points between. In one there is a single class with every method in it; in the other, I have static methods one-per class or maybe I’m using a language which doesn’t require me to even group by classes. In both cases, there is no organisation. No structure larger than methods with which to help organise or group related code.

Organising code is about making it easier for human beings to understand. The compiler or runtime doesn’t care how your code is organised, it will run it just the same. It will work the same and look the same – from the outside there is no difference. But for a developer making changes, the difference is critical. How do I find what I need to change? How can I be sure what my change will impact? How can I find other similar things that might be affected? These are all questions we have to answer when making changes to code and they require us to be able to reason about the code.

Bounded contexts help contain understanding – by limiting the size of the problem I need to think about at any one time I can focus on a smaller portion of it, but even within that bounded context organisation helps. It is much easier for me to understand how 100 methods work when they are grouped into 40 classes, with relationships between the classes that make sense in the domain – than it is for me to understand a flat list of 100 methods.

An Example

Let’s imagine we’re writing software to manage a small library (for you kids too young to remember: a library is a place where you can borrow books and return them when you’re done reading them; a book is like a physically printed blog but without the comments). We can imagine the kinds of things (methods) this system might support:

  • Add new title to the catalog
  • Add new copy of title
  • Remove copy of a title
  • User borrows a copy
  • User returns a copy
  • Register a new user
  • Print barcode for new copy
  • Fine a user for a late return
  • Pay outstanding fines for a user
  • Find title by ISBN
  • Find title by name, author
  • Find copy by scanned id
  • Change user’s address
  • List copies user has already borrowed
  • Print overdue book letter for user

This toy example is small enough that written in one-class it would probably be manageable; pretty horrible, but manageable. How else could we go about organising this?

Horizontal Split

We could split functionality horizontally: by technical concern. For example, by the database that data is stored in; or by the messaging system that’s used. This can work well in some cases, but can often lead to more god classes because your class will be as large as the surface area of the boundary layer. If this is small and likely to remain small it can be a good choice, but all too often the boundary is large or grows over time and you have another god class.

For example, functionality related to payments might be grouped simply into a PaymentsProvider – with only one or two methods we might decide it is unlikely to grow larger. Less obviously, we might group printing related functionality into a PrinterManager – while it might only have two methods now, if we later start printing marketing material or management reports the methods become less closely related and we have the beginnings of a god class.

Vertical Split

The other obvious way to organise functionality is vertically – group methods that relate to the same domain concept together. For example, we could group some of our methods into a LendingManager:

  • User borrows a copy
  • User returns a copy
  • Register a new user
  • Fine a user for a late return
  • Find copy by scanned id
  • List copies user has already borrowed

Even in our toy example this class already has six public methods. A coarse grained grouping like this often ends up being called a SomethingManager or TheOtherService. While this is sometimes a good way to group methods, the lack of clear boundary means new functionality is easily added over time and we grow ourselves a new god class.

A more fine-grained vertical grouping would organise methods into the domain objects they relate to – the recognisable nouns in the domain, where the methods are operations on those nouns. The nouns in our library example are obvious: Catalog, Title, Copy, User. Each of these has two or three public methods – but to understand the system at the outer level I only need to understand the four main domain objects and how they relate to each other, not all the individual methods they contain.

The fine-grained structure allows us to reason about a larger system than if it was unstructured. The extents of the domain objects should be relatively clear, if we add new methods to Title it is because there is some new behaviour in the domain that relates to Title – functionality should only grow slowly, we should avoid new god classes; instead new functionality will tend to add new classes, with new relationships to the existing ones.

What’s the Right Way?

Obviously there’s no right answer in all situations. Even in our toy example it’s clear to see that different structures make sense for different areas of functionality. There are a range of design choices and no right or wrong answers. Different designs will ultimately all solve the same problem, but humans will find some designs easier to understand and change than others. This is what makes design such a hard problem: the right answer isn’t always obvious and might not be known for some time. Even worse, the right design today can look like the wrong design tomorrow.


Categories: Programming, Testing & QA

Sacred Agile

From the Editor of Methods & Tools - Wed, 05/11/2016 - 08:47
You know that when you deal with this type of topic, you are writing about a touchy subject. When I write that Agile has been sometimes sacralized, my aim is not to hurt any religious or philosophical belief, but rather to see how men behave when adopting new ideas. Raised as a catholic, what will […]

Message Obsession

Mistaeks I Hav Made - Nat Pryce - Wed, 03/23/2016 - 23:19
I’ve noticed a “code smell” in object-oriented code that I call “Message Obsession”. I find Message Obsession causes similar difficulties to Primitive Obsession. However, Message Obsession appears to be the complete opposite of Primitive Obsession. Refactoring to address the difficulties caused by either Primitive Obsession or Message Obsession leads to the same design. To demonstrate the Message Obsession, and to compare it to Primitive Obsession, imagine a game in which a robot moves around a grid of tiles. Primitive Obsession The code below, which moves the robot east and then north, suffers from Primitive Obsession. Domain concepts – direction of movement, in this case – are held as multiple primitive data types instead of being modelled explicitly. robot.move(0,1) robot.move(-1,0) It’s awkward to work with this interface because you have to pass the deltaX and deltaY coordinates around as a pair of values instead of as a single Direction value. This can lead to code duplication. Any code that has to store moves will have to define a structure to do so. Code that has to perform calculations on directions – to add, invert or rotate directions, for example – will do so via helper functions or inline calculations. This logic ends up duplicated all over the code because programmers working in one area do not know that programmers working in other areas are writing the the same calculations. Primitive Obsession can also lead to errors. Are the coordinates passed to the move function as x and y or row and column? The type system won’t catch the error because both coordinates are integers. Named parameters help, of course, but not all languages support them. Message Obsession Programmers know about the drawbacks of Primitive Obsession, but sometimes create designs that go too far the other way, to the smell I call “Message Obsession”. A design that suffers from Message Obsession does not pass around primitive values – it does not pass around values at all. Instead, objects implement several similar methods, each of which manipulates the same fields used to store primitive values inside the object. Message Obsession leads to code like: robot.moveEast() robot.moveNorth() It’s awkward to work with this kind of interface because you can’t pass around, store or perform calculations on the direction at all. Symptoms Symptoms caused by this smell include… …class hierarchies that mirror the direction methods. For example, a hierarchy of Command classes with a concrete class for each direction the robot can move: interface MoveCommand { fun apply(r: robot) } object MoveNorth : MoveCommand { fun applyTo(r: robot) = r.moveNorth() } object MoveSouth : MoveCommand { fun applyTo(r: robot) = r.moveSouth() } ... etc. ... …dispatching with conditional statements: when (keyCode) { UP_ARROW -> robot.moveNorth() DOWN_ARROW -> robot.moveSouth() ... etc. ... } …using anonymous functions to treat directions as first-class values. val movesByKeyCode : MapUnit> = mapOf( UP_ARROW to {robot -> robot.moveNorth()}, DOWN_ARROW to {robot -> robot.moveSouth()}, ... etc. ... ) Cure The duplication in the method names shows that, as with primitive obsession, there’s a value type to be factored out. Let’s call it “Direction”. The names of the methods show that we’ll need some constant Direction values and what those constants should be called. We really want code that moves the robot to look like: robot.move(east) robot.move(north) We could define the Direction type as: data class Direction(val deltaX: Int, val deltaY: Int) And direction constants as: val north = Direction( 0, -1) val east = Direction( 1, 0) val south = Direction( 0, 1) val west = Direction(-1, 0) Now directions are first class values, they can be stored in collections, mapped to and from user input events, rotated, inverted, etc. For example: val movementKeyBindings: Map The value type acts as an “attractor” for behaviour. We can add operations to it to rotate or invert directions, for example. fun Direction.inverse() = Direction(-deltaX, -deltaY) fun Direction.rotatedClockwise() = Direction(deltaY, -deltaX) fun Direction.rotatedAnticlockwise() = Direction(-deltaY, deltaX) ... etc. ... Those rotatedClockwise and rotatedAnticlockwise methods also exhibit Message Obsession! Maybe we will need to introduce a RightAngleTurn type as the system evolves… Implicit Duplication Sometimes the Message Obsession smell is not obvious from the names. The duplication in naming is missing or implicit. For example, in the term “move” is not used in following code, but is implied by the behaviour of the east and north methods. robot.east() robot.north() The missing concept of “move” has to be named before the duplication stands out. Refactoring too Far, and Back Again I’m struck by how Message Obsession acts as an opposite of primitive obsession. I have often seen Message Obsession appearing as a reaction to Primitive Obsession, pushing the design too far in the opposite direction. The design eventually settles on a compositional style as the need for first-class values arises and the drawbacks of Message Obsession become clear.
Categories: Programming, Testing & QA

Early Impressions of Kotlin

Mistaeks I Hav Made - Nat Pryce - Thu, 12/31/2015 - 00:07
We’ve been using the Kotlin programming language for a few weeks on our latest project to perform technical experiments, explore the problem space, and write a few HTTP services. I’ve also ported Hamcrest to Kotlin, as HamKrest, to help us write tests, and written a small library for type safe configuration of our services. Why Kotlin? The organisation I’m working with has mature infrastructure for deploying JVM services in their internal PaaS cloud. They use a mix of Java and Scala but have found Scala builds too slow. Watching my colleague struggle to use Java 8 streams to write what should have been basic functional map-and-fold code, I decided to have a look at some other “post-Java” JVM languages. We wanted a typed language. We wanted language aware editing. And we wanted a language that had an organisation behind its development and enough people using it that we could get questions answered, even the stupid ones we’d be likely to ask while learning. That eliminated dynamically typed languages (Groovy, JavaScript, Clojure) and languages that are less popular or have small, informal development teams behind them (Xtend, Gosu, Fantom, Frege, etc.). In the end it came down to Red Hat’s Ceylon and JetBrains’ Kotlin. Of the two, Ceylon is the most innovative, and therefore (to me, anyway) the most interesting, but Kotlin met more of our criteria. Kotlin has a more active community, is being used for commercial development by JetBrains and a number of other companies1, has good editing support within IntelliJ, has an active community on social media, and promises easy interop with existing Java libraries. Good points Kotlin was very quick to learn. Kotlin is a conservative increment to Java that smooths off a lot of Java’s rough edges. Kotlin is small and regular, with few special cases and gotchas to learn. In many ways it feels a bit like a compiled, typed Python with curly brace syntax. The type system is a breath of fresh air compared to Java. Type inferencing makes code less cluttered. There is no distinction between primitive & reference types. Generics and subtyping work together far better than Java: the type system uses declaration site variance, not use site variance, and variance does not usually have to be specified at all for functions. You never have unavoidable compiler warnings, as you do with Java’s type system. Functional programming is more convenient in Kotlin than Java. You can define free-standing functions and constants at the top level of a module, instead of having to define them in a “Utils” class. Function definitions can be nested. There is language support for immutable value types (aka “data classes”) and algebraic data types (“sealed classes”). Null safety is enforced by the type system and variables and fields are non-nullable by default. The language defines standard function types and a lambda syntax for anonymous functions. You can define extension methods on existing types. They are only syntactic sugar for free-standing functions, but nevertheless can lead to more concise, expressive code. The Kotlin standard library defines a number of useful extension methods, especially on the Iterable and String types. Kotlin supports, and carefully controls, operator overloading. You can overload operators that act as functions (e.g. arithmetic, comparison, function call) but not those that perform flow control (e.g. short-cut logical operators). You cannot define your own operators, which will stop me going down the rabbit hole of “ascii-art programming”, which I found hard to resist when writing Scala. Class definitions require a lot less boilerplate than in Java. Using old fashioned getter-and-setter style Java code is made more convenient by language support for bean properties. Anonymous extension methods (borrowed from Groovy, I believe) make domain specific embedded languages easier to write. Method chaining builder APIs are unnecessary in Kotlin and more awkward than using the apply function to set properties of an object. Kotlin has a philosophy of preferring behaviour to be explicitly specified. For example, there is no automatic coercion between numeric types, even from low to high precision. That means you have to explicitly convert from Int to Long, for example. I like that, but I expect some people will find it annoying. Surprises I didn’t miss destructuring pattern match Kotlin doesn’t have destructuring pattern match & language support for tuples. A limited form of destructuring can be used in for loops and assignments. The when expression is an alternative conditional expression that doesn’t do destructuring. I was surprised to find that I didn’t really miss destructuring. Without tuples you have to use data classes with named fields. The flow-sensitive typing then works rather well where destructuring would be necessary in a language where the type system is not flow-sensitive. For example: sealed class Result { class Success(val value: T) : Result() class Failure(val exception: Throwable): Result() } ... val e : Result = doSomething() if (e is Result.Success) { // e is known to be a Result.Success and // can be used as such without a downcast println(e.value) } Sealed classes cannot be data classes An algebraic data type cannot be be a data class, so you have to write equals, hashCode, etc. yourself. This is a surprising omission, since I always want an algebraic data type to be a value type. The Kotlin developers say that this will be fixed after version 1.0 is released. Functions are not quite first-class objects There’s a difference between f1 and f2 below: val f1 = {i:Int -> i+1} fun f2(i:Int) = i + 1 Code can refer to the value of f1 directly, but must use the “::” operator to obtain a reference to f2: val f = ::f2 Null safe operators push me to write more extension functions The null safe operators (?. and ?:) only help when dereferencing a nullable reference. When you have a nullable reference and want to pass a value (if it exists) to a function as a parameter, you have to use the awkward construct: nullableThing?.let{ thing -> fn(thing, other, parameters) } I find myself refactoring my functions to extension functions to reduce the syntactic noise, letting me rewrite the code above as: nullableThing?.fn(other, parameters) Is that a good thing? I’m not sure. Sometimes it feels a little forced. Companion objects Kotlin borrows the concept of companion objects from Scala. Why can’t classes be objects? Perhaps it’s a limitation of the JVM but compared to, say, Python, it feels clunky. Frustrations I only have a couple of frustrations with Kotlin. Lack of polymorphism Operators and the for loop are syntactic sugar that desugar to method calls. The methods are invoked statically and structurally — the target does not need to implement an interface that defines the method. For example, a + b desugars to a.add(b). However, there’s no way to write a generic function that sums its parameters, because add is not defined on an interface that can be used as an upper bound. There’s no way to define the following function for any type that has an add operator method. fun sum(T first, T second) : T = first + second You would have to define overloads for different types, but then wouldn’t be able to call sum within a generic function. fun sum(first: Int, second: Int) = first + second fun sum(first: Long, second: Long) = first + second fun sum(first: Double, second: Double)= first + second fun sub(first: Matrix, second: Matrix)= first + second fun sum(first: Money, second: Money)= first + second ... That kind of duplication is what generics are meant to avoid! Extension methods are also statically bound. That means you cannot write a generic function that calls an extension method on the value of a type parameter. For example, the standard library defines the same extension methods on unrelated types — forEach, map, flatMap, fold, etc. But there is no concept of, for example, “mappable” or “foldable”. Nor is there a way of defining such a concept and applying it to existing types to allow you to write generic functions over unrelated types. Kotlin doesn’t have higher kinded types that would let you express this kind of generic function or type classes that would let you add polymorphism without modifying existing type definitions. Compared to Rust’s traits, which support extension methods, operator overloading, type classes for parametric polymorphism, and interfaces for subtype polymorphism, Kotlin’s monomorphic extension methods and operator overloading are quite limited and do not help refactor duplicated logic. However, for typical monomorphic, procedural Java code this is probably not an issue. Optionality is a Special Case Kotlin’s support for nullable types is implemented by a special case in the type system and special case operators that only apply to nullable references. The operators do not desugar to methods that can be implemented for other types. For example, optionalThing?.foo can be considered a map of the function {thing -> thing.foo} over the option optionalThing. If foo itself is nullable, then it can be considered a flatMap. But the expressions do not desugar to map and flatMap And if you want to map or flatMap a function, you have to use a different syntax: optionalThing?.let(theFunction). For typical Java code, which is monomorphic and uses null references with wild abandon, language support for nullability is invaluable. But I would find it more convenient if it could be used polymorphically, or if Kotlin used a common naming convention for optional and other functor types. Null safety is not enforced when you interop with Java code. And you do that a lot. Kotlin doesn’t have many libraries or much of a runtime and makes it easy to call existing Java libraries directly. Kotlin expects you to know what you’re doing with respect to null references when calling Java code, and doesn’t force you to treat every value returned from Java as nullable. As a result, null safety is a bit of an illusion in a lot of our Kotlin code and it has come back to bite us. Conclusion The code we’ve been writing has been a mix of coordinating and piping data between existing Java libraries – Apache HTTP client, Undertow HTTP server, JDBC, Sesame, JSON and XML parsers, and so on – and algorithmic code that analyses human readable text. For that kind of work, Kotlin has been very useful. The Kotlin code is far more concise than the equivalent Java. In our domain models and algorithmic code, Kotlin’s type safety and especially null safety, has been a big help. Exactly how happy we’ve been with Kotlin has depended on the design style of the code we’re writing: For functional programming, Kotlin has occasionally been frustrating, because we cannot use parametric polymorphism to factor out duplicated logic as much as we’d like. For object-oriented programming, Kotlin’s concise syntax for class definitions and language support for delegation avoids a lot of Java’s boilerplate. However, most Java out there is monomorphic, procedural code moving data between “NOJOs” and APIs that expect objects to have “bean” getters and setters. Kotlin has made working with that kind of API much easier and far more concise than doing so in Java. As far as I can tell, RedHat sponsor development of Ceylon but do not actually use it to develop their own products. If I’m wrong, please let me know in the comments.↩
Categories: Programming, Testing & QA

Thought after Test Automation Day 2013

Henry Ford said “Obstacles are those frightful things you see when take your eyes off the goal.” After I’ve been to Test Automation Day last month I’m figuring out why industrializing testing doesn’t work. I try to put it in this negative perspective, because I think it works! But also when is it successful? A lot of times the remark from Ford is indeed the problem. People tend to see obstacles. Obstacles because of the thought that it’s not feasible to change something. They need to change. But that’s not an easy change.

After attending the #TAD2013 as it was on Twitter I saw a huge interest in better testing, faster testing, even cheaper testing by using tools to industrialize. Test automation has long been seen as an interesting option that can enable faster testing. it wasn’t always cheaper, especially the first time, but at least faster. As I see it it’ll enable better testing. “Better?” you may ask. Test automation itself doesn’t enable better testing, but by automating regression tests and simple work the tester can focus on other areas of the quality.

images

And isn’t that the goal? In the end all people involved in a project want to deliver a high quality product, not full of bugs. But they also tend to see the obstacles. I see them less and less. New tools are so well advanced and automation testers are becoming smarter and smarter that they enable us to look beyond the obstacles. I would like to say look over the obstacles.

At the Test Automation Day I learned some new things, but it also proved something I already know; test automation is here to stay. We don’t need to focus on the obstacles, but should focus on the goal.

Categories: Testing & QA

The Toyota Way: The need for doing it right the first time

After WWII Toyota started developing its Toyota Production System (TPS); which was identified as ‘Lean’ in the 1990s. Taiichi Ohno, Shigeo Shingo and Eiji Toyoda developed the system between 1948 and 1975. In the myth surrounding the system it was not inspired by the American automotive industry, but from a visit to American supermarkets, Ohno saw the supermarket as model for what he was trying to accomplish in the factor and perfect the Just-in-Time (JIT) production system. While accomplishing this low inventory levels were a key outcome of the TPS, and an important element of the philosophy behind its system is to work intelligently and eliminate waste so that only minimal inventory is needed.

As TPS and Lean have their own principles as outlined by Toyota:

  • Long-term Philosophy
  • Right process will produce the right results
  • Value to organization by developing people
  • Solving root problems drives organizational learning

As these principles were summed up and published by Toyota in 2001, by naming it “The Toyota Way 2001”. It consists the above named principles in two key areas: Continuous Improvement, and Respect for People.

the-toyota-way

The principles for a continuous improvement include establishing a long-term vision, working on challenges, continual innovation, and going to the source of the issue or problem. The principles relating to respect for people include ways of building respect and teamwork. When looking at the ALM all these principles come together in the ‘first time right’ approach already mentioned. And from Toyota’s view they were outlined as followed:

  • The right process will produce the right results
    • Create continuous process flow to bring problems to the surface.
    • Use the ‘pull’ system to avoid overproduction (kanban)
    • Level out the workload (heijunka).
    • Build a culture of stopping to fix problems, to get quality right from the first (jidoka)
  • Continuously solving root problems drives organizational learning
    • Go and see for yourself to thoroughly understand the situation (Genchi Genbutsu);
    • Make decisions slowly by consensus, thoroughly considering all options (nemawashi); implement decisions rapidly;
    • Become a learning organization through relentless reflection (hansei) and continuous improvement (kaizen).
Let’s do it right now!

As the economy is changing and IT is more common sense throughout ore everyday life the need for good quality software products has never been this high. Software issues create bigger and bigger issues in our lives. Think about trains that cannot ride due to software issues, bank clients that have no access to their bank accounts, and people oversleeping because their alarm app didn’t work on their iPhone. As Capers Jones [Jones, 2011] states in his 2011 study that “software is blamed for more major business problems than any other man-made product” and that “poor quality has become one of the most expensive topics in human history”. The improvement of software quality is a key topic for all industries.

 Right the first time vs jidoka

In both TPS and Lean autonomation or jidoka are used. Autonomation can be described as ‘intelligent autonomation’, it means that when an abnormal situation arises the ‘machine’ stops and fix the abnormality. Autonomation prevents the production of defective products, eliminates overproduction, and focuses attention on understanding the problem and ensuring that it never recurs; a quality control process that applies the following four principles:

  • Detect the abnormality.
  • Stop.
  • Fix or correct the immediate condition.
  • Investigate the root cause and install a countermeasure.
Find defects as early as possible

In other words autonomation helps to get quality right the first time perfectly. With IT projects being different from the Toyota car production line, ‘perfectly’ may be a bit too much, but the process around quality assurance should be the same:

  • Find the defect.
  • Stop.
  • Fix or correct the error.
  • Investigate the root cause and take countermeasures.

The defect should be found as early as possible to be fixed as early as possible. And as with Lean and TPS the reason behind this is to make it possible to address the identification and correction of defects immediately in the process.

Categories: Testing & QA