Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator/categories/2&page=1' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Testing & QA
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

The Silver Bullet Syndrome

From the Editor of Methods & Tools - Thu, 02/25/2016 - 16:48
We love our silver bullets don’t we? We are constantly chasing the dream that the next big thing will solve all our past problems. It doesn’t matter if it’s a language, framework, platform or library, we’re out there chasing it. Why? Well because it’s going to solve our needs, it’s going to solve the business […]

Longevity of Source Code

Actively Lazy - Wed, 02/24/2016 - 22:10

Take a look at the code you work in day-to-day. How long has it been there? How old is it? Six months old? A year? Maybe five years old? Ten? Twenty?! How much of the code is old? Less than 10%? Half? Or as much as 90%? Curious to know the answers to these questions I’ve been investigating how long code sticks around.

Software archeology

Work for any company that’s been around for more than a couple of years and there will be source code that’s been there for a while. Writing software in an environment like this is often an exercise in software archeology – digging down into the application is like digging down into an old city, slowly uncovering the past layer by layer.

Once you get past the shiny new containerized micro-services, you start delving into the recent past: perhaps the remnants of the company’s first foray into a Service Oriented Architecture; now a collection of monolithic services with a complex tangle of business logic, all tied together with so much Spring. Dig deeper still and we get back to the EJB era; some long-forgotten beans still clinging on to life, for lack of any developer’s appetite to understand them again. Down here is where the skeletons are.

If it ain’t broke, don’t fix it

What’s so bad about old code? It’s fulfilling some important purpose, no doubt. At least, some of it probably is.

If you look at code you wrote a year ago and can’t see anything to change, you haven’t learnt a thing in the last year

We’re always learning more: a better understanding of the domain, a better understanding of how our solution models the domain, new architectural styles, new tools, new approaches, new standards and new ideas. It is inevitable the code you wrote a year ago could be improved somehow. But how much of it have you gone back and improved recently?

The trouble with old code is that it gets increasingly hard to change. What would happen if a change in business requirements led you all the way down to the Roman-era sewers that are the EJBs? Would you implement the change the way it would have been done a decade ago? Or would you spend time trying to extract the parts that need to change? Perhaps building another shiny, new containerized micro-service along the way? That change isn’t going to be cheap though.

And this is the problem: paying back this “technical debt” is the right thing to do, but it will be slower to change this ancient code than the stuff you wrote last week or last month. The more ancient code you have the slower it will be to make changes, the slower you can develop new features. The worst part of maintaining a long running code base isn’t just paying back the debt from the things we know we did wrong; it’s the debt from things that were done right (at the time), but only now seem wrong.

How old is our code?

I’ve been looking through a variety of source code: some commercial, some open source. Across a variety of languages (Java, C#, Ruby). Generally it seems that most code bases follow a pretty similar pattern:

About 70% of the lines of code you wrote today will still be in head, unchanged, in 12 months time

Perhaps unsurprisingly, code changes most often in the first couple of months after being written. After that, it seems to enter a maintenance mode, where code changes relatively rarely.

image

I found this pretty surprising: after a year around 75% of the code I’ve written is still there. Imagine how much better I understand the problem today. Imagine the since-forgotten design ideas, the changed architectural vision, the new tools and libraries it could be refactored to use today. Imagine how much better every single one of those lines could be. And yet, even in a code base where we’re constantly working to pay back technical debt, we’re barely making a dent in how old the code is.

The how

How did I do this analysis? Thanks to the magic power of git, it is surprisingly easy. I can use git to do a recursive git blame on the entire repo. This lists, for each line currently in head, the commit that introduced it, who made it and when. With a bit of shell-fu we can extract a count by day or month:

git ls-tree -r -rz --name-only HEAD -- | xargs -0 -n1 git blame -f HEAD | sed -e 's/^.* \([0-9]\{4\}-[0-9]\{2\}\)-[0-9]\{2\} .*$/\1/g' | sort | uniq -c

This outputs a nice table of lines of code last touched by month, as of today. But, I can just as easily go back in history, e.g. to go back to the start of 2015:

git checkout `git rev-list -n 1 --before="2015-01-01 00:00:00" master`

I can then re-run the recursive git blame. By comparing the count of lines last checked in each month I can see how much of the code written before 2015 still exists today. With more detailed analysis I can see how the number of lines last touched in a given month changes over time to see how quickly (or not!) code decays.

Conclusion

Code is written to serve some purpose, to deliver some business value. But it quickly becomes a liability. As this code ages and rots it becomes harder and harder to change. From the analysis above it is clear to understand why a code base that has been around for a decade is mostly all archaic: with so little of the old code being changed each year it just continues to hang around, all the while we’re piling up new legacy on top – most of which will still be here next year.

What are we to do about it? It seems to be a natural law of software that over time it only accumulates. From my own experience it seems that even concerted efforts on refactoring make little impact. What are we to do? Do we just accept that changes become progressively harder as source code ages? Or do we need to find a way to encourage software to have a shorter half-life, to be re-written sooner.


Categories: Programming, Testing & QA

Longevity of Source Code

Actively Lazy - Wed, 02/24/2016 - 22:10

Take a look at the code you work in day-to-day. How long has it been there? How old is it? Six months old? A year? Maybe five years old? Ten? Twenty?! How much of the code is old? Less than 10%? Half? Or as much as 90%? Curious to know the answers to these questions I’ve been investigating how long code sticks around.

Software archeology

Work for any company that’s been around for more than a couple of years and there will be source code that’s been there for a while. Writing software in an environment like this is often an exercise in software archeology – digging down into the application is like digging down into an old city, slowly uncovering the past layer by layer.

Once you get past the shiny new containerized micro-services, you start delving into the recent past: perhaps the remnants of the company’s first foray into a Service Oriented Architecture; now a collection of monolithic services with a complex tangle of business logic, all tied together with so much Spring. Dig deeper still and we get back to the EJB era; some long-forgotten beans still clinging on to life, for lack of any developer’s appetite to understand them again. Down here is where the skeletons are.

If it ain’t broke, don’t fix it

What’s so bad about old code? It’s fulfilling some important purpose, no doubt. At least, some of it probably is.

If you look at code you wrote a year ago and can’t see anything to change, you haven’t learnt a thing in the last year

We’re always learning more: a better understanding of the domain, a better understanding of how our solution models the domain, new architectural styles, new tools, new approaches, new standards and new ideas. It is inevitable the code you wrote a year ago could be improved somehow. But how much of it have you gone back and improved recently?

The trouble with old code is that it gets increasingly hard to change. What would happen if a change in business requirements led you all the way down to the Roman-era sewers that are the EJBs? Would you implement the change the way it would have been done a decade ago? Or would you spend time trying to extract the parts that need to change? Perhaps building another shiny, new containerized micro-service along the way? That change isn’t going to be cheap though.

And this is the problem: paying back this “technical debt” is the right thing to do, but it will be slower to change this ancient code than the stuff you wrote last week or last month. The more ancient code you have the slower it will be to make changes, the slower you can develop new features. The worst part of maintaining a long running code base isn’t just paying back the debt from the things we know we did wrong; it’s the debt from things that were done right (at the time), but only now seem wrong.

How old is our code?

I’ve been looking through a variety of source code: some commercial, some open source. Across a variety of languages (Java, C#, Ruby). Generally it seems that most code bases follow a pretty similar pattern:

About 70% of the lines of code you wrote today will still be in head, unchanged, in 12 months time

Perhaps unsurprisingly, code changes most often in the first couple of months after being written. After that, it seems to enter a maintenance mode, where code changes relatively rarely.

image

I found this pretty surprising: after a year around 75% of the code I’ve written is still there. Imagine how much better I understand the problem today. Imagine the since-forgotten design ideas, the changed architectural vision, the new tools and libraries it could be refactored to use today. Imagine how much better every single one of those lines could be. And yet, even in a code base where we’re constantly working to pay back technical debt, we’re barely making a dent in how old the code is.

The how

How did I do this analysis? Thanks to the magic power of git, it is surprisingly easy. I can use git to do a recursive git blame on the entire repo. This lists, for each line currently in head, the commit that introduced it, who made it and when. With a bit of shell-fu we can extract a count by day or month:

git ls-tree -r -rz --name-only HEAD -- | xargs -0 -n1 git blame -f HEAD | sed -e 's/^.* \([0-9]\{4\}-[0-9]\{2\}\)-[0-9]\{2\} .*$/\1/g' | sort | uniq -c

This outputs a nice table of lines of code last touched by month, as of today. But, I can just as easily go back in history, e.g. to go back to the start of 2015:

git checkout `git rev-list -n 1 --before="2015-01-01 00:00:00" master`

I can then re-run the recursive git blame. By comparing the count of lines last checked in each month I can see how much of the code written before 2015 still exists today. With more detailed analysis I can see how the number of lines last touched in a given month changes over time to see how quickly (or not!) code decays.

Conclusion

Code is written to serve some purpose, to deliver some business value. But it quickly becomes a liability. As this code ages and rots it becomes harder and harder to change. From the analysis above it is clear to understand why a code base that has been around for a decade is mostly all archaic: with so little of the old code being changed each year it just continues to hang around, all the while we’re piling up new legacy on top – most of which will still be here next year.

What are we to do about it? It seems to be a natural law of software that over time it only accumulates. From my own experience it seems that even concerted efforts on refactoring make little impact. What are we to do? Do we just accept that changes become progressively harder as source code ages? Or do we need to find a way to encourage software to have a shorter half-life, to be re-written sooner.


Categories: Programming, Testing & QA

Software Development Conferences Forecast February 2016

From the Editor of Methods & Tools - Tue, 02/23/2016 - 15:09
Here is a list of software development related conferences and events on Agile project management ( Scrum, Lean, Kanban), software testing and software quality, software architecture, programming (Java, .NET, JavaScript, Ruby, Python, PHP), DevOps and databases (NoSQL, MySQL, etc.) that will take place in the coming weeks and that have media partnerships with the Methods […]

The New Testing Conference

James Bach’s Blog - Mon, 02/22/2016 - 01:34

Anna Royzman is starting a new testing conference which she has decided to call the New Testing Conference. Anna has asked me to tell you about some of the ways we will be trying to live up to that bold name.

(Disclosure: although it doesn’t belong to me and I am not in charge of anything, bear in mind that I have an economic interest in it. I’m being paid to present at it and I’m advising Anna on the design of the program and invitation of speakers. I will be doing a tutorial and probably other things, too.)

Position Talks as Gentle Debate

We were talking about what it means to have a test conference that focuses on “newness” and one of the things we realized is that new is always controversial. New is always sketchy. For any new practice, there will be lots of practitioners who roll their eyes about it or scowl darkly. Therefore, if we want to talk about new things, we have to deal with the clash between “tried and comfortable” and “new and nervous.” So, this conference must help good ideas move beyond the new and nervous stage, while letting the not so good ideas fall back into obscurity (at least for a while… until the next generation unearths it again like a cursed monkey paw).

A structure we want to try is this:

  1. Hold two or more short position talks around a particular topic. For instance “Is BDD worth doing or a vain waste of time?”
  2. The speakers discuss and debate the topic BEFORE the conference. That way, at the conference, they would be able to put their best ideas forward and avoid misrepresenting the other side’s argument.
  3. They each speak for 10-15 minutes explaining their arguments.
  4. There is a 20-minute break for the audience, during which they may speak with the speakers to give them ideas or continue the debate. The speaker don’t get a break.
  5. The speakers give 5-minute follow-up lightning talks to respond to the other speakers or amend their previous statements.
Each Track Talk Includes a Demonstration, Exercise, or Experience Report

We feel that just talking about concepts isn’t enough. So, each track talk will include at least one of the following:

  • Demonstration: Show how an idea works in practice, then take questions and comments from the audience.
  • Exercise: Get the audience to try something, then debrief.
  • Experience Report: Tell a specific story about something that you experienced at a particular time and place, then take questions and comments.
“360 degree” Tutorials

I’m not sure if that’s quite the right name, but we want to do some workshops based on a particular structure I’ve been experimenting with:

  1. The instructor offers a challenge (which he has previously performed and has results ready to share).
  2. The students perform the challenge.
  3. Debrief with instructor commentary and critique.
  4. The instructor shows what he did and challenges the students to critique it.
  5. Students critique instructors work.
  6. Instructor critiques his own work.

Part of the fun of a “360” kind of workshop, as an instructor, is that I try to anticipate every criticism that the students will make of my work. I usually miss something, but then I add it to my list of critiques and I am even more prepared for the next time I run the workshop. I end up looking smarter and smarter– but of course the punchline is: I’m smarter because I opened myself to all this criticism. And when we all get comfortable hearing critical reactions to our work, our whole community grows smarter, faster.

Categories: Testing & QA

Software Development Linkopedia February 2016

From the Editor of Methods & Tools - Wed, 02/17/2016 - 14:36
Here is our monthly selection of knowledge on programming, software testing and project management. This month you will find some interesting information and opinions about web development, limiting software testing, story mapping, software architecture, changing organizations, UX testing, technical debt, good code and bad tests. Blog: The Sad State of Web Development Blog: Just Say […]

EarlGrey - iOS Functional UI Testing Framework

Google Testing Blog - Wed, 02/17/2016 - 00:14
By Siddartha Janga on behalf of Google iOS Developers 

Brewing for quite some time, we are excited to announce EarlGrey, a functional UI testing framework for iOS. Several Google apps like YouTube, Google Calendar, Google Photos, Google Translate, Google Play Music and many more have successfully adopted the framework for their functional testing needs.

The key features offered by EarlGrey include:

  • Powerful built-in synchronization : Tests will automatically wait for events such as animations, network requests, etc. before interacting with the UI. This will result in tests that are easier to write (no sleeps or waits) and simple to maintain (straight up procedural description of test steps). 
  • Visibility checking : All interactions occur on elements that users can see. For example, attempting to tap a button that is behind an image will lead to test failure immediately. 
  • Flexible design : The components that determine element selection, interaction, assertion and synchronization have been designed to be extensible. 


Are you in need for a cup of refreshing EarlGrey? EarlGrey has been open sourced under the Apache license. Check out the getting started guide and add EarlGrey to your project using CocoaPods or manually add it to your Xcode project file.

Categories: Testing & QA

Quote of the Month February 2016

From the Editor of Methods & Tools - Mon, 02/15/2016 - 10:47
Ask somebody in the building industry to visually communicate the architecture of a building and you’ll likely be presented with site plans, floor plans, elevation views, cross-section views and detail drawings. In contrast, ask a software developer to communicate the software architecture of a software system using diagrams and you’ll likely get a confused mess […]

Running a static website with Hugo on Google Cloud Storage

Agile Testing - Grig Gheorghiu - Fri, 02/12/2016 - 21:29
I've played a bit with Hugo, the static web site generator written in golang that has been getting a lot of good press lately. At the suggestion of my colleague Warren Runk, I also experimented with hosting the static files generated by Hugo on Google Cloud Storage (GCS). That way there is no need for launching any instances that would serve those files. You can achieve this by using AWS S3 as well of course.

Notes on GCS setup
You first need to sign up for a Google Cloud Platform (GCP) account. You get a 30-day free trial with a new account. Once you are logged into the Google Cloud console, you need to create a new project. Let's call it my-gcs-hugo-project.

You need to also create a bucket in GCS. If you want to serve your site automatically out of this bucket, you need to give the bucket the same name as your site. Let's assume you call the bucket hugotest.mydomain.com. You will have to verify that you own mydomain.com either by creating a special CNAME in the DNS zone file for mydomain.com pointing to google.com, or by adding a special META tag to the HTML file served at hugotest.mydomain.com (you can achieve the latter by temporarily CNAME-ing hugotest to www.mydomain.com and adding the HEAD tag to the home page for www).

If you need to automate deployments to GCS, it's a good idea to create a GCP Service Account. Click on the 'hamburger' menu in the upper left of the GCP console, then go to Permissions, then Service Accounts. Create a new service account and download its private key in JSON format (the key will be called something like my-gcs-hugo-project-a37b5acd7bc5.json.

Let's say your service account is called my-gcp-service-account1. The account will automatically be assigned an email address similar to my-gcp-service-account1@my-gcs-hugo-project.iam.gserviceaccount.com.

I wanted to be able to deploy the static files generated by Hugo to GCS using Jenkins. So I followed these steps on the Jenkins server as the user running the Jenkins process (user jenkins in my case):

1) Installed the Google Cloud SDK



$ wget https://dl.google.com/dl/cloudsdk/channels/rapid/google-cloud-sdk.tar.gz$ tar xvfz google-cloud-sdk.tar.gz$ cd google-cloud-sdk/$ ./install.sh
- source .bashrc
$ which gcloud
/var/lib/jenkins/google-cloud-sdk/bin/gcloud


2) Copied the service account's private key my-gcs-hugo-project-a37b5acd7bc5.json to the .ssh directory of the jenkins user.

3) Activated the service account using the gcloud command-line utility (still as user jenkins)

$ gcloud auth activate-service-account --key-file .ssh/my-gcs-hugo-project-a37b5acd7bc5.jsonActivated service account credentials for: [my-gcp-service-account1@my-gcs-hugo-project.iam.gserviceaccount.com]
4) Set the current GCP project to my-gcs-hugo-project


$ gcloud config set project my-gcs-hugo-project
$ gcloud config listYour active configuration is: [default]
[core]account = my-gcp-service-account1@my-gcs-hugo-project.iam.gserviceaccount.comdisable_usage_reporting = True
project = my-gcs-hugo-project
5) Configured GCS via the gsutil command-line utility (this may actually be redundant since we already configured the project with gcloud, but I leave it here in case you encounter issues with using just gcloud)

$ gsutil config -eIt looks like you are trying to run "/var/lib/jenkins/google-cloud-sdk/bin/bootstrapping/gsutil.py config".The "config" command is no longer needed with the Cloud SDK.To authenticate, run: gcloud auth loginReally run this command? (y/N) yBacking up existing config file "/var/lib/jenkins/.boto" to "/var/lib/jenkins/.boto.bak"...This command will create a boto config file at /var/lib/jenkins/.botocontaining your credentials, based on your responses to the followingquestions.What is the full path to your private key file? /var/lib/jenkins/.ssh/my-gcs-hugo-project-a37b5acd7bc5.json
Please navigate your browser to https://cloud.google.com/console#/project,then find the project you will use, and copy the Project ID string from thesecond column. Older projects do not have Project ID strings. For such projects,click the project and then copy the Project Number listed under that project.
What is your project-id? my-gcs-hugo-project
Boto config file "/var/lib/jenkins/.boto" created. If you need to usea proxy to access the Internet please see the instructions in that
file.
6) Added the service account created above as an Owner for the bucket hugotest.mydomain.com

7) Copied a test file from the local file system of the Jenkins server to the bucket hugotest.mydomain.com  (still logged in as user jenkins), then listed all files in the bucket, then removed the test file

$ gsutil cp test.go gs://hugotest.mydomain.com/Copying file://test.go [Content-Type=application/octet-stream]...Uploading   gs://hugotest.mydomain.com/test.go:             951 B/951 B
$ gsutil ls gs://hugotest.mydomain.com/gs://hugotest.mydomain.com/test.go
$ gsutil rm gs://hugotest.mydomain.com/test.goRemoving gs://hugotest.mydomain.com/test.go...
8) Created a Jenkins job for uploading all static files for a given website to GCS
Assuming all these static files are checked in to GitHub, the Jenkins job will first check them out, then do something like this (where TARGET is the value selected from a Jenkins multiple-choice dropdown for this job):
BUCKETNAME=$TARGET
# upload all filee and disable caching (for testing purposes)gsutil -h "Cache-Control:private" cp -r * gs://$BUCKETNAME/
# set read permissions for allUsersfor file in `find . -type f`; do    # remove first dot from file name    file=${file#"."}    gsutil acl ch -u allUsers:R gs://${BUCKETNAME}${file}done
The first gsutil command does a recursive copy (cp -r *) of all files to the bucket. This will preserve the directory structure of the website. For testing purposes, the gsutil command also sets the Cache-Control header on all files to private, which tells browsers not to cache the files.
The second gsutil command is executed for each object in the bucket, and it sets the ACL on that object so that the object has Read (R) permissions for allUsers (by default only owners and other specifically assigned users have Read permissions). This is because we want to serve a public website out of our GCS bucket.
At this point, you should be able to hit hugotest.mydomain.com in a browser and see your static site in all its glory.
Notes on Hugo setup
I've only dabbled in Hugo in the last couple of weeks, so these are very introductory-type notes.
Installing Hugo on OSX and creating a new Hugo site
$ brew update && brew install hugo$ mkdir hugo-sites$ cd hugo-sites$ hugo new site hugotest.mydomain.com$ git clone --recursive https://github.com/spf13/hugoThemes themes$ cd hugotest.mydomain.com$ ln -s ../themes .
At this point you have a skeleton directory structure created by Hugo (via the hugo new site command) under the directory hugotest.mydomain.com:
$ lsarchetypes  config.toml content     data        layouts     static themes
(note that we symlinked the themes directory into the hugotest.mydomain.com directory to avoid duplication)

Configuring your Hugo site and choosing a theme
One file you will need to pay a lot of attention to is the site configuration file config.toml. The default content of this file is deceptively simple:
$ cat config.tomlbaseurl = "http://replace-this-with-your-hugo-site.com/"languageCode = "en-us"title = "My New Hugo Site"
Before you do anything more, you need to decide on a theme for your site. Browse the Hugo Themes page and find something you like. Let's assume you choose the Casper theme. You will need to become familiar with the customizations that the theme offers. Here are some customizations I made in config.toml, going by the examples on the Casper theme web page:
$ cat config.tomlbaseurl = "http://hugotest.mydomain.com/"languageCode = "en-us"title = "My Speedy Test Site"newContentEditor = "vim"
theme = "casper"canonifyurls = true
[params]  description = "Serving static sites at the speed of light"  cover = "images/header.jpg"  logo = "images/mylogo.png"  # set true if you are not proud of using Hugo (true will hide the footer note "Proudly published with HUGO.....")  hideHUGOSupport = false
#  author = "ValĂšre JEANTET"#  authorlocation = "Paris, France"#  authorwebsite = "http://vjeantet.fr"#  bio= "my bio"#  googleAnalyticsUserID = "UA-79101-12"#  # Optional RSS-Link, if not provided it defaults to the standard index.xml#  RSSLink = "http://feeds.feedburner.com/..."#  githubName = "vjeantet"#  twitterName = "vjeantet"  # facebookName = ""  # linkedinName = ""
I left most of the Casper-specific options commented out and only specified a cover image, a logo and a description. 
Creating a new page
If you want blog-style posts to appear on your home page, create a new page with Hugo under a directory called post (some themes want this directory to be named post and others want it posts, so check what the theme expects). 
Let's assume you want to create a page caled hello-world.md (I haven't even mentioned this so far, but Hugo deals by default with Markdown pages, so you will need to brush up a bit on our Markdown skills). You would run:
$ hugo new post/hello-world.md
This creates the post directory under the content directory, creates a file called hello-world.md in content/post, and opens up the file for editing in the editor you specified as the value for newContentEditor in config.toml (vim in my case). The default contents of the md file are specific to the theme you used. For Casper, here is what I get by default:
+++author = ""comments = truedate = "2016-02-12T11:54:32-08:00"draft = falseimage = ""menu = ""share = trueslug = "post-title"tags = ["tag1", "tag2"]title = "hello world"
+++

Now add some content to that file and save it. Note that the draft property is set to false by the Casper theme. Other themes set it to true, in which case it would not be published by Hugo by default. The slug property is set by Casper to "post-title" by default. I changed it to "hello-world". I also changed the tags list to only contain one tag I called "blog".
At this point, you can run the hugo command by itself, and it will take the files it finds under content, static, and its other subdirectories, turn them into html/js/css/font files and save it in a directory called public:
$ hugo0 draft content0 future content1 pages created3 paginator pages created1 tags created0 categories createdin 55 ms
$ find publicpublicpublic/404.htmlpublic/csspublic/css/nav.csspublic/css/screen.csspublic/fontspublic/fonts/example.htmlpublic/fonts/genericons.csspublic/fonts/Genericons.eotpublic/fonts/Genericons.svgpublic/fonts/Genericons.ttfpublic/fonts/Genericons.woffpublic/index.htmlpublic/index.xmlpublic/jspublic/js/index.jspublic/js/jquery.fitvids.jspublic/js/jquery.jspublic/pagepublic/page/1public/page/1/index.htmlpublic/postpublic/post/hello-worldpublic/post/hello-world/index.htmlpublic/post/index.htmlpublic/post/index.xmlpublic/post/pagepublic/post/page/1public/post/page/1/index.htmlpublic/sitemap.xmlpublic/tagspublic/tags/blogpublic/tags/blog/index.htmlpublic/tags/blog/index.xmlpublic/tags/blog/pagepublic/tags/blog/page/1public/tags/blog/page/1/index.html
That's quite a number of files and directories created by hugo. Most of it is boilerplate coming from the theme. Our hello-world.md file was turned into a directory called hello-world under public/post, with an index.html file dropped in it. Note that the Casper theme names the hello-world directory after the slug property in the hello-world.md file.
Serving the site locally with Hugo
Hugo makes it very easy to check your site locally. Just run

$ hugo server
0 draft content
0 future content
1 pages created
3 paginator pages created
1 tags created
0 categories created
in 35 ms
Watching for changes in /Users/grig.gheorghiu/mycode/hugo-sites/hugotest.mydomain.com/{data,content,layouts,static,themes}
Serving pages from memory
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop


Now if you browse to http://localhost:1313 you should see something similar to this:


Not bad for a few minutes of work.

For other types of content, such as static pages not displayed on the home page, you can create Markdown files in a pages directory:

$ hugo new pages/static1.md
+++
author = ""
comments = true
date = "2016-02-12T12:24:26-08:00"
draft = false
image = ""
menu = "main"
share = true
slug = "static1"
tags = ["tag1", "tag2"]
title = "static1"

+++

Static page 1.

Note that the menu property value is "main" in this case. This tells the Casper theme to create a link to this page in the main drop-down menu available on the home page.

If you run hugo server again, you should see something the menu available in the upper right corner, and a link to static1 when you click on the menu:




To deploy your site to GCS, S3 or regular servers, you need to upload the files and directories under the public directory. It's that simple.

I'll stop here with my Hugo notes. DigitalOcean has a great tutorial on installing and running Hugo on Ubuntu 14.04.



Some notes on Ansible playbooks and roles

Agile Testing - Grig Gheorghiu - Thu, 02/11/2016 - 23:09
Some quick notes I jotted down while documenting our Ansible setup. Maybe they will be helpful for people new to Ansible.
Ansible playbooks and roles
Playbooks are YAML files that specify which roles are applied to hosts of certain type.
Example: api-servers.yml
$ cat api-servers.yml---
- hosts: api  sudo: yes  roles:    - base    - tuning    - postfix    - monitoring    - nginx    - api    - logstash-forwarder
This says that for each host in the api group we will run tasks defined in the roles listed above.
Example of a role: the base role is one that (in our case) is applied to all hosts. Here is its directory/file structure:
roles/baseroles/base/defaultsroles/base/defaults/main.ymlroles/base/filesroles/base/files/newrelicroles/base/files/newrelic/newrelic-sysmond_2.0.2.111_amd64.debroles/base/files/pubkeysroles/base/files/pubkeys/id_rsa.pub.jenkinsroles/base/files/rsyslogroles/base/files/rsyslog/50-default.confroles/base/files/rsyslog/60-papertrail.confroles/base/files/rsyslog/papertrail-bundle.pemroles/base/files/sudoers.droles/base/files/sudoers.d/10-admin-usersroles/base/handlersroles/base/handlers/main.ymlroles/base/metaroles/base/meta/main.ymlroles/base/README.mdroles/base/tasksroles/base/tasks/install.ymlroles/base/tasks/main.ymlroles/base/tasks/newrelic.ymlroles/base/tasks/papertrail.ymlroles/base/tasks/users.ymlroles/base/templatesroles/base/templates/hostname.j2roles/base/templates/nrsysmond.cfg.j2roles/base/varsroles/base/vars/main.yml
An Ansible role has the following important sub-directories:
defaults - contains the main.yml file which defines default values for variables used throughout other role files; note that the role’s files are checked in to GitHub, so these values shouldn’t contain secrets such as passwords, API keys etc. For those types of variables, use group_vars or host_vars files which will be discussed below.
files - contains static files that are copied over by ansible tasks to remote hosts
handlers - contains the main.yml file which defines actions such as stopping/starting/restarting services such as nginx, rsyslog etc.
meta - metadata about the role; things like author, description etc.
tasks - the meat and potatoes of ansible, contains one or more files that specify the actions to be taken on the host that is being configured; the main.yml file contains all the other files that get executed
Here are 2 examples of task files, one for configuring rsyslog to send logs to Papertrail and the other for installing the newrelic agent:
$ cat tasks/papertrail.yml- name: copy papertrail pem certificate file to /etc  copy: >    src=rsyslog/{{item}}    dest=/etc/{{item}}  with_items:    - papertrail-bundle.pem
- name: copy rsyslog config files for papertrail integration  copy: >    src=rsyslog/{{item}}    dest=/etc/rsyslog.d/{{item}}  with_items:    - 50-default.conf    - 60-papertrail.conf  notify:     - restart rsyslog
$ cat tasks/newrelic.yml- name: copy newrelic debian package  copy: >    src=newrelic/{{newrelic_deb_pkg}}    dest=/opt/{{newrelic_deb_pkg}}
- name: install newrelic debian package  apt: deb=/opt/{{newrelic_deb_pkg}}
- name: configure newrelic with proper license key  template: >    src=nrsysmond.cfg.j2    dest=/etc/newrelic/nrsysmond.cfg    owner=newrelic    group=newrelic    mode=0640  notify:     - restart newrelic
templates - contains Jinja2 templates with variables that get their values from defaults/main.yml or from group_vars or host_vars files. One special variable that we use (and is not defined in these files, but instead is predefined by Ansible) is inventory_hostname which points to the hostname of the target being configured. For example, here is the template for a hostname file which will be dropped into /etc/hostname on the target:
$ cat roles/base/templates/hostname.j2{{ inventory_hostname }}
Once you have a playbook and a role, there are a few more files you need to take care of:
  • hosts/myhosts - this is an INI-type file which defines groups of hosts. For example the following snippet of this file defines 2 groups called api and magento.

[api]api01 ansible_ssh_host=api01.mydomain.coapi02 ansible_ssh_host=api02.mydomain.co
[magento]mgto ansible_ssh_host=mgto.mydomain.co
The api-servers.yml playbook file referenced at the beginning of this document sets the hosts variable to the api group, so all Ansible tasks will get run against the hosts included in that group. In the hosts/myhosts file above, these hosts are api01 and api02.
  • group_vars/somegroupname - this is where variables with ‘secret’ values get defined for a specific group called somegroupname. The group_vars directory is not checked into GitHub. somegroupname needs to exactly correspond to the group defined in hosts/myhosts.

Example:
$ cat group_vars/apises_smtp_endpoint: email-smtp.us-west-2.amazonaws.comses_smtp_port: 587ses_smtp_username: some_usernameses_smtp_password: some_passworddatadog_api_key: some_api_key. . . other variables (DB credentials etc)

  • host_vars/somehostname - this is where variables with ‘secret’ values get defined for a specific host called somehostname. The host_vars directory is not checked into GitHub. somehostname needs to exactly correspond to a host defined in hosts/myhosts.

Example:
$ cat host_vars/api02insert_sample_data: false
This overrides the insert_sample_data variable and sets it to false only for the host called api02. This could also be used for differentiating between a DB master and slave for example.
Tying it all together
First you need to have ansible installed on your local machine. I used:
$ pip install ansible
To execute a playbook for a given hosts file against all api server, you would run:
$ ansible-playbook -i hosts/myhosts api-servers.yml
The name that ties together the hosts/myhosts file, the api-servers.yml file and the group_vars/groupname file is in this case api.
You need to make sure you have the desired values for that group in these 3 files:
  • hosts/myhosts: make sure you have the desired hosts under the [api] group
  • api-server.yml: make sure you have the desired roles for hosts in the api group
  • group_vars/api: make sure you have the desired values for variables that will be applied to the hosts in the api group
Launching a new api instance in EC2I blogged about this here.Updating an existing api instance
Make sure the instance hostname is the only hostname in the [api] group in the hosts/myhosts file. Then run:
$ ansible-playbook -i hosts/myhosts api-servers.yml

Old Age Code

Actively Lazy - Wed, 02/10/2016 - 22:34

Is your code ready for retirement? Is it suffering from the diseases of old age? Do you have code you can’t even imagine retiring? It’s just too critical? Too pervasive? Too legacy?

Jon & The Widgets //www.flickr.com/photos/qchristopherConveyor Belt – thanks to https://www.flickr.com/photos/qchristopher

Jon’s first job out of school was in the local widget factory, WidgetCo. Jon was young, enthusiastic and quickly took to the job of making widgets. The company was pleased with Jon and he grew in experience, learning more about making widgets, taking on ever more responsibility; until eventually Jon was responsible for all widget production.

After a couple of years one of WidgetCo’s biggest customers started asking about a new type of square widget. They had only made circular widgets before, but with Jon’s expertise they thought they could take on this new market. The design team got started, working closely with Jon to design a new type of square widget for Jon to produce. It was a great success, WidgetCo were the first to market with a square widget and sales went through the roof.

Unfortunately the new more complex widget production pipeline was putting a lot of pressure on the packing team. With different shape widgets and different options all needing to be sorted and packed properly mistakes were happening and orders being returned. Management needed a solution and turned to Jon. The team realised that if Jon knew a bit more about how orders were going to be packed he could organise the widget production line better to ensure the right number of each shape widget with the right options were being made at the right time. This made the job much easier for the packers and customer satisfaction jumped.

//www.flickr.com/photos/foshydogSneeze – thanks to https://www.flickr.com/photos/foshydog

A few years down the line and Jon was now a key part of the company. He was involved in all stages of widget production from the design of new widgets and the tools to manufacture them through to the production and packaging. But one day Jon got sick. He came down with a virus and was out for a couple of days: the company stopped dead. Suddenly management realised how critical Jon was to their operations – without Jon they were literally stuck. Before Jon was even back up to speed management were already making plans for the future.

Shortly after, the sales team had a lead that needed a new hexagonal widget. Management knew this was a golden opportunity to try and remove some of their reliance on Jon. While Jon was involved in the initial design, the team commissioned a new production line and hired some new, inexperienced staff to run it. Unfortunately hexagonal widgets were vastly more complex than the square ones and, without Jon’s experience the new widget production line struggled. Quality was too variable, mistakes were being made and production was much too slow. The team were confident they could get better over time but management were unhappy. Meanwhile, Jon was still churning out his regular widgets, same as he always had.

But the packing team were in trouble again – with two, uncoordinated production lines at times they were inundated with widgets and at other times they were idle. Reluctantly, management agreed that the only solution was for Jon to take responsibility for coordinating both production lines. Their experiment to remove their reliance on Jon had failed.

//www.flickr.com/photos/neilmoraleeThe Deckhand – thanks to https://www.flickr.com/photos/neilmoralee

A few years later still and the new production line had settled down; it never quite reached the fluidity of the original production line but sales were ok. But there was a new widget company offering a new type of octagonal widget. WidgetCo desperately needed to catch up. The design team worked with Jon but, with his workload coordinating two production lines, he was always busy – so the designers were always waiting on Jon for feedback. The truth was: Jon was getting old. His eyes weren’t what they used to be and the arthritis in his fingers made working the prototypes for the incredibly complex new widgets difficult. Delays piled up and management got ever more unhappy.

So what should management do?

Human vs Machine

When we cast a human in the central role in this story it sounds ridiculous. We can’t imagine a company being so reliant on one frail human being that it’s brought to its knees by an illness. But read the story as though Jon is a software system and suddenly it seems totally reasonable. Or if not reasonable, totally familiar. Throughout the software world we see vast edifices of legacy software, kept alive way past their best because nobody has the appetite to replace them. They become too ingrained, too critical: too big to fail.

14256058429_f7802658c8_zFor all it’s artificial construct, software is not so different from a living organism. A large code base will be of a level of complexity comparable with an organism – too complex for any single human being to comprehend in complete detail. There will be outcomes and responses that can’t be explained completely, that require detailed research to understand the pathway that leads from stimulus to response.

But yet we treat software like it is immortal. As though once written software will carry on working forever. But the reality is that within a few years software becomes less nimble, harder to change. With the growing weight of various changes of direction and focus software becomes slower and more bloated. Each generation piling on the pounds. Somehow no development team has mastered turning back time and turning a creaking, old age project into the glorious flush of youth where everything is possible and nothing takes any time at all.

It’s time to accept that software needs to be allowed to retire. Look around the code you maintain: what daren’t you retire? That’s where you should start. You should start planning to retire it soon, because if it’s bad now it is only getting worse. As the adage goes: the best time to start fixing this was five years ago; the second best time is now.

//www.flickr.com/photos/ewwhiteSad dog – thanks to https://www.flickr.com/photos/ewwhite

After five years all software is a bit creaky; not so quick to change as it once was. After ten years it’s well into legacy; standard approaches have moved on, tools improved, decade old software just feels dated to work with. After twenty years it really should be allowed to retire already.

Software ages badly, so you need to plan for it from the beginning. From day one start thinking about how you’re going to replace this system. A monolith will always be impossible to replace, so constantly think about breaking out separate components that could be retired independently. As soon as code has got bigger than you can throw away, it’s too late. Like a black hole a monolith sucks in functionality, as soon as you’re into run away growth your monolith will consume everything in it’s path. So keep things small and separate.

What about the legacy you have today? Just start. Somewhere. Anywhere. The code isn’t getting any younger.


Categories: Programming, Testing & QA

Setting up a mailinator-like test mail server with postfix and MailHog

Agile Testing - Grig Gheorghiu - Wed, 02/10/2016 - 01:09
The purpose of this exercise is to set up a mailinator-style mail server under our control. If you haven't used mailinator, it's a free service which provides an easy way to test sending email to random recipients. If you send mail to somebody189@mailinator.com and then go to
https://mailinator.com/inbox.jsp?to=somebody189, you will see a mailbox associated with that user, and any incoming email messages destined for that user. It's a handy way to also sign up for services that send confirmation emails.

I have been playing with MailHog, which is a mail server written in Golang for exactly the same purpose as mailinator. In fact, MailHog can happily intercept ANY recipient at ANY mail domain, provided it is set up properly. In my case, I didn't want to expose MailHog on port 25 externally, because that is a recipe for spam. Instead, I wanted to set up a regular postfix server for mydomain.com, then set up a catch-all user which will receive mail destined for anyuser@maildomain.com, and finally send all that mail to MailHog via procmail. Quite a few moving parts, but I got it to work and I am hastening to jot down my notes before I forget how I did it.
The nice thing about MailHog is that it provides a Web UI where you can eyeball the email messages you sent, including in raw format, and it also provides a JSON API which allows you to list messages and search for specific terms within messages. This last feature is very useful for end-to-end testing of your application's email sending capabilities.

I set up everything on a Google Cloud Engine instance running Ubuntu 14.04.
  • instance name: mailhog-mydomain-com
  • DNS/IP: mailhog.mydomain.com / 1.2.3.4
Install go 1.5.3 from source

First install the binaries for go 1.4, then compile go 1.5.
# apt-get update
# apt-get install build-essential git mercurial bazaar unzip
# cd /root
# wget https://storage.googleapis.com/golang/go1.4.3.linux-amd64.tar.gz
# tar xvfz go1.4.3.linux-amd64.tar.gz
# mv go go1.4
# git clone https://go.googlesource.com/go
# cd go
# git checkout go1.5.3
# cd src
# ./all.bash

# mkdir /opt/gocode Edit /root/.bashrc and add:
export GOPATH=/opt/gocode
export PATH=$PATH:/root/go/bin:$GOPATH/bin

then source /root/.bashrc.

# go version
go version go1.5.3 linux/amd64


Set up postfix

Install postfix and mailutils

# apt-get install postfix mailutils

- specified System mail name as mydomain.com

Set up catch-all mail user (local Unix user)

# adduser catchall
Edit /etc/aliases and replace content with the lines below.

# cat /etc/aliases

# See man 5 aliases for format
mailer-daemon: postmaster
postmaster: root
nobody: root
hostmaster: root
usenet: root
news: root
webmaster: root
www: root
ftp: root
abuse: root
root: catchall

Run:
# newaliasesEdit /etc/postfix/main.cf and add lines:

luser_relay = catchall
local_recipient_maps =

Restart postfix:

# service postfix restart

Use Google Cloud Platform Web UI to add firewall rule called allow-smtp for the “default” network associated with the mailhog-mydomain-com instance. The rule allows incoming traffic from everywhere to port tcp:25.
Set up DNS

Add A record for mailhog.mydomain.com pointing to 1.2.3.4.

Add MX record for catchallpayments.com pointing to mailhog.mydomain.com.

Test the incoming mail setup

Send mail to catchall@mydomain.com from gmail.

Run mail utility on GCE instance as user catchall:

catchall@mailhog-mydomain-com:~$ mail

"/var/mail/catchall": 1 message 1 new

>N 1 Some Body     Tue Feb 9 00:23 52/2595 test from gmail

? 1

Return-Path: <some.body@gmail.com>
X-Original-To: catchall@mydomain.com
Delivered-To: catchall@mydomain.com



Send mail to random user which doesn’t exist locally catchall333@mydomain.com and verify that user catchall receives it:

catchall@mailhog-mydomain-com:~$ mail

"/var/mail/catchall": 1 message 1 new

>N 1 Some Body     Tue Feb 9 18:32 52/2702 test 3

? 1

Return-Path: <some.body@gmail.com>
X-Original-To: catchall333@mydomain.com
Delivered-To: catchall333@mydomain.com


Install and configure MailHog

Get MailHog

# go get github.com/mailhog/MailHog - this will drop several binaries in /opt/gocode/bin, including mhsendmail and MailHog (for reference, the code for mhsendmail is here)

# which MailHog
/opt/gocode/bin/MailHog

# which mhsendmail
/opt/gocode/bin/mhsendmail
Configure HTTP basic authentication

MailHog supports HTTP basic authentication via a file similar to .htpasswd. It uses bcrypt for password (see more details here). The MailHog binary can also generate passwords with bcrypt.

I created a password with MailHog:

# MailHog bcrypt somepassword
somebcryptoutput

Then I created a file called .mailhogrc in /root and specified a user called mailhogapi with the password generated above:

# cat /root/.mailhogrc
mailhogapi:somebcryptoutput

Create upstart init file for MailHog

I specified the port MailHog listens on (I chose the same port as its default which is 1025) and the filed used for HTTP basic auth.

# cat /etc/init/mailhog.conf
# MailHog Test SMTP Server (Upstart unit)
description "MailHog Test SMTP Server"
start on (local-filesystems and net-device-up IFACE!=lo)
stop on runlevel [06]

exec /opt/gocode/bin/MailHog -smtp-bind-addr 0.0.0.0:1025 -auth-file /root/.mailhogrc
respawn
respawn limit 10 10
kill timeout 10
See more command line options for MailHog in this doc.

Start mailhog service

# start mailhog
mailhog start/running, process 25458

# ps -efd|grep Mail
root 7782 1 0 22:04 ? 00:00:00 /opt/gocode/bin/MailHog -smtp-bind-addr 0.0.0.0:1025 -auth-file /root/.mailhogrc

At this point MailHog is listening for SMTP messages on port 1025. It also provides a Web UI on default UI port 8025 and a JSON API also on port 8025.

Install procmail and configure it for user catchall

This is so messages addressed to user catchall (which again is our catch-all user) can get processed by a script via procmail.

# apt-get install procmail

Add this line to /etc/postfix/main.cf:

mailbox_command = /usr/bin/procmail -a "$EXTENSION" DEFAULT=$HOME/Maildir/ MAILDIR=$HOME/Maildir

(this will send all messages to procmail instead of individual user mailboxes)

Then su as user catchall and create .procmailrc file in its home directory:

catchall@mailhog-mydomain-com:~$ cat .procmailrc
:0
| /opt/gocode/bin/mhsendmail --smtp-addr="localhost:1025"

This tells procmail to pipe the incoming mail message to mhsendmail, which will format it properly and pass it to port 1025, where MailHog is listening.

Test end-to-end

Use Google Cloud Platform Web UI to add firewall rule called allow-mailhog-ui for the “default” network associated with the mailhog-mydomain-com instance. The rule allows incoming traffic from everywhere to tcp:8025 (where the MailHog UI server listens). It’s OK to allow traffic to port 8025 from everywhere because it is protected via HTTP basic auth.

The MailHog UI is at http://mailhog.mydomain.com:8025

Any email sent to xyz@mydomain.com should appear in the MailHog Inbox.

By default, MailHog stores incoming messages in memory. Restarting MailHog (via ‘restart mailhog’ at the cmdline) will remove all messages.

MailHog also supports MongoDB as a persistent storage backend for incoming messages (exercise left to the reader.)

Use the MailHog JSON API to verify messages

List all messages:

$ curl --user 'mailhogapi:somepassword' -H "Content-Type: application/json" "http://mailhog.mydomain.com:8025/api/v2/messages"

Search messages for specific terms (for example for the recipient’s email):

$ curl -i --user 'mailhogapi:somepassword' -H "Content-Type: application/json" "http://mailhog.mydomain.com:8025/api/v2/search?kind=containing&query=test1%40mydomain.com"

See the MailHog API v2 docs here.

That's it, hope it makes your email sending testing more fun!

Managing Programmers

From the Editor of Methods & Tools - Tue, 02/09/2016 - 15:24
Programmers are not like the other kids. They cannot and should not be managed like normal people if your intent is to produce high quality software. The things you would do to make the process go faster will actually make things go slower. This session gives you insight on the care and feeding of programmers, […]

Setting up Jenkins to run headless Selenium tests in Docker containers

Agile Testing - Grig Gheorghiu - Sat, 02/06/2016 - 01:40
This is the third post in a series on running headless Selenium WebDriver tests. Here are the first two posts:
  1. Running Selenium WebDriver tests using Firefox headless mode on Ubuntu
  2. Running headless Selenium WebDriver tests in Docker containers
In this post I will show how to add the final piece to this workflow, namely how to fully automate the execution of Selenium-based WebDriver tests running Firefox in headless mode in Docker containers. I will use Jenkins for this example, but the same applies to other continuous integration systems.
1) Install docker-engine on the server running Jenkins (I covered this in my post #2 above)
2) Add the jenkins user to the docker group so that Jenkins can run the docker command-line tool in order to communicate with the docker daemon. Remember to restart Jenkins after doing this.
3) Go through the rest of the workflow in my post above ("Running headless Selenium WebDriver tests in Docker containers") and make sure you can run all the commands in that post from the command line of the server running Jenkins.
4) Create a directory structure for your Selenium WebDriver tests (mine are written in Python). 
I have a directory called selenium-docker which contains a directory called tests, under which I put all my Python WebDriver tests named sel_wd_*.py. I also  have a simple shell script I named run_selenium_tests.sh which does the following:
#!/bin/bash
TARGET=$1 # e.g. someotherdomain.example.com (if not specified, the default is somedomain.example.com)
for f in `ls tests/sel_wd_*.py`; do    echo Running $f against $TARGET    python $f $TARGETdone
My selenium-docker directory also contains the xvfb.init file I need for starting up Xvfb in the container, and finally it contains this Dockerfile:
FROM ubuntu:trusty
RUN echo "deb http://ppa.launchpad.net/mozillateam/firefox-next/ubuntu trusty main" > /etc/apt/sources.list.d//mozillateam-firefox-next-trusty.listRUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE49EC21RUN apt-get updateRUN apt-get install -y firefox xvfb python-pipRUN pip install seleniumRUN mkdir -p /root/selenium/tests
ADD tests /root/selenium/testsADD run_all_selenium_tests.sh /root/selenium
ADD xvfb.init /etc/init.d/xvfbRUN chmod +x /etc/init.d/xvfbRUN update-rc.d xvfb defaults
ENV TARGET=somedomain.example.com
CMD (service xvfb start; export DISPLAY=:10; cd /root/selenium; ./run_all_selenium_tests.sh $TARGET)
I explained what this Dockerfile achieves in the 2nd post referenced above. The ADD instructions will copy all the files in the tests directory to the directory called /root/selenium/tests, and will copy run_all_selenium_tests.sh to /root/selenium. The ENV variable TARGET represents the URL against which we want to run our Selenium tests. It is set by default to somedomain.example.com, and is used as the first argument when running run_all_selenium_tests.sh in the CMD instruction.
At this point, I checked in the selenium-docker directory and all files and directories under it into a Github repository I will call 'devops'.
5) Create a new Jenkins project (I usually create a New Item and copy it from an existing project).
I specified that the build is parameterized and I indicated a choice parameter called TARGET_HOST with a few host/domain names that I want to test. I also specified Git as the Source Code Management type, and I indicated the URL of the devops repository on Github. Most of the action of course happens in the Jenkins build step, which in my case is of type "Execute shell". Here it is:
#!/bin/bash
set +e
IMAGE_NAME=selenium-wd:v1
cd $WORKSPACE/selenium-docker
# build the image out of the Dockerfile in the current directory/usr/bin/docker build -t $IMAGE_NAME .
# run a container based on the imageCONTAINER_ID=`/usr/bin/docker run -d -e "TARGET=$TARGET_HOST" $IMAGE_NAME`
echo CONTAINER_ID=$CONTAINER_ID  # while the container is still running, sleep and check logs; repeat every 40 secwhile [ $? -eq 0 ];do  sleep 40  /usr/bin/docker logs $CONTAINER_ID  /usr/bin/docker ps | grep $IMAGE_NAMEdone
# docker logs sends errors to stderr so we need to save its output to a file first/usr/bin/docker logs $CONTAINER_ID > d.out 2>&1
# remove the container so they don't keep accumulatingdocker rm $CONTAINER_ID
# mark jenkins build as failed if log output contains FAILEDgrep "FAILED" d.out
if [[ $? -eq 0 ]]; then    rm d.out   exit 1else  rm d.out  exit 0fi
Some notes:
  • it is recommended that you specify #!/bin/bash as the 1st line of your script, to make sure that bash is the shell that is being used
  • use set +e if you want the Jenkins shell script to continue after hitting a non-zero return code (the default behavior is for the script to stop on the first line it encounters an error and for the build to be marked as failed; subsequent lines won't get executed, resulting in much pulling of hair)
  • the Jenkins script will build a new image every time it runs, so that we make sure we have updated Selenium scripts in place
  • when running the container via docker run, we specify -e "TARGET=$TARGET_HOST" as an extra command line argument. This will override the ENV variable named TARGET in the Dockerfile with the value received from the Jenkins multiple choice dropdown for TARGET_HOST
  • the main part of the shell script stays in a while loop that checks for the return code of "/usr/bin/docker ps | grep $IMAGE_NAME". This is so we wait for all the Selenium tests to finish, at which point docker ps will not show the container running anymore (you can still see the container by running docker ps -a)
  • once the tests finish, we save the stdout and stderr of the docker logs command for our container to a file (this is so we capture both stdout and stderr; at first I tried something like docker logs $CONTAINER_ID | grep FAILED but this was never successful, because it was grep-ing against stdout, and errors are sent to stderr)
  • we grep the file (d.out) for the string FAILED and if we find it, we exit with code 1, i.e. unsuccessful as far as Jenkins is concerned. If we don't find it, we exit successfully with code 0.


Tooling in the .net World

Actively Lazy - Wed, 02/03/2016 - 21:51

As a refugee Java developer first washing up on the shores of .net land, I was pretty surprised at the poor state of the tools and libraries that seemed to be in common use. I couldn’t believe this shanty town was the state of the art; but with a spring in my step I marched inland. A few years down the road, I can honestly say it really isn’t that great – there are some high points, but there are a lot of low points. I hadn’t realised at the time but as a Java developer I’d been incredibly spoiled by a vivacious open source community producing loads of great software. In the .net world there seem to be two ways of solving any problem:

  1. The Microsoft way
  2. The other way*

*Note: may not work, probably costs money.

IDE

The obvious place to start is the IDE. I’m sorry, but Visual Studio just isn’t good enough. Compared to Eclipse and IntelliJ for day-to-day development Visual Studio is a poor imitation. Worse, until recently, it was expensive as a lone developer. While there were community editions of Visual Studio, these were even more pared down than the unusable professional edition. Frankly, if you can’t run ReSharper, it doesn’t count as an IDE.

I hear there are actually people out there who use Visual Studio without ReSharper. What are these people? Sadists? What do they do? I hope it isn’t writing software. Perhaps this explains the poor state of so much software; with tools this bad – is it any wonder?

But finally Microsoft have seen sense and recently the free version of Visual Studio supports plugins – which means you can use ReSharper. You still have to pay for it, but with their recent licensing changes I don’t feel quite so bad handing over a few quid a month renting ReSharper before I commit to buying an outright license. Obviously the situation for companies is different – it’s much easier for a company to justify spending the money on Visual Studio licenses and ReSharper licenses.

With ReSharper Visual Studio is at least closer to Eclipse or IntelliJ. It still falls short, but there is clearly only so much lipstick that JetBrains can put on that particular pig. I do however have great hope for Project Rider, basically IntelliJ for .net.

Restful Services

A few years back another Java refugee and I started trying to write a RESTful, CQRS-style web service in C#. We’d done the same in a previous company on the Java stack and expected our choices to be similarly varied. But instead of a vast plethora of approaches from basic HTTP listeners to servlet containers to full blown app servers we narrowed the field down to two choices:

  1. Microsoft’s WCF
  2. ServiceStack

My fellow developer played with WCF and decided he couldn’t make it easily fit the RESTful, CQRS style he had in mind. After playing with ServiceStack we found it could. But then begins a long, tortuous process of finding all the things ServiceStack hadn’t quite got right; all the things we didn’t quite agree with. Now, this is not entirely uncommon in any technology selection process. But we’d already quickly narrowed the field to two. We were now committed to a technology that at every turn was causing us new, unanticipated problems (most annoyingly, problems that other dev teams weren’t having with vanilla WCF services!)

We joked (not really joking) that it would be simpler to write our own service layer on top of the basic HTTP support in .net. In hindsight, it probably would have been. But really, we’d paid the price for having the temerity to step off the One True Microsoft Way.

Worse, what had started as an open source project went paid for during the development of our service – which meant that our brand new web service was immediately legacy as the service layer it was built on was no longer officially supported.

Dependency Management

The thing I found most surprising on arrival in .net land was that people were checking all their third party binaries into version control like it was the 1990s! Java developers might complain that Maven is nothing more than a DSL for downloading the internet. But you know what, it’s bloody good at downloading the internet. Instead, I had the internet checked in to version control.

However, salvation was at hand with NuGet. Except, NuGet really is a bit crap. NuGet doesn’t so much manage my dependencies as break them. All the damned time. Should we restrict all versions of a library (say log4net) to one version across the solution? Nah, let’s have a few variations. Oh, but nuget, now I get random runtime errors because of method signature mis-matches. But it doesn’t make the build fail? Brilliant, thank you, nuget.

So at my current place we’ve written a pre-build script to check that nuget hasn’t screwed the dependencies up. This pre-build check fails more often than I would like to believe.

So managing a coherent set of dependencies isn’t the job of the dependency tool, so what does it do? It downloads one file at a time from the internet. Well done. I think wget can do that, too. It’s a damned sight faster than the nuget power shell console, too. Nuget: it might break your builds, but at least it’s slow.

The Microsoft Way

And then I find things that blow me away. At my current place we’ve written some add-ins to Excel so that people who live and die by Excel can interact with our software. This is pretty cool: adding buttons and menus and ribbons into Excel, all integrated to my back-end services.

In my life as a Java developer I can never even imagine attempting this. The hoops to jump through would have been far too numerous. But in Visual Studio? Create a new, specific type of solution, hit F5, Excel opens. Set a breakpoint, Excel stops. Oh my God – this is an integrated development environment.

Obviously our data is all stored in Microsoft SQLServer. This also has brilliant integration with Visual Studio. For example, we experimented with creating a .net assembly to read some of the binary data we’re storing in the DB. This way we could run efficient queries on complex data types directly in the DB. The dev process for this? Similarly awesome: deploy to the DB and run directly from within Visual Studio. Holy integrated dev cycle, batman!

When there is a Microsoft way, this is why it is so compelling. Whatever they do will be brilliantly integrated with everything else they do. It might not have the flexibility you want, it might not have all the features you want. But it will be spectacularly well integrated with everything else you’re already using.

Why?

Why does it have to be this way? C# is really awesome language; well designed with a lot of expressive power. But the open source ecosystem around it is barren. If the Microsoft Way doesn’t fit the bill, you are often completely stuck.

I think it comes down to the history of Visual Studio being so expensive. Even as a C# developer by day, I am not spending the thick end of ÂŁ1,000 to get a matching dev environment at home, so I can play. Even if I had a solid idea of an open source project to start, I’m not going to invest a thousand quid just to see.

But finally Microsoft seem to be catching on to the open source thing. Free versions of Visual Studio and projects like CoreCLR can only help. But the Java ecosystem has a decade head start and this ultimately creates a network effect: it’s hard to write good open source software for .net because there’s so little good open source tooling for .net.


Categories: Programming, Testing & QA

Successful Software Development Project Failures

From the Editor of Methods & Tools - Mon, 02/01/2016 - 15:10
Software development projects have always been challenging. The most referenced source for metrics about project success is the Standish Group yearly CHAOS report. According to this source, around 30% of software development project were considered as a “failure” in the recent years. The failure rate of larger projects is a little bit higher than the […]

We. Use. Tools.

James Bach’s Blog - Sun, 01/31/2016 - 12:17

Context-Driven testers use tools to help ourselves test better. But, there is no such thing as test automation.

Want details? Here’s the 10,000 word explanation that Michael Bolton and I have been working on for months.

Editor’s Note: I have just posted version 1.03 of this article. This is the third revision we have made due to typos. Isn’t it interesting how hard it is to find typos in your own work before you ship an article? We used automation to help us with spelling, of course, but most of the typos are down to properly spelled words that are in the wrong context. Spelling tools can’t help us with that. Also, Word spell-checker still thinks there are dozens of misspelled words in our article, because of all the proper nouns, terms of art, and neologisms. Of course there are the grammar checking tools, too, right? Yeah… not really. The false positive rate is very high with those tools. I just did a sweep through every grammar problem the tool reported. Out of the five it thinks it found, only one, a missing hyphen, is plausibly a problem. The rest are essentially matters of writing style.

One of the lines it complained about is this: “The more people who use a tool, the more free support will be available…” The grammar checker thinks we should not say “more free” but rather “freer.” This may be correct, in general, but we are using parallelism, a rhetorical style that we feel outweighs the general rule about comparatives. Only humans can make these judgments, because the rules of grammar are sometimes fluid.

Categories: Testing & QA

Rich domain objects with DivineInject

Actively Lazy - Thu, 01/28/2016 - 22:01

DivineInject is a .net dependency injection framework, designed to be simple to use and easy to understand.

You can find the source for DivineInject on github. In this second part in the series we’ll look at creating rich domain objects, the first part covers getting started with Divine Inject.

Simple Domain Objects

As an example, imagine I have a simple shopping basket in my application. The shopping basket is encapsulated by a Basket object, for which the interface looks like this:

interface IBasket
{
    IList<IBasketItem> GetBasketContents();
    void AddItemToBasket(IBasketItem item);
}

The contents of the basket are actually backed by a service to provide persistence:

interface IBasketService
{
    IList<IBasketItem> GetBasketContents(Guid basketId);
    void AddItemToBasket(Guid basketId, IBasketItem item);
}

I can create a very simple implementation of my Basket:

class Basket : IBasket
{
    private readonly IBasketService _basketService;
    private readonly Guid _basketId = Guid.NewGuid();

    public Basket(IBasketService basketService)
    {
        _basketService = basketService;
    }

    public IList<IBasketItem> GetBasketContents()
    {
        return _basketService.GetBasketContents(_basketId);
    }

    public void AddItemToBasket(IBasketItem item)
    {
        _basketService.AddItemToBasket(_basketId, item);
    }
}

Since I’m using dependency injection, IBasketService sounds like a dependency, so how do I go about creating an instance of Basket? I don’t want to create it myself, I need the DI framework to create it for me, passing in dependencies.

I want to do something with how the Basket is created, so let’s start with a simple interface for creating baskets:

interface IBasketFactory
{
    IBasket Create();
}

When I’m creating a basket I don’t care about IBasketService or other dependencies; calling code just wants to be able to create a new, empty basket on demand. How would we implement this interface? Well, I could do the following – although I shouldn’t.

class BadBasketFactory : IBasketFactory
{
    public IBasket Create()
    {
        // DON'T DO THIS - just an example
        return new Basket(
            DivineInjector.Current.Get<IBasketService>());
    }
}

Now I’d never suggest actually doing this, explicitly calling the dependency injector. The last thing I want from my DI framework is to have references to it smeared all over the application. However, what this class does is basically what I want to happen.

DivineInject however can generate a class like this for you; this is configured at the same time you define the rest of your bindings:

DivineInjector.Current
    .Bind<IBasketFactory>().AsGeneratedFactoryFor<Basket>();

This generates an IBasketFactory implementation, which can create new IBasket implementations on demand (they will all actually be instances of Basket); all without having references to the DI framework smeared across my code. If I want to use the IBasketFactory, for example from my Session class, I declare it as a constructor arg the same as I would any other dependency:

public Session(IAuthenticationService authService, 
               IBasketFactory basketFactory)
{
    _authService = authService;
    _basketFactory = basketFactory;
}

The DI framework takes care of sorting out dependencies and I get a Session class with no references to DivineInject. When I need a new basket I just call _basketFactory.Create(). Since I have nice interfaces everywhere, everything is easy to mock so I can TDD everything.

Rich Domain Objects

Now what happens as my domain object becomes more complex? Say, for example, I want to be able to pass in some extra arguments to my constructor. Returning to our basket example: as well as starting a new, empty basket – isn’t there a possibility that I want to continue using an existing basket? E.g. in case of load balanced servers or fail-over. What do I do then?

I start by changing Basket, to allow me to pass in an existing basket id:

private readonly IBasketService _basketService;
private readonly Guid _basketId;

public Basket(IBasketService basketService)
{
    _basketService = basketService;
    _basketId = Guid.NewGuid();
}

public Basket(IBasketService basketService, Guid basketId)
{
    _basketService = basketService;
    _basketId = basketId;
}

I now have two constructors, one of which accepts a basket id. Since all Basket instances are created by an IBasketFactory, I need to change the factory interface, too:

interface IBasketFactory
{
    IBasket Create();
    IBasket UseExisting(Guid id);
}

I now have a new method on my IBasketFactory, if I was hand-coding the factory class I’d expect this second method to call the second constructor, passing in the basket id.

What do we need to tell DivineInject to make it generate this more complex IBasketFactory implementation? Nothing! That’s right, absolutely nothing – DivineInject will already generate a suitable IBasketFactory. Our original declaration above, is still sufficient:

DivineInjector.Current
    .Bind<IBasketFactory>().AsGeneratedFactoryFor<Basket>();

This generates an IBasketFactory implementation, returning a Basket instance for each method it finds on the interface. Since one of these methods takes a Guid, it tries to find a matching constructor which also takes a Guid, plus any dependencies it knows about. DivineInject can automatically wire up the right factory method to the right constructor, using the arguments it finds in each. Now, when a session wants to re-use an existing basket it just calls:

_basketFactory.UseExisting(existingBasketId)

This creates a new Basket instance, with dependencies wired up, passing in the basket id. Everything is still using interfaces so all your collaborations can be unit tested. Behind the scenes DivineInject generates the IL code to implement your factory interfaces, leaving you free to worry about your design.

By following this pattern we can create rich domain objects that include both state and dependencies: it becomes possible to create stateful objects that have behaviours (methods) that make sense in the domain. Successfully modelling your domain is critical to creating code that’s easy to understand and easy to change. DivineInject helps you model your domain better.


Categories: Programming, Testing & QA

Software Development Conferences Forecast January 2016

From the Editor of Methods & Tools - Tue, 01/26/2016 - 09:34
Here is a list of software development related conferences and events on Agile project management ( Scrum, Lean, Kanban), software testing and software quality, software architecture, programming (Java, .NET, JavaScript, Ruby, Python, PHP), DevOps and databases (NoSQL, MySQL, etc.) that will take place in the coming weeks and that have media partnerships with the Methods […]

Reinventing Testing: What is Integration Testing? (part 2)

James Bach’s Blog - Mon, 01/25/2016 - 10:45

These thoughts have become better because of these specific commenters on part 1: Jeff Nyman, James Huggett, Sean McErlean, Liza Ivinskaia, Jokin Aspiazu, Maxim Mikhailov, Anita Gujarathi, Mike Talks, Amit Wertheimer, Simon Morley, Dimitar Dimitrov, John Stevenson. Additionally, thank you Michael Bolton and thanks to the student whose productive confusion helped me discover a blindspot in my work, Anita Gujarathi.

Integration testing is a term I don’t use much– not because it doesn’t matter, but because it is so fundamental that it is already baked into many of the other working concepts and techniques of testing. Still, in the past week, I decided to upgrade my ability to quickly explain integration, integration risk, and integration testing. This is part of a process I recommend for all serious testers. I call it: reinventing testing. Each of us may reinvent testing concepts for ourselves, and engage in vigorous debates about them (see the comments on part 1, which is now the most commented of any post I have ever done).

For those of you interested in getting to a common language for testing, this is what I believe is the best way we have available to us. As each of us works to clarify his own thinking, a de facto consensus about reasonable testing ontology will form over time, community by community.

So here we go…

There several kinds of testing that involve or overlap with or may even be synonymous with integration testing, including: regression testing, system testing, field testing, interoperability testing, compatibility testing, platform testing, and risk-based testing. Most testing, in fact, no matter what it’s called, is also integration testing.

Here is my definition of integration testing, based on my own analysis, conversations with RST instructors (mainly Michael Bolton), and stimulated by the many commenters from part 1. All of my assertions and definitions are true within the Rapid Software Testing methodology namespace, which means that you don’t have to agree with me unless you claim to be using RST.

What is integration testing?

Integration testing is:
1. Testing motivated by potential risk related to integration.
2. Tests designed specifically to assess risk related to integration.

Notes:

1. “Motivated by” and “designed specifically to” overlap but are not the same. For instance, if you know that a dangerous criminal is on the loose in your neighborhood you may behave in a generally cautious or vigilant way even if you don’t know where the criminal is or what he looks like. But if you know what he looks like, what he is wearing, how he behaves or where he is, you can take more specific measures to find him or avoid him. Similarly, a newly integrated product may create a situation where any kind of testing may be worth doing, even if that testing is not specifically aimed at uncovering integration bugs, as such; OR you can perform tests aimed at exposing just the sort of bugs that integration typically causes, such as by performing operations that maximize the interaction of components.

The phrase “integration testing” may therefore represent ANY testing performed specifically in an “integration context”, or applying a specific “integration test technique” in ANY context.

This is a special case of the difference between risk-based test management and risk-based test design. The former assigns resources to places where there is potential risk but does not dictate the testing to be performed; whereas the latter crafts specific tests to examine the product for specific kinds of problems.

2. “Potential risk” is not the same as “risk.” Risk is the danger of something bad happening, and it can be viewed from at least three perspectives: probability of a bad event occurring, the impact of that event if it occurs, and our uncertainty about either of those things. A potential risk is a risk about which there is substantial uncertainty (in other words, you don’t know how likely the bug is to be in the product or you don’t know how bad it could be if it were present). The main point of testing is to eliminate uncertainty about risk, so this often begins with guessing about potential risk (in other words, making wild guesses, educated guesses, or highly informed analyses about where bugs are likely to be).

Example: I am testing something for the first time. I don’t know how it will deal with stressful input, but stress often causes failure, so that’s a potential risk. If I were to perform stress testing, I would learn a lot about how the product really handles stress, and the potential risk would be transformed into a high risk (if I found serious bugs related to stress) or a low risk (if the product handled stress in a consistently graceful way).

What is integration?

General definition from the Oxford English Dictionary: “The making up or composition of a whole by adding together or combining the separate parts or elements; combination into an integral whole: a making whole or entire.”

Based on this, we can make a simple technical definition related to products:

Integration is:
v. the process of constructing a product from parts.
n. a product constructed from parts.

Now, based on General Systems Theory, we make these assertions:

An integration, in some way and to some degree:

  1. Is composed of parts:
  • …that come from differing sources.
  • …that were produced for differing purposes.
  • …that were produced at different times.
  • …that have differing attributes.
  1. Creates or represents an internal environment for its parts:
  • …in which its parts interact among themselves.
  • …in which its parts depend on each other.
  • …in which its parts interact with or depend on an external environment.
  • …in which these things are not visible from the outside.
  1. Possesses attributes relative to its parts:
  • …that depend on them.
  • …that differ from them.

Therefore, you might not be able to discern everything you want to know about an integration just by looking at its parts.

This is why integration risk exists. In complex or important systems, integration testing will be critically important, especially after changes have been made.

It may be possible to gain enough knowledge about an integration to characterize the risk (or to speak more plainly: it may be possible to find all the important integration bugs) without doing integration testing. You might be able to do it with unit testing. However, that process, although possible in some cases, might be impractical. This is the case partly because the parts may have been produced by different people with different assumptions, because it is difficult to simulate the environment of an integration prior to actual integration, or because unit testing tends to focus on what the units CAN do and not on what they ACTUALLY NEED to do. (If you unit test a calculator, that’s a lot of work. But if that calculator will only ever be asked to add numbers under 50, you don’t need to do all that work.)

Integration testing, although in some senses being complex, may actually simplify your testing since some parts mask the behavior of other parts and maybe all you need to care about is the final outputs.

Notes:

1. “In some way and to some degree” means that these assertions are to be interpreted heuristically. In any specific situation, these assertions are highly likely to apply in some interesting or important way, but might not. An obvious example is where I wrote above that the “parts interact with each other.” The stricter truth is that the parts within an integration probably do not EACH directly interact with ALL the other ones, and probably do not interact to the same degree and in the same ways. To think of it heuristically, interpret it as a gentle warning such as  “if you integrate something, make it your business to know how the parts might interact or depend on each other, because that knowledge is probably important.”

By using the phrase “in some way and to some degree” as a blanket qualifier, I can simplify the rest of the text, since I don’t have to embed other qualifiers.

2. “Constructing from parts” does not necessarily mean that the parts pre-existed the product, or have a separate existence outside the product, or are unchanged by the process of integration. It just means that we can think productively about pieces of the product and how they interact with other pieces.

3. A product may possess attributes that none of its parts possess, or that differ from them in unanticipated or unknown ways. A simple example is the stability of a tripod, which is not found in any of its individual legs, but in all the legs working together.

4. Disintegration also creates integration risk. When you takes things away, or take things apart, you end up with a new integration, and that is subject to the much the same risk as putting them together.

5. The attributes of a product and all its behaviors obviously depend largely on the parts that comprise it, but also on other factors such as the state of those parts, the configurations and states of external and internal environments, and the underlying rules by which those things operate (ultimately, physics, but more immediately, the communication and processing protocols of the computing environment).

6. Environment refers to the outside of some object (an object being a product or a part of a product), comprising factors that may interact with that object. A particular environment might be internal in some respects or external in other respects, at the same time.

  • An internal environment is an environment controlled by the product and accessible only to its parts. It is inside the product, but from the point vantage point of some of parts, it’s outside of them. For instance, to a spark plug the inside of an engine cylinder is an environment, but since it is not outside the car as a whole, it’s an internal environment. Technology often consists of deeply nested environments.
  • An external environment is an environment inhabited but not controlled by the product.
  • Control is not an all-or-nothing thing. There are different levels and types of control. For this reason it is not always possible to strictly identify the exact scope of a product or its various and possibly overlapping environments. This fact is much of what makes testing– and especially security testing– such a challenging problem. A lot of malicious hacking is based on the discovery that something that the developers thought was outside the product is sometimes inside it.

7. An interaction occurs when one thing influences another thing. (A “thing” can be a part, an environment, a whole product, or anything else.)

8. A dependency occurs when one thing requires another thing to perform an action or possess an attribute (or not to) in order for the first thing to behave in a certain way or fulfill a certain requirement. See connascence and coupling.

9. Integration is not all or nothing– there are differing degrees and kinds. A product may be accidentally integrated, in that it works using parts that no one realizes that it has. It may be loosely integrated, such as a gecko that can jettison its tail, or a browser with a plugin. It may be tightly integrated, such as when we take the code from one product and add it to another product in different places, editing as we go. (Or when you digest food.) It may preserve the existing interfaces of its parts or violate them or re-design them or eliminate them. The integration definition and assertions, above, form a heuristic pattern– a sort of lens– by which we can make better sense of the product and how it might fail. Different people may identify different things as parts, environments or products. That’s okay. We are free to move the lens around and try out different perspectives, too.

Example of an Integration Problem

bitmap

This diagram shows a classic integration bug: dueling dependencies. In the top two panels, two components are happy to work within their own environments. Neither is aware of the other while they work on, let’s say, separate computers.

But when they are installed together on the same machine, it may turn out that each depends on factors that exclude the other. Even though the components themselves don’t clash (the blue A box and the blue B boxes don’t overlap). Often such dependencies are poorly documented, and may be entirely unknown to the developer before integration time.

It is possible to discover this through unit testing… but so much easier and probably cheaper just to try to integrate sooner rather than later and test in that context.

 

Categories: Testing & QA