Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Process Management

Agile Program Manager

Individually, each boy is a project. Together they're a program to be managed.

Individually, each boy is a project. Together they’re a program to be managed.

Scaling Agile project management to large, complex endeavors requires an Agile Program Manager to address the big picture coordination of programs.  Program management is the discipline of coordinating and managing large efforts comprised of a number of parallel and related projects. Scrum leverages a concept called the Scrum of Scrums to perform many of the activities need for program management.  Agile Program Management is not just repurposed project management or a part-time job for a Scrum Master.

Agile Program Managers coordinate and track expectations across all projects under the umbrella of the program, whether the projects are using Agile or not. Coordination includes activities like identifying and tracking dependencies, tracking risks and issues and communication. Coordination of the larger program generally requires developing a portfolio of moving parts at the epic or function level across all of the related projects (epics are large user stories that represent large concepts that will be broken down later). Agile Program Managers layer each project’s release plans on top of the program portfolio to provide a platform for coordinated release planning. Techniques like Kanban can be used for tracking and visualizing the portfolio.  Visualization show how the epics or functions are progressing as they are developed and staged for delivery to the program’s customers.

Facilitating communication is one the roles of an Agile Program Managers. The Scrum of Scrums is the primary vehicle for ensuring communication.  The Scum of Scrums is a meeting of the all of the directly responsible individuals (DRIs) from each team in the program. The DRI has the responsibility to act as the conduit of information for his or her team to the Agile Program Manager and other DRIs. The DRI raises issues, risks, concerns and needs. In short, the DRI communicates to the team and the Scrum of Scrums. The Scrum of Scrums is best as a daily meeting of the DRIs chaired by the Agile Program Manager, however the frequency can be tailored to meet the program’s needs.  A pattern I have seen used to minimize overhead is varying the frequency of the Scrum of Scrums based on project risk.

Another set of activities that generally fall to the Agile Program Manager is the development and communication of program status information. Chairing high-level status meetings, such as those with sponsor or other guidance groups, is a natural extension of the role. However this requires the Agile Program Manager to act as a conduit of information by transferring knowledge from the Scrum of Scrums to the sponsors and back again. Any problem with information flow can potentially cause bad decisions and will affect the program.

It is important to recognize that Agile Program Management is more than a specialization within the realm of project management or a side job a Scrum Master can do in his or her spare time.  Agile Program Managers need to be well versed in both Agile techniques and in standard program management techniques because the Agile Program Manager is a hybrid from both camps. Agile Program Managers build the big picture view that a portfolio view of all of the related projects will deliver. They also must facilitate communication via the Scrum of Scrums and standard program status vehicles.  The Agile Program Manager many times must straddle the line between both the Agile and waterfall worlds.


Categories: Process Management

Agile Teams Making Decisions (Refined)

Team members working together.

Team members working together (or not).

In Agile projects, the roles of the classic project manager in Scrum are spread across the three basic roles (Product Owner, Scrum Master and Development Team). A fourth role, the Agile Program Manager (known as a Release Train Engineer in SAFe), is needed when multiple projects are joined together to become a coordinated program.  The primary measures of success in Agile projects are delivered business value and customer satisfaction.  These attributes subsume the classic topics of on-time, on-budget and on-scope. (Note: Delivered value and customer satisfaction should be the primary measure of success in ALL types of projects, however these are not generally how project teams are held accountable.)

As teams learn to embrace and use Agile principles, they need to learn how to make decisions as a team. The decisions that teams need to learn how to make for themselves always have consequences, and sometimes those consequences will be negative. To accomplish this learning process in the least risky manner, the team should use techniques like delaying decisions as late as is practical and delivering completed work within short time boxes. These techniques reduce risk by increasing the time the team has to gather knowledge and by getting the team feedback quickly. The organization also must learn how to encourage the team to make good decisions while giving them the latitude to mess up. This requires the organization to accept some level of explicit initial ambiguity that is caused by delaying decisions, rather than implicit ambiguity of making decisions early that later turn out to be wrong. The organization must also learn to evaluate teams and individuals less on the outcome of a single decision and more on the outcome of the value the team delivered.

Teams also have to unlearn habits, for example, relying on others to plan for them. In order to do that, all leaders and teams must have an understanding of the true goals of the project (listen to my interview with David Marquet) and how the project fits into the strategic goals of the organization.

Teams make designs daily that affect the direction of the sprint and project. The faster these decisions are made the higher the team’s velocity or productivity. Having a solid understanding of the real goals of the project helps the team make decisions more effectively. Organizations need to learn how share knowledge that today is generally compartmentalized between developers, testers or analysts.

The process of learning and unlearning occurs on a continuum as teams push toward a type of collective self-actualization. As any team moves toward its full potential, the organization’s need to control planning and decisions falls away. If the organization doesn’t back away from the tenants of command and control and move towards the Agile principles, the ability of any team to continue to grow will be constrained.  The tipping point generally occurs when an organization realizes that self-managing and self-organizing teams deliver superior value and higher customer satisfaction and that in the long run is what keeps CIOs employed.


Categories: Process Management

Debian Size Claims - New Lecture Posted

10x Software Development - Steve McConnell - Tue, 06/30/2015 - 19:17

In this week's lecture (https://cxlearn.com) I demonstrate how to use some of the size information we've discussed in other lectures by diving into the Wikipedia claims about the sizes of various versions of Debian.  The point of this week's lecture is to show how to apply critical thinking to size information presented by an authoritative source (Wikipedia), and how to arrive at a confident conclusion that that information is not credible. Practicing software professionals should be able to look at size claims like the Debian size claims and, based on general knowledge, immediately think, "That seems far from credible." Yet, few professionals actually do that. My hope is that working through public examples like this in the lecture series will help software professionals improve their instincts and judgment, which can then be applied to projects in their own organizations. 

Lectures posted so far include:  

0.0 Understanding Software Projects - Intro
     0.1 Introduction - My Background
     0.2 Reading the News
     0.3 Definitions and Notations 

1.0 The Software Lifecycle Model - Intro
     1.1 Variations in Iteration 
     1.2 Lifecycle Model - Defect Removal
     1.3 Lifecycle Model Applied to Common Methodologies
     1.4 Lifecycle Model - Selecting an Iteration Approach  

2.0 Software Size
     2.05 Size - Comments on Lines of Code
     2.1 Size - Staff Sizes 
     2.2 Size - Schedule Basics 
     2.3 Size - Debian Size Claims (New)

Check out the lectures at http://cxlearn.com!

Understanding Software Projects - Steve McConnell

 

Succeeding with Geographically Distributed Scrum Teams - New White Paper

10x Software Development - Steve McConnell - Tue, 06/30/2015 - 19:02

We have a new white paper, "Succeeding with Geographically Distributed Scrum Teams." To quote the white paper itself: 

When organizations adopt Agile throughout the enterprise, they typically apply it to both large and small projects. The gap is that most Agile methodologies, such as Scrum and XP, are team-level workflow approaches. These approaches can be highly effective at the team level, but they do not address large project architecture, project management, requirements, and project planning needs. Our clients find that succeeding with Scrum on a large, geographically distributed team requires adopting additional practices to ensure the necessary coordination, communication, integration, and architectural work. This white paper discusses common considerations for success with geographically distributed Scrum.

Check it out!

Prioritize and Optimize Over a Slightly Longer Horizon

Mike Cohn's Blog - Tue, 06/30/2015 - 15:00

A lot of agile literature stresses that product owners must prioritize the delivery of value. I’m not going to argue with that. But I am going to argue that product owners need to optimize over a slightly longer horizon than a single sprint.

A product owner’s goal is to maximize the amount of value delivered over the life a product. If the product owner shows up at each sprint planning meeting focused on maximizing the value delivered in only that one sprint, the product owner will never choose to invest in the system’s future.

The product owner will instead always choose to deliver the feature that delivers the most value in the immediate future regardless of the long-term benefit. Such a product owner is like the pleasure-seeking student who parties every night during the semester but fails, and has to repeat the course during the summer.

A product owner with a short time horizon may have the team work on features A1, B1, C1 and D1 in four different parts of the application this sprint because those are the highest valued features.

A product owner with a more appropriate, longer view of the product will instead have the team work on A1, A2, A3 and A4 in the first sprint because there are synergies to working on all the features in area A at the same time.

Agile is still about focusing on value. And it’s still about making sure we deliver value in the short term. We just don’t want to become shortsighted about it.

How to create the smallest possible docker container of any image

Xebia Blog - Tue, 06/30/2015 - 10:46

Once you start to do some serious work with Docker, you soon find that downloading images from the registry is a real bottleneck in starting applications. In this blog post we show you how you can reduce the size of any docker image to just a few percent of the original. So is your image too fat, try stripping your Docker image! The strip-docker-image utility demonstrated in this blog makes your containers faster and safer at the same time!


We are working quite intensively on our High Available Docker Container Platform  using CoreOS and Consul which consists of a number of containers (NGiNX, HAProxy, the Registrator and Consul). These containers run on each of the nodes in our CoreOS cluster and when the cluster boots, more than 600Mb is downloaded by the 3 nodes in the cluster. This is quite time consuming.

cargonauts/consul-http-router      latest              7b9a6e858751        7 days ago          153 MB
cargonauts/progrium-consul         latest              32253bc8752d        7 weeks ago         60.75 MB
progrium/registrator               latest              6084f839101b        4 months ago        13.75 MB

The size of the images is not only detrimental to the boot time of our platform, it also increases the attack surface of the container.  With 153Mb of utilities in the  NGiNX based consul-http-router, there is a lot of stuff in the container that you can use once you get inside. As we were thinking of running this router in a DMZ, we wanted to minimise the amount of tools lying around for a potential hacker.

From our colleague Adriaan de Jonge we already learned how to create the smallest possible Docker container  for a Go program. Could we repeat this by just extracting the NGiNX executable from the official distribution and copying it onto a scratch image?  And it turns out we can!

finding the necessary files

Using the utility dpkg we can list all the files that are installed by NGiNX

docker run nginx dpkg -L nginx
...
/.
/usr
/usr/sbin
/usr/sbin/nginx
/usr/share
/usr/share/doc
/usr/share/doc/nginx
...
/etc/init.d/nginx
locating dependent shared libraries

So we have the list of files in the package, but we do not have the shared libraries that are referenced by the executable. Fortunately, these can be retrieved using the ldd utility.

docker run nginx ldd /usr/sbin/nginx
...
	linux-vdso.so.1 (0x00007fff561d6000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd8f17cf000)
	libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fd8f1598000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fd8f1329000)
	libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007fd8f10c9000)
	libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fd8f0cce000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd8f0ab2000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd8f0709000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fd8f19f0000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd8f0505000)
Following and including symbolic links

Now we have the executable and the referenced shared libraries, it turns out that ldd normally names the symbolic link and not the actual file name of the shared library.

docker run nginx ls -l /lib/x86_64-linux-gnu/libcrypt.so.1
...
lrwxrwxrwx 1 root root 16 Apr 15 00:01 /lib/x86_64-linux-gnu/libcrypt.so.1 -> libcrypt-2.19.so

By resolving the symbolic links and including both the link and the file, we are ready to export the bare essentials from the container!

getpwnam does not work

But after copying all essentials files to a scratch image, NGiNX did not start.  It appeared that NGiNX tries to resolve the user id 'nginx' and fails to do so.

docker run -P  --entrypoint /usr/sbin/nginx stripped-nginx  -g "daemon off;"
...
2015/06/29 21:29:08 [emerg] 1#1: getpwnam("nginx") failed (2: No such file or directory) in /etc/nginx/nginx.conf:2
nginx: [emerg] getpwnam("nginx") failed (2: No such file or directory) in /etc/nginx/nginx.conf:2

It turned out that the shared libraries for the name switch service reading /etc/passwd and /etc/group are loaded at runtime and not referenced in the shared libraries. By adding these shared libraries ( (/lib/*/libnss*) to the container, NGiNX worked!

strip-docker-image example

So now, the strip-docker-image utility is here for you to use!

    strip-docker-image  -i image-name
                        -t target-image-name
                        [-p package]
                        [-f file]
                        [-x expose-port]
                        [-v]

The options are explained below:

-i image-name           to strip
-t target-image-name    the image name of the stripped image
-p package              package to include from image, multiple -p allowed.
-f file                 file to include from image, multiple -f allowed.
-x port                 to expose.
-v                      verbose.

The following example creates a new nginx image, named stripped-nginx based on the official Docker image:

strip-docker-image -i nginx -t stripped-nginx  \
                           -x 80 \
                           -p nginx  \
                           -f /etc/passwd \
                           -f /etc/group \
                           -f '/lib/*/libnss*' \
                           -f /bin/ls \
                           -f /bin/cat \
                           -f /bin/sh \
                           -f /bin/mkdir \
                           -f /bin/ps \
                           -f /var/run \
                           -f /var/log/nginx \
                           -f /var/cache/nginx

Aside from the nginx package, we add the files /etc/passwd, /etc/group and /lib/*/libnss* shared libraries. The directories /var/run, /var/log/nginx and /var/cache/nginx are required for NGiNX to operate. In addition, we added /bin/sh and a few handy utilities, just to be able to snoop around a little bit.

The stripped image has now shrunk to an incredible 5.4% of the original 132.8 Mb to just 7.3Mb and is still fully operational!

docker images | grep nginx
...
stripped-nginx                     latest              d61912afaf16        21 seconds ago      7.297 MB
nginx                              1.9.2               319d2015d149        12 days ago         132.8 MB

And it works!

ID=$(docker run -P -d --entrypoint /usr/sbin/nginx stripped-nginx  -g "daemon off;")
docker run --link $ID:stripped cargonauts/toolbox-networking curl -s -D - http://stripped
...
HTTP/1.1 200 OK

For HAProxy, checkout the examples directory.

Conclusion

It is possible to use the official images that are maintained and distributed by Docker and strip them down to their bare essentials, ready for use! It speeds up load times and reduces the attack surface of that specific container.

Checkout the github repository for the script and the manual page.

Please send me your examples of incredibly shrunk Docker images!

Software Development Conferences Forecast June 2015

From the Editor of Methods & Tools - Mon, 06/29/2015 - 14:17
Here is a list of software development related conferences and events on Agile project management ( Scrum, Lean, Kanban), software testing and software quality, software architecture, programming (Java, .NET, JavaScript, Ruby, Python, PHP) and databases (NoSQL, MySQL, etc.) that will take place in the coming weeks and that have media partnerships with the Methods & […]

SPaMCAST 348 - Woody Zuill, #NoEstimates

Software Process and Measurement Cast - Sun, 06/28/2015 - 22:00

The Software Process and Measurement Cast features our interview with Woody Zuill.  We talked about the concept and controversy swirling around #NoEstimates. Even if the concept is a bridge too far for you, the conversation is important because we talked about why thinking and questioning is a critical survival technique. As Woody points out, it is important to peer past the “thou musts” to gain greater understanding of what you should be doing!

Woody Zuill has been programming computers for 30+ years. Over the last 15+ years he has worked as an Agile Coach, Trainer, and Extreme Programmer and now works with Industrial Logic as a Trainer/Coach/Consultant for Agile and Lean software development. He believes code must be simple, clean, and maintainable to realize the Agile promise of Responding to Change. 

Contact Information
Mob Programming: http://mobprogramming.org/
Blog: http://zuill.us/WoodyZuill/
Twitter: https://twitter.com/woodyzuill

Call to action!

I have a challenge for the Software Process and Measurement Cast listeners for the next few weeks.  I would like you find one person that you think would like the podcast and introduce them to the cast.  This might mean sending them the URL or teaching how to download podcasts.  If you like the podcast and think it is valuable they will be thankful to you for introducing them to the Software Process and Measurement Cast! Thank you in advance!

Re-Read Saturday News

We have just begun the Re-Read Saturday of The Mythical Man-Month. We are off to a rousing start beginning with the Tar Pit.   Get a copy now and start reading!

The Re-Read Saturday and other great articles can be found on the Software Process and Measurement Blog.

Remember: We just completed the Re-Read Saturday of Eliyahu M. Goldratt and Jeff Cox’s The Goal: A Process of Ongoing Improvement, which began on February 21nd. What did you think?  Did the re-read cause you to read The Goal back up for a refresher? Visit the Software Process and Measurement Blog and review the whole re-read.

Note: If you don’t have a copy of the book, buy one.  If you use the link below it will support the Software Process and Measurement blog and podcast.

Dead Tree Version or Kindle Version 

Upcoming Events

Software Quality and Test Management 

September 13 – 18, 2015

San Diego, California

http://qualitymanagementconference.com/

I will be speaking on the impact of cognitive biases on teams!  Let me know if you are attending!

More on other great conferences soon!

Next SPaMCast

The next Software Process and Measurement Cast is a magazine installment.  We will feature our essay on Agile Testing. The flow of testing is different in an Agile project.  In many cases, organizations have either not recognized the change in flow, or have created Agile/waterfall hybrids with test groups holding onto waterfall patterns.  While some of the hybrids are driven by mandated contractual relationships, the majority are driven by lack of understanding or fear of how testing should flow in Agile projects.

We will also have new installments from Jeremy Berriault’s QA Corner.  Jeremy, is a leader in the world of quality assurance and testing and was originally interviewed on the Software Process and Measurement Cast 274. The third column features Steve Tendon discussing more of his great new book, Hyper-Productive Knowledge Work Performance. 

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.

Categories: Process Management

SPaMCAST 348 – Woody Zuill, #NoEstimates

 www.spamcast.net

http://www.spamcast.net

Listen Now

Subscribe on iTunes

The Software Process and Measurement Cast features our interview with Woody Zuill.  We talked about the concept and controversy swirling around #NoEstimates. Even if the concept is a bridge too far for you, the conversation is important because we talked about why thinking and questioning is a critical survival technique. As Woody points out, it is important to peer past the “thou musts” to gain greater understanding of what you should be doing!

Woody Zuill has been programming computers for 30+ years. Over the last 15+ years he has worked as an Agile Coach, Trainer, and Extreme Programmer and now works with Industrial Logic as a Trainer/Coach/Consultant for Agile and Lean software development. He believes code must be simple, clean, and maintainable to realize the Agile promise of Responding to Change.

Contact Information
Mob Programming: http://mobprogramming.org/
Blog: http://zuill.us/WoodyZuill/
Twitter: https://twitter.com/woodyzuill

Call to action!

I have a challenge for the Software Process and Measurement Cast listeners for the next few weeks.  I would like you find one person that you think would like the podcast and introduce them to the cast.  This might mean sending them the URL or teaching how to download podcasts.  If you like the podcast and think it is valuable they will be thankful to you for introducing them to the Software Process and Measurement Cast! Thank you in advance!

Re-Read Saturday News

We have just begun the Re-Read Saturday of The Mythical Man-Month. We are off to a rousing start beginning with the Tar Pit.   Get a copy now and start reading!

The Re-Read Saturday and other great articles can be found on the Software Process and Measurement Blog.

 

Remember: We just completed the Re-Read Saturday of Eliyahu M. Goldratt and Jeff Cox’s The Goal: A Process of Ongoing Improvement, which began on February 21nd. What did you think?  Did the re-read cause you to read The Goal back up for a refresher? Visit the Software Process and Measurement Blog and review the whole re-read.

Note: If you don’t have a copy of the book, buy one.  If you use the link below it will support the Software Process and Measurement blog and podcast.

Dead Tree Version or Kindle Version 

Upcoming Events

Software Quality and Test Management 

September 13 – 18, 2015

San Diego, California

http://qualitymanagementconference.com/

I will be speaking on the impact of cognitive biases on teams!  Let me know if you are attending!

 

More on other great conferences soon!

 

Next SPaMCast

The next Software Process and Measurement Cast is a magazine installment.  We will feature our essay on Agile Testing. The flow of testing is different in an Agile project.  In many cases, organizations have either not recognized the change in flow, or have created Agile/waterfall hybrids with test groups holding onto waterfall patterns.  While some of the hybrids are driven by mandated contractual relationships, the majority are driven by lack of understanding or fear of how testing should flow in Agile projects.

We will also have new installments from Jeremy Berriault’s QA Corner.  Jeremy, is a leader in the world of quality assurance and testing and was originally interviewed on the Software Process and Measurement Cast 274. The third column features Steve Tendon discussing more of his great new book, Hyper-Productive Knowledge Work Performance.

 

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.


Categories: Process Management

Re-Read Saturday: Mythical Man-Month, Part 1

The Mythical Man-Month

The Mythical Man-Month

The Mythical Man-Month: Essays on Software Engineering was originally published in 1975. That was the year I took my first course in programming – FORTRAN, at Memphis State University. Mythical Man-Month has been a must read for every professional involved in delivering value since it was published. The core themes in the book, which include Brook’s Law (adding people to a late project just makes it later – more in a future installment), are just as important today as they were when they were published because they still true for all types of software development and maintenance.   The Mythical Man-Month, I think on reflection you will agree, is still important because we are still trying to come to grips with most if not all of the concepts that Brooks exposed.

Here is the game plan for the Re-Read. We will be reading from Anniversary Edition of the Mythical Man-Month (Addison Wesley 1995). My copy is the 14th printing from 2014. Part of this re-read will actually be a first read for me. I believe I originally read Man-Month in the early 1980’s, and this edition has a new preface and four new chapters. My intent during the re-read is to cover two essays/chapters per week so the re-read takes ten weeks “ish.”  Today will be the exception to the rule. We will tackle the first essay, The Tar Pit (I already used the ideas in the essay to make a point in the essay on Scrum of Scrum membership).

Preface(s):
The original preface provides background for Brooks’ perspective. Brooks came to his conclusions based on a history as a hardware architect then as the manager of the OS/360 project (IBM’s highly successful mainframe operating system). In the preface Brooks telegraphs one of his central theorems – large projects are different than small projects. It took a few years and the Agile Manifesto (2001) to recognize this fact and begin to act on it.

The anniversary edition includes a new preface in addition to the original. There were two notes in the new preface that caused me pause. The first was the observation by Brooks was that few of the central thoughts in the book had been critiqued, proven, and disproven by research and experience. I would like your feedback as we review the essays. I will explicitly point those points out during the re-read and if miss any skip ahead to Chapter 18 to keep me honest. The second point, based on my interest in the publishing industry,  from the new preface was that Brooks was able to publicly thank Addison Wesley staff that he found instrumental when preparing the first edition. The publishing world has changed. The question I pose to the Software Process and Measurement Cast blog readers is has our profession changed as much?

Chapter 1: The Tar Pit

In the Tar Pit Brooks sets the context for the system programming as a craft that is more than just the lone coder sitting in his or her office creating a stand alone app but rather a member of a team building large scale systems. The Tar Pit describes why the complexity and level of effort is different and then exposes the joy and woes of the profession.

The first takeaway from the Tar Pit is that as the product of a piece of work progresses from the delivery of a single program to programming product or system then to a programming systems product, the complexity of managing and understanding the work increases. As size and complexity increase so do tasks and activities to ensure the product works together. Brook suggests that the difference between level of effort between the creation of stand-alone program and a systems product is 9 times! The complexity of coordinating all of these additional tasks and activities while solving complex business problems impacts our ability to deliver on-time, on-budget and on-scope. At times, it also impact our ability to deliver WOW.

The second takeaway are the joys of programming.  Joys motivate programmers to put forth the effort needed to deliver value while trying to navigate the tar pit.  Brooks points out that the rewards of programming are at least four fold:

  1. The joy of making things.
  2. The joy of making things that are useful to others.
  3. The joy of problem solving.
  4. The joy of learning.

These joys are offset by several woes.  These woes are the part of the tar pit that make programming hard.

  1. Expectation of perfection. Unless the code is correct and solves the requirements, it is wrong. There is little to no gray area.
  2. Dependence on others. Others set your objectives, provide resources and often control the source of information. This often leads to the tension caused when individual or teams have the responsibility to deliver without commensurate authority. Agile principles, when applied to teams, are a start to addressing these woes.
  3. Finding the bugs in the big picture.  Designing the big picture is significantly more fun than the tedious effort required to find bugs.
  4. Dead on arrival projects.  Long running projects are occasionally dead on arrival or the market has changed and what has been delivered is not what is needed.

In 1975 when this essay was published Agile was not a word one would attach to building applications or systems. However I think we can see the seeds of the Agile revolution in this essay.


Categories: Process Management

Scala development with GitHub's Atom editor

Xebia Blog - Sat, 06/27/2015 - 14:57
.code { font-family: monospace; background-color: #eeeeee; }

GitHub recently released version 1.0 of their Atom editor. This post gives a rough overview of its Scala support.

Basic features

Basic features such as Scala syntax highlighting are provided by the language-scala plugin.

Some work on worksheets as found in e.g. Eclipse has been done in the scala-worksheet-plus plugin, but this is still missing major features and not very useful at this time.

Navigation and completion Ctags

Atom supports basic 'Go to Declaration' (ctrl-alt-down) and 'Search symbol' (cmd-shift-r) support by way of the default ctags-based symbols-view.

While there are multiple sbt plugins for generating ctags, the easiest seems to be to have Ensime download the sources (more on that below) and invoke ctags manually: put this configuration in your home directory and run the 'ctags' command from your project root.

This is useful for searching for symbols, but limited for finding declarations: for example, when checking the declaration for Success, ctags doesn't know whether this is scala.util.Success, akka.actor.Status.Success, spray.http.StatusCodes.Success or some other 3rd-party or local symbol with that name.

Ensime

This is where the Ensime plugin comes in.

Ensime is a service for Scala IDE support, originally written for the Scala support in Emacs. The project metadata for Ensime can be generated with 'sbt gen-ensime' from the ensime-sbt sbt plugin.

Usage

Start the Ensime server from Atom with 'cmd-shift-p' 'Ensime: start'. After a small pause the status bar proclaims 'Indexer ready!' and you should be good to go.

At this point the main features are 'jump to definition' (alt-click), hover for type info, and auto-completion:

atom.io ensime completion

There are some rough edges, but this is a promising start based on a solid foundation.

Conclusions

While Atom is already a pleasant, modern, open source, cross platform editor, it is clearly still early days.

The Scala support in Atom is not yet as polished as in IDE's such as IntelliJ IDEA or as stable as in more mature editors such as Sublime Text, but is already practically useful and has serious potential. Startup is not instant, but I did not notice a 'sluggish feel' as reported by earlier reviewers.

Feel free to share your experiences in the comments, I will keep this post updated as the tools - and our experience with them - evolve.

Git Subproject Compile-time Dependencies in Sbt

Xebia Blog - Fri, 06/26/2015 - 13:33

When creating a sbt project recently, I tried to include a project with no releases. This means including it using libraryDependencies in the build.sbt does not work. An option is to clone the project and publish it locally, but this is tedious manual work that needs to be repeated every time the cloned project changes.

Most examples explain how to add a direct compile time dependency on a git repository to sbt, but they just show how to add a single project repository as a dependency using an RootProject. After some searching I found the solution to add projects from a multi-project repository. Instead of RootProject the ProjectRef should be used. This allows for a second argument to specify the subproject in the reposityr.

This is my current project/Build.scala file:

import sbt.{Build, Project, ProjectRef, uri}

object GotoBuild extends Build {
  lazy val root = Project("root", sbt.file(".")).dependsOn(staminaCore, staminaJson, ...)

  lazy val staminaCore = ProjectRef(uri("git://github.com/scalapenos/stamina.git#master"), "stamina-core")
  lazy val staminaJson = ProjectRef(uri("git://github.com/scalapenos/stamina.git#master"), "stamina-json")
  ...
}

These subprojects are now a compile time dependency and sbt will pull in and maintain the repository in ~/.sbt/0.13/staging/[sha]/stamina. So no manual checkout with local publish is needed. This is very handy when depending on an internal independent project/module and without needing to create a new release for every change. (One side note is that my IntelliJ currently does not recognize that the library is on the class/source path of the main project, so it complains it cannot find symbols and therefore cannot do proper syntax checking and auto completing.)

Scaling Agile: Scrum of Scrums, Membership Revisited

Pick a direction?

Pick a direction?

A Scum of Scrums (SoS) is a mechanism that has been adopted to coordinate a group of teams so that they act as a team of teams.  Historically the concept of a SoS was not part of the original Scrum framework and is considered an add-on to the framework.  In other words a Scrum of Scrums is an optional technique that can be added to more canonical Scrum framework if useful however because the technique is optional the amount of guidance is patchy.  For example, when an organization adopts a SoS who should participate is often hotly debated.  The participation debate popped up after we published Scrum of Scrums, The Basics. I had a number of conversations with readers to discuss the topic. Consolidating the discussions suggest the type of membership the person supports depends on what they want to get from using the SoS. All of the readers felt that the SoS should always be focused on coordination however there can be two flavors; one focused on activities and the second on technical questions.

Coordination of Activities:  Those that believe the SoS is primarily a tool for coordinating team activities support the idea that Scrum Masters should be chosen for SoS membership.  The rationale put forward focuses on the idea that the Scrum Master is uniquely positioned to gather and communicate information related to the coordination of teams. Readers in this group view feel that since Scrum Masters interact with all team members as a facilitator rather than a technical leader they are better coordinators. The alternate view held by many Agilistas believe that Scrum Masters acting in this role violate Scrum principles and represent a prima facie reinstitution of the role of the project manager. Further the Agilista view suggests that when technical issues need to be dealt with in SoS meetings populated with Scrum Masters they can’t make decisions rather often need to act as a conduit between technical team members. When SoS become a conduit, a version of the classic telephone game, decision making effectiveness is reduced..

Technical Coordination:  Fred Brooks in his essay Tar Pit (foreshadowing the next installment of Re-Read Saturday) suggests that software development increases in complexity as it progresses past development of an individual program. Integrating work with the work of other requires sharing technical information and making technical decisions that often can impact more than a single person or team. Scrum of scrums that are being used for technical coordination require participants with relevant technical acumen.  Participants with technical acumen generally come from the technical part of the team (developers, architects or testers).

Pushing aside the noise of whether the coordination of activities is less Agile than using the SoS for technical coordination, a more pragmatic approach is to recognize the needs of the team of teams is context driven.  The type of decision the team of team needs to make will change across a sprint or a SAFe Program Increment therefore who should attend a SoS needs to vary. The downside to varying membership is ensuring the right people are in attendance to address the SoS’s current need. One solution I have observed is to develop a cadence for topics.  For example tackle coordination every fourth SoS gathering with other more technical topics being focused on in-between.  Predictability makes it easy to plan who should attend.  Regardless of approach I suggest that any SoS should agree upon a mechanism to decide on the type of to hold. Flexibility to identify the type of SoS will ensure the team does not fall prey to meeting schedule paralysis or the equally evil telephone game.


Categories: Process Management

Scaling Agile: Scrum of Scrums – The Basics

Scaling - going from one project to two (or more!)

Scaling – going from one project to two (or more!)

The Scrum of Scrums (SoS) is a technique for scaling Scrum and other Agile techniques to larger efforts. On paper, the SoS is a simple extrapolation of the daily Scrum or stand-up meeting that has become a hallmark of the practice of Agile. A typical stand-up meeting occurs on a daily basis so that all of the team members of can coordinate and plan daily activities. Classically the team uses the three question technique to organize the meeting. In a similar manner, a typical Scrum of Scrums acts as a focal point to help synchronize a team of teams or even teams of teams. There are four basics of a SoS that need to be understood before any nuances can be addressed.

  1. Who Attends: There are two basic schools of thought in picking SoS attendees. The first (and most common) school suggests that the Scrum Master attend the SoS. The Scaled Agile Framework Enterprise (SAFe) is an example of a methodology that leverages the Scrum Master during both the planning event and during the program increment. Alternately, the second group takes a more egalitarian approach (possibly more pragmatic) allowing the each team to select an attendee that can most easily convey or understand the current issues the team and the larger group are having at the time. For example, if design decisions are at the front, perhaps team members with the most UX acumen would make sense. In this scenario attendees would vary over time.
  2. Who Facilitates: Small SoS groups, for example a handful of teams that are co-located, may not require facilitation. The SoS can self-organize and execute the meeting with little overhead. However as the number of participants increases (I bound SoS meetings using the same 5 -9 members rule used in Scrum teams) or as the distribution of the members becomes more varied facilitation becomes more important. Facilitators help the team use the SoS practice, ensure logistics are set up and champion the schedule. In larger efforts, a program manager often delivers facilitation, or if practicing SAFe the Release Train Engineer performs the facilitation role.
  3. Frequency: Scrum of Scrums often follows the same pattern as the daily Scrum/stand-up meeting. A second frequency pattern is risk-based; the frequency of the SoS meetings varies depending on need. Early in a project the SoS meets daily as teams are forming, norming and as early decisions are being made. The SoS meeting become twice weekly in the middle of the project and then returns to daily as a release approaches.
  4. Format: Daily stand-up meetings commonly leverage the classic three question approach (What did I do? What am I going to do? and What are my blockers?). The Scrum of Scrums generally follows a VERY similar format with each participant answering the following four questions:
    1. What did my team do since the last meeting?
    2. What will my team do before we meet again?
    3. Is my team experiencing blockers?
    4. Is my team going to put something in another team’s path?

The meeting follows the same round robin approach with each participant interacting with each other. The facilitator (if used) should never be the focal point of the meeting, nor should the SoS devolve into a purely a status meeting.

The daily stand-up and the SoS are very similar meetings. Both meetings are held to share information, coordinate activities and to identify problems. The scope of SoS meeting is broader than a single team with the meeting providing coordination and planning activities within and across teams.

Next: Using the Scrum of Scrums to Scale Planning and Scrum of Scrum Anti-patterns


Categories: Process Management

You Don’t Need a Complicated Story Hierarchy

Mike Cohn's Blog - Tue, 06/23/2015 - 15:00

Consultants and tool vendors seem to have a penchant for making things complicated. It seems the more complicated we make things, the more our clients need us. And that sells tools and services, I suppose.

On the other hand, I find unnecessary complexity extremely frustrating. It’s like the novel I read this week by a first-time author. It was good, but it had too many minor characters who complicated the plot and made the book hard to follow.

The same thing happens when people introduce complicated hierarchies or taxonomies for user stories like this:

You don’t need this. When teams are forced to use complicated taxonomies for their stories, they spend time worrying about whether a particular story is an epic, a saga or merely a headline. That discussion is like the minor character who walks into the novel and needlessly complicates the plot.

But, Mike -- I can hear you asking -- you’ve written about epics and themes before.

Yes, but those are labels. A story is a story so my recommended story taxonomy is this:

A story is a story is a story.

Some stories are big and they can be labeled as epics. I’ve used the analogy of movies before. All movies are movies but some movies are romantic comedies—that’s a label, just like epic is.

Similarly, theme refers to a group of related stories, but not does have to work within a hierarchy. Again using movies, I could have a group of spy movies that would include the James Bond movies and the Austin Powers movies. But a group of comedy movies would include Austin Powers but not James Bond.

So, again, theme and epic are labels not an implied hierarchy. Don’t make things more complicated than they need to be. I haven’t come across any reasons to have fancy story hierarchies or taxonomies.

Selecting an Iteration Approach - New Lecture Posted

10x Software Development - Steve McConnell - Tue, 06/23/2015 - 11:17

In this week's lecture (https://cxlearn.com) I explain how the lifecycle model can be used to show the incredibly large number of variations in approaches to software projects, especially including numerous variations in kinds of iteration. I identify approaches that work if you need predictability, if you need flexibility, if you need to attack uncertainty in requirements (i.e., unknown requirements), and if you need to attack uncertainty in architecture (i.e., technical risk).  

The overarching message is that there are lots of different ways to organize the activities on a software project, and the way you organize the activities significantly affects what a project will accomplish. 

Lectures posted so far include:  

0.0 Understanding Software Projects - Intro
     0.1 Introduction - My Background
     0.2 Reading the News
     0.3 Definitions and Notations 

1.0 The Software Lifecycle Model - Intro
     1.1 Variations in Iteration 
     1.2 Lifecycle Model - Defect Removal
     1.3 Lifecycle Model Applied to Common Methodologies
     1.4 Lifecycle Model - Selecting an Iteration Approach 

2.0 Software Size
     2.05 Size - Comments on Lines of Code
     2.1 Size - Staff Sizes 
     2.2 Size - Schedule Basics 

Check out the lectures at http://cxlearn.com!

Understanding Software Projects - Steve McConnell

 

Variations in Iteration - New Lecture Posted(1)

10x Software Development - Steve McConnell - Tue, 06/23/2015 - 11:17

In this week's lecture (https://cxlearn.com) I explain how the lifecycle model can be used to show the incredibly large number of variations in approaches to software projects, especially including numerous variations in kinds of iteration. I identify approaches that work if you need predictability, if you need flexibility, if you need to attack uncertainty in requirements (i.e., unknown requirements), and if you need to attack uncertainty in architecture (i.e., technical risk).  

The overarching message is that there are lots of different ways to organize the activities on a software project, and the way you organize the activities significantly affects what a project will accomplish. 

Lectures posted so far include:  

0.0 Understanding Software Projects - Intro
     0.1 Introduction - My Background
     0.2 Reading the News
     0.3 Definitions and Notations 

1.0 The Software Lifecycle Model - Intro
     1.1 Variations in Iteration 
     1.2 Lifecycle Model - Defect Removal
     1.3 Lifecycle Model Applied to Common Methodologies
     1.4 Lifecycle Model - Selecting an Iteration Approach 

2.0 Software Size
     2.05 Size - Comments on Lines of Code
     2.1 Size - Staff Sizes 
     2.2 Size - Schedule Basics 

Check out the lectures at http://cxlearn.com!

Understanding Software Projects - Steve McConnell

 

Software Gardening, Entropy, Hybrid Requirements, Lean UX in Methods & Tools Summer 2015 issue

From the Editor of Methods & Tools - Mon, 06/22/2015 - 14:29
Methods & Tools – the free e-magazine for software developers, testers and project managers – has just published its Summer 2015 issue that discusses  Software Gardening, Software Entropy, Hybrid Requirements, Lean UX and agileMantis. * Software Gardening – Yet another crappy analogy or a reality? * Entropy for Measuring Software Maturity * The READ, RATT, […]

SPaMCAST 347 – Agile Project Management, Conway’s Law and Microservices, Hardcore Testing

Software Process and Measurement Cast - Sun, 06/21/2015 - 22:00

The Software Process and Measurement Cast includes three columns.  The first is our essay on managing Agile projects and teams. I often say project management is dead. That does not mean that the pressures that drive the need to manage work have gone away. In the end the “what” of project management is important because control, discipline and coordination are needed tools to deliver value; the journey toward Agile is the reframing of the “how” of project management.

This week Gene Hughson returns with an entry from his Form Follows Function column.  Gene tackles the topic of whether the application of Conway’s Law makes microservices more of an organizational approach than an architecture.   After listening, check out Gene’s Form Follows Function blog!

The third column in this SPaMCAST magazine is from the Software Sensei, Kim Pries.  Kim tackles hardcore testing.  Kim discusses the implications and uses of this aggressive type of testing in hardware, software and wetware.

A great line up!

Call to action!

Reviews of the Podcast help to attract new listeners.  Can you write a review of the Software Process and Measurement Cast and post it on the podcatcher of your choice?  Whether you listen on ITunes or any other podcatcher, a review will help to grow the podcast!  Thank you in advance!

Re-Read Saturday News

We just completed the Re-Read Saturday of Eliyahu M. Goldratt and Jeff Cox’s The Goal: A Process of Ongoing Improvement which began on February 21nd. What did you think?  Did the re-read cause you to pick  The Goal back up for a refresher? Visit the Software Process and Measurement Blog and review the whole re-read.

Note: If you don’t have a copy of the book, buy one.  If you use the link below it will support the Software Process and Measurement blog and podcast.

Dead Tree Version or Kindle Version 

Next week we will begin re-reading The Mythical Man-Month. Get a copy now and start reading!

Upcoming Events

Software Quality and Test Management 

September 13 – 18, 2015

San Diego, California

http://qualitymanagementconference.com/

I will be speaking on the impact of cognitive biases on teams!  Let me know if you are attending!

 

More on other great conferences next week.

 

Next SPaMCast

The next Software Process and Measurement Cast will feature our interview with Woody Zuill.  Some people might think “that there is no Woody only Zuul” (apologies to the Ghostbusters) when it comes to topics like #NoEstimates.  However as Woody points out, it is important to peer past the “thou musts” to gain greater understanding of what you should be doing!

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.

Categories: Process Management

SPaMCAST 347 – Agile Project Management, Conway’s Law and Microservices, Hardcore Testing

 www.spamcast.net

http://www.spamcast.net

Listen Now

Subscribe on iTunes

The Software Process and Measurement Cast includes three columns.  The first is our essay on managing Agile projects and teams. I often say project management is dead. That does not mean that the pressures that drive the need to manage work have gone away. In the end the “what” of project management is important because control, discipline and coordination are needed tools to deliver value; the journey toward Agile is the reframing of the “how” of project management.

This week Gene Hughson returns with an entry from his Form Follows Function column.  Gene tackles the topic of whether the application of Conway’s Law makes microservices more of an organizational approach than an architecture.   After listening, check out Gene’s Form Follows Function blog!

The third column in this SPaMCAST magazine is from the Software Sensei, Kim Pries.  Kim tackles hardcore testing.  Kim discusses the implications and uses of this aggressive type of testing in hardware, software and wetware.

A great line up!

Call to action!

Reviews of the Podcast help to attract new listeners.  Can you write a review of the Software Process and Measurement Cast and post it on the podcatcher of your choice?  Whether you listen on ITunes or any other podcatcher, a review will help to grow the podcast!  Thank you in advance!

Re-Read Saturday News

We just completed the Re-Read Saturday of Eliyahu M. Goldratt and Jeff Cox’s The Goal: A Process of Ongoing Improvement which began on February 21nd. What did you think?  Did the re-read cause you to pick  The Goal back up for a refresher? Visit the Software Process and Measurement Blog and review the whole re-read.

Note: If you don’t have a copy of the book, buy one.  If you use the link below it will support the Software Process and Measurement blog and podcast.

Dead Tree Version or Kindle Version 

Next week we will begin re-reading The Mythical Man-Month. Get a copy now and start reading!

Upcoming Events

Software Quality and Test Management 

September 13 – 18, 2015

San Diego, California

http://qualitymanagementconference.com/

I will be speaking on the impact of cognitive biases on teams!  Let me know if you are attending!

 

More on other great conferences next week.

 

Next SPaMCast

The next Software Process and Measurement Cast will feature our interview with Woody Zuill.  Some people might think “that there is no Woody only Zuul” (apologies to the Ghostbusters) when it comes to topics like #NoEstimates.  However as Woody points out, it is important to peer past the “thou musts” to gain greater understanding of what you should be doing!

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.


Categories: Process Management