Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

SPaMCAST 317 – Questions, Answers and Controversy, Robust Software

Software Process and Measurement Cast - Sun, 11/23/2014 - 23:00

SPaMCAST 317 tackles a wide range of frequently asked questions, ranging from the possibility of an acceleration trap, the relevance of function points, whether teams have a peak loads and safe to fail experiments. Questions, answers and controversy!

We will also have the next installment of Kim Pries’s column, The Software Sensei! This week Kim discusses robust software.

The essay starts with “Agile Can Contribute to an Acceleration Trap”

I am often asked whether Agile techniques contribute to an acceleration trap in IT.  In an article in The Harvard Business Review, Bruch and Menges (April 2010) define an acceleration trap as the malaise that sets in as an organization fails prey to chronic overloading. It can be interpreted as laziness or recalcitrance, which then elicits even more pressure to perform, generating an even deeper malaise. The results of the pressure/malaise cycle are generally a poor working atmosphere and employee loss. Agile can contribute to an acceleration trap but only as a reflection of poor practices. Agile is often perceived to induce an acceleration trap in two manners: organizational change and delivery cadence.

Listen to the rest now

Call to action!

We are in the middle of a re-read of John Kotter’s classic Leading Change of on the Software Process and Measurement Blog.  Are you participating in the re-read? Please feel free to jump in and add your thoughts and comments!

After we finish the current re-read will need to decide which book will be next.  We are building a list of the books that have had the most influence on readers of the blog and listeners to the podcast.  Can you answer the question?

What are the two books that have most influenced you career (business, technical or philosophical)?  Send the titles to

First, we will compile a list and publish it on the blog.  Second, we will use the list to drive future  “Re-read” Saturdays. Re-read Saturday is an exciting new feature that began on the Software Process and Measurement blog on November 8th.  Feel free to choose you platform; send an email, leave a message on the blog, Facebook or just tweet the list (use hashtag #SPaMCAST)!


SPaMCAST 318 features our interview with Rob Cross.  Rob and I discussed his INFOQ article “How to Incorporate Data Analytics into Your Software Process.”  Rob provides ideas on how the theory of big data can be incorporated in to big action.


Upcoming Events

DCG Webinars:

Agile Risk Management - It Is Still Important
Date: December 18th, 2014
Time: 11:30am EST

Register Now

The Software Process and Measurement Cast has a sponsor.

As many you know I do at least one webinar for the IT Metrics and Productivity Institute (ITMPI) every year. The ITMPI provides a great service to the IT profession. ITMPI’s mission is to pull together the expertise and educational efforts of the world’s leading IT thought leaders and to create a single online destination where IT practitioners and executives can meet all of their educational and professional development needs. The ITMPI offers a premium membership that gives members unlimited free access to 400 PDU accredited webinar recordings, and waives the PDU processing fees on all live and recorded webinars. The Software Process and Measurement Cast some support if you sign up here. All the revenue our sponsorship generates goes for bandwidth, hosting and new cool equipment to create more and better content for you. Support the SPaMCAST and learn from the ITMPI.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.

Categories: Process Management

Software Estimating for Non Trivial Projects

Herding Cats - Glen Alleman - Sun, 11/23/2014 - 16:26

When we read on a blog post that estimates are not meaningful unless you are doing very trivial work, † I wonder if the poster has worked on any non-trivial software domain. Places like GPS OCX, SAP consolidation, Manned Space Flight Avionics, or maybe Health Insurance Provider Networks. Because without some hands on experience in those non-trivial domains, it's be hard to actually knowing what you're talking about when it comes to estimating the spend of other peoples money.

Maybe some background on estimates for nontrivial work will shed light on this ill informed notion that only trivial projects can be estimated.

These are a small sample of papers from one journal on software estimating for misison critical, some times National Asset projects. 

Go to Cross Talk, The Journal of Defense Software Engineering, and search for "estimating" to get 10 pages of 10 articles on this topic alone. This notion of estimating in non-trivial domains is well developed, well documented, and many examples of tools, processes, and principles. 

If Do Your Homework and the Test is much easier.

It could be that the original poster has little experience in mission critical, national asset, enterprise class, software intensive systems. Or it could be the poster simply doesn't know what making estimates for project that spends other peoples money, many times significant amounts of money, is all about.

And of course most of the problems describes as the basis for Not Estimating - the illogical notion that if we can't do something well, let's stop doing it - starts with not knowing what Done Looks Like in any units of measure meaningful to the decision makers. 

So start here with my favorite enterprise architect blog amd his list of books when you follow the link to the bottom.

Screen Shot 2014-11-23 at 7.06.35 AM

So when you have some sense of what DONE looks like in terms of capabilities, the estimating process is now on solid ground. From that solid ground you can ask have we done any like this before? Or better yet can we f ind someone who has done something like this before? Or maybe can we look around to see what looks like our problem and figure out how long it took them by simply asking them? I

If the answer to any of those questions is NO and you're NOT working in a research and development domain, then don't start the project because you're not qualified to do the work, you don't know what you're doing and you're going to waste your customers money.


† Scroll to the bottom of and search for "A Thing I Can Estimate," to see the phrase, and remember the questions and the answers above. If you're not answering those in some positive way, you're now on a death march project starting day one, because you don't know what done looks like for the needed capabilities. Not the requirements, not the code, not the testing - that's all straight forward. Without some notion of what the system is supposed to do, you're never recognize it if it were ever to come into view. And since the customer doesn't know as well, all the money they're spending to find out has to be written off as IRAD or flushed down the toliet as a waste of time and effort in the end. And then you'll know why  Standish (improperly) reports projects fail. 

Screen Shot 2014-11-23 at 7.38.19 AM

Related articles Basis of #NoEstimates are 27 Year Old Reports Estimating Guidance Should I Be Estimating My Work? Assessing Value Produced By Investments Trust but Verify How to Estimate Software Development
Categories: Project Management

R: dplyr – “Variables not shown”

Mark Needham - Sun, 11/23/2014 - 02:02

I recently ran into a problem where the result of applying some operations to a data frame wasn’t being output the way I wanted.

I started with this data frame:

words = function(numberOfWords, lengthOfWord) {
  w = c(1:numberOfWords)  
  for(i in 1:numberOfWords) {
    w[i] = paste(sample(letters, lengthOfWord, replace=TRUE), collapse = "")
numberOfRows = 100
df = data.frame(a = sample (1:numberOfRows, 10, replace = TRUE),
                b = sample (1:numberOfRows, 10, replace = TRUE),
                name = words(numberOfRows, 10))

I wanted to group the data frame by a and b and output a comma separated list of the associated names. I started with this:

> df %>% 
    group_by(a,b) %>%
    summarise(n = n(), words = paste(name, collapse = ",")) %>%
    arrange(desc(n)) %>%
Source: local data frame [5 x 4]
Groups: a
   a  b  n
1 19 90 10
2 24 36 10
3 29 20 10
4 29 80 10
5 62 54 10
Variables not shown: words (chr)

Unfortunately the words column has been excluded and I came across this Stack Overflow post which suggested that the print.tbl_df function was the one responsible for filtering columns.

Browsing the docs I found a couple of ways to overwrite this behaviour:

> df %>% 
    group_by(a,b) %>%
    summarise(n = n(), words = paste(name, collapse = ",")) %>%
    arrange(desc(n)) %>%
    head(5) %>%
    print(width = Inf)


> options(dplyr.width = Inf)
> df %>% 
    group_by(a,b) %>%
    summarise(n = n(), words = paste(name, collapse = ",")) %>%
    arrange(desc(n)) %>%

And now we see this output instead:

Source: local data frame [5 x 4]
Groups: a
   a  b  n                                                                                                         words
1 19 90 10 dfhtcgymxt,zpemxbpnri,rfmkksuavp,jxaarxzdzd,peydpxjizc,trdzchaxiy,arthnxbaeg,kjbpdvvghm,kpvsddlsua,xmysfcynxw
2 24 36 10 wtokzdfecx,eprsvpsdcp,kzgxtwnqli,jbyuicevrn,klriuenjzu,qzgtmkljoy,bonbhmqfaz,uauoybprrl,rzummfbkbx,icyeorwzxl
3 29 20 10 ebubytlosp,vtligdgvqw,ejlqonhuit,jwidjvtark,kmdzcalblg,qzrlewxcsr,eckfgjnkys,vfdaeqbfqi,rumblliqmn,fvezcdfiaz
4 29 80 10 wputpwgayx,lpawiyhzuh,ufykwguynu,nyqnwjallh,abaxicpixl,uirudflazn,wyynsikwcl,usescualww,bkvsowfaab,gfhyifzepx
5 62 54 10 beuegfzssp,gfmegjtrys,wkubhvnkkk,rkhgprxttb,cwsrzulnpo,hzkvjbiywc,gbmiupnlbw,gffovxwtok,uxadfrjvdn,aojjfhxygs

Much better!

Categories: Programming

Re-read Saturday: Establish a Sense of Urgency, Leading Change, John P. Kotter Chapter Three

Some times you need help urgently.

Some times you need help urgently!

The first two chapters of John P. Kotter’s Leading Change established why change in organizations fails and the forces that shape successful changes in organizations and ultimately introduces the eight-stage process for creating major change.  Chapter three dives into the model by exploring the first phase.  A sense of urgency provides the source of energy that can drive change.  Without a sense of urgency that is shared by the majority of stakeholders, the level of effort required to deliver change will overwhelm any progress.  Period!

Simply put, change is difficult. In all but the newest organizations the status quo has an enormous amount of inertia.  Organizations have invested time and effort into building planning and control processes that lock in strategies and techniques that have been successful.  Breaking that inertia requires an application of energy to disrupt the process equilibrium most organizations develop.  For an example of inertia and established equilibriums, fill a bath tub with water, open the drain and then cover the drain hole with a plug (or your hand).  An equilibrium quickly forms between the water pushing down and the vacuum of the drain pipe.  If the pull of the vacuum or water pressure increased too much the system would fail and you would have to buy new plumbing. In order to break the seal and let the water out of the tub energy is needed to break the seal of the plug.  A sense of urgency is the energy that can be applied to break the seal and propel the change processes.

Complacency is the force that opposes urgency.  Complacency is a feeling of uncritical self-satisfaction with yourself or your organization’s achievements. Complacency is caused by numerous scenarios and activities including lack of a visible crisis, past success, trappings of success to “happy talk.”  Early in my career I worked for a major women’s garment manufacturer.  The firm was the largest and most successful junior clothing manufacturer.  The year after I joined the firm we achieved our highest sales year. Enter MTV and the ascendancy of boutique mall retailers (this was also the beginning of the significant consolidation of the department store market).  Our market research and sale leaders focused on talking primarily to accounts that were successful in selling our merchandise.  This was akin to having conversations in a fishbowl (closed, self-reinforcing environment). Other signs of behaviors and environmental conditions reinforced the feeling of complacency that kept the organization from generating a sense of urgency.  The first is was slow erosion of sales (no visible immediate crisis) and lots of trappings of success, such as holding sales meetings in luxury resorts even while sales plummeted. Complacency has to be broken for a sense of urgency to take hold.

Bold moves are required to disrupt and confront complacency. For example, a few years ago I was sitting in on a staff meeting of an IT organization that was actively being shopped to outsourcers. It was common knowledge that the “business” felt that the organization was unproductive and cost too much. The first thirty minutes of the meeting was a self-congratulatory presentation on the organization’s past successes.  When I discussed it afterwards with the CIO, his defense of the presentation was that he was trying to generate higher moral.  Side note: the IT organization did not measure productivity or cost per unit, therefore they had little understanding of whether they were efficiently spending the business’ money. Later that year I was told the IT organization had been outsourced and that out of approximately 5k people in the department, only 35 still had jobs.  Focusing on past successes and failing to transparently discuss hard truths allow complacency to throttle a sense of urgency.

Significant changes require significant effort. A sense of urgency which defines the need to change generates the energy needed to apply the effort and resources needed for change. Without a sense of urgency it will be difficult to generate buy-in amongst the management team in an organization.  Establishing a sense of urgency is the first stage of eight-stage process for creating major change.  It is an absolute requirement for a change of any significance, but it is not sufficient to complete the cycle of change.

Categories: Process Management

Show Me Your Math

Herding Cats - Glen Alleman - Sat, 11/22/2014 - 21:02

IScreen Shot 2014-11-21 at 12.32.21 PMn a recent email exchange, the paper by Todd Little showing projects that exceeded their estimates was used as an example for how porrly we estimate, and ultimately one of the reasons to adopt the #NoEstimates paradigm of making decisions in the absence of estimates of cost, schedule, and the probability that the needed capabilities will show up on time and be what the customer wanted.

Sherlock here had it right. This picture by the way  is borrowed from Mike Cohn's eBook of 101 quotes for agile.

I've written about Little's paper before, but it's worth repeating.

It's very sporty to use examples of bad mathematics, bad management, bad processes, bad practices as the basis for something new. This is essential the basis the book Hard Facts, dangerous Half-Truths: Profiting from Evidence-Based Management. When we start talking about something new and disruptive in the absence of the data, facts, root causes, and underlying governance and principles, we're treading on very thin ground in terms of credibility.

Here's the core principle of all software development

Customers exchange money for value. The definition of  value needs to be in units of measure that allows them to make decisions about the future value of that value. That value is exchanged for a cost. A future cost as well. 

Both this future cost and future value are probabilistic in nature, due to the uncertainties in the work processes, technologies, markets, productivity and all the ...ilities associated with project work. In the presence of uncertainty, nothing is for certain - a tautology. There are two type of uncertainty - reducible and irreducible. Reducibe we can do something about. We can spend money to reduce the risk associated with the uncertainty. Irreducible, we can't. We can only have marginmanagement reserve, or a Plan B.

To make decisions in the presence of these uncertainties - reducible and irreducible - we need to estimate the uncertainty, the cost of handling the uncertainty, and the value produced by the work driven by these uncertainties. When we fail to make these estimates, the uncertainties don't go away. When we slice the work into small chunks, we might also slice the uncertainties into small chunks - this is the basis of agile and the paradigm of Little Bets. But the uncertainties are still there, unless we've explicitly bought them down or installed margin and reserve. They didn't go away. And what you don't know - or choose to explicitly ignore - can hurt you.

So Sherlock is right don't put forth a theory without the data. 


Related articles Should I Be Estimating My Work? Basis of #NoEstimates are 27 Year Old Reports Estimating Guidance Baloney Claims: Pseudo - science and the Art of Software Methods Assessing Value Produced By Investments Trust but Verify Anecdotal Evidence is not Actually Evidence
Categories: Project Management

Models, Models Everywhere


CMMI, ITIL, Six Sigma, Agile, waterfall, software development life cycle and eXtreme Programming . . .what do all of these terms have in common?  They are models.  In a perfect world, models are abstractions that we find useful to explain the world around us.  Models work by rendering complex ideas more simply.  For example, both a road map and picture rendered in Google Earth are models.  Two very different types of models: an abstraction of a set roads, buildings, parks and plants that exhibit can provide more information than rendering.  Real life is complex, Google Earth is less complex and the road maps are the least complex.  Simplifying a concept to a point allows understanding, while too much simplification renders the concept as a pale reflection.  Oversimplification can lead to misunderstandings or misconceptions, for example the conception that Agile methods are undisciplined or that waterfall methods are bureaucratic.  Both of these are misconceptions (individual implementations are another story).  According to Kenji Haranabe, software development is a form of communication game.  Communication requires that groups understand a concept so that it can be implemented.  Communication and understanding requires finding a level where common understanding based on common words can occur.  Words provide the simplification of real life into a form of model.

Unfortunately it is difficult to determine when enough simplification is enough.  Oversimplification of ideas can allow trouble to creep in.  Oversimplification can water down a concept to the point that it can not provide useful information to be used operationally.  An example of a very simple model is the five maturity levels commonly connected to the CMMI.  The maturity levels build awareness, but provide little operational information.  I do not know how many times I have heard people talk about an individual maturity level as if the name given to that level was all you needed to know about a maturity level.   The less simplified version with process areas, goals and practices provides substantial operational information.  ‘Operationalizing’ an overly simplified model will yield unanticipated results and that is not what we want from a model.  I once built a model of the battleship Missouri that had horrible directions (directions are a model of a model), I used three firecrackers to remodel the thing I ended up with (which was not a very good model).

Models abound in the world of information technology.  If we don’t have a model for it, we at least have TLA (three letter acronym) for it and are working on a model that will incorporate it.  The models that have lasting power provide structure, discipline and control.  They’re also used as a tool to guide work (tightly or loosely depends on the organization) and as a scaffold to define improvements in a structured manner.  Models are powerful; molding, bending and guiding legions of IT practitioners.  The dark side of this power is that the choice of models can be definitional statement for a group or organization.  Selecting a model can elicit all of the passions of politics or religion.  Just consider the emotions you feel when describing Six Sigma, CMMI, eXtreme Programming, waterfall or Agile.  One of those words will probably be a hot button.  The power of models can become seductive and entrenched so as to reduce your ability to change and adapt as circumstances demand.  A model is never a goal!  Define what your expectations are for the model or models that you are you using in business terms.  Examples of goals I would expect are of increased customer satisfaction, improved quality or faster time-to-market, rather than attaining CMMI Maturity Level 2 or implementing daily builds.  Know what you want to accomplish then integrate the models and tactics to achieve those goals.  Do not let the tool be the goal.

Models are powerful, useful tools to ensure understanding, they provide structure and discipline.  Perform a health check.  Questions to ask about the models in your organization begin with:

  1. Is there is a deep enough understanding of the model being used? – With knowledge comes the background to operationalize the model.
  2. What are your expectations of value from the model? – Knowing what you want from a model helps ensure that the model does not become the goal and that you retain the ability to be flexible.

There are many other questions you can ask on your heath check, however if you can’t answer these questions well stop and reassess, re-evaluate, re-train and re-plan your effort.

CMMI, ITIL, Six Sigma, Agile, waterfall, software development life cycle and eXtreme Programming. . . powerful tools or a straight jacket. Which is it for you?

Categories: Process Management

Want to preview our new DataGrid for Xamarin.Forms?

Eric.Weblog() - Eric Sink - Fri, 11/21/2014 - 19:00

Zumero.DataGrid is a Xamarin.Forms control for displaying data in rows and columns.

If you would be interested in testing and previewing a trial version of this product, email us:


  • We plan to submit this to the Xamarin Component Store where (pending Xamarin's approval) it would be a commercial product. Pricing has not been set.

  • You'll receive a file in Xamarin Component format (a .xam file) which you will need to install into your IDE.
    (command line: xamarin-component.exe install ZumeroDataGrid-1.0.0.xam)
    The component file contains the libraries for all platforms, plus documentation and samples.

  • The test component is a trial version. It overlays the contents of some (randomly chosen) grid cells with an orange label that says "Trial". It is otherwise identical to the full version.

  • This component is unrelated to the one I mentioned in a blog entry back on 3 October. That approach was based on drawing. It was not capable of hosting Xamarin.Forms views, so it couldn't do things like in-place editing. This component is entirely different.

  • We are eager to hear feedback of any kind.

  • Support for cell contents to be any Xamarin.Forms.View
  • Scrolling, both horizontal and vertical
  • Optional top frozen header row
  • Optional left frozen column
  • Separate View template for each column
  • View recycling (If you have 1000 rows but only 10 can be visible, DataGrid creates 10 Views and reuses them.)
  • Data row objects (provided to the DataGrid as an IList<object>)
  • Strong integration with the Xamarin.Forms binding mechanism
  • Automatic updates if an ObservableCollection is used for rows and columns
  • Configurable row height (for all rows in the grid)
  • Configurable column widths (can be different for each column)
  • Configurable spacing between rows and columns
  • Works with C#, F#, and XAML
  • Support for iOS, Android, and Windows Phone
  • Full API documentation in MonoDoc
  • Technical support service from Zumero


Stuff The Internet Says On Scalability For November 21st, 2014

Hey, it's HighScalability time:

Sweet dreams brave Philae. May you awaken to a bright-throned dawn for one last challenge.


  •  80 million: bacteria xferred in a juicy kiss;
  • Quotable Quotes:
    • James Hamilton: Every day, AWS adds enough new server capacity to support all of Amazon's global infrastrucrture when it was a $7B annual revenue enterprise.
    • @iglazer: What is the test that could most destroy your business model?  Test for that. @adrianco #defragcon
    • @zhilvis: Prefer decoupling over duplication. Coupling will kill you before duplication does by @ICooper #buildstufflt
    • @jmbroad: "Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful." ~George Box
    • @RichardWarburto: Optimisation maybe premature but measurement isn't.
    • @joeerl: Hell hath no version numbers - the great ones saw no need for version numbers - they used port numbers instead. See, for example, RFC 821,
    • JustCallMeBen: tldr: queues only help to flatten out burst load. Make sure your maintained throughput is high enough.
    • @rolandkuhn: «the event log is a database of the past, not just of the present» — @jboner at #reactconf
    • @ChiefScientist: CRUD is dead. -- @jboner #reactconf 
    • @fdmts: 30T of flash disks cabled up and online.  Thanks @scalableinfo!
    • monocasa: Immutable, statically linked, minimal system images running micro services on top of a hypervisor is a very old concept too. This is basically the direction IBM went in the 60's with their hypervisors and they haven't looked back.
    • Kiril Savino: Scaling is the process of decoupling load from latency.

  • Perhaps they were controlled by a master AI? Google and Stanford Built Similar Neural Networks Without Knowing It: Neural networks can be plugged into one another in a very natural way. So we simply take a convolutional neural network, which understands the content of images, and then we take a recurrent neural network, which is very good at processing language, and we plug one into the other. They speak to each other—they can take an image and describe it in a sentence.

  • You know how you never really believed the view in MVC was ever really separate? Now this is MVC. WatchKit apps run on the iPhone as an extension, only the UI component runs on the watch. XWindows would be so proud.

  • Shopify shows how they Build an Internal Cloud with Docker and CoreOS: Shopify is a large Ruby on Rails application that has undergone massive scaling in recent years. Our production servers are able to scale to over 8,000 requests per second by spreading the load across 1700 cores and 6 TB RAM.

  • Machine learning isn't just about creating humavoire AIs. It's a technology, like electricity, that will transform everything it affixes with its cyclops gaze. Here's a seemingly mundane example from Google, as discussed on the Green (Low Carbon) Data Center Blog. Google has turned inward, applying Machine Learning to its data center fleet. The result:  Google achieved from 8% to 25% reduction in its energy used to cool the data center with an average of 15%.  Who wouldn’t be excited to save an average of 15% on their cooling energy costs by providing new settings to run the mechanical plant? < And this is how the world will keep those productivity increases reaching skyward.

  • Does anyone say "I love my water service"? Or "I love my garbage service"? Then why would anyone say "I love Facebook"? That's when you've arrived. When you are so much a part of the way things are that people don't even think of loving them or not. They just are. The Fall of Facebook

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Make Stories Small When You Have "Wicked" Problems

If you read my Three Alternatives to Making Smaller Stories, you noticed one thing. In each of these examples, the problem was in the teams’ ability to show progress and create interim steps. But, what about when you have a “wicked” problem, when you don’t know if you can create the answer?

If you are a project manager, you might be familiar with the idea of “wicked” problems from   from the book Wicked Problems, Righteous Solutions: A Catalog of Modern Engineering Paradigms. If you are a designer/architect/developer, you might be familiar with the term from Rebecca Wirfs-Brock’s book, Object Design: Roles, Responsibilities, and Collaborations.

You see problems like this in new product development, in research, and in design engineering. You see it when you have to do exploratory design, where no one has ever done something like this before.

Your problem requires innovation. Maybe your problem requires discussion with your customer or your fellow designers. You need consensus on what is a proper design.

When I taught agile to a group of analog chip designers, they created landing zones, where they kept making tradeoffs to fit the timebox they had for the entire project, to make sure they made the best possible design in the time they had available.

If you have a wicked problem, you have plenty of risks. What do you do with a risky project?

  1. Staff the project with the best people you can find. In the past, I have used a particular kind of “generalizing specialist,” the kind where the testers wrote code. The kind of developers who were also architects. These are not people you pick off the street. These are people who are—excuse me—awesome at their jobs. They are not interchangeable with other people. They have significant domain expertise in how to solve the problem. That means they understand how to write code and test.
  2. Help those generalizing specialists learn how to ask questions at frequent points in the project. In my inch-pebble article, I said that with a research project, you use questions to discover what you need to know. The key is to make those questions small enough, so you can show progress every few days or at least once week. Everyone in the project needs to build trust. You build trust by delivering. The project team builds trust by delivering answers, even if they don’t deliver working code.
  3. You always plan to replan. The question is how often? I like replanning often. If you subscribed to my Reflections newsletter (before the Pragmatic Manager), back in 1999, I wrote an article about wicked projects and how to manage the risks.
  4. Help the managers stop micromanaging. The job of a project manager is to remove obstacles for the team. The job of a manager is to support the team. Either of those manager-types might help the team by helping them generate new questions to ask each week. Neither has the job of asking “when will you be done with this?” See Adam Yuret’s article The Self-Abuse of Sprint Commitment.

Now, in return, the team solving this wicked problem owes the organization an update every week, or, at the most, every two weeks about what they are doing. That update needs to be a demo. If it’s not a demo, they need to show something. If they can’t in an agile project, I would want to know why.

Sometimes, they can’t show a demo. Why? Because they encountered a Big Hairy Problem.

Here’s an example. I suffer from vertigo due to loss of (at least) one semi-circular canal in my inner ear. My otoneurologist is one of the top guys in the world. He’s working on an implantable gyroscope. When I started seeing him four years ago, he said the device would be available in “five more years.”

Every year he said that. Finally, I couldn’t take it anymore. Two years ago, I said, “I’m a project manager. If you really want to make progress, start asking questions each week, not each year. You won’t like the fact that it will make your project look like it’s taking longer, but you’ll make more progress.” He admitted last year that he took my advice. He thinks they are down to four years and they are making more rapid progress.

I understand if a team learns that they don’t receive the answers they expect during a given week. What I want to see from a given week is some form of a deliverable: a demo, answers to a question or set of questions, or the fact that we learned something and we have generated more questions. If I, as a project manager/program manager, don’t see one of those three outcomes, I wonder if the team is running open loop.

I’m fine with any one of those three outcomes. They provide me value. We can decide what to do with any of those three outcomes. The team still has my trust. I can provide information to management, because we are still either delivering or learning. Either of those outcomes provides value. (Do you see how a demo, answers or more questions provides those outcomes? Sometimes, you even get production-quality code.)

Why do questions work? The questions work like tests. They help you see where you need to go. Because you, my readers, work in software, you can use code and tests to explore much more rapidly than my otoneurologist can. He has to develop a prototype, test in the lab and then work with animals, which makes everything take longer.

Even if you have hardware or mechanical devices or firmware, I bet you simulate first. You can ask the questions you need answers to each week. Then, you answer those questions.

Here are some projects I’ve worked on in the past like this:

  • Coding the inner loop of an FFT in microcode. I knew how to write the inner loop. I didn’t know if the other instructions I was also writing would make the inner loop faster or slower. (This was in 1979 or so.)
  • Lighting a printed circuit board for a machine vision inspection application. We had no idea how long it would take to find the right lighting. We had no idea what algorithm we would need. The lighting and algorithm were interdependent. (This was in 1984.)
  • With clients, I’ve coached teams working on firmware for a variety of applications. We knew the footprint the teams had to achieve and the dates that the organizations wanted to release. The teams had no idea if they were trying to push past the laws of physics. I helped the team generate questions each week to direct their efforts and see if they were stuck or making progress.
  • I used the same approach when I coached an enterprise architect for a very large IT organization. He represented a multi-thousand IT organization who wanted to revamp their entire architecture. I certainly don’t know architecture. I know how to make projects succeed and that’s what he needed. He used the questions to drive the projects.

The questions are like your tests. You take a scientific approach, asking yourself, “What questions do I need to answer this week?” You have a big question. You break that question down into smaller questions, one or two that you can answer (you hope) this week. You explore like crazy, using the people who can help you explore.

Exploratory design is tricky. You can make it agile, also. Don’t assume that the rest of your project can wait for your big breakthrough.  Use questions like your tests. Make progress every day.

I thank Rebecca Wirfs-Brock for her review of this post. Any remaining mistakes are mine.

Categories: Project Management

Scaling Agile – SAFe Program Increment: Inspect and Adapt


Most Agile frameworks are built on the basis empirical models of process control. These, in turn, are based on periodic observation and adaptation. Each instantiation of the process may be different due to the circumstances any specific team faces. The empirical mode is often compared to the defined process approach, which posits a standard process for all teams that evolves based on the needs of the collective whole rather than a team or small group of teams. Scaled Agile Framework Enterprise (SAFe) program increments are no different. Program increments are similar to a large Scrum sprint beginning with planning and completing with a demonstration and retrospective, known as inspect and adapt. The empirical process of inspect and adapt is one of the core principles in SAFe. At the team level, each sprint culminates with the classic demonstration and retrospective. The same type of discipline is leveraged for program increments. PI level inspect and adapt is composed of three components:

  1. The teams involved in the program increment demonstrate the system with the new features, architectural changes and non-functional requirements that have been completed during the PI. The goal of the demonstration is to evoke feedback based on the overall current status of the overall application from the Agile release train (ART) stakeholders.
  2. Metrics Review. Program metrics are reviewed and discussed as a team. The goal of the metrics review and the ensuing discussion is to ensure that everyone involved understands performance based on tactical experience and measured performance. The combination of experience and measurement helps the team see through their individual cognitive biases.
  3. Retrospective/Problem Solving Workshop. The problem solving workshop provides a platform for the PI participants to first identify the issues experienced during the increment, then to identify which of those that they would like to solve and finally to develop a corrective action plan. SAFe recommends that a structured, root cause analysis is used to analyze the issue and define a corrective action plan (the preferred method uses the “five whys” and Ishikawa Diagrams/Fishbone diagrams – both of these are tried and true techniques used in the quality and process improvement arena). The goal of the process is to generate tangible improvement actions that can be taken into the release planning event for the next program increment.

The inspect and adapt process in SAFe is a means of generating feedback on the product that is being developed and the processes being used to do the work. Feedback is a means of ensuring the Agile release train stays on track. Inspect and Adapt provide feedback so that changes can be made the trajectory of the product being build and to make changes in how work is being done.

Categories: Process Management

Episode 215: Gang of Four – 20 Years Later

Johannes Thönes talks with Erich Gamma, Ralph Johnson and Richard Helm from the Gang of Four about the 20th anniversary of their book Design Patterns. They discuss the following topics: the definition of a design pattern and each guest’s favorite design pattern; the origins of the book in architecture workshops; the writing of the book […]
Categories: Programming

Google Analytics Demos & Tools

Google Code Blog - Thu, 11/20/2014 - 18:05
As a member of the Google Analytics Developer Relations team, I often hear from our community that they want to do more with GA but don't always know how. They know the basics but want to see full examples and demos that show how things should be built.

Well, we've been listening, and today I'm proud to announce the launch of Google Analytics Demos & Tools, a new website geared toward helping Google Analytics developers tackle the challenges they face most often.

The site aims to make experienced developers more productive (we use it internally all the time) and to show new users what's possible and inspire them to leverage the platform to improve their business through advanced measurement and analysis.

Google Analytics Developers Image
Some highlights of the site include a full-featured Enhanced Ecommerce demo with code samples for both Google Analytics and Google Tag Manager, a new Account Explorer tool to help you quickly find the IDs you need for various Google Analytics services and 3rd party integrations, several examples of easy-to-build custom dashboards, and some old favorites like the Query Explorer.

Google Analytics Demos & Tools not only shows off Google Analytics technologies, it also uses them under the hood. All pages that require authorization use the Embed API to log users in, and usage statistics, including outbound link clicks, authorization status, client-side exceptions, and numerous other user interaction events are measured using analytics.js.

Every page that makes use of a Google Analytics technology lists that information in the footer, making it easy for developers to see how all the pieces fit together. In addition, the entire site is open sourced and available on Github, so you can dive in and see exactly how everything works.

Feedback is welcome and appreciated!

Posted By Philip Walton, Developer Programs Engineer
Categories: Programming

Built to Kill

Software Requirements Blog - - Thu, 11/20/2014 - 17:00
During a conversation with a lead engineer working on the Google self-driving car project, it was mentioned that the car would be programmed to consistently break the speed limit. On average, the car will travel 10 mph over any posted speed limit. Why design a car to deliberately break the law? Safety, primarily; since every […]
Categories: Requirements

How Do You Estimate What You Don’t Know?

Making the Complex Simple - John Sonmez - Thu, 11/20/2014 - 16:00

In this video I answer a question about how to estimate something that you don’t know how to do yet.

The post How Do You Estimate What You Don’t Know? appeared first on Simple Programmer.

Categories: Programming

musiXmatch drives user engagement through innovation

Android Developers Blog - Thu, 11/20/2014 - 15:54

Posted by Leticia Lago, Google Play team

musiXmatch is an app that offers Android users the unique and powerful feature FloatingLyrics. FloatingLyrics pops up a floating window showing synched lyrics as users listen to tracks on their favorite player and music services. It’s achieved through a seamless integration with intents on the platform, something that’s technically possible only on Android.

As a result musiXmatch has seen “a dramatic increase in terms of engagement’, says founder Max Ciociola, “which has been two times more active users and even two times more the average time they spend in the app.”

The ability to deliver lyrics to a range of different devices — such as Chromecast, Android TV, and Android Wear — is creating opportunities for musiXmatch. It’s helping them turn their app into a smart companion for their users and getting them closer to their goal of reaching 100 million people.

In the following video, Max and Android engineer Sebastiano Gottardo talk about the unique capabilities that Android offers to musiXmatch:

To learn about achieving great user engagement and retention and reaching more users through different form factors, be sure to check out these resources:

  • Convert installs to active users — Watch this video to hear from Matteo Vallone, Partner Development Manager for Google Play, about the best practices in engaging and retaining app users using intents, identity, context, and rich notifications as well as delivering a cross-platform user experience.
  • Expanding to new form factors: Tablet, Wear & TV — Watch this panel discussion with Google experts talking about cross-platform opportunities and answering developer questions.
Join the discussion on

+Android Developers
Categories: Programming

Populist versus Technical View of Problems

Herding Cats - Glen Alleman - Thu, 11/20/2014 - 04:17

On a twitter discussions and email exchanges there is a notion of populist books versus technical books  used to address issues and problems encountered in our project management domains. My recent book Performance-Based Project Management® is a populist book. There are principles, practices, and processes in the book that can be put to use on real projects, but very few equations and numbers. It's mostly narrative about increasing the probability of project success. But the to calculate that probability based on other numbers, processes, and systems is not there. That's the realm of Technical books and journal papers.

The content of the book was developed with the help of editors at American Management Association, the publisher. The Acquisition Editor contacted me about writing a book for the customers of AMA. He explained up front AMA is in the money making business of selling books. And that although I may have many good ideas, even ideas that people might want to read about, it's an AMA book and I'll be getting lots of help developing those ideas into a book that will make money for AMA.

The distinction between a populist book and a technical book are the differences between a book that addresses a broad audience with a general approach to the topic and a deep dive book focused on a narrow audience.

But one other disticntion is for most of the technical approaches, some form of calculation takes place to support the materials found in the populist material. One simple example is estimating. There are estimating articles and some books that lay out the principles of estimates. We have those in our domain in the form of guidelines and a few texts. But to calculate the Estimate To Complete in a statistically sound manner, technical knowledge and the underlying mathematics of non-linear, non-stationary, stochastic processes (Monte Carlo Simulation of the projects work structure) is needed. 

Two examples of populist versus technical

Two from my past two from my current work. 

Screen Shot 2014-11-19 at 8.02.28 PM

These two books are about the same topic. General relativity and its description of the shape of our universe.  One is a best selling popularization of the topic, found in many home libraries of those interested in this fascinating topic. The one on the left is on my shelf from a graduate school course on General Relativity along with Misner, Thorne, and Wheeler's Gravity.

Dense is an understatement for the math and the results of the book on the left. So if you want to calculate something about a rapidly spinning Black Hole, you're going to need that book. The book on the right will talk about those Black Holes in non-mathematical terms, but no numbers come out from that description.

Screen Shot 2014-11-19 at 8.03.36 PMThe book on the left is about probabilistic processes in everyday life that we misunderstand or are biased to misunderstand. The many cognitive biases we use to convince ourselves we are making the right decisions on projects are illustrated through nice charts and graphs.

We use the book on the left in our work with non-stationary stochastic process of complex project cost and schedule modeling. Making these decisions is critical to quantifying how technical and economics risk may affect a system's cost. This book is a treatment of how probability methods are applied to model, measure, and manage risk, schedule, and cost engineering for advanced systems. Garvey's shows how to construct models, do the calculations, and make decisions with these calculations.

Here's The Point - Finally

If you come across a suggestion that decisions can be made in the absence of knowing anything about the future numbers or about actually doing the math, put that suggestion in the class of populist descriptions of a complex topic.

If you can't calculate something, then you can't make a decision based on the evidence represented by numbers. If you can't decide based on the math, then the only way left is to decide on intuition, hunchs, opinion, or some other seriously flawed non-analytical basis.

Just a reminder from Mr. Deming stated in yesterday's post


If it's not your money, there's likley an expectation that those providing the money are intestered in the calculations needed to make those decisions. 

Action accordingly

Related articles Estimating Guidance When the Solution to Bad Management is a Bad Solution Measures of Program Performance Should I Be Estimating My Work? Decision Making Without Estimates? Trust but Verify Anecdotal Evidence is not Actually Evidence Why Trust is Hard Baloney Claims: Pseudo - science and the Art of Software Methods
Categories: Project Management

Chinese Developers Can Now Offer Paid Applications to Google Play Users in More Than 130 countries

Android Developers Blog - Thu, 11/20/2014 - 03:07

By Ellie Powers, product manager for Google Play

Google Play is the largest digital store for Android users to discover and purchase their favorite mobile app and games, and the ecosystem is continuing to grow globally. Over the past year, we’ve expanded the list of countries where app developers can sign up to be merchants on Google Play, totaling 60 countries, including Lebanon, Jordan, Oman, Pakistan, Puerto Rico, Qatar and Venezuela most recently.

As part of that continued effort, we’re excited to announce merchant support in China, enabling local developers to export and sell their apps to Google Play users in more than 130 countries. Chinese developers can now offer both free and paid applications through various monetization models, including in-app purchasing and subscriptions. For revenue generated on Google Play, developers will receive payment to their Chinese bank accounts via USD wire transfers.

If you develop Android apps in China and want to start distributing your apps to a global audience through Google Play, visit and register as a developer. If you want to sell apps and in-app products, you'll need to also sign up for a Google Wallet merchant account, which is available on the “Revenue” page in the Google Play Developer Console. After you’ve uploaded your apps, you can set prices in the Developer Console and later receive reports on your revenue. You’ll receive your developer payouts via wire transfer. For more details, please visit our developer help center.

We look forward to continuing to roll out Google Play support to developers in many more countries around the world.

中国开发者可以向全球130个国家的Google Play用户提供付费应用啦

发表者:Ellie Powers, Google Play产品经理

Google Play是一个可让Android用户发现和购买他们喜爱的移动应用程序和游戏的全球最大的应用商店,这个生态系统在全球迅速成长。过去一年中,我们已经扩展到60个国家,让应用程序开发人员可以注册成为 Google Play的商家,其中新近支持的国家包括黎巴嫩、约旦、阿曼、巴基斯坦、波多黎各、卡塔尔和委内瑞拉。

作为持续改进 Google Play努力的一部分,我们很高兴地宣布在中国增加了对商家的支持,让中国的开发者能售卖应用程序到130个国家的 Google Play 用户。中国的开发者现在可以提供通过各种盈利模式的免费和付费应用,包括应用内购买和订阅。在 Google Play 产生的营收将通过美元电汇的方式支付给开发者的中国的银行账户。

如果你在中国开发Android应用程序,并希望通过 Google Play 把应用程序推广到全球,请登录 并建立你的 Google Play 开发者账户。如果你想售卖付费的应用程序和应用程序内的产品,则需要再注册一个Google 电子钱包商家帐户,通过Google Play开发者控制台里的”营收”页面进行设置。上传应用程序后,你可以通过开发者控制台设定价格,之后就可以收到营收报告,你将会通过电汇的方式获得收入。

我们将继续增加更多 Google Play 商家支持的国家,敬请关注。


Join the discussion on

+Android Developers
Categories: Programming

Scaling Agile – SAFe Agile Release Planning: Program Increment Objectives

Program increment (PI) objectives are summaries.

Program increment (PI) objectives are summaries.

Program increment (PI) objectives are summaries of business and technical goals at multiple levels within a program increment.  In the Scaled Agile Framework (SAFe) there are two levels of PI objectives. Team PI objectives reflect the summary of the specific business and technical goals that have been planned during release planning. Program PI Objectives are at a higher level of abstraction, reflecting the summary of the teams PI objectives to describe the overall program increment. The process of generating the PI objectives serves three primary purposes. They are to:

  1. Create alignment. Teams in the ART synthesize the visions (business context, product and architecture visions) presented at the beginning of the planning event along with the prioritized backlog to generate business and technical stories which are then planned. Stories and spikes are added and planned by the team until they reach capacity. When process of planning is complete the work that has been planned is summarized into the team PI objectives. PI objectives are communication vehicles for the team to share what they are doing to those outside the team and for other teams and stakeholders (those outside the team boundary) to consume and understand. The PI objectives are a stand in for a laundry list user stories that allows all of the ART teams to compare objectives.  The comparison process helps to ensure teams are aligned so that the program increment vision can be delivered. Program PI objectives are generated by synthesizing the teams PI objectives into program PI objectives. Program PI objectives are another tool to ensure all of the teams are aligned.
  2. Generate agreement of feasibility. The process of synthesizing the stories into team PI objectives and then from team PI objectives into program PI objectives is a mechanism to communicate what each team believes feasible in the program increment time frame. As team-level PI objectives are communicated and synthesized into program PI objectives, the ART has another opportunity to expose dependencies or to ensure that dependencies are understood.
  3. Set Expectations. PI objectives, whether at the team or program level, are commitments to perform. PI objectives can be thought of as line in the sand based on plans generated by the teams that are committed to doing the work (rather than plans and objectives made for the teams).

The release planning process is a mechanism for teams to plan the work they are going to do in a program increment. PI objectives are a mechanism to summarize the detail that planning creates so that everyone involved in the Agile release train can agree upon what will be delivered. At the end of a program increment those involved in the increment can reflect on the objectives both at a program and team level and use what was learned to continuously improve.

Categories: Process Management

Keeping Your Saved Games in the Cloud

Android Developers Blog - Wed, 11/19/2014 - 19:47

Posted by Todd Kerpelman, Developer Advocate

Saved Games Are the Future!

I think most of us have at least one or two games we play obsessively. Me? I'm a Sky Force 2014 guy. But maybe you're into matching colorful objects, battling monsters, or helping avians with their rage management issues. Either way, there's probably some game where you've spent hours upon hours upgrading your squad, reaching higher and higher levels, or unlocking every piece of bonus content in the game.

Now imagine losing all of that progress when you decide to get a new phone. Or reinstall your game. Yikes!

That's why, when Google Play Games launched, one of the very first features we included was the ability for users to save their game to the cloud using a service called the AppState API. For developers who didn't need an entire server-based infrastructure to run their game, but didn't want to lose players when they upgraded their devices, this feature was a real life-saver.

But many developers wanted even more. With AppState, you were limited to 4 slots of 256k of data each, and for some games, this just wasn't enough. So this past year at Google I/O, we launched an entirely new Saved Games feature, powered by Google Drive. This gave you huge amounts of space (up to 3MB per saved game with unlimited save slots), the ability to save a screenshot and metadata with your saved games, and some nice features like showing your player's saved games directly in the Google Play app.

...But AppState is Yesterday's News

Since the introduction of Saved Games, we've seen enough titles happily using the service and heard enough positive feedback from developers that we're convinced that Saved Games is the better offering and the way to go in the future. With that in mind, we've decided to start deprecating the old cloud save system using AppState and are encouraging everybody who's still using it to switch over to the new Saved Games feature (referred to in our libraries as "Snapshots").

What does this mean for you as a game developer?

If you haven't yet added Saved Games to your game, now would be the perfect time! The holidays are coming up and your players are going to start getting new devices over the next couple of months. Wouldn't it be great if they could take your game's progress with them? Unless, I guess, "not retaining users" is part of your business plan.

If you're already using the new Saved Games / Snapshot system, put your feet up and relax. None of these changes affect you. Okay, now put your feet down, and get back to work. You probably have a seasonal update to work on, don't you?

If you're using the old AppState system, you should start migrating your player's data over to the new Saved Games service. Luckily, it's easy to include both systems in the same game, so you should be able to migrate your users' data with their ever knowing. The process would probably work a little something like this:

  • Enable the new Saved Game service for your game by
    • Adding the Drive.SCOPE_APPFOLDER scope to your list of scopes in your GoogleApiClient.
    • Turning on Saved Games for your game in the Google Play Developer Console.
  • Next, when your app tries to load the user's saved game
    • First see if any saved game exists using the new Saved Games service. If there is, go ahead and use it.
    • Otherwise, grab their saved game from the AppState service.
  • When you save the user's game back to the cloud, save it using the new Saved Games service.
  • And that should be it! The next time your user loads up your game, it will find their saved data in the new Saved Games service, and they'll be all set.
  • We've built a sample app that demonstrates how to perform these steps in your application, so we encourage you to check it out.

In a few months, we will be modifying the old AppState service to be read-only. You'll still be able to read your user's old cloud save games and transfer them to the new Saved Games service, but you'll no longer be able to save games using the old service. We are evaluating early Q2 of 2015 to make this change, which should give you enough time to push your "start using Saved Games" update to the world.

If you want to find out more about Saved Games and how they work, feel free to review our documentation, our sample applications, or our Game On! videos. And we look forward to many more hours of gaming, no matter how many times we switch devices.

Join the discussion on

+Android Developers
Categories: Programming

Quote of the Day

Herding Cats - Glen Alleman - Wed, 11/19/2014 - 19:39

All ideas require credible evidence to be tested, suspect ideas require that even more so - Deep Inelastic Scattering, thesis adviser, University of California, 1978

Carl Sagan 1When the response to questions about the applicability of an idea is push back with accusations that those asking the questions in an attempt to determine the applicability and truth of the statement are somehow afraid of that truth, suggests there is little evidence as a test of those conjectures.

When there are proposals that ignore the principles of business, microeconomics, control systems theory, and are based on well know bad management practices, with well know and easy to apply corrective actions - there is no there, there

Deming 2

So without a testable process, in a testable domain, with evidence based assessment of appliability, outcomes, and benefits, any suggestion is opinion at best and blather at worse.

Related articles Trust but Verify Assessing Value Produced By Investments Should I Be Estimating My Work?
Categories: Project Management