Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Herding Cats - Glen Alleman
Syndicate content
Performance-Based Project Management¬ģ Principles, Practices, and Processes to Increase Probability of Success
Updated: 4 hours 41 min ago

No Estimates Needs to Come In Contact With Those Providing the Money

Wed, 10/01/2014 - 17:36

For all the words written and posted around estimating or not estimating - and I've contributed my share - the basis of estimates has yet to be addressed outside of a few people. @PeterKretzman @aritanninen @kalapaistos@fscavo come to mind.

The gap here is simple. No one seems to ask - or even want to ask - Who are the estimates for? They are not likely for developers, who rightly, so in some cases see estimating as taking away from their valuable development duties.

Who Are Estimates For?

Estimates are for business managers providing the money that appears in the developers paycheck. Estimates are for those same business managers accountable for the Profit & Loss statement of the firm employing the developers writing the code. Those estimates forecast confidence intervals of profit or loss on a project or service before that profit or loss arrives and is irrevocable. 

Estimates are for the business marketing staff in a product firm, who are forecasting the "break even" plan for the sunk cost of developing  software that will be sold in the market. Whose revenue will pay back the short term loan (line of credit) used to pay the salaries of the developers. Without this forecast, decisions about spending or further spending have to be made in the dark.

Estimates are for the business development staff in a professional services and development firm to forecast the confidence in the assure that the contractual obligations to provide working software will not cost more - including management reserve and contingency - than they quoted the customer during the early phases of the project. Since all forecasting are probabilistic, this confidence is - or should be - discussed as the probability of cost at of below or completing on or before. The dysfunction of using estimates as commitments, is recognized as just that - dysfuntion. But as a dysfunction, it's classified as Bad Management. Don't Do Stop Things on Purpose is good advice for any business.

Estimates are for the internal business finance staff accountable for managing and forecasting costs for internal software development or procurement used to run the business - and likely used to generate revenue - and assure the senior finance people that the "value" produced by this software measured in monetized units of "money" will exceed the cost to achieve that value when the project completes. And some sense of when the date will be, so those monetized benefits can start to appear on the balance sheet using FASB 86 accounting rules.

The estimates are not for the developers

Those talking about #NoEstimates from the developers point of view are talking to the wrong people. They appeat to be talking to their own self-selected group and not the group that provides the money for their work. As my former NASA Cost Director colleague reminds me "follow the money." So follow the money. Unless the developers are providing the money themselves, the question of estimating or not estimating is a self-referencing conversation in the absence of these people. Because of that, those best to say if estimates are of value or not are not in the conversation. 

So Back To The Original Question

Ignoring for the moment the observed or perceived dysfunctions found in low maturity software development organizations of the misuse of estimates. Ignore for the moment the preception that making estimates of the future cost, duration, and probabilistic outcomes of development work is part of normal engineering processes. Ignore the emotional rhetoric of the Dilbert approach to management. 

The core principle of Microeconomics of software development requires we  have some approximation of the future to make decisions about alternatives. The opportunity cost, the trade-space of decision making, requires we approximate the cost and outcomes of our decisions. 

Now add the core business process of managing expenditures against a planned and targeted Return on Investment, which has both Value and Cost in it's equation. 

Then ask those conjecturing there are:

  • Decision making frameworks for project that do not require estimates
  • Investment models for software projects that do not require estimates
  • Project management approaches of dealing with risk, scope management, progress reporting that do not require estimates

To connect the dots to those conjectures with Microeconomics of software development and ROI assessments of standard business processes.


Related articles How NOT to Estimate Anything How To Fix Martin Fowler's Estimating Problem in 3 Easy Steps More #NoEstimates All Project Numbers are Random Numbers - Act Accordingly How To Estimate, If You Really Want To Resources for Moving Beyond the "Estimating Fallacy" Back To The Future How to "Lie" with Statistics


Categories: Project Management

What Can Lean Learn From Systems Engineering?

Tue, 09/30/2014 - 16:33

The Lean Aerospace Initiative and the Lean Aerospace Initiative Consortium define processes applicable in many domains for applying lean. At first glance there is no natural connection between Lean and System Engineering. The ideas below are from a paper Igave at a Lean conference.

Key Takeaways

  • Lean and Systems engineering are cousins.
  • All but trivial projects are systems and many are systems of systems. Thinking like a systems engineer is the basis of implementing Lean processes. Thinking in the absence of systems, does little to add sustaining value to any process improvement.
  • Product development is a value stream process, but how the components interact at the technical, business, financial, and operational levels is a systems engineering process. Lean itself does not possess the vocabulary to speak to these systems complexity issues¬†[1]

Core Concepts of Systems Engineering

  1. Capture and understand the requirements in terms of Capabilities assessed through Measures of Effectiveness (MOE) and Measures of Performance (MOP).
  2. Ensure requirements are consistent with what is predicted to be possible in a solution in these MOEs and MOPs.
  3. Treat goals as desired characteristics for what may not be possible.
  4. Define the MOE, MOP, goals, and solutions for the whole lifecycle of the project in units meaningful to the buyer.
  5. Maintain the distinction between the statement of the problem and the description of the solution.
  6. Baseline each statement of the problem and the statement of the solution.
  7. Identify descriptions of alternative solutions.
  8. Develop descriptions of the solution.
  9. Except for simple problems, develop a logical  solution description.
  10. Be prepared to iterate in design to drive up effectiveness.
  11. Base the solution of the evaluation of its effectiveness, in units of measure meaningful to the buyer.
  12. Independently verify all work products.
  13. Validate all work products from the perspective of the stakeholders.
  14. Some management is needed to plan and implement effective and efficient transformation of requirements and goals into a description of the solution.

Typical System Engineering Activities

  1. Technical management
  2. System design
  3. Product realization
  4. Technical analysis and evaluation
  5. Product control
  6. Process control
  7. Post implementation support

Steps to Lean Thinking [2]

  1. Specify value
  2. Identify value stream
  3. Make value flow continuously
  4. Let customers pull value
  5. Pursue perfection

Differences and Similarities between Lean and Systems Engineering

  1. Both emerged from practice. Only later were the principles and theories codified.
  2. Both have focused on different phases of the product lifecycle. SE is generally on product development. SE is more focused on planning. Lean generally on product production. While Lean is more focused on empirical action.
  3. Unlike Lean, SE has less focus on quality, except for Integrated Product and Product Development (IPPD).

Despite these differences and similarities both Lean and Systems Engineering are focused on the same objectives ‚Äď delivering products or lifecycle value to the stakeholders.

It is the lifecycle value that drives both paradigms and must drive any other process paradigm associated with Lean and Systems Engineering. Paradigm like software development, the management of any form of a project and the very notion of agile. A critical understanding often missed is that Lifecycle Value includes the cost of delivering that value.

Value can't be determined in the absence of knowing the cost. ROI and Microeconomics of decision making require both variables to be used to make decisions.

What do we mean by lifecycle?

Generally lifecycle is a combination of product performance, quality, cost and fulfillment of the buyers needed capabilities.[3]

Lean and Systems Engineering share this common goal. The more complex the system, the more contribution there from Lean and SE.

Putting Lean and Systems Engineering Together on Real Projects

First some success factors on complex projects [4]

  1. Dedicated and stable interdisciplinary teams
  2. Use of prototypes and models to generate tradeoffs
  3. Prioritizing product features
  4. Engagement with senior management and customers at every point in the project
  5. Some form of high performing front end decision process that reduces instability of key inputs and improve the flow of work throughout the product lifecycle.

This last success factor is core to any complex environment, no matter what the process is called. In the absence of stability of requirements and funding, improvements to the flow of work is constrained.

The notion of adapting to changing requirements is not the same as having the requirements ‚Äď and the associated funding ‚Äď be unstable.

Mapping of the Value Stream to the work process requires some level of stability. It is the search for this stability where Systems Engineering ‚Äď as a paradigm ‚Äď adds measureable value to any Lean initiative.

The standardization and commonality of processes across complex systems is the basis for this value. [5]


  1. Lean and SE are two side of the same coin regarding the objective of creating value for the stakeholder
  2. Lean and SE complement each other during different phases of the project ‚Äď ideation, product trades for SE and production waste removal for Lean anchor both ends of the spectrum of improvement opportunities.
  3. Value stream thinking makes visible the paths to be taken in transitioning to a Lean paradigm while maintaining the principles of systems engineering. [6]
  4. The result is the combination of Speed and Robustness ‚Äď systems are easily adaptable to change while maintaining fewer surprises, using leading indicators to make decisions and decreasing sensitivity to production and use variables.

[1]¬†‚ÄúThe Lean Enterprise ‚Äď A Management Philosophy at Lockheed Martin,‚ÄĚ Joyce and Schechter,¬†Defense Acquisition Review Journal, 2004.

[2] Lean Thinking, Womack and Jones, Simon and Schuster, 1996

[3] Lean Enterprise Value: Insights from MIT’s Lean Aerospace Initiative, Murman, et al, Palgrave 2002.

[4]¬†‚ÄúLean Systems Engineering: Research Initiatives in Support of a New Paradigm,‚ÄĚ Rebentisch, Rhodes, and Murman,¬†Conference on Systems Engineering, April 2004.

[5] LM21 Best Practices, Jack Hugus, National Security Studies, Louis A. Bantle Symposium, Syracuse University Maxwell School, October 1999

[6] ‚ÄúEnterprise Transition to Lean Roadmap,‚ÄĚ MIT Lean Aerospace Initiative, 2004 Plenary Conference.


Related articles Why Projects Fail, No Matter the Domain When We Say Risk What Do We Really Mean? How to Deal With Complexity In Software Projects? Big Systems Acquisitions - Lessons for ACA Web Site
Categories: Project Management

The Cognitive Illusion of Bad Software Project Outcomes

Tue, 09/30/2014 - 00:41

Daniel Kahneman's and Amos Tversky's paper On The Reality of Cognitive Illusion. ‡ They suggest, through their research, that intuitive predictions and judgements are often mediated by a small number of distinctive mental operations, called judgement heuristics. These heuristics often lead to characteristic errors and biases.

For example, the effect of aerial perspective on apparent distance is confirmed by the observation that the same mountain appears closer on a clear day rather than a hazy day. The intuitive predication and judgement of probability are often based on the relations of similarity between evidence and possible outcomes. This representativeness is an assessment of the degree of correspondence between a sample and a population. 

The next heuristic is the availability bias in which the probability is estimated by assessing availability or associative distance. † Experience shows and experiments confirm that large classes are recalled better and faster than instances of less frequent classes. That likely occurrences are easier to imagine than unlikely ones. And associative connections are strengthened when two events frequently co-occur. That these associative bonds are strengthened by repetition is the basis of memory. 

So Here's the Issue

When we hear or read that software projects fail often or Standish report says ..., or a personal anecdote that resonates with our own personal experience, we recall that experience from memory. The actual data from the population of all data are not used for comparison. Rather we assume - by applying the cognitive illusion - that the sample sata represents the large class of population data, since our repeated observations of the sample data class has reinforced our illusion that that sample data IS the population.

This is the core issue with anecdotal information when making decisions in the presence of uncertainty. Or speaking about a condition in the absence of statistically testable hypothesis. Or attempting to convey a message in the absence of external confirmation that the message is on solid footing compared to the population of data.

Why This Is Not Good Management

When we hear we're all bad at making estimates, in the absence of actual population statistics about estimate making, we're using Cognitive Illusions and Availability Heuristics. Because we have personal experience with making bad estimates and the majority of people we associate with have the same experience.

This experience is in no way representative of the population of people tasked with making estimates. This would be irrelevant of course if the conversation were simple chatter at the bar. But once that conversation enters the realm of policy making, method development, or suggestions that the anecdotal observations need to result in changing how business conducts its business - we're bad at making estimates so the solution is to stop making estimates - then both availability bias and Cognitive Illusions have displaced the actual conversation about the very validity of the anecdotal concepts. And it is replaced by strong defense of the cognitively biased dea, no matter the credibility of the concept - which is most often weak at best and simply false at worse.

So next time you hear some statement about something involving observational and anecdotal data, ask a simple question.

What's the process by which these anecdotal observation have been tested in the broader population of conditions?

This is the core issue with the Standish Reports. They are self selected samples of projects that are troubled in the absence of the population of projects that are troubled and not troubled. 

Always ask for references, data representative of the references, and an assessment of the statistical confidence that the anecdotal data is in fact correlated with the population data. Otherwise it's just an opinion, and very likely an uniformed opinion.

And if you're paying money to listen to someone tell you ancedotal data and don't speak up and ask those questions, you've participated in the availability heuristic and cognitive illusion along with the speaker.

† Availability: A Heuristic for Judging Frequency and Probability, Amos Tversky and Daniel Kahneman, a chapter appearing in Cognitive Psychology, 1973

‡ On the Reality of Cognitive Illusions, Daniel Kahneman and Amos Tversky, Psychological Review, Vol. 103, No. 3, pp. 582-591

Categories: Project Management

Don Yaeger's 16 Consistent Characteristics of Greatness

Mon, 09/29/2014 - 04:40

Don Yeager has a small book mark sized card on 16 Consistent Characteristics of Greatness. I got my card at a PMI conference where he spoke. I'm repeating them here. Don's talk was about sports people he interviewed for magazines and books. The audience was hard-bitten Government and Industry project and program managers. Those accountable for millions and billions of dollars of high risk, high reward endeavors. After Don finished his talk, no a person in the room had dry eyes. Subscribe to Don's daily message at

How They Think

1. It's personal - they hate to lose more than they love to win.

2. Rubbing elbows - they understand the value  of association.

3. Believe - they have faith in a higher power.

4. Contagious Enthusiasm - they are positive thinkers... They are enthusiastic... and that enthusiasm rubs off.

How They Prepare

5. Hope for the best, but ... They prepare for all possibilities before they step on the field.

6. What Off-Season? They are always working towards the next game... The goal is whart's ahead, and there's always something ahead.

7. Visualize Victory - They see victory before the game begins.

8. Inner Fire - they use adversity as fuel.

How They Work

9. Ice in Their Veins - they are risk-takers and don't fear making a mistake.

10. When All Else Fails - they know how - and when - to adjust their game plan.

11. Ultimate Teammate - they will assume whatever role is necessary for the team to win.

12. Not Just About the Benjamin's - they don't just play for the money.

How They Live

13. Do Unto Others - they know character is defined by how they treat those who cannot help them.

14. When No One Is Watching - they are comfortable in the mirror... They live their life with integrity.

15. When Everyone is Watching - they embrace the idea of being a role model.

16. Records Are Made To Be Broken - they know their legacy isn't what they did on the field. They are well rounded.

Categories: Project Management

Making Choices in the Absence of Information

Sat, 09/27/2014 - 15:57

Making Choices with Scant InformationDecision making in the presence of uncertainty is a normal business function as well as a normal technical development process. The world is full of uncertainty.

Those seeking certainty will be woefully disappointed. Those conjecturing that decisions can't be made in the presence of uncertainty are woefully misinformed. 

Along with all this woefulness is the boneheaded notion that estimating is guessing, and that decisions can actually be made in the presence of uncertainty in the absence of estimating.

Here's why. When we are faced with a decision, a choice between multiple decisions, a choice between multiple outcomes, each is probabilistic. If it were not - that is we have 100% visibility into the consequences of our decision, the cost involved in making that decision, the cost impact or benefit impact from that decision - it's no longer a decision. It's a choice to pick between several options based on something other than time, money, or benefit.

We're at the farmers market every Saturday morning. Apples are in season. Honey Crisp are my favorite. Local growers all know each other and price their apples pretty much the same. What they don't sell on Saturday, they take to private grocers. What doesn't go there, goes to the chains and labeled Locally Grown. Each step in the supply chain has a mark up, so buying at the Farmers Market is the lowest price. So deciding which apples to buy is usually an impulse for me and my wife. The cost is the same, the benefit is the same, it's just an impulse.

Let's look at the broad issue here - not about apples, from Valuation of Software Initiatives Under Uncertainty, Hakan Erdogmus, John Favaro, and Michael Halling, (Erdogmus is well known for his work in Real Options).

Screen Shot 2014-09-26 at 5.06.49 PM

Buying an ERP system, or funding the development of a new product, or funding the consolidation of the data center in another city is a much different choice process than picking apples. These decisions have uncertainty. Uncertainty of the cost. Uncertainty of the benefits, revenue, savings, increasing in reliability and maintainability. Uncertainty in almost every variable. 

Managing in the presence of uncertainty and the resulting risk, is called business management. It's also called how adults manage projects (Tim Lister)

So with the uncertainty conditions established for our project work, how can we make decisions in the presence of the uncertainties of cost, schedule, resource utilization, delivered capabilities, and all the other attributes and all the ...ilities of the inputs and outcomes of our work?

The Presence of Uncertainty is one of most Significant Characteristics of Project Work

Managing in the presence of uncertainty is unavoidable. Ignoring this uncertainty is also unavoidable. It's still there even if you ignore it.

Uncertainty comes in many forms

  • Statistical uncertainty - Aleatory uncertainty, only margin can address this uncertainty.
  • Subjective judgement - bias, anchoring, and adjustment.
  • Systematic error - lack of understanding of the reference model.
  • Incomplete knowledge - Epistemic Uncertainty, this lack of knowledge can be improved with effort.
  • Temporal variation - instability in the observed and measured system.
  • Inherent stochasticity - instability between and within collaborative system elements

Agile is an Approach to Dealing With Software Project Uncertainty

Going on 12 years ago the topic of managing in the presence of uncertainty was an important topic that spawned approaches to ERP using agile. This work has progressed to more formal principles and practices around software development in the presence of uncertainty and the acquisition of software products.

So Back To the Problem at Hand If decisions - credible decisions - are to be made in the presence of uncertainty, then some how we need information to address the sources of that uncertainty in the bulleted list above. This information can be obtained through many means. Modeling, sampling, parametrically, past performance, reference classes. Each of these sources has in itself an inherent uncertainty.  So in the end, it comes done to this... To make a credible decision in the presence of uncertainty, we need to estimate the factors that go into that decision. We Need To Estimate There's no way out of it. We can't make a credible decision of any importance without an estimate of the impact of that decision, the cost incurred from making that decision, the potential benefits from that decision, the opportunity cost of NOT selecting an outcome from a decision. Picking one Honey Crisp basket over another, not much at risk, low cost, low probability of disappointment. Planning, funding, managing, deploying, operating an ERP system, not likley done in the absence of estimating all variables up front, every time we produce the next increment, every time we have new information, every time we need to make a decision. To suggest otherwise violates the core principles of Microeconomics. If it's your money no one cares - apples or ERP, proceed at your own risk, ignore microeconomics. If it's not your money, it's going to require intentional ignorance of the core principles of successful business management. Behave appropriately.   Related articles Time to Revisit The Risk Discussion How to Deal With Complexity In Software Projects? An Example of Complete Misunderstanding of Project Work Uncertainty is the Source of Risk When We Say Risk What Do We Really Mean? Both Aleatory and Epistemic Uncertainty Create Risk Unceratinty and Risk Four Types of Risk Bayesian probability theory banned by English court
Categories: Project Management

An Example of Complete Misunderstanding of Project Work

Sat, 09/27/2014 - 01:06

Choose-to-Know-Stop-the-Misinformation-Profile-Picture-1WARNING RANT AGAINST INTENTIONAL IGNORANCE FOLLOWS

This is one of those pictures tossed out at some conference that drives me crazy. It's uninformed, ignores the disciplines of developing software for money, and is meant to show how smart someone is, without actually understanding the core processes needed for actually being knowledgeable of the topic - in this case statistical processes of project work. Then the picture gets circulated, re-posted, and becomes the basis of all kinds of other misunderstanding, just like the Dilbert cartons that represent cartons of the problem, but have no corrective actions associated.

It is popular in some circles of agile development to construct charts showing the strawman of deterministic and waterfall approaches, then compare them to the stochastic approaches and point out how much better the latter is than the former. Here's an example.

These strawman approaches are of course not only misinformed, they're essentially nonsense in any domain where credible project management is established, and the basis of the their response with Don't Do Stupid Things on Purpose.

Screen Shot 2014-09-25 at 11.27.59 AM

Let's look at each strawman statement for the Deterministic View in light of actual project management processes, either simply best practice or mandated practice.

  • Technologies are stable - no one believes this that has been around in the last 50 years. And if they do, they've been under a rock. Suggesting this is the case ignores even¬†the simplest observations of technology and it's path of progress.
  • Technologies are predictable - anyone who has any experience in any engineering disciplne knows this is not the case. Beyond the simplest single¬†machine unintended consequence and emergent behavior with obvious.
  • Requirements are stable - no they're not. Not even in the bone head simplest project. This would require precognition and clairvoyance.
  • Requirements are predictable - no they're not. Read any Requirements Management guidance, any requirements elicitation process, or work any non-trivial project to learn this as a cub developer.
  • Useful information is available at the start - this would require¬†clairvoyance.
  • Decisions are front loaded - this ignores completely the principles of microeconomics of decision making in the presence of uncertainty. Good way to set fire to your money. For a good survey of when and how to make front loaded decisions see¬†Making Essential Choices with Scant Information. Also buy Williams other book¬†Modelling Complex Projects. Along with another book Project Governance: Getting Investments Right.¬†This statement is a prime example of not doing your homework before deciding to post something in public.
  • Task durations are predictable - all task duration are driven by aleatory uncertainty. For this to be true, the laws of stochastic process would have to be suspended. Another example of been asleep in the High School statistics class.
  • Task arrival times are predictable - same as above. Must have be a classics major in college. With full apologies to our daughter who was a classics major.
  • Our work path is linear, unidirectional - this would require the problem to be so simple it can be modeled as a step by step assembly of Lego parts. Unlikely in any actual non-trivial project. When system of systems becomes the problem - any enterprise IT project, any complex product - the conditions of linear and unidirectional go out the window.
  • Variability is always harmful - this violates the basis of all engineering systems, where Deming variability is built into the system. Didn't anyone who made this chart read Deming?
  • The math we need is arithmetic - complete ignorance of the basic processes of all systems - they are statistical generating functions creating probabilistic outcomes.

The only explanation here is the intentional ignorance of basic science, math, engineering, and computer science. 

In the stochastic View there are equally egregious errors.

  • Technologies are evolving - of course they are. Look at any technology to see rapid and many times disruptive evolution. Project management is Risk Management. Risks are created by uncertainty - reducible and irreducible. Managing in the presence of uncertainty is how adults manage projects.
  • Technologies are unpredictable - in some sense, but we're building systems from parts in the market place. If you're a researcher at Apple this likely the case. If you're integrating an ERP system, you'd better understand the process, technology, and outcomes, or you're gonna fail. Don't let people who believe this to spend your money.
  • Requirements are evolving - of course they are. But the needed capabilities had better be stable or you've signed up for a Death March project, with no definition of done. But requirements aren't the starting point, Capabilities are. Capabilities Based Planning is how enterprise and complex projects are managed.
  • Requirements are degree of freedom - have no idea what this means. Trade Space is part of all good engineering process. Wonder if the author or those referencing this chart know that.
  • Useful information is continuously arriving - of course it is. This work is called engineering and development. Both are¬†verbs.
  • Decisions are continuous - of course they are. This is the core principle of microeconomics of all business decision making. But front-end decisions are mandatory. See "Issues on Front-end decision making¬†for some background before believing this statement is credible. And a summary of the concept of the Williams book above.¬†
  • Task arrival times are unpredictable - this is intentional ignorance of stochastic processes. Prediction always includes confidence and a probability distribution. Predictions¬†is simply saying something about the future. For task arrival time to be unpredictable, those time would have to be completly chaotic, with no underlying process to produce them. This would be unheard of in project work. And if so the project would be chaotic and destination to fail starting on day way. Another example of asleep in the stats class.
  • Our work path is networked and recursive - of course it is. But this statement is counter to the INVEST condition of agile which is present in only the simplest project.¬†
  • Variability is required - all processes are stochastic processes. A tautology. Natural variability is irreducible. Event based variability is disruptive to productivity, quality, cost and schedule performance and the forecasting of when, how much, and what will be delivered in terms of Capabilities. Uncontrolled Variability is counter proper stewardship of your customer's money.
  • The math we need is probability and statistics - yes, and you'd better have been paying attention in the High School statistics class and stop using terms you can't refer to in the books on your office shelf.¬†

In the End

For some reason using charts like this one, re-posting of Dilbert cartons, making statements using buzz words - we're using Real Options and Bayesian Statistics to manage our work - are may favorite ones - seems to be more common the closer we get to the sole contributor¬†point of view. Along with¬†look at my 22 samples of self-selected data with a ¬Ī70% variance as how¬†to forecast future performance.

It may be because sole contributors are becoming more prevalent. Sole contributors have certainly changed the world of software development in wasy never possible by larger organizations. But without the foundation of good math, good systems engineering - and I don't mean "data center systems engineering," I mean INCOSE Systems Engineering - those sole contributor points of view simply don't scale.

Always ask when you hear a piece of advice - in what domain have you applied this advice with success? 

Related articles Why is Statistical Thinking Hard? The Misunderstanding of Chaos - Again Deterministic versus Stochastic Trends in Earned Value Management Data Management is Prediction - W. Edwards Deming How To Assure Your Project Will Fail
Categories: Project Management

Quote of the Day

Thu, 09/25/2014 - 17:01

Most problems are self imposed and usually can be traced to lack of discipline. The foremost attribute of successful programs is discipline: Discipline to evolve and proclaim realistic cost goals; discipline to forego appealing but nonessential features; discipline to minimize engineering changes; discipline to do thorough failure analysis; discipline to abide by test protocols; and discipline to persevere in the face of problems that will occur in even the best-managed programs - Norm R. Augustine

Related articles Agile Requires Discipline, In Fact Successful Projects Require Discipline
Categories: Project Management

Quote of the Day

Thu, 09/25/2014 - 04:00

A cynic is a man who knows the price of everything and the value of nothing. ‚ąí Oscar Wilde

The inverse is true as well, we can't know the value of something until we know it's cost. Both the cost and the value can be tangible or intangible. But both are needed when faced with deciding between alternatives. This is the very foundation of microeconomics of business decision making.

Categories: Project Management

Building a Credible Performance Measurement Baseline - Update

Wed, 09/24/2014 - 15:36

 An Update

Screen Shot 2014-09-24 at 8.33.49 AM

Categories: Project Management

The Cost of Value in the Future

Tue, 09/23/2014 - 01:02

CIBCOn my way to a client in Toronto, the logo advertisment for CIBC said...

The future emerges

And since that future is rarely known in advance, the notion of Value in the future and the cost of producing that value came to me.

Value is what happnes to something when it it useful. Anything material or immaterial is of value only because it is useful for or to somebody. 

Thr cost of this value is used to assess its usefulness. This is the affordability of value principle of Microeconomics. As well the affordability of the Better Buying Power initiatives in our domain. The Microeconomics principles askswhat is the lost opportunity cost of any decision made by spending - or not spending - a scarce resource  to acquire something of value?

If this value and its cost are in the future, then there is uncertainty (reducible and irreducible) in both the Value and the Cost of that value. In the absence of estimates of the Value and the Cost, decisions about choices between alternatives can not be informed.

The notion of making decisions about future outcomes, based onValue and the Cost of that value without estimating the cost, risk, and produced value violates the principle of decision making in the presence of scarce resources - time, money, talent.

So when we hear there are...

  • Decision making framework for project that do not require estimates
  • Investment models for software projects that do not require estimates
  • Project management approaches using risk management, scope management, and progress reporting that do not require estimates
  • Ways toProve that estimates are not needed

Then ask some hard questions of how the principles of microeconomics of software development can be set aside, and demand tangible evidence showing these conjectures are actually possible in any development domain where the future emerges while spending other peoples money.


Related articles Why Is It Hard To Manage Projects? What Do We Mean When We Say "Agile Community?" Incremental Commitment Spiral Model Principles Trump Practices and Processes Focus on Value is Only ¬Ĺ The Equation How Not To Make Decisions Using Bad Estimates The Value of Information Microeconomics
Categories: Project Management

Project Driven Organizations

Mon, 09/22/2014 - 16:18

The notion that projects are somehow no longer needed fails to address how to replace the processes, governance, and stewardship of a business's assets while producing the needed value from those expenditures.  Here's a framework for management a firms funds in producing value in exchange of those funds.

Lifecycle from Glen Alleman   This governance process can be guided by Project governance from Glen Alleman Related articles Agile as a Systems Engineering Paradigm Performance-Based Project Management(sm) Released Capabilities Based Planning and Development Why Is It Hard To Manage Projects?
Categories: Project Management

Incremental Commitment Spiral Model

Sun, 09/21/2014 - 15:00

Incremental CommitmentUnless you're building sofwtare as a hobby, someone is paying you to do that work. Those paying aren't likley doing it as a hobby either. They have some expectation of getting their money back sometime in the future. Somewhere in the discussion of writing software for money, the notion of writing software for money was lost. 

Those with money pay those with software writing capabilities to produce products that can be sold or put to use to create a value in return. Along the way was a disconnect that software is an end in itself. That the needs to developers trumps the needs of those providing the money for the developers. That those spending the money get to say what they'll do, how they'll do it, or what they won't do with that money.

Writing software for money as practiced in a sole contributor paradigm provides nearly infinite flexibility on requirements, cost and schedule forecasting, and the current notion of making business, programmatic, and technical decisions in the absence of estimating the cost and impact of those decisions.

When that paradigm leaks into a larger domain of producing a return on the investment from that cost, there are two varaibles that must enter every conversation. The Value generated by expending a Cost to produce an assessment of both those variables.

ROI = (Value - Cost) / Cost

A Value at Risk is one approach to assessing what processes should be in place when spending othe people money. The larger the Value at Risk requires a larger discipline of managing both the Cost and Value. There are many paradigms of Agile and the domain and context of software development, or any project for that matter, is important to assess before stating any method is applicable outside the ancedotal domain of the speaker.

The first assessment is always Value at Risk. That is, what is the cost of making a wrong decision? This is the basis of Microeconomics. This is the oppotunity cost assessment of decision making.

Microeconomics studies the behavior of individuals and small impacting organizations in making decisions on the allocation of limited resources. Cost, schedule, and technical capabilities are certainly a limited resource.

Those conjecturing decisions can be made in the absence of estimating the cost and impact have yet to show the viability of those ideas in practice, at least outside small projects with low Value at Risk.

The book The Incremental Commitment Spiral Model: Principles and Practices for Successful Systems and Software, Barry Boehm and Jo Ann Lane is a good bridge book between small low value at risk agile, Scaled Agile for Enterprise, and the full up formal DOD 5000.02 acquisition processes that are trying very hard to move into the agile domain.

The book starts with four principles:

  1. Stakeholder value-based guidance
  2. Incremental commitment and accountability
  3. Concurrent multi-discipline engineering
  4. Evidence and risk-based decisions

There are extensions to these principles:

  • Risk Meta-Principles of Balance - balancing the risk of doing to little and the risk of doing too much to find the middle course¬†sweet spot.
  • Theory W (Win-Win) theorem - in which a system will succeed of and only of it makes winners of its success-critical stakeholders.
  • System Success Realization Theorem - in which making winners of success-critical stakeholders requires:
    • Identifying all of the success-critical stakeholders.
    • Understanding how each stakeholder wants to win.
    • Having the success-critical stakeholders negotiate among themselves a win-win set of product and process plans.
    • Controlling progress toward the negotiated win-won realization, including adapting it to change.
Related articles Agile Requires Discipline, In Fact Successful Projects Require Discipline Why Johnny Can't Estimate or the Dystopia of Poor Estimating Practices What Do We Mean When We Say "Agile Community?" Why Project Management is a Control System Critical Thinking Skills Needed for Any Change To Be Effective
Categories: Project Management

Technical Performance Measures

Sat, 09/20/2014 - 16:59

We're preparing for a Webinar on 25 September 2014, now titled Using Techncial Performance to Inform Earned Value, which addresses the disconnect in EAI-748-C between two statement

  • Page 2, 5 bullet - Objectively assess accomplishments at the work performance level. ¬†
  • ¬ß3.8 - Earned Value is a direct measurement of the quantity of work accomplished. The quality and technical content of the work performed is controlled by other processes. ¬†

Two reconcile these two statement, we need to have a process of informing Earned Value (BCWP) with the Techncial Performance of the products being built. After the Webinar, we'll post the link. 

In the mean time here's a list of resources gathered to support this topic.

  • Technical Performance Measurement - A Program Manager's Barometer: DCMA Pilots a Modified Approach to TPMs, Mike Ferraro,¬†Program Manager, November-December, 2002
  • Technical Performance Measures, Jim Oaks, Rock Botta and Terry Bahill, BAE Systems.
  • Technical Measurement: A collaborative Project of PSM, INCOSE, and Industry, Garry Roedker and Cheryl Jone, 27 December 2005, INCOSE-TP-2003-020-01.
  • Technical Performance Measurement Earned Value, and Risk Management: Integrated Diagnostic Tool for Program Management, Commander N. D. Pisano, SC, USN, PEO Air SW and Special Mission Programs.
  • CPM-500-C/D/E: Integrated Systems Engineering with Earned Value Management, Partick K. Barker, Paul Solomon, November 2009, Alexandria, VA
  • CPM-500: Principles of Technical Management, Lesson D: Implementing Technical Performance Measurement, Mike Ferraro, IPMC 2002.
  • Technical Performance Measures Module: Space Systems Engineering Version 1.0, NASA.
  • Technical Performance Measurement - Systematic Approach for Planning, Integration, and Assessment, Part 1, Kathyyn A. Kulick.
  • Technical Performance Measurement - Kinking Technical Performance to Cost and Schedule, Part 2 Kathyyn A. Kulick.
  • Technical Performance Measurement - Assessment Techniques and Earned Value Calculation, Part 3, Kathyyn A. Kulick.
  • Leading Indicators for Systems Engineering, Sixth Annual NASA Project Management Challange, Dr. Donna H. Rhodes, MIT, Feb 2009.
  • The Difficult Problems of Establishing Measures of Effectivenss for Command and Control: A Systems Engineering Perspective, Noel Sproles,¬†Systems Journal, Volume 4, Number 2, 2001.
  • Coming to Grips with Measures of Effectiveness, Noel Sproles,¬†Systems Journal, INCOSE 2000.
  • Formulating Measures of Effectiveness, Nole Sproles,¬†Systems Engineering Volume 5, Number 4, 2002
  • Systems Engineering at NASA: Selected Topics, Jim Andary, NASA Goddard Space Flight Center, Nov 5, 2009.
  • Space and Missile Command, Systems Engineering Primer and Handbook
  • NASA Systems Engineering Handbook
  • Engineering Complex Systems with Models and Objects,David W. Oliver Timothy P. Kelliher James G. Keegan, Jr., McGraw Hill.
  • Engineering Complex Systems, Douglas O. Norman and Michael L. Kuras, The MITRE Corporation
  • National Aerospace System - Systems Engieering Manual, FAA
  • MITRE Systems Engineering Guide, Collected Wisdom from MITRE's Systems Engineers.
Related articles Is There Such a Thing As Making Decisions Without Knowing the Cost? More Uncertainty and Risk Agile as a Systems Engineering Paradigm
Categories: Project Management

Project Driven Organizations

Fri, 09/19/2014 - 21:09

There is a popular notion that Projects are some how not needed to develop software. Just fund the Team and the business outcomes needed to fulfill a strategy and deliver value to the balance sheet will appear. 

This may work in  domains where the project team, the business and its processes and the funding sources are in fact all the same - a small startup for example. When the business moves beyond this self-contained startup mode, other  processes become needed to Govern the use of funds - Stewardship of Funding. 

Here's how to approach the process, once the firm moves beyond a collection of individuals personally all interacting with their customer based. These processes, data, and outcomes belong to the realm of Governance. In IT, Governance †

  • Clarifies business strategies and the role of IT in achieving them.
  • Measures and manages the amount of funding spent on the value received from IT.
  • Assigns accountability for the organizational changes required to benefit from new IT capabilities.
  • Learns from each implementation and becoming more adept at sharing and reusing IT assets.

Lifecycle from Glen Alleman † IT Governance: How Top Performers Manage IT Decision Rights for Superior Results, Peter Weill and Jenne W. Ross, Harvard Business School Press. Related articles Delivering Needed Capabilities Staying on Plan Means Closed Loop Control The Purpose Of Guiding Principles in Project Management Performance-Based Project Management(sm) Released What Do We Mean When We Say "Agile Community?"
Categories: Project Management

Principles Trump Practices and Processes

Thu, 09/18/2014 - 17:05

5 Immutable PrinciplesA new Book The Incremental Commitment Spiral Model: Principles and Practices of Successful Systems and Software, is now out.

From another source The right principles trump practices everytime - Dean Leffingwell.

This notion that practices and processes can be put forward in the absence of testing them against  principles has become popular.

The most visible of course is that decisions can be made in the absence of estimating the cost and impact of that decision. The principle of MicroEconomics of Software development was first stated by Dr. Boehm. Early in that #NoEstimates discussion was a comment that all those ideas are old and no longer applicable. Of course that ignores the principle of Microeconomics along with most every other principle of managing projects while spending other peoples money. 

As well there are other principles of project success

  • What does done look like in units of Effectiveness and Performance?
  • What is the Plan to reach Done on the needed date for the needed cost?
  • What are the resources required to reach Done?
  • What impediments will be encountered along the way that must be handled?
  • How are we going to measure progress toward Done in units of measure meaningful to the decision makers?

Here's how to develop the answers to those Principles questions.

Principles and Practices of Performance-Based Project Management¬ģ from Glen Alleman
Categories: Project Management

Statistical Significance

Tue, 09/16/2014 - 23:58

Screen Shot 2014-09-16 at 4.33.40 PMThe term statistical significance is critical to most every discussion about spending other peoples money with some new and innovative process.

When we here I know a CEO that uses my approach, we need to ask several critical questions both getting too excited about this idea that is being suggested. Especially is this new idea violates some core business processes, like Microeconomics, let alone FASB 86, GAAP cost and revenue recognition.

  1. Is that CEO the CEO of a publically traded firm, subject to governance processes? If so, some outside the developer communbity gets to say if the new and innovative idea hasn't violated the business rules.
  2. Does that CEO's company live a domain where what they do is like what other people do? You know a Reference Class that can be used outside the ancedote.
  3. Does that CEO have to report cash obliogations to his line or credit or banker for some planning horizon in the future? Those pesky bankers do like know the cash call demands from your firm for that LOC.

The notion of an anecdote is always interesting in conversation I knew a guy once who .... But can we make policy decisions based on anecdotes? Hopefully not. 

We can make policy decisions based on statistically sound observation - 8 out 10 dentist recommend Pepsident Toothpaste was a popular advertisement in the 1970's.

This leads us back to How To Lie With Statistics and the self-selected sample space. 

  • Let me ask all the people I ride with in our cycling club what they think of the local brewery where we leave from on Tuesday evenings, what they think of the Nitro¬†Milk Stout that is served for free.¬†We like it.

Without a statistical sound sample space, a statistically sounds sampling processes, any conclusion are just ancedote. This is the core issue with things like the Standish report and other surveys suggesting the sky is falling on IT projects. 

The same goes for thosie suggesting their favorite apporoach to spending other peoples can be done in the absence of knowing hwo much money, when that money will produce value, and what kinds of value will be produced.

Ask for data. No data, then as they say "Data Talks, BS walks"

† The carton above is from Hugh MacLoed, gapingvoid art, 1521 Alton Road, Suite #518, Miami Beach, FL 33139. I've been following him since day one. You shold do the same and buy his book

Related articles Another Nonsense Statistical Survey The Failure of Open Loop Thinking How to "Lie" with Statistics FASB Issues 2015 GAAP Financial Reporting Taxonomy for Public Review How Not To Make Decisions Using Bad Estimates Can Agile Be Integrated with Governance Based Development Processes? How To Fix Martin Fowler's Estimating Problem in 3 Easy Steps What Software Domain Do You Work In? All Project Work is Probabilistic Work
Categories: Project Management

Quote of the Day - Plan for the Future

Tue, 09/16/2014 - 01:05

Before you begin a thing remind yourself that difficulties and delays quite impossible to foresee are ahead. You can only see one thing clearly, and that is your goal. Form a mental vision of that and cling to it through thick and thin.
‚ÄĒ Kathleen Norris

Screen Shot 2014-09-15 at 3.58.10 PMAnd when they are impossible to see we need margin and reserve to protect our project from them. This margin is for the irreducible risks and the reserve is for buying down the reducible risks. Both irreducible (aleatory) and reducible (epistemic) uncertainties can be modeled for all projects. This is the role of risk management and project. To NOT model these uncertainties is to ignore them. To ignore them is to say to those providing the money that you're not following Tim Lister's advice...

Risk Management is how Adults manage projects

As well when you begin a thing remember another important quote...

Measurement is the first step that leads to control and eventually to improvement. If you can‚Äôt measure something, you can‚Äôt understand it. If you can‚Äôt understand it, you can‚Äôt control it. If you can‚Äôt control it, you can‚Äôt improve it. ‚Äē H. James Harrington

So if we're going to start a job and don't have an assessment - to some level of confidence - of how long it will take, how much it will cost, and what we will be capable of delivering when we get to the end of the money and the time - it is very likely that those paying for our work will be very disappointed in our efforts.

When we here that we can make decisions in the absence of knowing the probabilistic cost, schedule, and likelihood of producing the needed capabilities, think back to the two quotes above. And consider the conjectures below that requires us to ignore those quote and instead follow those statements in the absence of any evidence they are applicable outside of the Value at Risk being low enough that those providing the money don't really care if it's all a loss.


† Orginal post from Mark Anderson's email from ExecuNet, 9/14/2014

Related articles Uncertainty is the Source of Risk The World of Probability and Statistics How to Deal With Complexity In Software Projects? Both Aleatory and Epistemic Uncertainty Create Risk Time to Revisit The Risk Discussion
Categories: Project Management

GAO Reports on ACA Site

Fri, 09/12/2014 - 19:08

With all the speculation on what went wrong with the ACA site and all the agile pundits making statements about how agile could have saved the site, here's some actual facts beyond all the opinions - that Daniel Patrick Moynihan would remind us...

Every man is entitled to his own opinion, but not his own facts


  Screen Shot 2014-09-12 at 11.40.52 AM


Screen Shot 2014-09-12 at 11.42.53 AM

The Key Findings are

  • Oversight weakness and lack of adherence to planning requirements.
  • Acquisition planning carried high levels of risk.
  • Requirements for developing the FFM were not well defined.
  • Contract type carried risk for the government.
  • New IT development approach was supposed to save time, but carried unmitigated risk.
  • Contractor did not fully adhere to HHS procurement guidelines, missing opportunities to capture and consider risk important to program success.
  • Changing requirements and oversight contributed to significant cost growth, schedule delays, and reduced capabilities.
  • Unclear contract oversight responsibilities exacerbated cost growth.
  • Significant contractor performance issues without corrective actions.

So when we hear 


Think in what domain, with what value at risk, with what complexity of project, and what business process in which these could possibly be applicable. In fact this goes back to the core of the agile manifesto. And when we hear "pure agile," Scrum Masters produce Scrum Slaves, Mob Programming, "we all want a seat at the table with equal voices, and many other "opinions," remember Moynihan and ask for facts, domain, past performance, experience, and examples of success.

As agile starts to scale to larger domains and the government seeks better ways to develop software beyond the failed processes described above - what parts of this manifesto are applicable outside of a small group of people in the same room with the customer directing the work of the developers?

As my colleague (former NASA Cost Director) reminds our team if you see something that doesn't make sense - follow the money. In the case of ACA and in the case of the Work Shop outcomes above.

Related articles Can Agile Be Integrated with Governance Based Development Processes? Agile Means ... Can Enterprise Agile Be Bottom Up? Agile as a Systems Engineering Paradigm How To Create Confusion About Root Causes Agile and the Federal Government Quote of the Day - Just a Little Process Check Sailing and Agile
Categories: Project Management

Making Decisions

Sat, 09/06/2014 - 23:07

Making Multi-Objective DecisionsAlmost all decision problems involve the simultaneous considerations of different objectives that are many times in conflict.

In the software development world this might be characterized by a customer wanting a set of features to be delivered for a budget on a certain date so those features can be put to work earning back their cost.

Since the list of features is likely to needed to be developed in a specific order with varying costs, the question is what features should be done first and what features down next. 

The traditional response from an agile developer is the define the value of those features and produce them in that order. What is the unit of measure of value. That's rarely stated. But along with the Value is the cost of that value and other attributes of the development process. Risk, resource demand, inter dependencies between other features, inter dependencies between these features and external processes - the externalities of all complex problems.

The formal discipline is this process is called Multi-Criteria Decision Analysis (MCDA). MCDA has the following elements:

  • A goal, objective, or criteria to be achieved
  • A ¬†need to be fulfilled
  • Constraints and requirements associated with and affected by the decision
  • Decision options or alternatives
  • The Environment in which the decision must be made
  • The Experience and background of the decision makers

The methods to solve these types of problems started in the 1950's with Churchman and Ackoff, and were axiomatized by Debreu in 1960, along with Luce and Tukey, Krantz, and Scott.

The principles in these early papers and the continued development of multi-criteria decision making have now moved to nearly every domain where scarce resources, probabilistic outcomes, uncertainty of the impact of the decisions, and value at risk is high enough to ask the question

What will be the outcome of this decision on the future of the processes, cost, technology, environment, or any other external domain?

So when we hear that

  • Decision making frameworks for project that don't require estimates.
  • Investment models for software project that don't require estimates.
  • Project management (risk management, scope management, process reports) approaches that don't require estimates.

We have to wonder if these frameworks, investment models, and project management methods have come in contact with the reality of how business works. As Carl saying in a somewhat harsh manner

Extraordinary claims require extraordinary evidence. Carl Sagan

Related articles Positive Deviance 5 Questions That Need Answers for Project Success
Categories: Project Management

Focus on Value is Only ¬Ĺ The Equation

Fri, 09/05/2014 - 16:54

ValueWhenever I hear "we need to focus on value" it tells me that only ¬Ĺ the equation of business ¬†success is understood.

Absolutely value is needed and is the focus of our work efforts. No customer is going to pay for a product or service that doesn't have the needed value. This value is usually in a unit of measure around a "capability" to do something for the that in turn solves a problem, generates revenue, enables another process to function.

Measures of Effectiveness to get the job done. Measures of Performance while getting the job done. The Technical Performance Measures of the underlying hardware, software, people processes that enable the job to get done at the right cost.

But this value is an enabler. The units of measure of the value are defined by the customer, not the provider. The Marketing department of the provider may have suggestions about the value of the product or service, but it is the customer that confirms that value.

In doing the confirmation of Value, the buyer (internal or extermal customer) makes this assessment using a foundational principles of all business decisions...


Microeconomics is a branch of economics that studies the behavior of individuals and small impacted organizations in making decisions on the allocation of limited resources. Those limited resources to the buyer are:

  • Time - the need for the capability provided by the¬†Value produced by the developer usually has a time frame on it. If that¬†Value could arrive any time, then it is unlikely the¬†Value of the Value is very high.
  • Money - the other limited resource is money. How much does the Value cost?

The time value of money is then used by the buyer to enable another decision making process

ROI = (Value - Cost ) / Cost

So when we hear, we don't need to know the cost, that is we don't need to estimate the cots of producing the value, think again. This can only be true if the delivered value takes place in the presence of unlimited resources. That is we have all the time we need and no one really cares about the cost. That is the principle of microeconomics of business decision making is not in play.

As a final point. Budget is NOT Cost. Setting the budget, only sets the budget. The cost of the work is called Actual Cost and actual cost is generated by exchanging labor and materials for money. So setting the budget is needed. But setting the budget only says what the expected cost might be. If you're given an budget and an expected set of Value outcomes, estimates of the confidence of delivering that Value will be needed. Otherwise your budget is just a number, that will easily be blown before you delivery the needed capabilities on the needed date.

If asked what should our budget be for the expected Value, then solving the ROI or Expected Monetary Value, or Value at Risk needs to take place. In order to do this for a decision about the future ROI, EMV, or VAR, we need to estimate both the Cost and Value - then manage the project that produces the Capabilities that deliver the Value toward both those cost and time variables. And of course those variables are random variables.

When it is stated in an work shop where the participants will learn about

  • Decision making frameworks for project that don't require estimates.
  • Investment models for software project that don't require estimates.
  • Project management (risk management, scope management, process reports) approaches that don't require estimates.

Then those processes are not operating in a governance domain where microeconomics of product developments are in place. Can't say where they are operating, but it's not on this planet of a for profit business. Or those proffering this approach have discovered a secret sauce that gets around the basic principles of all business -

Profit = Revenue - Cost of Goods Sold

The only place I know this principle is not in place, is here in Colorado, just outside of Fairplay, CO, is South Park.


Here in South Park, the local businessmen have a clever plan to get rich without be subject to microeconomics of making decisions about how to do that.

Screen Shot 2014-09-05 at 9.39.44 AM

Categories: Project Management