Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
The notion¬†of the Cone of Uncertainty has been around for awhile. Barry Boehm's work in ‚ÄúSoftware Engineering Economics‚ÄĚ. Prentice-Hall, 1981. ¬†The poster below is from Steve McConnell's site and makes several things clear.
Here's a simple definition of the Cone of Uncertainty:¬†
The Cone of Uncertainty describes the evolution of the amount of uncertainty during a project. Uncertainty not only decreases over time passing, but it also diminishes its impact by risk management, specifically by decision-making.At the beginning of a project, comparatively little is known about the product or work results, and so estimates are subject to large uncertainty. As more research and development is done, more information is learned about the project, and the uncertainty then tends to decrease, reaching 0% when all residual risk has been terminated or transferred. This usually happens by the end of the project.
So the question is? - How much variance reduction needs to take place - in any and all the project attributes (risk, effectiveness, performance, cost, schedule - shown below) at what points in time, to increase the probability of project success.
This is the paradigm of the Cone of Uncertainty - it's a planned development compliance engineering tool, not an after the fact data collection tool
The Cone is NOT the result of the project's past performance. The Cone IS the Planned reduction of uncertainty as the project proceeds. When actual measures of cost, schedule, and technical performance are outside the planned cone of uncertainty, corrective actions must be taken to move those uncertanties¬†inside the cone of uncertanty, if the project is going to meet it's cost, schedule, and technical performance goals.¬†
The Measure that¬†is modeled in the Cone¬†the variable¬†is the Quantitative basis of a¬†control process that establishes the goal for the performance measure. Capturing the actual performance, comparing it to the planned performance, and compliance with the upper and lower control limits provides guidance for making adjustments to maintain that variables performance with acceptable limits.
The Benefits of the Use of the Cone of Uncertainty¬†
The planned value, the upper and lower control limits, the measures of actual values form a¬†Close Loop Control System - a measurement based feedback process to improve the effectiveness and efficiency of the project management processes by 
A critical success factor for all project work is¬†Risk Management. And risk management includes the managements of all kinds of risks. Risks from all kinds of sources of uncertainty, including technical risk, cost risk, schedule, management risk. Each of these uncertainties¬†and the risks they produce can take on a range of values described by probability and statistical distribution functions. Knowing what ranges are possible and knowing what ranges¬†are acceptable is a critical project success factor.
We need to know the Upper Control Limits (UCL) and Lower Control Limit (LCL) of the ranges of all the variables¬†that will impact the success of our project. With this paradigm we have logically connected project management processes with Control System processesIf the variances, created¬†by uncertainty going outside the UCL and LCL. Here's a¬†work in progress paper "Is there an underlying Theory of Project Management," that addresses some of the issues with¬†control of project activities.
Here are some examples of¬†Planned variances and managing of the actual¬†variances to make sure the project¬†stays on plan.
A product¬†weight as a function of the programs increasing maturity. In this case, the projected¬†base weight is planned and the planned weights¬†of each of the major subsystems are laid out as a function of time. Tolerance bands for the¬†project base weight¬†provide management with actionable information about the progression of the program. If the vehicle gets overweight, money and time are needed to correct the undesirable variance. This is a¬†closed loop control system¬†for managing the program with a Technical Performance Measure¬†(TPM). There can be¬†cost and¬†schedule performance measures as well.
Below is another example of a Weight¬†reduction attribute that has error bands. In this example (an actual vehicle like the example above) the weight must be reduced as the program proceeds left to right. We have a target weight at Test Readiness Review of 23KG. A 25KG vehicle was¬†sold in the proposal, and we need a target weight that has a safety margin, so 23KG is our target.
As the program proceeds, there are UCL and LCL bands that follow the¬†planned weight. ¬†The Orange dots are the¬†actual weights from a variety of sources - a Design Model (3D Catia CAD system), a detailed design model, a bench scale model that can be measured, a non-flying prototype, and then the 1st Flight Article). As the program progresses each of the weight measurements for each of the models through to a final article is compared to the planned weight. We need to keep these values¬†inside the error bands of¬†NEEDED weight reduction if we are to stay on plan.
This is the critical concept in successful project management
We must have a Plan for the critical attributes - Mission Effectiveness, Technical Performance, Key Performance Parameters - for the items. If these are not compliant with the plan will one of the Root Causes of program performance shortfall. We must have a¬†burndown or¬†burnup plan for producing the end item deliverables for the program that match those parameters over the course of the program. Of course, we have a wide range of possible outcomes for each item in the beginning. And as the program proceeds the variances measures on those items move toward compliance of the target number in this case Weight.
Here's another example of the Cone of Uncertainty, in this case, the¬†uncertainty is the temperature of an oven being designed by an engineering team. The UCL and LCL are defined BEFORE the project starts. These are used to inform the designer of the progress of the project as it proceeds. Staying¬†inside¬†the controls limits is the¬†Planned progress path to the final goal - in this case, temperature.
The Cone of Uncertanty, is the signaling boundaries of the¬†Closed Loop Control system used to manage the project to success
It turns out the¬†cone can also be a¬†flat range with Upper and Lower Control Limits of the variable that is being developed - a design to variable - in this example a Measure of Performance. In this case, a¬†Measure of Performance that needs to stay within the Upper and Lower limits as the project progress through its gates. If this variable is¬†out of bounds the project will have to¬†pay in some way to get it back to Green
A¬†Measure of Performance characterizes physical or functional attributes relating to the system operation, measured or estimated under specific conditions.¬†Measures of Performance are (1) Attributes that assure the system has the capability and capacity to perform and (2) Assessment of the system to assure it meets design requirements to satisfy the Measures of Effectiveness.
Another¬†cone style¬†is the¬†cone of confidence in a delivery date. This Actual case it's an Actual Launch date. In this case, as the program moves from left to right, we need to assure that the Launch Date moves from a¬†low confidence¬†Date to a date that has a¬†chance of being correct. The BLUE bars are the probabilistic ranges of the current estimate date. As the program moves forward those ranges must be reduced if we're going to show up as needed. The Planned date and a date with a¬†margin¬†are the¬†build to dates. As the program moves the confidence of the date must increase and move toward the need date.
Just a reminder again -¬†the Cone of Uncertainty is a DESIRED path, NOT the result of an¬†unmanaged project outcome.
Risk Management, as shown below, is how Adults Manage Projects
Wrap Up On the Misunderstanding of the Purpose and Value of the Cone of Uncertainty
When you hear...¬†
I have data that shows that uncertainty (or any other needed attribute) doesn't reduce and therefore the COU is a FAKE ... OR¬†... I see data on my projects where the variance is getting worse as we move forward, instead of narrowing as the Planned COU tells us it should be to meet our goals ...
...then that project is¬†out of control,¬† starting with a missing steering target¬†that means it's¬†Open¬†Loop Control¬†and will be late, over budget, and likely¬†not perform to the needed effectiveness and performance parameters. And when you see these¬†out of control situations, go find the Root Cause and generate the Corrective Act.¬†
This¬†data is an observation of a project not being managed as Tim Lister suggests -¬†Risk Management is How Adults Manage Projects.¬†
And if these¬†observations are taking place without corrective actions of the Root Causes of the performance shortfall, the management is behaving badly. Their just observers of the train wreck that is going to happen real soon.
The Engineering Reason for the Cone of Uncertainty Model and the Value it Provides the Designing Makers
The Cone of Uncertainty is NOT¬†an output from the project's behaviour, by then that's too late.
It's a Steering Target Input to the Management Framework for increasing the probability of the project's success.
This is the¬†Programmatic Management of the project in support of the Technical Management of the project. The processes is an engineering discipline. Systems Engineering, Risk Engineering, Safety and Mission Assurance Engineering, are typical roles where we work.
To suggest otherwise is to invert the paradigm and removes any value from the post-facto observations of the project's performance. At that point it's Too Late, the Horse has left and there's no getting him back.
Defining the¬†planned and needed variance levels at planned points in the project is the basis of the closed loop control system needed increase the probability of success.
When variances outside the planned variance appear, the Root Cause of those must be found and corrective action take.
¬†Systems Engineering Measurement Primer, INCOSE
¬†System Analysis, Design, and Development Concepts, Principles, and Practices, Charles Wasson, John Wiley & Sons
 SMC Systems Engineering Primer & Handbook: Concept, Processes, and Techniques,¬†Space & Missle Systems Center, U.S. Air Force
¬†Defense Acquisition Guide, Chapter 4,¬†Systems Engineering, 15 May 2013.
¬†Program Managers Tool Kit, 16th Edition, Defense Acquisition University.
 "Reducing Estimation Uncertainty with Continuous Assessment: Tracking the 'Cone of Uncertainty',"¬†Pongtip Aroonvatanaporn, Chatchai Sinthop, Barry Boehm.¬†ASE‚Äô10, September 20‚Äď24, 2010, Antwerp, Belgium.¬†
¬†Boehm, B. ‚ÄúSoftware Engineering Economics‚ÄĚ. Prentice-Hall, 1981.
 Boehm, B., Abts, C., Brown, A. W., Chulani, S., Clark, B. K., Horowitz, E., Madachy, R., Reifer, D. J., and Steece, B. Software Cost Estimation with COCOMO II, Prentice-Hall,
 Boehm, B., Egyed, A., Port, D., Shah, A., Kwan, J., and Madachy, R. "Using the WinWin Spiral Model: A Case Study," IEEE Computer, Volume 31, Number 7, July 1998, pp. ¬†33-44¬†
 Cohn, M. Agile Estimating and Planning, Prentice-Hall, 2005
 DeMarco, T. Controlling Software Projects: Management, Measurement, and Estimation, Yourdon Press, 1982.
 Fleming, Q. W. and Koppelman, J. M. Earned Value Project Management, 2nd edition, Project Management Institute, 2000
 Galorath, D. and Evans, M. Software Sizing, Estimation, and Risk Management, Auer-bach, 2006
Jorgensen, M. and Boehm, B. ‚ÄúSoftware Development Effort Estimation: Formal Models or Expert Judgment?‚ÄĚ IEEE Software, March-April 2009, pp. 14-19
 Jorgensen, M. and Shepperd, M. ‚ÄúA Systematic Review of Software Development Cost Estimation Studies,‚ÄĚ IEEE Trans. Software Eng., vol. 33, no. 1, 2007, pp. 33-53
 Krebs, W., Kroll, P., and Richard, E. Un-assessments ‚Äďreflections by the team, for the team. Agile 2008 Conference
 McConnell, S. Software Project Survival Guide, Microsoft Press, 1998
 Nguyen, V., Deeds-Rubin, S., Tan, T., and Boehm, B. "A SLOC Counting Standard," COCOMO II Forum 2007
 Putnam L. and Fitzsimmons, A. ‚ÄúEstimating Software Costs, Parts 1,2 and 3,‚ÄĚ Datamation, September through December 1979
 Stutzke, R. D. Estimating Software-Intensive Systems, Pearson Education, Inc, 2005.¬†
Related articlesComplex, Complexity, Complicated Economics of Software Development Herding Cats: Economics of Software Development Estimating Probabilistic Outcomes? Of Course We Can! I Think You'll Find It's a Bit More Complicated Than That Risk Management is How Adults Manage Projects
Overheard on Twitter
The Start of a Project Is the Worst Time to Estimate Its Duration or Cost
This is only the case if those you've hired know nothing about what capabilities are needed to produce value the project, what Features are needed to produce that Value, when those Value-producing Features are needed to meet the time cost of value payback process, what risks there are for meeting those value producing outcomes, and how the work effort to produce that value are to be measured (physical percent complete) to increase the probability of success for your¬†project. As the project progresses this understanding will, of course, improve with feedback, working product, and learning.¬†
If those you've hired don't have some sense of these needs, to some level of confidence, you've hired the wrong people.
From Estimatiing and Reporting Agile Projects with the SRDRRelated articles Managing in Presence of Uncertainty How We Make Decisions is as Important as What We Decide. Planning is the basis of decision making in the presence of uncertainty
Risk is everywhere¬†on projects. This risk comes from two types of uncertainty. Aleatory uncertainty, which is the naturally occurring yields variances in the underlying processes. This uncertainty is handled with cost, schedule, and technical performance margins. Epistemic uncertainty comes from probabilistic processes that can be addressed with handling responses.
The idea of risk and its management and handling is a critical success factor for all software development.
One of the most rigorous theorems of economics  proves that the existing means of production yields greater economic performance only through greater uncertainty that is, through greater risk. While it is futile to try and eliminate risk, and questionable to try and minimize it, it is essential that risk be taken be the right risks...
We must be about to choosing rationally among risk-taking courses of action, rather than plunge into uncertainty on the basis of hunch, hearsay, or incomplete experience, no matter how meticulously quantified.
- Peter Drucker (1975)¬†Management (From¬†The Principles of Software Engineering, Chapter 6, Tom Glib, 1988).
Managing in the presence of risk - and the uncertainty that creates the risk - requires we make¬†risk-informed decisions. These decisions are informed by the probabilistic and statistical outcomes of those decisions in the future. In order to make¬†risk-informed decisions, we must estimate the outcomes and the¬†impacts of those outcomes on future activities (cost, schedule, and technical performance of products and services). Without these estimates, there is no risk management. And as Tim Lister reminds us¬†
Without these estimates, there can be no risk management. And as Tim Lister reminds us¬†
Risk Management is how Adults Manage Projects. Be an adult, make estimates of the future outcomes of your risk informed decisions.¬†
¬†Control or Economic Law Paperback,¬†Eugen von Boehm-Bawerk, 2010.
 "How Much Risk is Too Much Risk," Tim Lister, Boston SPIN
 "Risk Management is How Adults Manage Projects," Susanne Madsen¬†
 "Risk Management and Agile Software Development," Glen B AllemanRelated articles Essential Reading List for Managing Other People's Money Risk Management is How Adults Manage Projects Herding Cats: Economics of Software Development Herding Cats: Managing Uncertainty, Risk, Threat, and Opportunity What is a Team? Logically Fallacious Friday Herding Cats: Risk Management Capabilities Based Planning Herding Cats: Pictures About Managing in the Presence of Uncertainty Intellectual Honesty of Managing in the Presence of Uncertainty
Risk Management is How Adults Manage Projects - Tim Lister
Risk Management requires making estimates of many things. The uncertainties that create the risk - reducible (Epistemic) and irreducible (Aleatory), the impacts from the risks, the efficacy¬†of the corrective actions, the residual reducible uncertainty, and any changes in the irreducible uncertainties.
Just like risk management, estimating is how adults manage projects. No risk management no adult managemnt. No estimates, no adult management. Since as that.¬†
Making decisions in the presence of uncertainty with limited resources is the domain of Macro and Micro Economics
Making decisions¬†in resource-limited situations¬†at the national or global scale is¬†macroeconomics. Macroeconomics is the study of how people make decisions influenced by tax rates, interest rates¬†foreign policy, and trade policy. Microeconomics is the study of how people make decisions on a personal scale and treats decisions that individual and organizations make. For example, about¬†which software to buy, which Features in the development backlog should¬†be implemented next, what prices to charge for products and services.
Software development is an exercise in microeconomics, since it deals with limited resources - time, cost, and what value is produced in exchange for the time and money.
Because of the limitations of resources, projects need to¬†operate¬†withing a¬†world¬†of limited resources, the uncertainties - both reducible and irreducible - that create risk, and the emerging attributes of all project work. This is the foundation for estimates. Estimates¬†with accuracy and precision values needed to make credible decisions. These estimates are critical to both developers and customers. Underestimating software engineering costs could result in management approving proposed systems that potentially exceed budget allocations, or underdeveloped functions with poor quality, or a failure to complete a project on time. Overestimating costs may result in too many resources committed to a project, or, during contract bidding, result in losing a contract and loss of jobs.
These estimates are used for generating requests for proposals, contract negotiations, scheduling, monitoring, and control.
Managing in the presence of uncertainty requires a Closed Loop Control process. Where¬†targets are set, work is performed, feedback received, corrective actions taken it¬†steer toward¬†the target. Without that¬†target and the error bands on the target and the processes used to steer, Close Loop Control will be ineffective - constantly¬†chasing a moving target, with understanding of what¬†could result, versus what¬†should¬†result.¬†
Accurate and precise estimates - with predefined¬†values for the accuracy and precision needed to satisfy the attributes of the Closed Loop Control system - are needed because:
Since uncertainty creates risk, managing in the presence of uncertainty is Risk Management. Since¬†risk management is how adults manage projects, ‚Ä† making estimates in the presecnce of uncertanty is adult management.
No Estimates? No Adult Management.
Here are three starting resources for Software Economics:
‚Ä† Tim Lister, Risk Management is Project Management for Grownups
¬†Related articles Pune: Bangalore man arrested in connection with brutal murder of software engineer Economics of Software Development I Think You'll Find It's a Bit More Complicated Than That Logically Fallacious Friday Complex, Complexity, Complicated Herding Cats: Managing Uncertainty, Risk, Threat, and Opportunity Essential Reading List for Managing Other People's Money
Increasing the Probability of Project Success
Simple in Theory, Complex in Practice
When we hear any suggestion about improving the probability of success of a project, there are some simple tests to confirm there actually is such a thing. This means having a set of Principles, Process, and Practices to test the suggestion against. Let's start with the Principles
Let's start with the Principles
These Five Immutable principles are time phased into Processes that provide answers to the Five Principles.
and the connections between each Process are made to form a Closed Loop control systems needed to manage any project.
With some details for each process area
To support these Principles and Processes, a set of Practices are needed
These charts are an extract from the book¬†Performance-Based Project Management: Increasing the Probability of Project Success¬†and the abstracted training materials Handbook.
I received a book over the Holidays -¬†Managing Project Risk and Uncertainty: A Constructively¬†Simple Approach to Decision Making, Chris Chapman and Stephen Ward. This is a seminal work on risk management in the presence of uncertainty. The introduction has this...
Uncertainty in the plan English sense is¬†lack of certainty has important implication for what can be acheived by organizations. All Management decision should take uncertainty into account. Sometime the implcations of uncertainties are¬†risk in the sense of¬†significant potential unwlecome effects on orgainzation performance.¬†
Success in the presence of uncertainty requires a process be followed. Here's a recommend¬†one:¬† Stage in the Decision Process Uncertainty About Monitor the environment and current operations within the orgainzaiton Completness, veracity, and accuracy of information received, meaning of information, interpretation of implications Recognize the Issue Significance of issue, need for action Scope of decisions Appropriate frame¬†of reference, scope of relevant organization activities who is involved, who should be involved, extent of separation¬†from¬†other decision issues ¬†Determine the performance criteria relevant performance criteria, whose criteria, appropriate metrics, appropriate priorities, and¬†trade-offs¬†between different criteria. ¬†Identify alternative course of action ¬†Nature of alternatives available (scope, timing, and logistics), what is possible, level of detail required, time available to identify alternatives. Predict the outcomes of courses of action Consequence, nature of influencing¬†factors, size of influencing factors, effects and interactions¬†between influencing factors (variability and timing), nature and significance of assumptions made ¬†Chose the course of action How to weight and compare predicted outcomes ¬†Implement¬†the chosen¬†alternatives How alternatives will work in practices Monitor and review performance ¬†¬† What to monitor, how of term to monitor, when to take¬†further actions
In order to make decisions in presence of uncertainty, we need to estimate all the partially¬†elements of the decision process. Withoitn these estimates thetre is no Risk Management. Without Risk Management, there is no SAdult management of the project.¬†
Risk Management is How Adults Manage Projects - Tim Lister
One of the principles of agile development is¬†self-organizing teams. ¬†Self-Organizing is a powerful process. But Self-Directed is not the same as Self-Organizing. On all but de-minimis projects, there is some external organization¬†that is defining what Done looks like at the business capabilities level. In Scrum, this is the Product Owner, who is a member of the team. The PO is defined as:
The keeper of the requirements. The Product Owner provides the ‚Äúsingle source of truth‚ÄĚ for the Team regarding requirements and their planned order of implementation. In practice, the Product Owner is the interface between the business, the customers, and their product related needs on one side, and the Team on the other. The Product Owner buffers the Team from feature requests that come from many sources, and is the single point of contact for all questions about product. The PO maintains the Product Backlog, sets the schedule for releasing completed work to customers, and makes the final call as to whether implementations have the features and quality required for release.¬†
But First a Statement of Principles
Without a set of principles, it's difficult to have a conversation about seeking a shared understanding of the problem and possible solutions for almosty any topic. So for projects that spend other people's money in the presence of uncertainty - Governance is a¬†means to establish those shared principles.
Governance is about Decision Rights. Specifying the decision rights and accoutability framework to encourage desirable behaviours in the developemnt and use of information technology and its supporting services. -¬†IT Governance: How Top Performers Manage IT Decision Risght for Superier Results, Weill and Ross, Harvard Business School Proess.
In this¬†Role¬†(like all members of Agile teams, the PO is a role, not a position) the business value stream is conveyed from the business to the development team through the Product Roadmap and Release Plan. With the Team's full contribution to these artifacts, the Product¬†Owner is Accountable to the business for delivering that value from the Scrum Team's outcomes. This is paradigm is usually found at the Enterprise level of software development. If you're working on a self-contained team, where the customer, PO, development team, and all supporting roles are all sitting at the same table, with a low ceremony around cost, schedule, or deliverable - you can stop reading now. You don't need governance, a Product Roadmap, and Release Plan. Just code and the person paying you will tell you what to do next.
But if you're on a team that gets its funding outside the team, then there is likely a governance process in place for how to spend that money. In this case, there are others who are Accountable for delivery working software. Not designing and developing the software. But those that have a business role for the use of that software to make money or provide a mandated service.This is the Team itself¬†as a collective, but there is a separation of concerns on any non-trivial project as well. The UX/UI designers are Accountable for developing human factors and compliance -
In this governance paradigm, the Team itself is still a collective, but there is a separation of concerns on any non-trivial project as well. The UX/UI designers are Accountable for developing human factors and compliance - 508 Compiance for example - as well as compliance¬†with the current or future¬†UX/UI processes. The Database performance lead on the team is accountable¬†for assuring the code and web service maintain the needed database performance. The Cyber Security lead is Accountable for assure the work of the developers adheres with NIST Cyber Security Framework when the system is public facing with controlled content. The Architect is accountable for assuring the code developed by the Team is compliant with the established Architecture.¬†For example TOGAF or in the DOD, DODAF, or in some other domains industry architectures. IEEE 1553 for real-time embedded system.
In these examples, there is usually some overarching governance process. If yu have no governance process, then none of this accountability¬†discussion is applicable - do what you want not one cares. But if someone does care, usually the customer and those paying, then the notion of Accountability¬†is the basis of project success.
The Responsibility Assignment Matrix
First, there are many alternatives to RACI, so this post is about a Principle and in this case a Practice and Process. On projects where governance is in place, a Responsibility Assignment Matrix (RAM) is a means of defining who is participating in what¬†Role on the project. The RAM defines the single points¬†of accountability for project¬†Leadership Team.¬†These assignments start by identifying who is accountable for which project roles before the project starts. As the project matures, a delegation of these responsibilities down to the project team members using the RACI tool.
In an enterprise project¬†here's an example of the locations for Accountability¬†at the enterprise.¬†
¬†For agile programs, replace the PM with the PO and the diagram above remains the same. From this business governance process, we can build a RAM.
The RACI paradigm should actually start with Accountability - but ARCI which isn't as snappy. RACI provides the means to¬†flow down¬†responsibility from the Accountability. There cannot be multiple people Accountable, but there can be multiple people Responsible. Without a single point of integrative responsibility, it's not clear who gets to say what about the spending of other people's money. Again if you have no governance process, spend as you wish, do as you wish, no one care. But if there is governance, one place for developers to looks as to¬†WHY governance is needed (rather than saying it's a waste) is Essentials of Managerial Finance, Besley and Brigham.¬†This book explains¬†why and¬†how to manage other people's money when producing products or services in exchange for that money.¬†
In The End
If it's your money or maybe your parents money, no one carees how you spend it. But if it's not your money - investors, relatives, the firms money, the stockholders money, the owners money - then some form of governance is likely needed. With this governance paradigm in place, some structure around¬†who gets to decide how to spend that money is likely needed. With that need in place RACI (or its relatives) is a way to have a conversation about¬†who, what, when, and why descioons can be made.
The governance process is based on 3 pillars:
This is the basis of Governance - It's About Decision Rights.¬†
No need for decision rights? Spend away
 COBIT 5 ISACA's new Framework for IT Governance, Risk, and Security Auditing: An Overview
 The Role of ITIL in IT Governance: Leveraging IT Governance around IT Service ManagementRelated articles Essential Reading List for Managing Other People's Money The Art of Systems Architecting Architecture -Center ERP Systems in the Manufacturing Domain
For projects at scale, meaning the success or failure of the outcome impacts the firm in ways that cannot be corrected if it fails - loss of business, non-recoverable sunk cost, or other unfavorable impacts, there is usually a formal¬†governance¬†process for managing the project, the funding, and the outcomes in a manner that provide visibility to the project progress to plan to the highest levels of the organization.
Many of these Enterprise IT projects apply agile methods. Just as many of the Agile projects in this domain misuse the ¬†Definition of Done. Used as an excuse for not having a plan. Or as an excuse for not defining tangible evidence to needs to be produced in exchange for the investment. Here's a collection of PP&C processes¬†that impact the probability of program success.
Here's the framework for applying Program Planning and Control to Enterprose IT projects as well as the other projects and programs where it is currently used.
Let's with a basic idea. All work on projects is uncertain. Reducible uncertainty and irreducible uncertainty. This uncertainty is applicable to all work elements no matter the development method, the system architecture, or any other attributes of the project. Uncertainty is universally applicable to everything¬†we do.
Some would¬†suggest¬†that haveing no dependencies¬†would remove the impact of uncertainties.¬†But that is no true on any non-trivial project. For example in an enterprise¬†system, like the one below, the needed capabilities to provide value to the business have a logical order. This is a health insurance provider enrollment¬†system. We can have the¬†shared group matrix¬†reports¬†and interfaces until we have¬†a demonstration¬†of the conversion process and member reconciliation. Order, predecessor, and successor relationships¬†are part of all development work.¬†
Let's start with some simple statistics. These uncertainties¬†come from the underlying probability and statistics of the work processes. It's important to separate probability from statistics. Both are needed, but they are not the same. And more importantly, they have different impacts on the project. We must learn about the behaviors of both. Reducible uncertainties¬†come from probabilities. Irreducible uncertainties come from statistical processes. One we can do something about and other we must have margin because they are¬†irreducible.
If you hear about some probabilistic or statistical process, there are some terms that are useful. Here's one - a Probability Distribution Function. This tells us the¬†probability that some value will appear. In this example, those values range from 0.0 ¬†to 5.0. As managers, we may only be interested in 80% of the possible value. This 10/90 phrase says that numbers from the 10th percentile to the 90th percentile are the ones we're interested. If there is possibility¬†that a value in the less than 10th percentile or greater than the 90th percent¬†was to appear, we'd need to consider that an outlier or a value that must be prevented from appeared by some means
As PP&C people we now need to put these concepts to work. The first question to answer is¬†what are the behaviors of the schedule of work. We start with the schedule, rather than cost, because most every non-trivial projects have some deadline to start earning back the invested cost. It's not that we're not interested in cost, but we can usually go beg for more money. We get to beg for more time in any serious project. Why is this true, simple¬†time cost of money. Those investing in the project¬†have a need to start making money from their investment. Or, in many cases, those investing in the project have some external dependency - a¬†launch date for a product. Or a physical¬†launch date - a literal launch date. You can only go to Mars in a small window (weeks) every 3 years. Miss that date, you have to wait - usually at your own cost.
So here's a simple project example. This is from a larger briefing where we use the Wright Brothers contract to deliver a¬†flying machine on a specific date for a specific amount of money. The contract for this machine is 3 pages and calls out dates, cost, measures of effectiveness, measures of performance, and key performance parameters. They had already flown many many times, this was a procurement contract for a production machine. That date was¬†
That date was August 14, 1908. Orville and Wilbur were probably not professional Program Planning and Cost analyst, but they knew they to have a¬†credible schedule complete with risk management and risk buy down, margins for that cost and schedule as well as margins for the technical performance of the product. Just like any¬†credible¬†plan for any non-trivial development effort.¬†The Wright Brothers is a good place to learn how it was actually done, and learn that many of things we learning in grade school are not factually correct. The contract¬†Signal Corps Specification No. 486 calls out the details. Wilburn and Orville had he needed margins - developed from¬†reference classes of past flights, experiments, models, prototypes, and intuition.¬†
In our current domain, the primary tool used to develop the needed margins, assess the impacts of risk reduction activities, find¬†blocking steps and a variety of other reducible and irreducible uncertainties is Monet Carlo Simulation. These simulations are applied to the three core attributes of all project work - time, cost, and technical performance and their interactions.
Here's what that looks like for the¬†planned completion of Work Package #3 on March 12th, 2012. The chart shows there is a 60% probability of completing¬†on or before the need date. If this is acceptable, then fine. If not we need a better plan.
One of the roles of PP&C is the develop models for the cost and schedule of the project. Then interact with the engineers to assess the plan for¬†increasing the maturity of the deliverables and how hat maturing assessment will impact the cost and schedule when that maturity is not being delivered to plan. This is the basis of a Closed Loop Control system.
Without a plan for delivering the needed capabilities at the needed maturity on the needed date for the needed cost, there is no way to have a control system that will provide¬†actionable information to the decision makers.
¬†While these charts are notional in nature, here's a real one for dependencies of a large software-intensive system of systems for a flight vehicle. This chart is blurry enough to not be recognizable, but it is real and it represent several billion dollars of work over 5 years
So In the¬†End
This is what Program Planning and Controls does. It does it on large space flight programs. road building, power plant building, and most of all on Agile Software Development projects. Not you're 5 guys at the same table with their customer and an open check book projects. But large software development project (ours are typically $20M or greater up to Billions) with a deadline for working software - all the individual Sprints, Release up to Full Working systems with the needed capabilities to accomplish the mission.
If you have no deadline, no¬†not to exceed¬†budgets, no mandatory capabilities to meet the mission need, no defined benfist from the project, then none of this is usefulRelated articles Making Decisions In The Presence of Uncertainty Intellectual Honesty of Managing in the Presence of Uncertainty The Flaw of Empirical Data Used to Make Decisions About the Future Making Decisions in the Presence of Uncertainty Managing in the Presence of Uncertainty
To increase the probability of project success, many things have to happen at the same time. Here's are Five principles and practices that can increase that probability of success
Principles and Practices of Performance-Based Project Management¬ģ from Glen Alleman Related articles Managing in Presence of Uncertainty Applying the Right Ideas to the Wrong Problem Good Project and Bad Project
I'm working an Implementation Review (IR) of a major space flight vehilce, that includes Software Intensive System of Systems.¬†
Here are the guidelines for¬†a credible cost estimate (GAO-16-620)
These guidelines are comprehensive and not likely¬†to cover most projects or programs. But it's good to ask how many of these attributes can be found in your cost estimate?
¬†Related articles Information Technology Estimating Quality Managing in the Presence of Uncertainty Want To Learn How To Estimate? Five Estimating Pathologies and Their Corrective Actions The Art of Systems Architecting The Fallacy of the Planning Fallacy Architecture -Center ERP Systems in the Manufacturing Domain IT Risk Management
Most people know nothing about learning; many despise¬†it. Dummies reject as too hard whatever is not dumb - Thomas More,¬†Utopia
The Fallacy - We can't know much of anything about the Future. But in fact, the future is always knowable to some degree of precision and accuracy unless it is truly Unknowable
¬†Software developers and the IT managers they work for operating in the¬†future. Many of this practitioners¬†view the future as an esoteric, abstract, impractical realm, But the future is where the value of the software is earned. The future¬†is where to cost to develop that software is paid back. The future is where the users of the system will be satisfied with the provided capabilities.
The primary¬†job of management and those developing value for management is to find the future, not just the future in general, but the specific¬†futures for the customers. - Al Ries, in¬†Positioing the Battle for Your Mind.
¬†This future is not in the resulting products and services but the cost, schedule, and technical attributes needed to produce these products and services. Knowing this future can be on large-grained¬†boundaries - sometimes called waterfall. Or on fine grained¬†boundaries¬†- sometimes called spiral, incremental commit and agile.
All Project¬†Work Operates in the Presence of Uncertainty
All work¬†even production line work operates in the presence of uncertainty. Uncertainty comes in tow forms - Aleatory and Epistemic
Managing in the presence of these two uncertainties¬†requires one or two approaches:
Now to the Logical Fallacy
In the presence of Aleatory and Epistemic uncertanty, risk is created to the cost, schedule, and technical performance of the project. To assess the impact of that risk, devise the protective actions needed to address the resulting risk - ESTIMATING is required. Without estimates the Aleatory and Epistemic uncertanty that exists on All project will go unaddressed and the probablity of project sucess will be significanlty reduced, perhaps to Zero.¬†
Without estimating both the probability of occurance (for reducible uncertanties) and the statistical processes (for irreducible uncertanties) informed decsions cannot be made.
The falalcy is that decsions can be made in the presence of these uncertanties - that exist on all project work - willfully ignores these principles.
There are two type of errors made, of which this fallacy¬†adheres:
¬†Software Cost Estimation with COCOM II, Barry Boehm, et al
¬†The Incremental Commitment Spiral Model, Barry¬†Boehm, and Jo Ann Lane
¬†The Economics of Iterative Software Development, Walker Royce
¬†Facts and Fallacies of Software Engineering, Robert Glass
 Distinguishing Two Dimensions of Uncertainty, Craig Fox and G√ľlden √úlkumen, in Perspectives of Thinking, Judging, and Decision Making
¬†Decision Analysis for Professionals, Peter McNamee and John Celona
¬†Related articles Economics of Software Development I Think You'll Find It's a Bit More Complicated Than That Herding Cats: Estimating is a Learned Skill Complex, Complexity, Complicated
The suprising discovery of Newton's is just this, the clear seperation of laws of nature on one hand and initial conditions on the other - Eugene Wigner, in¬†Newton's Principis for the Common Reader, S. Chandrasekhar
There are 5 immutable principles of project success. Without the initial conditions of these five principles, the project has little chance of success.
All project work is uncertain. Uncertainty comes in two types - Reducible (Epistemic) and Irreducible (Aleatory). These uncertainties create the risk to the success of all projects. Without managing in the presence of risk, the probability of project success is significantly¬†reduced, most likely reduced to zero.
First a definition
A risk is an issue or event that could prevent a program or project from meeting its technical, schedule, cost, or safety objectives.
Management in the presence of risk has the following steps: 
For this process to work, each activity - in the presence of uncertainty - must be estimated.¬†
So in the end, if we're going to be adults when managing projects, especially projects funded by other people's money, we need to act like adults and estimate. Without estimates of the uncertainties, the risks created by those uncertainties, the effectiveness of our risk handling processes -¬†research, accept, watch, or mitigate in the NASA paradigm, the effectiveness of the controlling processes, and even the effectiveness of the communication processes - there will be little chance of success for our risk management process.
Let's change Tim Lister's quote¬†and call it as it is
Estimating is how adults manage projects. No estimates No adult management
and the story in our neighborhood when our sons were in the Scouts...¬†
what's the difference between our organization and the Boy Scouts? The Boy Scouts have adult supervision.¬†
 NASA's approach to Continuous Risk Management, described in "NASA's Management of the Orion Multi-Purpose Crew Vehicle Program," September 2016Related articles Bayesian Statistics and Project Work Making Decisions in the Presence of Uncertainty Making Decisions In The Presence of Uncertainty Five Estimating Pathologies and Their Corrective Actions The Use, Misuse, and Abuse of Complexity and Complex Capabilities Based Planning Just Because You Say Words, It Doesn't Make Then True
The Due Date doesn't mean the Do Date
If you have a Due date, you need the plan to show up on that date, with what is Due for the cost of that Due Item and have that Due Item be compliant with the attributes (effectiveness and performance) needed by those paying you.
I was asked to look at a corrective¬†action plan this year, that had a list of deliverables¬†that were supposed correct the root causes of the problems¬†identified¬†as the reason the project was not performing as planned. The right two columns¬†were the person performing the work and the DONE date.
A big smile came to my face.¬†I see the real root cause of your troubles here.¬†You don't know what Done looks like. That's the first immutable principle of project success.¬†
Without knowing what Done looks like in units of measure meaningful to the decision makers, the project has little hope of success.
20 Logical Fallacy Found in Agile and related topics
Those making the original¬†assertion need to bring evidence. Without that evidence, those challenging the assertion have no need to bring evidence. For those making the assertation to ask for evidence¬†as to why their conjecture is correct is the¬†Burden of Proof fallacy, which is common when the original¬†conjecture has no basis in principle, fact, or experience¬†- it's just a conjecture. All attempts¬†by the original¬†assertion author to invert this fallacy should be strongly¬†rejected, since it shows a lack of understanding of who ideas are put forth and tested in the larger marketplace of ideas.Related articles Making Conjectures Without Testable Outcomes Are Estimates Really The Smell of Dysfunction? Why Guessing is not Estimating and Estimating is not Guessing Architecture -Center ERP Systems in the Manufacturing Domain IT Risk Management