Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
In a CrossTalk article Risk Management for Dummies, Tom DeMarco speaks about late software projects and the approaches for the solution. The notion of¬†early start as the solution for¬†late finish needs to address the core question, but fails to address the root cause of¬†lateness - no schedule margin.
How soon should we have started to finish¬†on or before¬†the needed finish time?
The answer to this is simple:
Good question. And of course like all questions about some future activity we'll need to¬†Estimate those values. We can't actually manage - in any credible manner - without estimating. Anyone suggesting otherwise has likely only encountered trivial projects where the¬†Value at Risk¬†was low enough that no one cared is you were late or over budget.
An Example of Managing In The Presence of Uncertainty
Columbus proposed a voyage to Ferdinand and Isabella in 1486. The monarchs promptly set up a special commission of¬†learned men to study the proposal, which they found vague and arcane. After four years of unsatisfactory discussions with the master mariner, the commission's finally rejected the proposal.
To their credit they did not abandon Columbus and soon¬†recalled Columbus not to plan again, but to name his price for carrying out the voyage. The price was enormous. Columbus insisted on being knighted, appointed a grand admiral and viceroy, and given the unwillingness to compromise, the king and queen dismissed Columbus. Eventually, they relented and accepted all of Columbus's demand.
I (Norm Augustine) put forth this example because all parties involved could have benefited from risk management. Columbus could have presented his proposal containing each element of risk, but consider the payoff, well worth the investment.
Ferdinand and Isabella could have assessed the "price of doing nothing." The commission of experts could have have a framework in which they could evaluate the¬†technical¬†feasibility of the proposal. If the parties had studied the technical risk management, perhaps we'd be celebrating 1486, not 1492
For an simple schedule, like the Wright Brothers meeting their contractual data to the U.S. Army of August 31 1908 for the delivery of the Wright Flyer, they needed a confidence level of 80% for the¬†on or before¬†date. The schedule margin of 10 days provided the¬†protection for that date.
What Does This Mean?
The first meaning, missing from the CrossTalk article is.
Schedules without margin are late on day one
It means¬†Estimating is Risk Management. No estimates, no risk management. No risk management, lower probability of project success. For schedule risk we need¬†irreducible and¬†reducible¬†uncertainties that drive the schedule. The for the¬†reducible uncertainties, we need activities to reduce the risk from that uncertainty. For¬†irreducible¬†uncertainties, we need margin to protect each major deliverable.
Risk Management is How Adults Manage Projects - Tim Lister
Remember that next time you hear¬†we can't estimate, we don't want to estimate, estimates are a waste.
Here's How to Manage in the Presence of Uncertainty
"We suck at designing good database schemas." "I can never figure out how to configure the IIS VPN tunneling server." "This Sharepoint Server UI in SP 2012 is so much different than 2007, I can't possibly figure out how to help my customer."
Or, this 401(k) calculation for estimating my retirement is just beyond my ability to comprehend." Or, I can't possibly figure out how much money I need for my summer vacation with my family on a European trip, with relatives I haven't even met, and hotels and rental cars, train tickets, and meals, and other expenses, and airline reservations, and all the unknown and may be unknowable stuff. I'm just going to sit here and mope. Yep, that's me, Eeyore, woo is me. ¬†
What is it in some software domains that allows well educated, experienced developers to simply give up, walk way, allow themselves the be subjected to Dilbert Bosses? When they'd never talk that way about something they are skilled, experienced, and capable of doing?
It's a repeating theme in the #NoEstimates domain. You'd never hear a Flight Avionics engineer at a major space and defense company here in Denver say "we suck at estimating, we suck at testing, we suck at systems integration." Those engineers are asked all the time to "invent new physics to solve a problem." Or other colleagues literally inventing solutions of complex oil & gas software based process control algorithms. Or other colleagues at a major consulting firms deploying ERP and encountering legacy systems they've never seen or integrated before, saying "we can't figure this out, it's too hard and our managers will take our estimates as commitments, so we're simply not going to estimate our work."
Is it missing the engineering discipline taught in college. Where a probability and statistics class is mandatory for all 4 years to be called an engineer. Or even in the business and finance discipline estimating, using complex approaches - time series analysis, principle component analysis, bayesian statistics - is mandatory to be called a finance major. Or in the sciences in general. I say this from direct experience with our children and their significant others who are in finance, bio-sciences, and mechanical engineering all writing software for money as part of their actual profession.
And this whole notion that somehow writing software is not engineering is simply BS. It is engineering in many business domains. So maybe it's a domain issue. Maybe non-engineering cultures can't figure this out.¬†
Or is it maturity? Many appear to be very mature developers? Or is it culture, not geographic culture, but lack of having experienced a development culture where business and engineering work together to determine risks, corrective actions, increasing maturity of the deliverables development processes that allow for growth in both technical and business capabilities - all using agile by the way.
Everything is Uncertain
Business operates on risk taking. Risk taking is about making decisions in the presence of uncertainty. Uncertainty is about probable outcomes - positive or negative - in the future. To make decisions about uncertain outcomes in the future, we need to estimate those outcomes, the cost to achieve them, and the possible benefits from that cost.
No credible business person fails to understand this.
If they do, they won't be in business for long. It's taught in school - business school. Same for engineering and computer science school.
What is it that brings smart, capable, intelligent people to say "we suck," when the facilities (classes, books, articles, tools, external advisors) for learning how to estimate for the required reasons of making decisions about other peoples money for future outcomes, is the very basis of all business - is at hand, with simple learning and practice. These facilities are also the basis of all learning. "We suck" says "we're not capable of learning."
Are those of the #NoEstimate ilk saying "I can't learn?" or are they saying "I don't want to learn?" Because those providing them the money have a vested interest in knowing how much money is needed to produce that treasured¬†Value they claim to be producing.¬†
Just like learning a programming language. Moving from SP 2007 to SP 2012 (that actually sucks), dealing with cyber security. We don‚Äôt say ‚Äúwe suck‚ÄĚ and give up. What is actually going on here? Is some portion of the development community become Eeyore?Related articles Your Project Needs a Budget and Other Things Building the Perfect Schedule Don't Manage By Quoting Dilbert Qualitative Risk Management and Quantitative Risk Management Estimating is Risk Management
Risk Management is about many things - but it is first and foremost about estimating future outcomes.¬†
Suppose one of you wants to build a tower. Will he not first sit down and estimate the cost to see if he has enough money to complete it? For if he lays the foundation and is not able to finish it, everyone who sees it will ridicule him saying, "This fellow began to build and was not able to finish." - Luke 14:28-30
To manage the risk of¬†not enough money, not enough time, the probability of unacceptable outcomes we need to estimate. It's as simple as that and as complex as that.
Risk management is essential for development and production of products and services because key information about cost schedule, and technical performance are uncertain and many times unknown until later in the project.
Proceeding to spend other peoples money in the presence of these uncertainties is not only bad management, it's bad economics - the microeconomics of decision making requires estimating the impacts of any decision - the¬†opportunity cost¬†of that decision.
Risk management is concerned with the outcome of future events, whose exact outcome is unknown, and with how to deal with these uncertainties as a range of possible outcomes. -¬†Effective Risk Management: Some Keys to Success, 2nd Edition, Edmund H. Conrow.
So when you here¬†we can make decisions about the future without estimating ask yourself, did that speaker ever read about risk management. And as Tim Lister says
Risk Management is How Adults Manage ProjectsRelated articles Probability and Statistics of Project Work Qualitative Risk Management and Quantitative Risk Management Decision Making in the Presence of Uncertainty All The World's A Random Process Your Project Needs a Budget and Other Things
Plans are critical, a schedule to implement that plan comes next. With the Plan, the sequence of the work is needed to assure the value to the customer is delivered in the proper order. That order is an order because the business (or the mission) usually can't accept¬†all the features and functions at once. So the Plan is the top level sequence, and the schedule is the next level down.
Building the perfect schedule (v6) from Glen Alleman Related articles Your Project Needs a Budget and Other Things Qualitative Risk Management and Quantitative Risk Management
When we read about a big IT problem, like...
The first impulse is to use this information to support some or other position, like¬†here's an example of estimate driven bad management. Without the logical conclusion of finding the actual Root Cause of the problem, as shown in the IDA report. Other examples usually start with bogus Standish data. Take a look on page iv below to see the real root cause, and see if¬†Not Estimating¬†would have been able to address the issues with ECSS? Not likley.¬†
This document seems to not load everytime, refresh the page if it doesn't
So we're back to the same place we always seem to come to. Domain and knowledge of the domain, before conjecturing any solution to any problem and the conjectured solution that occurs outside the domain of experience.¬†
For those not able to read the details here's a summary from the final section.
The notion of some that "estimates" are the root cause of the problems and that making decision in the absence of Estimates¬†is the solution to the problem based on¬†un-informed opinion.
So down load the report, read it in it's entirety, then assess other opinions in the light of actually reading the Root Cause Analysis, then drawing conclusions from the actual data and its analysis by the professionals tasked by the sponsors of the RCA who are accountable for looking after the money and the measurable outcomes.
And then the quote that says it all...
"Everyone is entitled to his own opinion, but not his own facts." -¬†Daniel¬†Patrick Moynihan¬†
¬†Related articles Taxonomy of Logical Fallacies
Many times I hear about¬†Cost of Delay,¬†Deliver Value,¬†Measure story points,¬†or¬†Measure Stories. And a myriad of other assessments of project performance, all of which - OK, most of which are examples of Open Loop Control.
Back in 2014, we had a paper in a publication of the¬†College of Performance Management,¬†starting on Page 17. As well, a colleague Nick Pisano (CDR US Navy Retired) has a post on the same topic at his blog.
The notion of a¬†baseline let alone a Performance Measurement Baseline is at the heart of¬†Closed Loop Control of all processes, from your heating and air conditioning system in your house, to the flight controls on the 737-700 winging its way back home to Denver, to the project you're working - using what ever project management method or software development method of your choosing.
The notion that we can manage anything, the temperature of the room, the¬†nice soft ride¬†in the 737, or the probability of showing up¬†on or before¬†the need date,¬†at or below the needed cost, with the needed capabilities - and NOT have a baseline to steer to is simply wrong.¬†
Below is the framework for Closed Loop control. This paradigm says simply:
ESTIMATING IS THE BASIS OF DECISION MAKING - it can't be any clearer than that.
Control systems from Glen Alleman Related articles Your Project Needs a Budget and Other Things The Actual Science in Management Science Probability and Statistics of Project Work What is Governance? Don't Manage By Quoting Dilbert
In project work we're looking to create or change something, in some defined period of time, for a defined cost. This means we're going to spend money now for some future outcome. The elements that go into this effort to produce some change in the future include (but are not limited to) scope of our efforts (requirements for the outcomes), technical performance (including quality and other ...ilities of the outcomes), the schedule for the work (so we don't have to do everything at once), the budget so we know the cost of the value produced), resources that do the work in exchange for money defined in the budget, risk to cost, schedule, and technical performance goals, and other attributes. A specific project will have specific constraints from each of these attributes.¬†
The relationships between these attributes is usually non-linear, random in some way (stochastic), and affects future outcomes. Because of the random nature of the attributes and the random nature of their relationship, simple linear, non-statistical projections of past performance used for future performance is most likely to be a disappointment.
To answer the question¬†what does the future look like when the past is a non-linear stochastic process, we need to be able to¬†manage in the presence of uncertainty. With this ability, the future simply¬†emerges and many times this futute is not what was expected.¬†
This is the role of planning. The best description of planning is
To be successful at planning we need to do many things. Since it is the future we're planning for each of these things requires an
Yes, we're never going to see it coming if we don't Plan. And to Plan, we need to estimate. And to estimate we need to learn how to estimate. So if we want to manage in the presence of uncertainty, here's a starting point...
When managing in the presence of uncertainty we're going to need to make estimates of the impacts of that uncertainty on our decision making processes for that¬†emerging future. When we hear we can make decision in the absence of estimates, it's simply not true in any non-trivial manner. For trivial decisions,¬†Do I start with the enrollment UI or the validation of IRS information UI, or¬†do I buy a Mac with 4GB or 6GB¬†of memory (assuming I can upgrade if I need more), making estimates is likely is little value. But for a decision like,¬†do we switch all our legacy systems to SAP or JD Edwards, I going to need a credible estimate of the cost, schedule, resources, risks, and tangible beneficial¬†outcomes for my enterprise.¬† So without a domain, context, the¬†Value at Risk, the underlying processes for uncertainty (reducible and irreducible) and some other attributes, the notion of making decisions in the absence of estimating the cost and impact of those decisions in simply bad management.¬† For Those Interested in Decision Making Processes in the Presence of Uncertainty
¬†Related articles Bayesian Statistics and Project Work Decision Making in the Presence of Uncertainty Your Project Needs a Budget and Other Things
When we hear descriptons of Dilbert style management, with posting and reposting Dilbert cartons showing pointy haired bosses Doing Stupid Things on Purpose and no actionable advice on how to take corrective actions - then you know for sure the poster is just whinning - move on nothing of value here.
Addressing programs we all encounter in the project management work, means answering - with the % Whys - real 5 whys, not just the suggestion of the 5 whys - the symptom, the problem, and the root cause in the broad categories of:
Only with the answers to these categories in the three classes of assessment - what's the symptom, what's the problem that that produces the symptom, and what is the root cause of the problem - can corrective actions start.
None of this available - the conversation is a non-starter.Related articlesWhat is Governance?Taxonomy of Logical FallaciesYour Project Needs a Budget and Other ThingsThe 5 Whys Process We Use to Understand the Root of Any Problem
The use of Bayesian Statistics in project work is well developed. Here's an example of the approach. The drivers for each uncertainty can be modeled to produce on estimate of the¬†risk of the project not producing the desired outcomes.
What is needed is an understanding of the¬†prior probabilities of the drivers of the probabilistic outcomes, the relationships between the events created by these networks.
The Bayesian Network Approach ‚Ä†
Bayesian networks provide decision support processes for a wide variety of problems where uncertainty and probabilistic reasoning is involved. The Bayesian Network is a¬†directed graph with associated probability tables. The graph is the standard¬†nodes and arcs. The nodes represent uncertain variables. Each node has a set of states that represent causal or influential relationships between the variables. There is a probability table for each node, providing the probabilities of each state of the variable.
For prior variables - variables without parents the table contains¬†marginal probabilities. This is referred to as the¬†prior distribution which represents the prior belief - the state of knowledge - for that variable. For each variable with predecessors (parents), the probability table has the conditional probabilities for each combination of the predecessor states. This is the¬†likelihood function that represents how likely is a state of a variable given a particular state of its predecessor.
Bayesian Network are applied in situation that require statistical inference. The use in this instance knows some¬†evidence - some variable states or events that have¬†actual¬†observations. These observed values represent the¬†posterior¬†probability of the event occurring. By applying Reverend Bayes rile in each affect node, they can influence other Bayesian Network node by propagating and modifying the probability distributions.¬†
Bayesian Networks Have A Natural Connection With Project Planning
These networks can:
Bayesian Networks is a tool for decision support based on¬†Estimating¬†outcomes.
So when we hear that decisions can be made in the absence of estimates, ask for tangible examples of how this can be done, the basis (mathematics) of these processes, and examples in specific domains of how this can be done. If no answer is forth coming, question the veracity of the claims.
‚Ä† "Inference in Hybrid Bayesian Networks using dynamic discretisation," Martin Neil, Manesh Tailor,¬†and David Marquez,¬†Department of Computer Science, Queen Mary, University of LondonRelated articles The Actual Science in Management Science Qualitative Risk Management and Quantitative Risk Management What is Governance?
When I hear¬†what are the odds of success for a project, it tells me there is an undertstanding gap in how statistical processes are used in projects to produce probabilistic estimates of three things:
First let's echo Tim Lister's advice...
Risk Management is How Adults Manage Projects
All the World's a Statistical Process
First let's look at a network of work activities. These are tasks with dependencies, whose durations have naturally occurring variances. These durations can never be an exact number, since the work is emerging or simply varying naturally. Each activity has a unique Probability Distribution Function, which may be similar, but also unique. This is the case as well when there are no dependencies. The probabilistic processes are still in place. Even some place like the Toyota production line, no work process takes exactly the same duration. So if these natural variances are unaccounted for, you're going to be late, likely over budget, and your favorite gadget may not work. This concept is the basis of Statistical Process Control.
For each process, the upper and lower ranges, along with the Probability Distribution Function can be used to model the range of possible outcomes for duration - for example. In the picture below the probability of completing¬†on or before for the IOC (Initial Operating Capability) for Friday October 23, 2020 is 80%.
When we hear¬†the bet we'll¬†be successful, this is the number. It's not the right term¬†the bet but this is the term. There is an 80% confidence of completing¬†on or before 23 Oct 2020.
We can now¬†connect the dots between individual activities and a network of activities with the next chart. This shows the dependencies, each of their variances and how those drive the variances in the outcomes.
In the End the Discussion is About Domain and Context
When we hear about some new approach to making decisions in the absence of estimating the impact from those decisions, ask in what domain can that be possible. By possible I mean how can we make a decision by ignoring the principles of Microeconomics.
There may be domains in which that is completely possible. Below is a scale of projects I built awhile ago when working on an overall Program Governance engagement. From family gardening to building the USS Virginia there is a huge spectrum of techniques, processes, governance, tools, and approaches to increasing the probability of success. Having any discussion about the applicability of any idea has to start with¬†what domain are we in.Qualitative Risk Management and Quantitative Risk Management The Actual Science in Management Science What is Governance?
Qualitative risk analysis includes methods for prioritizing the identified risks for further action, such as risk response.
The project members must revisit qualitative risk analysis during the project‚Äôs lifecycle. When the team repeats qualitative analysis for individual risks, trends may emerge in the results. These trends can indicate the need for more or less risk management action on particular risks or even show whether a risk mitigation plan is working.
Quantitative risk analysis is a way of numerically estimating the probability that a project will meet its cost and time objectives. Quantitative analysis is based on a simultaneous evaluation of the impact of all identified and quantified risks, using Monte Carlo simulation.
Quantitative risk analysis simulation starts with the model of the project and either its project schedule or its cost estimate, depending on the objective. The degree of uncertainty in each schedule activity and each line‚Äźitem cost element is represented by a probability distribution. The probability distribution is usually specified by determining the optimistic, the most likely, and the pessimistic values for the activity or cost element. This is typically called the ‚Äú3‚Äźpoint estimate.‚ÄĚ The three points are estimated by the project team or other subject matter experts who focus on the schedule or cost elements one at a time.
Capturing risks, classifying them, prioritizing them, analyzing them is necessary to project success, but not sufficient.¬†
Mitigating (handling) the risk is needed. This is done in one for four ways: ‚Ä†
Avoid. Risk can be avoided by removing the cause of the risk or executing the project in a different way while still aiming to achieve project objectives. Not all risks can be avoided or eliminated, and for others, this approach may be too expensive or time‚Äźconsuming. However, this should be the first strategy considered.
Transfer. Transferring risk involves finding another party who is willing to take responsibility for its management, and who will bear the liability of the risk should it occur. The aim is to ensure that the risk is owned and managed by the party best able to deal with it effectively. Risk transfer usually involves payment of a premium, and the cost‚Äźeffectiveness of this must be considered when deciding whether to adopt a transfer strategy.
Mitigate. Risk mitigation reduces the probability and/or impact of an adverse risk event to an acceptable threshold. Taking early action to reduce the probability and/or impact of a risk is often more effective than trying to repair the damage after the risk has occurred. Risk mitigation may require resources or time and thus presents a tradeoff between doing nothing versus the cost of mitigating the risk.
Acceptance. This strategy is adopted when it is not possible or practical to respond to the risk by the other strategies, or a response is not warranted by the importance of the risk. When the project manager and the project team decide to accept a risk, they are agreeing to address the risk if and when it occurs. A contingency plan, workaround plan and/or contingency reserve may be developed for that eventuality.
‚Ä† Project Risk Management: A Scalable Approach, Risk Management Task Group, CalTrans, June 2012.Related articles The Actual Science in Management Science What is Governance? The Myth and Half-Truths of "Myths and Half-Truths"
When we hear about making decisions in the absence of estimates of the impact from that decision, the cost of making that decision - the opportunity cost, which is the basis of Microeconomics, or best of all the possible alternatives that might result from that decision - the opportunity costs - we need to stop and think.
Is it actually possible to make a decsion without knowing these things?
The answer is NO. But of course the answer is also YES. Since decisions can't be made in the absence of those estimates. They are made all the time. A little joke in the extreme sports domain, which our son participates in, goes like this.
What are the last four words uttered by a 22 year old back country skier in Crested Butte before arriving at the emergency room?
Hey everyone watch this!
Any estimating the probability of clearing the 20 foot gap? Oh Hell No, let's go...
The decision making process here is the same as the decision making process on projects. There are uncertainties that create risk. There are uncertainties that are irreducible and there are uncertainties that that are reducible. Risk of crashing and breaking your collarbone. Riks of crashing the project and breaking the bank or breaking the customer releationship.¬†
Both these uncertainty types must be addressed if the project has a chance of success, just as both uncertainty types need to be addressed if there is a chance of landing the jump without breaking something in a very painful way.
There are a set of environmental conditions on the project and on the slopes that are helpful and as well as a hindrance to success. Modeling those is the starting point for making the decision to¬†proceed. This is the taxonomy of uncertainty that must be assessed before proceeding with the decision.¬†
If you're 22 years old and believe you're immortal, then assessing these risks is rarely necessary. It's the¬†let's just try this view of the world. After breaking both collar bones (separate occasions), crashing mountain bikes as well as crashing on skis and being transported down the mountain in a Ski Patrol Sled, feedback prevails and a more mature assessment of the outcome results.
The word uncertainty¬†has a variety of meanings and has a variety of synonyms:¬†error, information, vagueness, scatter, unknown, discord, undefined, ambiguous, probability, stochastic,¬†distribution, confidence, and chance. These create confusion and from the confusion the opportunity to ignore them.
To evaluate the outcomes of our decisions, we need data
This data comes from a¬†model of the world¬†that allows us to translate our observations into information. In this model there are two types of abstraction. Aleatory and Epistemic. Aleatory implies an inherent¬†randomness of the outcomes of the process subject to our decision making. Flipping a coin is modeled as an aleatory process, as is rolling dice. When flipping the coin, the¬†random but observable data is the result. ¬†Since the underlying probability function for flipping the coin has no observable quantities (we can see all the processes that go into holding the coin, flipping with our fingers, the air, the rotation of the earth, etc.) but we can't model the world of coin flipping directly. Instead we can only observe the results from that model.
This is many times the case on projects. The underlying physical processes, which themselves may be deterministic, can't be observed. So all we get is the probability that an outcome will occur. This is a Bernoulli model of the process.¬†
A Bernoulli trial is an experiment outcome that can be assigned to one of two possible states (e.g., success / failure, heads / tails,¬†yes / no). The outcomes are assigned to two values, 0 and 1. A Bernoulli process is obtained by repeating the same Bernoulli trial,¬†where each trial is independent. If the outcome assigned to the value 1 has probability p, it can be shown that the summation of n¬†Bernoulli trials is binomial distributed.
The Epistemic uncertainty of the processes, both slope style skiing and projects, represents how precise our state of knowledge is about the¬†world model. We can measure the temperature of the snow, we can measure the performance of the database server. We know the wind speed at the top of the kicker, we know the density of the defects on the code base from our last quality assurance review.
The epistemic uncertainty of the process pertains to the degree of knowledge of the model and its parameters. Epistemic comes from the Greek word¬†episteme (knowledge).
We Need Aleatory and Epistemic information to make a decision
The system we want to make a decision about has a reality that can be modeled in some way, to some degree of confidence. When it is suggested otherwise, that simply is not true. Only when¬†Black Swans are introduced - the Unknown Unknowns are applied - does this model not work. In the project world, thsoe UNK UNKs mean one of three things:
This last situation is best represented In the famous Rumsfeld quote about UNKUNKs he failed to read¬†The Histories, by Herodotus,¬†(484-ca. 425 BCE), who cautioned not to go into that part of the world and engage in a ground war. It turned out bad for Alexander the Great.
So if you're the sort that accepts that decisions can be made in the absence of estimating the cost and impact of that decision - you're in the last two categories.¬†
A Final Thought
When it is suggested that businesses are seeking¬†deterministic or¬†predictable outcomes - which of course they are not, not can they since all business processes are probabilistic - those processes exist in only a few domains
Such precise processes are the¬†antitheses of aleatory., this is the type of model most familiar to
scientists and engineers and include relationships such as E=mc2,¬†F=ma, F=((G √ó¬†m1 √ó¬†m2) / r2).¬† So if you work with classical mechanics or the like, you can look for predictability. But if you work in the¬†real world of projects or the business of projects -¬†All The World's a Random Process - behave accordingly.
This, in turn, gives rise to the prospect of a Chief Probability Officer to manage the distributions that underlie risk, real portfolios, real options and many other activities in the global economy.
¬†- Sam Savage, Stefan Scholtes and Daniel Zweidler
There are some very serious misunderstandings going around¬†about how management in the presence of uncertainty takes places in business. The basic conjecture is
Management Science's Quest: in Search of Predictability ‚Ä†
Let's start with a basic fact for all projects, all business processes - everything is a stochastic process. So searching for predictability is not a goal for any informed business or technical person or organization. If it is, then that defines the maturity of that person or organization. It happens, but it states up front little understanding of the underlying stochastic processes that create probabilistic outcomes of - Everything¬†
Here's a quick review of both processes in play in all activties.¬†
In the Decision Making Business there are four reasons why they are hard.
So to start with the notion of¬†predictability¬†- it is simply not possible in any real project or business domain, to speak about predictability in the absence of the underlying statistical processes that create probabilistic outcomes.
Any credible business or technical manager knows this. If predictability is assumed or even desired, then the naivety of the manager is the only likely source, or maybe the intentional ignorance of the statistical and probabilistic nature of business and technical process. But predictability is not possible in the sense of absolutes, only probabilities.
So let's look at some less than informed concepts that are popular in some circles ...
The suggestions above indicate the lack of understanding of fundamental knowledge of¬†making decisions in the presence of uncertainty as described in the¬†Making Hard Decisions¬†book. The Journal Operations Research and Management Sciences, will put the science back in management science that those conjecturing the topics above seem to have missed.
In Journal papers and many books and related sources all the suggestions that we can't make decision in the presence of uncertainty, that simple minded conjectures like:
The basic problem with most perspectives on management today is that they are static analyses of a future environment. And all decisions are made because we believe we can predict the future.
Are simply not true, and better insight as to why they are not true can be had with straightforward reserch available by joining INFORMS or a variety of other professional societies.
So perhaps before making unsubstantiated¬†claims about how modern statistical and probabilistic management processes are applied to business, some¬†homework might be in order.Related articles What is Governance? Your Project Needs a Budget and Other Things The False Notion of "we haven't seen this before" Engineering in the face of uncertainty: Stochastic solutions to structural problems A comparison of MC and QMC simulation for the valuation of interest rate derivatives
When we hear ...
"Don't fall in love with your plan, it is almost certainly wrong "
let's reflect on advice from IEEE Std 730-2014
Plan Management: It is obvious that the project must be managed during its execution, but perhaps not so obvious that the plan itself must be managed. Despite the fact that nearly continuous change destabilizes any plan, the plan itself must not be allowed to float and become meaningless. The project must adopt a discipline for monitoring, reviewing and revising plans that they are stable in application while responsive to change.
The management process of IEEE/ISO 12207-2008 requires the manager to control the execution of the project by monitoring and reporting progress, and investigating, analyzing and resolving problems that arise.
The common phrase used by many in agile is Plan-Do-Check-Act.
This starts with Plan. In order to Do, Check, and Act, we need a place to start. The Plan. So falling in love with our Plan and then not, Doing, Checking the outcome, and Acting on the progress to that Plan, determining the varaincies from that Plan, assessing the impact from those variances, would mean we are not managing the project.
This would also mean that¬†falling in love with the Plan would require use ignorning the very basis of management in the presence of change.
So make plans, get feedback on the progress to plan, update the plan with this information, make a new or updated plan, schedule the work that the plan shows needed to be done, and measure progress to plan in units of measure meaningful to the decision makers.
Like the picture above, if you're going to embark on a day hike to Twin Sisters in Rocky Mountain National Park, you'd better have a¬†plan.¬†Not just for how long it's going to take, what your path is going to be, but have a Plan-B and likely a Plan-C when the weather goes bad, thunder storms come over the mountain, or any other¬†disruptive event.
But most of all PLAN, and don't fall for that non-actionable statement that plans are almost certainly wrong. Because without the plan, you have no clue about what DONE looks like, how to get there, how long it will take, how much it will cost, or how you'll recognize DONE when it arrives.
Plans are Strategies for Success
They can be neither Right or Wrong, they can only represent you're current understanding of the emerging situation that leads to the current path to success.Related articles Taxonomy of Logical Fallacies The False Notion of "we haven't seen this before" What is Governance? Your Project Needs a Budget and Other Things
When we hear about some new fangled way of writing software for money, first ask in what domain has this new idea been applied, and what were the measures of effectiveness and measures of performance for that approach?
Then ask if there is any external frameworks applicable in that domain for developing software based business systems that¬†govern the development, deployment, and operational aspects of the work?¬†
When the answer is yes, then next ask what are those governance frameworks? In a current engagement, ISO 12207 is the overarching framework for the development, testing, deployment, and operations¬†of software systems. These systems provide services for Health and Human Services, Center for Medicaid Services and the disbursement of $492 billion.¬†
The management of software development at the Federal, State, Local level, and the commercial providers of services to those levels is guided by ISO 12207 which is composed of the following processes.¬†
Now you may say¬†we never ever have a project of this size. Such projects are insanely large and outside any domain we'll ever see.
So the inverse question is¬†at what size do you no longer care about:
Then look through the 12207 list above and ask,¬†which of these process areas has NO value for my project?¬†That is, I don't need to know how to do this while spending other peoples money.¬†
I'll be just find in the absence of the guidance provided by this process area.
By the Way, the use of Agile methods fits right in with development work described in 12207.
With the answers to those questions, you can come back to see where you fit in the spectrum of projects.
Are you a¬†Lone Wolf writing code for yourself or a customer where only you work or are you a member of an enterprise development team working on systems that are essential to the business in which they cannot fail, even cannot fail the first time they¬†go live? Without the answer to this question, there can be no way to assess any suggestion, conjecture, or wild assed ideas about improving the probability of success for any software project. Related articles Taxonomy of Logical Fallacies Planning is the basis of decision making in the presence of uncertainty What is Governance? Good Project and Bad Project The Myth and Half-Truths of "Myths and Half-Truths"
Another twitter conversation - of sorts - prompted a thought about the purpose of planning, from the Quote of the Day post. In the enterprise IT world, planning and plans provide the road map for development and deployment of systems that implement the strategy of the business.
It is the process of planning that reveals the needed seqeunce of delivered capabilities. In the picture below a health insurance providers network application is deployed to replace several legacy systems over the course of time. Specific capabilities must be in place before later capabilities can be deployed.
This is a¬†Plan for the development and deployment of those capabilities. From the plan, technical and operational requirements ae developed, software developed, tested, IV&V's, confirmed to be compliant with security and regulatory guidelines, business users trained, and providers (medical) trained.
The sequence of these capabilities is well defined from the business operations side. The development of software inside each capability providing activity has some flexibility. But if we're going to arrive on the planned - NEED = date for say¬†Data Store Lookup, then the sequence of that software has some flexibility. But not arbitrary flexibility. Further levels of planning are needed for resource availability - both people, machines, services, facilities, etc.¬†
Planning is Planning for Capabilities to Be Available to Deliver Value
The planning process is driven by Capabilities Based Planning. This process has some simple straightforward steps.
So when we hear¬†deliver value on day one, we need to ask in what domain is that even possible. We need to deliver value on the day the value is needed for the business. Having a capability ready before the business is ready to use it is poor resource utilization planning.
We'll have spent money and time on something the business can't use yet. We may have made better use of that time and money on another capability. Which capabilities come in what order, at what time the business is capable of absorbing them is the first and primary role of¬†PLANNING.
So the conjecture that plans are almost certainly wrong, begs the question -¬†do you know what you're doing? If you're plans are almost certainly wrong - at least in the Enterprise IT business - you've got the wrong people managing the project. And you almost¬†certaintly are wasting money developing things that won't be used.
This doesn't mean we don't explore, try different things to see if they'll work. But that work is also planned. It's not our money.
Domain is King
If you've got a project that is just you or maybe you and a few close friends that's going to take a week or maybe 6 weeks, planning in this manner is likely a waste. Along with estimating your work, keeping track of progress to plan, and even counting the money.
But if you're on a $200M enterprise IT¬†development, integration, and deployment project with ~100 developers, testers, QA people, security, compliance, server ops, DBA's etc. you'd better have some notion of the order of work, the order of value delivery, the cost of that work, the probability of showing up on timem, on budget with that value in hand and how you going to¬†herd all the cats that surround the project.Related articles All About Me or All About the Mission? Your Project Needs a Budget and Other Things What is Governance? Taxonomy of Logical Fallacies Good Project and Bad Project Closed Loop Control The False Notion of "we haven't seen this before"
It was overheard on Twitter
"Don't fall in love with your plan, it is almost certainly wrong "
A plan is a strategy for success. Strategies are hypothesise . Hypothesise need tests to verify - just like you learned in High School science class. That test is the Measures of Effectiveness and Measures of Performance of the outcomes from your project as well as the Technical Performance Measures that are used to take corrective actions needed to reach the goal of delivering value in exchgange for time and money.
The Plan describes where we are going, the various paths we can take to reach our destination, and the progress or performance assessment points along the way to assure we are on the right path.
These assessment points measures the ‚Äúmaturity‚ÄĚ of the product or service against the planned maturity. This is the only real measure of progress ‚Äď not the passage of time or consumption of money.
Wrong in the planning sense can only be wrong if you are managing your project Open Loop with no assessment of Effectiveness, Performance, Risk reduction, cost absorption, time performance or all the¬†...ilities associated with spending other peoples money.
So it is true that the Plan is wrong if you're not managing in the presence of uncertainty, feedback, and taking corrective action.¬†
Decision theory is concerned with the problem of making decisions. Statistical¬†decision theory is decision making in the presence of statistical knowledge, by¬†understanding some of the uncertainties involved in the problem.
Decision theory deals with the situations where decisions have to be¬†made in the presence of uncertainty, and its goal is to provide a rational¬†framework for dealing with such situations. To make good choices¬†we must calculate and manage the resulting risks from those choices. Today, we have tools ¬†to perform these calculations.
A few hundred years ago decision making in the presence of uncertainty and the resulting risk had only tool faith, hope, and guesswork. This is because risk is a numbers game. Before the 17th century, our understanding of numbers did not provide us with the tools needed to make choices in the presence of uncertainty.
A good book about the history of¬†making choices in the presence of uncertainty - risk management - is¬†Against the Odds, The Remarkable Story of Risk,¬†Peter Bernstein. These efforts culminated in¬†Bernoulli's focused not on probabilistic events, but on the human beings who desire or fear certain outcomes to a greater or lesser degree.
Bernoulli¬†showed how to create mathematical tools to allow anyone to ‚Äúestimate his prospects from any risky undertaking in light of [his] specific financial circumstances.‚ÄĚ The is the basis of¬†Microeconomics of decision making, in which the opportunity cost of a collection¬†of choices can be assessed by estimating both the cost of that decision and the result beneficial outcome or loss.
In 1921, Frank Knight distinguished between¬†risk,¬†when the probability of an outcome is possible to calculate ‚ÄĒ¬†or is knowable ‚ÄĒ¬†and¬†uncertainty,¬†when the probability of an outcome is not possible to determine ‚ÄĒ¬†or is unknowable.
This becomes an argument that rendered insurance attractive and entrepreneurship¬†tragic.¬† 20 years ¬†later, John von Neumann and Oskar Morgenstern established the foundation of game theory, which deals in situations where people‚Äôs decisions are influenced by the unknowable decisions of live variables¬†‚ÄĒ in the gaming world, this means¬†other people.
Decision making in the presence of uncertainty is a normal business function as well as a normal technical development process. The world is full of uncertainty.
Those seeking certainty will be woefully disappointed. Those conjecturing that¬†decisionscan't be made in the presence of uncertainty are woefully¬†misinformed.¬†
Along with all this woefulness is the boneheaded notion that estimating is guessing, and that decisions can actually be made in the presence of uncertainty in the absence of estimating.
Here's why. When we are faced with a decision, a choice between multiple decisions, a choice between multiple outcomes, each is probabilistic. If it were not - that is we have 100% visibility into the consequences of our decision, the cost involved in making that decision, the cost impact or benefit impact from that decision - it's no longer a decision. It's a choice to pick between several options based on something other than time, money, or benefit.
Buying an¬†ERP¬†system, or funding the development of a new product, or funding the consolidation of the¬†data center¬†in another city is a much different¬†choice¬†process than picking apples. These decisions have uncertainty. Uncertainty of the cost.¬†Uncertainty¬†of the benefits, revenue, savings, increasing in reliability and maintainability.Uncertainty¬†in almost every variable.¬†
Managing in the presence of uncertainty and the resulting risk, is called business management. It's also called how adults manage projects (Tim Lister)
The Presence of Uncertainty is one of most Significant Characteristics of Project Work
Managing in the presence of¬†uncertainty¬†is¬†unavoidable. Ignoring this uncertainty is also unavoidable. It's still there even if you ignore it.¬†Uncertainty¬†comes in many forms
It has been conjectured that ...
What in the begining you thought you needed is never what you actually need
Fails to realize several critical success factors for project success...
I need the capability to move a brigade of 3,000 to 5,000 troops 100 miles up the coast in ten hours.¬†‚ÄĒ Gen Norman¬†Schwarzkopf
Capabilities decipher the intent of the leader (Commander)
Capabilities based planning (v2) from Glen Alleman So if we're on a project that doesn't know what¬†Done looks like, we've got to ask a serious question Do we know what we're doing? If the answer is¬†No, we may want to rethink why we're here. Here's a process to answer that question. And a framework to discover those answers Related articles What is Governance? Your Project Needs a Budget and Other Things The False Notion of "we haven't seen this before"
When we hear something like¬†estimates are the smell of dysfunction. That's a logical fallacy. Estimating is a business and engineering activities that produces numbers, confidence intervals on those numbers, and confidence interval on the confidence intervals.
We have an 80% confidence in completing on or before the 3rd week in November, 2011.
Or an example of¬†Doing Stupid Things on Purpose¬†
Let's give Mary this new proiect which is 427 times bigger than any project she's every worked in her life and then expect it to be successfull and be shocked when it goes in the ditch and blame the business governance process of estimating as the root cause of her problems.
So don't fall victim to the logical fallacy of using the symptom in place of the root cause. Identifying the symptoms is very easy. It's the root cause and the corrective actions that's the had part.
The¬†smell of dysfunction¬†in estimating the cost, schedule, and technical performance in software development is not the estimate. It's bad management, it's uninformed application of microeconomics of decision making, it's laziness in learning how to manage the expenditure of other peoples money. It's missing governance.Related articles Three Increasingly Mature Views of Estimate Making in IT Projects The Myth and Half-Truths of "Myths and Half-Truths" Your Project Needs a Budget and Other Things What is Governance? The False Notion of "we haven't seen this before"