There is a quote from George Box that is used - misused actually - by many populist blogers and authors that goes like this...

All models are, some are useful

Of course this quote is used to avoid asking and answering questions around models, forecasting, and assessment of possible future states of systems, a project being a system.

The actual quote is from Science and Statistics, George E. P. Box, Journal of the American Statistical Association, December 1976, pp. 791-799.

So the actual reading would suggest "all model are wrong" ... and the correct answer cannot be obtained by making the model more elaborate. A simple model is needed.

But how simple? That's always the challenge. 

The nonsense that #NoEstimates is the answer is of cource just as foolish as overparameterized and overelaborated models. This of course is lost on both sides of the modeling discussion. 

Related articles Models ... Calling BS on "All Models Wrong" Everything is a Random Variable Models All models are wrong

I've concluded the #NoEstimates advocates have difficulty stating the domain and context in which their exploration is applicable. Without a Domain and a Context in that Domain, it's difficult to have a conversation about the merits of the idea.

Can I apply it to embedded flight control systems? How about emerging databases for nuclear safety and safeguards? Enterprise health insurance processing? How about pipeline right of way document management? Maybe editorial systems in media? Turbine controls, emergency shutdown, fire and gas monitoring? Radar signal processing?

You see the problem, right? General claims like making estimates is a waste of time seem to beg the question - in what domain? Answer comes back Software Development is a domain. Ah, not really. Kinda ends the discussion doesn't it.

Related articles Complex Problems Require Better Solutions Estimates and all that... Estimate Anything

Shim has asked a question - a recurring question - about having a PMP. Since everyone is selling all the time - it's part of being human. There are those in the community that encourage a PMP. There are those who oppose the PMP on principles. There are those selling alternatives.

Here's my sales pitch for a PMP

1. If you are willing to read a boring set of study guides, sit for a really boring test that may or may not be applicable to your domain, pay some money, and get a certificate that demonstrates you have “some” level of commitment to learning how to do your job from a broader set of practices — You’ve stated to everyone that you’re not a lone wolf, making things up, creating your own version of how to manage projects, redefining terms and processes and claiming they are applicable to “projects in general.”
2. With that certificate, you’re now connected to a community of others like you. If you don’t like that community, or you disagree with how that community works, then don’t join. Standing on the side of the road being a Blustering Bully about how evil that organization is, or how that organization is somehow not doing it right, is pretty much consider “bitching” here in the US. Get over it and get back to work. Unless you of course are “selling the alternative,” and need to slam the competition as part of your sales strategy. Having an MBA, allows me to understand that is a bad sales tactic and makes you look naive and uninformed about the selling process.
3. No one can connect certification with competency. I have personal experience with my mother-in-law and the misdiagnosis of a brain tumor from a renowned neurosurgeon here in Boulder County. Board certified, head of the department. He was fundamentally incompetent. We went back to Toronto and got a second opinion. She was born with that small mass in her head, get back to work. The symptoms had nothing to do with the mass. Certification does not mean competency in exactly the same way a PhD does not mean anything in the absence of tangible contributions to the professional referenced and acknowledged by peers. I know this from graduate work in Physics. Count the number citations of your thesis in the citation databases to see if anyone on the planet acknowledges your “thesis” (a hypothesis by the way) has been accepted or verified by others. This is the basis of the physics world and it likely the same in all research based PhD’s, including the soft-science domain of project management.
4. Everyone is selling all the time. Test the message the seller is selling against the broader background of your own experience, the experience of trusted others, and the rationality of the message. “You can lose weight in 10 days, with my special diet,” “You can take 6 strokes off your golf game with my special utility wood,” “You can ride much faster on your bike if you only adopt my training program.” You don’t find many used women’s golf clubs in the “play it again” stores, because women know you can’t “buy a game,” so they rarely trade up. You can’t “buy” competency, but a certificate says you are at least trying to understand a few things about the “game.”
5. In the end having more knowledge may not help, but it can't hurt. Being a member of a group with similar background let's you test the question every month - do I have sufficient experience to do my job?

The #NoEstimates notion has merit in the right domain. If you're looking for motivation for estimates outside of the domain where the customer doesn't really care about the final cost, just the final product developed inside a "level of effort" budget, look here and remember.

When you are spending other people's money - billions of dollars of other people's money - you'd better have a credible answer when they ask "how much is this going to cost and when will you be done? This is separate from "will I get what I asked for?" That's another topic all together.

There are several things we need to remember about agile:

• Agile means paying to discover the requirements as we go. No problem if you know that up front.
• #NoEstimate means you don't have an Estimate At Completion (EAC) other than the passage of time multiplied by the total labor load. No problem if you understand that.
• When there is a budget cap, and deadline for delivery, and a minimal set of useable features, then some notion other than we don't make no stink'in estimates probably needs to be in place.

Related articles No Estimates of Costs and Schedule? Unrealistic Cost and Schedule Estimates

PMI and most other project management educators use the notion of the Triple Constraint - Cost, Schedule, and Scope. This is sometimes called the Iron Triangle. It is call Iron because when you change one thing, the other two change as well. This has been talked about here before.

Cost we know about. Cost starts with the Budget and then includes the Actual Cost of performing the work - consuming that budget. Schedule is the same. We have a Plan and we execute the project against that Plan. As the project proceeds, we are either on plan or off plan. That is we are ahead or behind schedule.

Scope is supposed to be a description of the deliverables from the project. The Outcomes of the projects schedule and budget. But in the PMI vernacular, scope fails to explicitly include several things. 

• Quality - does the product work as defined?
• Technical Performance - does the product work as needed?
• Key Performance Parameters - does the product meet the needs of the consumer?

These are all wrapped into the word scope. Here's a better way to look at the Triple Constraint.

If our project management method uses the PMI paradigm, we could be on budget, on schedule, and providing the scope but have no clue about the items shown below:

• Technical Performance Measures
• Measures of Effectiveness
• Measures of Performance
• Key Performance Parameters

And of course failure to assess these are the root cause of most project failures.

Related articles Performance Based Management Showing Up On Time Project Controls is the Basis of Project Management Failure is NOT an Option!

Here is a handbook for developing credible Integrated Master Schedules. There are many suggestions but this one comes from the place where schedules are critical.

Once you have your credible Integrated Master Schedule, you're going to want to keep it that way and assess its credibility.

With these two documents, you'll have a starting point for creating schedules that actually describe what done looks like in units of measure meaningful to the decision makers.

Related articles The Train Wreck of all Projects Starts With These Problems Scheduling Guides Unrealistic Cost and Schedule Estimates Uncertainty is the Source of Risk Probabilistic Cost and Schedule Processes

There is a popular quote from George Box about all models are wrong, but some models are useful. This quote is many times used by people suggesting some new or innovative approach to a problem that has been around long time.

While this quote has a pithy ring to it, I'm almost certain those using the quote do so without actually reading the book where is was used.

In the early 1970s econometric models had been constructed for a number of countries using time series data. They were largely static, with responses to  change assumed to take place within one period, irrespective of whether it was a year or a quarter.

The time series models of Box and Jenkins stood in stark contrast to these naive and static models. The Box and Jenkins used a single variable in isolation and dynamics played the central role. A few studies compared the two approaches and concluded that univariate time series models provided the more accurate short-term forecasts.

This was a turning point because it implied that dynamics were more important for understanding short-run movements than economic relationships as then understood. The emphasis in time series econometrics therefore shifted from modelling large simultaneous systems to taking account of dynamic interactions.

Box and Jenkins were influential not so much because what they said was new, but because they said it well. Their contribution was to show how the dynamic properties of observed series could be matched to those of theoretical models. 

The Project Management Point

Models are a critical component of any credible forecast of cost, schedule, and technical performance. Without these models it is actually Guessing as so many critics of estimating are fond of stating. With credible models, forecasting becomes a tool used to increase the probability of project success.

So next time some self-proclaimed person uses Box's quote, ask if they have his book in the shelf and on what page that quote appears.

Related articles Models All models are wrong George Box, the Accidental Statistician

I was presenting at the College of Performance Management conference in Florida this week. The topic of credible estimating is always at the top of the list. Here's a phrase that says it all for anyone thinking twice about estimating, not making estimates #NoEstimates, or simply being lazy about how they are estimating

We don't make predictions about the future because we want them to come true. But because we want them to not come true. - Professor Frank Anbari. "Earned Value Project Management Method and Extensions," Project Management Journal, December 2003.

I'll get the attribution from the person who told me this. In the end, to be credible we must manage with our eyes wide open. If you don't have a credible estimate of what it is going to cost, when you are going to be done, and what features you will be able to deliver, in what order, then you'd better find a customer that doesn't really care about those things.

Related articles Probabilistic Cost and Schedule Processes Credible Estimating Processes Measurement of Uncertainty How to be a Master Scheduler

The more a man is imbued with the ordered regularity of all events, the firmer becomes his conviction that there is no room left by the side of this ordered regularity for causes of a different nature - Albert Einstein, in Ideas and Opinions, from an address at Princeton Theological Seminary, May 19, 1939.

When I hear people talking at #NoEstimates I have to smile. When someone is paying you to spend their money and produce something of value with that money. When that someone has an expectation of that value arriving sometime in the future. You're response, probably shouldn't be we don't do estimates, just give us the money and we'll spend it.

That's a bit over the top I know. Ron Jeffries has a rational approach to actually developing the software, ONCE the needed capabilities have been defined.

But without some high level estimating processes, the software development efforts are considered Level of Effort. Which means keep going until the money runs out. You can certainly do this using the methods defined by Ron. It's called maintenance. PayPal does this. But you have to pick your projects carefully.

How many legs does a dog have if you call the tail a leg? Four. Calling a tail a leg doesn't make it a leg
- Abraham Lincoln

The notion that we can rename things and then assume they are the same doesn't work, just like Lincoln said. The notion of measuring progress with the passage of time and calling it Earned Value, doesn't mean we're doing Earned Value.

Doing Agile without doing Scrum, XP, Crystal, or DSDM is not doing agile.

The LinkedIn forum The Project Manager Network - #1 Group for Project Managers which I sometimes wonder if that is actually the case, has a conversation going on around project failure. A Master's student asked about project failure modes. That led to the use of Earned Value. One participants made some pretty boneheaded statement around EV:

1. (EV) It's a tool and anyone who thinks it is an actual depiction of real progress is fooling themselves.
2. Percent complete, unless 100%, is at best an educated guess.
3. I've heard of the 0/100 and the 0/50/100 methodologies. I disagree with the 0/100. If you are trying to accurately portray progress, it is best to develop a set of standard credits, for example when doing drawings or, actual measurement such as in construction. Each activity has a value from which percent complete can be derived. 
4. An equipment general arrangement drawing is a good example (percentages shown are individual contributions) - using 40 hrs/drawing as the allowance (we get)...

5% (2 hrs)for applying the border and title
20% (8 hrs) for defining the area - boundaries and overall dimensions
30% (12 hrs) for showing all the equipment
5% (2 hrs) for the drawing being checked
10% (4 hrs) for it being Issued for Approval
15% (6 hrs) for it being Issued for Bid
15% (6 hrs) for it being Issued for Construction.
100% = 40 hrs. 

So, silly me, I responded that determining physical percent complete was a straight forward process once you have Quantifiable Backup Data or measures of Physical Percent Complete, or even better, Technical Performance Measures. Let's look at each concept

1. EV, when percent complete is measured in tangible units of physical percent complete is a depiction of real progress. Why because the progress is physical percent complete. It's a tautology.
2. Percent complete is NOT an educated guess, it is tangible proof. This is call Quantifiable Backup Data. Bring the number of drawings to the table you said you were going to do on the day you said you were going to do them, and you'll get the prorated share of the earned value for the total drawing package. Bring 8 of the planned 10 on the day you planned to produce 8 and you'll get 80% of the total work package for producing 10 drawings.
3. 0/100 is best, all others skew the performance. The last piece of that concept is actually correct. For a 10 drawing package, with each drawing weighted the same, that is 10%, 5 drawings, completed on the planned day, gets you 50% physical percent complete.
4. No we go straight into the ditch. Progress is NEVER measured with the passage of time except in Level of Effort work. The actual costs (ACWP) is interesting to cost accounting, but it tells us nothing about the progress of the project except how much money we have spent compared to our budget. If the assigned (apportioned) value were all there was, then OK. But the conversation included the number of hours, and there where the problem starts.

Then the final come back from the OP.

If you're doing a drawing and it takes 40 hours to do the drawing and no two drawings are alike, how do you determine what percentage complete you are? When estimates of the work to be completed are put together, they will estimate X drawings at 40 hrs/drawing. They will have different standards by discipline. Will a person ever report 41% complete on a drawing? Most likely not. It will be shown as 40%. Collectively all the drawings may be 41%, but not individual. No matter how you look at it, it is still an estimate.

First measuring progress to plan by the number of hours consumed is called Level of Effort. Second LOE is a very poor measure of anything other than the passage of time and consumption of resources.

In the end the poster was insistent that ANSI-748-B is just a guide, and doing EV was a local issue. No doubt that is true in his domain. But as a practitioner of EVM using ANSI-748-B for programs subject to DCMA Validation, as well as engagements with the DOD office that own the Earned Value Management processes for the DOD, I'd be laughed out of the room with an approach like that.

For a summary Earned Value in a Nutshell using a similar drawing example.

So Here's The Way TO Do Earned Value

1. Using the WBS define the Work Packages that produce the deliverables for the program
2. Assign an Earned Value Technique to each of these Work Packages - 0/100 for the individual tasks inside the Work Package is the absolute best. You're either done with the task or you are not. Weight the tasks if you want, but sum up the "done ones" and compare that to the total work and compute a percent complete.
3. Sequence these Work Packages in an Integrated Master Schedule
4. Assign budget (BCWS) to each of the Work Packages
5. Collect the actual costs to perform the work (ACWP)
6. Compute the Earned Value (BCWP) using a simple formula BCWP = BCWS x Physical Percent Complete.
7. Compute Cost and Schedule variance. Make management decisions based on these and take corrective actions to get back to GREEN.
8. Repeat step 5, 6, and 7 every month at a minimum, more often if possible.

That's all there is - essentially - for doing Earned Value Management. Of course if you read ANSI-748-B, the NDIA Earned Value Management Intent Guide, and the DCMA Intent Guide, there is a lot more.

You Can't Make Stuff Up

OK, you can. But if you do, it makes having a conversation about EV much harder. 

Related articles What is Earned Value? Definitions, Yes They Are Important In Earned Value, Money Spent is Not a Measure of Progress! Rings of Success Earned Value says its Name Issues with Deploying Earned Value Management

The conversation on Twitter around No Estimates #NoEstimates brings a smile. Here's some samples

• Marketing needs a rough idea of when alpha-level code will be available. Allowing a few weeks for alpha testing and a few more for beta testing provides a time line. That may well be good enough. 

That's called an estimate. A rough estimate for sure, but it's an estimate

From the same blog (a good one BTW)...

• We're on an enterprise system now and management needs to know how long and how much cost it will take to get to a Minimal Viable Product. The first release has to be functionally complete in order to be competitive. Everyone understands that this effort will take several months to get to beta-level code.  Management needs some idea of what they’re getting into, how long it will take, and how much it will cost.

Now we're into the realm of actual estimating

Are No Estimates Better Than Estimates?

Before answering this another question needs to be answered...

What is the value at risk?

This means how much money and time are you willing to risk without understanding how much time and money is at risk.

So before anyone can answer the question about estimating value, that question needs to be answered by the owner of the money. NOT to consumer of the money. 

This at risk question rarely comes up in the agile world. It doesn't come up enough in our formal FAR/DFARS acquisition work either. The at risk question is a risk management question. Agile claims to be a risk management process - which of course is laughable in our DOD/NASA/NRC/FDA/NNSA risk management world. All software intensive by the way.

So before venturing further, what's the value at risk must be answered. Then the owner of that risk, can tell those who should be working in the best interest of the owner of the risk, if an estimate is needed. Not the other way aorund.

Without this answer, the discussion on #NoEstimate is just sharing of personal opinions in the absence of any value at risk.

Related articles Time to Revisit The Risk Discussion When We Say Risk What Do We Really Mean? More Uncertainty and Risk Measurement of Uncertainty Estimates in Software Development. New Frontiers. Risk Management

There is a twitter discussion going on around Neil's post of People Need Estimates. This is a typical agile approach. No real domain or context, just a principle looking for a problem to solve. 

First let's establish some ground rules:

• I'm a strong advocate of agile development. I work prorgams that use agile. Big programs. Billion dollar programs. 
• When we are spending other people's money it's not about "us" and our favorite method of spending that money. It's about governance. Don't like governance, then don't work on projects that spend other peoples money.

And the final one ...

• Agile - in many cases - is spending people's money to discover the requirements. That's fine. But acknowledge it. Requirements emerge. Requirements change. Requirement are wrong many times. But don't say it'll be OK to use agile and ignore that you're spending - maybe even wasting - other people's money and calling it progress.

Neil says "People need certainty about what they will get and how much they have to spend. " This is not factually true in many domains, not is it actually possible. Those with the money need to know the confidence in the spend plan and the project duration. Certainty is not possible. But don't use that as an excuse for not having a confidence interval estimate:

• We will complete this project on or before the 3rd week in November, 2014 with an 80% confidence and a 12% error band on that confidence. This is easily developed and is done every month on the programs we work.
• We will complete this project at $1.6M or less with 80% confidence and a 10% error on that confidence.

This is what those spending the money need to know. Otherwise the project is Level of Effort. Work producing good things till the money runs out, get some more money continue doing good things. No problem. That is called maintenance. This is how PayPal uses Scrum. They budget annually for features, develop those features within that budget, repeat forever.

I love the notion "When estimating a date or cost you are creating uncertainty around those things, because you are guessing. "

Well stop guessing. Start being a software professional and start estimating. Have you done this before? How long did it take. Has anyone you know done it before? Go ask them. Guessing is lame. No estimates is even lamer. Worse, it's just being lazy. 

One last thing, "People still need 500-page business requirement documents." This is bogus. We work multi billion programs, with 100's of millions of dollars of embedded sofwtare and don't have 500 page requirements documents. Nonsense. 

If you don't know where you are going, if you don't have some notion of what done looks like, if you can't tell me approximately how much and how long, please don't start. You can be a binary search process for duration and cost for any tangible piece of software and within five question get without 20%. That's good enough to start. So start and take corrective actions with ne estimate along the way.

Finally "However, I would argue that #NoEstimates gives greater certainty than estimating does." There is absolutely no evidence for this. This is pure speculation and therefore also nonsense. Think about it,

• A builder comes to your house for a kitchen remodel.
• You tell him roughly what you want done. 
• You ask "how much will this cost, approximately?"
• "Oh I don't know, we're not good at making estimates, let's just get started and see where this takes us."

Are you out of your !@#$'ing mind. It is no different for software. Here's a nice short intro from Target Process, that actually makes sense.

Let's say the CIO comes and says "I need these features", and asks "how much and how long?" You say "Oh we get better outcomes when we don't estimates." Really, and some one believes that nonsense? I guess so.

Related articles Probabilistic Cost and Schedule Processes Estimates and all that... Cost (and Schedule) Estimating Foundations A Point Measures Need A Variance

Yes, this is an actual post on a LinkedIn forum

When the role of the project manager is to push the rock up the hill and then let it roll down again, there is little chance of success, and in the end little need for project management or the project manager. Cancel the project, everyone just go home.

Risk Management is how adults management projects - Tim Lister

 

There have several rounds of how to use analogies and how not to use analogies in the past few years. 

These involved notions like agile is an untended garden. Actually and untended garden and called a weed patch.

Or we can't really stop and develop a strategy, because we're always putting out fires. Actually firemen rarely put out fires. They spend the majority of their time preventing fire with fire safety, inspections, fire inspections. Their job is to never have fires start.

And of course for false analogies of the double pendulum as a stand in for chaos and unpredictability. Since the equations of motion are easily defined - an exercise for any upper division physics student - and a MathLab plugin, you can plot the path of the double pendulum. 

And my favorite the attractor analogy, in which it is presumed that some how in chaos theory the attractor attracts. Without understanding  that those pretty pictures of the attractor are the result of the underlying equations for the system.

So with that in mind, there is a new book from one of my favorite authors, Douglas Hofstadter, Surfaces and Essences. In the book Hofstadter makes the case for analogy as the fuel for creative thinking. Using Robert Oppenheimer's quote... 

whether or not we talk about discovery or of invention, analogy is inevitable in human thought, because we come to new things in science with what equipment we have, which is how we have learned to think, and above all how we learned to think about the relatedness of things.

But as always we need to take care to assure that those analogies we use to expand the conversation, don't violate the laws of physics, gardening, or mathematics.

Related articles The Misunderstanding of Chaos - Again Happy 50th Anniversary, Chaos Managing in the presence of all things

Everything is simpler than you think and at the same time more complex than you imagine. ‒ Johann Wolfgang Goethe

More discussion on LinkedIn about risk, risk management, types of risk, and definitions of risk and opportunity. The "risk management" community in the domains I work is based on Department of Defense, Department of Energy, Department of Homeland Security, NASA, ITIL, AACEI (Association for the Advancement of Cost Engineering International), Nuclear Regulatory Commission, INCOSE (International Council of Systems Engineering), OGC Management of Risk, GAO, and Deloitte's Risk Management Framework.

Several core concepts get confused. So before taking any deep dive into "managing risk," these need to be established.

What is Risk

There are many definitions of risk from a variety of sources. In the end these definitions are all pretty much the same. 

• Risk involves the probability of something happening in the future.
• When that something happens it impacts the project in ways that are not good.
• There can be a probability of the effect of the impact as well.
• Handling the risk means knowing what kind of risk it is, and what the choices are for handling the risk.

Here's a good definition that can be used in probably any domain.

Risk refers to the uncertainty that surrounds future events and outcomes. It is the expression of the likelihood (probability of occurrence) and (probability of) impact of an event with the potential to influence the achievement of an organization’s objectives. - Managing Risk in Government: An Introduction to Enterprise Risk Management, IBM Center for The Business of Government.

I've edited this a bit to remove the likelihood measure and replace it with the probability just to keep the math straight. You'll see why below when it comes to ordinal values in the risk matrix - a bad idea.

There is a time honored  approach to ranking a risk, by calculating some number. This number is meanigless of course, because it has no scale, it is just a ordinal number. We need cardinal numbers for actual risk management.

Risk = Probability of Occurrence x Impact  

This is a trick question. The risk matrix approach that results from this calculation treats these numbers as likelihood not probabilities. But in fact they are probabilities. As well they are Ordinal numbers (1, 2, 3, 4, 5 usually) and as such represents the relative ranking of the likelihood of occurrence, not the actual probability of occurrence.

This approach starts with a major problem - a show stopper problem. The two values on either side of the multiplication sign are probability distributions, not real numbers. There is no MULTIPLY operator between two probability distributions. The is the CONVOLUTION  operator, but that is not likely to be of much value to ordinary project management staff. One starting point for a better understanding of the problems with the risk multiplier and risk matrix approach is the work of Tony Cox. Optimal Design of Qualitative Risk Matrices to Classify Quantitative Risks. Tony's source paper provides the basis for this discussion, "What's Wrong with Risk Matricies."

To put it in blunt terms.

The combination of Ordinal Scales and Risk Matricies are Fatally Flawed

In this link, it is stated again, so I'll restate it here

It is a category mistake to multipy ordinal numbers

By category error it means the objects, the categories, do not have the proper properties to work with the operators. For some more discussion of this topic and a bit of counter discussion, see What's Right with Risk Matricies.

Where does risk come from?

Risk comes from uncertainty. There are two kinds of uncertainty.

• Uncertainty that can be reduced - reducible uncertainty. This is called epistemic uncertainty. This uncertainty is characterized by the lack of knowledge. We can buy this knowledge.
• Uncertainty that cannot be reduced - irreducaable uncertainty. This is called aleatory uncertainty. This uncertainty is an inherent variability in the project processes and characterized by a probability distribution. We can not buy more information about this uncertainty.

The critical point here is the reducibility and the irreducibility of these uncertainties. 

• If the uncertainty is irreducible, then only margin can be used to protect the project from the risk. This is scehdule margin, cost margin, techncial performance margin.
• If the uncertainty is reducible, then we can buy down the uncertainty and therefore buy down the risk. That is we can spend more to find out more information - increase our knowledge. This is done in one of two ways:
  • Spend money in the project to increase our knowledge.
  • Provide a Management Reserve or Contingency to handle the risk when it comes true.

How is Risk Maanged?

We must start with a risk management process. There are several, but they are essentially the same. Here's my favorite, there are others of course, but this one covers all the steps.

So What's the Point Here?

• There is no need to make up definitions for risk and risk management. There are plenty around and all of them are in use on actual projects by actual project and program managers, managing risks every day. Providing your own definitions, is sporty at best. At worst, it shows you haven't done your homework.
• Uncertainty is the source of risk. There are two types of uncertainty - reducible, epistemic and irreducible, aleatory. If you do not separate these two, you cannot credibly have a risk handling plan. If you do not have a credible handling plan, then you're not actually managing risk. 
• Ordinal numbers cannot be used to make risk decisions. Cardinal numbers are needed. No ranking likelihood of occurrence as 1,2,3,4,5. Not allowed, it's bogus. Same for impact. Bogus. And certainly no multiplying those two, even more bogus.
• Risk Management is How Adults Manage Projects - Tim Lister

Related articles Four components of risk Both Aleatory and Epistemic Uncertainty Create Risk Time to Revisit The Risk Discussion When We Say Risk What Do We Really Mean? Uncertainty is the Source of Risk The Barriers to Effective Risk Management

Cost estimating methods have been around for a long time. The current processes found in agile use a points system, sometimes a Fibonacci series to bin the values of the points.

The challenge with this approach is the estimate in agile is not monetized so we can't really tell if the Total Allocated Budget (TAB) is sufficient for the project at any point in time, unless the capacity for and the quality of the ourcomes is steady - that is Level of Effort. 

With the LOE approach, the capacity for work is the critical measurement needed for estimating the cost at completion. As well continuous updating of this capacity for work is needed and correctly done on good agile projects.

But there are other issues with this LOE approach on larger projects:

• Do we know the variances in the capacity for work and how those variances will impact the final Cost at Completion?
• Do we know the interdependencies between the various work products and how they impact the final cost?
• Do we know the Aleatory uncertainty - the natural occurrence that can't be fixed and has to have margin. 
• Do we know the Epistemic uncertainty - the event based risks that need a risk handling plan?

So the examples like that found at Projects @ Work, don't really consider any of the underlying uncertainties in estimating. Without the next level down - statistically adjusted estimates of the work effort, the capacity for work, the quality of that work, and the interdependencies between those work activities and their products, the simple and maybe even simple minded approaches to estimating have limits to scaling.

This is one of those topics where everyone is right in some way, depending on the domain, context in the domain, and scale of that domain. As agile enters the larger acquisition community, where we're spending other peoples money - maybe 100's of millions of dollars, care needs to be taken when applying un-monetized, non-probabilistic, non-joint probability (cross correlations between work element) and non-stochastic forecasting models. The real world is not that simple.

Related articles Deterministic versus Stochastic Trends in Earned Value Management Data Probabilistic Cost and Schedule Processes Uncertainty is the Source of Risk Time to Revisit The Risk Discussion When We Say Risk What Do We Really Mean? Complex Problems Require Better Solutions A Point Measures Need A Variance

Over the past weeks there have been several discussions on the forums and Blogs amount risk and risk management. Here's a short post on those topics and their impact on project performance.

Risk Comes from Uncertainty

Risk does not exist by itself. Risk is created when there is uncertainty. If I am certain that it is going to rain this afternoon, then there is no risk of rain. It's going to rain with 100% probability. There is no uncertainty about the forecast of rain.

So first we need uncertainty to have a risk. But there are two classes of uncertainty:

• Aleatory uncertainty - is uncertainty that comes from a random process. Flipping a coin and predicting either HEADS or TAILS is aleatory uncertainty. In other words, the uncertainty we are observing is random, it is part of the natural processes of what we are observing.

Aleatory uncertainty refers to the inherent uncertainty due to the probabilistic variability.

This type of uncertainty is Irreducible, in that there will always be variability in the underlying variables.

These uncertainties are characterized by a probability distribution.

The parameter that is being measured - duration, RPM, discharge from a river flow, is stochastic and cannot be reduced.

• Epistemic uncertainty - is uncertainty that comes from the lack of knowledge. This lack of knowledge comes from many sources. Inadequate understanding of the underlying processes, incomplete knowledge of the phenomena, or imprecise evaluation of the related characteristics are common sources of epistemic uncertainty. In other words we don't know how this thing works so there is uncertainty about its operation.

Epistemic uncertainty refers to limited knowledge we may have about the system (modeled or real). This type of uncertainty is reducible. If we have more information, we can take more measurements, conduct more tests, "buy" more information. This type of uncertainty can be reduced. The parameter being measures is usually a characteristic of the material or the physical process. The uncertainty is related to the "lack of knowledge," about this parameter.

Handling Risks Created from Uncertainty

• Aleatory risk is not a lack of information. It is a naturally occurring process. We cannot buy more information, so we have to provide margin for this type of risk. Schedule Margin to cover the naturally occurring variances in how long it takes to do the work. Cost Margin to cover the naturally occurring variances in the price of something we are consuming in our project.

Aleatory uncertainty and the resulting risk is modeled with a Probability Distribution Function. This PDF describes all the possible values the process can take and the probability of each value. For a single toss of a coin, there is a 50% probability it will be either heads or tails. For multiple tosses of a fair coin the probability distribution of the total number of heads or the total number of tails is a binomial distribution that  looks like this for the numbers of HEADs from fair coin being tossed 20 times.

The PDF for the possible durations for the work in the project can be determined in several ways. It turns out we can buy knowledge about aleatory uncertainty through Reference Class Forecasting and past performance modeling. This new information then allows us to update - adjust - our past performance on similar work will provide information about our future performance. But the  underlying processes is still random, and our new information simply created a new aleatory uncertainty PDF.

• Epistemic risk comes from the lack of knowledge. Epistemology is the branch of philosophy concerned with the nature and scope of knowledge. Lack of knowledge is epistemic uncertainty. 

Epistemic risk is modeled by defining the probability that the risk will occur, the time frame in which that probability is active, and the probability of an impact or consequence from the risk when it does occur.

Risk statements are used to define and model these event based risk:

• IF-THEN - says if we miss our next milestone then project will fail to achieve its business value during the next quarter.
• CONDITION-CONCERN - our subcontractor has not provided enough information for us to status the schedule, and our concern is the schedule is slipping and we don't know it.
• CONDITION-EVENT-CONSEQUENCE - our status shows there are some tasks behind schedule, so we could miss our milestone, and the project will fail to achieve its business value in the next quarter.

For these types of risks we can have an explicit or an implicit risk handling plan. I use the work handling with special purpose. We handle risks in a variety of ways. Mitigation is one of those ways. But the risk handling work is actual work. It is in the schedule. We are doing work to mitigate the risk. We are buying down the risk, or we are retiring the risk. In all cases, we are spending money, and consuming time to reduce the probability that the risk will occur. Or we could be spending money and consuming time to reduce the impact of the risk when it does occur. In both cases we are taking action to address the risk. The second approach to handling an epistemic risk is the have Management Reserve to cover the cost of the consequences when the risk occurs. Sometimes the term contingency is used. Both Management Reserve and Contingency may be used together. In both cases, money is set aside to handle the risk. We also need time as well, so we may have schedule reserve. But this gets confused many times with schedule margin, but it is still needed.

Risk Management is how Adults Manage Projects

One of the posters stated what would be considered a Lame response to the processes and seeming conplexity of managing risks on non-trivial projects, by stating you're making this to complex - Just Do It. It was lame. Here's the response to those who objective in what ever way to doing risk management.

First answer the question what is the value at risk for your project? Don't know? Go find out. Then ask the project sponsor or the person giving you money to manage the project, if they would be willing to lose that money outright. Just write it off when the risk comes true. Probably not would be the answer. So go do the risk management process. 

Here's Tim Lister's advice. The section title is Lister's quote and should be used every time some lame response comes back about risk management.

Related articles Four components of risk Time to Revisit The Risk Discussion Uncertainty is the Source of Risk Deterministic versus Stochastic Trends in Earned Value Management Data