Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Herding Cats - Glen Alleman
Syndicate content
Performance-Based Project Management¬ģ Principles, Practices, and Processes to Increase Probability of Success –Ē–ĺ–≤–Ķ—Ä—Ź–Ļ, –Ĺ–ĺ –Ņ—Ä–ĺ–≤–Ķ—Ä—Ź–Ļ
Updated: 10 hours 39 min ago

Late Start = Late Finish

Sat, 01/31/2015 - 18:31

In a CrossTalk article Risk Management for Dummies, Tom DeMarco speaks about late software projects and the approaches for the solution. The notion of early start as the solution for late finish needs to address the core question, but fails to address the root cause of lateness - no schedule margin.

How soon should we have started to finish on or before the needed finish time?

Screen Shot 2015-01-29 at 7.36.47 PM

The answer to this is simple:

  • If we have a schedule for the project, which contains the needed work efforts, in the planned sequence with an estimated duration for that effort - the Most Likely duration - then we can model the probability of a completion date with Monte Carlo Simulation.
  • The question is¬†what is the range of possible durations an activity can have, before we actually start performing that work?

Good question. And of course like all questions about some future activity we'll need to Estimate those values. We can't actually manage - in any credible manner - without estimating. Anyone suggesting otherwise has likely only encountered trivial projects where the Value at Risk was low enough that no one cared is you were late or over budget.

An Example of Managing In The Presence of Uncertainty

From the Forward of Technical Risk Management, Jack V. Michaels, by Norman Augustine author of Augustine's Laws.

Columbus proposed a voyage to Ferdinand and Isabella in 1486. The monarchs promptly set up a special commission of learned men to study the proposal, which they found vague and arcane. After four years of unsatisfactory discussions with the master mariner, the commission's finally rejected the proposal.

To their credit they did not abandon Columbus and soon recalled Columbus not to plan again, but to name his price for carrying out the voyage. The price was enormous. Columbus insisted on being knighted, appointed a grand admiral and viceroy, and given the unwillingness to compromise, the king and queen dismissed Columbus. Eventually, they relented and accepted all of Columbus's demand.

I (Norm Augustine) put forth this example because all parties involved could have benefited from risk management. Columbus could have presented his proposal containing each element of risk, but consider the payoff, well worth the investment.

Ferdinand and Isabella could have assessed the "price of doing nothing." The commission of experts could have have a framework in which they could evaluate the technical feasibility of the proposal. If the parties had studied the technical risk management, perhaps we'd be celebrating 1486, not 1492

For an simple schedule, like the Wright Brothers meeting their contractual data to the U.S. Army of August 31 1908 for the delivery of the Wright Flyer, they needed a confidence level of 80% for the on or before date. The schedule margin of 10 days provided the protection for that date.

Screen Shot 2015-01-31 at 8.22.36 AM

What Does This Mean?

The first meaning, missing from the CrossTalk article is.

Schedules without margin are late on day one

It means Estimating is Risk Management. No estimates, no risk management. No risk management, lower probability of project success. For schedule risk we need irreducible and reducible uncertainties that drive the schedule. The for the reducible uncertainties, we need activities to reduce the risk from that uncertainty. For irreducible uncertainties, we need margin to protect each major deliverable.

Risk Management is How Adults Manage Projects - Tim Lister

Remember that next time you hear we can't estimate, we don't want to estimate, estimates are a waste.

Here's How to Manage in the Presence of Uncertainty

Related articles Qualitative Risk Management and Quantitative Risk Management We Suck At Estimating Building the Perfect Schedule Estimating is Risk Management Probability and Statistics of Project Work
Categories: Project Management

We Suck At Estimating

Fri, 01/30/2015 - 17:40

SadeeyoreWhen I hear "we suck at estimating," or "we can't make good estimates, because people take them as commitments," I wonder what would be the answer if those statements were restated as... 

"We suck at designing good database schemas." "I can never figure out how to configure the IIS VPN tunneling server." "This Sharepoint Server UI in SP 2012 is so much different than 2007, I can't possibly figure out how to help my customer."

Or, this 401(k) calculation for estimating my retirement is just beyond my ability to comprehend." Or, I can't possibly figure out how much money I need for my summer vacation with my family on a European trip, with relatives I haven't even met, and hotels and rental cars, train tickets, and meals, and other expenses, and airline reservations, and all the unknown and may be unknowable stuff. I'm just going to sit here and mope. Yep, that's me, Eeyore, woo is me.  

What is it in some software domains that allows well educated, experienced developers to simply give up, walk way, allow themselves the be subjected to Dilbert Bosses? When they'd never talk that way about something they are skilled, experienced, and capable of doing?

It's a repeating theme in the #NoEstimates domain. You'd never hear a Flight Avionics engineer at a major space and defense company here in Denver say "we suck at estimating, we suck at testing, we suck at systems integration." Those engineers are asked all the time to "invent new physics to solve a problem." Or other colleagues literally inventing solutions of complex oil & gas software based process control algorithms. Or other colleagues at a major consulting firms deploying ERP and encountering legacy systems they've never seen or integrated before, saying "we can't figure this out, it's too hard and our managers will take our estimates as commitments, so we're simply not going to estimate our work."

Is it missing the engineering discipline taught in college. Where a probability and statistics class is mandatory for all 4 years to be called an engineer. Or even in the business and finance discipline estimating, using complex approaches - time series analysis, principle component analysis, bayesian statistics - is mandatory to be called a finance major. Or in the sciences in general. I say this from direct experience with our children and their significant others who are in finance, bio-sciences, and mechanical engineering all writing software for money as part of their actual profession.

And this whole notion that somehow writing software is not engineering is simply BS. It is engineering in many business domains. So maybe it's a domain issue. Maybe non-engineering cultures can't figure this out. 

Or is it maturity? Many appear to be very mature developers? Or is it culture, not geographic culture, but lack of having experienced a development culture where business and engineering work together to determine risks, corrective actions, increasing maturity of the deliverables development processes that allow for growth in both technical and business capabilities - all using agile by the way.

Everything is Uncertain

Business operates on risk taking. Risk taking is about making decisions in the presence of uncertainty. Uncertainty is about probable outcomes - positive or negative - in the future. To make decisions about uncertain outcomes in the future, we need to estimate those outcomes, the cost to achieve them, and the possible benefits from that cost.

No credible business person fails to understand this.

If they do, they won't be in business for long. It's taught in school - business school. Same for engineering and computer science school.

What is it that brings smart, capable, intelligent people to say "we suck," when the facilities (classes, books, articles, tools, external advisors) for learning how to estimate for the required reasons of making decisions about other peoples money for future outcomes, is the very basis of all business - is at hand, with simple learning and practice. These facilities are also the basis of all learning. "We suck" says "we're not capable of learning."

Are those of the #NoEstimate ilk saying "I can't learn?" or are they saying "I don't want to learn?" Because those providing them the money have a vested interest in knowing how much money is needed to produce that treasured Value they claim to be producing. 

Just like learning a programming language. Moving from SP 2007 to SP 2012 (that actually sucks), dealing with cyber security. We don‚Äôt say ‚Äúwe suck‚ÄĚ and give up. What is actually going on here? Is some portion of the development community become Eeyore?

Related articles Your Project Needs a Budget and Other Things Building the Perfect Schedule Don't Manage By Quoting Dilbert Qualitative Risk Management and Quantitative Risk Management Estimating is Risk Management
Categories: Project Management

Estimating is Risk Management

Thu, 01/29/2015 - 18:06

Risk Management is about many things - but it is first and foremost about estimating future outcomes. 

Suppose one of you wants to build a tower. Will he not first sit down and estimate the cost to see if he has enough money to complete it? For if he lays the foundation and is not able to finish it, everyone who sees it will ridicule him saying, "This fellow began to build and was not able to finish." - Luke 14:28-30

To manage the risk of not enough money, not enough time, the probability of unacceptable outcomes we need to estimate. It's as simple as that and as complex as that.

Risk management is essential for development and production of products and services because key information about cost schedule, and technical performance are uncertain and many times unknown until later in the project.

Proceeding to spend other peoples money in the presence of these uncertainties is not only bad management, it's bad economics - the microeconomics of decision making requires estimating the impacts of any decision - the opportunity cost of that decision.

Risk management is concerned with the outcome of future events, whose exact outcome is unknown, and with how to deal with these uncertainties as a range of possible outcomes. - Effective Risk Management: Some Keys to Success, 2nd Edition, Edmund H. Conrow.

So when you here we can make decisions about the future without estimating ask yourself, did that speaker ever read about risk management. And as Tim Lister says

Risk Management is How Adults Manage Projects

Related articles Probability and Statistics of Project Work Qualitative Risk Management and Quantitative Risk Management Decision Making in the Presence of Uncertainty All The World's A Random Process Your Project Needs a Budget and Other Things
Categories: Project Management

Building the Perfect Schedule

Wed, 01/28/2015 - 16:30

Plans are critical, a schedule to implement that plan comes next. With the Plan, the sequence of the work is needed to assure the value to the customer is delivered in the proper order. That order is an order because the business (or the mission) usually can't accept all the features and functions at once. So the Plan is the top level sequence, and the schedule is the next level down.

Building the perfect schedule (v6) from Glen Alleman Related articles Your Project Needs a Budget and Other Things Qualitative Risk Management and Quantitative Risk Management
Categories: Project Management

Turning Editorials into Root Cause Analysis (Update)

Tue, 01/27/2015 - 23:50

When we read about a big IT problem, like...

Screen Shot 2015-01-26 at 7.24.24 PM

The first impulse is to use this information to support some or other position, like here's an example of estimate driven bad management. Without the logical conclusion of finding the actual Root Cause of the problem, as shown in the IDA report. Other examples usually start with bogus Standish data. Take a look on page iv below to see the real root cause, and see if Not Estimating would have been able to address the issues with ECSS? Not likley. 

Screen Shot 2015-01-26 at 7.14.04 PM

This document seems to not load everytime, refresh the page if it doesn't

So we're back to the same place we always seem to come to. Domain and knowledge of the domain, before conjecturing any solution to any problem and the conjectured solution that occurs outside the domain of experience. 

For those not able to read the details here's a summary from the final section.

Screen Shot 2015-01-27 at 1.53.38 PM

The notion of some that "estimates" are the root cause of the problems and that making decision in the absence of Estimates is the solution to the problem based on un-informed opinion.

So down load the report, read it in it's entirety, then assess other opinions in the light of actually reading the Root Cause Analysis, then drawing conclusions from the actual data and its analysis by the professionals tasked by the sponsors of the RCA who are accountable for looking after the money and the measurable outcomes.

And then the quote that says it all...

"Everyone is entitled to his own opinion, but not his own facts." - Daniel Patrick Moynihan 


Related articles Taxonomy of Logical Fallacies
Categories: Project Management

Building a Credible Performance Measurement Baseline

Thu, 01/22/2015 - 03:03

Many times I hear about Cost of Delay, Deliver Value, Measure story points, or Measure Stories. And a myriad of other assessments of project performance, all of which - OK, most of which are examples of Open Loop Control.

Back in 2014, we had a paper in a publication of the College of Performance Management, starting on Page 17. As well, a colleague Nick Pisano (CDR US Navy Retired) has a post on the same topic at his blog.

Screen Shot 2015-01-21 at 4.59.00 PM

The notion of a baseline let alone a Performance Measurement Baseline is at the heart of Closed Loop Control of all processes, from your heating and air conditioning system in your house, to the flight controls on the 737-700 winging its way back home to Denver, to the project you're working - using what ever project management method or software development method of your choosing.

The notion that we can manage anything, the temperature of the room, the nice soft ride in the 737, or the probability of showing up on or before the need date, at or below the needed cost, with the needed capabilities - and NOT have a baseline to steer to is simply wrong. 

Below is the framework for Closed Loop control. This paradigm says simply:

  • State where we are going in units of time, cost, and technical performance:
    • I need this set of features (Capabilities) to be available for use by the customer on or before this date, with some confidence level, for some cost - again with a confidence level.
    • With these features - provided on a planned date, for a planned cost - I can then assess the progress toward that planned date, planned cost, and planned capabilities.
  • With the Planned data and the assessment of the actual data - cost, schedule, and technical performance:
    • Technical Performance is actually not enough
    • Measures of Effectiveness are needed
    • Measures of Performance as well
    • And other¬†...ilities of the outcomes - reliability, maintainability, serviceability, stability, etc.
  • Then with these measures we can generate an¬†error signal - between planned to date and actual to date - to determine several critical things - without which¬†we're flying Open Loop.
    • Given our ¬†Performance to Date and the Planned Performance at this point, how far behind are we, how over budget are we, how close are we to getting this gadget to work as needed?
    • With this data, we can then make a decision.
  • Making those decision means
    • ESTIMATING both the¬†to be¬†target - where should we be at this point in the project for cost, schedule and technical performance. And where should we be to close the gaps between our target and the actual progress to date.
    • ESTIMATING what cost, effort, changes in technical direction are needed to close those gaps.

ESTIMATING IS THE BASIS OF DECISION MAKING - it can't be any clearer than that.

Control systems from Glen Alleman Related articles Your Project Needs a Budget and Other Things The Actual Science in Management Science Probability and Statistics of Project Work What is Governance? Don't Manage By Quoting Dilbert
Categories: Project Management

Planning And Estimating Is Required for Project Success

Mon, 01/19/2015 - 20:06

In project work we're looking to create or change something, in some defined period of time, for a defined cost. This means we're going to spend money now for some future outcome. The elements that go into this effort to produce some change in the future include (but are not limited to) scope of our efforts (requirements for the outcomes), technical performance (including quality and other ...ilities of the outcomes), the schedule for the work (so we don't have to do everything at once), the budget so we know the cost of the value produced), resources that do the work in exchange for money defined in the budget, risk to cost, schedule, and technical performance goals, and other attributes. A specific project will have specific constraints from each of these attributes. 

The relationships between these attributes is usually non-linear, random in some way (stochastic), and affects future outcomes. Because of the random nature of the attributes and the random nature of their relationship, simple linear, non-statistical projections of past performance used for future performance is most likely to be a disappointment.

Welcome to the Future

To answer the question what does the future look like when the past is a non-linear stochastic process, we need to be able to manage in the presence of uncertainty. With this ability, the future simply emerges and many times this futute is not what was expected. 

This is the role of planning. The best description of planning is

planning constantly peers into the future for indications as to where a solution may emerge. A Plan is a complex situation, adapting to an emerging solution. 
- Mike Dwyer, Big Visible

To be successful at planning we need to do many things. Since it is the future we're planning for each of these things requires an


Yes, we're never going to see it coming if we don't Plan. And to Plan, we need to estimate. And to estimate we need to learn how to estimate. So if we want to manage in the presence of uncertainty, here's a starting point...

When managing in the presence of uncertainty we're going to need to make estimates of the impacts of that uncertainty on our decision making processes for that emerging future. When we hear we can make decision in the absence of estimates, it's simply not true in any non-trivial manner. For trivial decisions, Do I start with the enrollment UI or the validation of IRS information UI, or do I buy a Mac with 4GB or 6GB of memory (assuming I can upgrade if I need more), making estimates is likely is little value. But for a decision like, do we switch all our legacy systems to SAP or JD Edwards, I going to need a credible estimate of the cost, schedule, resources, risks, and tangible beneficial outcomes for my enterprise.  So without a domain, context, the Value at Risk, the underlying processes for uncertainty (reducible and irreducible) and some other attributes, the notion of making decisions in the absence of estimating the cost and impact of those decisions in simply bad management.  For Those Interested in Decision Making Processes in the Presence of Uncertainty


Related articles Bayesian Statistics and Project Work Decision Making in the Presence of Uncertainty Your Project Needs a Budget and Other Things
Categories: Project Management

Don't Manage By Quoting Dilbert

Mon, 01/19/2015 - 00:02


When we hear descriptons of Dilbert style management, with posting and reposting Dilbert cartons showing pointy haired bosses Doing Stupid Things on Purpose and no actionable advice on how to take corrective actions - then you know for sure the poster is just whinning - move on nothing of value here.

Nothing_to_see_here (1)

Addressing programs we all encounter in the project management work, means answering - with the % Whys - real 5 whys, not just the suggestion of the 5 whys - the symptom, the problem, and the root cause in the broad categories of:

  • Process
  • Tools
  • Training
  • Environment
  • Communication
  • Management

Only with the answers to these categories in the three classes of assessment - what's the symptom, what's the problem that that produces the symptom, and what is the root cause of the problem - can corrective actions start.

None of this available - the conversation is a non-starter.

Related articlesWhat is Governance?Taxonomy of Logical FallaciesYour Project Needs a Budget and Other ThingsThe 5 Whys Process We Use to Understand the Root of Any Problem
Categories: Project Management

Bayesian Statistics and Project Work

Sun, 01/18/2015 - 18:42

BayesPictureThe use of Bayesian Statistics in project work is well developed. Here's an example of the approach. The drivers for each uncertainty can be modeled to produce on estimate of the risk of the project not producing the desired outcomes.

What is needed is an understanding of the prior probabilities of the drivers of the probabilistic outcomes, the relationships between the events created by these networks.

Screen Shot 2015-01-16 at 10.42.17 PM

The Bayesian Network Approach †

Bayesian networks provide decision support processes for a wide variety of problems where uncertainty and probabilistic reasoning is involved. The Bayesian Network is a directed graph with associated probability tables. The graph is the standard nodes and arcs. The nodes represent uncertain variables. Each node has a set of states that represent causal or influential relationships between the variables. There is a probability table for each node, providing the probabilities of each state of the variable.

For prior variables - variables without parents the table contains marginal probabilities. This is referred to as the prior distribution which represents the prior belief - the state of knowledge - for that variable. For each variable with predecessors (parents), the probability table has the conditional probabilities for each combination of the predecessor states. This is the likelihood function that represents how likely is a state of a variable given a particular state of its predecessor.

Bayesian Network are applied in situation that require statistical inference. The use in this instance knows some evidence - some variable states or events that have actual observations. These observed values represent the posterior probability of the event occurring. By applying Reverend Bayes rile in each affect node, they can influence other Bayesian Network node by propagating and modifying the probability distributions. 

Bayesian Networks Have A Natural Connection With Project Planning

These networks can:

  • Explicitly quantify uncertainty and model the casual relationships between the variables.
  • Enable reasoning from effect to cause as well as from cause to effect.
  • Make it possible to overturn previous beliefs in light of new data.
  • Make predictions ¬†with incomplete data.
  • Combine subjective and objective data.
  • Arrive at decision based on visible and auditable reasoning.

Bayesian Networks is a tool for decision support based on Estimating outcomes.

So when we hear that decisions can be made in the absence of estimates, ask for tangible examples of how this can be done, the basis (mathematics) of these processes, and examples in specific domains of how this can be done. If no answer is forth coming, question the veracity of the claims.

† "Inference in Hybrid Bayesian Networks using dynamic discretisation," Martin Neil, Manesh Tailor, and David Marquez, Department of Computer Science, Queen Mary, University of London

Related articles The Actual Science in Management Science Qualitative Risk Management and Quantitative Risk Management What is Governance?
Categories: Project Management

Probability and Statistics of Project Work

Sat, 01/17/2015 - 16:25

When I hear what are the odds of success for a project, it tells me there is an undertstanding gap in how statistical processes are used in projects to produce probabilistic estimates of three things:

  • The probability of completing¬†on or before¬†a need date?
    • All projects have a¬†need date for the delivered value.
    • If there is no¬†need date, then either the project is a R&D effort or the stakeholder of the project outcomes cares little about putting that project value to work to recover the investment.
  • The probability of completing¬†at of below a needed cost?
    • All projects spend money, usually someone elses money.¬†
    • How much will the project spend?
    • This is NOT the same as the budget for the project.
    • The Estimate to Complete (ETC) and the Estimate At Completion (EAC) are two numbers management uses to assure the project will show up¬†at or below¬†the Budget At Completion.
  • The probability of the delivered technology meets the required specifications?
    • When the Technical Performance Measures are written down, we can assess our progress toward meeting them in a probabilistic manner as well.
    • Will our database server farm be fast enough, big enough, reliable enough to meet the business need?
    • Waiting till we're done is not a good idea.

First let's echo Tim Lister's advice...

Risk Management is How Adults Manage Projects

Screen Shot 2015-01-16 at 3.34.55 PM

All the World's a Statistical Process

First let's look at a network of work activities. These are tasks with dependencies, whose durations have naturally occurring variances. These durations can never be an exact number, since the work is emerging or simply varying naturally. Each activity has a unique Probability Distribution Function, which may be similar, but also unique. This is the case as well when there are no dependencies. The probabilistic processes are still in place. Even some place like the Toyota production line, no work process takes exactly the same duration. So if these natural variances are unaccounted for, you're going to be late, likely over budget, and your favorite gadget may not work. This concept is the basis of Statistical Process Control.

Probabilistic Networks

For each process, the upper and lower ranges, along with the Probability Distribution Function can be used to model the range of possible outcomes for duration - for example. In the picture below the probability of completing on or before for the IOC (Initial Operating Capability) for Friday October 23, 2020 is 80%.

When we hear the bet we'll be successful, this is the number. It's not the right term the bet but this is the term. There is an 80% confidence of completing on or before 23 Oct 2020.

Monte Carlo

We can now connect the dots between individual activities and a network of activities with the next chart. This shows the dependencies, each of their variances and how those drive the variances in the outcomes.

Probabilistic Schedule

In the End the Discussion is About Domain and Context

When we hear about some new approach to making decisions in the absence of estimating the impact from those decisions, ask in what domain can that be possible. By possible I mean how can we make a decision by ignoring the principles of Microeconomics.

There may be domains in which that is completely possible. Below is a scale of projects I built awhile ago when working on an overall Program Governance engagement. From family gardening to building the USS Virginia there is a huge spectrum of techniques, processes, governance, tools, and approaches to increasing the probability of success. Having any discussion about the applicability of any idea has to start with what domain are we in.

Types of Projects

Related articles Qualitative Risk Management and Quantitative Risk Management The Actual Science in Management Science What is Governance?
Categories: Project Management

Qualitative Risk Management and Quantitative Risk Management

Fri, 01/16/2015 - 14:56


Qualitative risk analysis includes methods for prioritizing the identified risks for further action, such as risk response.

The project members must revisit qualitative risk analysis during the project’s lifecycle. When the team repeats qualitative analysis for individual risks, trends may emerge in the results. These trends can indicate the need for more or less risk management action on particular risks or even show whether a risk mitigation plan is working.

Quantitative risk analysis is a way of numerically estimating the probability that a project will meet its cost and time objectives. Quantitative analysis is based on a simultaneous evaluation of the impact of all identified and quantified risks, using Monte Carlo simulation.

Quantitative risk analysis simulation starts with the model of the project and either its project schedule or its cost estimate, depending on the objective. The degree of uncertainty in each schedule activity and each line‚Äźitem cost element is represented by a probability distribution. The probability distribution is usually specified by determining the optimistic, the most likely, and the pessimistic values for the activity or cost element. This is typically called the ‚Äú3‚Äźpoint estimate.‚ÄĚ The three points are estimated by the project team or other subject matter experts who focus on the schedule or cost elements one at a time.

Risk Response

Capturing risks, classifying them, prioritizing them, analyzing them is necessary to project success, but not sufficient. 

Mitigating (handling) the risk is needed. This is done in one for four ways: †

  • Avoid. Risk can be avoided by removing the cause of the risk or executing the project in a different way while still aiming to achieve project objectives. Not all risks can be avoided or eliminated, and for others, this approach may be too expensive or time‚Äźconsuming. However, this should be the first strategy considered.

  • Transfer. Transferring risk involves finding another party who is willing to take responsibility for its management, and who will bear the liability of the risk should it occur. The aim is to ensure that the risk is owned and managed by the party best able to deal with it effectively. Risk transfer usually involves payment of a premium, and the cost‚Äźeffectiveness of this must be considered when deciding whether to adopt a transfer strategy.

  • Mitigate. Risk mitigation reduces the probability and/or impact of an adverse risk event to an acceptable threshold. Taking early action to reduce the probability and/or impact of a risk is often more effective than trying to repair the damage after the risk has occurred. Risk mitigation may require resources or time and thus presents a tradeoff between doing nothing versus the cost of mitigating the risk.

  • Acceptance. This strategy is adopted when it is not possible or practical to respond to the risk by the other strategies, or a response is not warranted by the importance of the risk. When the project manager and the project team decide to accept a risk, they are agreeing to address the risk if and when it occurs. A contingency plan, workaround plan and/or contingency reserve may be developed for that eventuality.

† Project Risk Management: A Scalable Approach, Risk Management Task Group, CalTrans, June 2012.

Related articles The Actual Science in Management Science What is Governance? The Myth and Half-Truths of "Myths and Half-Truths"
Categories: Project Management

All The World's A Random Process

Thu, 01/15/2015 - 18:00

When we hear about making decisions in the absence of estimates of the impact from that decision, the cost of making that decision - the opportunity cost, which is the basis of Microeconomics, or best of all the possible alternatives that might result from that decision - the opportunity costs - we need to stop and think.

Is it actually possible to make a decsion without knowing these things?

The answer is NO. But of course the answer is also YES. Since decisions can't be made in the absence of those estimates. They are made all the time. A little joke in the extreme sports domain, which our son participates in, goes like this.

What are the last four words uttered by a 22 year old back country skier in Crested Butte before arriving at the emergency room?

Hey everyone watch this!

Any estimating the probability of clearing the 20 foot gap? Oh Hell No, let's go...


The decision making process here is the same as the decision making process on projects. There are uncertainties that create risk. There are uncertainties that are irreducible and there are uncertainties that that are reducible. Risk of crashing and breaking your collarbone. Riks of crashing the project and breaking the bank or breaking the customer releationship. 

Both these uncertainty types must be addressed if the project has a chance of success, just as both uncertainty types need to be addressed if there is a chance of landing the jump without breaking something in a very painful way.

There are a set of environmental conditions on the project and on the slopes that are helpful and as well as a hindrance to success. Modeling those is the starting point for making the decision to proceed. This is the taxonomy of uncertainty that must be assessed before proceeding with the decision. 

If you're 22 years old and believe you're immortal, then assessing these risks is rarely necessary. It's the let's just try this view of the world. After breaking both collar bones (separate occasions), crashing mountain bikes as well as crashing on skis and being transported down the mountain in a Ski Patrol Sled, feedback prevails and a more mature assessment of the outcome results.

The word uncertainty has a variety of meanings and has a variety of synonyms: error, information, vagueness, scatter, unknown, discord, undefined, ambiguous, probability, stochastic, distribution, confidence, and chance. These create confusion and from the confusion the opportunity to ignore them.

To evaluate the outcomes of our decisions, we need data

This data comes from a model of the world that allows us to translate our observations into information. In this model there are two types of abstraction. Aleatory and Epistemic. Aleatory implies an inherent randomness of the outcomes of the process subject to our decision making. Flipping a coin is modeled as an aleatory process, as is rolling dice. When flipping the coin, the random but observable data is the result.  Since the underlying probability function for flipping the coin has no observable quantities (we can see all the processes that go into holding the coin, flipping with our fingers, the air, the rotation of the earth, etc.) but we can't model the world of coin flipping directly. Instead we can only observe the results from that model.

This is many times the case on projects. The underlying physical processes, which themselves may be deterministic, can't be observed. So all we get is the probability that an outcome will occur. This is a Bernoulli model of the process. 

A Bernoulli trial is an experiment outcome that can be assigned to one of two possible states (e.g., success / failure, heads / tails, yes / no). The outcomes are assigned to two values, 0 and 1. A Bernoulli process is obtained by repeating the same Bernoulli trial, where each trial is independent. If the outcome assigned to the value 1 has probability p, it can be shown that the summation of n Bernoulli trials is binomial distributed.

The Epistemic uncertainty of the processes, both slope style skiing and projects, represents how precise our state of knowledge is about the world model. We can measure the temperature of the snow, we can measure the performance of the database server. We know the wind speed at the top of the kicker, we know the density of the defects on the code base from our last quality assurance review.

The epistemic uncertainty of the process pertains to the degree of knowledge of the model and its parameters. Epistemic comes from the Greek word episteme (knowledge).

We Need Aleatory and Epistemic information to make a decision

The system we want to make a decision about has a reality that can be modeled in some way, to some degree of confidence. When it is suggested otherwise, that simply is not true. Only when Black Swans are introduced - the Unknown Unknowns are applied - does this model not work. In the project world, thsoe UNK UNKs mean one of three things:

  1. We couldn't know - it was a surprise. We didn't understand our world model.
  2. We didn't know - we didn't have enough time or money to find out, or we were simply too lazy to find out. The world model was understandable, but we didn't look hard enough.
  3. We don't what to know - let's just try it and see what happens. We know the world model, but don't want to acknowledge the consequences.

This last situation is best represented In the famous Rumsfeld quote about UNKUNKs he failed to read The Histories, by Herodotus, (484-ca. 425 BCE), who cautioned not to go into that part of the world and engage in a ground war. It turned out bad for Alexander the Great.

So if you're the sort that accepts that decisions can be made in the absence of estimating the cost and impact of that decision - you're in the last two categories. 

A Final Thought

When it is suggested that businesses are seeking deterministic or predictable outcomes - which of course they are not, not can they since all business processes are probabilistic - those processes exist in only a few domains

Such precise processes are the antitheses of aleatory., this is the type of model most familiar to
scientists and engineers and include relationships such as E=mc2, F=ma, F=((G × m1 × m2) / r2).  So if you work with classical mechanics or the like, you can look for predictability. But if you work in the real world of projects or the business of projects - All The World's a Random Process - behave accordingly.

Related articles Decision Making in the Presence of Uncertainty The Actual Science in Management Science Planning is the basis of decision making in the presence of uncertainty Confidence vs. Credibility Intervals What is Governance? Your Project Needs a Budget and Other Things The False Notion of "we haven't seen this before"
Categories: Project Management

The Actual Science in Management Science

Tue, 01/13/2015 - 19:38

Screen Shot 2015-01-12 at 5.05.40 PMPlanning for an uncertain future calls for a shift in information management ‚ÄĒ from single numbers to probability distributions ‚ÄĒ in order to correct the "flaw of averages."

This, in turn, gives rise to the prospect of a Chief Probability Officer to manage the distributions that underlie risk, real portfolios, real options and many other activities in the global economy.

 - Sam Savage, Stefan Scholtes and Daniel Zweidler

There are some very serious misunderstandings going around about how management in the presence of uncertainty takes places in business. The basic conjecture is

Management Science's Quest: in Search of Predictability †

Let's start with a basic fact for all projects, all business processes - everything is a stochastic process. So searching for predictability is not a goal for any informed business or technical person or organization. If it is, then that defines the maturity of that person or organization. It happens, but it states up front little understanding of the underlying stochastic processes that create probabilistic outcomes of - Everything 

Here's a quick review of both processes in play in all activties. 


In the Decision Making Business there are four reasons why they are hard.

  1. Decisons are hard because of its complexity.
  2. Decisions are difficult because of the inherent uncertainty of the situation.
  3. A decision maker may be interested in working toward multiple objectives, but progress in one direction may impede progress in other directions.
  4. A problem may be more difficult if different perspectives lead to different conclusions. 

So to start with the notion of predictability - it is simply not possible in any real project or business domain, to speak about predictability in the absence of the underlying statistical processes that create probabilistic outcomes.

Any credible business or technical manager knows this. If predictability is assumed or even desired, then the naivety of the manager is the only likely source, or maybe the intentional ignorance of the statistical and probabilistic nature of business and technical process. But predictability is not possible in the sense of absolutes, only probabilities.

So let's look at some less than informed concepts that are popular in some circles ...

  • Predictability is a form of causality - predicting is separated from the source of predictions. And certainty the causality associated with prediction need not be there. Bayesian statistics and Monte Carlo Simulation, need not connect the predicted outcomes with the source of those outcomes - other than the source of the random variables from a¬†generating function.
  • Planning rests on the assumption we can predict - a Plan is a strategy for guiding our efforts to change something in the future or arrive at some place in the future. The Strategy is a Hypothesis and that hypothesis needs an¬†experiment¬†to test the current situation to determine if it will result in the desired outcomes in the future. This is core¬†design of experiments that we all learned in our High School science class. Plans describe an¬†emerging outcome.
  • Goals change with the observation of reality -¬†This dynamic adaptation process is what we, in the Agile community, call a feedback loop - this is true, but a target value is needed to compare that feedback information to generate an error signal. This is called Closed Loop Control and is the foundation of all control systems including Statistical Process Control system. And control systems that are adaptive in the presence of emerging dynamic systems. This is the basis of¬†Learning Systems in stochastic adaptive control.
  • Management techniques must not be based on the existence of a perfect, predictable future - this is a naive understanding of management. Perfect, predictable futures simplay do not exist anywhere for anything. All processes are random processes, many times not even stationary random process.

The suggestions above indicate the lack of understanding of fundamental knowledge of making decisions in the presence of uncertainty as described in the Making Hard Decisions book. The Journal Operations Research and Management Sciences, will put the science back in management science that those conjecturing the topics above seem to have missed.

In Journal papers and many books and related sources all the suggestions that we can't make decision in the presence of uncertainty, that simple minded conjectures like:

The basic problem with most perspectives on management today is that they are static analyses of a future environment. And all decisions are made because we believe we can predict the future.

Are simply not true, and better insight as to why they are not true can be had with straightforward reserch available by joining INFORMS or a variety of other professional societies.

So perhaps before making unsubstantiated claims about how modern statistical and probabilistic management processes are applied to business, some homework might be in order.

Related articles What is Governance? Your Project Needs a Budget and Other Things The False Notion of "we haven't seen this before" Engineering in the face of uncertainty: Stochastic solutions to structural problems A comparison of MC and QMC simulation for the valuation of interest rate derivatives
Categories: Project Management

Plan Management

Mon, 01/12/2015 - 15:05

When we hear ...

"Don't fall in love with your plan, it is almost certainly wrong "

let's reflect on advice from IEEE Std 730-2014

Plan Management: It is obvious that the project must be managed during its execution, but perhaps not so obvious that the plan itself must be managed. Despite the fact that nearly continuous change destabilizes any plan, the plan itself must not be allowed to float and become meaningless. The project must adopt a discipline for monitoring, reviewing and revising plans that they are stable in application while responsive to change.

Plan Do Check Act CIThe management process of IEEE/ISO 12207-2008 requires the manager to control the execution of the project by monitoring and reporting progress, and investigating, analyzing and resolving problems that arise.

The common phrase used by many in agile is Plan-Do-Check-Act.

This starts with Plan. In order to Do, Check, and Act, we need a place to start. The Plan. So falling in love with our Plan and then not, Doing, Checking the outcome, and Acting on the progress to that Plan, determining the varaincies from that Plan, assessing the impact from those variances, would mean we are not managing the project.

This would also mean that falling in love with the Plan would require use ignorning the very basis of management in the presence of change.

Plan First

So make plans, get feedback on the progress to plan, update the plan with this information, make a new or updated plan, schedule the work that the plan shows needed to be done, and measure progress to plan in units of measure meaningful to the decision makers.

Like the picture above, if you're going to embark on a day hike to Twin Sisters in Rocky Mountain National Park, you'd better have a plan. Not just for how long it's going to take, what your path is going to be, but have a Plan-B and likely a Plan-C when the weather goes bad, thunder storms come over the mountain, or any other disruptive event.

But most of all PLAN, and don't fall for that non-actionable statement that plans are almost certainly wrong. Because without the plan, you have no clue about what DONE looks like, how to get there, how long it will take, how much it will cost, or how you'll recognize DONE when it arrives.

Plans are Strategies for Success

They can be neither Right or Wrong, they can only represent you're current understanding of the emerging situation that leads to the current path to success.

Related articles Taxonomy of Logical Fallacies The False Notion of "we haven't seen this before" What is Governance? Your Project Needs a Budget and Other Things
Categories: Project Management

The Notion of Enterprise Software Development

Sun, 01/11/2015 - 20:09

The Road Map to SWEWhen we hear about some new fangled way of writing software for money, first ask in what domain has this new idea been applied, and what were the measures of effectiveness and measures of performance for that approach?

Then ask if there is any external frameworks applicable in that domain for developing software based business systems that govern the development, deployment, and operational aspects of the work? 

When the answer is yes, then next ask what are those governance frameworks? In a current engagement, ISO 12207 is the overarching framework for the development, testing, deployment, and operations of software systems. These systems provide services for Health and Human Services, Center for Medicaid Services and the disbursement of $492 billion. 

The management of software development at the Federal, State, Local level, and the commercial providers of services to those levels is guided by ISO 12207 which is composed of the following processes. 

Screen Shot 2015-01-11 at 10.50.17 AM

Now you may say we never ever have a project of this size. Such projects are insanely large and outside any domain we'll ever see.

So the inverse question is at what size do you no longer care about:

  • Budgeting?
  • Scheduling?
  • Resource management?
  • Estimate to Complete?
  • Estimate at Completion?
  • Risk informed estimates of cost, schedule, and technical performance of the deliverables?
  • Risk informed decision making in general?
  • Definitive descriptions of the deliverables in units of measure meaningful to the decision makers?

Then look through the 12207 list above and ask, which of these process areas has NO value for my project? That is, I don't need to know how to do this while spending other peoples money. 

I'll be just find in the absence of the guidance provided by this process area.

By the Way, the use of Agile methods fits right in with development work described in 12207.

With the answers to those questions, you can come back to see where you fit in the spectrum of projects.

Are you a Lone Wolf writing code for yourself or a customer where only you work or are you a member of an enterprise development team working on systems that are essential to the business in which they cannot fail, even cannot fail the first time they go live? Without the answer to this question, there can be no way to assess any suggestion, conjecture, or wild assed ideas about improving the probability of success for any software project. Related articles Taxonomy of Logical Fallacies Planning is the basis of decision making in the presence of uncertainty What is Governance? Good Project and Bad Project The Myth and Half-Truths of "Myths and Half-Truths"
Categories: Project Management

Planning is the Basis of Decision Making in the Presence of Uncertainty

Sat, 01/10/2015 - 23:46

Another twitter conversation - of sorts - prompted a thought about the purpose of planning, from the Quote of the Day post. In the enterprise IT world, planning and plans provide the road map for development and deployment of systems that implement the strategy of the business.


It is the process of planning that reveals the needed seqeunce of delivered capabilities. In the picture below a health insurance providers network application is deployed to replace several legacy systems over the course of time. Specific capabilities must be in place before later capabilities can be deployed.

This is a Plan for the development and deployment of those capabilities. From the plan, technical and operational requirements ae developed, software developed, tested, IV&V's, confirmed to be compliant with security and regulatory guidelines, business users trained, and providers (medical) trained.

Screen Shot 2015-01-10 at 11.41.06 AM

The sequence of these capabilities is well defined from the business operations side. The development of software inside each capability providing activity has some flexibility. But if we're going to arrive on the planned - NEED = date for say Data Store Lookup, then the sequence of that software has some flexibility. But not arbitrary flexibility. Further levels of planning are needed for resource availability - both people, machines, services, facilities, etc. 

Planning is Planning for Capabilities to Be Available to Deliver Value

The planning process is driven by Capabilities Based Planning. This process has some simple straightforward steps.


So when we hear deliver value on day one, we need to ask in what domain is that even possible. We need to deliver value on the day the value is needed for the business. Having a capability ready before the business is ready to use it is poor resource utilization planning.

We'll have spent money and time on something the business can't use yet. We may have made better use of that time and money on another capability. Which capabilities come in what order, at what time the business is capable of absorbing them is the first and primary role of PLANNING.

So the conjecture that plans are almost certainly wrong, begs the question - do you know what you're doing? If you're plans are almost certainly wrong - at least in the Enterprise IT business - you've got the wrong people managing the project. And you almost certaintly are wasting money developing things that won't be used.

This doesn't mean we don't explore, try different things to see if they'll work. But that work is also planned. It's not our money.

Domain is King

If you've got a project that is just you or maybe you and a few close friends that's going to take a week or maybe 6 weeks, planning in this manner is likely a waste. Along with estimating your work, keeping track of progress to plan, and even counting the money.

But if you're on a $200M enterprise IT development, integration, and deployment project with ~100 developers, testers, QA people, security, compliance, server ops, DBA's etc. you'd better have some notion of the order of work, the order of value delivery, the cost of that work, the probability of showing up on timem, on budget with that value in hand and how you going to herd all the cats that surround the project.

Related articles All About Me or All About the Mission? Your Project Needs a Budget and Other Things What is Governance? Taxonomy of Logical Fallacies Good Project and Bad Project Closed Loop Control The False Notion of "we haven't seen this before"
Categories: Project Management

Quote of the Day

Fri, 01/09/2015 - 17:17

It was overheard on Twitter

"Don't fall in love with your plan, it is almost certainly wrong "

A plan is a strategy for success. Strategies are hypothesise . Hypothesise need tests to verify - just like you learned in High School science class. That test is the Measures of Effectiveness and Measures of Performance of the outcomes from your project as well as the Technical Performance Measures that are used to take corrective actions needed to reach the goal of delivering value in exchgange for time and money.

The Plan describes where we are going, the various paths we can take to reach our destination, and the progress or performance assessment points along the way to assure we are on the right path.

These assessment points measures the ‚Äúmaturity‚ÄĚ of the product or service against the planned maturity. This is the only real measure of progress ‚Äď not the passage of time or consumption of money.

Wrong in the planning sense can only be wrong if you are managing your project Open Loop with no assessment of Effectiveness, Performance, Risk reduction, cost absorption, time performance or all the ...ilities associated with spending other peoples money.

So it is true that the Plan is wrong if you're not managing in the presence of uncertainty, feedback, and taking corrective action. 

Categories: Project Management

Decision Making in the Presence of Uncertainty

Fri, 01/09/2015 - 06:00

Decision theory is concerned with the problem of making decisions. Statistical decision theory is decision making in the presence of statistical knowledge, by understanding some of the uncertainties involved in the problem.

Decision theory deals with the situations where decisions have to be made in the presence of uncertainty, and its goal is to provide a rational framework for dealing with such situations. To make good choices we must calculate and manage the resulting risks from those choices. Today, we have tools  to perform these calculations.

A few hundred years ago decision making in the presence of uncertainty and the resulting risk had only tool faith, hope, and guesswork. This is because risk is a numbers game. Before the 17th century, our understanding of numbers did not provide us with the tools needed to make choices in the presence of uncertainty.

A good book about the history of making choices in the presence of uncertainty - risk management - is Against the Odds, The Remarkable Story of Risk, Peter Bernstein. These efforts culminated in Bernoulli's focused not on probabilistic events, but on the human beings who desire or fear certain outcomes to a greater or lesser degree.

Bernoulli¬†showed how to create mathematical tools to allow anyone to ‚Äúestimate his prospects from any risky undertaking in light of [his] specific financial circumstances.‚ÄĚ The is the basis of¬†Microeconomics of decision making, in which the opportunity cost of a collection¬†of choices can be assessed by estimating both the cost of that decision and the result beneficial outcome or loss.

In 1921, Frank Knight distinguished between¬†risk,¬†when the probability of an outcome is possible to calculate ‚ÄĒ¬†or is knowable ‚ÄĒ¬†and¬†uncertainty,¬†when the probability of an outcome is not possible to determine ‚ÄĒ¬†or is unknowable.

This becomes an argument that rendered insurance attractive and entrepreneurship¬†tragic.¬† 20 years ¬†later, John von Neumann and Oskar Morgenstern established the foundation of game theory, which deals in situations where people‚Äôs decisions are influenced by the unknowable decisions of live variables¬†‚ÄĒ in the gaming world, this means¬†other people.

Decision making in the presence of uncertainty is a normal business function as well as a normal technical development process. The world is full of uncertainty.

Those seeking certainty will be woefully disappointed. Those conjecturing that decisionscan't be made in the presence of uncertainty are woefully misinformed. 

Along with all this woefulness is the boneheaded notion that estimating is guessing, and that decisions can actually be made in the presence of uncertainty in the absence of estimating.

Here's why. When we are faced with a decision, a choice between multiple decisions, a choice between multiple outcomes, each is probabilistic. If it were not - that is we have 100% visibility into the consequences of our decision, the cost involved in making that decision, the cost impact or benefit impact from that decision - it's no longer a decision. It's a choice to pick between several options based on something other than time, money, or benefit.

Buying an ERP system, or funding the development of a new product, or funding the consolidation of the data center in another city is a much different choice process than picking apples. These decisions have uncertainty. Uncertainty of the cost. Uncertainty of the benefits, revenue, savings, increasing in reliability and maintainability.Uncertainty in almost every variable. 

Managing in the presence of uncertainty and the resulting risk, is called business management. It's also called how adults manage projects (Tim Lister)

The Presence of Uncertainty is one of most Significant Characteristics of Project Work

Managing in the presence of uncertainty is unavoidable. Ignoring this uncertainty is also unavoidable. It's still there even if you ignore it. Uncertainty comes in many forms

  • Statistical uncertainty¬†- Aleatory uncertainty, only margin can address this uncertainty.
  • Subjective judgement¬†- bias, anchoring, and adjustment.
  • Systematic error¬†- lack of understanding of the reference model.
  • Incomplete knowledge¬†- Epistemic Uncertainty, this lack of knowledge can be improved with effort.
  • Temporal variation¬†- instability in the observed and measured system.
  • Inherent¬†stochasticity¬†- instability between and within collaborative system elements
So Back To the Problem at Hand   If decisions - credible decisions - are to be made in the presence of uncertainty, then some how we need information to address the sources of that uncertainty in the bulleted list above. This information can be obtained through many means. Modeling, sampling, parametrically, past performance, reference classes. Each of these sources has in itself an inherent uncertainty.  So in the end, it comes done to this...   To make a credible decision in the presence of uncertainty, we need to estimate the factors that go into that decision. We Need To Estimate   There's no way out of it. We can't make a credible decision of any importance without an estimate of the impact of that decision, the cost incurred from making that decision, the potential benefits from that decision, the opportunity cost of NOT selecting an outcome from a decision. Anyone suggesting we can make decisions in the absence of estimating needs to provide clear, concise, actionable information with examples of how this can be done in the presence of uncertainty created by he underlying statistical processes of project work and the resulting probabilistic outcomes of those processes.   If you have certainty, you don't need to estimate. Measure your emperical performance to date, using the Most Likely value from thay performance and the variance project the future performance. Ignore risk, ignore naturally occuring (aleatory) and event based (epistemic) uncertainties in the uncerlying processes and proceed to spend yor customers money by applying faith, hope, and guesswork, just like they did before the 17th century.   Related articles What is Governance? Your Project Needs a Budget and Other Things The False Notion of "we haven't seen this before" Conveniently Unburdened by Evidence Taxonomy of Logical Fallacies
Categories: Project Management

Capabilities Based Planning

Thu, 01/08/2015 - 01:11

It has been conjectured that ...

What in the begining you thought you needed is never what you actually need

Fails to realize several critical success factors for project success...

  • Without stating what capabilities we need from the project, we have no means to assess any value produced by the project are worth our investment. These capabilities aren't requirements - yet. They're the mechanisms to earn back the investment. Typical capabilities sound like...
    • We need to process provider enrollment processes for $0.07 per transaction versus of current $0.12 per transaction.
    • I need the capability to move a brigade of 3,000 to 5,000 troops 100 miles up the coast in ten hours.¬†‚ÄĒ Gen Norman¬†Schwarzkopf

Capabilities decipher the intent of the leader (Commander)

  • Requirement always emerge. But capabilities should not, without rethinking why we're doing the project.
  • If we're changing our needed capabilities, we don't likley know what¬†Done looks like in any meaningful way and therefore are wasting out money¬†exploring.
    • Exploring in a research and development domain is mandated, but we should do that with the full knowledge and participation of the people paying for work.
    • Agile is essential a process to¬†buy¬† knowledge about things we don't know about.
    • Ask before spending our customer's money experimenting,¬†can we gain this knowledge in other, cheaper

Capabilities based planning (v2) from Glen Alleman So if we're on a project that doesn't know what Done looks like, we've got to ask a serious question Do we know what we're doing? If the answer is No, we may want to rethink why we're here. Here's a process to answer that question. Screen Shot 2015-01-07 at 4.08.01 PM And a framework to discover those answers Screen Shot 2015-01-07 at 4.10.20 PM Related articles What is Governance? Your Project Needs a Budget and Other Things The False Notion of "we haven't seen this before"
Categories: Project Management

Taxonomy of Logical Fallacies

Mon, 01/05/2015 - 04:50


When we hear something like estimates are the smell of dysfunction. That's a logical fallacy. Estimating is a business and engineering activities that produces numbers, confidence intervals on those numbers, and confidence interval on the confidence intervals.

We have an 80% confidence in completing on or before the 3rd week in November, 2011.

Or an example of Doing Stupid Things on Purpose 

Let's give Mary this new proiect which is 427 times bigger than any project she's every worked in her life and then expect it to be successfull and be shocked when it goes in the ditch and blame the business governance process of estimating as the root cause of her problems.

So don't fall victim to the logical fallacy of using the symptom in place of the root cause. Identifying the symptoms is very easy. It's the root cause and the corrective actions that's the had part.

The smell of dysfunction in estimating the cost, schedule, and technical performance in software development is not the estimate. It's bad management, it's uninformed application of microeconomics of decision making, it's laziness in learning how to manage the expenditure of other peoples money. It's missing governance.

Related articles Three Increasingly Mature Views of Estimate Making in IT Projects The Myth and Half-Truths of "Myths and Half-Truths" Your Project Needs a Budget and Other Things What is Governance? The False Notion of "we haven't seen this before"
Categories: Project Management