Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
It has been suggested that context is somehow irrelevant. This notion seems to start around the criticism of the¬†MoSCoW requirements method where requirements that implement the needed capabilities of a system are categorized.
MoSCoW means: (how it got the moniker I have no idea) and is a well developed method of eliciting requirements in a paradigm where the customer has some notion of what done looks like. If this is not the case, no method will help and the norms of project management are no longer effective - it's a¬†Chaos¬†mode. So in fact MoSCoW has no value and the suggestion may be true - in Chaos we don't need requirements elicitation methods.
But let's assume we're not in chaos and we actually would like to know something about what Done looks like, how much it might cost to get to done, how long it might take, and what of value will be produced when we reach done.¬†
But for projects operating in the other three uncertainty modes - variaton, foreseen uncertainty, and¬†unforeseen uncertainty,¬†the first question is actually the¬†inverse question ...
Why would we NOT prioritize the requirements using this or a similar mechanism?
Let's look at some content in the notion that prioritizing requierements should be ignored.¬†A requirements¬†Trade Space is mandatory on any non-trivial project for some very simple reasons:
Below is an example (at the end) of a program that has a minimum set of¬†capabilities fulfilled by the requirements, where the Program Manager and the Customer managed the success with¬†ruthless adherence to a schedule and the needed capabilities.
This example might serve as a paradigm for other¬†mission critical projects.¬†
Now the question is¬†can this paradigm or¬†context be applicable to small group, agile software projects? Turns out Star Dust had small team software intensive software components. As Shim says¬†think about it.
Focus on the nine I's (v9) from Glen Alleman Related articles What's Missing from Project Management in IT Agile Paradigm A Common Myth in Software Development They Never Saw It Coming
Proper Preparation Prevents Poor Performance
This simple phrase describes the core behavior of all project related work. For any platitude to survive contact with reality, it must be¬†true,¬†actionable, and¬†applicable¬†in your own domain.¬†
The notion that ‚Äúemergence‚ÄĚ is the driver for the participants in a project requires careful consideration. The technical or business requirements of the outcomes of the project are always ‚Äúemergent‚ÄĚ in some form. To be otherwise would require a preset group of activities, materials, technology, and personnel.
Failing to understand the subtleties of the continuous emergence of requirements, that enable the capabilities to be delivered, ¬†means failing to understand any project requires planning to deal with these emerging requirements. This emergence also pertains to the tools and processes used to manage and deliver the project. emergence is applicable to all elements.
Preparing for Emergence is a critical success factor in project work. Proper preparation is the foundation of programmatic and technical risk management. This means asking and answering the ‚Äúif ‚Äď than‚ÄĚ question, rather than the ‚Äúwhat ‚Äď if‚ÄĚ question.
Managing in the presence of emergence requires¬†directed decision making. Just letting things happen disconnects the outcomes of the project from the neeed capabilities produced by the project. These capabilities are the¬†immutable part of the business process. They can only change, when the strategy for the success of the business enabled by the project changes. To do otherwise, would be to disconnect between the investment in the project and the value produced by the project.
‚ÄúIf ‚Äď Than‚ÄĚ means knowing what can go wrong and how to respond to the following:
¬†Plans are nothing, Planning is Everything
The notion that plans are nothing but planning is everything is a common mis-applied quote. The clip art ¬†and content extracted from the current edition of Defense Acquisition Journal, November-December, 2013.
The ever-quotable Dwight D. Eisenhower, speaking to a group of industry executives who could be mobilized for war at a moment‚Äôs notice, was echoing an old adage¬†about warfare: ‚ÄúNo battle plan survives first contact with the enemy.‚ÄĚ
Eisenhower‚Äôs message, like the man himself, was straightforward: ‚ÄúThe reason it is so important to plan [is] to keep yourselves steeped in the character of the problem that you may one day be called upon to solve‚ÄĒor to help to solve.‚ÄĚ He was reminding them that warfare is inherently fluid, and that the only way to adjust to quickly changing circumstances is to have planned for such contingencies in advance.
For our project management domain, Plans are considered Strategies for the successful delievry of the project's capabilities. But this strategy is actually a hypothesis and this needs continual¬†testing. The best way of course if testing the hypothesis is working product at periodic points in the project. Some would say this test should be every few weeks. This of course - as always - is domain dependent. But at a minimum every month.
In our Earned Value Management domain, the montly Contract Performance Report (CPR), requires an assessment of physical percent complete to calculate the BCWP (Budgeted Cost of Work Performed). ¬†The planned cost of producing this value is BCWS (Budgeted Cost of Work Scheduled).
BCWP = BCWS x¬†Physical Percent Complete
On the planned day for the planned cost. Don't show up on the planned day for the planned cost and the planned physical percent complete? You can't be on schedule and on budget.
Now Add Technical Performance Measures and Quantified Backup Data
The determination of¬†Physical Percent Complete starts with Quantifiable Backup Data. Percent complete is never a¬†guess or an¬†opinion. Tangible evidence is needed. But tangible evidence against the planned percent complete, on the day we planned to be that percent complete. This is nearly identical to Agile's approach to delivering working software at the end of an iteration. Not quite, since agile allows you to drop planned deliverables without an penalty to the current performance measure. This¬†mortgaging the future by pushing undelivered features into the future would not be counted as complete in Earned Value.
We hear all the time, pithy statements about teams, teaming, team building, enabling team that provide innovation, and most every¬†soft skill ever thought of, applied to managing projects. At the project below, where I was one of many program managers, we came to realize one fundamental truth about teams -¬†
If you have a team who is your opponent?
Having played team sports at the High School, College, and competitive adult level, our teams always played against an opposing team. Much of the team building platitudes seem to ignore that teams are formed to play ¬†other teams, score points, win games, overcome obstacles, and participate in competition.
Who is the¬†other team if we're on a project team?
See if the notions below resonant when you hear short, cleaver words about the role of teams on projects?
Making the impossible possible from Glen Alleman Related articles Agile Requires Discipline, In Fact Successful Projects Require Discipline Trust 5 Benefits Team-building Consultants Can Offer Your Business Competency Model For Project Managers
The first question is¬†who pays for people learning from their mistakes? Then the next question -¬†with these mistakes, did the project advance in ways it would not have before we made the mistake?¬†
And did the money we spent learning from our mistake "earn its value?"¬†Or put in another way¬†was there a better use of the money to advance our learning other than building something that we considered a mistake?
There are other questions as well.¬†Why didn't we know this was going to be a mistake before we started? Or¬†could we have known this before we discovered it failed? Or even better,¬†was the failure we just experienced knowable at all?
This knowability question is the key to all project planning processes. If something is not knowable, then we could not have known, so we only discover our mistake after the fact. If it was knowable and we choose not to address it - ignore the potential problem - then we'll overrun our plan for no good reason other than we choose to ignore the knowable facts.
We may have a knowable issue, but can't afford to¬†learn (pay money to learn), but that's different.
These questions and others need to be asked and answered before we can make any assessment if¬†learning from our mistakes is a good idea. The alternative to¬†learning from our mistakes is to¬†do the job right the first time.
This of course is overstating the solution and somewhat silly, but it brings out a critical concept that must be addressed in any credible management process of spending other peoples money...
Who pays for the development team to discover their mistakes, if these mistakes could be avoided?
Once we have the answer to that question, we now have some decisions to make.
This is all fancy words for¬†someone has to pay for us to learn what we don't currently know. And we need to make the cost of this learning visible as soon as possible if we're going to be good stweards of other peoples money.
So What's the Point?
The notion of¬†Hiring smart people is pointless if they aren't allowed to make any mistakes¬†ignores the managerial finance obligation to determine who pays for these mistakes, is the budget for making these mistakes in the baseline authorization, and if not, are we going to overrun our planned cost for the project and dilute the Return on Investment calculation¬†that we sold to the owner of the project?
We seem to forget that writing software for money, means spending other peoples money for the expected amount of money (within the variance bounds) and showing up on or before some expected time (within the varaince bounds), with more or less the expected capabilities.¬†
This is a¬†Prime Directive (to borrow a phrase) for all projects. It's very doubtful the owner of the money says to us¬†hey boys and girls, go out there and experiment around to figure out what will work and what won't work without actually budgeting for that¬†experimenting work.
When there is explicit budget to¬†experiment that's called¬†explicit work for discovering what we dont' know, that is needed before we can proceed. We'd find that budgeted work in our plan for the project. No problem, all part of the normal project management process.
But the notion of¬†Hiring smart people is pointless if they aren't allowed to make any mistakes¬†needs to address¬†Who, What, When, Where, Why, and¬†How this discovery process is going to get paid for FIRST, then we can start¬†failing on purpose to make the follow on work better.
But let's start with a fundamental principle of all business, not matter the domain. If we expect to generate revenue or produce some measurable value from our work efforts, we need to measure those beneficial outcomes in some agreed upon units.¬†
Dollars is one unit of measures common in the business world. If we work are a¬†for profit company, it is likely the managers of the business use that unit when they discuss project work. If we work for the government, budget is used many times in place of cost, but cost - in dollars - is still in the equation. If we work for a non-profit or a not-for-profit budget and cost are half the equation. Intangible benefits the other half.¬†
But it is unlikely any place we work, there is not some discussion of cost, budget, or expenses. So no matter what our personal opinion creating estimates of our work effort are, we can't escape the fundamentals of business.¬†
We need to know the cost of something before we can assign a value
Let's look at some deep¬†truths about estimating
So The Big Question?
These issues are not unique to software development. They are found in every project domain. Public works, bio-pharma, energy, government contracting.¬†
Estimating is hard, it's important, and needs to be addressed in much more detail, before being dismissed as simply something management wants.
The recent post about the 5 Ways To Rethink Software Projects has elicited a response from the author stating I was being unfair to his thesis that in the end - Number 5 - we didn't need to make estimates. Of course without any domain or business processes governing the expenditure of other peoples money, it's hard to tell where the proposed ideas would fit it.
Going back to basics with the clip below, I'm reminded we seem to have lost all visibility to why estimating is needed in the first place. The clip below is from a 1970 Naval Post Graduate School (NPS) thesis about the troubles with estimating. Many of the problems are still in place today. So you may say we haven't move beyond the orginal problem. But that would be too simple. The papers below, including this 1970 thesis shows how far we have come.
I'd conjecture to move beyond twitter exchanges and interviews of personal anecdotes we need to test the notion of how to improve estimates in the market place. Anecdotes are fine, if derived from actual hands-on experience in a domain or similar domain. The bottom's up discussion many times ignores governance and simple business processes and is stuck on the¬†you're not listening to my great idea¬†that XP was in early on. As Carl Sagan suggests
"Extraordinary claims require extraordinary evidence."¬†Carl¬†Sagan
The¬†systems referenced below are not in the domain of the CIO magazine article. But the principles for forecasting the cost of a project before - or near the start - work is still an important part of Project Governance.¬†¬†The notion of project governance seems to have been skipped over when we look at the problems of estimating. From the management of the ACA web site, to large ERP systems, to major weapons, to major science projects - managing the project with the paradigm of¬†governance is actually harder than it looks. It is hard because the¬†value at risk is more important than the interests of the individual. The subordination of self interest some times is more difficult than seeing the bigger picture of managing other peoples money.
So we can choose to explore how to improve our estimating capabilities or we can seek out ways to avoid estimating all the together. But in the end, those with the money likely want us to learn to be better estimators. We should ask them, before assuming they want us to stop estimating the cost of work - #5 in the list of ways.
That's the message here. If you have a customer who will pr ovide you with funding in the absence of knowing - in some way, with some confidence - how much it will cost in the end, I'd suggest you proceed and hope for the best.
There is a common idea found in some software places, especially one that start with the notion that we don't want to do estimating, or estimating is not needed when spending other peoples money, that estimating is hard, is the same a guessing, and produces results not valuable or not needed by the customers.
Let's start with a paradigm. If you choose to not estimate because you don't want to learn, well not much to do about that. But let's read what Barry has to say.¬†
There is no good way to perform a software cost-benefit analysis, break-even analysis, or make-or-buy analysis without some reasonably accurate method of estimating software costs and their sensitivity to various product, project, and environmental factors.
Professional organizations for cost estimating
Ignoring for the moment the problems with the logic of ignoring the needs of those providing us money, here's some places to start in the domain where estimating is the basis of good project management
There are many more approaches. Google will get you them all. But in the end you actually have to want to learn how to do software estimating. If you start with the notion that estimates are either not needed, serve no purpose, or you simply have heard this from a source that has no connection to your domain, nor wants to - then you'll probably want to ask a few more questions:
So when you hear someone suggest we need¬†question everything then question the questioner if he has any experience at all spending other peoples money on a project where the¬†value at risk is substantial to the point if the project fails he'll ge fired.
No? not working in that domain -¬†manifestly important and nearly impossible - according to Edwin Land, then that advice about estimating may be a bit too narrowly focused, limited in domain for your need. You get to decide.
But if you do maybe these will help as well
It's not clear how this notion came about, since I haven't talked directly with those making statement like that. I'll have to estimate myself why this might be true.
One reason might be they haven't been exposed to the statistical processes used to make estimates in a variety of domains, not just project management. When we encounter the need for estimates, we need to come in contact with probability and statistics. All processes on projects are statistical in nature. This produces uncertainty and uncertainty results in probabilistic outcomes.¬†
Estimating means searching in this probabilistic "space" for a number that is¬†close enough¬†to answer a question.¬†How many people are in line for movie tickets inside? Should I buy my tickets at the kiosk outside? Or maybe, how long will it take me to drive from my office to the airport parking garage for my flight on Wednesday morning? These are not exact numbers, these ae quick estimates. The answer to the first question can be made with a quick look inside the theater. The second comes from the experience of driving to the airport several times a month over the past 20 years. One is informed by observeation of the current situation, one informed by past experience under a variety of situations.
Let's start with a simple definition of an estimate - it is an approximation of some value for some purpose, even if the data is incomplete, uncertain, or even unstable. Typically the estimate is derived from a statistical source of data - either observed or referenced in some way.
Making Simple Estimates
When we don't have the exact answer of some value, we can make an estimate of that value. The result is a number and a confidence on that value. Say we need an estimate of how long it will take to drive to the airport. It's 52 miles. Some on surface streets, some on open 2 lane roads, some on freeways. Knowing the speed limit for each of these segments can get us an approximate time portal-to-portal. We can improve the estimate, knowing the weather conditions and the traffic flow on the surface streets.¬†
Let's look at five easy steps for making estimates (abstracted from¬†The "Mother of All Guesses" -¬†A User-Friendly Guide to Statistical Estimation,¬†by Francois Melese and David Rose¬†COPYRIGHT ARMED FORCES COMPTROLLER, 1998)
Making Estimates for Project Work
With the steps above we can start to make estimates of our project work.¬†
But first let's kill some myths used to not to make estimates.
We've heard them all and more. There is an endless list of reasons for not doing most anything on a project, but that doesn't remove our obligation to be stewards of other people's money we are spending with expectation (estimate) of some value in return.
Let's start with a core principle of all project work, and possibly a principle of all life's work when we encounter money.¬†
We can't determine what something is worth until we know what it costs.
This is a crass capitalist point of view I know. But we live in a crass capitalist society and¬†them's the rules. Don't like the rules, it might be better to look somewhere else for work other than spending other peoples money without knowing how much it will cost in the end and when we'll be done.
We all know and maybe even experience a Dilbert's paradigm when it comes to projects. But that doesn't make it right. We hear this some times from management. But we also hear it as an excuse for not making estimates from those who should be looking after the cost needed to compute the¬†Return on Investment.
A Few More Myths
Below are some links to other Blogs on this topic, explore, think about our role in managing other peoples money.Related articles How To Estimate Almost Any Software Deliverable Measurement of Uncertainty Credible Estimating Processes Everything is a Random Variable Estimating Accuracy
When we hear about software development methods, it's critical to understand in what domain they are applicable. Here's an analogy for that question. Without an answer to this question there is no basis for comparing units of measure. What might be a wonderful approach in one domain would be unusable in another - this is the case for all the examples.
A solo or 3 team project with a customer in the same building would never apply DOD 5000.02, FAR 34.2 Earned Value flow downs. Building the flight avionics to fly to ISS would never use #NoEstimates and emergent requirements on a weekly basis for a 200 person development team spread across the United States.
So before any conversation - beyond shared ancedotes and asking audiences to¬†think about it, because I can't tell you what to do¬†- can take palce, we need to establish the domain, the¬†value at risk, and the governance model applied to our work. Then we can talk about shared ideas.
Paradigm of agile project management from Glen Alleman Related articles Performance Based Management It's Not the People It's the Process? Five Ways To Rethink Software Projects Agile Project Management
Matt Heusser's article brings up some interesting points. Let's look to see if there are any limitations from a domain or context point of view. By domain I mean,¬†in what taxonomy are you writing software for money. By context I mean¬†what are the constraints or governance guidelines¬†in that domain.
1. Make the amount of money small
This is a version of time boxing. It limits the value at risk¬†for the development process. This bounds the risk process. In exchange for the¬†total loss of doing the wrong thing information can be found. This is also called tuition cost.¬†
Issue:¬†We may not have all we need to forecast the total cost and schedule. Projects in many domains aren't made up of small chunks of themselves. So we'll need to confirm that the sum of the parts results in the whole. Integration, test, verification, architecture, interface management, and many other¬†Systems Engineering aspects need to be involved in some way.
2. Fund a pilot that delivers working software and use this model to forecast schedule
This is buying a¬†reference class. With the reference class - all be it a class of 1 - a forecast of future cost, schedule, and technical performance. We need all three in the reference class.¬†
Issue: this is a larger issue of Number 1. With the pilot can we be assured that work can be scaled? Verification of that will have to be part of the pilot or a follow on. Then comes the confidence intervals for how that scaling will interact with the other - yet to be developed - components of the system. Is the scaling linear, nonlinear, stochastic interactions, and a whole raft of other¬†discoverable processes. Each needs to be planned and budgeted.The ACA web site is a recent example. A UAV I worked where the engine didn't have enough thrust after the final integration. Etc. etc.
3. Move from contract negoiation to partnership
You've simply transferred to responsibility to estimate the cost and schedule to someone else. A single example - in the article - is not the basis of a syndicatable¬†process. So this example, while interesting, probably isn't going to go too far with someout means to address the¬†Estimate at Complete needed in most non-trivial software development projects.
Issue: Still don't know how much the project will cost in the end.
4. Employ Start Stop Heuristics
Seems like just another version of time boxed budget and schedule.¬†
Issue: still doesn't address the¬†Estimate at Complete before actually reaching the end or near the end of the project.
5. Drop Estimation From Your Estimation Process All Together
Another version of time boxed budget. Someone has to know how much money is needed to run the business on an annual basis. Or how much money will be allocated to do some work on an annual basis. This is called¬†Level of Effort. Work until the money runs out, we give you more money, or tell you to stop. PayPal works in this way - sorta. The prioritization of the work is the responsibility asking for the outcomes. They have a budget. Give that budget (not funds, budget) to the development organization in exchange for delivered code.¬†
Issue: The project will cost a know amount. We don't know what we'll get for that amount. But that might be OK. Once work is been going for awhile, a¬†Reference Class¬† can be built to allow that question to be answered, assuming the requested software fits inside the reference class in some way.
So In The End
The 5 suggestions don't have a domain beyond the single examples. And the suggestions don't seem to have a way to forecast the bounds of the project with an¬†Estimate at Completion beyond the use of the reference class of the project itself. This¬†self referencing¬† reference class seems a bit sporty.
So yes, there are some ways to develop software in the absence of formal estimating. Although 2 of the 5 are actually using reference classes to forecast.¬†
Those paying for the work to be done, still have to come up with some upper bound on cost, schedule, and technical capabilities for that cost and schedule. These 5 suggestions are a start. But we don't yet know where they can be applied. If they have been applied outside of specific anecdotes, and if they are scalable beyond the personal anecdotes.
No problem. This notion of not estimating is still evolving. At some point, answers need to be forth coming and the¬†Yoda-style¬†conversation replaced by business conversation -¬†what did you do with my money?
In a meeting today on the difficulties of increasing the probability of project success in our domain (DOD/NASA). There were three sources of uncertainty around cost, schedule, and technical forecasting
The program runs into things that cause cost and schdeule overruns:
We really shouldn't be doing #1 and #3. #1 means we didn't look hard enough. #3 means we¬†can't handle the truth. #2 is the definition of uncertainty. But is it uncertainty because the project, didn't know or didn't want to know.
Doing stupid things in purposeRelated articles Reference Class Forecasting for Software Estimating Facts and Fallacies of Estimating Software Cost and Schedule Who pays? How NOT to Estimate Anything Cost (and Schedule) Estimating Foundations
When managing projects that are funded by other people's money, we are obligated to know something about the probabilistic outcomes of our work. Without the ability to make forecasts and estimates of these outcomes , we are relegated to the category of labor.
As Project Managers, we must learn to estimate and forecast to some level of confidence. This is not guessing, that is child's play. Choosing not to estimate is also child's play. We must learn to make estimates based in sound statistical and probabilistic principles. Without that, the very notion of Return on Investment is intentionally ignored.
Return on Investment = (Value produced - Cost to Produce) / (Cost to Produce)
It's that simple and it's that hard. The cost and value variables are probabilistic estimates and must be treated in that way. But without knowing either of these, to some degree of confidence, we cannot make informed management decisions.
That is the primary role of project management or engineering development -
Make Decisions based on evidence in the presence of uncertainty
The NPR story about the Affordable Care Act web site is one of those¬†out of your domain stories. (follow picture link to story) that we see in the popular press.
First is the¬†way over generalized descriptions of Waterfall and Agile. Things like¬†Waterfall development favors listing a huge set of requirements for a system up front, letting developers go away for months (if not longer) and expecting a huge software product in the end.¬†
The simple minded description of agile is stated as¬†The agile method does the opposite, favoring work done in phases, delivering "minimum shippable" parts of a software system in weekly or biweekly cycles. This allows for iterating ‚ÄĒ or adjusting to hiccups discovered in the previous cycle, changing features or quashing bugs quickly and avoiding getting an end product that doesn't look a thing like what your users need.
Let's get some clarity here. The conjecture that agile would have somehow¬†safed the ACA program is just that pure conjecture.
But let's first look at some possible root causes (page 5 of the linked briefing):
So let's look at page 15's corrective actions and their sources:
So What Does All Mean?
The five immutable principles, practices, and processes of project were violated.
The sofwtare development method is independent of these principles, practices and processes. There is no way yet to determine if the ACA web site violated any of these 3 sets of 5. Neither could the author of the NPR story. Or anyone she used as the source of information. No one will likely know until GAO does a¬†root cause analysis of the management processes.¬† The report seemed to be missing a few things.
Attending and speaking at the Integrated Program Management Conference. Several themes have emerged during the past two days, one more to go.
Budgets are Mandatory for any Credible Management of Other Peoples Money
All project elements are actually random variables. Funding, productivity, quality, efficiency, cost, schedule, risk, reliability - all the¬†illities actually. All the independent and dependent variables of any project. The project can be a construction project - pouring concrete and welding pipe. It can be a software project - developing a capability used by an internal IT customer, all the way to a game used by millions on their smart phones. Or a project flying to Mars with intensive technologies of hardware, software, and even inventing new physics.
Walts paper speaks to how¬†statistical process control¬†can improve the software development process. This process does not assume what development method you are using.¬†
Since all the elements of the project interact in some way, many times in a non-linear way, most times stochastically (statistical dependencies), we need to not just acknowledge this but be able to manage in the presence of the uncertainties that result from these underlying stochastic processes. As well these processes may not be¬†stationary, they may change as a function of time, as a function of other changes, or just randomly change -¬†stuff happens.¬†
So what does this mean in practice:
Since there are connections between all the elements, the first connection - the one most interesting to those paying for the project are the cost and technical risk connections. I know you may think the¬†produced products are the most important - the production of working software for example. But that software appears from the project through the expenditure of time and money. And without knowing how much time and money is needed to produce that outcome - to some level of confidence - all the modern product development techniques are going to help. Why you say?¬†'Cause you've got no money.¬†
In the end software intensive projects - enterprise class software intensive projects - are complex systems. Projects where all project participants are in the same room, with the shared vision of the outcome - not so complex. But that's not where the bulk of the IT spend lives. As well the notion - perhaps naive - that complex projects can be broken down into simple projects ignore the core principle of all complex systems.¬†The coupling and cohesion of the system elements may not be known upfront and in fact may not be knowable until it is too late. If we look at the primary driver of project failure - the people problem - we can see that¬†Coupling and Cohesion is the primary source of difficulties.
So In The End
To increase our probability of project success we must learn to manage in the presence of uncertainty. This uncertainty is created by either the¬†lack of knowledge (Epistemic uncertainty) or the statistical uncertainty created by the naturally occurring variances in the processes (Aleatory uncertainty).¬†
Both these uncertainties create risk to the project variables - cost, schedule, technical performance. These uncertainties directly participate in the estimating and forecasting processes designed to seek out possible future behaviour. Attempts to control the behaviour that results from the aleatory uncertainty is fruitless. Muda. A waste of time. The only way to protect against the aleatory uncertainties is with margin. Cost margine, schedule margin, technical margin.¬†
The epistemic uncertainty can be addressed by spending money to¬†buy knowledge. This is the definition of epistemic - Epistemology. We can spend money upfront to reduce the uncertainty - this is the basis of all iterative and incremental development systems, be they Scrum or DOD 5000.02. Or we can¬†hold in reserve, money, time, capacity to address problems when they¬†come true probabilistically.¬†
Pay me know or pay me later.¬†
So Now What?
To have any hope of success we must have some level of confidence that all the probabilistic processes, driven by their underlying statistical processes have some level of understanding. Blindly pulling work off a queue, without knowing the arrival rate to the queue, the throughput of the processes servicing the queue, the quality of the products produced by that servicing, and the acceptance rate of the resulting products by the consumer - naive at best and simply bad management at worst.Related articles Uncertainty is the Source of Risk Both Aleatory and Epistemic Uncertainty Create Risk Time to Revisit The Risk Discussion Statistics, Bad Statistics, and Damn Lies Deterministic versus Stochastic Trends in Earned Value Management Data
All decision making starts with a domain and context. Without domain and context we have no way to understand the environment in which we operate and the well defined objectives needed to assess our decision making process. We have no way to address the natural uncertainty of our project work. We can not make a decision in the absence of alternatives.
We can only address uncertainties - both reducible and irreducible - by understanding the probabilistic outcomes created by the underlying statistical processes. Assessing the resulting outcomes against our goals is the basis of all probabilistic estimating and forecasting activities of any credible decision making processes.¬†
It ¬†is the probabilistic and statistical information that informs our decisions. To ignore this means we are driving in the dark with our head lights off.Related articles Quote of the Day - More Statistics for Decision Making Quote of the Day - Correlation, Causation, and Statistics Time to Revisit The Risk Discussion Both Aleatory and Epistemic Uncertainty Create Risk Deterministic versus Stochastic Trends in Earned Value Management Data
There is a great blog post on TechWell about Black Swans and project failure. IT project failure, but any project failure is appropriate.
There some ¬†important points here on several fronts. For projects to actually fail means there were events that occured that the participants did not see coming. There are several sources of¬†not seeing it coming. The first of course is you didn't look. You didn't look because you choose not to look. Or you didn't know where to look. Our you couldn't recognize the train wreck coming before it happened. Or when you looked the wreck didn't seem as bad as it turned out to be.
The second issue is¬†Black Swans are used improperly in the project domain. Taleb's paradigm is in the sociologically driven finance world where systems operate in the edge of chaos. This chaos is a market process where hidden information drives behaviour. Many time information is not only hidden the people creating and using the information intentionally lie to each other, the public, and the oversight agencies. The result actually is a¬†Black Swan. But of course in the best of example of a¬†Black Swan - the finance collapse - many saw it coming and made a killing. See¬†The Big Short and¬†All the Devils Are Here¬†are two sources describing how many saw it coming, knew what to do about making money and followed through on their plans.
The last thing is, many in project management, especially on the agile side for some reason, use¬†Black Swans as an excuse for not doing their job of looking, planning, mitigating, controlling, and actually managing the processes of a project. All elements of¬†doing stupid things on purpose. This phrase came from an incident at a DOE Nuclear Weapons plant when a fiber optic cable was cut¬†by acident that knocked out the safety and safeguards controls to several buildings. All processes, all drawings, all step-action-plans were followed. But the final decision was made without any actual test of what woudl happen is they cut the wrong cable.¬†
So Are There Black Swans On Projects?
If we're deploying an ERP system, build a web site - even the ACA web site, integrating a collection of legacy systems, probably not. I say probably, because there may be an unknown behaviour of the system that can't be determined before it is put into place.
If we're building a particle accelerator for the first time and haven't developed the control systems for the super-conducting magnets there may be unknowns. But of the power system for those magnets is located under trees that could fall, well that's not a black swan. That's not looking for risk.
The whole Black Swan paradigm comes down to risk management. Risk management has several issues as well. We may not be able to afford to look for risks. When we find them we may not be able to afford the mitigation costs. Or we may not actually have to political will to take mitigation's. The O-Ring and Foam mishaps at NASA are examples of a combination of these.
For IT projects, we're not inventing new physics, we not flying to outer space, we're just writing software for money. So why blame failure of Black Swans? Because it's the easiest thing to do. And as a colleague is fond of saying¬†They Can't Handle the Truth.
So no more Black Swans until you've exhausted every avenue of exploration of the source of project failure. Identified those sources - risks, categorized them, analyzed them, made handling plans, and adjusted your project plans and schedule in the response to them. Then the only thing left are the things that were¬†Unknowable. Only then can you invoke the¬†Black Swan¬†excuse.
The answer is simple. If you don't look ahead to what something might cost, when it might be done, what impediments you might encounter, how you're going to measure progress beyond the passage of time and consumption of money, then the pesky¬†Black Swans are going to bite you. Estimating is not looking forward in an attempt to control, but looking forward in an attempt to avoid. Not doing that? You're a train wreck waiting to happen.
Unless we know what capabilities¬†we need at the end of the project, and know those somewhere near the beginning of the project, have a Plan¬†to produce those capabilities at the needed time for an anticipated budget, we're going to be disappointed in the result.
Identifying the needed capabilities is the first and most important practice for the success of any project. To achieve the project's objectives or a particular end state, we need to define these capabilities through¬†Measures of Effectiveness. We can do this through scenarios from the customer's point of view. The effectiveness measures describe how well the results from the project enable a business process or fulfill a strategic mission of the business - in this case enforcing the will of the emperor. These measures must be in units meaningful to the decision makers. In the case Darth Vadar.
When we let those needed capabilities emerge¬†we have likely¬†disconnected the products of the project from the mission¬† or vision¬†of the business. Requirements can emerge to fulfill the capabilities. But the capabilities need to be steady and ¬†focused on mission outcome, otherwise we are just on a spend plan. We'll have no way to know what done looks like until the money runs out.
Here's an example of an increasing maturity of the capabilities for a health insurance provider network ERP system.
This approach is not usually found in traditional or agile projects. These approaches start with requirements, stories, features - all related to the technical aspects of the project. A Master Plan¬†is developed that shows the needed capabilities (above), the order of delivery of these capabilities, and the dependencies between these capabilities.
The increasing maturity¬†of the project's outcomes is then stated in this map. Technical requirements implement the capabilities, so they need a plan - schedule actually - as well. Small incremental capability delivery is the best approach. This is obvious and the basis of agile, but not always planned that way.
This plan¬†contains all the steps (delivered capabilities) needed to increase the maturity of the of the project from the start. This does not mean we know how¬†these capabilities are going to be delivered, but it does show what DONE¬†looks like at the end of the project. In this example Phase 1 - Go Live.¬†
This approach does not define any specific development process - agile or traditional. It is the basis of success for any development process.
Capabilities Are the Vehicle For Delivering Value
The Capabilities and their¬†Measures of Effective need to remain stable over the life of the project. Without this reached¬†DONE¬†with any sense of budget and schedule will not only be difficult, you will be¬†rubber banding the baseline used to measure your performance. You'll have turn a¬†project into an operational activity. Projects have defined outcomes, defined periods of performance and associated budget.¬†Operations can be fixed budget, open ended periods of performance, and emerging outcomes. But reaching a defined description of¬†DONE becomes much more difficult.Related articles Agile Project Management Requirements Elicitation Performance Based Management Capabilities Based Planning Good Project Management Advice
For as soon as the distribution of labour comes into being, each man has a particular, exclusive sphere of activity, which is forced upon him and from which he cannot escape. He is a hunter, a fisherman, a herdsman, or a critical critic, and must remain so if he does not want to lose his means of livelihood.
(from Marx‚Äôs German Ideology)
Marx goes on to say...
[I]n communist society, where nobody has one exclusive sphere of activity but each can become accomplished in any branch he wishes, society regulates the general production and thusmakes it possible for me to do one thing today and another tomorrow, to hunt in the morning, fish in the afternoon, rear cattle in the evening, criticise after dinner, just as I have a mind, without ever becoming hunter, fisherman, herdsman or critic.¬†This fixation of social activity, this consolidation of what we ourselves produce into an objective power above us, growing out of our control, thwarting our expectations, bringing to naught our calculations, is one of the chief factors in historical development up till now.
With this in mind, our ability to influence the outcome of any endeavor, as specilist in our narrow field, that involves spending other people's money is limited. Without understanding this basic tenet of economics, we have little to say unless we can show what something will cost in exchange for its value. Those we're asking to fund our efforts in exchange for that value will have a difficult time listening. This may be considered undesirable by some, but it is a simple fact of life in our modern world of capitalism.
When we ask the question what software domain are you applying your improvement approach to? and the answer is software, it might be useful to put a framework around the answer to provide a basis for discussion. The figure below is a sample from several sources. An expansion of this picture is at the link below. This is the taxonomy used for the financial assessment of the software¬†business. It's one way to look at the various domains, focused here on commercial products and services.
So when you hear of the next big idea in development of software, and ask what kind ¬†of software do you work on, this might be a guide. When the answer is silence, maybe those proffering the solution need to look as well and test their ideas outside their personal anecdotal experience to see of anyone else shares their enthusiasm. Along with this, maybe asking if that wonderful idea has come in contact with any governance processes of spending other peoples money.Related articles Facts and Fallacies of Estimating Software Cost and Schedule