Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Woody asks some important questions that have straight foreword answers in the¬†software for money domain. These answers come for the business side of a software development organization. It's the business of software that needs estimates. Given a choice no developer really¬†wants to estimate her work. I never did, I just wanted to code. But once there is an understanding that my paycheck didn't come from the Bank of America, even thought that's what it said at the top (above TRW), then I slowly learned our customers paid of some value (in that first job a missile defense radar system) they needed to know how much it was going to cost to produce the needed capabilities.¬†
So here's some good questions that have answers from the point of view of those¬†paying for the cost to produce the software for the customer.
So some further questions?
Knowing what it costs to produce a specific value is a critical success factor in any business. Writing software for money is no different.Related articles Estimates and all that... Resources for Moving Beyond the "Estimating Fallacy" Danger Will Robinson Managing Risk and Uncertainty in Agile Why Estimating Is Mandatory It's an Estimating Problem not A Problem with Estimating The Value of Making An Estimate
The discussion (of sorts) on Twitter around "no estimates" - what ever that actually means, since there is no definitive description other than exploring¬†- brings me back to my core program management, project management, writing software for money, designing algorithms for identifying moving targets in radar systems, and other¬†software engineering experiences.
Let's start with a fundamental pricniple of all product or service development, either internal or external. While leading a couple hands full of project managers at a large Department of Energy environmental cleanup site, where software development was a critical success factor - and by the way we introduced eXtreme Programming into an ANSI validated Earned Value Management System - our external consulting firm gave us some good advice. We were bidding our technology and services at another DOE site, with similar cleanup problems. We were working on strategies, balanced scorecards, systems architectures, and the like.¬†
That's all nice boys and girls, but here's some fundamental advice - our customer has money and we want it. What's it gonna cost and when will you be done providing the capabilities to close this site?
That's it, that's the winning strategy. The customer has a need, we want to providea ¬†solution to it. How much will it cost and when will we have it. If we can answer those three questions - cost, schedule, delivered capabilities, with attached unassailable beneficial outcomes - we will win. This is a business strategy. All the Balanced Scorecard presentations, examples of past performance, deep references of success, are all for naught if the customer can't afford our solution. It comes down to this - and this is where I learned this from the Managing Partner of the Big 6 (at that time) consulting firm.
You can't know the value of something until you know its cost
That's a fundamental principle of all business transactions. Value is always exchanged for cost. We do this when we buy a Venti Nonfat Latte at Star Bucks. We do this when we pay the lawn care company to mow and trim weekly. We do this when we buy anything, including software or the services that produce the software.
This is an immutable principle of commerce
So when we hear, there are alternative ways of writing software for money that don't involves estimating the cost of doing that work, think again.¬†How did you get around the immutable principle of commerce? Now notice I used the word estimate in the same post as¬†know. Yes, estimating allows us to¬†know something to some level of confidence.¬†
I'll estimate that my 1 hour drive to work everyday, will be extended to 1 hour 20 minutes when the snow storm arrives tomorrow night.
I¬†know I'd better have margin in my drive schedule, if I want to attend the 8:30 stand up.¬†
I estimate that it will take 3 days to install and verify the database for the system, given the historical data from the last 3 times we did this.
This¬†knowledge¬† can then be used to plan the access to the server room, arrange for all the verification and validation data we'll need to certify the contents are¬†ready for use by the customer.¬†
We¬†estimate to a degree of confidence, things (time, cost, performance) we'll need to know about to do our job.
So How Can We Learn To Estimate?
Here's where we start. We start with what has taken place in the past.¬†We've never done this before you say. I'd suggest, working literally in the¬†rocket science world, there is very little in the commercial software world that hasn't been done by someone, somewhere in the past. You may not know these people, but it's been done. And more importantly it's the people issues that muck up the project most of the time, not the technical, unless of course it actually is rocket science, or stealth fighter science, or bioscience.¬†
So with the second best basis of estimate approach -¬†What is this like?¬†We've done similar things in the past, how is this problem like those solutions? Next comes the 10 questions approach. The¬†Planning Game. Then a parametric approach. Function Points, COCOMO, SEER, Price-S, SLIM, CoStar, and a long list of other¬†basis of estimate¬†tools, some free can guide you. So when you hear¬†software can't be estimated, change the phrase to¬†I don't know how to estimate software projects, but I can sure look into learning how.
Finally the least desirable way to estimate is to ask the expert. This only works if the expert has been calibrated with a reference class, has her¬†optimism bias in check, knows all about anchoring and adjustment, and has a track record for producing credible estimates. If not, you're going to be disappointed in the result.
But our management uses our estimates against us. Our management doesn't understand the notion of probability and statistics. Our management behaves like Dilbert's boss. This has nothing to do with the need to estimate the cost, schedule, and technical performance of the product and service needed by your customer. It has everyone to do with¬†managing up. And if that's not possible, producing a credible estimate with those risks baked in to protect yourself. And if that's not possible start looking for a better manager or even a better job because your company is going to be in the ditch before long.
So when we hear that¬†estimating is the smell of dysfunction -¬†without ever listing one single dyfunction - remember there are lots of dysfunctions in business. This is normal, because humans are involved. But that dysfunction is not¬†caused by the need to estimate. The need to estimate is a core business process. Doing bad estimates, doing estimates for the wrong reasons, doing estimates wrong - that's a dysfunction that is universal.
In the end you need to either nut up or shut up as Woody says. Yes, that Woody. Learn to estimate for all the right reasons, then when there is an opportunity to have an enlightened manager at your current firm or a new firm, you'll be prepared to contribute value to the business process in ways that benefit the top line.
Since that top line, minus the costs to produce the goods or services is the bottom line (in it's simplest¬†form) is what writing software for money is all about. Knowing the middle line -¬†costs of goods sold¬†(COGS) is critical to actually staying in business.Let's Stop Guessing and Learn How to Estimate Facts and Fallacies of Estimating Software Cost and Schedule
There was a question posed on LinkedIn
How do you get Project Manager understand the importance of Earned Value Management?
It's been shown over and again when we make EV a compliance process and hold managers accountable for compliance it is a necessary condition for success, but far from sufficient.
Until EVM starts providing actionable information to the program management staff in the form of "leading indicators" it will always be one of those compliance processes. This information must reveal where in the program, technical and programmatic changes can be made, the ¬†testable outcomes of those changes on program performance, and the impacts on cost and schedule forecasts and the EAC against the BAC. Starting at the program level, but going down to the Control Account and Work Package.
In other words...
Nice data there in your Formats 1-5 and really nice IMS, what should I do next?
EV numbers themselves are lagging indicators, non-statistically adjusted, non risk adjusted, not connected to the effectiveness and performance measures of the project - unless enlightened users do so - and not connected to the needed capabilities of the customer as delivered by the program. There is no DID for the IMP, so Systems Engineering rarely flows down MOEs and MOPs - except where they do, because of the knowledge of the power of this approach.
Next is a real problem. 748-C has a loop hole big enough to fly a 787 through. In section 3.8 "Earned value is a direct measurement of the quantity of work accomplished. The quality and technical content of work performed is controlled by other processes." Measuring ¬†quantity is a construction centric view of producing value. Not a¬†value centric view. Our defense, space, biopharma, software intensive programs that apply EVM are not pouring concrete or welding pipe. (I used to work in that domain as a SW developers on piping design systems). If we see EV as measuring¬†quantity we ignore all the concerns Paul Solomon has brought forward, starting with the missing Technical Performance Measures for the product.
Sure we have¬†exit¬†criteria for the Work Packages. But these need to trace vertically through the IMP to the¬†Capability ¬† to accomplish the mission or provide a solution to the business need. In other ¬†do something with the resulting product or service. It has to be the right thing of course. But EV - as stated in 748 - only measures to quantity of¬†parts needed to delivery a capability, not the ability of the system to fulfill the needed capabilities. This by the way is essentially a Systems Engineering issue. With the missing IMP, most of the motivation for ¬†connecting the dots between EV and mission success is simply not there. It's back to the immutable principle
We don't know what done looks like in units of measure meaningful to the decision makers
"Earning value" means assessing the efficacy of the BCWS. This means assessing the Technical Performance Measures of the work be delivered. Many implementations assign TPMs and QBD's this work. But this is not called out in 748-C nor any formal EV guidance. It is called out in the DAG and the old SEMP DID. No SEMP DID is in place.¬†
So as EV practitioners we need to make the business case of EVM, not just the compliance case. When we add "leading indicators" that are statistically sound (Eric Druker has written about this as have others, including me), forecasting of EAC based on ARIMA processes rather than our linear, non-statistical, non risk adjusted CPI/SPI measures. As well those measure "roll up" the variances of all the past time series data, just like Darrell Huff tells us to do in "How To Lie With Statistics".¬†
All the data is available in the EV engine and in the CR at PARCA for DOD jobs. What's next is to start using EVM in the same way supply chain managers, quality control managers, systems engineers, and every design and manufacturing engineer does - compare statistically adjusted performance against the probabilistic plan to see leading indicators emerge of where we are going to go into the ditch in the future, show the PM, that corrective action "before the fact" rather than after.¬†
This will be a topic discussed at EVM World and ISCEA conferences coming soon.
Shim Marom provides thought provoking posts. The one on Estimates and Not Estimates was a perfect straight line requesting a response.
So some other thoughts:
If No Estimates is about making decisions, then No Estimates better come up with the cost of outcomes of having made that decision or that statement is completely bogus. Time to start looking for ¬†business value from our work efforts in building software for money. That means know the cost of our efforts to some degree of confidence before and during the project.¬†Knowing afterward is simple and worthless, that horse ran away and know we're saying how much it costs.Related articles It's an Estimating Problem not A Problem with Estimating How To Estimate, If You Really Want To Moving from "Why" to "What" Actions are Needed? Why Hasn't #NoEstimates Produced Actionable Outcomes Yet?
Project success is elusive in all business and technical domains, including software development, construction, new drug development, any project involving multiple participants, irregular funding, and emerging requirements and risks that haven't yet been identified and handled.
To increase the probability of project success we must start with principles and apply practices implemented by processes known to produce benefical outcomes. Without these principles the practices and processes have no home.
These principles provide the framework for practice and process. Let's establish them first:
Identify the Technical and Operational Capabilities
The capabilities needed for success of the project, mission, or business ¬†are defined by the customer. Through a strategy¬†development process or some other process that states why¬†these are needed in units of measure meanigful to the decision maker. These measures are stated as effectiveness and performance goals.
Construct the Sequence of Capabilities
The capabilities must be delivered in an order that maximizes business value, minimizes risk, and maximizes opportunities along the way to make improvements for the customer.
With the sequence of work in place, we can now look for risks to our success and opportunities for improvement. Risk is created by uncertainty. Uncertainty comes in two flavors:
With sequenced capabilities, risk handling plans, and opportunity plans we are now able determine the cost and schedule of the work needed to deliver the capabilities. This work starts with packages of work holding the budget for the work and describing the period of performance.¬† This result shows us the cost needed to produce each capability. This cost can be compared to the beneficial outcomes from the capability to confirm the business case or mission strategy if viable from a business point of view.
For each chunk of work in the plan to implement the needed capabilities we need some method to measure progress to our plan. These measures must be based on tangible evidence of physical percent complete. This can be done through three basic approaches.
Each of these assessments of progress to plan is based on a pre-defined unit of measure. This avoids the opinion of progress we often see on projects stuck at 80% to 90% complete. It also prevents measuring progress with the passage of time and consumption of money. Even the notion of¬†working software has to have a tangible measure of¬†working. Passing a test is fine. Is it the right test to confirm that a requirements is fulfilled to deliver a capability?¬†
Each capability of the insurance provider network ERP system above is developed in a planned sequence to provide value in support of a business strategy. This order includes minimizing risk and maximizing opportunities. Each point where capabilities join the business can put these to work in generating value.
We need to know what DONE looks like, how we‚Äôre going to arrive at DONE on-time, on-budget, with the planned capabilities, what resources, funding, and work sequences we need, what risk handling and opportunity management can be performed, and how we‚Äôre going to measure tangible physical percent complete along the way.
These Principles enable 5 Practices and 5 Process to be implemented to increase the probability of project success. The details of all these can be found in¬†Performance-Based Project Management(sm).Related articles The "Real" Root Cause of IT Project Failure Performance Based Management
It's not you money, behave apprioriately.
When asked to develop software in exchange for money - this is leaglly called¬†work for hire - we have an obligation to have some notion of the final cost. If not, we're likley working a¬†staff augmentation and not doing¬†work for hire. So let's assume we are on a WFH engagement. Knowing something about the final cost starts with estimating that cost. This estimate says its name - it's an estimate. It's not a firm price. It's not even a firm anything. It's an estimate.
This estimate can come in several forms
The answer to both of these approaches comes from making estimates. Estimates of cost and schedule. Estimates of capacity for work. The process for making these estimates is called ¬†Basis of Estimate and ¬†usually starts with an ¬†anchoring nbsp;process.¬†
By anchoring it means making the estimate from something that can be used as a reference. It's been done before, there is a model of what ¬†could ¬†be done. Some reference class. This anchoring process is then followed by an ¬†adjustment process. Adjustments can come in a short time, or over a longer time. But the anchoring and adjustment paradigm is well developed.
One research study has shown
One way to make judgments under uncertainty is to anchor on information that comes to mind and adjust until a plausible estimate is reached. This anchoring-and-adjustment heuristic is assumed to underlie many intuitive judgments, and insufficient adjustment is commonly invoked to explain judgmental biases. However, despite extensive research on anchoring effects, evidence for adjustment-based anchoring biases has only recently been provided, and the causes of insufficient adjustment remain unclear.¬†
Anchoring and adjustment is well studied in behaviour finance - why people make financiual decsisions that they do. Anchoring and adjustment is also well studied in estimating starting with Oil & Gas estimates of reserves to estimating public works projects. A complete litertaure search can be found from Google of course.
But estimates needed for making business and techncial decisions must take this research into consideration is they are to be successful. ¬†For cost and schedule estimates the best place to start is past performance. Here's an orginal drawing of Flybjerg's¬†Reference Class Forecasting process flow. Google will find you all his work as well. Many about infrastructure and some about IT.
When it is mentioned¬†we haven't done this before so we have no reference classes, we have to pause and think. Is this a trueGreen Field¬†project. Technology that has actually never beed done before is rare in the IT world. If this project is truly without precedent, it's likely an R&D project and need to be planned much differently.
So assuming this is not an R&D project, what can we do about creating estimates when we have little past performance? There are many tools available, some free, to start the estimating process. To produce an¬†anchor estimate, we can start to refine before the project starts, or even after the project starts.¬†
There are three types of activities that paerticipate in the estimating process
So when we hear¬†estimating is the smell of dysfunction and no dysfunctions are listed let alone any credible and¬†in use estimating processes, it's time to question the wisdom is assuming estimates are the problems with software projects.
So let's invert the conversation for a moment.
In the end it's the business that needs the estimates of the developers have decided they are not useful. It's not the developers money - if it is no one cares how they spend it. The business has a obligation to the shareholders, investors, the funding agency, to have some understanding of what the cost for the products or services will be when they are done.Related articles How To and How Not To Make Credible Estimates of Cost and Schedule Facts and Fallacies of Estimating Software Cost and Schedule It's an Estimating Problem not A Problem with Estimating 8 Reasons Why Estimates Are Too Low The Value of Making An Estimate
The AMACOM Book¬†Performance-Based Project Management has a blog post from the publisher.
Much of the discussion on building products and services is focused on eliciting requirements, developing solutions around these requirements. Ignoring for a moment the silliness of not knowing the cost or duration of this work effort, we ¬†still need to address the business side of spending other peoples money in exchange for delivery a known value.
Over the past months I've come to see the world through the eyes of Systems Engineering. I have a MS in Systems Management, but haven't applied it in 25 years.
My current assignment is on a large software¬†intensive system that is a national asset. This is a code word for¬†it has to work as planned or 100's of million of people, sovereigns, and other users will be disappointed at best and possibly be put in harms way at worst.
This systems view means: (Systems Thinking for the Enterprise: New and Energing Perspectives)
So it all comes down to this:
If you don't know what someting costs - in units of money, schedule, manpower, infrastructure, tools, process risk, delay - you're not going to be able to know what it is worth.
That's it, It's that simple. Anyone suggesting their processes or even a HashTag masquerading¬†as a suggested approach to problems, that this solution will increase the Probability of Project Success (POPS) ¬†or their suggested solution is focused on providing value to the customer, must address how that suggestion will work in the presence of the reality of building products and services in exchange for someone else's money.
Earned Value measures the production of tangible outcomes from work efforts on a project using¬†Physical Percent Complete. This P%C is calculated used Quantifiable Backup Data shows what technical or operation goals must be met on a planned date in order to claim some¬†Percent Complete. Measures of this physical percent are NOT how much money you spent or how much time you took to do the work.
They are only measured in unit of¬†tangible technical progress to plan. This tangible technical progress is defined¬†BEFORE the work starts. This progress goal is derived from the time phased Measures of Effectiveness and Measures of Performance goals of the project.
These MOE, MOP, KPP, and TPMs are defined below and their measures - in units meaningful to the decision makers - established before work starts, put on¬†baseline and used to compare progress to plan (time phased) to produce that Physical Percent Complete in the Earned Value (BCWP) formula in the first chart.
This branch of mathematics [probability] is the only one, I believe, in which good writers get results entirely erroneous - Charles Sanders Pierce
When it is said -¬†you can't estimate the future - or¬†we don't know total cost, think of Mr. Pierce. All things project management are probabilistic drive by the underlying statistical processes of irreducible and reducible uncertainty. Rarely, if ever, are these uncertainties¬†Unknowable, in the mathematical sense.¬†
Performance-Based Project Management(sm) on Amazon
Here's the Table of Contents and Chapter 2. This chapter is the anchor for the Principles, Practices, and Processes of increasing the probability of project success.
Performance-Based Project Management(sm) from Glen Alleman Here's a quick overview of the rest of the book with Principles, Practices, and Processes 5 immutable principles and 5 processes in 60 seconds from Glen Alleman On a bit more detail about how the parts all come together
Performance based planning in a nut shell (V5) from Glen Alleman ¬† ¬† Related articles Flaws and Fallacies in Statistical Thinking What is Project Management About?
The processes by which we have discussions about project processes must be based on probability and statisics. Every project is subject to aleatory (irreducible) and epistemic (reducible) uncertainties. When we talk about cost, schedule, and technical performance are all statistical in nature.¬†
Here's a book that should be read before engaging anyone in a conversation about the processes of project management, estimating, risk, performance management.
Software development ranges from straight forward, with small a team with a continuous improvement effort. Say a web site or warehouse management application were the customer is happy to just keep the improvements coming. Every improvement or new feature can be put to use. Along the way new ideas enter into the mix and since there is no real¬†deadline, the features can be deployed when they are ready.
On the other end of this wide spectrum of projects is the mission critical, business critical project. For example a Mergers and Acquisition strategy of two large firms that need to¬†join their ERP systems before any benefits from the merger for the customers, operations, and cash flow can be realized. Below is an example of the¬†Plan for such a project.
The first thing to put to rest is the naive and misinformed notion that planning is somehow not needed. The¬†Plan above is a strategy for the integration of legacy systems with a new ERP system. The¬†Planned date for this system is tied to business success. Long before this Plan was developed, the business strategy identified the needed capabilities of the integrated system, the planned savings from the integrated system, and a customer facing set of¬†abilities of the result.¬†
This is the fundamental Return on Investment equation. ROI = (Value - Cost)/Cost. We know the value, since the analysis of the value of the integrated system was done before the merger. The cost was estimated at the highest level from experience of integrating ERP in the past. Details come next, usually after the merger.¬†
Each of the¬†boxes provides a needed capability. End¬†box is an incremental deployment of this capability, with feedback that informs the downstream capabilities. No project can have an end-to-end detailed schedule with any hope of that schedule remaining intact. But the¬†Plan above describes the order of delivery of the needed capabilities. These are not arbitrary, these are not random, these are not subject to change. At least not without understanding the impact on the business merger process and the resulting cost savings and cash flow generation.
So What's The Point
If the¬†value at risk is sufficient to call into question business failure, or at least major issues, if the project fails to meet its goals, then we need a Plan, an estimated cost, and an estimated delivery date. To ignore these business needs is to ignore the obligation to provide that needed information.¬†
You project may not be a merger of two multi-billion dollar firms. But the question isn't¬†when to estimate but¬†when do we no need to estimate. The asnswer to that starts with the¬†Value at Risk. The business providing the funding gets to say what the value is.¬†
What is the business willing to¬†Risk on the project without knowing that value before starting?
So here's an immutable principle of all business:
For a business to stay in business it must manage its costs. Without managing cost there can be no hope of profit. Profit is why a business is in business. Sure there is that "let's change the world" mission statement. Or the "world domination" statement. But without profit those glorious goals cannot be met - unless your a sovereign. So managing cost and knowing cost is a precondition for profit. Profit is a condition for staying in business.
We can know cost from the past - these are called actuals or estimated actuals if the invoice hasn't arrived yet. Knowing costs in the future is called forecasting. Forecasting involves estimating. Estimating future cost is part of the immutable process of staying in business. If we can't forecast future cost, it's going to be hard to hang on to those revenues customers are giving us in exchange for some value we've delivered.¬†
Enough of Business 101. Now on to project based business processes.¬†Overrun our project cost for an external customer on a non-government job and our profit margin is reduced.
Overrun our baseline cost on a government cost-plus job and you're going to have to explain why we're a¬†bone head to some contracting officer.
Overrun our government contract by 25% or greater and we'll have to explain to congress why you're a¬†bone head that breached the Nunn-McCurdy limit.
Overrun our spend plan for staff, and someone is going to come and ask why we have more people than we planned. Have no plan for that number of people, the balance sheet is going to show the overrun to people you may have never met and they'll be asking us questions we may not have answers for.
Project performance starts with cost management. It may be likely that our salary is a recoverable (allowable) cost. It may be our salary is wrapped into the bill rate to the customer. But no matter the source of funds that pay your salary, the light bill, the really nice capichino maker, the pool table, Income - Cost is retained earnings. If we overrun our¬†Planned cost, we're going to erode what remains from Revenue - Cost = Profit.
Any one not agreeing must be working for a not-for-profit or a non-profit organization. Cost management translates directly into profit margin goals. Profit Margin is revenue minus direct cost plusoverhead, indirects, and all other non-recoverable costs. Profits are what keeps the lights on. Profit is Net Margin.
So Now What?
Cost is one of two numbers in the Return on Investment equation every managerial accounting class taught us.¬†
ROI = (Value - Cost) / Cost
Unplanned growth in cost reduces ROI. When we hear¬†It's all about delivering customer value, nothing but value, starts and ends with value, ask at what cost? Don't know the cost, you can't determine the NET Value. Simple managerial accounting again. Not developer accounting in story points, not drip accounting in budget fed to your project.¬†Dead Presidents accounting from the bank customers pay into to your payroll account your pay your mortgage from.
Any business not knowing what things cost for the work they are performing, be it internal work or external work, is not going to be in business for long.¬†
Cost is KING
Once we come to understand that, we can now talk to business people and talk to engineers.¬†Because most engineers work on¬†budget. And¬†budget is not the same as¬†funding. Funding can buy bread at the store, It's real money. Budget is a cell in Excel or a number in SAP, or a number written on the White Board.
So Now What, Really?
If we're on the customer side, or we're in the accounting managers office, or the product managers office, or the account executives office, we'll be having a conversation about the¬†cost of the wonderful work we're doing for our internal or external clients or our customers we're selling our product to. This is how business is run.¬†
There is a mythical notion that places like Google, Facebook, Twitter don't care about cost. I'd suggest you go look at the 10-K and search for the term cost. You'll see things like¬†cost of revenue. How much do we have to spend to bring in every dollar in revenue. Business runs on cost. It also runs on products, service, cool things no one can produce except for them, stuff we never ever thought about but they did. But when paychecks go out every two weeks to Chase,¬†dead presidents are removed from one account and transferred to another - you're bank account. The cost of building all those cool, unimagined things requires real money. Spend too much - more than planned - profits go down.¬†
How Can The Business Know What Costs It's Obligated to Pay Next Week, Month, Quarter, Year?
How you ask? They make estimates. Now these estimates may never reach us in the development department. We may only see budget, funding allocation, release of funds, or just a direction to go forth and build cool stuff. But someone, somewhere in the building needs to know the cost of our wonderful work.¬†
This cost can be flat spread - peanut butter spread as we say where I work. A fixed number of Full Time Equivalents with a base cost and burden (fringe, overhead, and indirects). That's the budget for all the people working here. What happens then is we need to know our capacity for work. What can we produce for that cost? Unless we're building transmissions at Toyota, we'll have a variable cost of production. Somethings we work will cost more per the value produced, some will cost less. Software for example is NOT a production line. ¬†So if we've fixed the cost of production, the¬†value produced will be variable. How variable? Oh I don't know, let's estimate.¬†
Estimate from past performance. That's the absolute best way. What of there is no past performance, or no applicable past performance? Then estimate anyway. You're seeing a trend here right?
We have to estimate our cost of production in order to know, with any confidence, our future earnings. Where earnings are Revenue minus Cost.
No cost estimating process at the business level is simply nonsense. Now we as a developers may not be involved in cost estimating. But someone is. Let's go find that someone and ask.¬†Do you really not want to know my cost of being here?¬†Related articles Financial Accounting Versus Project Acounting It's an Estimating Problem not A Problem with Estimating How To Estimate Almost Anything If You Really Want To What's Gonna Cost and When Will We Be Done? Book for Cost and Schedule Forecasting
Software intensive project work is indeterminate. Many times, customers don't know what they really want until they see something working. Development's capacity for work is not well understood early in the project. Technology emerges and impacts the effectiveness of the project. Reducible and irreducible uncertainties abound. These conditions are¬†normal, they can't be wished away.¬†
So what are the options? Let's start with some advice and see what we can do to improve the¬†Probability of Project Success. Wise leaders from the past have words we can use to guide our efforts.
I know one thing: that I know nothing¬†
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt
‚ÄĒ Bertrand Russell
The fool doth think he is wise, but the wise man knows himself to be a fool.
‚ÄĒ William Shakespeare, As You Like It
Knowing nothing about the project is not good for increasing the probability of success. Knowing what¬†capabilities are needed to fulfill a mission or business need is a starting point. Capabilities are not requirements, those come next. Capabilities allow the user to¬†do something of value. This is the¬†value spoken about, most times spoken about in the absence of any units of measure. These measures, the measure of capability is the¬†Measure of Effectiveness. (The tool in the previous link is a systems engineering tool, that provides all project participants with clear, concise, and testable assessments of their understanding of what¬†Done looks like. The notion that tools get in the way of success is ill-informed at best, and just plain wrong at worst).
Measures of Effectiveness are operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in an operational environment, under a specific set of conditions.
If we don't know what capabilities we want the project to deliver, it's going to be a low probability that the project is a success. We don't know what done looks like. Now we can get paid to discover these capabilities.¬†Capabilities Based Planning is a process to discover these. Someone has to pay for this of course. It is usually the customer that pays. But someone has to pay.¬†
With the needed capabilities in hand, we can make our first estimate for cost, schedule, and the probability of success. Here's an example. Frank Cepollina¬†was speaking at the BAA (Broad Area Announcement) for the Hubble Robotic Service mission.
He stated his needed capabilities rather than in techncial requirements. Those always come after we know what¬†done looks like. Notice these capabilities are mission capabilities.¬†Get the job done descriptions.
How much does that cost and when can you have it ready to fly?
The three vendors needed to develop an estimate. The details of this process can be found in¬†Assessment of the options for extending the life of the Hubble Space Telescope. This example can be extended to IT and software development projects. What do we want the system to do? Don't know? Go find out. Make that the first step in the project. Read¬†Predictabilty: A Simple Approach to Creating Reliable Project Schedules, Steve Bockman for an example of how to think about this in a software development environment. No matter what our domain, size of project, we're going to need to understand something about what capabilities are needed for project success. We might be able just to start work and let these¬†emerge. But our probability of success is¬†unknown and that's not likely a good thing in the end.
So when we know something about the needed capabilities, we can start to discover the requirements. But first we're going to need some kind of measure for those requirements. These are performance measures.¬†
These are measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions. These are Measures of Performance, and are:
These are measure of how the requirements are going to perform in pursuit of the mission.
Final we have to discover the Technical Performance Measures. These are attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal. These:
Here another need for estimating. How fast do we have to process transactions to meet the needed capability? How much can we weigh and still have the capability to fly? How reliable (mean time between failure, mean time to first failure, mean time to recover from failure), and still meet the needed capabilities?
These estimates are focused¬†on technical outcomes. Thing that fly away, things that provide services in support of the customer.¬†Things.¬†
Where are we going here?
We must learn to estimate all kinds of things on a successful project. The credibility of our skills as project managers is always tied up with our ability to estimate. Can't estimate, choose not to estimate, see estimating as some kind of dysfunction, it is doubtful those providing us with money are going to have much confidence in our ability to deliver that mystical¬†value we are told is part of all projects.
Estimating is part of project work. To not estimate, means we won't know what done looks like until we arrive at the end. That's unlikley to be acceptable to those paying us to do work.¬†
The next step is to determine what requirements will be needed to implement the needed capabilities. These are defined as Measures of Performance that characterize physical or functional attributes relating to the system of operation, measured or estimated under specific conditions.
Notice something important here. Software project success always depends on development of working ¬†software. But what software, in what order, with what Technical Performance Measures, Measures of Performance, and Measures of Effectiveness all supporting the needed Capabilities.
Success doesn't start with coding. It's about knowing the answers to the CBP, MOE, MOP, TPM attributes. The notion of¬†emergent¬†anything works only when we can state in some unit of measure what¬†DONE looks like. Without that we're just wasting the customer money¬†exploring, challenging everything, and¬†looking for the mystical Unicorn hiding in the bushes.
So Here's the Punch LIne
When we spend other people's money on our project, private money, public money, government money, they have the expectation of knowing something about how much much and when we will be finsihed with the work in exchange for that money. This is pretty much the case for every project I've ever worked for the past 34 years. It's the basis of every successful business - knowing what your costs are for the operations. So when we hear about developing software ¬†for money and not having any concern about the cost of that effort it simply doesn't make any sense.Related articles Elements of Project Success The "Real" Root Cause of IT Project Failure Risk Management for Dummies Requirements Elicitation Agile Project Management How Not To Develop What "Done" Look Like Performance Based Management Five Core Processes of Project Success
The common picture of requirements elicitation looks like this. Which of course is an example of¬†doing stupid things on purpose. When this picture is used as an example of not doing something because it doesn't turn out right, is a further example of doing stupid things.
Let's see where the gaps appear that results in the outcome in the last panel:
The wheels fall off when there is no description of DONE shared between the customer and the provider. If the customer doesn't know what DONE looks like, who does? The developers? Probably not. Is DONE emerging? Then the customer has to pay for that?¬†
There is no way out of the need to know what DONE looks like in Measures of Effectiveness, Measures of Performance, Key Performance Parameters, and Technical Performance Measures. Without these the success of the project is in doubt from the start.
The notion that requirements emerge is will established. But the capabilities the customer needs to be stable enough to establish an Estimate At Complete and an Estimated Duration to Complete. Without these the customer has no understanding of the¬†all in cost of the project. These change as the requirements change. That's part of the project management process. This can't be ignored if the customer is to have any confidence in the project providing value at the needed time at needed to be put to use.¬†
So a reminder one more time...
You can't assess the value of the project outcome without knowing something about the cost to deliver that value and the time frame for that delivery.
No matter what anyone says, this is an immutable business principle. Anything is simply fooling yourself into believing that the customer doesn't care how you spend her money or when you spend it.
Focus on value, that's what the customer bought. Actually they bought a¬†capability to do something to improve their business or mission. But ignoring the cost of delivering that value is ignoring the balance sheet of the business.
The post 8 Reasons Estmates are too Low, is one of those pieces of material that on the surface seems plausible but has series flaws. First is the restating bad management practices and then arguing against them. This seems all too common in the Agile domain for some reason.
A poster campaign at Rocky Flats in the late 90's for safety and safeguards, but usable everywhere is...
Don't Do Stupid Things on Purpose
If we ignore the red herring approach of doing stupid things on purpose and then tilting at the resulting windmill, let's look further for each idea in the post. The picture to the right is used when engaging in a conversation about making improvement, but starting with a credible baseline. This is called Dead Horse on a Stick. Thanks to the Master Systems Engineer on our program for this concept. He uses this when he starts a review and the ideas are dead¬†before he got there. It's also appropriate for concepts that are¬†dead on arrival,¬†like suggesting that those paying for products don't have a need to know how much it's going to cost, before they start spending money.
So what's the point?
When we look at making improvements to anything from projects and sepnding other people's money, to better peddling of my road bike on century rides - start with
stop doing stupid things on purpose.
Only then can you make real and lasting improvements. If you don't do that, you are beating a dead horse.
Such as are your habitual thoughts, such also will be the character of your mind; for the soul is dyed by the thoughts. - Marcus Aurelius
Projects are composed of three fundamental elements. Cost, Schedule, and ¬†Technical outcomes. The Technical Outcomes go far beyond the PMI-style¬†scope terms. In this paradigm, the technical outcomes are at the end of a chain. Here's examples of that chain - Capabilities, Measures of Effectiveness, Measures of Performance, Key Performance Parameters¬†(there are 5 in our domain), and Technical Performance Measures. At the TPM level is where things like quality live, traceable to the KPPs.
These three elements are coupled in dynamic ways. Their connections are¬†springy, in that changes in one has an impact on the other two, But rarely is this impact linear and ridgid. The¬†Iron Triangle notion is really a¬†Three Body¬†problem, in which all three element impact each other and at the same time respond to that impact.
All projects have these three elements, coupled in this way. Changes in one impact the other two. Changes in two impact each other and the third. Without knowing the dynamics of cost, schedule, and technical performance, we can't have any credible understanding of these variables.¬†
Three Body Problem
The three body problem determines the possible motions of three point masses m1, m2, and m3, which attract each other according to Newton's law of inverse squares. It started with the perturbative studies of Newton himself on the inequalities of the lunar motion. In the 1740s there was a search for solutions (or at least approximate solutions) of a system of ordinary differential equations by the works of Euler, Clairaut and d'Alembert (with an explanation by Clairaut of the motion of the lunar apogee).
Developed further by Lagrange, Laplace, and their followers, the mathematical theory entered a new era at the end of the 19th century with the works of Poincar√© and since the 1950s with the development of computers. While the two-body problem is integrable and its solutions completely understood, solutions of the three-body problem¬†(Java 7 in 64 bit Browser needed) may be of an arbitrary complexity and are very far from being completely understood.
The forces between the bodies can be¬†self attractive or they can be a central force - the restrictive three body problem. Or a combination of the two. This is the basis of complex systems, where multiple forces are applied to ¬†objects, which in turn change the forces. As an aside, the double pendulum and the three body problem are used as examples of complex systems. Without acknowledging that the underlying mathematics is¬†deterministic since the Java example above draws the lines from an algorithm.
This is a common mistake by those unable to¬†do the math,¬†or who want to suggest the problems of the day are beyond solution.
Three Body Problem and Three Elements of Project Management
The three body problem uses gravity as the force between the masses. There is a simpler example of three masses connected with three springs. This model is found in chemistry and biology, at the molecular level. Gravity is not in effect of course, but electromagnetic force.
Consider a simplified model for the vibrations of an ozone molecule consisting of three equal oxygen atoms. The atoms are represented by three equal point masses in ¬†equilibrium positions at the vertices of an equilateral triangle. They are connected by equal springs of constant k that lie along the arcs of the circle circumscribing the triangle. Mass points and springs are constrained to move on the circle, so that, e.g., the potential energy of a spring is determined by the arc length covered.
Now to Projects
If we assume for the moment that cost, schedule, and technical performance are dynamic variables, with¬†forces between them described by their functional equations. In our functional equations for the force between them ar not constant, but are relationships like this:
The interaction between the three core elements (cost, schedule, technical) is a two way interaction, so the spring analogy is not quite correct, since the spring force doesn't know which end it is pushing or pulling.
It's not the¬†Iron Triangle, it's a springy triangle. The connections are non-linear and most importantly they are probabilistically driven by the underlying statistical processes of the project. Let's start with the picture below.
All project processes are¬†probabilistic. They have behaviours that are not fixed. The notion that you can¬†slice work into same sized¬†chunks and execute these chunks with the same effort would violate the basic¬†aleatory uncertainties of all work processes. With an understanding of the statistical processes, driven by either aleatory or¬†epsitemic uncertainties, is followed by asking probabilistic questions.¬†What's the probability that we'll complete on or before a date or¬†what's the probability we'll compete at or below a cost.
With a probability and statistics foundation, we can now put together a credible plan, driven by the underlying stochastic. All work is connected in dependent ways. The work effort, it's duration, and outcomes is also statistically driven. This picture is typical of such a project.
In The End
So we can: