Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
There is a nice post from Trent Hone on No Estimates. This triggered some more ideas about why we estimates, what the root cause of the problem #NoEstimates is trying to solve and a summary of the problem
A Few Comments
All project work is probabilistic, driven by the underlying statistical uncertainties. These uncertainties are of two types - reducible and irreducible. Reducible uncertainty is driven by the lack of information. This information can be increased with direct work. We can "buy down" the uncertainty, with testing, alternative designs, redundancy. Reducible uncertainty is "event based." Your power outage for example. DDay being pushed one day by weather.
Irreducible uncertainty is just "part of the environment." It's the natural varaibility embedded in all project work. The "vibrations" of all the variables. This is handled by Margin. Schedule margin, cost margin, technical margin.
Here's an approach to "managing in the presence of uncertainty"
For my experience in Software Intensive Systems in a variety of domains (ERP, Realtime embedded systems, defense, space, nuclear power, pulp and paper, New Drug Development, heavy manufacturing, and more) #NE is a reaction to Bad Management. This inverts the Cause and Effect model of Root Cause Analysis. The conjecture that "estimates are the smell of dysfunction" without stating the dysfunction, the corrective action for that dysfunction, applying that corrective action, then reassessing the conjecture is a hollow statement. So the entire notion of #NE is a house built on sand.
Lastly the Microeconomics of decision making in SWDev in the presence of uncertainty means estimating is needed to "decide" between alternatives - opportunity costs. This paradigm is the basis of any non-trivial business governance process
No Estimates is a solution looking for a problem to solve.
On any project with significant¬†Value at Risk¬†Economic Analysis provides visibility to the data needed for decision making. This¬†Value at Risk paradigm is a critical starting point for applying all processes of decision making. The choice of decision process must be matched to the¬†opportunity¬†cost (actually the value of the loss for the alternative not chosen).
With these probabilistic outcomes driven by the underlying statistical process of all project work, we need to be able to estimate all the values of the random variables and their impact on the processes above. ¬†¬†
Next is an example of applying this probabilistic decision making in the presence of uncertainty for cost and schedule assessment. This can be for other probabilistic variables on the project. Technical Performance Measures, Measures of Effectiveness, Measures of Performance, Key Performance Parameters, and many other¬†...ilities (maintainability, supportability, survivability, etc.)What Happened to our Basic Math Skills? Information Technology Estimating Quality Making Decisions In The Presence of Uncertainty What's the Smell of Dysfunction?
Do you have a title such as product manager, product owner, or business analyst?
We hear ¬†these titles all the time. What does each do?
Here is how I have seen successful agile projects and programs use people in these positions. Note that I am discussing agile projects and programs.
The product manager creates the roadmap. She has the product vision over the entire life of the product. Typically, what’s In the roadmap are larger than epics—they are themes or feature sets.¬†
Product management means thinking strategically about the product. You might require several projects or programs to achieve what the product manager wants as a strategic vision.
Product owners¬†(PO)¬†work with agile teams to translate the strategic vision into Minimum Viable Products. The PO decides when the team(s) have done enough to release. See the release frequency image to understand the cost and value of releasing.
The business analyst¬†may do any of these things. In my experience, I have seen business analysts focus on “what does this requirement/feature/story really mean to the team and/or the product?” I have seen fewer BAs do the strategic visioning of the product over its lifetime. I have seen BAs work with POs when the PO was not available enough for the team. I have seen BAs do great work breaking stories into smaller components of value, not architectural components.
Your team might have different names for these positions. Each team needs the strategic lifetime-of-the-product view; the tactical view for the next iteration or so and the knowledge of how to re-rank the backlog; and the ability to translate stories into small valuable chunks.¬†
Can one person do each of these things? It depends on the person. I have found it difficult to move quickly from the tactical to strategic and back again (or vice versa). Maybe you know how. For me, that is a form of multitasking.¬†
The more important questions are: do you have the roles you need, at the time you need them on your team? If you are one of these people, do you know how to perform these roles? If you are outside the organization in some way, do you know what you need to do, to perform these roles?
If you don’t know what to do to help your team, consider participating in Product Owner for Agencies training. Marcus Blankenship and I will help you learn what to do, and coach you in real time as to how to do it best for your team. I hope to see you there.
It’s July—find something to celebrate with your team. Unless your “down under,” you’re probably enjoying nicer weather than you have the rest of the year. We’re half way through the year, which means it’s a good time to look back on how far your team has come.
If you have retrospective notes, look back at the notes from six months ago. It’s sometimes amusing for a team to look back and see how insignificant an issue seems now that seemed huge back then. Six months of steady progress can do that.
One team I worked with was feeling really down about how few automated tests they were writing. They’d only written a few hundred in the recently completed sprint.
Considering that many were small unit tests, they felt like that was insufficient. I pulled out notes from six months earlier in which they’d set a goal of writing 10 automated tests in the sprint. That helped put their progress in perspective and gave them something to celebrate.
Equally as important to what we celebrate is how we celebrate. In the summer, simply having a few meetings outside can feel like a celebration.
I’m not a fan of giving team members cash bonuses for achieving milestones, but one time I gave everyone on a team a week off if they met a significant quarterly goal. For many people, the gift of time is about as good as it gets.
One of my favorite team celebrations was when a team decided to host a friendly competition in which team members each brought in a homemade salsa. Everyone voted and a winner was crowned. Recipes where shared on the project wiki.
My daughter, Delaney, introduced me to a celebration technique from her high school drama club. At the end of a play the actors award one another “paper plate awards,” which are simply hand-drawn awards on paper plates. I shared this with a Scrum team I worked with and they had fun with it. Of course some of the awards were a bit back-handed (“Most Spectacular Bug” and “Most Frequently Late to the Standup”), but that’s to be expected and was part of the fun.
So, what accomplishments will your team be celebrating this summer? And how will you be celebrating?
Project success starts with a simple principle. We have to¬†Know What Done¬†looks like before we start.
In¬†¬†Michelangelo's painting to the left, the two fingers are not touching. In the paradigm of a deity this may be sufficient to¬†complete the job. In our our mortal world this is a nice example of¬†almost done.¬†
It's more than common that we are stuck at 90% complete for a long time after the planned completion date. There are several independent variables here that are the sources of this problem.¬†
So What's a Project Manager to Do?
Here are six steps to creating a credible picture of what done looks like and execution to that understanding.
These six steps can be applied to any project management or product development approach from agile to formal DOD acquisition.¬†
The key here is to connect the Programmatic performance (cost and schedule) with the Technical performance of the project, measure the variances of actuals to plan and take corrective actions to get back on plan or better yet stay on plan.¬†
It is common to confuse strategy with operational effectiveness. Strategy for Information Technology (IT) projects¬†contains three major themes. These ¬†form the foundation of the IT Strategy as well as the tactical processes that will be deployed in support of these strategies.
What Is Strategy?
Strategy is creating fit among a company‚Äôs activities. The success of a strategy depends on doing many things well ‚Äď not just a few. The things that are done well must operate within a close nit system. If there is no fit among the activities, there is no distinctive strategy and little to sustain the strategic deployment process. Management then reverts to the simpler task of overseeing independent functions. When this occurs operational effectiveness determines the relative performance of the organization.¬†
Improving operational effectiveness is a necessary part of management, but it is not strategy. In confusing the two, managers will be unintentionally backed into a way of thinking about the business environment that drives the business processes (IT) away from the strategic support and toward the tactical improvement of operational effectiveness.
Managers must be able to clearly distinguish operational effectiveness from strategy. Both are essential, but the two agendas are different. The operational effectiveness agenda involves continual improvement business processes that have no trade‚Äďoffs associated with them. The operational effectiveness agenda is the proper place for constant change, flexibility, and relentless efforts to achieve best practices. In contrast, the strategic agenda is the place for making clear tradeoffs and tightening the fit between the participating business components. Strategy involves the continual search for ways to reinforce and extend the company‚Äôs position in the market place.
The concept of fit among functional units is one of the oldest ideas in strategy. Gradually however, it has been supplanted with new concepts of core competencies, critical resources and key success factors. In fact fit is far more critical to the success of the IT systems than realized.¬† Strategic fit among the various systems components and the business processes they support is fundamental not only to competitive advantage but also to the sustainability of that advantage.
Fit among a company‚Äôs activities creates pressures and incentives to improve operational effectiveness. Fit means that poor performance in one activity will degrade performance in others, so that weaknesses are exposed drawing management‚Äôs attention. Conversely, with increasing fit, improvements of one activity will pay dividends in other areas.
The challenge now is to create fit among the IT components and their matching business components.
Building A Strategy
To define our Vision, Strategic Objectives, Performance Goals, Critical Success Factors in achieving those, and the measures of effectiveness and performance in pursuit of those strategic goals and objectives, we need a method that collects all of these in a single place.¬†
If we are going to make tradeoffs in pursuit of strategy, we need to know what those tradeoffs are, how much will be the opportunity cost for each trade and how each trade impacts our strategic decision making.
To dive into the details, to make those opportunity cost tradeoffs about future outcomes in the presence of uncertainty we must of course¬†ESTIMATE.¬†There can be no execution of the strategy without make estimates of the benefits of the outcomes of the project that delivers the capabilities that implement the strategy.
The Balanced Scorecard presentation below shows how to build the strategy. Page 49 - 52 shows how to¬†connect the dots between strategy and project execution, where the work is done, at or below the planned cost, on or before the needed time, and with the planned effectiveness and performance of the delivered capabilities. Showing up late, over budget, and with missing capabilities will not enable the strategy to fulfill it's mission and vision. It's a closed loop system - all parts must work in combination for success.¬†
¬†‚ÄúWhat is Strategy,‚ÄĚ M. E. Porter, Harvard Business Review, Volume 74, Number 6, pp. 61‚Äď78.
Jack Welch Speaks: Wisdom from the World‚Äôs Greatest Business Leader, J. Welch and J. C. Lowe, John Wiley & Sons, 1998.
Control Your Destiny or Someone Else Will: Lessons in Mastering Change‚ÄďFrom the Principles Jack Welch Used to Revolutionize GE, N. M. Tichy and S. Sherman, Harpers Business, 1994.Related articles Information Technology Estimating Quality Capabilities Based Planning What Happened to our Basic Math Skills? Systems Thinking, System Engineering, and Systems Management Making Decisions In The Presence of Uncertainty Everything I Learned About PM Came From a Elementary School Teacher
The #NoEstimates advocates have asked us to¬†see estimates as a smell: an indication of possible decision making dysfunction. It might be useful to explore what's causing the smell.
In the normal business process world, when we encounter a dysfunction, Root Cause Analysis is an approach to discover the cause and effects of the dysfunction.
Since the Primary Effect is described as¬†dysfunction¬†but not stating what this dysfunction is, let's apply RCA in the form of the Apollo Method to the statement of the #NoEstimates advocates.¬†
But first some background on RCA and the Apollo Method.
What are some dysfunctions of software development management caused by the need to estimate?:
It's been suggested that¬†asking 5 Whys¬†is a place to start. It is well understood that simply¬†asking¬†may be necessary but far from sufficient to discover the Root Cause of any dysfunction. Source of the problem with the 5 Whys approach starts with our natural story telling approach to problem solving.
Finding the source of any dysfunction is straightforward:
None ¬†is this is in palce for the #NoEstimates conjecture that¬†estimates are the smell of dysfunction.¬†
So unless we have some understanding of the Dysfunction, conjecturing that estimates are the¬†smell and the Not Estimating will remove the dysfunction has little chance of actual success.
No Estimates is a solution looking for a problem to solve. And to date that problem has not been identified, and most importantly the conjecture that Not Estimating fixes the problem has no tangible evidence to confirm it will fix the problem.Related articles The Dysfunctional Approach to Using "5 Whys" Who's Budget is it Anyway? Mr. Franklin's Advice
With the principles of Capabilities Based Planning in place from the previous post, here's how to implement it.
The key here is to have a¬†capabilities delivery map in place showing what capabilities need to be delivered in what sequence for what cost to enable the business to receive the planned value in exchange for the cost to produce those capabilities.¬†
Here's an actual example for capabilities delivery. Each capabilities arrives with its dependent capabilities to provide the needed value to the business. This value enables the bsuienss to¬†do something of value in support of the business strategy and the planned revenue that results from the cost to produce that value
If we're managing the project by producing¬†Stories in aAgile, or fulfilling¬†Requirements¬†in traditional process, we don't actually know what DONE looks like other that are spend of money and passage of time. This is a critical issue in the enterprise software development domain. Fulfilled requirements, tested and running stories may or may not be the needed capabilities to meet the business strategy. Without a¬†Capabilities¬†Based Planning map like the one above there is no way to tell. Related articles Capabilities Based Planning Systems Thinking, System Engineering, and Systems Management What's the Smell of Dysfunction?
When I use a word¬†Humpty Dumpty said in a rather scornful tone, it means just what I choose to to mean - neither more nor less.
The question is, said Alice, whether you can make words mean so many different things.
The question is said Humpty Dumpty which is to be master.
Through the¬†Looking¬†Glass, Chapter 6
The mantra of #NoEstimates is that No Estimates is not about Not Estimating. Along with that oxymoron comes
Forecasting is Not Estimating
This of course redefines the standard definition of both terms.¬†Estimating is a rough calculation or judgment of a value, number, quantity, or extent of some outcome.¬†
An estimate is Approximation, prediction, or projection of a quantity based on experience and/or information available at the time, with the recognition that other pertinent facts are unclear or unknown.
Forecasting is a prediction of a future event
Both Estimating and Forecasting result in a probabilistic output in the presence of uncertainty
Slicing is Not Estimating??
Slicing work into smaller pieces so that "standard" size can be used to project the work effort and completion time. This is a standard basis of estimate in many domains.¬†So slicing is Not Estimating in the #NoEstimates paradigm.¬†In fact slicing is Estimating, another inversion of the term
No means Yes
using Past Performance to estimate future performance is core to all estimating processes. Time series used to estimate possible future outcomes is easily done with AIRMA, 4 lines of R, and some raw data as shown in¬†The Flaw of Averages. But as described there, care is needed to confirm the future is like the past.
When We Redefine Words to Suite Our Needs We're Humpty Dumpty
Lewis Carol's¬†Alice in Wonderland is political allegory of 19th century England. When #NoEstimates redefines established mathematical terms like¬†Forecasting and¬†Estimating and ignores the underlying mathematics for time series forecasting, ARIMA for example, they are willfully ignoring established practices and replacing them with their own untested conjectures.
Key here¬†ways to make decisions with NO ESTIMATES. OK, show how that is not actually an estimating technical, no matter how simple or flawed and estimating technical.Related articles Mr. Franklin's Advice There is No Such Thing as Free The Fallacy of the Planning Fallacy Do The Math Monte Carlo Simulation of Project Performance Essential Reading List for Managing Other People's Money
Over the years the success rate of traditional project management methods applied to software development projects has been underwhelming. ¬†Traditional project management methods are based on a retrospective approach, which measures variance against plan rather than providing a performance‚Äďforecast that can be used to guide projects in a chaotic environment. ¬†There are a number of programmatic control issues associated with IT projects that suggest a better approach is needed. 
Using this linear project planning paradigm ‚Äď sometimes referred to as waterfall ‚Äď but often derived from PMBOK linear style planning processes ‚Äď¬†there is little attention given to the forces that negatively impact the project. These project risks have no means of evaluation other than to acknowledge their presence, define mitigations and track the results. The impact on the business value of the capabilities of the system is not part of the project management process.
Capabilities Based Planning is anchored on producing Enterprise and Software Intensive Systems focused on strategic outcomes. Progress ¬†is measured through assessment of the effectiveness and performance of the deliverables in meeting those strategic objectives. This approach assures business value is connected with the strategy not just measures of the passage or time and consumption of money and the production of technical¬†features.¬†
In this approach avoiding or controlling change becomes the primary activity of project management. In this traditional model change is undesirable. In reality of business systems development, change is not only natural it is desirable. It is through change that the system can adapt to the needs of the business, which are themselves driven by external forces. These forces are rarely under the control of the project manager let alone the senior management of the business.
One project failure mode is when the participants and leaders of the project fail to recognize the difference between managing in the presence of change and managing change. It is managing in the presence of change that is a critical success factor of any modern business systems development.
Definition of Capability-Based Planning
‚Äú‚Ä¶ involves a functional analysis of operational requirements. Capabilities are identified based on the tasks required‚Ä¶ Once the required capability inventory is defined, the most cost effective and efficient options to satisfy the requirements are sought.‚ÄĚ
What Are Capabilities and Why Are They Better at Describing Maturity?
Measuring project and product maturity as a function of effort and time assures that project management adds value to the business. Simply controlling and measuring the expenditure of resources ‚Äď score keeping ‚Äď provides little value in the presence of change. We need measures in units meaningful to the decision makers. Physical Percent complete needs to be measured as increasing Effectiveness and Performance, with decreasing Risk to increase the Probability of Project Success.
Capabilities‚Äďbased planning provides a defined outcome that is not a final conclusion but lays the groundwork for the continued delivery of value. Objectives are reached and the operational value delivered when a defined capability is available for use.¬† Features and functions describe the static and dynamic behaviors of a system, but they are not directly connected to the business strategy. Milestones indicate that a position in a timeline has been reached, but do not forecast what value will be delivered to the business or how this value is traceable to the needs of the user community. Capabilities provide the answer to the following question: in order to achieve our objectives, what capabilities must we possess? 
Capabilities‚Äďbased planning transforms the delivery of features and functions into the delivery of processes that support a business strategy. Capabilities‚Äďbased planning is planning, under the conditions of uncertainty, to provide capabilities suitable for a wide range of business challenges and circumstances, while working within an economic framework. This approach emphasizes flexibility, adaptiveness, and robust capabilities, implying a modular building‚Äďblock approach to the delivery of enterprise applications.¬†
Capabilities are not the same as features and functions; they enable demands to be met without explicit specification of the solution. A capability is the ability to affect an outcome, react to an input, or change an effect.
A capability provides an outcome or an effect without an a priori specification. Features and functions require an a priori specification in order to test for their existence or conformance to the specification. Capabilities‚Äďbased planning can be understood at the execution level, but it needs to be raised to the level of enterprise process analysis:
Identify a needed capability in operational terms, using the set of capability options to assess the effectiveness in an operations paradigm, and make choices about requirements and the ways to achieve the capability using an integrated portfolio framework to produce an output set of options based on these operational paradigms.¬†
Putting capabilities‚Äďbased planning to work requires a change in our approach to planning ‚ÄĒ a set of business process improvement activities focused on assessing increasing maturity of the capabilities needed to fulfill the strategic objectives. Emphasis is placed on operational capabilities rather than features and functions. These operational capabilities become the building blocks of change. The emphasis is also placed on evaluating capabilities under conditions of uncertainty, which requires the deployment of robust building blocks capable of adapting to these changes. In both cases, analysis illuminates the feasibility of alternatives.¬†
Augmenting Our Strategy‚ÄďMaking with Capabilities
Strategy‚Äďmaking is the starting point for project management. It asks and answers the question why are we doing this?¬† Strategy making activities can be augmented through a capabilities‚Äďbased planning process by mapping strategies to the assessment of maturity evaluation points for each of the emerging capabilities. This approach connects the why of a project with the how. The result is the replacement of the measurement of progress as the passage of time with the measurement of progress as the delivery of capabilities.Capabilities‚Äďbased planning focuses on assessing the increasing maturity of functionality defined by the strategy. Planning under uncertainty provides capabilities suitable for a wide range of challenges and circumstances while working within an economic framework that necessitates choice, where the focus is on ‚Äúpossible uses‚ÄĚ rather than specified features and functions.¬† Connecting Capabilities to Strategy can be done in the following manner. ¬†
With a set of capabilities in mind, a plan for delivering the capabilities is needed. One approach to building this plan is an Event‚ÄďBased integrated master schedule. This has been discussed in the past, but the next article will describe the details on how to build such a schedule; derived from the capabilities. Wharton Strategic Management, ‚ÄúThree Reasons Why Good Strategies Fail: Execution, Execution, ‚Ä¶‚ÄĚ 10 August 2005.
 ‚ÄúUncertainty and Project Management: Beyond the Critical Path Mentality,‚ÄĚ Arnoud de Meyer, INSEAD Working Paper, 2001.
¬†‚ÄúAnalytical Architecture for Capabilities‚ÄďBased Planning, Mission‚ÄďSystem Analysis, and Transformation,‚ÄĚ Paul K. Davis, RAND National Defense Research Institute, MR‚Äď1513‚ÄďOSD, 2002.
As I work with more clients on their programs, I see that what might work for a product owner for a team does not work for a program. In a program, if the product owner is shortsighted, does not take advantage of agile/lean for updates, and does not have small features, the program loses momentum. Here are my three tips:
When the product owner (or, in this case, a program product owner) creates a several quarter rolling wave roadmap the teams can see the product direction. This helps the teams see avoid bad product decisions that might inadvertently prevent the product from taking a direction the PO would like.
There is a challenge with this. Teams can rarely, if ever, commit to an entire quarter’s worth of work. That’s okay. The roadmap¬†is a wishlist. The roadmap informs and guides the product development. Teams need detailed backlogs.
I know of just a couple of teams who can plan for up to three iterations and not replan. Some teams can plan for two iterations, some for one. Even if you can complete features according to a one-quarter roadmap, the product owner needs the ability to change the ranking of the remaining features.
As the teams finish features, it’s the product owner’s job to accept the features, or not. And, it’s the product owner’s job to update the one-quarter roadmap—and maybe the multiple quarter roadmap.
I’m working with some programs who have only 5 or 6 teams. I’m working with some who have 20, 30, 40 teams. That’s a ton of people. How do those people stay in sync? By integrating and creating tests that test the product, end to end. If the stories are large, the teams have a ton of work in progress and lose the ability to collaborate easily. This can be a huge challenge for the product owners and teams.
For one-team projects, I like one and two-day stories. That way the product owner can see progress every day. (I’m happy if it’s even more often than that! Two days is my max story size.)
I know product owners who can barely make time for one meeting an iteration with their teams. That does not provide the product owner feedback about the story size, or what is in the stories. Stories tend to become more complex. Then, the team doesn’t finish the stories and everyone wants to know why.
When you have smaller stories, you see the value in what the team(s) do, all the time. The teams build momentum.
Making stories small requires you think about how little you can do, not how much. That’s a huge change, especially in a program.
Those are my three tips for today:
If you liked this post, you might want to join Marcus Blankenship and me for our Product Owner for Agencies training. These things are tricky enough when you are inside the organization. They are more difficult when you are an agency/consultant working for the client.
There's a popular meme going around that asking for estimates and making estimates is¬†the smell of dysfunction. We can assume it's management dysfunction. So what are the dysfunctions of management that they ask for estimates from those spending their money to produce value in exchange?
Turns out there a few obvious ones, when we consider Dilbert-style management.
But this is bad management. Obvious to everyone who has ever attended a probability and statistics class in their engineering, computer science, or hard science education.¬†
So maybe the first dysfunction is those conjecturing¬†estimating is the smell of¬†dysfunction¬†is they don't understand the underlying mathematics of making estimates in the presence of uncertainty. This includes both management and those spending the money provided by management.
Since there is no domain, context, framing assumptions, or principles stated by those conjecturing¬†estimates are the smell of¬†dysfunction¬†let's look at one set of principles of writing software for money - other peoples money.
If we look for the root cause of projects going wrong, let's see how not following the 5 Principles can be a source of the dysfunction.
What Does Done Look Like?
Is there some notion of what capabilities the customer - those paying - need when we're done spending their money? Is there some units of measure of Done that are meaningful to those paying? If the answer is no, then we're likely to have little value for estimates, no matter the quality of the estimate.¬†
The smell of dysfunction is proceed to spend money without knowing what done looks like in any meaningful units of measure for those providing the money
What's the Path to Done?
Do we have any notion of the order of work to be performed? Let's assume there is some dependency in this work. The agile notion of INVEST must be tested first. Any non-trivial project has interdependencies. If there are non, then the work must be simple enough that all the pieces act independently from each other. No order of production, no order of operations, no order of use.¬†
The smell of dysfunction is not having a strategy to reach done on of before the need date for the capabilities that will earn back the investment for the money provided by those paying for your work.
Do We Have Enough Resources to Get to Done?
We need time, money, and resources to produce business value in exchange for the money we've been given. How much money? How much time? What resources?
The smell of dysfunction is not knowing how much of work will cots in the end to some level of confidence. Not knowing when we'll be done to some level of confidence. Our not knowing of what we've been asked to produce for that money and time will actually provide the needed capabilities those paying are expecting.
What Impediments Will We Encounter Along the Way?
All projects have uncertainty. Uncertainty produces risk. Managing in the presence of uncertainty means managing in the presence of risk.
Risk Management is How Adults Manage Projects - Tim Lister
Uncertainty comes in two forms reducible (epistemic) and irreducible (aleatory). Reducible uncertainty and its associated risk can be bought down. How much risk, what is the cost to buy it down? That means estimating.¬†
Irreducible uncertainty and its associated risk cannot be bought down. We need margin - cost margin, schedule margin, technical margin to protect the project from this unfavorable outcome. How much margin? We need an estimate. For both reducible and irreducible uncertainty answering that question comes easily with a Monte Carlo Simulation.
The smell of dysfunction is¬†not¬†having¬†a risk model for all the project work. Not have¬†estimating¬†for¬†the¬†probability¬†of occurance of an event based risks. Not have a Probability¬†Distribution¬†Function¬†of of the¬†naturally¬†occurring¬†variance¬†of irreducible¬†work - duration, cost, performance.
How Are We Going To Measure Progress to Plan?
To measure progress we need a plan. Then we need some assessment of phsycal percent complete. This measurement is an ideal paradigm for agile.¬†Working¬†products that meet the measures of effectiveness, measures of performance, technical performance measures, ley performance parameters are ways to assure what we produced actually does what is needed by those who are paying.
To start with a target to steer toward we need to estimate what are the possible outcomes of the project. That is what are the achievable goals in measures of Effectiveness and Performance. With this starting point and measures of actual performance we can create a¬†error¬†signal used to Close Loop Control to tag corrective actions to steer toward our target. The target can change of course, and many time does.
With the probabilistic target and the actual measurement we have an Open Loop Control system, which provides no steering signal and results in¬†we'll be done when we're done, we'll spend what we¬†spend, and you'll get what you get,. Could be better than planned, could be worse than plan - don't know.
The smell of dysfunction is to not have a¬†probabilistic¬†steering target developed from past¬†performance and models of future performance. Without this model we are¬†operating¬†open¬†loop. Not steering target that can be corrected with actual performance information., But¬†more¬†importantly¬†not steering¬†target¬†telling what our¬†performance¬†must be to meet the business goals of the project.¬†
So Want To Talk About Smells?
Tangible evidence of dysfunction is needed. Variance analysis needed.¬†Tangible corrective actions needed.¬†
Exploring is none of these. Exploring is talking about fixing the smell. Talking and exploring doesn't fix the dysfunction. ¬†Looking for¬†waste in the Muda sense - this is Muda. Do something tangible. Measure the result. Compare to plan. Make corrections to both action and plan - closed loop. Repeat till success.
Stop exploring - do something constructive. Correct the dysfunctions with actionable outcomes.¬†
In The End
Conjecturing to NOT do something without first identifying the root cause of the¬†smell is Open Loop decision making.
Conjecturing to NOT do something without saying what that something is so it can be tested is of little value to those paying money that need real help to increase the probability of success of their work efforts.
Conjecturing to¬†explore has little value to those seeking actionable corrections to the problems.¬†Exploring means o real commitment to improve.¬†We're just wandering around looking under rocks for interesting ideas. Having someone pay for that is called¬†pure research. Business that produces products and services in exchange for money are looking for value to result from their investment.¬†¬†Related articles What Happened to our Basic Math Skills? Systems Thinking, System Engineering, and Systems Management Making Decisions In The Presence of Uncertainty Everything I Learned About PM Came From a Elementary School Teacher The Art of Systems Architecting How We Make Decisions is as Important as What We Decide. Information Technology Estimating Quality
Populist books provide an important role in the processes of "thinking about things." They are simple, understandable in ways that resonant with the those not familiar with a topic, and are hopefully¬†gateway¬†sources to then next level of understanding. Populist books have a down-side as well. They are usually simplified versions of the underlying topic, devoid of the details, which unfortunately have mathematics that may be beyond the casual reader.
I've written about the issues with populist books before. There is a new set of issues that needs to be addressed. The¬†Think Fast Act Slow¬†book is a recent example of a populist book. It has useful materials, but leaves out all the ground work and heavy lifting needed to put these ideas to work.¬†
In graduate school, there are several things you learn before starting your thesis work. Do a literature search. You're bright idea may have already been done. Or worse your bright idea is a cockamamy idea on day one. If everyone tells you it's a cockamamy idea, you may be able to show the world they're wrong. To do that you need to get through a peer review and a test of your idea by strangers using actual data that holds up to ruthless testing by others. There have been a few of those, most have gone on to win the Nobel Prize.
So if you hear some idea that doesn't quite make sense, ask for the data that supports that idea, so you can do independent testing. Better yet if that idea is an obvious violating of the basic principles - either of physics (cold fusion) or of economics (#NoEstimates) ask those proposing the idea for direct evidence of its applicability that can also be independently tested.
Here's a list of supporting papers need to put the populist ideas to work from my library. Goggle will find these for you:
These are a small sample of the background that needs to be examined after read the populist book.
With this example, you can move beyond populist ideas - no matter how valid - to technical ideas and start putting them to work and testing the outcomes for their efficacy in your domain.
Here's a starting point for that effort in Populist versus Technical View of Problems¬†
We have an interesting problem in some projects. Agencies, consulting organizations, and consultants help their clients understand what the client needs in a product. Often, these people and their organizations then implement what the client and agency develop as ideas.
As the project continues, the agency manager continues to help the client identify and update the requirements. Because this a limited time contract, the client doesn’t have a product manager or product owner. The agency person—often the owner—acts as a product owner.
This is why Marcus Blankenship and I have teamed up to offer Product Owner Training for Agencies.
If you are an agency/consultant/outside your client’s organization and you act as a product owner, this training is for you. It’s based on my workshop Agile and Lean Product Ownership. We won’t do everything in that workshop. Because it’s an online workshop, you’ll work on your projects/programs in between our meetings.
If you are not part of an organization and you find yourself acting as a product owner, this training is for you. See¬†Product Owner Training for Agencies.
Making decisions in the presence of uncertainty of a future outcomes resulting from that decision is an important topic in the project management, product development, and engineering domains. The first question in this domain is...
If the future is not identical to the past, how can we make a decision in the presence of this future uncertainty?
The answer is we need some means of taking what we know about the past and the present and turning it into information about the future. This information can be measurements of actual activities - cost, duration of work, risks, dependencies, performance and effectiveness measures, models and simulation of past and future activities, reference classes, parametric models.
If the future is identical to the past and the present, then all this data can show us a simple¬†straight line¬†projection from the past to the future.
But there are some questions:
The answers to these and many other questions can be found in the mathematics of probability and statistics. Here's some popular misconceptions of mathematical concepts
Modeling is the Key to Decision Making
"All models are wrong, some are useful," George¬†Box and Norman R. Draper (1987). Empirical Model-Building and Response Surfaces, p. 424, Wiley. ISBN 0471810339.¬†
We can't possibly estimate activities in the future if we don't already know what they are
We actually do this all the time. But more importantly there are simple step-by-step methods for making credible estimates about unknown - BUT KNOWABLE - outcomes.
This know of unknown but knowable is critical. If we really can't know - it is unknowable - then the work is not a project. It is pure research. So move on, unless you're a PhD researcher.
Here's a little dialog showing how to estimating most anything in the software development world.¬†
With your knowledge and experience in the domain and a reasonable understanding of what the customer wants (no units of measure for reasonable by the way, sorry), let's ask some questions.
I have no pre-defined expectation of the duration. That is I have no anchor to start. If I did and didn't have a credible estimate I'd be a Dilbert manager - and I'm not.
Microeconomics of Decision Making
¬†Making decisions about the future in the presence of uncertainty can be addressed by microeconomics principles. Microeconomics¬†is a branch of economics that studies the behavior of individuals and small impacting organizations in making decisions on the allocation of limited resources. Projects have limited resources, business has limited resources. All human endeavors have limited resources - time, money, talent, capacity for work, skills, and other unknowns.¬†
The microeconomics of decision making involves several variables
Formally, defining this choice problem is simple: there is a state space S, whose elements are called states of nature and represent all the possible realizations of uncertainty; there is an outcome space X , whose elements represent the possible results of any conceivable decision; and there is a preference relation ‚™ł over the mappings from S to X.¬†‚Ä†
This of course provides little in a way to make a decision on a project. But the point here is¬†making decisions in the presence of uncertainty is a well developed discipline. Conjecturing it can't be done simply ignores this discipline.
The Valuation of Project Deliverables
It's been conjectured that focusing on value is the basis of good software development efforts. When suggested that this value is independent of cost this is misinformed.¬†Valuation¬†and the resulting Value used to compare choices,¬†is the process of determining the economic value of an asset, be it a created product, a service, or a process. Value is defined as the net worth, or the difference between the benefits produced by the asset and the costs to develop or acquire the asset, all adjusted appropriately for probabilistic risk, at some point in time.
This¬†valuation has several difficulties:
The valuation of the outcomes of software projects depends on the analysis of these underlying costs and benefits. A prerequisite for cost-benefit analysis is the identification of the relevant value and cost drivers to produce that value. Both cost and value are probabilistic, driven by ¬†uncertainty - both reducible and irreducible uncertainty
In addition to measurable benefits and costs of the software project, the valuation process must consider uncertainty. Uncertainty arises from different sources. Natural uncertainty (aleatory) which is ¬†irreducible. This uncertainty relates to variations in the environment variables. Dealing with irreducible uncertainty requires¬†margin¬†for cost, schedule, and the performance of the outcomes. For both value and cost.
Event based uncertainty (epistemic) which is reducible. That is we can buy down this uncertainty with out actions. We can pay money to find things out. We can pay money to improve the value delivered from the cost we invest to produce that value.
Parameter uncertainty relates to the estimation of parameters (e.g., the reliability of the average number of defects). Model uncertainty relates to the validity of specific models used (e.g., the suitability of a certain distribution to model the defects). There is a straightforward taxonomy of uncertainty for software engineering that includes additional sources such as scope error and assumption error. The standard approach of handling uncertainty is by defining probability distributions for the underlying quantities, allowing the application of a standard calculus. Other approaches based on fuzzy measures or Bayesian networks consider different types of prior knowledge. ‚Ä°
The Final Point Once Again
The conjecture we can make informed decisions about choices in an uncertain future can be done in the absence of making estimates of the impacts of these choices has no basis in the mathematics of decision making.
This conjecture is simply not true. Any attempt to show this can be done has yet to materialize in any testable manner. This is where the basic math skills come into play. There is no math that supports this conjecture. Therefore there is no way to test this conjecture. It's personal opinion uninformed by any mathematics.
Proceed with caution when you hear this.
‚Ä† Decision Theory Under Uncertainty,¬†Johanna Etner, Meglena Jeleva, Jean-Marc Tallon,¬†¬†Centre d‚ÄôEconomie de la Sorbonne 2009.64
‚Ä°¬†Estimates, Uncertainty and Risk. IEEE Software, 69-74 (May 1997),¬†Kitchenham and Linkman and "Belief Functions in Business Decisions. In: Studies in Fuzziness and Soft Computing, Vol. 88,¬†Srivastava and MockRelated articles Information Technology Estimating Quality Everything I Learned About PM Came From a Elementary School Teacher Carl Sagan's BS Detector Eyes Wide Shut - A View of No Estimates
In this week's lecture (https://cxlearn.com) I demonstrate how to use some of the size information we've discussed in other lectures by diving into the Wikipedia claims about the sizes of various versions of Debian. The point of this week's lecture is to show how to apply critical thinking to size information presented by an authoritative source (Wikipedia), and how to arrive at a confident conclusion that that information is not credible. Practicing software professionals should be able to look at size claims like the Debian size claims and, based on general knowledge, immediately think, "That seems far from credible." Yet, few professionals actually do that. My hope is that working through public examples like this in the lecture series will help software professionals improve their instincts and judgment, which can then be applied to projects in their own organizations.
Lectures posted so far include:
0.0 Understanding Software Projects - Intro
0.1 Introduction - My Background
0.2 Reading the News
0.3 Definitions and Notations
1.0 The Software Lifecycle Model - Intro
1.1 Variations in Iteration
1.2 Lifecycle Model - Defect Removal
1.3 Lifecycle Model Applied to Common Methodologies
1.4 Lifecycle Model - Selecting an Iteration Approach
2.0 Software Size
2.05 Size - Comments on Lines of Code
2.1 Size - Staff Sizes
2.2 Size - Schedule Basics
2.3 Size - Debian Size Claims (New)
Check out the lectures at http://cxlearn.com!
We have a new white paper, "Succeeding with Geographically Distributed Scrum Teams." To quote the white paper itself:
When organizations adopt Agile throughout the enterprise, they typically apply it to both large and small projects. The gap is that most Agile methodologies, such as Scrum and XP, are team-level workflow approaches. These approaches can be highly effective at the team level, but they do not address large project architecture, project management, requirements, and project planning needs. Our clients find that succeeding with Scrum on a large, geographically distributed team requires adopting additional practices to ensure the necessary coordination, communication, integration, and architectural work. This white paper discusses common considerations for success with geographically distributed Scrum.Check it out!