Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
"We suck at designing good database schemas." "I can never figure out how to configure the IIS VPN tunneling server." "This Sharepoint Server UI in SP 2012 is so much different than 2007, I can't possibly figure out how to help my customer."
Or, this 401(k) calculation for estimating my retirement is just beyond my ability to comprehend." Or, I can't possibly figure out how much money I need for my summer vacation with my family on a European trip, with relatives I haven't even met, and hotels and rental cars, train tickets, and meals, and other expenses, and airline reservations, and all the unknown and may be unknowable stuff. I'm just going to sit here and mope. Yep, that's me, Eeyore, woo is me. ¬†
What is it in some software domains that allows well educated, experienced developers to simply give up, walk way, allow themselves the be subjected to Dilbert Bosses? When they'd never talk that way about something they are skilled, experienced, and capable of doing?
It's a repeating theme in the #NoEstimates domain. You'd never hear a Flight Avionics engineer at a major space and defense company here in Denver say "we suck at estimating, we suck at testing, we suck at systems integration." Those engineers are asked all the time to "invent new physics to solve a problem." Or other colleagues literally inventing solutions of complex oil & gas software based process control algorithms. Or other colleagues at a major consulting firms deploying ERP and encountering legacy systems they've never seen or integrated before, saying "we can't figure this out, it's too hard and our managers will take our estimates as commitments, so we're simply not going to estimate our work."
Is it missing the engineering discipline taught in college. Where a probability and statistics class is mandatory for all 4 years to be called an engineer. Or even in the business and finance discipline estimating, using complex approaches - time series analysis, principle component analysis, bayesian statistics - is mandatory to be called a finance major. Or in the sciences in general. I say this from direct experience with our children and their significant others who are in finance, bio-sciences, and mechanical engineering all writing software for money as part of their actual profession.
And this whole notion that somehow writing software is not engineering is simply BS. It is engineering in many business domains. So maybe it's a domain issue. Maybe non-engineering cultures can't figure this out.¬†
Or is it maturity? Many appear to be very mature developers? Or is it culture, not geographic culture, but lack of having experienced a development culture where business and engineering work together to determine risks, corrective actions, increasing maturity of the deliverables development processes that allow for growth in both technical and business capabilities - all using agile by the way.
Everything is Uncertain
Business operates on risk taking. Risk taking is about making decisions in the presence of uncertainty. Uncertainty is about probable outcomes - positive or negative - in the future. To make decisions about uncertain outcomes in the future, we need to estimate those outcomes, the cost to achieve them, and the possible benefits from that cost.
No credible business person fails to understand this.
If they do, they won't be in business for long. It's taught in school - business school. Same for engineering and computer science school.
What is it that brings smart, capable, intelligent people to say "we suck," when the facilities (classes, books, articles, tools, external advisors) for learning how to estimate for the required reasons of making decisions about other peoples money for future outcomes, is the very basis of all business - is at hand, with simple learning and practice. These facilities are also the basis of all learning. "We suck" says "we're not capable of learning."
Are those of the #NoEstimate ilk saying "I can't learn?" or are they saying "I don't want to learn?" Because those providing them the money have a vested interest in knowing how much money is needed to produce that treasured¬†Value they claim to be producing.¬†
Just like learning a programming language. Moving from SP 2007 to SP 2012 (that actually sucks), dealing with cyber security. We don‚Äôt say ‚Äúwe suck‚ÄĚ and give up. What is actually going on here? Is some portion of the development community become Eeyore?Related articles Your Project Needs a Budget and Other Things Building the Perfect Schedule Don't Manage By Quoting Dilbert Qualitative Risk Management and Quantitative Risk Management Estimating is Risk Management
Risk Management is about many things - but it is first and foremost about estimating future outcomes.¬†
Suppose one of you wants to build a tower. Will he not first sit down and estimate the cost to see if he has enough money to complete it? For if he lays the foundation and is not able to finish it, everyone who sees it will ridicule him saying, "This fellow began to build and was not able to finish." - Luke 14:28-30
To manage the risk of¬†not enough money, not enough time, the probability of unacceptable outcomes we need to estimate. It's as simple as that and as complex as that.
Risk management is essential for development and production of products and services because key information about cost schedule, and technical performance are uncertain and many times unknown until later in the project.
Proceeding to spend other peoples money in the presence of these uncertainties is not only bad management, it's bad economics - the microeconomics of decision making requires estimating the impacts of any decision - the¬†opportunity cost¬†of that decision.
Risk management is concerned with the outcome of future events, whose exact outcome is unknown, and with how to deal with these uncertainties as a range of possible outcomes. -¬†Effective Risk Management: Some Keys to Success, 2nd Edition, Edmund H. Conrow.
So when you here¬†we can make decisions about the future without estimating ask yourself, did that speaker ever read about risk management. And as Tim Lister says
Risk Management is How Adults Manage ProjectsRelated articles Probability and Statistics of Project Work Qualitative Risk Management and Quantitative Risk Management Decision Making in the Presence of Uncertainty All The World's A Random Process Your Project Needs a Budget and Other Things
Is there room for functional managers, such as development and test managers, in agile organizations? Maybe. It depends on whether they take the role of an agile manager.
If you have organized as a feature teams-based organization, the functional managers (development, test, analysis, whatever) can take these responsibilities:
Functional managers are champions for the team, and shepherds for the process. They are servant leaders.
Here’s what functional managers do not do:
What does this mean? It means that the team members are leaders. Agile pushes responsibility into the teams, and away from traditional management. Agile requires leadership at all levels.
Agile challenges managers to recreate their jobs. An agile transformation requires managers work in an agile way, and work differently than before.
If you want to learn more about the role of leaders and managers in agile, join Gil Broza and me at The Influential Agile Leader, either in San Francisco or London this year. We still have an early bird price until mid-February. Don’t miss this opportunity to help your agile transition and your career.
Do you want to join me on my Virtual Street Team in February?
As my Virtual Street Team member you get:
Exclusive access to me via a private channel
Several exclusive on-line chats with me
A free Kindle or ePub copy of the #Workout book
Free signed copies of the book and my previous two books
The post Join Me on My Private Virtual Street Team in February! appeared first on NOOP.NL.
Plans are critical, a schedule to implement that plan comes next. With the Plan, the sequence of the work is needed to assure the value to the customer is delivered in the proper order. That order is an order because the business (or the mission) usually can't accept¬†all the features and functions at once. So the Plan is the top level sequence, and the schedule is the next level down.
Building the perfect schedule (v6) from Glen Alleman Related articles Your Project Needs a Budget and Other Things Qualitative Risk Management and Quantitative Risk Management
I don’t like it when people keep me waiting.
I reply to all emails within 24 hours. I pay my bills within two weeks, often within days. Last week, when one of my team members needed credit card information that I didn’t have with me, I did my best to send it within an hour. Web orders for my book? I validate shipping addresses within hours, and we send people’s orders within 48 hours. Invoices? I usually send them twice per month, or earlier when people ask for them.
When we read about a big IT problem, like...
The first impulse is to use this information to support some or other position, like¬†here's an example of estimate driven bad management. Without the logical conclusion of finding the actual Root Cause of the problem, as shown in the IDA report. Other examples usually start with bogus Standish data. Take a look on page iv below to see the real root cause, and see if¬†Not Estimating¬†would have been able to address the issues with ECSS? Not likley.¬†
This document seems to not load everytime, refresh the page if it doesn't
So we're back to the same place we always seem to come to. Domain and knowledge of the domain, before conjecturing any solution to any problem and the conjectured solution that occurs outside the domain of experience.¬†
For those not able to read the details here's a summary from the final section.
The notion of some that "estimates" are the root cause of the problems and that making decision in the absence of Estimates¬†is the solution to the problem based on¬†un-informed opinion.
So down load the report, read it in it's entirety, then assess other opinions in the light of actually reading the Root Cause Analysis, then drawing conclusions from the actual data and its analysis by the professionals tasked by the sponsors of the RCA who are accountable for looking after the money and the measurable outcomes.
And then the quote that says it all...
"Everyone is entitled to his own opinion, but not his own facts." -¬†Daniel¬†Patrick Moynihan¬†
¬†Related articles Taxonomy of Logical Fallacies
Don't blindly adopt anything.
Scrum is comprised of a self-organizing team that is given a challenge, and to meet that challenge, works in short, time-boxed iterations during which they meet daily to quickly synchronize their efforts.
At the start of each iteration, they meet to plan what they will accomplish. At the end, they demonstrate what has been accomplished and reflect on how well they worked together to achieve it.
That's it. Anything else—release planning, burndowns, and so on, is optional. Stick to the above and find the local optimizations that fit your environment. No expert knows more about your company than you do.
Let me tell you a little secret: Every now and then, I curse my customers. OK, it’s not really a secret, probably. I think the neighbors can hear it when I do. But that’s the way it is. I curse them.
I curse them because I care.
Many times I hear about¬†Cost of Delay,¬†Deliver Value,¬†Measure story points,¬†or¬†Measure Stories. And a myriad of other assessments of project performance, all of which - OK, most of which are examples of Open Loop Control.
Back in 2014, we had a paper in a publication of the¬†College of Performance Management,¬†starting on Page 17. As well, a colleague Nick Pisano (CDR US Navy Retired) has a post on the same topic at his blog.
The notion of a¬†baseline let alone a Performance Measurement Baseline is at the heart of¬†Closed Loop Control of all processes, from your heating and air conditioning system in your house, to the flight controls on the 737-700 winging its way back home to Denver, to the project you're working - using what ever project management method or software development method of your choosing.
The notion that we can manage anything, the temperature of the room, the¬†nice soft ride¬†in the 737, or the probability of showing up¬†on or before¬†the need date,¬†at or below the needed cost, with the needed capabilities - and NOT have a baseline to steer to is simply wrong.¬†
Below is the framework for Closed Loop control. This paradigm says simply:
ESTIMATING IS THE BASIS OF DECISION MAKING - it can't be any clearer than that.
Control systems from Glen Alleman Related articles Your Project Needs a Budget and Other Things The Actual Science in Management Science Probability and Statistics of Project Work What is Governance? Don't Manage By Quoting Dilbert
As I write the program management book, I am struck by how difficult it is to estimate large chunks of work.
What can you do? Here are some options:
Each of these shows your estimation audience you have uncertainty. The larger the project or program, the more you want to show uncertainty.
If you are agile, you may not need to estimate at all. I have managed many projects and programs over the years. No one asked me for a cost or schedule estimate. I received targets. Sometimes, those targets were so optimistic, I had to do a gross estimate to explain why we could not meet that date.
However, I am not convinced anything more than a gross estimate is useful. I am convinced an agile roadmap, building incrementally, seeing progress, and deciding what to do next are good ideas.
With internal releases, everyone can see the project or program progress.
In addition, we have a quarterly external release. Now, your project or program might not be able to release to your customers every quarter. But, that should be a business decision, not a decision you make because you can’t release. If you are not agile, you might not be able to meet a quarterly release. But, I’m assuming you are agile.
You might need to replace MVPs with MIFS,¬†Minimum Indispensable Feature Sets, especially at the beginning.
If you always make stories so small that you can count them, instead of estimate them, you will be close. You won’t spend time estimating instead of developing product, especially at the beginning.
You know the least about the risks and gotchas at the beginning of a project or program. You might not even know much about your MIFS or MVPs. However, if you can release something for customer consumption, you can get the feedback you need.
Feedback is what will tell you:
Here’s my experience with estimation. If you provide an estimate, managers won’t believe you. They pressure you to “do more with less,” or some such nonsense. They say things such as, “If we cut out testing, you can go faster, right?” (The answer to that question is, “NO. The less technical debt we have or create, the faster we can go.”)
However, you do need the planning of roadmaps and backlogs. If you don’t have a roadmap that shows people something like what they can expect when, they think you’re faking. You need to replan the roadmap, because what the teams deliver won’t be everything the product owner wanted. That’s okay. Getting feedback about what the teams can do early is great.
There are two questions you want to ask people who ask for estimates:
If you work on the most valuable project/program, why are you estimating it? You need to understand how much the organization wants to invest before you stop. If you’re not working on the most valuable project/program, you still want to know how much the organization wants to invest. Or, you need a target date. With a target date, you can release parts iteratively and incrementally until you meet the target.
This is risk management for estimation and replanning. Yes, I am a fan of #noestimates, because the smaller we make the chunks, the easier it is to see what to plan and replan.
We need planning and replanning. I am not convinced we need detailed estimation if we use iterative and incremental approaches.
The following was originally published in Mike Cohn's monthly newsletter. If you like what you're reading, sign up to have this content delivered to your inbox weeks before it's posted on the blog, here.
Suppose your boss has not bought into trying an agile approach in your organization. You schedule a meeting with the boss, and stress how your organization should use Scrum because Scrum:
And based on this discussion, your boss isn’t interested.
Why? Because you focused on the features of Scrum rather than its benefits.
Product owners and Scrum teams make the same mistake when working with the product backlog. A feature is something that is in a product—a spell-checker is in a word processor. A benefit is something a user gets from of a product. By using a word processor, the user gets documents free from spelling errors. The spell-checker is the feature, mistake-free documents is the benefit.
It is generally considered a good practice for the items at the top of a product backlog to be small. Each must be small enough to fit into a sprint, and most teams will do at least a handful each sprint.
The risk here is that small items are much more likely to be features than benefits. When a Scrum team (and specifically its product owner) becomes overly focused on features, it is possible to lose sight of the benefits.
Scrum teams commonly mitigate this risk in two common ways. First, they leave stories as epics until they move toward the top of the product. Second, they include a so-that clause in their user stories. These help, but do not fully eliminate the risk.
Let’s return to your attempt to convince your boss to let your team use Scrum. Imagine you had focused on the benefits of Scrum rather than its features. You told your boss how using Scrum would lead to more successful products, more productive teams, higher quality software, more satisfied stakeholders, happier teams, and so on.
Can you see how that conversation would have gone differently than one focused on short time boxes, self-organization and roles?
In project work we're looking to create or change something, in some defined period of time, for a defined cost. This means we're going to spend money now for some future outcome. The elements that go into this effort to produce some change in the future include (but are not limited to) scope of our efforts (requirements for the outcomes), technical performance (including quality and other ...ilities of the outcomes), the schedule for the work (so we don't have to do everything at once), the budget so we know the cost of the value produced), resources that do the work in exchange for money defined in the budget, risk to cost, schedule, and technical performance goals, and other attributes. A specific project will have specific constraints from each of these attributes.¬†
The relationships between these attributes is usually non-linear, random in some way (stochastic), and affects future outcomes. Because of the random nature of the attributes and the random nature of their relationship, simple linear, non-statistical projections of past performance used for future performance is most likely to be a disappointment.
To answer the question¬†what does the future look like when the past is a non-linear stochastic process, we need to be able to¬†manage in the presence of uncertainty. With this ability, the future simply¬†emerges and many times this futute is not what was expected.¬†
This is the role of planning. The best description of planning is
To be successful at planning we need to do many things. Since it is the future we're planning for each of these things requires an
Yes, we're never going to see it coming if we don't Plan. And to Plan, we need to estimate. And to estimate we need to learn how to estimate. So if we want to manage in the presence of uncertainty, here's a starting point...
When managing in the presence of uncertainty we're going to need to make estimates of the impacts of that uncertainty on our decision making processes for that¬†emerging future. When we hear we can make decision in the absence of estimates, it's simply not true in any non-trivial manner. For trivial decisions,¬†Do I start with the enrollment UI or the validation of IRS information UI, or¬†do I buy a Mac with 4GB or 6GB¬†of memory (assuming I can upgrade if I need more), making estimates is likely is little value. But for a decision like,¬†do we switch all our legacy systems to SAP or JD Edwards, I going to need a credible estimate of the cost, schedule, resources, risks, and tangible beneficial¬†outcomes for my enterprise.¬† So without a domain, context, the¬†Value at Risk, the underlying processes for uncertainty (reducible and irreducible) and some other attributes, the notion of making decisions in the absence of estimating the cost and impact of those decisions in simply bad management.¬† For Those Interested in Decision Making Processes in the Presence of Uncertainty
¬†Related articles Bayesian Statistics and Project Work Decision Making in the Presence of Uncertainty Your Project Needs a Budget and Other Things
I published a new Pragmatic Manager this past weekend. It’s called Discovering Your Leadership.
It has a pointer to the Influential Agile Leader event that Gil Broza and I are leading in San Francisco and then in London. You can still get early bird pricing, until Feb 15. Don’t miss this opportunity—we’re only leading it twice this year.
When we hear descriptons of Dilbert style management, with posting and reposting Dilbert cartons showing pointy haired bosses Doing Stupid Things on Purpose and no actionable advice on how to take corrective actions - then you know for sure the poster is just whinning - move on nothing of value here.
Addressing programs we all encounter in the project management work, means answering - with the % Whys - real 5 whys, not just the suggestion of the 5 whys - the symptom, the problem, and the root cause in the broad categories of:
Only with the answers to these categories in the three classes of assessment - what's the symptom, what's the problem that that produces the symptom, and what is the root cause of the problem - can corrective actions start.
None of this available - the conversation is a non-starter.Related articlesWhat is Governance?Taxonomy of Logical FallaciesYour Project Needs a Budget and Other ThingsThe 5 Whys Process We Use to Understand the Root of Any Problem
The use of Bayesian Statistics in project work is well developed. Here's an example of the approach. The drivers for each uncertainty can be modeled to produce on estimate of the¬†risk of the project not producing the desired outcomes.
What is needed is an understanding of the¬†prior probabilities of the drivers of the probabilistic outcomes, the relationships between the events created by these networks.
The Bayesian Network Approach ‚Ä†
Bayesian networks provide decision support processes for a wide variety of problems where uncertainty and probabilistic reasoning is involved. The Bayesian Network is a¬†directed graph with associated probability tables. The graph is the standard¬†nodes and arcs. The nodes represent uncertain variables. Each node has a set of states that represent causal or influential relationships between the variables. There is a probability table for each node, providing the probabilities of each state of the variable.
For prior variables - variables without parents the table contains¬†marginal probabilities. This is referred to as the¬†prior distribution which represents the prior belief - the state of knowledge - for that variable. For each variable with predecessors (parents), the probability table has the conditional probabilities for each combination of the predecessor states. This is the¬†likelihood function that represents how likely is a state of a variable given a particular state of its predecessor.
Bayesian Network are applied in situation that require statistical inference. The use in this instance knows some¬†evidence - some variable states or events that have¬†actual¬†observations. These observed values represent the¬†posterior¬†probability of the event occurring. By applying Reverend Bayes rile in each affect node, they can influence other Bayesian Network node by propagating and modifying the probability distributions.¬†
Bayesian Networks Have A Natural Connection With Project Planning
These networks can:
Bayesian Networks is a tool for decision support based on¬†Estimating¬†outcomes.
So when we hear that decisions can be made in the absence of estimates, ask for tangible examples of how this can be done, the basis (mathematics) of these processes, and examples in specific domains of how this can be done. If no answer is forth coming, question the veracity of the claims.
‚Ä† "Inference in Hybrid Bayesian Networks using dynamic discretisation," Martin Neil, Manesh Tailor,¬†and David Marquez,¬†Department of Computer Science, Queen Mary, University of LondonRelated articles The Actual Science in Management Science Qualitative Risk Management and Quantitative Risk Management What is Governance?
When I hear¬†what are the odds of success for a project, it tells me there is an undertstanding gap in how statistical processes are used in projects to produce probabilistic estimates of three things:
First let's echo Tim Lister's advice...
Risk Management is How Adults Manage Projects
All the World's a Statistical Process
First let's look at a network of work activities. These are tasks with dependencies, whose durations have naturally occurring variances. These durations can never be an exact number, since the work is emerging or simply varying naturally. Each activity has a unique Probability Distribution Function, which may be similar, but also unique. This is the case as well when there are no dependencies. The probabilistic processes are still in place. Even some place like the Toyota production line, no work process takes exactly the same duration. So if these natural variances are unaccounted for, you're going to be late, likely over budget, and your favorite gadget may not work. This concept is the basis of Statistical Process Control.
For each process, the upper and lower ranges, along with the Probability Distribution Function can be used to model the range of possible outcomes for duration - for example. In the picture below the probability of completing¬†on or before for the IOC (Initial Operating Capability) for Friday October 23, 2020 is 80%.
When we hear¬†the bet we'll¬†be successful, this is the number. It's not the right term¬†the bet but this is the term. There is an 80% confidence of completing¬†on or before 23 Oct 2020.
We can now¬†connect the dots between individual activities and a network of activities with the next chart. This shows the dependencies, each of their variances and how those drive the variances in the outcomes.
In the End the Discussion is About Domain and Context
When we hear about some new approach to making decisions in the absence of estimating the impact from those decisions, ask in what domain can that be possible. By possible I mean how can we make a decision by ignoring the principles of Microeconomics.
There may be domains in which that is completely possible. Below is a scale of projects I built awhile ago when working on an overall Program Governance engagement. From family gardening to building the USS Virginia there is a huge spectrum of techniques, processes, governance, tools, and approaches to increasing the probability of success. Having any discussion about the applicability of any idea has to start with¬†what domain are we in.Qualitative Risk Management and Quantitative Risk Management The Actual Science in Management Science What is Governance?