Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Herding Cats - Glen Alleman
Syndicate content
Performance-Based Project Management¬ģ - an integrated approach to Principles, Practices, and Processes to increase Probability of Project Success.
Updated: 7 hours 50 min ago

Process Improvement In A Complex World

9 hours 38 min ago

Much of the discussion around making improvements in processes fails to address the governance aspects of a business or organization. Instead it focuses on the personal aspects. Agile development of a software team, without the corporate impacts. The desire to stop doing something without an actual replacement, under the guise of we're exploring. Our the mention of the term intent of the commander without understanding that filling out that intent requires complete capabilities to act in in the absence of direct supervision.

My project management maturity was changed forever at Rocky Flats, under the management of senior leaders with experience and skill formed in the US Navy. The book Making the Impossible Possible is the story of that project and the learnings that can be applied in any high risk, complex, high reward domain.

Making the impossible possible from Glen Alleman The term Intent of the Commander is actualized in the concept of the Plan of the Day. Here's an actual plan of the day for shipboard life. This notion of plan of the day was applied late in the Rocky Flats program. Before that  we had Plan of the Week and then Plan of the Month. The approach is very simple. What do we plan to get done at the end of this period (day, week, month? Write that down, establish some measures of performance and effectiveness for those ourcomes. Measure them, take corrective actions if they don't match the expectations. Repeat until done. Plan of the Day
Categories: Project Management

Connecting the Dots Between Principles and Practices of Project Success

Tue, 04/22/2014 - 00:57

When we hear about all the methods of managing projects, the PMI Body of Knowledge, PRINCE2, home grown and commercial  solutions - always look at them in the light of these 5 Immutable Principles and the 5 Practices then implement the principles.

Principles and Practices of Performance-Based Project Management¬ģ from Glen Alleman
Categories: Project Management

We Can Know the Business Value of What We Build?

Sun, 04/20/2014 - 01:58

In a recent post titled #NoEstimates - Really? there was an interesting comment.

Clearly, the business value of any feature or project can not be known with much certainty in advance of it being implemented. Still, for the purpose of keeping the analysis simple for now, let’s table this issue for a bit.

This is not the case in a governance based organization or a Capabilities Based Planning organization, where the "valuation" of the resulting product, service, or purchased or built product is part of the planning process.

It's a "build to valuation"

Knowing - to some probabilistic level of confidence - what business value or mission fulfillment the project or product will produce is the core of any decision making process. Knowing the cost of the value is about making decisions, analysis of alterntaives, or assessing the trade space. 

With the "estimated" value and the "estimated" cost for that value, a decision can be made in what is called "analysis of alternatives" in our software intensive domain.

Only by having both estimates - value and  cost of acheiving  that value, their most likely numbers and the probabilistic range of those numbers (measured usually in dollars), can we make that "analysis of alternatives," or "trade space" needed to Govern both the business and the project and products that enable the business to meet its goals.


So there are popular myths around the estimating of cost and value discussion, and a few that are just flat out bogus:

  • We can't know the value of our outcomes, until they are done.
    • Start with a Balanced Scorecard strategy approach, where you define the¬†needed value of any work before starting, determine the estimated cost of achieving that value for the busines or mission and¬†do the math of ROI = (Value - Cost) / Cost.
  • No requirement can be confirmed to be correct until the software is complete and put to work.
    • Read any - my favorite The Requirements Engineering Handbook¬†- requirements management book to see if this makes any sense at all.
    • This assumes - the myth that is - that those developing the software really don't know what done looks like in any units of measure meanigful to the decision makers. Requirements definition is the role of Systems Engineering. So if the developers can't know, maybe they need to the Systems Engineers. Don't have Systems Engineers? Now that's a different problem.
    • All the Capabilities Based Planned and Requirements Management processes are based on assessing the value of those requirements¬†before the product or service arrives.
    • Assessing that value is part of all Systems Engineering process using Measures of Effectiveness and Measures of Performance.
    • During the project's execution, Technical Performance Measures and their supporting Quantifiable Backup Data are used to assue that what is being built is meeting the needs of the customer.
  • Making Estimates is a waste of time when we don;t know what the requirements are.
    • This is true.
    • But if you don't know what the requirements are, someone has to pay to find out. This someone is usually the customer. This is the basis of Agile, and it works.
      • But someone has to pay.¬†
    • Even in the realm where the requirements are vague, there is hopefully some ¬†notion of what¬†Capabilities are needed from the project or product.
    • What are you going to do with the results of the project or the product when it arrives?
    • How much are you willing to pay for those capabilities?
    • Don't have the answers to some level of confidence? - Just set fire to the money now and save all the effort.
  Related articles The "Real" Root Cause of IT Project Failure Resources for Moving Beyond the "Estimating Fallacy" 5 Questions That Need Answers for Project Success Why Johnny Can't Estimate or the Dystopia of Poor Estimating Practices
Categories: Project Management

Why Projects Fail, No Matter the Domain

Fri, 04/18/2014 - 23:07

There are endless reasons for why projects fail. Some are correct, some are bogus, some are down right nonsense. Into this list I'd like at add my opinion, informed by hands on experience on software intensives programs in defense, energy, bio-pharma, commercial products, and enterprise IT.

It starts and ends, with the

Failure to know what DONE looks like in units of measure meaningful to the decision makers

The starting point for the description of DONE is the clear and concise statement of the Needed Capabilities for the resulting project that produce value for the stakeholders. These capabilities are stated in Measures of Effectiveness.

These capabilities state what the results from the project will do for the business or mission when they are available. The planned availability date is part of their capability. Only then come the requirements, then planning for the delivery of the technical and operational components that enable these requirements. Then several more enablers of projects success are needed.

All these activities, outcomes, processes, methods are encapsulated in the paradigm of Systems Engineering. The picture below are the notional elements of Systems Engineering. This picture is from the FAA Systems Engineering Handbook. This handbook is for the National Airspace System (NAS), a software intensive program to upgrade the exiting software, hardware, and processes. Other agencies and business have similar pictures.

SE Elements

  • Synthesis - Transforms requirements into physical solutions.
  • Functional Analyses - describes the functional characteristics that are used to derive the products or services of the system resulting from the project
  • Requirements Management - identifies and manages the requirements that describe the desires capabilities of the project
  • Interface Management - identifies and manages the interactions between components of the system or interactions with external systems
  • Integrated Technical Planning - Plans the projects efforts and products
  • Speciality Engineering - Analyzes the system, requirements, functions, solutions, and/or interfaces using specialized skills and tools. Assists in the derivation of requirements, synthesis of solutions, selection of alternatives, and validation and verification of requirements.
  • Integrity of Analyses - Ensures that the analyses provide the required level of fidelity and accuracy.
  • Lifecycle Engineering - Identifies and manages requirements for system lifecycle attributes, including real estate management, deployment and transition, integrated logistics support, sustainment¬†/ technology evolution, and disposal.
  • Risk Management¬†- Identifies, analyzes, and manages the uncertainties of achieving program requirements by developing strategies to reduce the severity or likelihood of those uncertainties.
  • Trade Studies - assists decision making by analyuzing selecting the best-balanced solutions to match requirements that enable the capabilities
  • Configuration Management - Establishes and maintains consistency and manages change in the system performance, functional, and physical attributes.
  • Verification and Validation - Determines if system requirements are correct. Determines that the solution meets the validated requirements.

What Does All This Mean?

When we hear our customers don't know what they want. Or, our customers don't know the target budget for the work they are asking us to do for them. What they are saying is I don't know what DONE looks like in an tangible way that can be used to measure the performance of the project.

So one of two things has to happen when this is the case...

  1. The customer has to spend money to find out what they actually want. This is done all the time on our larger enterprise, defense, and space programs. It's paying to explore. It's establishing the credibility of the capabilities by spending money to do so
  2. Prepare for project failure.

So when we here that famous - infamous actually - phrase let's explore. Ask a serious question, and demand a serious answer ...

Who's pay for us to explore to discover what we should have already know? 

Don't get an answer? Run away from that project, it's going to be a failed project.


Categories: Project Management

EVM World Coming Soon

Fri, 04/18/2014 - 15:04

Speaking on Techncial Performance Measures and conducting workshop on same topic with Mr. Kranz and Tom Coonce of IDA.

Categories: Project Management

We're Asking the Wrong Question

Thu, 04/17/2014 - 04:15

The right question for any suggested improvement, change, or suggesting we stop doing that need to produce an answer to ...

Does this suggestion increase the probability of project success?

This means tracing the suggestion to the outcome of the project being better, faster, cheaper, or some other tangible measure of improvement. 

What does project success look like?

The delivery of agreed-upon capability within established resource constriants, e.g. funding, schedule, facilities.

Five factors are used to assess the success

  • Capabilities - do we know what capabilities will be needed, when they'll be needed, ¬†for what cost, to meet the business plan, competitive market position, or any other assessment of success?
    • It's the capabilities that enable the customer to do¬†valuable things for their customers, themselves, or their citizens.
  • Requirements - do we know what technical and operational requirements must be in place to fulfill the needed capabilities?
    • It's the capabilty to do something of value that the customer bought.
  • Resources - what resources will be needed to deliver these capabilities, fulfilled by the requirements within the expected cost and time needed to meet the business or mission goals for the Return on Investment?
    • ROI = (Value - Cost) / Cost
    • As a business or funding agency, this ROI is a primary assessment of success.
    • Did we get what we planned to get for the money we thought we would spend on the day we thought we would? All within the expected probabilistic confidence range?
  • Execution - do we have a means of measuring progress toward producing value for the customer?
    • This value needs to be available at a point in time to meet the plan for the business or the mission
    • This time and cost of the available capabilities is an essential need for any business.
    • Without a goal such as this there is no real crticality to the project. If the project is not critical, then any approach can be viable. The probability of success is really not an interesting question to ask.

So Now What?

When we hear about something new, anything new, how can we test it that suggestion against business needs, mission needs, governance, or strategy. This starts with establishing the domain in which the suggestion might possibly work. Then proposing the framework in which it has worked or might work. Then a proposed way to assess the possible benefits of performing the sugegsted solution.

Categories: Project Management

Quote of the Day

Wed, 04/16/2014 - 12:52

Wisdom begins when we learn the difference between "that makes no sense" and "I don't understand".
Mary Doria Russell

It is that difference that is needed to assess when a idea passes the smell test. 

Categories: Project Management

Software Creativity - Jazz or Beethoven's Ninth Symphony

Wed, 04/16/2014 - 01:18

Giants_of_JazzThere is a popular notion that writing software is like jazz. A loose collection of participants, improvising around a theme to produce an emergent outcome. 

These participants play off each other, react to emergent streams of melody, contribute their own special talent to the music and pretty much work in a a self directed manner over the course of the performance.

While I'm not a fan of analogies, this is a useful one for the purpose here. There are certainly domains where the Jazz analogy describes what is going on in the trio in the picture to the left.

Beethovne 9th Ode to JoyBut what about other music? Music that is just as creative, just as moving, just as impactful to the listener. Beethoven's Ninth Symphony's Ode to Joy with the poem from Friedrich Schiller 1785 work is an example.

Words that have moved nations and populations. Following Beethoven's Ninth was a movie we saw over the weekend that described this result.

In the Ninth, each performer has a score to follow, led by the conductor, but also by the concert master and the  senior players in each section. The vocals are also led by a senior performer.

In the software business there are likely similar domains and projects. Ones that can be improvised and ones that require conductors, a concert master, and players who follow the score.

In both cases - and this is where the analogy falls apart - is the players are highly skilled, experienced in the art, having played the basic themes 1,000 of times over before improvising or following the score. 

The jazz performance is not made up as it goes. OK, fusion is, but that crap makes my head hurt. Melodies, rules for cords are practiced for 10,000 hours (Gladwell), relationships between the players have magical connections not available to mere mortals. The Jazz Trio and the Berlin Philharmonic are populated with highly skilled and experienced professionals. We've all heard our children play in the school band and know what that sounds like. All the platitudes in the world about agile axioms are of no worth without the necessary capabilities to actually get the work done properly.

Applying the notion that agile software development is like jazz makes as much sense as saying I can sit in the 3rd chair of the trombone section (my high school band position) and play my contribution to Beethoven's Ninth without a Curtis Institute degree in performance and 10 years experience (my aunt was a professional pianist in the late 1950's from Curtis). 

It's not gonna happen - in both analogies - without the prerequisites of professional performance capabilities. Otherwise the result sounds like we're back in High School Music class with Mr. Meach (my teacher).

So how long will it take us to be capable performing to a level needed to not  smell like we're high school kids in the marching band? I don't know let's make an estimate.

Related articles Analogies, the Good, the Bad, and the Ugly The "Real" Root Cause of IT Project Failure
Categories: Project Management

What Does It Mean To Be DONE?

Tue, 04/15/2014 - 05:16

There is a continuing discussion in the agile community about delivering value in the order set by the customer. Along with this discussion is the use of the word DONE. A popular phrase is no requirement or piece of software can be considered DONE until it is put to use. 

This is a software developers point of view of course. But there is another view of software based products. It starts with the Measures of Effectiveness for the resulting product. These Measures of Effectiveness are:

Operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in an operation environment, under specific conditions.

This measure of DONE is not directly related to code, testing, requirements or anything like that. It is related to how Effective the software is in delivering the capabilities needed to fulfill the mission or business case. 

The individual requirements and pieces of code that implement them can be - or should be - traceable to these capabilities. For each Measure of Effectiveness, we then need a Measure of Performance. These measures characterise:

The functional or physical attributes relating to the system's operation, measured or estimated under specific conditions.

These are also not direclty related to producing code, running tests, or other direct software activities.

All the software design, testing, integration, etc. supports the creation of the ability to produce these Measures of Performance and Effectiveness. For the end user, all the development work is behind the scenes. What the customer actually bought was the ability to do something useful. To put a capability of the software system to work accomplishing a business need. Make money with this capability of course.

So What Does All This Mean?

It means that if you start at the bottom - with the software development processes - you're likley not going to see what the real picture is. This picture is that the customer paid for capabilities, measured in units of effectiveness and performance.

When we start with methods, paradigms, even cockamamie ones like not estimating the cost of the work effort needed to produce the capabilities, we loose the connection to why we are here. We are here to produce software that provides a capability. Likely more than one capability.

So when we hear words like - we can manage projects without knowing the cost or we'll let the requirements emerge, or the customer doesn't really know what they want, so we'll get started so they can decide along the way, ask how you are going to recognize DONE, before running out of time and money? 

How Do We Discover the Needed Capabilities?

Once we've decided that capabilities are in fact the place to start, how are they gathered. Here's the top level set of activities.

Screen Shot 2014-04-14 at 10.11.34 PM

Once we have these, we can start to elicit the technical and operational requirements that will fullfill these capabilities.

Screen Shot 2014-04-14 at 10.13.03 PM

These requirements can be emergent, they can evolve, they can be elicited incrementally and iteratively. But what ever way to appear they need to have a home. They need a reason for being here. They need to enable a capability to be available to the user. 

Categories: Project Management

Quote of the Day

Sun, 04/13/2014 - 21:40

Ninety percent of everything is crud - Theodore Sturgeon

In God We Trust

Related articles Models ...
Categories: Project Management

Many Explorers Don't Make It Home

Sat, 04/12/2014 - 22:15

When I hear the phrase we're exploring I'm reminded that in fact many who explore without a plan, measures of their progress progress against this plan, a risk management Plan-B for getting home when things go wrong, and without insufficient resources to survive the trip - come home empty handed or many time don't come home at all. Exploring without these items is called wandering around in the wilderness looking for something to eat. 

Here's a simple tale about an actual explorer, Ernest Shackleton, who experienced failure and near death on their first expedition to the South Pole (ADM Scott), that informed his attempt the reach the Pole a second time, only to experience failure again. In the first example prepartion was weak, management inconsistent, and lacking an actual strategy, no Plan-B. The second attempt, without Scott, was well planned, well provisioned, well staffed. When trouble started, Plan-B and then Plan-C were put in place and executed. 

  What's the Point About Managing Projects? So when we hear we're exploring and there is no destination in mind, or named problem to be solved, or even a description of possible root causes of the un-named problem, remember Shackleton's first trip and Scott's mismanagement of that exploration. On the second trip Shackleton had estimated what he would need, what route he would take, what skills his crew needed, what Plan-B would be, and even Plan-C when that didn't work, and most of all he estimated the probability of success to be high enough it was worth the risk to reach the South Pole and return to tell about it. The Polar expedition for Shackleton was a project, planned and executed with credible estimates of every step along the way, including the possibilities hat everything could go wrong and it did. Through is leadership, they lived to tell about it. Related articles Performance-Based Project Management(sm) Released Agile as a Systems Engineering Paradigm 1909 - Ernest Shackleton, leading the Nimrod Expedition to the South Pole, plants the British flag 97 miles (156 km) from the South Pole, the furthest anyone had ever reached at that time. Black Swans and "They Never Saw It Coming" 3 Impediments To Actual Improvement in the Presence of Dysfunction
Categories: Project Management

IT Governance and Decision Rights

Fri, 04/11/2014 - 16:15

GoveranceIn the discussions around #NoEstimates, it's finally dawned on me - after walking the book shelf in the office - the conversation is split across a chasm. Governance based organizations and non-governance based organizations. 

Same is the case for product development organizaitons. Those producing a software product for sale or providing a service in exchange for money. There are governance based product organizations and non-governance based product organizations. 

I can't say how those are differentiated, but there is broad research on the top of governance and business success using IT. The book on the left is a start. In this book there is a study of 300 enterprises around the world, with the following...

Companies with effective IT governance have profits that are 20% higher than other companies¬†pursuing similar strategies. One viable explanation for this differential is that IT governance¬†specifies accountabilities for IT-related business outcomes and helps companies align their IT¬†investments with their business priorities. But IT governance is a mystery to key decision makers¬†at most companies. Our research indicates that, on average, only 38% of senior managers in a¬†company know how IT is governed. Ignorance is not bliss. In our research senior management¬†awareness of IT governance processes proved to be the single best indicator of governance¬†effectiveness with top performing firms having 60, 70 or 80% of senior executives aware of how¬†IT is governed. Taking the time at senior management levels to design, implement, and¬†communicate IT governance processes is worth the trouble‚ÄĒit pays off.

IT Governance is a decision rights and accountability framework for encouraging desirable behaviours in the use of IT. And I'd add the creation of IT, IT products, and IT services. Since IT is a broad domain, let's exclude development effort for things like games, phone apps, plugins. and in general items that have low value at risk. This doesn't mean they don't have high revenue, but the investment value is low. So when they don't produce their desired beneficial outcome, the degree of loss is low as well.

Asessment of IT Governance focuses on four objectives:

  1. Cost effectiveness of IT
  2. Effective use of IT for asset utilization
  3. Effecrive use of IT for growth
  4. Effective use of IT for business flexibility

In all four, Weill and Ross provide guidance for assessing the capabilities of IT. In all four Cost is considered a critical success factor.

Without knowing the cost of a decision, the choices presented by that decision cannot be assessed. So when we hear #NoEstimates is about making decisions, ask of those decisions are being made in a governance based organization?

Then ask the question who has the decision rights to make those decisions? Who has the need to know the cost of the value produced by the firm in exchange for that cost. The developers, the management of the development team, the business management of the firm, the customers of the firm?

The three dependent variables of all projects are schedule, cost, and technical perfomance of produced capabilities (this is a wrapper word for everything NOT time and cost). The value at risk is a good starting point for deciding to apply governance processes or not. If you fix one of these variables - say budget (which is a place holder for cost until the actuals arrive), then the other two (time and technical) are now free to vary. Estimating their behaviour will be needed to assure the ROI meets the business goals. In the governance paradigm, these three dependent variables are part of the decision making process. Not knowing one of more puts at risk the value produced by the project or work effort. It's this value at risk that is the key to determining why, how, and when to estimate. 

What are you willing to loose (risk) if you don't need to know when you'll be done, or what you'll be able to produce on a planned day, or what that will cost, or determine if the ROI (return on investment), ROA (return on asset value), or ROE (return on equity) to some level of confidence to support your decision making - then estimating is a waste of time.

If on the other hand, the firm or customers you work for writing software in exchange for money have an interest in knowing any or all of those answers to support their decision making, you'll likley going to have to estimate, time, cost, produced capabilities to answer their questions.

It's not about you (the consumer of money). To find out who, follow the money, they'll tell you if they need an estimate or not.

Related articles Back To The Future Why Johnny Can't Estimate or the Dystopia of Poor Estimating Practices Danger Will Robinson Some more answers to the estimating questions Capabilties Based Planning, Use Cases, and Agile The Value of Making An Estimate
Categories: Project Management

5 Questions That Need Answers for Project Success

Thu, 04/10/2014 - 16:40

5 Success Questions

These 5 questions need credible answers in units of measure meanigful to the decision makers.

  1. Capabilities Based Planning starts with a Concept of Operations for each deliverable and the system as a whole. The ConOps describes how these capabilities will be used and what benefits will be produced as a result. This question is answered through Measures of Effectiveness (MOE) for each capability. MOEs are operational measures of success related to the achievement of the mission or operational objectives evaluated in an operational environment, under a specific set of conditions
  2. Technical and Operational requirements fulfill the needed capabilities. These requirements are assessed through their Measures of Performance (MOP), which characterize the physical or functional attributes relating the system operation, measured or estimated under specific conditions.
  3. The Integrated Master Plan and Integrated Master Schedule describes the increasing maturity of the project deliverables through Technical Performance Measures (TPM) to determine how well each deliverables and the resulting system elements satisfy or are expected to satisfy a technical requirement or goal in support of the MOEs and MOPs.
  4. Assessing progress to plan starts with Earned Value Management on a periodic basis. Physical Percent Complete is determine through adherence with the planned TPMs at the time of assessment and the adjustment of Earned Value (BCWP) to forecast future performance and the Estimate at Completion for cost and Estimated Completion Duration for the schedule.
  5. For each key planned deliverable and the work activities to produce it, Risks and their handling strategies are in place to adjust future performance assessment. Irreducible risks, such as duration and cost are handled with margin. Reducible risks are handled with risk retirement plans. Compliance with the risk buy down plan becomes a fifth assessment of progress to plan.

What Does All This Mean?

With these top level questions, many approaches are available, not matter what the domain or technology. But in the end if we don't have answers the probability of success will be reduced.

  1. If we don't have some idea of what DONE looks like in measures of effectiveness, then the project itself is in jeopardy from do one. The only way out is to pay money during the project to discover what DONE looks like. Agile does essentially this, but there are other ways. In all ways, knowing where we are going is mandatory. Exploring is the same as wandering around looking for a solution. If the customer is paying for this, the project is likely a R&D project. Even then the "D" part of R&D has a goal to discover something useful for those paying.
  2. When we separate capabilities based planning from requirements elicitation, we are freed up to be creative, emergent, agile, and maximize our investments. The technical and operational requirements can then be traced to the needed capabilities. This approach sets the stage for valiation of each requirement. Answering the question - why do we have this requirement? There is always mention of feature blot, this is an approach to test each requirement for its contirbution to the business or mission benefit of the project.
  3. The paradigm of Integrated Master Planning (IMP) provides the framework for assessing the increasing maturity of the project's deliverables. The IMP is the strategy for producing these deliverables along their path of increasing maturity to the final deliverables. Terms like preliminary, final, recviewed, accepted, installed, delivered, available, etc. are statements about the increasing maturity.
  4. Measuring physical percent complete, starts with the exit criteria for all packages of work on the project. This is a paradigm of agile, but more importantly it has always been the paradigm in defense and space domain. The foundation of this is Systems Engineering that is just starting to appear in enterprise IT projects.
  5. Risk Management is how adults manage projects - Tim Lister This says it all. Don't have a risk register, an assessment of the probability of occurrence, impacts, handling plans, residual risk and its impact - those risks are not going away. Each risk has to be monetized and their handling included in the budget. 


Categories: Project Management

Don't Start With Requirements Start With Capabilities

Tue, 04/08/2014 - 15:08

Lot's of myth floating around about requirements elicitation and management. Starting with requirements is not how large, complex, mission critical DOD and NASA programs work. Start with Capabilities. This cna be directly transferred to Enterprise IT.

Here's a map of the planned capabilities for an ERP system. This figure is from Performance-Based Project Management¬ģ where all the deatails of this and other principles, practices, and processes needed for project success can be found.

Here each business systems capability is outlined in the order needed to maximize the business value. In agile parlance, the customer has prioritized the deliverables. But in fact the prioritization is part of the strategic planning needed to assure that the cost investment returns the maximum value to assure the business receive maintains a positive ROI throughout the life of the project

Capabilities Map

The first step is to identify the needed capabilities. Here's the steps

Capabilities Based Planning

Only when we have the capabilities defined for each stage of the project can we start on the requirements. All the capabilities need to be identified and sequenced, otherwise we can't be assured to business goals can be met for the planned investment. With the planned capabilities, we can start on the requirements

Requirements Steps

With requirements in place for each capability, we can then start development. This is done incrementally and iterative using our favorite agile method. Doesn't mater as long an incremental delivery of value of the approach.

Categories: Project Management

Is There Such a Thing As Making Decisions Without Knowing the Cost?

Mon, 04/07/2014 - 18:46

How Much is that DogyyExtensive research has shown given that a current project is more than fifteen percent complete, the overrun at completion will not be less than the overrun incurred to date; and the percent overrun at completion will be greater than the percent overrun incurred to date. Assuming no change in scope or reduction in delivered capabilities, this overrun is locked.

Without knowing the original Estimate At Complete (EAC), the funders of the project have no way of making decisions about the project's total cost, its  incremental cost, or how to adjust scope and duration to meet the expected cost incurred in exchange for the expected value produced from this cost. Without this cost information, and related schedule, and techncial performanc information, the notion of decision making is nonsense. 

We can't make a decision without knowing the cost and benefits of the resulting decision

The absence of an estimated cost, duration, and delivered capability prevents the business from knowing if they are making the right decision about the investment in the project. So if decision-making is what management does, not knowing this information prevents credible decisions from being made

Not estimating these three data items ‚Äď cost, schedule, and technical performance (delivered capabilities) ‚Äď is simply bad business management. What ever unfavorable outcomes ‚Äď overruns, failed business capabilities, unhappy customers (ACA the most recent example) ‚Äď is well deserved.

The notion that we can make decision in the absence of estimates of their cost and benefit, appears to be unfounded conjecture, with no evidence of its validity. 

Categories: Project Management

The Calculus of Writing Software for Money, Part II

Sun, 04/06/2014 - 16:43

Boehm1981The dictionary definition of economics is a social science concerned with the description and analysis of production, distribution, and consumption of goods and services. [1]

Another definition of economics, closer to software developemnt is the study of how people make decisions in resource-limited situations. This definition matches the classical use of the term and is the basis of software economics. Since software product development always takes place in the presence of limited resoruces. Time, money, capabilities, even knowledge. And since software development always is the exchange of those resources for the production of value, looking at development from an economics point of view is a good start for any discussion around improving the process. 

Two other definitions are needed before continuing. Macroeconomics is the study of how people make decisions in a resource-limited situation on a national or global scale. Microeconomics is the study of how people make decisions in a resource-limited situation on an individual or organization scale. 

For software development, microeconomics deals more with the type of decision making needed for successful projects. And since much of the discussion these days is about making decisions on projects, let's see how the microeconomics paradigm may improve the communication.

There have been suggestions that the book above is old school and no longer applicable to the modern world of software development - e.g. Agile. Since the book is actually about engineering economics  not about software development methods, let's see what the book actually says for those who have not read it, heard Dr. Boehm speak, or in my case worked for the same firm where Dr. Boehm lead our process improvement management effort.

This book was a working text, when I attended USC as a Master's student while working at TRW (Boehm's home) for an Engineering Economics course. The book is still in print and available in used form for low prices. So those wishing to comment on the book, without having first read all 765 pages, can now do so at a low cost.

The preface of the book starts with the usual qualifiers, but contains three important objectives

  1. To make a book easy for students to learn from
  2. To make a book easy for professors to teach from
  3. To provide help for working professionals in the field

The major objective of the book is to provide a basis for a software engineering course, intended to be taken at the college senior or first year graduate level

So let's look at chapters to get a feel of the concepts of software engineering economics. My comments on the chapter are in italics.

  1. Chapter 1 - Case Study of the Scientific American subscription processing system.
  2. Chapter 2 - Case Study of an Urban School Attendance system
  3. Chapter 3 - Goals of Software Engineering
  4. Chapter 4 - The Software Lifecycle: Phases and Activities. This is where the book does not represent the current practice. 1980 Agile was not known. But TRW applied an iterative and incremental development process to Cobra Dane and TRDSS, two programs I participated in as a software engineer writing signal processing code in FORTRAN 77. 
  5. Chapter 5 - The Basic COCOMO Model. This model is in use today, along with SEER , QSM, Price-S, and many other software cost and schedule modeling tools. COCOMO was the first. But all are based on some sort of reference class estimating process.
  6. Chapter 6, 7, 8, and 9 - The  COCOMO Model: Development Modes,Activity Disribution, Product, and Component level 
  7. Chapter 10 - Performance Models and Cost-Effectiveness Models
  8. Chapter 11 - Production Functions: Economies of Scale
  9. Chapter 12 - Choosing Amoung Alternatives: Decision Criteria. This is where the microeconomics starts to enter the current discussion. If we intend to make decisions about how we are going to spend our customers money, we need to consider (from the chapter)
    • Minimum available budget
    • Minimum performance requirements Performance in this context is anything techncial. We have cost, schedule, and performance the three variables of all projects.
    • Maximum effectiveness / Cost Ratio
    • Maximum Effectiveness-Cost Difference
    • Composite options
    • The basis of all decision making in the software development paradigm starts and ends with cost. This is true, simply because writing software for money, costs money.
    • Value to customer, delievry dates are of course critical decision making attributes as well.
    • Knowing how much money, how that money is being utilized - efficacy of funds, how that money is time phased - absorption of funds, and the "return on investment" of those funds Starts and Stops with estimating (because we can't know for sure in the beginning and can't wait till the end) how much it will cost to deliver the needed capabilities to produce the value for the customer.¬†[2]
  10. Chapter 13 - Net Value and Marginal Analysis - another chapter on the effacay of money
  11. Chapter 14 - Present versus Future Expenditure and Income. Another microeconomics chapter on forecasting and estimating budget, performance, and value
  12. Chapter 15 - Figures of Merit - how to decide what to do, now that you have some notion of cost, schedule, performance, and risk
  13. Chapter 16 - Goals as Constraints. Some good discussion in the #NoEstimates world about contraints. Wonder if that author is famailar with this book?
  14. Chapter 17 - Systems Analysis and Constrained Optimization. The Masters degree TRW sent us to was about Systems Engineering and Systems Management. This is one of the chapter we paid close attention to.
  15. Chapter 18 - Coping with Un-reconcilable and Unquantifiable Goals. Lots of decisions deal with these two conditions. "Deciding" means performing an "Analysis of Alternatives" from a microeconomic point of view. This is course means knowing something about the three variables of all projects. One of which is cost.
  16. Chapter 19 - Coping with Uncertainties: Risk Analysis
  17. Chapter 20 - Statistical Decision Theory: The Value of Information. Much of the discussion around current "estimating" processes fails to deal with the statistical processes involved in those decision. Small samples - 5 cycles, self-selected samples, uncalibrated baselines, use of terms (bayesian) without working examples of their use - all seem to have entered the conversation. This chapter can provide insight to managing software projects using statistical processes.
  18. Chapter 21 - Seven Basic Steps in Software Cost Estimation. When it is mentioned "software can't be etimated," this chapter had better have been read and found flawed with working, reviewed examples of it not working. Conjecture is no substitute for facts.
  19. Chapter 22 - Alternative Software Cost Estimation Methods. When we hear "we're exploring for new ways to estimate" wonder if this chapter has been read?
    • Algorithmic models
    • Expert judgement
    • Estimation by analogy
    • Parkinsonian estimating
    • Price-to-Win estimating
    • Top-down estimating
    • Bottom-up estimating
    • Summary comparison of methods
    • So next time we hear "we're exploring altenative" ask straight out - "What are those alternatives, where have you had success with them compared to other methods, what domain have you had this success, and can you share the details of this success so others can asertain if they might be applicable in their domain?¬†
    • When there is no information forthcoming to these questions, think hard about the veracity of the speaker. Maybe they don't actually have a solution to this critically important problem that is still with us in 2014.
  20. Chapters 23 - 29 are about the details of the COCOMO model
  21. Chapters 30 - 33 are about software cost estimation and life-cycle management using more the COCOMO model

So What's the Point of All This?

When we hear estimating can't be done for software, we actually know better. It is being done in every software domain. Tools, processes, books, papers, conferences, vendors, professional organizations will show you how.

When we hear this, we now know better.

[1] "Software Engineering Economics," Barry W. Boehm, IEEE Transactions on Software Engineering, Volume SE-10, Number 1, Januarym 1984, pages 4-21.

[2] This is the crux of the post, the book and the discussion around the conjecture that we can make decisions about how to spend other peoples money with estimating that spend. 

Related articles Why Johnny Can't Estimate or the Dystopia of Poor Estimating Practices
Categories: Project Management

The Calculus of Writing Software for Money

Sat, 04/05/2014 - 15:43

Writing_is_like_sexThere is a nearly unlimited collection of views about how to write software for money.

From the anarchy of gaming coders sitting in the basement of the incubator on 28th and Pearl Street here in Boulder to the full verification and validation of ballistic missile defense system software, 7 miles up the road.

When I hear about how software should be written, how teams should be organized, how budgeting, planning, testing, deployment, maintenance, transiston to business, transistion to production, sustainment, and the myriad of other activities around software development should be done - the first question is always - what's the domain you're speaking about. 

Then - have you tested these ideas outside our personal experience. And finally have you tested these ideas in another domain to see if they carry over? If you're just exploring ideas, no problem. But that limits the credibility of the idea to being just and idea with no actionable outcome, other than a conversation. Those paying for the software you are writing for money, usually don't like paying for you to explore using their cash - unless that effort is actually in the plan.

There are of course fundamental - immutable actually - principles for any project based endeavor. These are the Five Immutable Principles of all project success, shown over and again in the root cause analysis of failed projects.

  1. Do we know what DONE looks like in some units of measure meaningful to the decision makers?
  2. Do have any idea on how to arrive at DONE at an expected time so the user can put our efforts to work when they are needed?
  3. Do we know what resources we need to perform Number 2? These include talent, facilities, and of course money.
  4. Do we have any idea what's going to go wrong along the way and how we're going to handle it so we can show up on time with the needed value the customer is expecting?
  5. And finally, how are we going to convince ourselves we are making progress at the rate expected by those paying for our effort?

All five of these principles need answers if we're going to have any hope of success. No matter how often it is repeated, insisted upon, or how clever the message is trying to avoid these principles, they're not going away. They are immutable. They need to be answered on day one and on every day until the project is over.

So if we are writing software for money - internal money, external money, maybe even our own money - ask these questions and see if our answers are credible.

  1. We don't know what done looks like, so let's get started and find out. This is a good way to spend more money. Write a few sentences about what capabilities will be provided by the software. Use these to test the business value. From those capabilities, requirements can emerge. Don't listen to anyone who suggests software requirements age if not put to use immediately. Think of the requirement of properly interfacing with the IEEE-1553 bus next time you're sitting in seat 23B on a 737 watching it fly through the air. That requirement was set down in 1973.
  2. Without a plan any path will get us lost. Make a plan. It can be a set of sticky notes on the wall. Or it can be 30,000 lines of an Integrated Master Schedule to fly to Mars and return. Never ever listen to someone who says planning isn't needed, they've probably not been accountable for delivering on-time, on-budget anything.
  3. How much will this cost? Don't know? It's going to cost more than you think. If those providing you the money aren't interested in how much you're going to spend, when you're going to spend it, cash their check up front, because they're going to run out of money before you're done.
  4. Risk Management is How Adults Manage Projects - Tim Lister. Behave appropriately.
  5. Tangible evidence of progress is always measured from the users point of view. In the end measures of effectiveness are the best assessment. The Measures of Performance. Compliance with requirements is weak. We have lots of examples of compliant, but no useable products. When someone says working software is our measure  ask what are our units of measure of "working?" Don't confuse effort with results. Show me it does what we planned it to do, on or near the day we planned it to do it.

More in next post about the economics of writing software for money.

Categories: Project Management

Do It Right or Do It Twice

Fri, 04/04/2014 - 15:23

Screen Shot 2014-04-03 at 10.47.18 AMI heard this phrase in a conference call yesterday with a DOD client and thought, how clever I'll write a blog about this. Only to find out there is a Forbes article with same name and several other articles as well. 

The Forbes article had a case study about¬†doing it right around a business process. It was the perfect framework (repeated here) for applying¬†Performance-Based Project Management¬ģ¬†

In the Forbes article there are five steps:

  1. The Vision Meeting - develop a set of needed capabilities for the outcome of the project. These provide the ability for the business to do something of value. There is all this discussion around creating value, but rarely are the units of measure for value mentioned.
    • Value cannot exist if we don't know both the units of measure¬†of the value itself and the cost to deliver those units of measure. This is where the naive and ill informed notion of #NoEstimates and the phrase¬†we are exploring for ways to make decisions with "No Estimates"¬†goes right in the ditch.
    • The¬†analysis of alternatives¬†is a starting point. Balanced Scorecard is a broader approach, but AoA will work for this post. If we have some idea about what capabilities we need to possess, then we can make decisions about them. What do they cost? How do they actually provide value to our business or mission.
    • What are the units of measure of that value. One good unit of measures is¬†effectiveness. How¬†effective will this new capability be in solving our problem?
  2. Build a strategy - what is the Plan to deliver the needed capabilities. The notion that we don't need to plan - we'll let the resulting capabilities and their technical and operational requirements emerge - is of course going to allow us to Do It Twice.
    • We need to know what DONE looks like in units of measure meaningful to the decision makers. The Plan is not the schedule. The Plan describes what will be delivered.
    • The schedule shows when it is needed to deliver the value. We need both.¬†
  3. Adapt if necessary - the needed capabilities should pretty much be fixed. If not we're wasting time and money exploring for what DONE looks like.
    • That's called Research. No problem, if we acknowledge we are on a research project. If the customer thinks - and has paid us - we're on a development and delivery project, there's going to be disappointment when we discover we've spent a bunch of money¬†exploring when we should have been delivering.
    • When I here about projects where the customer doesn't know what they want yet, so let's get started and we'll discover along the way. Go back to the office and get a bigger check book.
  4. Execute in time boxes - time boxing, rolling waves, incremental, iterative execution and delivery are  common sense. No one knows enough about anything at the detail level to know how to build it far into the future.
    • The Capabilities shouldn't be changing, but the mechanism for delivering these capabilities must be flexible and adaptable. The key outcome from executing in time boxes, is the answer to the question -¬†how long are you willing to wait before you find out you are late? The answer must be, short enough to take corrective action to NOT be late. This time interval is domain and project dependent. But answering this question will define your business rhythm.
  5. What are we working on now? What are we working on next? - make this visible. Have a plan of the month, a plan of the week, a plan for this quarter.
    • Have everyone on the project acknowledge they know what the outcomes of this plan are in units of performance, technical performance measures, and the quantifiable backup data showing physical percent complete.
    • We must measure progress in this manner. This is the notion in agile software of¬†working software. But the agile community doesn't have a formal way of stating the units of measure of¬†working. They leave that up to the customer, who may not know. The Systems Engineering paradigm does, through Measures of Effectiveness, Measures of Performance, and Technical Performance Measures.
    • Create a framework based on these, and only then insert your favorite agile development processes.

In the end project success is about knowing what done looks like, knowing how to get there, how to measure progress along the way. And of course knowing impediments to progress and handling them. These concepts are instantiated in two papers from a colleague Pat Barker, What is Your Estimate at Complete and Program Master Scehdule Can Improve Results, on page 20.

Not Sure Where We Are

Categories: Project Management

Agile as a Systems Engineering Paradigm

Thu, 04/03/2014 - 15:41

In yesterday's post, the notion of Systems Engineering was suggested as one solutuion to project failure. Here's the next step. The Agile notion started with a manifesto that turned into many interpretations and practices. In the standard project management paradigm, there is a set of principles, practices, and processes described in a variety of ways through several organizations. ITIL, PMI, APM, DOD, DOE, and other owners of project management activities.

When we take the Systems Engineering approach, we can put a wrapper around ALL project management, technical development, and deployment processes, that can be use to assess each practice and process to assure it is providing value. Here's a short overview of this paradigm. 

Agile project management is systems management from Glen Alleman The frameworks for Systems Engineering starts with several guides
  • ISO 15288
  • INCOSE Systems Engineering Handbook
  • FAA Systems Engineering¬†
Categories: Project Management

Slicing Work Into Small Pieces

Mon, 03/31/2014 - 19:32

One of the suggestions in #NoEstimates is the slicing of work - either Stories or any word needed to indicate an agile projects chunking of the work - into small pieces. This of course doesn't actually address the issue of producing and Estimate at Completion for the project. An estimate needed by those funding or authorizing the spending of funds to know how much and when.

But slicing is a process of reducing the exposure to uncertainty to a manageable size.  It's the next level down's answer to what's the value at risk? Make in small and reduce the value at risk of not showing up on time and on budget. Slicing answers the question, that has been around for some time.

How long are you willing to wait till you find out you are late (or over budget, or it doesn't work as planned)?

The answer to this  - how long - question varies according to the domain, value at risk, and other factors usually associated with risk tolerance. But it is a question that must be answered periodically (month;y for us) Recently this notion of slicing has been put forth as part of the solution to the estimating problem, which of course it's not. Since the size of the work chunks only reducing the uncertainty of the variance. Both the aleatory (irreducible uncertainty) and epistemic (reducible uncertainty) will be less when the exposure to the uncertainty is smaller. Beneficial to the project for sure. But the total all in cost and schedule are related to the slicing size only by the cumulative variance of the parts.

It many be interesting to know, that slicing is part of ANSI-748 Earned Value Management assessment by the Defense Contract Management Agency (DCMA). DCMA is the DOD agency that validates the Earned Value Management System's 32 Guidelines. DCMA performs a 14-point assessment of the Integrated Master Schedule (integrated because it is connected to the cost based) and the Performance Measurement Baseline (PMB) (the time phased planned budget for all the work).

DCMA Check 8 looks for high duration activities. These are know to cause issues with the exposure to programmatic risk for the program. The 44 day number represents 2 working months. The work then passes through one accounting period (monthly submission of the Integrated Program Management Report). At the end of each accounting period an assessment of Percent Complete is used to calculate the Budgeted Cost of Work Performed (BCWP) - the earned value for the tasks, works packages, and control accounts (funding buckets) for the program. 44 days may sound long for an agile software development project, but 44 days is short on a multi-year, many millions and likley billions of dollars of defesnse work.

Screen Shot 2014-03-30 at 4.18.49 PM

Since the agile comminity is  fond of saying there is nothing new here, while suggesting their ideas are new and unique, the above clip is from the DCMA guide, long used in our defense program management paradigm.

So it is worth repeating the principle of asking how long you are willing to wait before you find out something. The rule of thumb is to sample the status of the thing you are controlling are a rate twice your needed to control or determine a value. This is called the Nyquist rate from single processing. Signal processing is where I grew up writing software for Fast Fourier Transforms, Finite Impulse Response Filter, Kalman Filters for particle physics data streaming off the particle accelerator. When I didn't have an orginal idea to finish my PhD studies, I switched to writing the same software for radar signals intelligence, and electroninc warfare systems. Same principles work for any control system including a management control system.

Just as an aside in the control systems paradigm, there is discussion about monitoring and decision making from the information gathered from the monitoring. This is an Open Loop control systems. Without a planned value to seek, the SET POINT if you're using the room thermostat analogy, monitoring the value provide no value, since you don't know what you are seeking the system to do. Just monitoring is Open Loop, you've got numbers from the system the room temperature or the number of stories produced, but no target to control against.

To have a closed loop system, you need a SET POINT, a steering target, a goal, a desired outcome. Then the monitoring - sampling - can produce a variance, a difference between goal and actual - by which you cn take action. Raise or lower the temperature, speed up or sloe down the car, speed up or slow down the production of software outputs. Yes you can go too fast, the down stream user can't take the results and by the time they can, the requirements may have changed. This is the Closed Loop Control. 

Open Closed Loop

Categories: Project Management