Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator&page=4' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

Surface new proximity-based experiences to users with Nearby

Google Code Blog - Thu, 06/09/2016 - 17:16

Posted by Akshay Kannan, Product Manager

Today we're launching Nearby on Android, a new surface for users to discover and interact with the things around them. This extends the Nearby APIs we launched last year, which make it easy to discover and communicate with other nearby devices and beacons. Earlier this year, we also started experimenting with Physical Web beacons in Chrome for Android. With Nearby, we’re taking this a step further.

Imagine pulling up a barcode scanner when you’re at the store, or discovering an audio tour while you’re exploring a museum–these are the sorts of experiences that Nearby can enable. To make this possible, we're allowing developers to associate their mobile app or a website with a beacon.



A number of developers have already been building compelling proximity-based experiences, using beacons and Nearby:

Getting started is simple. First, get some Eddystone Beacons- you can order these from any one of our Eddystone-certified manufacturers. Android devices and and other BLE-equipped smart devices can also be configured to broadcast in the Eddystone Format.

Second, configure your beacon to point to your desired experience. This can be a mobile web page using the Physical Web, or you can link directly to an experience in your app. For users who don’t have your app, you can either provide a mobile web fallback or request a direct app install.

Nearby has started rolling out to users as part of the upcoming Google Play Services release and will work on Android devices running 4.4 (KitKat) and above. Check out our developer documentation to get started. To learn more about Nearby Notifications in Android, also check out our I/O 2016 session, starting at 17:10.

Categories: Programming

Starting An Agile Effort

Looking at the map to starting an Agile effort?

Looking at the map to starting an Agile effort?

What is needed to start an Agile project?  There are a number of requirements for beginning an Agile effort.  Those requirements typically include a big picture understanding of the business need, a budget, resources, and a team.   Somewhere in that mess, someone needs to understand if there are any unchangeable constraints. A high-level view of the five categories of requirements for starting an Agile effort are: 

  1. Business Need – All efforts need to begin with a goal firmly in mind.   While the absolute detail of how to achieve that goal may be discovered along the way, it is unconscionable to begin without firmly understanding the goal. Understanding the goal of the effort is the single most important requirement for starting any effort. Storytelling is a tool to develop and share the effort’s goal.
  2. Budget – In almost every organization, spending money, effort or time (the calendar kind) on an effort means that something else does not get funded. Very few efforts are granted the luxury of unlimited funds (money or effort). All efforts require a budget.  Budgeting begins at the portfolio level where decisions on which piece of work need to be addressed, then flows downward to programs or release trains where it divided up into finite pots of money.  The trickle down of the budget typically ends a team’s doorstep with a note saying that they have this much “money” to address a business need. The term money is used loosely as the effort is often used as a currency in many organizations.  If the business need or goal is the most important, then having a budget is a close second.
  3. Resources – In corporate environments resources generally include hardware, software, network resources and physical plant (a place to work).  People are not resources. In the late 90’s I participated in large bank merger projects.  In one of the projects the resources that had to be planned for were renting a floor in building (and lots of desks, chairs, phones, and stuff) and funding a route on an airline for the length of the project.
  4. People – People, often organized in teams, get the work accomplished.  Individual Agile teams should be cross-functional, self-organized and self-managed. A good team is a mixture of behaviors, capabilities and the right number of bodies. People are the third most important requirement for beginning an Agile effort.
  5. Constraints – Understanding the hard constraints for any effort has to operate within is important.  Some efforts are in response to legal mandates (income tax changes for example) or have to fit within specific hardware footprints (embedded code for example).  Constraints often are the impetuous for innovative solutions if they are known and anticipated. Note: constraints, like risks, can evolve, therefore they need to be revisited as an effort progresses.  

There is a hierarchy of requirements.  An effort needs a goal. A goal is needed to acquire a budget. A budget is needed to acquire a team and resources. Constraints are a wildcard that can shape the all of the other requirements.  Understanding and ensuring that the effort’s requirements are addressed is what is necessary for starting an Agile effort. 

The next few blog entries will explore each category in greater detail.


Categories: Process Management

The Case for and Against Estimates, Part 2

In the first part of this series, I said I liked order-of-magnitude estimates. I also like targets in lieu of estimates. I’ll say more about how estimates can be useful in part 3.

In this part, I’ll discuss when I don’t like estimates.

I find estimates not useful under these conditions:

  • When the people estimating are not the people doing the work.
  • When managers use old estimates for valuing the work in the project portfolio.
  • When management wants a single date instead of a date range or confidence level.

There are more possibilities for using estimates in not-so-hot ways. These are my “favorite” examples.

Let me take each of these in turn and explain how agile specifically helps these. That’s not because I think agile is the One and Only Answer. No, it’s because of the #noestimates discussion. I have used #noestimates in a staged-delivery project and on agile projects. I have not been able to do so on iterative or serial (waterfall/phase gate) projects. Of course, with my inch-pebble philosophy, I have almost always turned an iterative or serial project into some sort of incremental project.

People Estimate on Behalf of the Project Team

We each have some form of estimation bias. I have a pretty good idea of what it takes me to finish my work. When I pair with people, sometimes it takes longer as we learn how to work with each other. Sometimes, it takes much less time than we expected. I expect a superior product when I pair, and I don’t always know how long it will take us to deliver that product. (I pair-write with any number of people during the course of the year.) Even with that lack of knowledge, we can pair for a short time and project to a reasonable estimate. (Do a little work and then re-estimate.)

When people who are not part of the project team estimate on behalf of other people, they don’t know at least these things: what it will take the real project team to deliver, how the people will work together, and how/if/when the requirements will change. I have my estimation bias. You have yours. We might learn to agree if we work together. But, if we are “experts” of some sort, we don’t know what the team will encounter and how they will handle it.

I too often see experts ignore requirements risks and the potential for requirements changes. I don’t trust these kinds of software estimates.

Now, when you talk to me about construction, I might answer that we know more about construction. We have dollars per sq. foot for houses. We have dollars per road mile for roads. And, I live in Boston, the home of the Big Dig. Every time we remodeled/rebuilt our house, it came in at just 10% over the original number. We worked hard with the builder to manage that cost.

Those projects, including the Big Dig, were worth it.

How do we make software projects worth it? By delivering value as often as possible and asking these questions:

  • Is there still risk to manage?
  • Is there more value in the backlog?
  • How much more do we want to invest?

Software is not a hard product. It is infinitely malleable. What we deliver on Monday can change the estimate for what we want to deliver on Tuesday, Wednesday and Thursday. We can’t do that with hard products.

When other people estimate, we can’t use what we learn by working together and what we have learned already about this domain. Agile helps this specifically, because we deliver often and can re-estimate the backlog if we need to do so. We understand more about the remaining risks because we deliver.

Managers Use (Old) Estimates for the Project Portfolio

I have seen managers use estimates to value projects in the project portfolio. I wrote a post about that years ago: Why Cost is the Wrong Question for Evaluating Projects in Your Project Portfolio.

Here’s the problem with old estimates. Estimates expire. Estimates are good for some time period. Not forever, but for some number of weeks. Depending on how you work, maybe the estimate is good for a couple of months. Estimates expire because things change: the team might change. The codebase and the requirements have certainly changed.

However, project cost is only one part of the equation. Value has to be another part when you think about the project portfolio. Otherwise, you fall prey to the Sunk Cost Fallacy.

You might say, “We use ROI (return on investment) as a way to value projects in the project portfolio.” Now you depend on two guesses: what it will take for you to complete the project and the sales/adoption rate for the release.

ROI is a surrogate measure of value. When I have measured the actuals (what it actually took us to finish the project and the actual revenue at three, six, nine and twelve months out, we almost always did not meet the projected ROI. And, because we chose that project with great-looking ROI, we incurred a cost of delay for other projects. “If we don’t release this project because we are doing something else, what is the effect on our revenue/adoption/etc?” (See Diving for Hidden Treasures to read about the different costs of delay.)

People often say, “These two projects are equal in terms of project cost. If I don’t use ROI, how can I decide between these projects?”

I have never seen this to be true, and it’s quite difficult to predict which project will be shorter. Here are some options:

  • Use Cost of Delay as a way to value the projects in the project portfolio. See Diving for Hidden Treasures for ways to see Cost of Delay. See Manage Your Project Portfolio for many other ranking ideas for the project portfolio.
  • Determine the first releasable deliverable of value for each project. How long will that take? If you do one project, release something, does that provide you enough revenue so you can go to the other project and release something there?
  • Make all the deliverables small, so, if necessary, you could flow work from both projects through one team. The team can finish a feature/deliverable and move to the next one. I recommend using a kanban board and swarming over each feature so you get maximum throughput. Once the team has finished “enough” features, decide which project to spend more time on.

Agile helps the entire project portfolio problem because we can all see progress on an agile project: demos, product backlog burnup chart, and retrospective results. We know a lot more about what we finish and where we are headed. We can stop working on one project because we don’t leave work in an unfinished state.

Management Wants the Comfort of a Single Estimation Date

I supply a range of dates for my projects: possible, likely, pessimistic. I sometimes supply a confidence range. I have met many managers who do not want the reality of estimation. They want a single date: September 1, 2pm.

The problem is that an estimate is a guess. I can only know the exact duration or cost when I’m done with the project. I can get closer as we finish work, but I can’t know for sure months in advance. For a year-long project, I can guess as to which quarter/three month period. As we finish the project, I can spiral in on a date. By the last quarter, I can be within a couple of weeks of knowing.

Managers get paid the big bucks to manage the organization with assumptions, risks, and unknowns that we explain to them. When we work on projects, it’s our job to manage our risks and deliver value. The more value we deliver, the fewer unknowns our managers have.

Agile (and incremental approaches) help us manage those unknowns. Nothing is perfect, but they are better than other approaches.

I’ve worked with several managers who wanted one date. I gave them the pessimistic date. Sometimes, I provided the 90% confidence date. Even then, there were times we had more problems than we anticipated. Meeting that date became impossible.

A single-point estimate is something we like. Unfortunately, a single-point date is often wrong. Management wants it for any number of reasons.

If one of those reasons is assurance that the team can deliver, agile provides us numerous ways to get this result without a single-point estimate: set a target, see demos, see the product backlog burnup chart.

I have nothing against estimation when used properly. These are just three examples of improper estimate use. Estimates are guesses. In Part 3, I’ll talk when estimates might be useful.

(Sorry for the length of this post. I’ll stop writing because otherwise I’ll keep adding. Sigh.)

Categories: Project Management

SE-Radio Episode 259: John Purrier on OpenStack

John Purrier talks with Jeff Meyerson about OpenStack, an open-source cloud operating system for managing compute resources. They explore infrastructure-as-a-service, platform-as-a-service, virtualization, containers, and the future of systems development and management. Cloud service providers like Amazon, Google, and Microsoft provide both infrastructure-as-a-service and platform-as-a-service. Infrastructure-as-a-service gives developers access to virtual machines, servers, and network infrastructure. […]
Categories: Programming

Daydream Labs: VR plays well with others

Google Code Blog - Tue, 06/07/2016 - 17:04

Posted by Rob Jagnow, Software Engineer, Google VR

At Daydream Labs, we pair engineers with designers to rapidly prototype virtual reality concepts, and we’ve already started to share our learnings with the VR community. This week, we focus on social. In many of our experiments, we’ve found that being in VR with others amplifies and improves experiences in VR, as long as you take a few things into account. Here’s what we’ve learned so far:

Simplicity can be powerful: Avatars (or the virtual representations of people in VR) can be simplified to just a floating head with googly eyes and still convey a surprising degree of emotion, intent, and number of social cues. Eyes give people a location to look to and speak towards, but they also increase face-to-face communication by making even basic avatars feel more human. When we combine this with hands and a spatially-located voice, it comes together to create a sense of shared presence.



Connecting the real and the virtual: Even when someone is alone in VR, you can make them feel connected. For example, you can continue to carry a conversation even if you’re not in VR with them. Your voice can serve as a subtle reminder that they’re spanning two spaces—the real and the virtual. This asymmetric experience can be a fun way to help ground party games where one player is in VR but other players aren’t, like with charades or Pictionary.

But when someone else joins that virtual world with them, we’ve seen time and time again that the real world melts away. For most multiplayer activities, this is ideal because it makes the experience incredibly engaging.



Join the party: When you first start a VR experience with others, it can be tough to know where to begin. After all, it’s easier to join a party than to start one! Create shared goals for multi-player experiences. When you give people something to play with together, it can help them break the ice, allow them to make friends, and have more fun in VR.

You think you know somebody: Lastly, people who know each other offline immediately notice stature or differences in a person’s height in VR. We can re-calibrate environments to play with height and scale values to build a VR world where everyone appears to be the same height. Or we can adjust display settings to make each person feel like they’re the tallest person in the room. Height is such a powerful social cue in the real world and we can tune these settings in VR to nudge people into having more friendly, prosocial interactions.

If you’d like to learn more about Daydream Labs and what we’ve learned so far, check out our recent Lessons Learned from VR Prototyping talk at Google I/O.

Categories: Programming

Announcing the Certification of Agencies as part of Google Developers Agency Program

Google Code Blog - Tue, 06/07/2016 - 16:58

Posted by Uttam Kumar Tripathi, Global Lead, Developer Agency Program

Back in December 2015, we had shared our initial plans to offer a unique program to software development agencies working on mobile apps.

The Agency Program is an effort by Google’s Developer Relations team to work closely with development agencies around the world and help them build high quality user experiences. It includes providing agencies with personalized training through local events and hangouts, dedicated content, priority support from product and developer relations teams, and early access to upcoming developer products.

Over the past few months, the program drew a lot of interest from hundreds of Agencies and we have since successfully launched this program in a number of countries including India, UK, Russia, Indonesia, USA and Canada.

Having worked with various agencies for several months, the Agency Program has now launched certification for those partners that have undergone the required training and have demonstrated excellence in building Android applications using our platforms. The Agency Program hopes that doing so would make it easier for clients who’re looking to hire an agency to make an informed decision while also pushing the entire development agency ecosystem to improve.

The list of our first set of certified agencies Agencies is available here.


We do plan to review and add more agencies to this list over the year and also expand the program to other countries.

Categories: Programming

Don’t Pull My Finger

NOOP.NL - Jurgen Appelo - Tue, 06/07/2016 - 11:54
Dont Pull My Finger

As a public speaker, I get the weirdest requests.

Sometimes, event organizers want me to provide some “seed questions” that a moderator can ask me after a presentation. Usually, they use these when audience members do not immediately raise their hands when asked if they have any questions. In such a case, the moderator switches to a prearranged seed question after which the audience has usually awakened from its coma.

I don’t like giving organizers seed questions. Why should I be the one to tell them which questions they must ask me? It makes me think of a not-to-be-named family member who asked kids to pull on one of his fingers when he felt some gas coming up. And when they innocently pulled a finger, guess what happened? He thought it was hilarious.

Event organizers know their audiences better than I do. There’s no need to be lazy or to defer the bootstrapping of the Q&A to me. It is their job to make sure that we answer the important questions of their audience. And they shouldn’t care about anything that I most urgently want to get out of me.

When moderators do their job well, they generate a question or two on behalf of their audiences. And when they do, it happens often enough that they pose me questions I would never have imagined, and I need to think and offer an answer fast!

That prevents me from being lazy too.

I’m not going to let anyone pull my fingers. So don’t ask. If I have something relevant to say, I’ll say it. I don’t need to have it pulled out of me. Instead, I look forward to getting surprising questions that will challenge me!

(c) 2008 Derek Bridges, Creative Commons 2.0

My new book Managing for Happiness is available from June 2016. PRE-ORDER NOW!

Managing for Happiness cover (front)

The post Don’t Pull My Finger appeared first on NOOP.NL.

Categories: Project Management

Behind the scenes: Firebass ARG Challenge

Google Code Blog - Mon, 06/06/2016 - 21:47

Originally posted on Firebase blog

Posted by Karin Levi, Firebase Marketing Manager

This year's Google I/O was an exciting time for Firebase. In addition to sharing the many innovations in our platform, we also hatched a time-traveling digital fish named Firebass.

Firebass is an Alternate Reality Game (ARG) that lives across a variety of static web pages. If you haven’t played it yet, you might want to stop reading now and go fishing. After you’ve caught the Firebass and passed the challenge, come back -- we’re going to talk about how we built Firebass.

How we began

We partnered with Instrument, a Portland-based digital creative agency, to help us to create an ARG. We chose ARG because this allowed us to utilize developers’ own software tools and ingenuity for game functionality.

Our primary objective behind Firebass was to make you laugh, while teaching you a little bit about the new version of Firebase. The payoff for us? We had a blast building it. The payoff for you? A chance to win a free ticket to I/O 2017.

To begin, we needed to establish a central character and theme. Through brainstorming and a bit of serendipity, Firebass was born. Firebass is the main character who has an instinctive desire to time-travel back through prior eras of the web. Through developing the story, we had the chance to revisit the old designs and technologies from the past that we all find memorable -- as you can imagine, this was really fun.

Getting started

We put together a functional prototype of the first puzzle to test with our own developers here at Google. This helped us gauge both the enjoyment level of the puzzle and their difficulty. Puzzle clues were created by thinking of various ways to obfuscate information that developers would be able to recognize and manipulate. Ideas included encoding information in binary, base64, hex, inside images, and other assets such as audio files.

The core goal with each of the puzzles was to make them both logical but not too difficult -- we wanted to make sure players stayed engaged. A bulk of the game’s content was stored in Firebase, which allowed us to prevent players from accessing certain game details too early via inspecting the source code. As an added bonus, this also allowed us to demonstrate a use-case for Firebase remote data storage.

Driving the game forward

One of our first challenges was to find a way to communicate a story through static web pages. Our solution was to create a fake command line interface that acted as an outlet for Firebass to interact with players.

In order to ground our time travel story further, we kept the location of Firebass consistent at https://probassfinders.foo/ but changed the design with each puzzle era.

Continuing the journey

After establishing the Pro Bass Finders site and fake terminal as the centerpieces of the game, we focused on flushing out the rest of the puzzle mechanics. Each puzzle began with the era-specific design of the Pro Bass Finders home page. We then concepted new puzzle pieces and designed additional pages to support them. An example of this was creating a fake email archive to hide additional clues.

Another clue was the QR code pieces in puzzle 2.

The QR codes demonstrate Firebase time-based read permissions and provide a way to keep players revisiting the site prior to reaching the end of puzzle 2. There were a total of three pieces of a QR code that each displayed at different times during the day. It was really fun and impressive to see all of the different ways players were able to come up with the correct answer. The full image translates to ‘Locating’, making the answer the letter ‘L’, but many players managed to solve this without needing to read the QR code. You're all smart cookies.

Final part of the Puzzle

Puzzle 3 encompassed our deep nostalgia for the early web, and we did our best to authentically represent the anti-design look and feel of the 90s.

In one of the clues, we demonstrated Firebase Storage by storing an audio file remotely. Solving this required players to reference Firebase documentation to finish writing the code to retrieve the file.


<!-- connect to Firebase Storage below -->
<script>
console.log('TODO: Complete connection to Firebase Storage');
var storageRef = firebase.app().storage().ref();
var file = storageRef.child('spectrogram.wav');

// TODO: Get download URL for file (https://developers.google.com/firebase/docs/storage/web/download-files)
</script>
The finale

While the contest was still active, players who completed the game were given a URL to submit their information for a chance to win a ticket to Google I/O 2017. After the contest was closed, we simply changed the final success message to provide a URL directly to the Firebass Gift Shop, a treasure in and of itself. :)

Until next time

This was an unforgettable experience with a fervently positive reaction. When puzzle 3 unlocked, server traffic increased 30x! The community response in sharing photos, Slack channels, music, jokes, posts, etc. was incredible. And all because of one fish. We can’t wait to see all the swimmer winners next year at I/O 2017. Until then, try playing the game yourself at firebase.foo. Thank you, Firebass. Long may you swim.

><(((°<
Categories: Programming

Auto-generating Google Forms

Google Code Blog - Mon, 06/06/2016 - 21:07

Posted by Wesley Chun (@wescpy), Developer Advocate, Google Apps


pre.CICodeFormatter{ font-family:arial; font-size:12px; width:99%; height:auto; overflow:auto; line-height:20px; padding:0px; color:#000000; text-align:left; } pre.CICodeFormatter code{ color:#000000; word-wrap:normal; }
 function createForm() {  
// create & name Form
var item = "Speaker Information Form";
var form = FormApp.create(item)
.setTitle(item);

// single line text field
item = "Name, Title, Organization";
form.addTextItem()
.setTitle(item)
.setRequired(true);

// multi-line "text area"
item = "Short biography (4-6 sentences)";
form.addParagraphTextItem()
.setTitle(item)
.setRequired(true);

// radiobuttons
item = "Handout format";
var choices = ["1-Pager", "Stapled", "Soft copy (PDF)", "none"];
form.addMultipleChoiceItem()
.setTitle(item)
.setChoiceValues(choices)
.setRequired(true);

// (multiple choice) checkboxes
item = "Microphone preference (if any)";
choices = ["wireless/lapel", "handheld", "podium/stand"];
form.addCheckboxItem()
.setTitle(item)
.setChoiceValues(choices);
}

If you’re ready to get started, you can find more information, including another intro code sample, in the Google Forms reference section of the Apps Script docs. In the video, I challenge viewers to enhance the code snippet above to read in “forms data” from an outside source such as a Google Sheet, Google Doc, or even an external database (accessible via Apps Script’s JDBC Service) to generate multiple Forms with. What are other things you can do with Forms?

One example is illustrated by this Google Docs add-on I created for users to auto-generate Google Forms from a formatted Google Doc. If you’re looking to do integration with a variety of Google services, check out this advanced Forms quickstart that uses Google Sheets, Docs, Calendar, and Gmail! Finally, Apps Script also powers add-ons for Google Forms. To learn how to write those, check out this Forms add-on quickstart.

We hope the DevByte and all these examples inspire you to create awesome tools with Google Forms, and taking the manual creation burden off your shoulders! If you’re new to the Launchpad Online developer series, we share technical content aimed at novice Google developers, as well as discuss the latest tools and features to help you build your app. Please subscribe to our channel, give us your feedback below, and tell us what topics you would like to see in future episodes!

Categories: Programming

The Case for and Against Estimates, Part 1

After the article I referenced in Moving to Agile Contracts was published, there was a little kerfuffle on Twitter. Some people realized I was talking about the value of estimates and #noestimates. Some folks thought I was advocating never estimating anything.

Let me clarify my position.

I like order-of-magnitude estimates. I don’t hire people without either a not-to-exceed or an order-of-magnitude estimate. I explained how to do that in Predicting the Unpredictable: Pragmatic Approaches to Estimating Project Cost or Schedule.  That’s because I have to manage the risk of the money and the risk I won’t get what I want.

Notice there are two risks: money and value. When I need to manage both risks, I ask for order-of-magnitude estimation and frequent demos. When we did the remodel for the house we are living in now—almost a rebuild—we had an estimate from our builder. Our builder was great. He encouraged us to see the house every day to see their progress. The builder was transparent with problems. Was he truly agile? In order for him to create an estimate, we iterated on the designs for each room before he broke ground.

Construction, hardware development, mechanical products—all those “hard” products require iteration on the design before implementation. That’s because the cost of change is so high when you move to physical form.  In my experience, the cost of not iterating before you go to physical form is prohibitive.

So, what is the value of estimation for software? I have said (In Predicting) that software is learning, innovation. We learn from every software project. That makes estimation tricky, if not impossible. Can you estimate? Of course. The problem I see is in the value of the estimate. That value changes for the domain and customer.

If you have a reluctant-to-agile customer, you might want to do more estimation as you work through the project. That was the point of the Moving to Agile Contracts article. You might not even convince a customer that agile is good for them. If you get the value out of working in an agile way, great. You still get the value, even if the customer doesn’t.

Example.AgileRoadmapOneQuarterIf you have a regulated domain or a complex project that you might want to cancel, you might need more estimation as you proceed. I still like using my deliverable-based roadmaps and a not-to-exceed project cost. I would ask, “How much change do we expect?” If the deliverables are going to change every day or week, I don’t see how you can estimate and believe it. You can do a not-to-exceed for a date or cost.

In software, most of the cost is in the run rate for the project.

The image here is an example one-quarter roadmap from Agile and Lean Program Management. In a program, people often need to see further into the future than a backlog or two. I often see organizations requiring six-quarter roadmaps. That’s fine. The roadmap is a wish list. Why? Because it’s not possible to provide a reasonable estimate of that much work that far out without doing some work.

Here’s the tricky part: how much work do you need to do for estimation? I don’t know.

StagedDeliveryIn the Twitter conversation, Glen Alleman mentioned that Raytheon is doing a project using agile. I am pretty sure the agile Raytheon guy I know (socially) is on that project. Yes, they do 4-week iterations. They work feature-by-feature. I believe, although I am not positive, they started that project with a significant investigation period. To me, that project looks a lot more like staged delivery. On the other hand, does it matter??

Does that mean it’s not agile? Well, staged delivery does not require the same transparency and culture change that agile does. On the other hand, does it matter? Remember, I am not religious about what you do. I want your project to succeed.

So what about #noestimates? How does that figure into this conversation?

Here are times when you might not want to bother estimating:

  • You have a fixed target. Management said, “We need this project done by that date.” In that case, get a ranked backlog and get to work. Why fight with people or waste time estimating when you have no idea what you can do? In Predicting, I say something like this, “Get a ranked backlog. Get to work. Get some data. Show progress. If you can’t deliver what they want when they want it, you now have data for a discussion.”
  • You think things will change every day or every week. Management/your sponsor says, “Here’s the ranked backlog. We want to see what you can do so we know what we want to change.” Inviting change is why we use agile. Otherwise, we could used staged-delivery. Why estimate? I would use a not-to-exceed date or cost.
  • You are on the project to save the company. Get a ranked backlog and get to work. Determine how often you can release to get revenue.

I have been on all these kinds of projects. I have gotten a ranked backlog and gotten to work. I have succeeded. Oh, in one case, the company management started the project to save the company too late to make a difference. I didn’t succeed then. We needed four weeks to make a difference and had two.

I like delivering small chunks often. Yes, I use deliverables in my work that are small, often an hour or less.  I can stop when I get to the end of them and not worry about the next chunk. I am sure I do different work than you do.

That is why, as Glen says, the domain is critical. I think it’s also the customer. Maybe there are more things to consider.

In my next post, I will discuss when estimates are harmful.

Categories: Project Management

Moving to Agile Contracts

Marcus Blankenship and I wrote a follow-up piece to our first article, mentioned in Discovery Projects Work for Agile Contracts. That article was about when your client wants the benefit of agile, but wants you to estimate everything in advance and commit to a fixed price/fixed scope (and possibly fixed date) project. Fixing all of that is nuts.

The next article is Use Demos to Build Trust.

That post prompted much Twitter discussion about the purpose of estimates and trust. I’ll write a whole post on that because it deserves a thoughtful answer.

Categories: Project Management

The Inquiry Method for Test Planning

Google Testing Blog - Mon, 06/06/2016 - 14:07
by Anthony Vallone

Creating a test plan is often a complex undertaking. An ideal test plan is accomplished by applying basic principles of cost-benefit analysis and risk analysis, optimally balancing these software development factors:
  • Implementation cost: The time and complexity of implementing testable features and automated tests for specific scenarios will vary, and this affects short-term development cost.
  • Maintenance cost: Some tests or test plans may vary from easy to difficult to maintain, and this affects long-term development cost. When manual testing is chosen, this also adds to long-term cost.
  • Monetary cost: Some test approaches may require billed resources.
  • Benefit: Tests are capable of preventing issues and aiding productivity by varying degrees. Also, the earlier they can catch problems in the development life-cycle, the greater the benefit.
  • Risk: The probability of failure scenarios may vary from rare to likely, and their consequences may vary from minor nuisance to catastrophic.
Effectively balancing these factors in a plan depends heavily on project criticality, implementation details, resources available, and team opinions. Many projects can achieve outstanding coverage with high-benefit, low-cost unit tests, but they may need to weigh options for larger tests and complex corner cases. Mission critical projects must minimize risk as much as possible, so they will accept higher costs and invest heavily in rigorous testing at all levels.
This guide puts the onus on the reader to find the right balance for their project. Also, it does not provide a test plan template, because templates are often too generic or too specific and quickly become outdated. Instead, it focuses on selecting the best content when writing a test plan.

Test plan vs. strategy
Before proceeding, two common methods for defining test plans need to be clarified:
  • Single test plan: Some projects have a single "test plan" that describes all implemented and planned testing for the project.
  • Single test strategy and many plans: Some projects have a "test strategy" document as well as many smaller "test plan" documents. Strategies typically cover the overall test approach and goals, while plans cover specific features or project updates.
Either of these may be embedded in and integrated with project design documents. Both of these methods work well, so choose whichever makes sense for your project. Generally speaking, stable projects benefit from a single plan, whereas rapidly changing projects are best served by infrequently changed strategies and frequently added plans.
For the purpose of this guide, I will refer to both test document types simply as "test plans”. If you have multiple documents, just apply the advice below to your document aggregation.

Content selection
A good approach to creating content for your test plan is to start by listing all questions that need answers. The lists below provide a comprehensive collection of important questions that may or may not apply to your project. Go through the lists and select all that apply. By answering these questions, you will form the contents for your test plan, and you should structure your plan around the chosen content in any format your team prefers. Be sure to balance the factors as mentioned above when making decisions.

Prerequisites
  • Do you need a test plan? If there is no project design document or a clear vision for the product, it may be too early to write a test plan.
  • Has testability been considered in the project design? Before a project gets too far into implementation, all scenarios must be designed as testable, preferably via automation. Both project design documents and test plans should comment on testability as needed.
  • Will you keep the plan up-to-date? If so, be careful about adding too much detail, otherwise it may be difficult to maintain the plan.
  • Does this quality effort overlap with other teams? If so, how have you deduplicated the work?

Risk
  • Are there any significant project risks, and how will you mitigate them? Consider:
    • Injury to people or animals
    • Security and integrity of user data
    • User privacy
    • Security of company systems
    • Hardware or property damage
    • Legal and compliance issues
    • Exposure of confidential or sensitive data
    • Data loss or corruption
    • Revenue loss
    • Unrecoverable scenarios
    • SLAs
    • Performance requirements
    • Misinforming users
    • Impact to other projects
    • Impact from other projects
    • Impact to company’s public image
    • Loss of productivity
  • What are the project’s technical vulnerabilities? Consider:
    • Features or components known to be hacky, fragile, or in great need of refactoring
    • Dependencies or platforms that frequently cause issues
    • Possibility for users to cause harm to the system
    • Trends seen in past issues

Coverage
  • What does the test surface look like? Is it a simple library with one method, or a multi-platform client-server stateful system with a combinatorial explosion of use cases? Describe the design and architecture of the system in a way that highlights possible points of failure.
  • What are the features? Consider making a summary list of all features and describe how certain categories of features will be tested.
  • What will not be tested? No test suite covers every possibility. It’s best to be up-front about this and provide rationale for not testing certain cases. Examples: low risk areas that are a low priority, complex cases that are a low priority, areas covered by other teams, features not ready for testing, etc. 
  • What is covered by unit (small), integration (medium), and system (large) tests? Always test as much as possible in smaller tests, leaving fewer cases for larger tests. Describe how certain categories of test cases are best tested by each test size and provide rationale.
  • What will be tested manually vs. automated? When feasible and cost-effective, automation is usually best. Many projects can automate all testing. However, there may be good reasons to choose manual testing. Describe the types of cases that will be tested manually and provide rationale.
  • How are you covering each test category? Consider:
  • Will you use static and/or dynamic analysis tools? Both static analysis tools and dynamic analysis tools can find problems that are hard to catch in reviews and testing, so consider using them.
  • How will system components and dependencies be stubbed, mocked, faked, staged, or used normally during testing? There are good reasons to do each of these, and they each have a unique impact on coverage.
  • What builds are your tests running against? Are tests running against a build from HEAD (aka tip), a staged build, and/or a release candidate? If only from HEAD, how will you test release build cherry picks (selection of individual changelists for a release) and system configuration changes not normally seen by builds from HEAD?
  • What kind of testing will be done outside of your team? Examples:
    • Dogfooding
    • External crowdsource testing
    • Public alpha/beta versions (how will they be tested before releasing?)
    • External trusted testers
  • How are data migrations tested? You may need special testing to compare before and after migration results.
  • Do you need to be concerned with backward compatibility? You may own previously distributed clients or there may be other systems that depend on your system’s protocol, configuration, features, and behavior.
  • Do you need to test upgrade scenarios for server/client/device software or dependencies/platforms/APIs that the software utilizes?
  • Do you have line coverage goals?

Tooling and Infrastructure
  • Do you need new test frameworks? If so, describe these or add design links in the plan.
  • Do you need a new test lab setup? If so, describe these or add design links in the plan.
  • If your project offers a service to other projects, are you providing test tools to those users? Consider providing mocks, fakes, and/or reliable staged servers for users trying to test their integration with your system.
  • For end-to-end testing, how will test infrastructure, systems under test, and other dependencies be managed? How will they be deployed? How will persistence be set-up/torn-down? How will you handle required migrations from one datacenter to another?
  • Do you need tools to help debug system or test failures? You may be able to use existing tools, or you may need to develop new ones.

Process
  • Are there test schedule requirements? What time commitments have been made, which tests will be in place (or test feedback provided) by what dates? Are some tests important to deliver before others?
  • How are builds and tests run continuously? Most small tests will be run by continuous integration tools, but large tests may need a different approach. Alternatively, you may opt for running large tests as-needed. 
  • How will build and test results be reported and monitored?
    • Do you have a team rotation to monitor continuous integration?
    • Large tests might require monitoring by someone with expertise.
    • Do you need a dashboard for test results and other project health indicators?
    • Who will get email alerts and how?
    • Will the person monitoring tests simply use verbal communication to the team?
  • How are tests used when releasing?
    • Are they run explicitly against the release candidate, or does the release process depend only on continuous test results? 
    • If system components and dependencies are released independently, are tests run for each type of release? 
    • Will a "release blocker" bug stop the release manager(s) from actually releasing? Is there an agreement on what are the release blocking criteria?
    • When performing canary releases (aka % rollouts), how will progress be monitored and tested?
  • How will external users report bugs? Consider feedback links or other similar tools to collect and cluster reports.
  • How does bug triage work? Consider labels or categories for bugs in order for them to land in a triage bucket. Also make sure the teams responsible for filing and or creating the bug report template are aware of this. Are you using one bug tracker or do you need to setup some automatic or manual import routine?
  • Do you have a policy for submitting new tests before closing bugs that could have been caught?
  • How are tests used for unsubmitted changes? If anyone can run all tests against any experimental build (a good thing), consider providing a howto.
  • How can team members create and/or debug tests? Consider providing a howto.

Utility
  • Who are the test plan readers? Some test plans are only read by a few people, while others are read by many. At a minimum, you should consider getting a review from all stakeholders (project managers, tech leads, feature owners). When writing the plan, be sure to understand the expected readers, provide them with enough background to understand the plan, and answer all questions you think they will have - even if your answer is that you don’t have an answer yet. Also consider adding contacts for the test plan, so any reader can get more information.
  • How can readers review the actual test cases? Manual cases might be in a test case management tool, in a separate document, or included in the test plan. Consider providing links to directories containing automated test cases.
  • Do you need traceability between requirements, features, and tests?
  • Do you have any general product health or quality goals and how will you measure success? Consider:
    • Release cadence
    • Number of bugs caught by users in production
    • Number of bugs caught in release testing
    • Number of open bugs over time
    • Code coverage
    • Cost of manual testing
    • Difficulty of creating new tests


Categories: Testing & QA

Introducing Structurizr Express

Coding the Architecture - Simon Brown - Mon, 06/06/2016 - 13:35

I rolled out a new feature to Structurizr at the weekend called Structurizr Express, which is basically a way to create software architecture diagrams using text. Although the core concept behind Structurizr is to create a software architecture model using code, there are times when you simply want a quick diagram, perhaps for a presentation, pre-sales proposal, etc. Structurizr Express will let you do just that - quickly create a single software architecture diagram using a textual definition. Much like tools such as PlantUML, yUML, WebSequenceDiagrams, etc.

Structurizr Express

Despite the name, this is all still based around the C4 model although it only targets one diagram at a time. The three types of diagrams currently supported are System Context, Container and Component diagrams. Structurizr Express is available to use now and the help page provides a description and examples of the syntax. I hope you find it useful.

Categories: Architecture

Quote of the Month June 2016

From the Editor of Methods & Tools - Mon, 06/06/2016 - 12:56
The only software which doesn’t change is dead software. Stopping software from changing is a fast way to kill it. Treating software development as a temporary endeavour kills it. Regarding software as temporary also means the teams which create it get regarded as temporary. Such teams get disbanded at “the end”. The developers scatter to […]

Cumulative Flow Diagrams – Figures for SPaMCAST 397

Software Process and Measurement Cast - Sun, 06/05/2016 - 22:10

The Software Process and Measurement Cast 397 features our essay on cumulative flow diagrams.  The following file includes all of the figures referenced in the cumulative flow diagrams  essay which is part of SPaMCAST 397.

Categories: Process Management

Cumulative Flow Diagrams – Figures for SPaMCAST 397

Software Process and Measurement Cast - Sun, 06/05/2016 - 22:10

The Software Process and Measurement Cast 397 features our essay on cumulative flow diagrams.  The following file includes all of the figures referenced in the cumulative flow diagrams  essay which is part of SPaMCAST 397.

Categories: Process Management

SPaMCAST 397 – Cumulative Flow Diagrams, QA Sign, Project Strategy

SPaMCAST Logo

http://www.spamcast.net

Listen Now

Subscribe on iTunes                   Check out the podcast on Google Play Music

The Software Process and Measurement Cast 397 features our essay on cumulative flow diagrams.  A CFD can help everyone from team members to program managers to gain insight into issues, cycle time and likely completion dates. Cumulative flow diagrams are extremely versatile tools for managing work.

Download the figures that support this essay.

Our second column is a visit to the QA Corner. Jeremy Berriault weighs in on the thorny question of who signs off or approves the results of testing for projects. We discuss some strange behaviors that occur when responsibility and authority for the results of testing are ambiguous.

We also have the debut column from Jon M. Quigley. Jon inaugurates his column with a discussion of whether project risk, scope, and strategy are related.  The short answer is yes, and the longer answer suggests what happens when all of the options are not considered.  Jon is a principle at Value Transformation, LLC (www.valuetransform.com) along with being a teacher, coach, serial author and past guest on SPaMCAST 346.

Re-Read Saturday News

We continue the read of Commitment – Novel About Managing Project Risk by Maassen, Matts, and Geary.  Buy your copy today and read along (use the link to support the podcast). This week’s installment will address Chapter 7, which using the monomyth structure represents both atonement and the return home, the completion of the cycle.  Next week we will conclude with a few final thoughts.  The next book in the Re-read Saturday Series will be Kent Beck’s xP Explained, Second Edition. Visit the Software Process and Measurement Blog (www.tcagley.wordpress.com) to catch up on past installments of Re-Read Saturday.

Next SPaMCAST

The next Software Process and Measurement Cast features our interview of with bestselling author Kevin Kruse.  We discussed his new book, 15 Secrets Successful People Know About Time Management.  The ideas on managing time and more accurately managing focus are extremely useful and in some cases just a bit controversial.  Listen to the interview, then buy a copy of his book and become more productive!

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.


Categories: Process Management

SPaMCAST 397 – Cumulative Flow Diagrams, QA Sign Off, Project Strategy

Software Process and Measurement Cast - Sun, 06/05/2016 - 22:00

The Software Process and Measurement Cast 397 features our essay on cumulative flow diagrams. A CFD can help everyone from team members to program managers to gain insight into issues, cycle time and likely completion dates. Cumulative flow diagrams are extremely versatile tools for managing work.

Figures for this essay are included as a separate entry in the feed.

Our second column is a visit to the QA Corner. Jeremy Berriault weighs in on the thorny question of who signs off or approves the results of testing for projects. We discuss some strange behaviors that occur when responsibility and authority for the results of testing are ambiguous.

We also have the debut column from Jon M. Quigley. Jon inaugurates his column with a discussion of whether project risk, scope and strategy are related. The short answer is yes, and the longer answer suggests what happens when all of the options are not considered. Jon is a principle at Value Transformation, LLC (www.valuetransform.com) along with being a teacher, coach, serial author and past guest on SPaMCAST 346.

Re-Read Saturday News
We continue the read of Commitment – Novel About Managing Project Risk by Maassen, Matts, and Geary. Buy your copy today and read along (use the link to support the podcast). This week’s installment will address Chapter 7, which using the monomyth structure represents both atonement and the return home, the completion of the cycle. Next week we will conclude with a few final thoughts. The next book in the Re-read Saturday Series will be Kent Beck’s xP Explained, Second Edition. Visit the Software Process and Measurement Blog (www.tcagley.wordpress.com) to catch up on past installments of Re-Read Saturday.


Next SPaMCAST
The next Software Process and Measurement Cast features our interview of with bestselling author Kevin Kruse. We discussed his new book, 15 Secrets Successful People Know About Time Management. The ideas on managing time and more accurately managing focus are extremely useful and in some cases just a bit controversial. Listen to the interview, then buy a copy of his book and become more productive!

Shameless Ad for my book!
Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

Categories: Process Management

SPaMCAST 397 – Cumulative Flow Diagrams, QA Sign Off, Project Strategy

Software Process and Measurement Cast - Sun, 06/05/2016 - 22:00

The Software Process and Measurement Cast 397 features our essay on cumulative flow diagrams. A CFD can help everyone from team members to program managers to gain insight into issues, cycle time and likely completion dates. Cumulative flow diagrams are extremely versatile tools for managing work.

Figures for this essay are included as a separate entry in the feed.

Our second column is a visit to the QA Corner. Jeremy Berriault weighs in on the thorny question of who signs off or approves the results of testing for projects. We discuss some strange behaviors that occur when responsibility and authority for the results of testing are ambiguous.

We also have the debut column from Jon M. Quigley. Jon inaugurates his column with a discussion of whether project risk, scope and strategy are related. The short answer is yes, and the longer answer suggests what happens when all of the options are not considered. Jon is a principle at Value Transformation, LLC (www.valuetransform.com) along with being a teacher, coach, serial author and past guest on SPaMCAST 346.

Re-Read Saturday News
We continue the read of Commitment – Novel About Managing Project Risk by Maassen, Matts, and Geary. Buy your copy today and read along (use the link to support the podcast). This week’s installment will address Chapter 7, which using the monomyth structure represents both atonement and the return home, the completion of the cycle. Next week we will conclude with a few final thoughts. The next book in the Re-read Saturday Series will be Kent Beck’s xP Explained, Second Edition. Visit the Software Process and Measurement Blog (www.tcagley.wordpress.com) to catch up on past installments of Re-Read Saturday.


Next SPaMCAST
The next Software Process and Measurement Cast features our interview of with bestselling author Kevin Kruse. We discussed his new book, 15 Secrets Successful People Know About Time Management. The ideas on managing time and more accurately managing focus are extremely useful and in some cases just a bit controversial. Listen to the interview, then buy a copy of his book and become more productive!

Shameless Ad for my book!
Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

Categories: Process Management

Re-Read Saturday: Commitment – Novel about Managing Project Risk, Part 6

Picture of the book cover

This week’s installment of Commitment – Novel about Managing Project Risk by Olav Maassen, Chris Matts and Chris Geary (2nd edition, 2016) will address Chapter 7, which using the monomyth structure represents both atonement and the return home, the completion of the cycle.  Next week we will conclude with a few final thoughts.  The next book in the Re-read Saturday Series will be Kent Beck’s xP Explained, Second Edition.   

Chapter 7 begins when Duncan tracks Rose and her sister down to tell her that she may not have a job. The client pulled funding because it was too risky. Rose decides to tackle the problem head on and ropes her sister and Duncan into seeing if they can get things sorted out. All of this is occurring despite the fact that they were getting positive feedback on a weekly basis (later we find out that the client has not been implementing the new code rather waiting for a big bang, which IS risky).

When the trio gets to work, Rose gathers the team and does some scenario planning.  Scenario planning is a form of the real options approach proposed in Commitment. The team identifies four potential four scenarios for their situation. The first is that the work goes forward as normal. The second is the possibility that the company can salvage some of the work completed and use it for another client.  The third is that some of the people go to work with the “idiots” on the fourth floor (Duncan’s project).  The fourth scenario is scrapping everything and everyone is fired. Rose asks the team to split up into smaller teams of 2 to 3 to flesh out the scenarios so that everyone will know how to act if that scenario becomes a commitment. On a personal level, this type of scenario planning is deeply ingrained in my psyche. 

As with other points in the novel, after the introduction of a concept, a call out is used to provide exposition.  The call out is Lilly’s blogs entry on increasing your psychic odds using scenario planning.  The blog entry points out that scenario planning is another expression of real options. The scenario planning helps to reduce the angst of uncertainty without making a commitment (discussed in Chapter 6). The blog entry provides an example documented in Peter Senge’s book, The Fifth Discipline. The process generated communication and collaboration across siloed group and department lines, which did not happen normally. Scenario planning as a tool for risk management has positive benefits because people think through how they will act and react if the scenario occurs.  Scenario planning in Senge’s book occurred at an organizational level; however scenario planning is applicable at any scale.

The transition back to the action in the story begins with as Rose meets with the client and begins the meeting by explaining the impact of the ‘cancel the project’ scenario on the client.  Rose walks through a chart showing the two of the client’s competitors that have been updating their software monthly are taking market share from Rose’s client.   Rose’s client has not updated their software in over a year, even though they were getting weekly updates. Rose suggests that if the trend continues the client will have no market share. The meeting shows the risk issue as the client’s fear that if a change goes wrong they will lose market share even faster.  Rose points out that the weekly incremental change reduces risk and that because the change is incremental even if something goes wrong it will be “rolled back” quickly minimizing the impact.  The meeting ends with the client going off to reconsider (taking Rose’s phone number so they can go to the source to ask the questions). 

Skipping over the date with Duncan that includes a bad movie, piano playing, and dark apartment windows, we get to the decision.  The phone rings and everyone holds their collective breath waiting for Rose to answer.  The project is back on and after a brief non-celebration, they press forward to complete the project.

We next cut to the project retrospective and celebration.  Interrupting the end of project celebration, a group of suits enters and announces the sacking of Rose from the project at the request of the client.  The sacking from the project clears the way for her promotion to vice president. The scene provides a bit of melodrama for capping the chapter.

We will finish Commitment – Novel about Managing Project Risk next week with final thoughts and a word or two on the Epilogue.

Previous Installments:

Part 1 (Chapters 1 and 2)

Part 2 (Chapter 3)

Part 3 (Charter 4) 

Part 4 (Chapter 5)

Part 5(Chapter 6)


Categories: Process Management