Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
It is so easy to sit back and wait for things to get easier before you dive in and attempt to do them. You can’t start your fitness program or diet right now, because work is too hard or finances are too tight. You can’t start writing your book, because you just aren’t ready yet. […]
This week I spent more time hangout out with people, which means less time is available for my blog. Sorry about that! (Also, I have a book to finish…)
People are in need of different answers. They need answers to questions about how to implement better management with fewer managers. Creative networkers cannot rely on bosses to grow healthy organizations for them. They need to know how to manage things together.
I firmly believe that networked organizations own the future.
Recipes exist to get ideas, not to be followed to the letter‚Ä†
The recipe for a project is described by its¬†Capabilities. The technical and operational requirements need to support those capabilities, but defining those up front and following them to the letter restricts the creativity of the project team, just like it restricts the creativity of the chef.¬†
‚Ä† Epicurious web site comment on¬†Vichyssoise recipe
The most formal version of reviews and inspection are Fagan-style inspections. ¬†Inspections are very rigorous in removing defects from any type of deliverable. However, the cost and overhead of these inspections have led to the development of a wide range of less formal techniques.¬† The goal of the less formal techniques is find a balance between the benefit and cost of inspections. The least formal and most common review technique is the desk check.
Author ‚Äď The author is the person that created the deliverable being reviewed.¬† He or she sends the deliverable (or link) to the person doing the review, answers questions (if asked) and reacts to feedback.
Reviewer ‚Äď Reads the deliverable and provides feedback to the author.
Simple Desk Check Process:
The desk check process is very simple and very compact. All of my writings are desk checked by one or two editors. 99% percent of the time the review and my consumption of the information generated is asynchronous.¬† They edit the document and then at some point in the future I make corrections.¬† If my editors did not write down the feedback and just left a voice message I would never be able to remember the things I need to fix (it would also have to be a long message).¬† On the occasions that we are in the same location the process is far richer because we talk and make corrections interactively (this is the power of collaborative reviews).
There is almost always no pre-work for this type of review.
A second type of less formal review is the Walkthrough:
Author‚Äď Same as in the desk check, except when they act as the reader and scribe (see below)
Reader ‚Äď Reads or presents the deliverable being reviewed. Almost always the author.
Scribe ‚Äď Record comments and defects identified during the walk through.¬† Many times this role is filled by the author.
Reviewer ‚Äď Attends the walkthrough and provides feedback to the author. In more formal versions of a walkthrough, the reviewer may be asked to pre-read or pre-review the deliverable before the walk through.
A simple walkthrough process:
My brother owns a firm that builds custom houses. In his business the walkthrough is a critical quality and communication tool to ensure the customer and his build crews stay synchronized.
The desk check and the walkthrough are generally less effective than an inspection. However, they strike a good balance between degree of rigor and their ability to find and remove defects.¬† Rigor is highly correlated to cost and overhead. Informal and formal reviews and inspections can also be combined.¬† For example, I recently reviewed a process for an avionic firm that requires all designs and code to be desk checked before it is formally inspected. The firm‚Äôs goal is to remove 99.9+% of the defects prior to a production release on all life critical software. Just because a review technique is informal doesn‚Äôt mean that it is not valuable.
Every now and then in Cypher land we’ll end up with a collection of arrays, often created via the COLLECT function, that we want to squash down into one array.
For example let’s say we have the following array of arrays…
$ RETURN [[1,2,3], [4,5,6], [7,8,9]] AS result; ==> +---------------------------+ ==> | result | ==> +---------------------------+ ==> | [[1,2,3],[4,5,6],[7,8,9]] | ==> +---------------------------+ ==> 1 row
…and we want to return the array [1,2,3,4,5,6,7,8,9].
Many programming languages have a ‘flatten’ function and although cypher doesn’t we can make our own by using the REDUCE function:
$ WITH [[1,2,3], [4,5,6], [7,8,9]] AS result RETURN REDUCE(output = , r IN result | output + r) AS flat; ==> +---------------------+ ==> | flat | ==> +---------------------+ ==> | [1,2,3,4,5,6,7,8,9] | ==> +---------------------+ ==> 1 row
Here we’re passing the array ‘output’ over the collection and adding the individual arrays ([1,2,3], [4,5,6] and [7,8,9]) to that array as we iterate over the collection.
If we’re working with numbers in Neo4j 2.0.1 we’ll get this type exception with this version of the code:
==> SyntaxException: Type mismatch: expected Any, Collection<Any> or Collection<Collection<Any>> but was Integer (line 1, column 148)
We can easily work around that by coercing the type of ‘output’ like so:
WITH [[1,2,3], [4,5,6], [7,8,9]] AS result RETURN REDUCE(output = range(0,-1), r IN result | output + r);
Of course this is quite a simple example but we can handle more complicated scenarios as well by using nested calls to REDUCE. For example let’s say we wanted to completely flatten this array:
$ RETURN [[1,2,3], , [5, [6, 7]], [8,9]] AS result; ==> +-------------------------------+ ==> | result | ==> +-------------------------------+ ==> | [[1,2,3],,[5,[6,7]],[8,9]] | ==> +-------------------------------+ ==> 1 row
We could write the following cypher code:
$ WITH [[1,2,3], , [5, [6, 7]], [8,9]] AS result RETURN REDUCE(output = , r IN result | output + REDUCE(innerOutput = , innerR in r | innerOutput + innerR)) AS flat; ==> +---------------------+ ==> | flat | ==> +---------------------+ ==> | [1,2,3,4,5,6,7,8,9] | ==> +---------------------+ ==> 1 row
Here we have an outer REDUCE function which iterates over [1,2,3], , [5, [6,7]] and [8,9] and then an inner REDUCE function which iterates over those individual arrays.
If we had more nesting then we could just introduce another level of nesting!
How is it possible that a wooden Shinto shrine built in the 7th century is still standing? The answer depends on how you answer this philosophical head scratcher: With nearly every cell in your body continually being replaced, are you still the same person?
The Ise Grand Shrine has been in continuous existence for over 1300 years because every twenty years an exact replica has been rebuilt on an adjacent footprint. The former temple is then dismantled.
Now that's resilience. If you want something to last make it a living part of a culture. It's not so much the building that is remade, what is rebuilt and passed down from generation to generation is the meme that the shrine is important and worth preserving. The rest is an unfolding of that imperative.
You can see echoes of this same process in Open Source projects like Linux and the libraries and frameworks that get themselves reconstructed in each new environment.
The patterns of recurrence in software are the result of Darwinian selection process that keeps simplicity and value alive in human minds.
A blog post on Persuing Wabi has some fabulous photos of the shrine along with a brief description of why it's the way it is:
The common practice of starting with requirements leads to the common complaint that ¬†requirements change,¬†we don't know what we want yet, our users aren't very good at defining requirements so we'll let them emerge. While these are common, they are usually a symptom of a missing piece of information.
We don't know what ¬†capabilities ¬†are needed and what is the¬†Concept of Operations that those ¬†capabilities will implement, the project as likely failed before it starts. If we do know the¬†Capabilities and the¬†Concept of Operations, we can then measure progress of our work effort, not in the passage or time, consumption of resources (including money), or the production of¬†stories or ¬†story points (which are unit-less and therefore pretty much meaningless to those paying the our work), but in ¬† Measures of Effectiveness, Measures of Performance, and¬†Technical Performance Measures..
Concept of Operations
Let's start with a formal defininton of the¬†Concept of Operations
What this tells us is that we need to start with what DONE looks like. DONE is not a set of features. DONE is not stories or story points. DONE is not modules, databases, bent metal. DONE is the ability, the¬†capability to do something of value in echnage for the money we've spent.
The assessment of a¬†capability is it's Measure of Effectiveness. These are¬†operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of conditions.¬†We need to define these upfront. The Measures of Effectiveness:
They are not emergement. They are descriptions of¬†success. When we treat them as emergent, our project is chasing a moving target and is headed to the ditch.
Next are Measures of Performance. They ¬†characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions.¬†The Measures of Performance are:
Next comes the Technical Performance Measures. These are attributes ¬†that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal. The Technical Performance Measures:
Notice we have not mentioned coding, development methods like Scrum or XP, teams, paired programming of anything to so with building code. With these items in place, all those activities have no reason for being, other than to consume money and pass time. None of those items having anything with moving the project toward DONE, other spend money and pass time. Oh, you'll get a pile of stories implemented. Are they the right stories? How would you know. You'll perform lots of Test Driven Design. Is is the right design. How would you know?¬†
Oh your customer is going to prioritize those stories and features. How are they going to know in the absence of knowing what DONE looks like.
Capabilities Based Planning
This has been presentde before, but now it has a reason - the¬†Concept of Operations.
Capabilities based planning (v2) from Glen Alleman With the ConOps and CBP, we now have what we need to assess the development processes.¬†
We Know the Answer To That Rights?
Inspections are peer reviews by trained individuals using a formal, structured method. Inspections represent the GOLD standard for reviews.¬† According to Capers Jones, inspections are highly effective in removing over 97% of requirements defects. Data that I have collected supports the view that inspections are the most effective means of early defect removal. The issue is that very few people do inspections for two basic reasons.¬† The first is that inspections are expensive (they require time and effort) and inspections are uncomfortable for those being reviewed. Inspections require specific set of roles and a specific process.
Moderator ‚Äď The moderator leads the team through the process ensuring everyone participates and follows the rules.¬† The moderator will also ensure that metrics are collected and that the defects that are identified are correct and recorded.
Author ‚Äď The author is the person that created the deliverable being reviewed.¬† He or she will participate in the inspection, providing feedback and developing an understanding of the defects being identified.
Reviewers ‚Äď Reviewers read the deliverable and identify defects BEFORE the inspection meeting.¬† The pre-work is critical and needs to be turned into the scribe before the inspections meeting.
Scribe ‚Äď The scribe insures that all participants have provided their feedback before the inspection team meets.¬† The scribe also combines all of the feedback to ensure that only unique items are discussed in the inspection meeting.¬† During the inspection meeting, the scribe records feedback on the defects identified during the pre-work ¬†and records any new defects that are identified during the meeting. ¬†After the meeting they keep track of the defects to make sure they are tracked to completion. The data collected by the scribe feeds the measures and metrics.
Inspections are very effective if the team doing the inspection understands the process and the deliverable being reviewed.¬† For example, if you are doing a code review on a program written in Ruby don‚Äôt gather a group of reviewers that have never coded in Ruby.¬† They are not qualified. Everyone involved in the process needs to trust each other. All reviews, in general, and inspections (because they very formal), in specific, can be very stressful for the author. The level of stress can cause authors to avoid inspections or short cut the process which means that defects will be found later in the development process or in production.¬†¬†Inspections remove defects in code or any other deliverable before they get into production or before lots of time is spent on building something that will need to be changed before it goes into production.
Managers all over the world are faced with a critical challenge to their role. They ideas about management and their management style is being challenged. And this is even more important because many managers have reached a position of in their career where they thought they could "take it easy". Nothing could be further from the truth.
Today the role of managers in all industries is shifting. And in no industry more than the knowledge industry.In this video I explore why this is happening and where we may be able to look for solutions. I also present a concrete set of consequences that will affect you as a manager from the trends we are witnessing in the knowledge industry. Do you want to know more?
Sign-up here to read a white paper I wrote about the sources of disruption for managers and management in general. In it I explore where the key threats to the current management roles are coming from.Ready to explore what you as a manager can learn from The Science of Chaos?You came to the right place! :) Mystes in Finland organizes a workshop about Chaos Science applied to the challenges of managing small and large knowledge work organizations. You can visit their site to know more about the workshop and to sign up. Places are limited. In that workshop I will touch on the following topics:
Much of the discussion around making improvements in processes fails to address the governance aspects of a business or organization. Instead it focuses on the¬†personal aspects. Agile development of a software team, without the corporate impacts. The desire to¬†stop doing something without an actual replacement, under the guise of¬†we're exploring. Our the mention of the term¬†intent of the commander without understanding that filling out that intent requires complete capabilities to act in in the absence of direct supervision.
My project management maturity was changed forever at Rocky Flats, under the management of senior leaders with experience and skill formed in the US Navy. The book¬†Making the Impossible Possible is the story of that project and the learnings that can be applied in any high risk, complex, high reward domain.
Making the impossible possible from Glen Alleman The term¬†Intent of the Commander is actualized in the concept of the¬†Plan of the Day. Here's an actual plan of the day for shipboard life. This notion of¬†plan of the day was applied late in the Rocky Flats program. Before that ¬†we had¬†Plan of the Week and then Plan of the Month. The approach is very simple. What do we plan to get done at the end of this period (day, week, month? Write that down, establish some measures of performance and effectiveness for those ourcomes. Measure them, take corrective actions if they don't match the expectations. Repeat until done.
The following was originally published in Mike Cohn's monthly newsletter. If you like what you're reading, sign up to have this content delivered to your inbox weeks before it's posted on the blog, here.
I'm a child of the '60s, but only in the literal sense. I was born in 1962, so while I was alive during much of the excitement of that decade, I spent much of the decade playing "cowboys and Indians" and other non-politically correct games of the era. But I grew up in Southern California, and the vibe of the '60s lasted well into the '70s there. Further, I had a fair number of friends who had older brothers, some of whom I met only occasionally, but who were very much part of the hippie counterculture.
I remember one of these brothers-of-a-friend quite clearly. When I was 10 years old, he showed me a book called Be Here Now. Even then, I wanted to be a writer and this book really struck me as I paged through it. First, the book was square--books weren't supposed to be square unless they were picture books for babies. This book was for adults. At least I thought so. But it wasn't typeset like a normal book--it looked more like a poison pen letter; you know, like a ransom note with the text cut from newspaper headlines to make new words. There were drawings, but not drawings relegated to nice little boxes on the page. The drawings and the text were jumbled together.
It wasn't until I was 18 though that I actually read the book. I found a copy at my college library and read it. The New York Times called the book a "counterculture bible." And it was as common in the 1970s to see copies of that book around as it is today for programmers to have the Gang of Four patterns book.
I can't say the book's contents had any influence on me. It's a guide to Hinduism for Westerners who wish to become yogis. Not my thing. But the title of that book has always resonated with me: Be Here Now.
I think the ability to be here now is something too many of us are losing. We can't just be in the moment and in the place. Everyone has to be constantly on their cell phones, or checking email or Facebook on their laptops while supposedly making progress on something else. (I will admit to having paused once while writing this to investigate the bonk of a new email arriving. But I've so far withheld the temptation to look a second time.)
I witness the inability to be here now while training or doing consulting. I remember a particular client a couple of years ago where I was teaching a Certified ScrumMaster class. The boss had warned me that people there were busy so they'd have their laptops with them if needed. I spent much of the first day unable to make eye contact with a single person in the room. Not one person was truly in that room.
I really worry about this. I have daughters who are 14 and 18, and I often think about the work world they'll be entering. My wife and I have tried to instill in them the hippie mindset to be here now. Will others in their future companies share that? What will the short attention spans we are cultivating do to innovation? To productivity? To collaboration?
In the true spirit of the '60s, I'm going to leave this philosophical rather than trying to identify three steps agile teams can do to overcome short attention spans. I want to be no more practical than to ask each of you to be here now a bit more often, a bit longer, and a bit more intensely each day.
When we hear about all the methods of managing projects, the PMI Body of Knowledge, PRINCE2, home grown and commercial ¬†solutions - always look at them in the light of these 5 Immutable Principles and the 5 Practices then implement the principles.
Principles and Practices of Performance-Based Project Management¬ģ from Glen Alleman
Reviews and inspections are an integral part of building most everything. You find reviews and inspections in manufacturing, in construction and even in publishing. Software development and maintenance is no different. Reviews and inspections can be powerful tools to remove defects before they can impact production and to share knowledge with the team and stakeholders. They are part of a class of verification and validation techniques called static techniques. These techniques are considered static because the system or application being built is not executed. Instead of executing the code, the code or other written deliverables are examined either by people (generally called reviews and inspections) or mechanically by a tool (known as static analysis).¬† Reviews and inspections can be applied to any product generated as part of the development process, while static analysis can only be applied to code-based products.
Reviews and inspections come in various levels of formality to meet different situations and needs. Informal reviews typically do not follow a detailed written process and results are generally not documented for later review and analysis. They are often used to ensure knowledge sharing and training at a one-on-one level. One classic method used for informal reviews is called the desk check.¬† In a desk check, a team member emails a deliverable or code to another team member who reviews it and gives them feedback. Another form of informal review is pair programing.
Walkthroughs are a step up the formality ladder.¬† Walkthroughs are group sessions in which the author takes the group he or she is presenting to¬†through the deliverable. ¬†The attendees of walkthroughs generally include ¬†team members and technical specialists.¬†Walkthroughs can be very informal (an impromptu gathering) or the degree of formality can be increased by requiring meeting preparation and collection of¬†issues and defects. Walkthroughs are used to discover defects, make decisions and to distribute information.
Technical reviews leverage a defined process for defect detection, and include participation by peers, technical experts and often management personnel. They are more formal than the typical walkthrough and are much more formal than desk checks. A trained moderator, who is not the author, generally leads a technical review (to enforce independence) comparing the deliverable to organizational standards. In addition to defect discovery, decision making and information distribution, technical reviews are often used as a formal approval mechanism. For example, I recently observed an organization where all projects go through an architectural review. Technical reviews are a type of technical review usually based on defined organizational standards. The architecture review in the example was based on the organization‚Äôs published standard architecture.
Inspections are the most formal of the review and inspection techniques. The most well-known inspection technique is based on the process defined by Michael Fagan of Fagan Reviews.¬† The inspection process includes highly defined roles such as moderator, author, scribe and reviewer. All inspection processes typically include required pre-work, logging of defects, collection and publication of metrics, formal follow-up procedures and, in many cases, the use of statistical process control techniques. The goal of inspections is to find and remove defects.
Reviews and inspections are highly effective and powerful tools for finding and removing defects from software and other deliverables. Review and inspections are used in all software development and maintenance methods. The type of review and degree of formality is usually a function of the type of project. For example, inspections are almost always used on mission critical applications, such as medical devices and weapons systems, regardless of whether they are using Agile or plan-based techniques. Reviews and inspections remove defects and share knowledge so teams can maximize the value they deliver.