Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator&page=3' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

Quote of the Day

Herding Cats - Glen Alleman - Mon, 10/03/2016 - 16:33

One of the great mistakes is to judge policies and programs by their intentions rather than their results - Milton Friedman

When we hear any suggested corrective action for any supposed dysfunction, ask can you show me the testable results from your suggested change to the problem?

Related articles Doing the Math Don't Manage By Quoting Dilbert Estimating and Making Decisions in Presence of Uncertainty
Categories: Project Management

SPaMCAST 413 - Scaling Management, Throughput Accounting, QA Tools

Software Process and Measurement Cast - Mon, 10/03/2016 - 09:18

The Software Process and Measurement Cast 413 features our essay on Scaling Agile and Management Styles.  This essay builds on our recent discussion of servant leadership. It is not as simple as adding teams or building a hierarchy.

Steve Tendon joins the SPaMCAST this week to discuss Chapter 11 in Tame The Flow: Hyper-Productive Knowledge-Work Performance, The TameFlow Approach and Its Application to Scrum and Kanban, published J Ross (buy a copy here).   We discussed the concept of throughput accounting.  A powerful concept that that focuses on the delivery of value through the overall process. Visit Steve at www.tendon.net.

We cap this edition of the Software Process and Measurement Cast with a visit to the QA Corner with Jeremy Berriault. Jeremy discussed the impact of testing tools. There are significant plusses for using tools if you don’t let the tools use you! Connect with Jeremy on Linkedin.


Re-Read Saturday News

We continue the read/re-read of The Five Dysfunctions of a Team by Patrick Lencioni (published by Jossey-Bass).  The Five Dysfunctions of a Team is a business novel that uses a story to get important ideas across to the reader in a less threatening manner. This week we discuss the sections covering the background for the first major team crisis. Lots of behaviors you might, unfortunately, recognize in teams around you!

Visit the Software Process and Measurement Cast blog to participate in this and previous re-reads.


The Software Process and Measurement Cast 414 will feature our interview with Marcus Hammarberg. We often think of Agile as a tool to build or maintain software.  Marcus used Agile techniques to save a clinic and to change lives.  It is an amazing and uplifting story.


Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

Categories: Process Management

SPaMCAST 413 – Scaling Management, Throughput Accounting, QA Tools



Listen Now
Subscribe on iTunes
Check out the podcast on Google Play Music

The Software Process and Measurement Cast 413 features our essay on Scaling Agile and Management Styles.  This essay builds on our recent discussion of servant leadership. It is not as simple as adding teams or building a hierarchy.

Steve Tendon joins the SPaMCAST this week to discuss Chapter 11 in Tame The Flow: Hyper-Productive Knowledge-Work Performance, The TameFlow Approach and Its Application to Scrum and Kanban, published J Ross (buy a copy here).   We discussed the concept of throughput accounting.  A powerful concept that that focuses on the delivery of value through the overall process. Visit Steve at www.tendon.net.

We cap this edition of the Software Process and Measurement Cast with a visit to the QA Corner with Jeremy Berriault. Jeremy discussed the impact of testing tools. There are significant plusses for using tools if you don’t let the tools use you! Connect with Jeremy on Linkedin.

Re-Read Saturday News

We continue the read/re-read of The Five Dysfunctions of a Team by Patrick Lencioni (published by Jossey-Bass).  The Five Dysfunctions of a Team is a business novel that uses a story to get important ideas across to the reader in a less threatening manner. This week we discuss the sections covering the background for the first major team crisis. Lots of behaviors you might, unfortunately, recognize in teams around you!

Visit the Software Process and Measurement Cast blog to participate in this and previous re-reads.


The Software Process and Measurement Cast 414 will feature our interview with Marcus Hammarberg. We often think of Agile as a tool to build or maintain software.  Marcus used Agile techniques to save a clinic and to change lives.  It is an amazing and uplifting story.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.


Categories: Process Management

SPaMCAST 413 - Scaling Management, Throughput Accounting, QA Tools

Software Process and Measurement Cast - Sun, 10/02/2016 - 22:00

The Software Process and Measurement Cast 413 features our essay on Scaling Agile and Management Styles.  This essay builds on our recent discussion of servant leadership. It is not as simple as adding teams or building a hierarchy.

Steve Tendon joins the SPaMCAST this week to discuss Chapter 11 in Tame The Flow: Hyper-Productive Knowledge-Work Performance, The TameFlow Approach and Its Application to Scrum and Kanban, published J Ross (buy a copy here).   We discussed the concept of throughput accounting.  A powerful concept that that focuses on the delivery of value through the overall process. Visit Steve at www.tendon.net.

We cap this edition of the Software Process and Measurement Cast with a visit to the QA Corner with Jeremy Berriault. Jeremy discussed the impact of testing tools. There are significant plusses for using tools if you don’t let the tools use you! Connect with Jeremy on Linkedin.

Re-Read Saturday News

We continue the read/re-read of The Five Dysfunctions of a Team by Patrick Lencioni (published by Jossey-Bass).  The Five Dysfunctions of a Team is a business novel that uses a story to get important ideas across to the reader in a less threatening manner. This week we discuss the sections covering the background for the first major team crisis. Lots of behaviors you might, unfortunately, recognize in teams around you!

Visit the Software Process and Measurement Cast blog to participate in this and previous re-reads.


The Software Process and Measurement Cast 414 will feature our interview with Marcus Hammarberg. We often think of Agile as a tool to build or maintain software.  Marcus used Agile techniques to save a clinic and to change lives.  It is an amazing and uplifting story.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

Categories: Process Management

The Long Forgotten Nine Best Practices of Project Success

Herding Cats - Glen Alleman - Sun, 10/02/2016 - 18:00

There are Nine Best Practices for success in enterprise Software Development. Here's a paper describing them and their use.

Nine Best Practices from Glen Alleman Related articles Strategy is Not the Same as Operational Effectiveness What Can Lean Learn From Systems Engineering? An Example of Complete Misunderstanding of Project Work
Categories: Project Management

Neo4j: Procedure call inside a query does not support passing arguments implicitly (pass explicitly after procedure name instead)

Mark Needham - Sun, 10/02/2016 - 11:13

A couple of days I was trying to write a Cypher query to filter the labels in my database.

I started with the following procedure call to get the list of all the labels:

CALL db.labels
│label     │
│Airport   │
│Flight    │
│Airline   │
│Movie     │
│Person    │
│Engineer  │

I was only interested in labels that contained the letter ‘a’ so I tweaked the query to filter the output of the procedure:

CALL db.labels
YIELD label 
WITH label WHERE tolower(label) contains "a"
RETURN label

Unfortunately that didn’t work as I expected:

Procedure call inside a query does not support passing arguments implicitly (pass explicitly after procedure name instead) (line 1, column 9 (offset: 8))
"CALL db.labels"

The mistake I made was calling the procedure implicitly without using parentheses. If you want to do any post processing on the output of a procedure you need to call it explicitly otherwise Cypher gets very confused.

If we add back the parentheses it’s much happier:

CALL db.labels()
YIELD label 
WITH label WHERE tolower(label) contains "a"
RETURN label
│label     │
│Airport   │
│Airline   │

It stumped me for a while until I figured out what the error message meant! I think I’ll just use explicit parentheses all the time from now on to save me running into this one again.

Categories: Programming

Five Dysfunctions of a Team, Patrick Lencioni:  Re-Read Week 2

The Five Dysfunctions of a Team Cover

The “Book” during unboxing!

Today we continue our re-read of  The Five Dysfunctions of a Team by Patrick Lencioni (Jossey-Bass, Copyright 2002, 33rd printing). This is a business novel that highlights the dysfunctions of teams that reduce their effectiveness.  Teams are one of the most ubiquitous features of all organizations (once you get past solopreneurs); therefore problems in teams directly translate to problems in organizations.  If you do not have a copy of the book, please buy a copy from the link above and read along.  Today I begin by exploring the different personalities on “The Staff” (DecisionTech’s executive staff) and conclude with the run up to the off-site meeting in Napa which exposes some of the team’s problems that are hurting DecisionTech.

The Staff

The executive team at DecisionTech is known as “The Staff”. The nickname was a veiled reference to the perception that the executives did not act as a team.  This perception was not lost on Kathryn, the newly appointed CEO. The executive team is the canvas upon which Lencioni paints the five dysfunctions of a team.  The members of The Staff are:

  • Former CEO – Jeff Stanley is the former CEO that has stayed on as the business development lead.  Jeff  is described as knowing Silicon Valley and being a networking maven. Jeff had raised most the early money for the firm and hired most of the executives.  He was a force within the organization. Kathryn feels that he was not a manager, but that his heart was in the right place.
  • Marketing  – Mikey is that brand-building genius ,but “lacked a few key social graces.” Kathryn feels that Mikey is unaware that people saw her rough edges and that how that negatively impacted her effectiveness.
  • Chief Technologist – Martin is the person that sits in the meeting with his laptop open apparently not paying attention, but providing a sarcastic commentary. Martin’s style of interaction is wearing on his peers and staff (the rank and file) of the organization. I have known many a Martin in my career; technically brilliant but a wearing on the organization’s’ morale.
  • Sales – JR is the guy that always says ‘yes’ to everything that is needed, but rarely follows through. Whenever anyone calls him on dropping the ball he apologizes profusely. JR’s saving grace is the long track record of sales success before he came to DecisionTech.
  • Customer Support – Carlos is solid. He listens, he participates when needed, He is self-deprecating, always saying something constructive while downplaying his prior accomplishments.  Carlos works long hours and is a solid guy. Kathryn doesn’t believe she will have to worry about Carlos. 
  • CFO – Jen is the money person and is a “stickler” for detail.  Jan raised a significant amount of the money needed to get the company started. Her fiscal discipline allowed the board to give the rest of the staff free reign because Jan would keep them under control.
  • COO – Nick had a great resume, however, the COO role is ill-defined.  The lack of role definition led Nick to be frustrated because he did not view his work a very meaningful.  He kept the frustration bottled up.  On top of the nebulous role, Nick felt he was the only one in the firm qualified to be CEO and that his colleagues were inferior to him.

On the surface, members of “The Staff” seem to present most possible problems that a team could face. In Part Two the problems begin to surface.

Part Two

Part Two begins to present the problems that indicate that members of The Staff were not working well together. Remember that in week 1 Kathryn announced that after observing she was going to hold a series of offsite meetings to set a direction. The offsite meetings were a cornerstone of Kathryn’s change process. Early in my career, a subsidiary of a firm I worked for brought in a turnaround specialist.  The first day he required everyone in the company to physically move their desk from one side the space they were into the other.  His goal was to send a message that from then on, nothing would be the same. I view the offsite in the same vein.

First Test

The first direct test of Kathryn’s authority came in the form of an email from Martin.  He indicated that he had gotten a great opportunity to sell DecisionTech’s product to another firm and would be away from the office for a few days.  He did not acknowledge that he would have to miss part of the offsite, exacerbating the clash with Kathryn. Instead of responding by email Kathryn met with Martin face-to-face and delivered the message that he should reschedule and not miss the meeting.  After a bit of back and forth, Martin decided to stop fighting but not to drop the issue.  The first test highlighted passive aggressive behavior both in terms of Martin not addressing the conflict and then pretending to agree but in reality not to drop this issue.

End Run

Martin leveraged the old-boys network and cornered Jeff in the parking lot to enlist his help. Jeff asked Kathryn to lunch in order to broach the subject of the value of Martin attending a customer meeting versus Kathryn’s internal meeting. Kathryn sent a message by stating she was OK with Jeff disagreeing and discussing it face-to-face; however, she was in charge of the organization now. In a similar manner instead of directly dealing with the conflict, Jeff (like Martin) disengaged to “fight again another day”. Jeff’s behavior still ended in a passive aggressive manner.  Kathryn knew that The Staff was very broken.

The End Run sets the stage for the Napa offsite, which we will address next week.  However, through the combination of the staff descriptions and the first two tests of her leadership Lencioni paints the picture of a highly dysfunctional team and foreshadows significant headaches.

Three quick take-ups:

  1.   Email is not a great tool to use when disagreeing with your boss.
  2.   Not dealing with issues when they occur leads to passive aggressive behavior.
  3.   Hiding behind a computer in meetings and sniping is not a great way to win friends and influence people. 

Previous Installments in the re-read of  The Five Dysfunctions of a Team by Patrick Lencioni:

Week 1 – Introduction through Observations


Categories: Process Management

Accountability for What You Say is Dangerous and That’s Okay

James Bach’s Blog - Sat, 10/01/2016 - 20:33

[Note: I offered Maaret Pyhäjärvi the right to review this post and suggest edits to it before I published it. She declined.]

A few days ago I was keynoting at the New Testing Conference, in New York City, and I used a slide that has offended some people on Twitter. This blog post is intended to explore that and hopefully improve the chances that if you think I’m a bad guy, you are thinking that for the right reasons and not making a mistake. It’s never fun for me to be a part of something that brings pain to other people. I believe my actions were correct, yet still I am sorry that I caused Maaret hurt, and I will try to think of ways to confer better in the future.

Here’s the theme of this post: Getting up in front of the world to speak your mind is a dangerous process. You will be misunderstood, and that will feel icky. Whether or not you think of yourself as a leader, speaking at a conference IS an act of leadership, and leadership carries certain responsibilities.

I long ago learned to let go of the outcome when I speak in public. I throw the ideas out there, and I do that as an American Aging Overweight Left-Handed Atheist Married Father-And-Father-Figure Rough-Mannered Bearded Male Combative Aggressive Assertive High School Dropout Self-Confident Freedom-Loving Sometimes-Unpleasant-To-People-On-Twitter Intellectual. I know that my ideas will not be considered in a neutral context, but rather in the context of how people feel about all that. I accept that.  But, I have been popular and successful as a speaker in the testing world, so maybe, despite all the difficulties, enough of my message and intent gets through, overall.

What I can’t let go of is my responsibility to my audience and the community at large to speak the truth and to do so in a compassionate and reasonable way. Regardless of what anyone else does with our words, I believe we speakers need to think about how our actions help or harm others. I think a lot about this.

Let me clarify. I’m not saying it’s wrong to upset people or to have disagreement. We have several different culture wars (my reviewers said “do you have to say wars?”) going on in the software development and testing worlds right now, and they must continue or be resolved organically in the marketplace of ideas. What I’m saying is that anyone who speaks out publicly must try to be cognizant of what words do and accept the right of others to react.

Although I’m surprised and certainly annoyed by the dark interpretations some people are making of what I did, the burden of such feelings is what I took on when I first put myself forward as a public scold about testing and software engineering, a quarter century ago. My annoyance about being darkly interpreted is not your problem. Your problem, assuming you are reading this and are interested in the state of the testing craft, is to feel what you feel and think what you think, then react as best fits your conscience. Then I listen and try to debug the situation, including helping you debug yourself while I debug myself. This process drives the evolution of our communities. Jay Philips, Ash Coleman, Mike Talks, Ilari Henrik Aegerter, Keith Klain, Anna Royzman, Anne-Marie Charrett, David Greenlees, Aaron Hodder, Michael Bolton, and my own wife all approached me with reactions that helped me write this post. Some others approached me with reactions that weren’t as helpful, and that’s okay, too.

Leadership and The Right of Responding to Leaders

In my code of conduct, I don’t get to say “I’m not a leader.” I can say no one works for me and no one has elected me, but there is more to leadership than that. People with strong voices and ideas gain a certain amount of influence simply by virtue of being interesting. I made myself interesting, and some people want to hear what I have to say. But that comes with an implied condition that I behave reasonably. The community, over time negotiates what “reasonable” means. I am both a participant and a subject of those negotiations. I recommend that we hold each other accountable for our public, professional words. I accept accountability for mine. I insist that this is true for everyone else. Please join me in that insistence.

People who speak at conferences are tacitly asserting that they are thought leaders– that they deserve to influence the community. If that influence comes with a rule that “you can’t talk about me without my permission” it would have a chilling effect on progress. You can keep to yourself, of course; but if you exercise your power of speech in a public forum you cannot cry foul when someone responds to you. Please join me in my affirmation that we all have the right of response when a speaker takes the microphone to keynote at a conference.

Some people have pointed out that it’s not okay to talk back to performers in a comedy show or Broadway play. Okay. So is that what a conference is to you? I guess I believe that conferences should not be for show. Conferences are places for conferring. However, I can accept that some parts of a conference might be run like infomercials or circus acts. There could be a place for that.

The Slide

Here is the slide I used the other day:


Before I explain this slide, try to think what it might mean. What might its purposes be? That’s going to be difficult, without more information about the conference and the talks that happened there. Here are some things I imagine may be going through your mind:

  • There is someone whose name is Maaret who James thinks he’s different from.
  • He doesn’t trust nice people. Nice people are false. Is Maaret nice and therefore he doesn’t trust her, or does Maaret trust nice people and therefore James worries that she’s putting herself at risk?
  • Is James saying that niceness is always false? That’s seems wrong. I have been nice to people whom I genuinely adore.
  • Is he saying that it is sometimes false? I have smiled and shook hands with people I don’t respect, so, yes, niceness can be false. But not necessarily. Why didn’t he put qualifying language there?
  • He likes debate and he thinks that Maaret doesn’t? Maybe she just doesn’t like bad debate. Did she actually say she doesn’t like debate?
  • What if I don’t like debate, does that mean I’m not part of this community?
  • He thinks excellence requires attention and energy and she doesn’t?
  • Why is James picking on Maaret?

Look, if all I saw was this slide, I might be upset, too. So, whatever your impression is, I will explain the slide.

Like I said I was speaking at a conference in NYC. Also keynoting was Maaret Pyhäjärvi. We were both speaking about the testing role. I have some strong disagreements with Maaret about the social situation of testers. But as I watched her talk, I was a little surprised at how I agreed with the text and basic concepts of most of Maaret’s actual slides, and a lot of what she said. (I was surprised because Maaret and I have a history. We have clashed in person and on Twitter.) I was a bit worried that some of what I was going to say would seem like a rehash of what she just did, and I didn’t want to seem like I was papering over the serious differences between us. That’s why I decided to add a contrast slide to make sure our differences weren’t lost in the noise. This means a slide that highlights differences, instead of points of connection. There were already too many points of connection.

The slide was designed specifically:

  • for people to see who were in a specific room at a specific time.
  • for people who had just seen a talk by Maaret which established the basis of the contrast I was making.
  • about differences between two people who are both in the spotlight of public discourse.
  • to express views related to technical culture, not general social culture.
  • to highlight the difference between two talks for people who were about to see the second talk that might seem similar to the first talk.
  • for a situation where both I and Maaret were present in the room during the only time that this slide would ever be seen (unless someone tweeted it to people who would certainly not understand the context).
  • as talking points to accompany my live explanation (which is on video and I assume will be public, someday).
  • for a situation where I had invited anyone in the audience, including Maaret, to ask me questions or make challenges.

These people had just seen Maaret’s talk and were about to see mine. In the room, I explained the slide and took questions about it. Maaret herself spoke up about it, for which I publicly thanked her for doing so. It wasn’t something I was posting with no explanation or context. Nor was it part of the normal slides of my keynote.

Now I will address some specific issues that came up on Twitter:

1. On Naming Maaret

Maaret has expressed the belief that no one should name another person in their talk without getting their permission first. I vigorously oppose that notion. It’s completely contrary to the workings of a healthy society. If that principle is acceptable, then you must agree that there should be no free press. Instead, I would say if you stand up and speak in the guise of an expert, then you must be personally accountable for what you say. You are fair game to be named and critiqued. And the weird thing is that Maaret herself, regardless of what she claims to believe, behaves according to my principle of freedom to call people out. She, herself, tweeted my slide and talked about me on Twitter without my permission. Of course, I think that is perfectly acceptable behavior, so I’m not complaining. But it does seem to illustrate that community discourse is more complicated than “be nice” or “never cause someone else trouble with your speech” or “don’t talk about people publicly unless they gave you permission.”

2. On Being Nice

Maaret had a slide in her talk about how we can be kind to each other even though we disagree. I remember her saying the word “nice” but she may have said “kind” and I translated that into “nice” because I believed that’s what she meant. I react to that because, as a person who believes in the importance of integrity and debate over getting along for the sake of appearances, I observe that exhortations to “be nice” or even to “be kind” are often used when people want to quash disturbing ideas and quash the people who offer them. “Be nice” is often code for “stop arguing.” If I stop arguing, much of my voice goes away. I’m not okay with that. No one who believes there is trouble in the world should be okay with that. Each of us gets to have a voice.

I make protests about things that matter to me, you make protests about things that matter to you.

I think we need a way of working together that encourages debate while fostering compassion for each other. I use the word compassion because I want to get away from ritualized command phrases like “be nice.” Compassion is a feeling that you cultivate, rather than a behavior that you conform to or simulate. Compassion is an antithesis of “Rules of Order” and other lists of commandments about courtesy. Compassion is real. Throughout my entire body of work you will find that I promote real craftsmanship over just following instructions. My concern about “niceness” is the same kind of thing.

Look at what I wrote: I said “I don’t trust nice people.” That’s a statement about my feelings and it is generally true, all things being equal. I said “I’m not nice.” Yet, I often behave in pleasant ways, so what did I mean? I meant I seek to behave authentically and compassionately, which looks like “nice” or “kind”, rather than to imagine what behavior would trick people into thinking I am “nice” when indeed I don’t like them. I’m saying people over process, folks.

I was actually not claiming that Maaret is untrustworthy because she is nice, and my words don’t say that. Rather, I was complaining about the implications of following Maaret’s dictum. I was offering an alternative: be authentic and compassionate, then “niceness” and acts of kindness will follow organically. Yes, I do have a worry that Maaret might say something nice to me and I’ll have to wonder “what does that mean? is she serious or just pretending?” Since I don’t want people to worry about whether I am being real, I just tell them “I’m not nice.” If I behave nicely it’s either because I feel genuine good will toward you or because I’m falling down on my responsibility to be honest with you. That second thing happens, but it’s a lapse. (I do try to stay out of rooms with people I don’t respect so that I am not forced to give them opinions they aren’t willing or able to process.)

I now see that my sentence “I want to be authentic and compassionate” could be seen as an independent statement connected to “how I differ from Maaret,” implying that I, unlike her, am authentic and compassionate. That was an errant construction and does not express my intent. The orange text on that line indicated my proposed policy, in the hope that I could persuade her to see it my way. It was not an attack on her. I apologize for that confusion.

3. Debate vs. Dialogue

Maaret had earlier said she doesn’t want debate, but rather dialogue. I have heard this from other Agilists and I find it disturbing. I believe this is code for “I want the freedom to push my ideas on other people without the burden of explaining or defending those ideas.” That’s appropriate for a brainstorming session, but at some point, the brainstorming is done and the judging begins. I believe debate is absolutely required for a healthy professional community. I’m guided in this by dialectical philosophy, the history of scientific progress, the history of civil rights (in fact, all of politics), and the modern adversarial justice system. Look around you. The world is full of heartfelt disagreement. Let’s deal with it. I helped create the culture of small invitational peer conferences in our industry which foster debate. We need those more than ever.

But if you don’t want to deal with it, that’s okay. All that means is that you accept that there is a wall between your friends and those other people whom you refuse to debate with. I will accept the walls if necessary but I would rather resolve the walls. That’s why I open myself and my ideas for debate in public forums.

Debate is not a process of sticking figurative needles into other people. Debate is the exchange of views with the goal of resolving our differences while being accountable for our words and actions. Debate is a learning process. I have occasionally heard from people I think are doing harm to the craft that they believe I debate for the purposes of hurting people instead of trying to find resolution. This is deeply insulting to me, and to anyone who takes his vocation seriously. What’s more, considering that these same people express the view that it’s important to be “nice,” it’s not even nice. Thus, they reveal themselves to be unable to follow their own values. I worry that “Dialogue not debate” is a slogan for just another power group trying to suppress its rivals. Beware the Niceness Gang.

I understand that debating with colleagues may not be fun. But I’m not doing it for fun. I’m doing it because it is my responsibility to build a respectable craft. All testing professionals share this responsibility. Debate serves another purpose, too, managing the boundaries between rival value systems. Through debate we may discover that we occupy completely different paradigms; schools of thought. Debate can’t bridge gaps between entirely different world views, and yet I have a right to my world view just as you have a right to yours.

Jay Philips said on Twitter:

@jamesmarcusbach pointless 2debate w/ U because in your mind you’re right. Slide &points shouldn’t have happened @JokinAspiazu @ericproegler

— Jay Philips (@jayphilips) September 30, 2016

I admire Jay. I called her and we had a satisfying conversation. I filled her in on the context and she advised me to write this post.

One thing that came up is something very important about debate: the status of ideas is not the only thing that gets modified when you debate someone; what also happens is an evolution of feelings.

Yes I think “I’m right.” I acted according to principles I think are eternal and essential to intellectual progress in society. I’m happy with those principles. But I also have compassion for the feelings of others, and those feelings may hold sway even though I may be technically right. For instance, Maaret tweeted my slide without my permission. That is copyright violation. She’s objectively “wrong” to have done that. But that is irrelevant.

[Note: Maaret points out that this is legal under the fair use doctrine. Of course, that is correct. I forgot about fair use. Of course, that doesn’t change the fact that though I may feel annoyed by her selective publishing of my work, that is irrelevant, because I support her option to do that. I don’t think it was wise or helpful for her to do that, but I wouldn’t seek to bar her from doing so. I believe in freedom to communicate, and I would like her to believe in that freedom, too]

I accept that she felt strongly about doing that, so I [would] choose to waive my rights. I feel that people who tweet my slides, in general, are doing a service for the community. So while I appreciate copyright law, I usually feel okay about my stuff getting tweeted.

I hope that Jay got the sense that I care about her feelings. If Maaret were willing to engage with me she would find that I care about her feelings, too. This does not mean she gets whatever she wants, but it’s a factor that influences my behavior. I did offer her the chance to help me edit this post, but again, she refused.

4. Focus and Energy

Maaret said that eliminating the testing role is a good thing. I worry it will lead to the collapse of craftsmanship. She has a slide that says “from tester to team member” which is a sentiment she has expressed on Twitter that led me to say that I no longer consider her a tester. She confirmed to me that I hurt her feelings by saying that, and indeed I felt bad saying it, except that it is an extremely relevant point. What does it mean to be a tester? This is important to debate. Maaret has confirmed publicly (when I asked a question about this during her talk) that she didn’t mean to denigrate testing by dismissing the value of a testing role on projects. But I don’t agree that we can have it both ways. The testing role, I believe, is a necessary prerequisite for maintaining a healthy testing craft. My key concern is the dilution of focus and energy that would otherwise go to improving the testing craft. This is lost when the role is lost.

This is not an attack on Maaret’s morality. I am worried she is promoting too much generalism for the good of the craft, and she is worried I am promoting too much specialism. This is a matter of professional judgment and perspective. It cannot be settled, I think, but it must be aired.

The Slide Should Not Have Been Tweeted But It’s Okay That It Was

I don’t know what Maaret was trying to accomplish by tweeting my slide out of context. Suffice it to say what is right there on my slide: I believe in authenticity and compassion. If she was acting out of authenticity and compassion then more power to her. But the slide cannot be understood in isolation. People who don’t know me, or who have any axe to grind about what I do, are going to cry “what a cruel man!” My friends contacted me to find out more information.

I want you to know that the slide was one part of a bigger picture that depicts my principled objection to several matters involving another thought leader. That bigger picture is: two talks, one room, all people present for it, a lot of oratory by me explaining the slide, as well as back and forth discussion with the audience. Yes, there were people in the room who didn’t like hearing what I had to say, but “don’t offend anyone, ever” is not a rule I can live by, and neither can you. After all, I’m offended by most of the talks I attend.

Although the slide should not have been tweeted, I accept that it was, and that doing so was within the bounds of acceptable behavior. As I announced at the beginning of my talk, I don’t need anyone to make a safe space for me. Just follow your conscience.

What About My Conscience?
  • My conscience is clean. I acted out of true conviction to discuss important matters. I used a style familiar to anyone who has ever seen a public debate, or read an opinion piece in the New York Times. I didn’t set out to hurt Maaret’s feelings and I don’t want her feelings to be hurt. I want her to engage in the debate about the future of the craft and be accountable for her ideas. I don’t agree that I was presuming too much in doing so.
  • Maaret tells me that my slide was “stupid and hurtful.” I believe she and I do not share certain fundamental values about conferring. I will no longer be conferring with her, until and unless those differences are resolved.
  • Compassion is important to me. I will continue to examine whether I am feeling and showing the compassion for my fellow humans that they are due. These conversations and debates I have with colleagues help me do that.
  • I agree that making a safe space for students is important. But industry consultants and pundits should be able to cope with the full spectrum, authentic, principled reactions by their peers. Leaders are held to a higher standard, and must be ready and willing to defend their ideas in public forums.
  • The reaction on Twitter gave me good information about a possible trend toward fragility in the Twitter-facing part of the testing world. There seems to be a significant group of people who prize complete safety over the value that comes from confrontation. In the next conference I help arrange, I will set more explicit ground rules, rather than assuming people share something close to my own sense of what is reasonable to do and expect.
  • I will also start thinking, for each slide in my presentation: “What if this gets tweeted out of context?”

(Oh, and to those who compared me to Donald Trump… Can you even imagine him writing a post like this in response to criticism? BELIEVE ME, he wouldn’t.)

Categories: Testing & QA

Closed Loop Control and Granularity of the Estimating Process

Herding Cats - Glen Alleman - Sat, 10/01/2016 - 16:30

For any closed loop control system ‒ let’s assume we want to manage our project with such a system ‒ has a signal representing the current state of the system. For software, this can be value produced (assuming we have a unit of measure for that value in the for of effectiveness, performance, key performance parameters, or technical performance measures). https://goo.gl/DP6Jw is an overview of this process.

To control this process using feedback and corrective actions ‒ in the same way, your closed loop controller for your air conditioner or heater does - a sampling rate is determined based on the rate of change of the underlying processes.

For a software project, let’s start by answering a critical question – how long are you willing to wait before you find out you’re late? For your Honeywell or Nest controller on the wall, that sample rate is measured in seconds. So the answer to the software question needs to be determined by two things for a closed loop control system:

  • The pace of software deliverables – if your sampling the state of the software outcomes longer than you are producing those outcomes, errors unfavorable outcomes are accumulating faster than you are measuring them.
  • The response to change rate ‒ how fast can you make corrective actions to the system to keep it on plan. The plan can be the temperature of the room. It can be the speed of your car. It can be the cost, schedule, and technical performance measures of the software project.

These measures, corrective actions, and resulting outcomes operate in the presence of statistical and probabilistic processes. Stochastic process control is the field of study. I cut my teeth writing FORTRAN 77 code for the control system for flying, driving, and swimming machines and then for stationary machines, like a power plant, refinery and paper mills.

For the software machines the principles of closed loop control in the presence of noise and changing conditions are the same ‒ https://goo.gl/cUcXNj

The sample rate is also applied to the target to steer toward – the set point in the case of the temperature or cruise control. The estimate of Desired (or even contractual) schedule, cost, or technical performance of the software project.

The granularity of these estimates determining the dynamic behavior of the system ‒ the software project. Too large and accumulation of errors occur. Too small and wasted information is produced and possibly even over controlling the system.

When we hear the question at what level of an agile project do we estimate – Vision, Theme, Release, Feature, Story, or even Task ‒ the first question is again ­

How long are you  willing to wait before you are late, or the room is too hot, or you've driven into the ditch, or the radar station steering control has lost track with the incoming target, or your exothermic reactor has gone over-temperature and blown up?

This answer tells you the granularity of the control systems sampling and corrective actions as well as telling you the granularity of the steering target needed to maintain control of the underlying process.

So don't let anyone suggest estimating at any level - Feature, Stroy, or Task is right or wrong until they can answer...

  • How long are you willing to wait before you find out you are late?
  • What is the dynamic behaviour  of the underlying process?
  • How much error debt are we willing to build up before taking corrective actions?
  • What are the consequences losing  lock on the corretive actions (the project is out of control)?
  • What is the value at risk you are willing to right off if you lose control?

Managing software development projects is a Closed Loop control system. Subject to all the principles and practices of Closed Loop control. Any suggestion of another method of managing must be tested against these principles first before continuing the conversation. 

Related articles Complex Project Management Hope is not a Strategy Herding Cats: The Origins of Scrum and Empirical Closed Loop Control Closed Loop Control Doing the Math Herding Cats: The #NoEstimates Paradigm and Response
Categories: Project Management

Using BigQuery and Firebase Analytics to understand your mobile app

Google Code Blog - Fri, 09/30/2016 - 23:41
Originally posted on Google Cloud Platform Blog Posted by Sara Robinson, Developer Advocate

At Google I/O this May, Firebase announced a new suite of products to help developers build mobile apps. Firebase Analytics, a part of the new Firebase platform, is a tool that automatically captures data on how people are using your iOS and Android app, and lets you define your own custom app events. When the data's captured, it’s available through a dashboard in the Firebase console. One of my favorite cloud integrations with the new Firebase platform is the ability to export raw data from Firebase Analytics to Google BigQuery for custom analysis. This custom analysis is particularly useful for aggregating data from the iOS and Android versions of your app, and accessing custom parameters passed in your Firebase Analytics events. Let’s take a look at what you can do with this powerful combination.

How does the BigQuery export work?
After linking your Firebase project to BigQuery, Firebase automatically exports a new table to an associated BigQuery dataset every day. If you have both iOS and Android versions of your app, Firebase exports the data for each platform into a separate dataset. Each table contains the user activity and demographic data automatically captured by Firebase Analytics, along with any custom events you’re capturing in your app. Thus, after exporting one week’s worth of data for a cross-platform app, your BigQuery project would contain two datasets, each with seven tables:

Diving into the data
The schema for every Firebase Analytics export table is the same, and we’ve created two datasets (one for iOS and one for Android) with sample user data for you to run the example queries below. The datasets are for a sample cross-platform iOS and Android gaming app. Each dataset contains seven tables — one week’s worth of analytics data.

The following query will return some basic user demographic and device data for one day of usage on the iOS version of our app:


Since the schema for every BigQuery table exported from Firebase Analytics is the same, you can run any of the queries in this post on your own Firebase Analytics data by replacing the dataset and table names with the ones for your project.

The schema has user data and event data. All user data is automatically captured by Firebase Analytics, and the event data is populated by any custom events you add to your app. Let’s take a look at the specific records for both user and event data.

User data
The user records contain a unique app instance ID for each user (user_dim.app_info.app_instance_id in the schema), along with data on their location, device and app version. In the Firebase console, there are separate dashboards for the app’s Android and iOS analytics. With BigQuery, we can run a query to find out where our users are accessing our app around the world across both platforms. The query below makes use of BigQuery’s union feature, which lets you use a comma as a UNION ALL operator. Since a row is created in our table for each bundle of events a user triggers, we use EXACT_COUNT_DISTINCT to make sure each user is only counted once:
user_dim.geo_info.country as country,
EXACT_COUNT_DISTINCT( user_dim.app_info.app_instance_id ) as users
users DESC

User data also includes a user_properties record, which includes attributes you define to describe different segments of your user base, like language preference or geographic location. Firebase Analytics captures some user properties by default, and you can create up to 25 of your own.

A user’s language preference is one of the default user properties. To see which languages our users speak across platforms, we can run the following query:

user_dim.user_properties.value.value.string_value as language_code,
EXACT_COUNT_DISTINCT(user_dim.app_info.app_instance_id) as users,
user_dim.user_properties.key = "language"
users DESC

Event data
Firebase Analytics makes it easy to log custom events such as tracking item purchases or button clicks in your app. When you log an event, you pass an event name and up to 25 parameters to Firebase Analytics and it automatically tracks the number of times the event has occurred. The following query shows the number of times each event in our app has occurred on Android for a particular day:

COUNT(event_dim.name) as event_count
event_count DESC

If you have another type of value associated with an event (like item prices), you can pass it through as an optional value parameter and filter by this value in BigQuery. In our sample tables, there is a spend_virtual_currency event. We can write the following query to see how much virtual currency players spend at one time:

event_dim.params.value.int_value as virtual_currency_amt,
COUNT(*) as num_times_spent
event_dim.name = "spend_virtual_currency"
event_dim.params.key = "value"
num_times_spent DESC

Building complex queries
What if we want to run a query across both platforms of our app over a specific date range? Since Firebase Analytics data is split into tables for each day, we can do this using BigQuery’s TABLE_DATE_RANGE function. This query returns a count of the cities users are coming from over a one week period:

COUNT(user_dim.geo_info.city) as city_count
TABLE_DATE_RANGE([firebase-analytics-sample-data:android_dataset.app_events_], DATE_ADD('2016-06-07', -7, 'DAY'), CURRENT_TIMESTAMP()),
TABLE_DATE_RANGE([firebase-analytics-sample-data:ios_dataset.app_events_], DATE_ADD('2016-06-07', -7, 'DAY'), CURRENT_TIMESTAMP())
city_count DESC

We can also write a query to compare mobile vs. tablet usage across platforms over a one week period:

user_dim.app_info.app_platform as appPlatform,
user_dim.device_info.device_category as deviceType,
COUNT(user_dim.device_info.device_category) AS device_type_count FROM
TABLE_DATE_RANGE([firebase-analytics-sample-data:android_dataset.app_events_], DATE_ADD('2016-06-07', -7, 'DAY'), CURRENT_TIMESTAMP()),
TABLE_DATE_RANGE([firebase-analytics-sample-data:ios_dataset.app_events_], DATE_ADD('2016-06-07', -7, 'DAY'), CURRENT_TIMESTAMP())
device_type_count DESC

Getting a bit more complex, we can write a query to generate a report of unique user events across platforms over the past two weeks. Here we use PARTITION BY and EXACT_COUNT_DISTINCT to de-dupe our event report by users, making use of user properties and the user_dim.user_id field:

STRFTIME_UTC_USEC(eventTime,"%Y%m%d") as date,
COUNT(*) totalEvents,
EXACT_COUNT_DISTINCT(IF(userId IS NOT NULL, userId, fullVisitorid)) as users
FORMAT_UTC_USEC(openTimestamp) firstOpenedTime,
MAX(userIdSet) OVER(PARTITION BY fullVisitorid) userId,
FORMAT_UTC_USEC(eventTimestamp) as eventTime,
user_dim.app_info.app_instance_id as fullVisitorid,
user_dim.first_open_timestamp_micros as openTimestamp,
IF(user_dim.user_properties.key = 'user_id',user_dim.user_properties.value.value.string_value, null) as userIdSet,
user_dim.app_info.app_platform as appPlatform,
event_dim.timestamp_micros as eventTimestamp,
event_dim.name AS eventName,
TABLE_DATE_RANGE([firebase-analytics-sample-data:android_dataset.app_events_], DATE_ADD('2016-06-07', -7, 'DAY'), CURRENT_TIMESTAMP()),
TABLE_DATE_RANGE([firebase-analytics-sample-data:ios_dataset.app_events_], DATE_ADD('2016-06-07', -7, 'DAY'), CURRENT_TIMESTAMP())
), user_dim.user_properties)
date, appPlatform, eventName

If you have data in Google Analytics for the same app, it’s also possible to export your Google Analytics data to BigQuery and do a JOIN with your Firebase Analytics BigQuery tables.

Visualizing analytics data
Now that we’ve gathered new insights from our mobile app data using the raw BigQuery export, let’s visualize it using Google Data Studio. Data Studio can read directly from BigQuery tables, and we can even pass it a custom query like the ones above. Data Studio can generate many different types of charts depending on the structure of your data, including time series, bar charts, pie charts and geo maps.

For our first visualization, let’s create a bar chart to compare the device types from which users are accessing our app on each platform. We can paste the mobile vs. tablet query above directly into Data Studio to generate the following chart:
From this chart, it’s easy to see that iOS users are much more likely to access our game from a tablet. Getting a bit more complex, we can use the above event report query to create a bar chart comparing the number of events across platforms:
Check out this post for detailed instructions on connecting your BigQuery project to Data Studio.

What’s next?If you’re new to Firebase, get started here. If you’re already building a mobile app on Firebase, check out this detailed guide on linking your Firebase project to BigQuery. For questions, take a look at the BigQuery reference docs and use the firebase-analytics and google-bigquery tags on Stack Overflow. And let me know if there are any particular topics you’d like me to cover in an upcoming post.
Categories: Programming

Stuff The Internet Says On Scalability For September 30th, 2016

Hey, it's HighScalability time:


Everything is a network. Map showing the global genetic interaction network of a cell. 


If you like this sort of Stuff then please support me on Patreon.
  • 18: Google can now drink and drive in Washington DC.; $10 billion: cost of a Vision Quest to Mars; 620 Gbps: DDoS attack on KrebsOnSecurity; 1 Tbps: DDoS attack on OVH; $200,000: cost of a typical cyber incident; 8 million: video training dataset labeled with 4800 labels; 180: Amazon warehouses in the US; 10: bits of info per photon; 16: GPUs in new AI killer P2 instance type;

  • Quotable Quotes:
    • @markmccaughrean: 1,000,000 people to Mars in 100 yrs. 10 people/launch? That's 3 a day, every day, for a century. 1% failure rate? One explosion every month
    • @jeremiahg: Any sufficiently advanced exploit is indistinguishable from a 400lb hacker.
    • BrianKrebs: I suggested to Mr. Wright perhaps a better comparison was that ne’er-do-wells now have a virtually limitless supply of Stormtrooper clones that can be conscripted into an attack at a moment’s notice.
    • Sonia: Academia’s not-so-subtle distain for applied research does more than damage a few promising careers; it renders our field’s output useless, destined to collect dust on the shelves of Elsevier. 
    • Monica L. Smith: Nobody builds their own infrastructure. You don’t build your own highway, train line, water pipe, your own sewer. Those are things that connect you and your household to everybody else sequentially in your neighborhood, in your region, from the city out into the broader hinterlands.
    • @olesovhcom: This botnet with 145607 cameras/dvr (1-30Mbps per IP) is able to send >1.5Tbps DDoS. Type: tcp/ack, tcp/ack+psh, tcp/syn.
    • kenrose: We see this pattern at PagerDuty over the majority of our customers. There is a definite lull in alert volume over the weekends that picks up first thing Monday morning.It's led to my personal conclusion that most production issues are caused by people, not errant hardware or systems.
    • @rseroter: "We Crammed this Monolith Into a Container and Called it a Microservice"
    • @mweagle: I really don’t want to run my own k8s in AWS, but ECS is so opaque to debug that k8s seems like a good choice.
    • Werner Vogels~ We have this overarching goal which is customer centricity. Doing anything that benefits the customer gets priority above everything else. Working on eliminating all single points of failure in the company purely benefits the customer because it really improves the customer experience.
    • Cory Doctorow~ The thing open source software had going for it was the Ulysses Pact...the  irrevocable license, the failure mode of open source software, having founded an open source software company, I can tell you there are moments where it feels like your survival turns on being able to close the code you had opened when you were idealistic. There are moments of desperation when that happens. 
    • @lightbend: "We've been using #Akka in production for over two years, without a single crash." -@CruiseNorwegian |
    • @cloud_opinion: Monolithic -> Microservices -> "which container image?" -> "Screw it, lets do PaaS" ->  CF  or AWS?
    • Etsy: concurrency proved to be great for logical aggregation of components, and not so great for performance optimization. Better database access would be better for that.
    • Yaniv Nizan: the number of users actually contributing ad revenue in your app is a lot lower than 6.5% and much closer to the 1% or 2% that contribute revenue from In-app purchases. 
    • @reckless: Elon is basically putting on an Apple event, for going to Mars.
    • @potch: DRY: Don't Repeat Yourself / DAMP: Do Abstraction/Minimalism Pragmatically / MOIST: Maybe Only Innovate Some Times?
    • @dannysullivan: In the Facebook video metrics thing, spare a thought for the poor BuzzFeed watermelon, less viral than it thought :)
    • Addison Snell: If the promise of cloud computing is overblown, it because of the amplification it gets from its loyal converts, enterprises who have found liberation and agility in outsourcing IT. 
    • @psaffo: In 1990, the size of the US software industry was $3.2 billion -- the same size as the gourmet popcorn industry in that same year.
    • David Rosenthal: [Storage] Revenues are flat or decreasing, profits are decreasing for both companies. These do not look like companies faced by insatiable demand for their products; they look like mature companies facing increasing difficulty in scaling their technology.
    • @legind: Let's Encrypt now the 3rd largest CA, after Comodo and Symantec, comprising over 13% of the SSL cert market share 
    • @stewartbrand: “In the long run, the technology driving activities in space will be biological.” Rousing essay by Freeman Dyson.
    • @jessitron: Constructing causal ordering at the generic level of "all messages received cause all future messages sent" is expensive and also less meaningful than a business-logic-aware, conscious causal ordering. This conscious causal ordering gives us external consistency, accurate legibility, and visibility into what we know to be causal.

  • In an article light on details, written more with a marketing flourish, we still learn some interesting details on the infrastructure behind Pokemon Go. Bringing Pokémon GO to life on Google Cloud. It runs on Google Cloud, Kubernetes, Google Container Engine, HTTP/S Load Balancer, and Cloud Datastore. Keep in mind Alphabet is invested in Niantic and Ingress, the forerunner of Pokemon Go, ran on App Engine. So it sounds like a new backend implementation that had to scale from zero to the size of Twitter in a matter of weeks, with a much more complicated work load. Growth was explosive. Player traffic was 50x larger than initial estimates. An implication is the problems experienced during launch were not infrastructure related. Google, in the form of Customer Reliability Engineer (CRE), worked closely with Niantic to make sure the infrastructure scaled. The problems must have been elsewhere in the application stack, which is perfectly understandable. That sort of load could not have been predicted. The design decisions you make for 5x expected traffic are very different than they are for 50x. Nobody will spend the money or take the time to build a system for 50x. Nobody. Lots of good comments on HackerNews. Good question by ksec, would Poekemon Go even be possible in a pre-cloud era? 

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Five Immutable Principles of Project Success and Project Failure

Herding Cats - Glen Alleman - Fri, 09/30/2016 - 02:43

I saw a blog post about the Top 5 Reasons Your Project Fails recently. They were all good reasons, but those reasons were symptoms, not causes. We seem to always identify the symptoms, but until we fix the cause of failure, those symptoms will return.

The symptoms were:

  1. Priorities change.
  2. Incomplete requirements.
  3. Lack of Resources.
  4. Lack of User Support.
  5. Lack of Executive Support.

But these are simply missing practices and processes of the 5 Immutable Principles of project success

5 Immutable Principles

So let's look at the symptoms and the principle that could have addressed that symptom

  1. What Does Done Look Like?
  2. What is the path to Done?
    • Without a Plan, we have no visibility to the steps needed to reach Done.
    • As Yogi Berra says If you don't know where you're going, you'll end up someplace else.
  3. Do we have enough  time, resources, and money to get to Done?
    • Without a plan, we can't know how many resources will be needed, what kind of resources, and when they will be needed
    • That time, resources, and money are actually random variables, drawn from an underlying population. The distribution of this population can be determined through a variety of means.
  4. What impediments will we encounter along the way to Done?
    • Risk Management is how Adults Manage Projects - Tim Lister
    • What are the risk, what are their mitigations
  5. How do we know we are making progress toward Done?
    • Measuring Physical Percent Complete is the foundation of all good project management
    • This physical percent complete can be represented as effectiveness, performance, key performance parameter, and other technical measures

So with the 5 symptoms assigned to the 5 Principles, corrective actions can be put in place to avoid the outcomes.

Related articles Want To Learn How To Estimate? Capabilities Based Planning First Then Requirements Risk Management is How Adults Manage Projects Root Cause of Project Failure
Categories: Project Management

Agile Risk Management: Recognizing Risks

You can't capture risk with your camera. You need to have a conversation with a diverse group of stakeholders.

You can’t capture risk with your camera. You need to have a conversation with a diverse group of stakeholders.

At a recent Q&A session I was asked: Where could a person get their project risks? I stifled a smart-alecky answer that would have included driving to the grocery store, and decided that the question that was being asked was really: How do I go about recognizing and capturing risks? Perhaps a more boring question, but far more important. If I answered the first question the answer would have been that risks are generated by the interaction of the project with other projects, applications, the business, technology and world (risk categories) – pretty much the existence of a project could be considered a risk magnet. The answer to the second question is that once you have a risk magnet (a project) you will need to ask as many different people as is feasible to recognize the possible risks. The discussion of risk is always appropriate; however, the typical meeting/events and the types of people to consider in the conversation needs to be planned. The discovery process typically follows the requirements/user story discovery process outlined below.

  1. Carve out time when you are developing the backlog and ask as diverse a group as possible to identify the potential problems that could get in the away of delivering the value promised by the project. Prompt the group to consider business, technical, operational and organizational factors. Diversity is incredibly important to inject different perspectives, so that the team does not fall prey to only seeing the risks they expect (a form of cognitive bias).
  2. Form a small team (consider the Three Amigos) to interview stakeholders that were not part of the planning exercise. Explain the project and use the same category prompts to generate a risk discussion.
  3. Gather risk data though surveys when the program stakeholders are geographically diverse. (Note: I have only seen this used well in very large programs with professional market research staffs)
  4. Interview customers or potential customers. Customer interviews are not generally used as a standalone risk discovery tool, but rather as a tool to gather requirements/user stories. However, piggybacking a few questions to solicit potential risks is useful to add a diversity of thought to risk identification.
  5. Periodically ask about risks either as an agenda item or as a follow-on to standard meetings. For example, I have seen teams that have successfully added a five-minute follow on to the last daily stand-up of the week in order to consider risks. A quick risk recognition session can easily be added to other standard meeting many projects have. Other standard scrum meetings that can be used to identify risks include: demonstrations, retrospectives and sprint planning. Each of these meetings will provide a different perspective on the project and the team therefore could expose other potential risks.

The baseline answer to the question of how can I recognize and capture risks is by involving all of the projects stakeholders in a discussion of potential risks. The process of collaborative discussion will help increase diversity of thought, reducing (but NOT eliminating) the potential number of unknowns – unknowns that could impact the projects ability to deliver value.

Categories: Process Management

Android Wear 2.0 Developer Preview 3: Play Store and More

Android Developers Blog - Thu, 09/29/2016 - 18:00

Posted by Hoi Lam, Developer Advocate

Today we’re launching the third developer preview of Android Wear 2.0 with a big new addition: Google Play on Android Wear. The Play Store app makes it easy for users to find and install apps directly on the watch, helping developers like you reach more users.

Play Store features

With Play Store for Android Wear, users can browse recommended apps in the home view and search for apps using voice, keyboard, handwriting, and recommended queries, so they can find apps more easily. Users can switch between multiple accounts, be part of alpha and beta tests, and update or uninstall apps in the “My apps” view on their watch, so they can manage apps more easily. Perhaps the coolest feature: If users want an app on their watch but not on their phone, they can install only the watch app. In fact, in Android Wear 2.0, phone apps are no longer necessary. You can now build and publish watch-only apps for users to discover on Google Play.

Why an on-watch store?

We asked developers like you what you wanted most out of Android Wear, and you told us you wanted to make it easier for users to discover apps. So we ran studies with users to find out where they expected and wanted to discover apps––and they repeatedly looked for and asked for a way to discover apps right on the watch itself. Along with improvements to app discovery on the phone and web, the Play Store on the watch helps users find apps right where they need them.

Publish your apps

To make your apps available on Play Store for Android Wear, just follow these steps. You’ll need to make sure your Android Wear 2.0 apps set minSdkVersion to 24 or higher, use the runtime permissions model, and are uploaded via multi-APK using the Play Developer Console. If your app supports Android Wear 1.0, the developer guide also covers the use of product flavors in Gradle.

Download the New Android Wear companion app

To set up Developer Preview 3, you’ll need to install a beta version of the Android Wear app on your phone, flash your watch to the latest preview release, and use the phone app to add a Google Account to your watch. These steps are detailed in Download and Test with a Device. If you don’t have a watch to test on, you can use the emulator as well.

Other additions in Developer Preview 3 Developer Preview 3 also includes:
  • Complications improvements: Starting with Developer Preview 3, watch face developers will need to request RECEIVE_COMPLICATION_DATA permission before the watch face can receive complication data. We have added ComplicationHelperActivity to make this easier. In addition, watch face developers can now set default complications, including a selection of system data complications which do not require special permission (e.g. battery level and step count), as well as data providers that have whitelisted the watch face. Lastly, there are behavior changes related to ComplicationData to 1) help better differentiate various scenarios leading to “empty data” and 2) ease development by returning a default value for fields not supported by a complication type instead of throwing a runtime exception.
  • New WearableRecyclerView: This new UI component helps developers display and manipulate vertical lists of items while optimizing for round displays.
  • Inline Action for Notifications: A new API makes it easy to take action on a notification right from the stream. Developers can specify which action is displayed inline at the bottom of the notification by calling setHintDisplayActionInline:
    NotificationCompat.Action replyAction =
        new NotificationCompat.Action.Builder(R.drawable.ic_message_white_24dp,
                "Reply", replyPendingIntent)
                .extend(new NotificationCompat.Action.WearableExtender()
  • Smart Reply: Android Wear now generates Smart Reply responses for MessagingStyle notifications. Smart Reply responses are generated by an entirely on-watch machine learning model using the context provided by the MessagingStyle notification, and no data is uploaded to the cloud to generate the responses.
  • And much more: Read about the complete list of changes in the Android Wear developer preview release notes. Timeline

    We’ve gotten tons of great feedback from the developer community about Android Wear 2.0––thank you! We’ve decided to continue the preview program into early 2017, at which point the first watches will receive Android Wear 2.0. Please keep the feedback coming by filing bugs or posting in our Android Wear Developers community, and stay tuned for Android Wear Developer Preview 4.

Categories: Programming

Quote of the Day

Herding Cats - Glen Alleman - Thu, 09/29/2016 - 15:23

Never attribute to malevolence what is explicable by incompetence. - Robert J Halon

When we hear all the bad things that go  wrong with projects. The misuse and abuse of data, people, tools, and processes, I get a smile when I remember Halon's quote.

Removing things, changing things, installing new things will not address the root cause of bad management and especially bad project management. Only replacing the people will fix the root cause when they are willfully ignorant of how to do it right. 

Related articles Architecture -Center ERP Systems in the Manufacturing Domain IT Risk Management Why Guessing is not Estimating and Estimating is not Guessing
Categories: Project Management

Estimating Resources

Herding Cats - Glen Alleman - Wed, 09/28/2016 - 19:54

There's a never-ending opportunity to learn how to estimate in the presence of uncertainty. Here's some resources for informing that learning process. 

When you hear that estimates are a waste (we'd rather be coding), estimates are fiction, we're bad at estimating, and the plethora of other excuses for not learning how to estimate, ask if that person has done the minimal homework to learn how to estimate needed to make decisions in the presence of uncertainty found on all software development projects.   Don't need estimates? - the project is de minimis  Related articles Want To Learn How To Estimate? How We Make Decisions is as Important as What We Decide. Herding Cats: Where Are We Going, Doesn't Much Matter It Seems Herding Cats: Estimating on Non-Trivial Software Projects Taxonomy of Logical Fallacies
Categories: Project Management

How to set up Ads on your AMP Pages

Google Code Blog - Wed, 09/28/2016 - 17:03

Posted by Arudea Mahartianto, Google AMP Specialist

From conception, the open source Accelerated Mobile Pages Project has had a clear goal --- to make the mobile web experience better and faster for users. This extends beyond content to creating a user-first approach to advertising as well.

To realize this vision, the AMP team created an advertising solution that follows four core principles:

  • Faster is better - There is no reason ads in AMP can’t be as fast as the AMP document itself.
  • Beautiful matters - Ensure ads in AMP are beautiful and relevant.
  • Security is a must - Require all creatives to utilize the HTTPS protocol.
  • We’re better together - AMP isn’t about supporting a single advertising entity, but an entire industry. Success requires broad industry participation.

Ads in AMP are delivered using the amp-ad component. Using this component you can configure your ads in a number of ways such as the width, length, layout mode and ad loading strategy. Different ad networks might allow even more options.

Here is an example of an DoubleClick responsive ad implementation in AMP:


The type attribute informs the amp-ad component which ad platform to use. In this case we want DoubleClick and therefore the type value is doubleclick. For above the fold responsive ad implementation please use layout="fixed-height" instead and limit the ad height so users will get a fast loading content-focused experience from the very start.

Any attributes starting with data- in amp-ad are ad platform-specific attributes, including the data-slot attribute in the snippet above. Each ad platform will have different attributes available to configure. For example, compare the above DoubleClick example with another AMP ad example that uses the Rubicon platform:

<amp-ad width=320 height=50

For more amp-ad implementation examples, please check out AMP By Example. You can also check out the amp-ad documentation for the complete list of supported ad networks and their configuration semantics.

The team is also developing newer, better ways to bring the benefits of AMP to the ads ecosystem with initiatives like AMP for Ads and AMP Ad Landing Pages. These solutions will enable advertisers to design creatives and ad landing pages that are more consistent with the AMP experience publishers are bringing to users. We believe this will bring us closer to the goal of making the entire mobile web experience faster and better for everybody.

Categories: Programming

How Uber Manages a Million Writes Per Second Using Mesos and Cassandra Across Multiple Datacenters

If you are Uber and you need to store the location data that is sent out every 30 seconds by both driver and rider apps, what do you do? That’s a lot of real-time data that needs to be used in real-time.

Uber’s solution is comprehensive. They built their own system that runs Cassandra on top of Mesos. It’s all explained in a good talk by Abhishek Verma, Software Engineer at Uber: Cassandra on Mesos Across Multiple Datacenters at Uber (slides).

Is this something you should do too? That’s an interesting thought that comes to mind when listening to Abhishek’s talk.

Developers have a lot of difficult choices to make these days. Should we go all in on the cloud? Which one? Isn’t it too expensive? Do we worry about lock-in? Or should we try to have it both ways and craft brew a hybrid architecture? Or should we just do it all ourselves for fear of being cloud shamed by our board for not reaching 50 percent gross margins?

Uber decided to build their own. Or rather they decided to weld together their own system by fusing together two very capable open source components. What was needed was a way to make Cassandra and Mesos work together, and that’s what Uber built.

For Uber the decision is not all that hard. They are very well financed and have access to the top talent and resources needed to create, maintain, and update these kind of complex systems.

Since Uber’s goal is for transportation to have 99.99% availability for everyone, everywhere, it really makes sense to want to be able to control your costs as you scale to infinity and beyond.

But as you listen to the talk you realize the staggering effort that goes into making these kind of systems. Is this really something your average shop can do? No, not really. Keep this in mind if you are one of those cloud deniers who want everyone to build all their own code on top of the barest of bare metals.

Trading money for time is often a good deal. Trading money for skill is often absolutely necessary.

Given Uber’s goal of reliability, where out of 10,000 requests only one can fail, they need to run out of multiple datacenters. Since Cassandra is proven to handle huge loads and works across datacenters, it makes sense as the database choice.  

And if you want to make transportation reliable for everyone, everywhere, you need to use your resources efficiently. That’s the idea behind using a datacenter OS like Mesos. By statistically multiplexing services on the same machines you need 30% fewer machines, which saves money. Mesos was chosen because at the time Mesos was the only product proven to work with cluster sizes of 10s of thousands of machines, which was an Uber requirement. Uber does things in the large.

What were some of the more interesting findings?

  • You can run stateful services in containers. Uber found there was hardly any difference, 5-10% overhead, between running Cassandra on bare metal versus running Cassandra in a container managed by Mesos.

  • Performance is good: mean read latency: 13 ms and write latency: 25 ms, and P99s look good.

  • For their largest clusters they are able to support more than a million writes/sec and ~100k reads/sec.

  • Agility is more important than performance. With this kind of architecture what Uber gets is agility. It’s very easy to create and run workloads across clusters.

Here’s my gloss of the talk:

In the Beginning
Categories: Architecture

Software Development Conferences Forecast September 2016

From the Editor of Methods & Tools - Wed, 09/28/2016 - 15:01
Here is a list of software development related conferences and events on Agile project management ( Scrum, Lean, Kanban), software testing and software quality, software architecture, programming (Java, .NET, JavaScript, Ruby, Python, PHP), DevOps and databases (NoSQL, MySQL, etc.) that will take place in the coming weeks and that have media partnerships with the Methods […]

Discovering Your Team’s Risk Tolerance


Risk tolerance can be visualized as a curve. Above the curve represents a combination of high probability and a potential negative impact that will prevent the team from accepting the risk. Below the curve, the risk is deemed acceptable.  Outside of a few psychologically damaged individuals, everyone has a risk curve (whether they know it or not). On a team, everyone’s natural risk tolerance differs.  Complicating the discussion is that risk tolerance changes depending on context the person or team faces.  For example, at one point in my life riding my bike down a hill at top speed to see if I could slalom stop at the bottom was an acceptable risk.  I have the scars to prove I was that silly. Thinking back, I am not sure why I am alive today.  My risk tolerance is different now. While reminiscing about my unsafe days as a seven-year-old is fun, what is more important is to recognize that the same lesson can be applied to teams and in organizations. This leads us to the conclusion that we must talk about risk tolerance.  

Knowing and being able to predict a team’s risk tolerance is important.  For example, a few years ago I was asked to assess a team that had operated like clockwork for several years delivering superb value and quality work. However recently their quality had been questionable (their clients were finding significant defects in production).  There had been no significant personnel changes, nor was the type of work that they were doing significantly different. In the end, we determined that someone up the hierarchy had decided to remove quality from the team’s objectives (therefore how raises and promotions would be assessed) and doubled down on making dates and meeting budgets.  The change had the unintended consequence of changing the team’s risk tolerance curve.  On the surface at least, taking chances that might impact quality became less risky to the team, therefore more easy to change.  

Two relatively simple ways to approach a discussion of risk tolerance are:

  1. Every team and project has an implicit risk tolerance curve; some risks are acceptable and some are not.  Shifting team or organization’s risk tolerance from implicit to explicit requires explicit discussion.  In the project environment, the simplest approach is to hold an explicit discussion of risk. Specifically, ask participants to achieve consensus on whether examples of risks should be accepted or not.  It is powerful for the examples to be risks that have been recognized by the team and organization in the past, peppered with a few examples that are possible but more external to the team. The discussion will tend to touch on probability and potential impact and expose the participant’s perception of the risk. The team must end by agreeing on whether the team would accept or not accept the risk (accept can include taking on mitigation tasks).  While an explicit risk tolerance curve is not generated, the team will develop a clearer understanding of which risks it will tolerate and which it will not.  The examples also provide a set of analogies that can be used to assess risks as there are recognized. A handy set of analogies is EXTREMELY useful for every team member (Using analogies is a form of pattern recognition which is a cognitive bias).
  2. A more quantitative approach to quantify risk  uses a approach popularized by Michael Lant (any other quantitative scheme can be used), which assess each risk based on impact and probability to determine just how risky the risk is. Lant’s model equates a low impact/low probability  to a 1, and the highest probability/highest impact to 25. Based on the quantification, the team can quickly develop a consensus that any combination of impact and risk above a certain number can’t be accepted by the team.  In essence, the team says that up to a certain point they can mitigate or deal with a potential risk, but after that someone outside the team needs to own or indemnify the team from the potential that the risk turns into an issue or they can’t go forward.  The quantification provides a proxy for the line in the risk tolerance curve, and the rated risks can be used as a set of analogies for team members to do a real-time triage of newly discovered risks.

Both of these approaches represent a mechanism to have an explicit and structured discussion of risk tolerance.  Both approaches have an advantage over less structured approaches because they generate group knowledge and memory and artifacts that can be used to aid in using the team’s consensus and as a tool to reinforce that memory.  

Categories: Process Management