Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator&page=7' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

My Improved Travel Checklist

NOOP.NL - Jurgen Appelo - Wed, 06/22/2016 - 17:20
My Improved Travel Checklist

Last week in London, I wanted to shoot a video, but it appeared that I had left my camera’s memory cards at home. On my previous trip, it had been my Android tablet that I had forgotten to bring with me. The trip before that, it was my stack of local currency, my power adapters, my sunglasses, or whatever, which I hadn’t properly packed.

Sure, I have a travel checklist. But clearly, it had turned into a checklost. With at least 50 confirmed upcoming events around the world, it was time for me to redesign it.

Four Preferred Places

My original checklist was one large unorganized list of reminders. This regularly led to problems because, by scanning the list too quickly, I easily overlooked an item that was buried among all the other things that I only knew too well. So I divided the checklist into four parts:

  • Personal items (that I carry on my body)
  • Shoulder bag
  • Handbag (typically carry on luggage)
  • Extra bag (typically check in luggage)

Each travel item on my improved checklist now has a preferred place. This not only helps to keep the individual lists smaller, and easier to check more carefully, but it also helps me keep things in the right bag. I don’t want to have that situation again where I thought I had my universal adapters or chargers packed in the other bag (but found out later that I hadn’t).

(And preferred means that items can change places depending on context. For example, my keys will have moved to my shoulder bag before I arrive at the airport, while my passport may temporarily move to my pocket while suffering the security and customs rituals.)

Standard versus Extra

Another thing I noticed messing up my packing efforts was that some standard items are by default in my bags (such as passports and adapters), other extra things are by default out of my bags (such as clothes and toiletries), and some items had a Schrödinger-kind of existence, not clearly being in or out, until I opened my bags to have a look (such as device chargers and headphones).

After the reorganization, packing my bags with my improved checklist now consists of two distinct activities:

  1. Checking that all standard items are where they should be;
  2. Adding all extra items to the place where I want them.

Hopefully, this will save me some stress and headaches in the future.

For example, I once had to interrupt my trip to the airport because my passport was not in my bag: it was still under the scanner next to my computer. I always know that my passports are in my shoulder bag, except for the one or two times when, apparently, I was wrong. And thus, I made it a separate activity to check that I am not deceiving myself, thinking that the standard items are where they should be before adding all extra items. And a Schrödinger-kind of existence is not part of the improved design.

Optional Stuff

And then, of course, there are the optional items that mainly depend on the weather forecasts. I remember once nearly freezing to death in Helsinki because I had no warm coat or gloves. I was once drying my clothes in my hotel room in London because, stupidly, I had brought no umbrella. And more than once, I have been sweating in the sun because I was silly enough to bring only dark blue jeans and long-sleeve shirts.

Short versus Long trips

Finally, things can always change a bit depending on context.

On short trips (one or two nights), I don’t need to take the larger bag with me. But this means I must jam any books, running gear, and clean clothes into my hand luggage, which is not always possible, or else just leave some of it at home. And on long trips (ten or more nights), the large bag magically changes into an extra large suitcase, and this also changes what I can bring with me (usually a lot more clothes).

Well, there you have it: the philosophy behind my new-and-improved travel checklist. I include the full list below (as it is now).

Is there anything missing that you have on your travel checklist, and that may be useful for me as well?

House keys
Car keys

Shoulder bag – standard items
Credit cards and bank cards
Travel cards and loyalty cards
Pens and markers
Presentation clicker
Spare batteries
Memory sticks
Display adapter

Shoulder bag – extra items
Tablet + charger
Foreign currency

Handbag – standard items
Spare underwear, socks, and shirt
Spare medicine, contact lenses
GoPro camera
GoPro stick
GoPro batteries
GoPro charger
GoPro memory
GoPro microphone
Universal adapters
USB chargers
Business cards

Handbag – extra items
Notebook + charger
Gloves, scarf, ear muffs [IF forecast = cold]
Umbrella [IF forecast = wet]
Travel clothes [IF distance = long]

Large bag – standard items
Laundry bag

Large bag – extra items
Extra shoes
Running shoes and clothes
Book / giveaway
Fleece jacket [IF forecast = cold]
Warm socks and sweater [IF forecast = cold]
Raincoat [IF forecast = wet]
Short pants [IF forecast = hot]

photo (c) 2013 Chris Lott, Creative Commons 2.0

My new book Managing for Happiness is available from June 2016. PRE-ORDER NOW!

Managing for Happiness cover (front)

The post My Improved Travel Checklist appeared first on NOOP.NL.

Categories: Project Management

Product Owners and Learning, Part 1

When I work with clients, they often have a “problem” with product ownership. The product owners want tons of features, don’t want to address technical debt, and can’t quite believe how long features will take. ¬†Oh, and the POs want to change things as soon as they see them.

I don’t see this as problems.To me, this is all about learning. The team learns about a feature as they develop it. The PO learns about the feature once the PO sees it. The team and the PO can learn about the implications of this feature as they proceed. To me, this is a significant value of what agile brings to the organization. (I’ll talk about technical debt a little later.)

AgileRoadmap.copyright-1080x794One of the problems I see is that the PO sees the big picture. Often, the Very Big Picture. The roadmap here is a 6-quarter roadmap. I see roadmaps this big more often in programs, but if you have frequent customer releases, you might have it for a project, also.

I like knowing where the product is headed. I like knowing when we think we might want releases. (Unless you can do continuous delivery. Most of my clients are not there. They might not ever get there, either. Different post.)

Here’s the problem with the big picture. No team can deliver according to the big picture. It’s too big. Teams need the roadmap (which I liken to a wish list) and they need a ranked backlog of small stories they can work on now.

Example.AgileRoadmapOneQuarter In Agile and Lean Program Management, I have this picture of what an example roadmap might look like.

This particular roadmap works in iteration-based agile. It works in flow-based agile, too. I don’t care what a team uses to deliver value. I care that a team delivers value often. This image uses the idea that a team will release internally at least once a month. I like more often¬†if you can manage it.

Releasing often (internally or externally) is a function of small stories and the ability to move finished work through your release system. For now, let’s imagine you have a frictionless release system. (Let me know if you want a blog post about how to create a frictionless release system. I keep thinking people know what they need to do, but maybe it’s as clear as mud to ¬†you.)

The smaller the story, the easier it is for the team to deliver. Smaller stories also make it easier for the PO to adapt. Small stories allow discovery along with delivery (yes, that’s a link to Ellen Gottesdiener’s book). And, many POs have trouble writing small stories.

That’s because the PO is thinking in terms of feature sets, not features. I gave an example for secure login in How to Use Continuous Planning. It’s not wrong to think in feature sets. Feature sets help us create the big picture roadmap. And, the feature set is insufficient for the frequent planning and delivery we want in agile.

I see these problems in creating feature sets:

  • Recognizing the different stories in the feature set (making the stories small enough)
  • Ranking the stories to know which one to do first, second, third, etc.
  • What to do when the PO realizes the story or ranking needs to change.

I’ll address these issues in the next posts.

If you want to learn how to deliver what your customers want using agile and lean, join me in the next Product Owner workshop.

Categories: Project Management

Product Owners and Learning, Part 2

In Part 1, I talked about the way POs think about the big picture and the ranked backlog. The way to get from the big picture to the ranked backlog is via deliverables in the form of small (user) stories. See the wikipedia page about user stories. Notice that they are a promise for a conversation.

I talked about feature sets in the first post, so let me explain that here. A feature set is several related stories. (You might think of a feature set as a theme or an epic.) Since I like stories the team can complete in one day or less, I like those stories to be small, say one day or less. I have found that the smaller the story, the more feedback the team gets earlier from the product owner. The more often the PO sees the feature set evolving, the better the PO can refine the future stories. The more often the feedback, the easier it is for everyone to change:

  • The team can change how they implement, or what the feature looks like.
  • The PO can change the rest of the backlog or the rank order of the features.

I realize that if you commit to an entire feature set or a good chunk for an iteration, you might not want to change what you do in this iteration. If you have an evolving feature set, where the PO needs to see some part before the rest, I recommend you use flow-based agile (kanban). A kanban with WIP limits will allow you to change more often. (Let me know if that part was unclear.)

Now, not everyone shares my love of one-day stories. I have a client whose team regularly takes stories of size 20 or something like that. The key is that the entire team swarms on the story and they finish the story in two days, maybe three. When I asked him for more information, he explained this it in this way.

“Yes, we have feature sets. And, our PO just can’t see partial finishing. Well, he can see it, but he can’t use it. Since he can’t use it, he doesn’t want to see anything until it’s all done.”

I asked him if he ever had problems where they had to redo the entire feature. He smiled and said,

“Yes. Just last week we had this problem. Since I’m the coach, I explained to the PO that the team had effectively lost those three days when they did the “entire” feature instead of just a couple of stories. The PO looked at me and said, “Well, I didn’t lose that time. I got to learn along with the team. My learning was about flow and what I really wanted. It wasn’t a waste of time for me.”

“I learned then about the different rates of learning. The team and the PO might learn differently. Wow, that was a big thing for me. I decided to ask the PO if he wanted me to help him learn faster. He said yes, and we’ve been doing that. I’m not sure I’ll ever get him to define more feature sets or smaller stories, but that’s not my goal. My goal is to help him learn faster.”

Remember that PO is learning along with the developers and testers. This is why having conversations about stories works. As the PO explains the story, the team learns. In my experience, the PO also learns. It’s also why paper prototypes work well. Instead of someone (PO or BA or anyone) developing the flow, when the team develops the flow in paper with the PO/BA, everyone learns together.

Small stories and conversations help the entire team learn together.

Small features are about learning faster. If you, too, have the problem where the team is learning at a different rate than the PO, ask yourself these questions:

  • What kind of acceptance criteria do we have for our stories?
  • Do those acceptance criteria make sense for the big feature (feature set) in addition to the story?
  • If we have a large story, what can we do to show progress and get feedback earlier?
  • How are we specifying stories? Are we using specific users and having conversations about the story?

I’ve written about how to make small stories in these posts:

The smaller the story, the more likely everyone will learn from the team finishing it.

I’ll address ranking in the next post.

If you want to learn how to deliver what your customers want using agile and lean, join me in the next Product Owner workshop.

Categories: Project Management

How to Have Better Fridays

Recently, I’ve been teaching more people Agile Results. 

I teach them really fast, because I just focus on teaching them the most important tool in Agile Results:

Monday Vision, Daily Wins, Friday Reflection

But before I walk through, I share a quick story of how it all started.

Monday Vision was Born for Better Fridays

It was a warm, sunny, Friday afternoon.
My colleague and I were on our way to our favorite pizza place.
It was a beautiful day.  It should have been a great day.
But I felt like a beast of burden with the weight of the world on my back.
Our backlog was overflowing, we didn’t make it through what we thought we would, and we had been slogging away.
And for what?
Well, I caught myself looking in the rear view mirror, more than looking ahead.
Instead of feeling great, I felt like crap.
So I turned to my colleague and asked him how much we realistically have to spend on work when we get back.
We lied to each other and ourselves.  Then we got real.
We figured the best thing we could possibly do would be to prioritize the value we could deliver next week.
With that, we enjoyed our pizza, and when we got back to work, we figured out what a great next week would look like.
I never wanted us to have another Friday where we couldn’t go into weekend feeling good about what we had accomplished for the week.
And that’s how Monday Vision was born.

Monday Vision, Daily Wins, Friday Reflection for Better Fridays

One I tell that little story, people get it pretty quickly.  They‚Äôve been there.  They feel the pain.

They slogged away all week and at the end of the week, instead of feeling good about their achievements, they feel like they haven’t done enough.

They are never done.  They are overwhelmed. 

Instead of feeling like they earned their weekend for rest and relaxation, they feel guilty that they should work on their never-ending backlog and laundry-list of To-Dos.

Monday Vision for Better Fridays

So then I walk them through how to do Monday Vision, so they can have better Fridays.

On Mondays, imagine if it were Friday.  Really step into your future Friday and feel it.   What are Three Wins you really want to have under your belt?

What would you want to be able to say to your manager or to your team or to yourself, about what you accomplished or achieved for the week?

Get clarity on that.

Use that simple story of your Three Wins that you want to be able to talk about on Friday, as your way to prioritize your focus for the week on Monday.

Now you are doing ‚ÄúMonday Vision.‚ÄĚ

Daily Wins for Better Fridays

Each day, identify your Three Wins for that day.  This is the ‚ÄúDaily Wins‚ÄĚ practice.

You will have those days where your Three Wins might be, ‚ÄúGreat Breakfast,‚ÄĚ ‚ÄúGreat Lunch‚ÄĚ, ‚ÄúGreat Dinner.‚ÄĚ

There will be days that knock you down, and you wonder how you will get back up.

But then you will also have those days where you are on top of the world and your Three Wins for today will be magnificent.

You might even say, they will be your masterpiece.

Either way, get in the habit of starting your day by identifying Three Wins you want to achieve.

That will help you focus and prioritize all that you do in a more meaningful way for results that matter.

If you don’t know how to do this, just imagine if you were closing out your day, what are Three Wins that you want to be able to say you’ve achieved, either to you, your manager, your team or that someone special.

Friday Reflection for Better Fridays

Lastly, there is Friday Reflection.

Friday Reflection is your chance to dig deep and gain some new personal productivity insights.

Ask yourself, ‚ÄúWhat are three things going well?‚ÄĚ and ask yourself, ‚ÄúWhat are three things to improve?‚ÄĚ

Be honest with yourself about your answers.

You are the one that will win or lose from what you learn.

This is your chance to change any long-standing patterns of your personal productivity challenges.

Do you bite off more than you can chew?  Do you get randomized during the week?

Do you have a hard time figuring out what is actually valued?

Use what you learn to feed into next week.  This is your chance to change and practice your Growth Mindset.

Don’t expect any of these exercises to be easy, but they get easier with practice.

And that‚Äôs just it.  This isn‚Äôt a one-time trick.

This is a very precise set of productivity habits and practices that you can use for your lifetime to master time management, master your energy, master your motivation, and become more of what you are capable of.

Unleash your productivity, the Agile Way.

Best wishes for better Fridays.

Categories: Architecture, Programming

Beginning Agile Efforts: 4 Critical Factors For Team Dynamics

Team Dynamics

Team Dynamics

Teams are a common theme in the discussion of how Agile delivers value.  Teams are a collection of individuals that bring a range of capabilities.  Some people are specialists, others generalists and a very few are  renaissance people that are great at a wide range of activities.  Understanding the depth and breadth of capabilities in team members provides the team with the flexibility to dynamically allocate capabilities based on the technical context and business need (staff liquidity).  This is an incredibly powerful theory that only works if the team dynamics are conducive.  Team dynamics are an expression of how the team interacts with each other and those outside the team. When assessing the dynamics of a team there are many factors that are important; however, a few are more critical than others.

  1. Team Cohesiveness РCohesiveness is a reflection of how well a team sticks to together to accomplish a goal.  Cohesive teams know each other’s capabilities and capacity.  Cohesiveness is one factor that leads to psychological safety. A feeling of safety  allows team members to take risks without causing insecurity or fear of embarrassment if everything does not work out.  The cohesiveness of the team can be intuited from the length of time a team stays together and how invested team members are in the success of the team.
  2. Roles and Norms. Even the most Agile team is composed of different people playing different roles. Team performs well when team members recognize the roles that a team needs and then serve in those roles and are comfortable shifting roles when needed by the team. Roles include technical tasks, as well as roles such as leader, researcher and reviewer.  Clarity of roles norms helps build trust within the team and outside the team.  Trust reinforces team cohesion.  Roles and norms can be best assessed by an outsider by observing the teamwork. Alternately, retrospectives can be used to surface discussions of potential role and norm problems.
  3. Conflict Resolution. ¬†All teams will have conflict. Teams need to have a pallet of conflict resolution techniques. ¬†Conflict resolutions techniques can include simple ¬†talking, active listening, multi-voting, cost/benefit analysis or coaching. Teams need to prepare for inevitable conflict both by being aware of the signs of conflict and then have the wherewithal to deal ¬†the conflict. ¬†Observation can tell a coach a lot about a team’s ability deal with conflict. Alternately, one means to assess how a team will react is to give them a scenario and ask how they would solve the problem (this is a backdoor mechanism to teach resolution techniques). ¬†Teams with a good capability for conflict resolution will tend to be more cohesive and struggle less with roles.
  4. Impactful Goals.  Work that the team believes is important both to the organization and team members is motivating.  Motivation provides teams with a well of energy that can be invested in delivering value, resolving conflict, playing the roles that are needed and working together. The simplest way to know whether a team believes the goal they are pursuing is impactful is to ask and then to listen to the emotion in the answer. If there is no emotion, the team does not perceive what they are doing as important.

The four most critical drivers of Agile team dynamics can be assessed by observing and talking with the team.  The right dynamics will unlock a team’s potential; however most can’t get there without a help.  Coaching based on interacting and observing the team is a very powerful tool to give teams the tools needed to become more effective.  When beginning an Agile effort, spend the time and effort needed to tune every team to the four critical factors of team dynamics so they have the best start possible.

Categories: Process Management

Incentives and Deterrents for Starting Daily Scrums On Time

Mike Cohn's Blog - Tue, 06/21/2016 - 15:00

I recently emailed everyone who subscribes to my weekly tips a list of suggestions for ways to motivate team members to arrive on time to the daily scrum. For example, many teams have a rule that if you arrive late, you put a dollar in a jar as punishment for being late. Ideally the collected money is donated to a charity at the end of a project or after it reaches a certain amount.

I included a number of other techniques I’ve seen Scrum Masters use. Many were similar to paying a dollar in that they were punishments for being late. Wow, did I get called out for that. The general argument was that a Scrum Master should use an incentive for being on time rather than a punishment for being late.

That’s a really good point. But I have to say that in the vast majority of teams I’ve seen, they use a punishment rather than an incentive. Many of the punishments involve at least the potential for embarrassing the offender--sing a song, do a pushup, tell a joke, etc.

I’ve thought about why this is. It may be because it’s more natural (for many of us?) to harass the one rule-breaker than reward everyone else. Or maybe it’s because embarrassing the late arriver by making that person sing (for example) is more fun for everyone else.

Incentives Rather than Punishments

I do think, though, that it’s a great suggestion that we look for incentives rather than punishments. Besides, punishment may be too strong of a term. I was told by a couple of people that deterrent might be the better term.

In fact, Rob Dull pointed out an underlying benefit to using a deterrent, saying, “It's sooo healthy for people to be able to be silly/awkward in front of their teammates.”

Others considered the entire idea “stupid,” as one email I received succinctly put it. While others pointed out that the only real fix is to tap into an intrinsic motivation to get latecomers to arrive on time.

Isaias Fritsch took that type of approach by preparing a 30-minute presentation for the team on the importance of the daily scrum. He said that after that presentation, the team decided to come up with some changes to their daily scrum approach. The result of this has been that “the daily scrum is now way more interesting to everyone because relevant information is shared and everyone understands its importance/why we are doing it.”

Ted Morris Dawson had an interesting take on the punishment vs. incentive idea. One technique he has used was if all team members are on time for all daily scrums, then it’s the Scrum Master who does the embarrassing thing like sing, tell a joke or so on. Yikes!

So with the idea that we can help teams overcome the problem through either deterrents or incentives, I’ve split the suggestions I was emailed into lists of each.

  • Read a tongue-twister to the team (“Seventy-seven benevolent elephants” or “Scum-sucking scrum teams scrounge scrumptious scraps.”)
  • Dance alone without music for five seconds. Rob Dull reported, “It's quick, amusing for everyone, and it's a healthy encouragement of vulnerability.”
  • Bring coffee or a snack for the team the next day. Be careful--a few people emailed about gaining weight from this one! :)
  • Have a “tardy board” as Nadine Sullivan called it that tracks late arrivals or absences. Add rules like everyone is allowed one late arrival during some period. But after reaching some number of late arrivals or absences, the person has to bring something in such as a snack for the team.
  • Toss a beach ball, stuffed animal or some other item to the person who is to speak next. Throw, don’t toss, it at any late arrivers.
  • Buy the team an afternoon snack or tea.
  • Present a 30-minute knowledge sharing session on a topic. This was pointed out as being particularly helpful in team building.
  • Facilitate the next review or retrospective.
  • Facilitate the next daily scrum, meaning everyone is waiting for tomorrow’s meeting to start if today’s late arriver is late again.
  • Be the team’s “slave for a day” by doing things like getting snacks, soda and coffee for anyone who needed anything from the kitchen.
  • Take detailed, precise notes for the meeting.
  • Solve a tricky math problem on the board.
  • Buy a book for the office library. How about one one my books? :)
  • Close the door. A late arriver has to open it to enter, which focuses attention on the person.
  • Take on a short (perhaps 15-minute) administrative task the team needs done.
  • Simply put an indicator to each person’s name on a wall to show how frequently each team member was late.
  • Reward the first people to arrive with small bits of chocolate.
  • Collect a dollar or two for being late, but when the money is donated to a charity, donate it in the name of the person who arrived first at the meeting most often. Since charitable contributions are tax deductible in many countries, this gives incentives for being early and for not being late. Or as Michel Biron pointed out, it was both “carrot and stick.”
  • Give everyone who was prompt for each daily scrum a reward like a two-hour lunch or permission to come in late and leave early one day.
  • Everyone who was on time for each daily scrum goes to a movie during the day, one day during the next sprint.
  • Establish a policy of “if one of us is late, we’re all late,” and then provide a significant team reward to everyone if everyone is on time. One person told me he did this, but extended it to include being on time for all meetings. He knew that it would be hard to do, so he needed a significant motivation and they agreed on a nice lunch paid for by the company. As a consolation prize, if team members were only on time 90 percent of the time, everyone would receive a chocolate bar.

    I was told, “The result even surprised me. The team completely shifting their behavior. I was seeing team members loading up the digital agile board and setting up the conference connection for the remote team members minutes in advance of the standup. The team was holding each other accountable by friendly shoulder taps to make sure they all make it on time. I even observed a team member running across the building to grab someone out of a meeting to make sure she also makes it on time as well.”
  • Ice cream if everyone is on time every time.
  • A photo of the team holding a sign saying: “We did it!”
  • Start the meeting 15 minutes before everyone wants to go to lunch. Viktor Buzga reports that the company cafeteria in his company gets very crowded at 11:30 when it opens. Arrive at 11:30 and there’s no wait for lunch. Arrive two minutes later and there’s a 10-minute wait. So he starts daily scrums at 11:15. If the team finishes in 10-12 minutes, they’re perfectly timed for lunch. If not ...

So, there are many ways you can go about this. And, as I said in my emailed weekly tip, I love getting the chance to learn from all of you. I definitely learned some good techniques from this. Also, I’ve learned that I really should think more about incentives rather than deterrents.

Whatever you choose to do, you really need to make sure the entire team buys into it. The last thing you want is for someone to go to human resources complaining that you’re making him do a pushup for every minute he arrives late to a meeting.

But, for a team on which being late is a problem, these approaches can help address the problem. A common thread in many replies I received was that many of these incentives and deterrents work especially well because the entire team starts holding each other accountable. This is a huge benefit over just a Scrum Master holding people accountable.

What Do You Think?

What’s missing? What other techniques have you tried to encourage people to be on time for daily scrums? Please share your thoughts in the comments below.

Agile Hiring, Load Testing & Goal Management in Methods & Tools Summer 2016 issue

From the Editor of Methods & Tools - Tue, 06/21/2016 - 08:02
Methods & Tools has published its Summer 2016 issue that discusses hiring for agility, load testing scripts errors, managing with goals on every level and Behavior-Driven Development (BDD) with the open source Turnip tool. Methods & Tools is a free e-magazine for software developers, testers and project managers. * Hiring for Agility ‚Äď Mindset Matters […]

How to Keep flowtype Running and Report Errors on Save

Xebia Blog - Tue, 06/21/2016 - 08:00
We use flow from Facebook to run type checking on our codebase. When you run ‚Äėflow status‚Äô it starts a flow server in the background and keeps it running. That way after the first run the results of each next run are almost instant. The only thing currently lacking is a watch mode, but there

Control, Stability, Short Term, Long Term all needed for Success

Herding Cats - Glen Alleman - Tue, 06/21/2016 - 02:27
Single Track

When riding a single track like this one behind our neighborhood, I came to an understand of the tradeoffs between stability and control. In our conference sessions, we speak about how the Wright Brothers learned this concept as well.

Nationals Here's another trail being ridden by our son at Nationals in 2013. who's massively better than I will ever be. But same tradeoff between stability and control is needed to be ranked 33rd Cross Country and 23rd Short Track at DII in the nation, with team taking 2nd overall.

In both cases, nationally ranked and rank amaetur, control of bike is the key. Stability is a relative term, depending on speed, terrain, skill, risk taking tolerance,  experience, technical equipment, and other intangible factors.

We speak at conference where program planning and controls is the topic. Starting 2 years ago, our foundation for speaking about managing in the presence of  uncertanty is the Wright Brothers.

Most earlier experimenters in powered flight focused only on one or two of the primary problems - (a) A set of lifting surfaces, or wings, (b) A method of balancing and controlling the aircraft, and (c) A means of propulsion -  and did not consider the final design from the outset. The Wrights recognized that each of these areas had to be successfully addressed to build a working airplane. They believed that the aerodynamic and propulsion problems would be comparatively easier to solve, so they first concentrated on how to maintain balance and control.

Many believed that air currents were too swift and unpredictable for human reflexes. Therefore, an aircraft had to be inherently stable for the pilot to be able to maintain control.¬†Because of the Wrights‚Äô extensive experience with the bicycle‚ÄĒa highly unstable but controllable machine (just like the mountain bike)‚ÄĒthey saw no reason why an airplane could not be unstable yet controllable as well.

The notion of Control is missed used in software development projects. Misused and misunderstood. Control inside the upper and lower control bounds is a critical success factor. What are those upper and lower control bounds? Good question. They have to be estimated in the start. They become cleared as the project progresses. They have to be adjusted as new information emerges.

In projects, the question for Stability is a false quest. All project work is uncertain. Stability is short lived. New inputs arrive every day. Just like new inputs arrive while riding down the flat trail for cruising around the open space in the neighborhood or more so on the bumpy high speed trail of collegiate racing.

Even if the trail is defined in from of you, the small disruptions and many times big distributions input your control of the bike. The key is Control over Stability. Staying in control will get you across the finish line, down the trail, and home again to ride another day.

Skills of Maintaining Control on the Mountain Bike and the Project

Starting from our house there is a nice loop Left Hand Trail, that is easy, has a few climbs, and a few descents, uncrowded, and provides a nice view of the small hills before the real mountains start . Combine that with the Eagle Trail, Sage Trail, and North Rim Trail and it's a nice 7 mile loop

  1. Short term feedback - what's happening right in front of me? What do I need to do NOW to maintain stability, to survive for the next 10 feet on the trail 
    • On projects,¬†short term feedback is for the¬†next day or week.
    • What's coming due?
    • A Plan of the Week or even a Plan of the Day is ¬†very useful.
  2. Long term feedback - I see things coming on the trail. A big drop. A huge climb. What am I going to do to prepare for both? Do I have a plan of action to get through these?
    • On projects, I need to see this coming before I am¬†Overcome¬†By¬†Events¬†(OBE). The term OBE is used often in our domain. It says, they didn't see it coming. They didn't have a plan to respond to emerging events.
    • The naive term in agile of¬†respond to emerging requirements means you have to have a plan to meet that emerging requirement. If you don't, just like on¬†the bike, you're going to end ¬†up on the side¬†to the trail, landing on a cactus¬† or worse a snake where we live.
  3. Corrective actions - what corrective actions am I capable of performing? On the bike, I must know my limits. Can I power through the steep climb? Maybe if it's short. No likley if it's 1/2 mile at 18% grade.
    • On projects knowing the limits of the team is a critical success factor.
    • Who can work the problem with the most rapid response? Who's got skills and experience needed to¬†power through till we get back in control
  4. Alternative choices - as the trial unfolds in front of me, choices present themselves. Some are optional, some are mandatory alternatives. Other riders coming up the trail have to be addressed. If their coming up hill and I'm going down hill, they have the right-of-way. I can easily start again going down hill. Stopping and starting again going up hill is a real pain. Obstacles on the trail have to be dealt with. In the spring branches hang in th trail from the trees and bushes. Many have sharp leaves or thorns, riding through them is painful.
    • On project having alternatives is also mandatory.¬†
    • Nothing ever goes right as planned. Having an alternative ready to go, means when trouble arrives, there is no delay in making a choice to take another path.
  5. Estimate what's right in front of you - the trail is rough, bumpy, rutted, off camber, muddy, thorny, and many times some animal runs across my path - rabbitt, prairie dog, a snake. A quick estimate is needed to decide what to do. Keep going, speed up, slow down, stop? All depends on the situation.
    1. Estimating on projects is part of the close loop control system.
    2. Both project controls and business control depend on making estimates in the presence of uncertanty about future outcomes.
    3. These estimates keep the project Green, just like estimates on the bike keep me rubber side down.
  6. Estimates of needed for solutions coming up the trail - with short term estimating there are a limited number of choices, bounded by the physical terrain. For longer term choices, there are more options. Do I turn left at the next opportunity, go over the mesa, then back down to rejoin the trial at the parking lot of the hiking trailhead? 
    • For projects a longer view of what's coming is needed.
    • Points of integration or incorporation need to be in the Plan.
    • Alternative Points of Incorporation (APOI) are needed as well.
    • Many time these APOI are part of the master plan, since external facilities may be booked, resources become scarce, technology changes.¬†
    • A ¬†Plan B is needed as part of the normal process
  7. Estimates of needed resources - food, water, spare equipment are part of the normal riding gear. For really big rides, Bear Spray is common, since we live in Bear Country. Energy resources must be managed. Knowing how fast you can go, when is the next resting spot, next shady spot?
    • The resource plan for the project is needed no matter the method used to develop the project.
    • What skills, how many of those skills, the availability of those skills?¬†
  8. Estimates to complete - rarely do we go out without some plan of when we'll be back. This means some estimate of when we'll be back. Telling wife's we're going up to Hygiene and we'll be back in 2 hours is always good protocol. 
    • On projects knowing something about when you plan to finish is part of managing the project.
    • Anyone working on a project that doesn't have some type of deadline is working on a de minimis project - it's too small for anyone to care about when you'll be down.
    • Same for the budget
    • Same for the needed technical performance.
  9. A Plan B and even a Plan C - planning in depth is a good idea when riding anywhere. The longer the ride, the steeper the terrain, the higher the exposure, means having more plans to deal with the emerging situations
    • Without a plan, you're driving in the dark with the lights off.
    • What ever comes up is likely to be a surprise.
    • While surprises are nice for birthday parties, surprises 25 miles out from home, with a 25 mile return can turn unpleasant real quick.
    • Planning in depth is always a good idea.
    • Knowing the¬†Value at Risk¬†defines the magnitude and detail of the plan. If the impact is minimal and can be handled with a few changes, no problem. If the impact is major, having a detailed plan when it occurs is the basis of success
    • Risk Management is How Adults Manage Projects Tim Lister should never be forgotten
  10. Knowledge of capacity for work - how big is a big ride? Good question. When we have friends ask hey want to go for a ride, I have to make sure I have the capacity for work. Our neighborhood has many cyclist. A Silver Medalist rider. A semi-pro road rider. A former pro road rider. A semi-pro mountain biker all the way down to casual riders like me and our group. Know what the ride will demand and what I'm capable of doing is a starting point. Along the way reassessing both those informs decisions about routes, pace, and overall strategy to get home in one piece
    • On projects, if we don't our capacity for work, we can't make informed estimates if we can get to the end in¬†one piece.
    • If we¬†bite off more than we can chew, then we're going to be late and over budget before we start.
    • Know our capacity for work comes from past performance.
    • But it also means estimating what we can do, knowing that past performance, and he future demands for work.
Related articles Late Start = Late Finish Eyes Wide Shut - A View of No Estimates All The World's A Random Process Project Risk Management, PMBOK, DoD PMBOK and Edmund Conrow's Book Architecture -Center ERP Systems in the Manufacturing Domain Staying on Plan Means Closed Loop Control Strategy is Not the Same as Operational Effectiveness Estimating Processes in Support of Economic Analysis Qualitative Risk Management and Quantitative Risk Management
Categories: Project Management

Quick Summary of Estimating Advice

Herding Cats - Glen Alleman - Mon, 06/20/2016 - 23:23

There's lots of misinformation going around again in the #NoEstimates community

  • We can't estimate things we've never done before - this is simply not true. There is not much that hasn't been done before in some form. If you truly haven't done the work before, you're probably not the right person to be estimating for those wanting to pay you.¬†
  • Estimates are guesses, because we don't know that the future is - this is a fundamental error,¬†either of commision¬†or¬†omission) on how estimates are made.
  • Just deliver often in small chunk and we won't need to estimate - this of course ignores the need to produce an Estimate To Complete and an Estimate At Completion.
  • Estimates are a waste, we should be coding to generate value for the customer -¬†the¬†customer has a fiduciary¬†need to know how much it will cost to get that value and when that value will be delivered. This is core business. If there is no need to know estimated cost and schedule, the project is likely a de minimis effort, or the customer has enough money to burn to not care about how much and when.
  • Deadlines kill innovation and reduce quality - this is only true of your a really bad planner. All project work is probabilistic. This probabilistic (and statistical) behaviour mandates a plan to manage in the presence of uncertainty. Bout reducible and irreducible uncertainties have specific¬†handling plans. Either¬†margins¬†to protect from the naturally occurring ¬†variances or specific¬†buy down¬†activities to protect against probabilistic occurrences of undesirable outcomes.¬†

Here's the starting point to clarify and debunk those concepts

  • How To Estimate Almost Any Software Deliverable in¬†90 Seconds¬†- this the starting point for learning how to estimate for all software projects. Is it a¬†bigger than a breadbox, or¬†Twenty Questions approach. It's actually a Binary Search approach, that will¬†get you an¬†85%¬†confidence number in 90- seconds or¬†less. No more excuses for not estimating anything, including things you know nothing about - assuming you are a developer with any experience in the domain. If you have no experience in the domain, then you shouldn't be estimating to begin with - go find someone who is.

Here's some more details, once it's been confirmed you have domain experience

  • How Do You Estimate Work That's Never Been Done Before?¬†- This is a common lament in the #Noestimates community.¬†We've never done this before, so how can we¬†possibly¬†estimate how long it will take?¬†Lot's of ways - Reference Classes, Parametric models. Monte Carlo models of activity networks.
  • Let's Stop Guessing and Learn How to Estimate¬†- estimates are NOT guesses. Estimate - it is an approximation of some value for some purpose, even if the data is incomplete, uncertain, or even unstable. Typically the estimate is derived from a statistical source of data - either observed or referenced in some way.
  • How ¬†to Estimate if you really want to¬†(Updated)¬†- the key here¬†is if you want to. There is NO reason to not estimate. Estimates are need to make decisions in presence of uncertainty.¬†
  • How to Estimate Almost Anything if You Really Want To¬† - this starts with¬†what's the¬†coastline¬†of England¬†problem and uses a similar red herring about a hike down the coast of California.

In the end it's simple. Anyone speaking about the difficulties or the waste of estimating is likely on the spend side of the software development equation. Those on the pay side know (or should know) they needed estimates. They can get the estimates from those most qualified to provide them - the developes. Or they can just make them up. Probably better to have a bottom up estimate.

Related articles Humpty Dumpty and #NoEstimates Want To Learn How To Estimate? Let's Stop Guessing and Start Estimating
Categories: Project Management

Notifications in Android N

Android Developers Blog - Mon, 06/20/2016 - 20:10

Posted by Ian Lake, Developer Advocate

Android notifications are often a make-or-break interaction between your Android app and users. To provide a better user experience, notifications on Android N have received a visual refresh, improved support for custom views, and expanded functionality in the forms of Direct Reply, a new MessagingStyle, and bundled notifications.

Same notification, new look

The first and most obvious change is that the default look and feel of notifications has significantly changed. Many of the fields that were spread around the notifications have been collapsed into a new header row with your app’s icon and name anchoring the notification. This change ensured that the title, text, and large icon are given the most amount of space possible and, as a result, notifications are generally slightly larger now and easier to read.

Given the single header row, it is more important than ever that the information there is useful. When you target Android N, by default the time will be hidden - if you have a time critical notification such as a messaging app, you can re-enable it with setShowWhen(true). In addition, the subtext now supersedes the role of content info and number: number is never shown on Android N devices and only if you target a previous version of Android and don’t include a subtext will content info appear. In all cases, ensure that the subtext is relevant and useful - don’t add an account email address as your subtext if the user only has one account, for example.

Notification actions have also received a redesign and are now in a visually separate bar below the notification.

You’ll note that the icons are not present in the new notifications; instead more room is provided for the labels themselves in the constrained space of the notification shade. However, the notification action icons are still required and continue to be used on older versions of Android and on devices such as Android Wear.

If you’ve been building your notification with NotificationCompat.Builder and the standard styles available to you there, you’ll get the new look and feel by default with no code changes required. Better Support for Custom Views

If you’re instead building your notification from custom RemoteViews, adapting to any new style has been challenging. With the new header, expanding behavior, actions, and large icon positioning as separate elements from the main text+title of the notification, we’ve introduced a new DecoratedCustomViewStyle and DecoratedMediaCustomViewStyle to provide all of these elements, allowing you to focus only on the content portion with the new setCustomContentView() method.

This also ensures that future look and feel changes should be significantly easier to adapt to as these styles will be updated alongside the platform with no code changes needed on the app side.

Direct Reply

While notification actions have already been able to launch an Activity or do background work with a Service or BroadcastReceiver, Direct Reply allows you to build an action that directly receives text input inline with the notification actions.

Direct Reply uses the same RemoteInput API - originally introduced for Android Wear - to mark an Action as being able to directly receive input from the user.

The RemoteInput itself contains information like the key which will be used to later retrieve the input and the hint text which is displayed before the user starts typing.

// Where should direct replies be put in the intent bundle (can be any string)
private static final String KEY_TEXT_REPLY = "key_text_reply";

// Create the RemoteInput specifying this key
String replyLabel = getString(R.string.reply_label);
RemoteInput remoteInput = new RemoteInput.Builder(KEY_TEXT_REPLY)

Once you’ve constructed the RemoteInput, it can be attached to your Action via the aptly named addRemoteInput() method. You might consider also calling setAllowGeneratedReplies(true) to enable Android Wear 2.0 to generate Smart Reply choices when available and make it easier for users to quickly respond.

// Add to your action, enabling Direct Reply for it
NotificationCompat.Action action =
    new NotificationCompat.Action.Builder(R.drawable.reply, replyLabel, pendingIntent)

Keep in mind that the pendingIntent being passed into your Action should be an Activity on Marshmallow and lower devices that don’t support Direct Reply (as you’ll want to dismiss the lock screen, start an Activity, and focus the input field to have the user type their reply) and should be a Service (if you need to do work on a separate thread) or BroadcastReceiver (which runs on the UI thread) on Android N devices so as the process the text input in the background even from the lock screen. (There is a separate user control to enable/disable Direct Reply from a locked device in the system settings.)

Extracting the text input in your Service/BroadcastReceiver is then possible with the help of the RemoteInput.getResultsFromIntent() method:

private CharSequence getMessageText(Intent intent) {
    Bundle remoteInput = RemoteInput.getResultsFromIntent(intent);
    if (remoteInput != null) {
        return remoteInput.getCharSequence(KEY_TEXT_REPLY);
    return null;

After you’ve processed the text, you must update the notification. This is the trigger which hides the Direct Reply UI and should be used as a technique to confirm to the user that their reply was received and processed correctly.

For most templates, this should involve using the new setRemoteInputHistory() method which appends the reply to the bottom of the notification. Additional replies should be appended to the history until the main content is updated (such as the other person replying).

However, if you’re building a messaging app and expect back and forth conversations, you should use MessagingStyle and append the additional message to it.


We’ve optimized the experience for displaying an ongoing conversation and using Direct Reply with the new MessagingStyle.

This style provides built-in formatting for multiple messages added via the addMessage() method. Each message supports passing in the text itself, a timestamp, and the sender of the message (making it easy to support group conversations).

builder.setStyle(new NotificationCompat.MessagingStyle("Me")
    .setConversationTitle("Team lunch")
    .addMessage("Hi", timestampMillis1, null) // Pass in null for user.
    .addMessage("What's up?", timestampMillis2, "Coworker")
    .addMessage("Not much", timestampMillis3, null)
    .addMessage("How about lunch?", timestampMillis4, "Coworker"));

You‚Äôll note that this style has first-class support for specifically denoting messages from the user and filling in their name (in this case with ‚ÄúMe‚ÄĚ) and setting an optional conversation title. While this can be done manually with a BigTextStyle, by using this style Android Wear 2.0 users will get immediate inline responses without kicking them out of the expanded notification view, making for a seamless experience without needing to build a full Wear app.

Bundled Notifications

Once you’ve built a great notification by using the new visual designs, Direct Reply, MessagingStyle, and all of our previous best practices, it is important to think about the overall notification experience, particularly if you post multiple notifications (say, one per ongoing conversation or per new email thread).

Bundled notifications offer the best of both worlds: a single summary notification for when users are looking at other notifications or want to act on all notifications simultaneously and the ability to expand the group to act on individual notifications (including using actions and Direct Reply).

If you’ve built stacking notifications for Android Wear, the API used here is exactly the same. Simply add setGroup() to each individual notification to bundle those notifications together. You’re not limited to one group, so bundle notifications appropriately. For an email app, you might consider one bundle per account for instance.

It is important to also create a summary notification. This summary notification, denoted by setGroupSummary(true), is the only notification that appears on Marshmallow and lower devices and should (you guessed it) summarize all of the individual notifications. This is an opportune time to use the InboxStyle, although using it is not a requirement. On Android N and higher devices, some information (such as the subtext, content intent, and delete intent) is extracted from the summary notification to produce the collapsed notification for the bundled notifications so you should continue to generate a summary notification on all API levels.

To improve the overall user experience on Android N devices, posting 4 or more notifications without a group will cause those notifications to be automatically bundled.

N is for Notifications

Notifications on Android have been a constant area of progressive enhancement. From the single tap targets of the Gingerbread era to expandable notifications, actions, MediaStyle, and now features such as Direct Reply and bundled notifications, notifications play an important part of the overall user experience on Android.

With many new tools to use (and NotificationCompat to help with backward compatibility), I’m excited to see how you use them to #BuildBetterApps

Follow the Android Development Patterns Collection for more!

Categories: Programming

The Technology Behind Apple Photos and the Future of Deep Learning and Privacy

There’s a war between two visions of how the ubiquitous AI assisted future will be rendered: on the cloud or on the device. And as with any great drama it helps the story along if we have two archetypal antagonists. On the cloud side we have Google. On the device side we have Apple. Who will win? Both? Neither? Or do we all win?

If you would have asked me a week ago I would have said the cloud would win. Definitely. If you read an article like Jeff Dean On Large-Scale Deep Learning At Google you can’t help but be amazed at what Google is accomplishing. Impressive. Wide ranging. Smart. Systematic. Dominant.

Apple has been largely absent from the trend of sprinkling deep learning fairy dust on their products. This should not be all that surprising. Apple moves at their own pace. Apple doesn’t reach for early adopters, they release a technology when it’s a win for the mass consumer market.

There’s an idea because Apple is so secretive they might have hidden away vast deep learning chops we don’t even know about yet. We, of course, have no way of knowing.

What may prove more true is that Apple is going about deep learning in a different way: differential privacy + powerful on device processors + offline training with downloadable models + a commitment to really really not knowing anything personal about you + the deep learning equivalent of perfect forward secrecy.

Photos vs Photos
Categories: Architecture

SPaMCAST 399 ‚Äď Storytelling and The Big Picture, Manifesto, Deliberate Practice



Listen Now

Subscribe on iTunes                   Check out the podcast on Google Play Music

The Software Process and Measurement Cast 399 features our essay titled, Storytelling: Developing The Big Picture for Agile Efforts. Agile reminds us that the focus of any set of requirements needs to be on an outcome rather than a collection of whats and whos.  Storytelling is a powerful tool to elevate even the most diehard requirements analyst from a discussion of individual requirements to a discussion of outcomes. Before we can generate a backlog composed of features, epics, and user stories, we need to understand the big picture.

Our second column is a visit to Gene Hughson’s Form Follows Function Blog.  We discussed an entry titled A Meaningful Manifesto for IT.  Do we need a manifesto to know that how well we are meeting the needs of our customers is a reflection of how fit IT is for purpose? Perhaps the answer is yes, if for no other purpose than to ensure we make sure that what we deliver is not a waste of money.

Anchoring the cast this week is the Software Sensei, Kim Pries.  Kim discusses the role of deliberate practice in increasing the capability and capacity of teams. Kim’s provides practical advice on improving team performance.

Re-Read Saturday News

This week we begin the Re-read Saturday of  Kent Beck’s XP Explained, Second Edition with a discussion of the Preface and Chapter 1.  These sections provide a definition of XP and context for the diving into the principles and techniques. Using the link to XP Explained when you buy your copy to read along will support both the blog and podcast. Visit the Software Process and Measurement Blog (www.tcagley.wordpress.com) to catch up on past installments of Re-Read Saturday.


The next Software Process and Measurement Cast, #400!, features our interview with Jim Benson. Jim and I talked about personal Kanban, micromanagement, work-in-process limits, pattern matching, pomodoro and more. This was a marvelous interview to commemorate our first 400 shows!

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: ‚ÄúThis book will prove that software projects should not be a tedious process, for you or your team.‚ÄĚ Support SPaMCAST by buying the book here. Available in English and Chinese.

Categories: Process Management

SPaMCAST 399 ‚Äď Storytelling and The Big Picture, Manifesto, Deliberate Practice

Software Process and Measurement Cast - Sun, 06/19/2016 - 22:00

The Software Process and Measurement Cast 399 features our essay titled, Storytelling: Developing The Big Picture for Agile Efforts. Agile reminds us that the focus of any set of requirements needs to be on an outcome rather than a collection of whats and whos.  Storytelling is a powerful tool to elevate even the most diehard requirements analyst from a discussion of individual requirements to a discussion of outcomes. Before we can generate a backlog composed of features, epics, and user stories, we need to understand the big picture.

Our second column is a visit to Gene Hughson’s Form Follows Function Blog.  We discussed an entry titled A Meaningful Manifesto for IT.  Do we need a manifesto to know that how well we are meeting the needs of our customers is a reflection of how fit IT is for purpose? Perhaps the answer is yes, if for no other purpose than to ensure we make sure that what we deliver is not a waste of money.

Anchoring the cast this week is the Software Sensei, Kim Pries.  Kim discusses the role of deliberate practice in increasing the capability and capacity of teams. Kim’s provides practical advice on improving team performance.


Re-Read Saturday News

This week we begin the Re-read Saturday of  Kent Beck’s XP Explained, Second Edition with a discussion of the Preface and Chapter 1.  These sections provide a definition of XP and context for the diving into the principles and techniques. Using the link to XP Explained when you buy your copy to read along will support both the blog and podcast. Visit the Software Process and Measurement Blog (www.tcagley.wordpress.com) to catch up on past installments of Re-Read Saturday.



The next Software Process and Measurement Cast, #400!, features our interview with Jim Benson. Jim and I talked about personal Kanban, micromanagement, work-in-process limits, pattern matching, pomodoro and more. This was a marvelous interview to commemorate our first 400 shows!


Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: ‚ÄúThis book will prove that software projects should not be a tedious process, for you or your team.‚ÄĚ Support SPaMCAST by buying the book here. Available in English and Chinese.


Categories: Process Management

Unix: Find all text below string in a file

Mark Needham - Sun, 06/19/2016 - 09:36

I recently wanted to parse some text out of a bunch of files so that I could do some sentiment analysis on it. Luckily the text I want is at the end of the file and doesn’t have anything after it but there is text before it that I want to get rid.

The files look like this:

# text I don't care about
= Heading of the bit I care about
# text I care about

In other words I want to find the line that contains the Heading and then get all the text after that point.

I figured sed was the tool for the job but my knowledge of the syntax was a bit rusty. Luckily this post served as a refresher.

Effectively what we want to do is delete from the beginning of the file up until the line after the heading. We can do this with the following command:

$ cat /tmp/foo.txt 
# text I don't care about
= Heading of the bit I care about
# text I care about
$ cat /tmp/foo.txt | sed '1,/Heading of the bit I care about/d'
# text I care about

That still leaves an extra empty line after the heading which is a bit annoying but easy enough to get rid of by passing another command to sed that strips empty lines:

$ cat /tmp/foo.txt | sed -e '1,/Heading of the bit I care about/d' -e '/^\s*$/d'
# text I care about

The only difference here is that we’re now passing the ‘-e’ flag to allow us to specify multiple commands. If we just pass them sequentially then the 2nd one will be interpreted as the name of a file.

Categories: Programming

Unix: Split string using separator

Mark Needham - Sun, 06/19/2016 - 08:22

I recently found myself needing to iterate over a bunch of ‘/’ separated strings on the command line and extract just the text after the last ‘/’.

e.g. an example of one of the strings


I wanted to write some code that could split on ‘/’ and then pick the 3rd item in the resulting collection.

One way of doing this is to echo the string and then pipe it through cut:

$ string="A/B/C"
$ echo ${string} | cut -d"/" -f3

or awk:

$ echo ${string} | awk -F"/" '{ print $3}'

I don’t like having to echo the string – it feels a bit odd so I wanted to see if there was a way to do the parsing more ‘inline’.

I came across this post which explains how to change the internal field separator (IFS) on the shell and then parse the string into an array using read. I gave it a try:

$ IFS="/" read -ra ADDR <<< "${string}"; echo ${ADDR[2]}

Works! We can even refer to the last item in the array using -1 instead of it’s absolute position:

$ IFS="/" read -ra ADDR <<< "${string}"; echo ${ADDR[-1]}

I’d not come across this use of the ‘read’ function before. The key is the ‘-a’ parameter. From the man page:

-a aname
The words are assigned to sequential indices of the array variable aname,
starting at 0. All elements are removed from aname before the assignment.
Other name arguments are ignored.

So we’re resetting the internal field separator and then reading the string into another variable as an array split on the ‘/’ character.

Pretty neat although now it’s longer than the original command and I’m sure I’ll forget the syntax.

Further down the page is another suggestion which seems even harder to remember but is much shorter:

$ echo ${string##*/} 

This drops from the beginning of the string up until the last occurrence of ‘/’ which is exactly what we want.

This way is the nicest and doesn’t require any echoing if we just want to assign the result to a variable. The echo is only used here to see the output.

Categories: Programming

Android N APIs are now final, get your apps ready for Android N!

Android Developers Blog - Sun, 06/19/2016 - 02:41

Posted by Dave Burke, VP of Engineering

As we put the finishing touches on the next release of Android, which will begin to roll out to consumers later this summer, we’re releasing the 4th Developer Preview of Android N, including the Android N final SDK. And thanks to your continued feedback over the last three releases, all of the APIs are now final as well. If you’ve already enrolled your device in the Android Beta Program, (available at android.com/beta) you will receive an update to this Developer Preview shortly.

Get your apps ready for Android N

The final SDK for Android N is now available for download through the SDK Manager in Android Studio. It gives you everything you need to develop and test against the official APIs in the Android N platform. Once you’ve installed the final SDK, you can update your project’s compileSdkVersion to API 24 to develop with the Android N APIs and build and test on the new platform, for new features such as Multi-window support, direct-reply notifications, and others. We also recommend updating your app’s targetSdkVersion to API 24 to opt-in and test your app with Android N specific behavior changes. For details on how to setup your app with the final SDK, see Set up the Preview. For details on API level 24 check out the API diffs and the updated API reference, now hosted online.

Along with the Android N final SDK, we’ve also updated the Android Support Library to 24.0.0. This allows you to use multi-window and picture-in-picture callbacks, new notification features, methods for supporting Direct Boot, and new MediaBrowser APIs in a backward compatible manner.

Publish your apps to alpha, beta or production channels in Google Play

Now that you have a final set of APIs, you can publish updates compiling with, and optionally targeting, API 24 to Google Play. You can now publish app updates that use API 24 to your alpha, beta, or even production channels in the Google Play Developer Console. In this way, you can test your app’s backward-compatibility and push updates to users whose devices are running Developer Preview 4.

To make sure that your updated app runs well on Android N, as well as older versions, a common strategy is to use Google Play‚Äôs beta testing feature to get early feedback from a small group of users -- including developer preview users — and then do a staged rollout as you release the updated app to all users.

How to Get Developer Preview 4

Developer Preview 4 includes updated system images for all supported Preview devices as well as for the Android emulator. If you are already enrolled in the Android Beta program, your devices will get the Developer Preview 4 update right away, no action is needed on your part. If you aren’t yet enrolled in Android Beta, the easiest way to get started is by visiting android.com/beta and opt-in your eligible Android phone or tablet -- you’ll soon receive this (and later) preview updates over-the-air. As always, you can also download and flash this update manually. The N Developer Preview is available for Nexus 6, Nexus 5X, Nexus 6P, Nexus 9, and Pixel C devices, as well as General Mobile 4G [Android One] devices and the Sony Xperia Z3.

Thanks so much for all of your feedback so far. Please continue to share feedback or requests either in the N Developer Preview issue tracker, N Preview Developer community, or Android Beta community as we work towards the consumer release later this summer. We’re looking forward to seeing your apps on Android N!

Categories: Programming

Re-Read Saturday: Extreme Programming Explained: Embrace Change Second Edition Week 1

XP Explained

This week we begin the re-read of Kent Beck’s Extreme Programing Explained, Second Edition (2005). I remember the first time I read Extreme Programing Explained (First Edition), it was flying back and forth between Cleveland and Los Angeles.  It took two flights and then a lot of time to consider. XP Explained provides not only an explanation of Extreme Program, but also a philosophy for XP and Agile.  The 189 pages are organized into two sections and 25 relatively short chapters and are crammed with information and ideas. I used a whole highlighter on the book during my first read (the pages are now florescent yellow.  The book was written by Kent Beck, one of my heroes even before Agile.  The Second Edition now has Cynthia Andres joining Mr. Beck.

Extreme Programming Explained: Embrace Change, Second Edition begins with two forwards and a preface. ¬†I freely admit that my natural tendency is to plow through reading forewords and prefaces. ¬†In this case, it was interesting to compare the difference between the forewords from the first and second editions, both written by Erich Gamma. ¬†¬†The main point is that the second edition is a full rewrite. ¬†Gamma says¬†that Beck applied the XP paradigm of ‚Äústay aware, adapt, change.‚ÄĚ

Extreme Programming reflects a philosophy of adapting and changing based on the environment that the developer is operating in.  In the Preface, Beck rephrased the message that he used to open the first edition.  The message Beck opens the book with is that:

  1. No matter the circumstances you can always improve.
  2. You can always start improving with yourself.
  3. You can always start improving today. 

The message of XP Explained is one of hope, discipline, deliberate practice, and improvement.

Chapter 1 – What is XP?

Chapter 1 is the context from which the rest of the book builds. ¬†During my original read, I remember breezing through this chapter to get to the ‚Äúbeef.‚ÄĚ However, over the years, I have returned to this chapter to reground my understanding of Beck‚Äôs vision. ¬†In the second edition, Beck builds on his initial definition: ‚ÄúXP is a lightweight methodology for small-to-medium-sized teams developing software in the face of vague or rapidly changing requirements.‚ÄĚ In the second edition, Beck provides a list of attributes to augment the original definition.¬†

  • XP is lightweight
  • XP is a methodology
  • XP can work with teams of any size
  • XP adapts to vague or rapidly changing requirements.

The obvious difference is the recognition that XP can be used with teams of any size. As interesting is the implicit recognition in the last attribute that XP is can be applied even when requirements are well understood.  The newer definition recognizes that XP is both a style and philosophy of software development that can be applied to any scenario. The overarching philosophy provides a container to hold a set of complementary principles that support a set of techniques.  

Beck suggests philosophically that in XP you don’t prepare for failure, rather you play all out to win. ¬†XP is a philosophy and methodology that is based on a¬†mentality of sufficiency. Rather than focusing on the constraints, a project lives within, XP focuses on delivering software and continuous improving regardless of constraints. All efforts and projects have constraints, spending too much time worrying about constraints is a distraction from the goal of the work. ¬†

XP addresses many of the risks in the software development process, such as:

  1. Schedule slippage by leveraging short release cycles.
  2. Avoidance of not delivering anything due to project cancellation by the use of small releases that make business sense.  
  3. Chances that the system goes ‚Äúsour‚ÄĚ by leveraging continuous automated testing.
  4. High defect rates by having both programmers and customers write tests.
  5. Business needs are misunderstood by including business-oriented people as first class members of the team.
  6. Business needs change before delivery is addressed through the shortened release cycle.
  7. False feature rich releases are avoided by planning that includes the business and an insistence on working on the highest priority items first.
  8. Staff turnover is reduced by allowing programmers to take the responsibility of estimating their own work and then to perform to those estimates. The frustration of being asked (or told) to do the impossible is reduced improving the working environment.

When I was flying to Los Angeles reading Extreme Programming Explained the first time, I was a little bit too impatient and breezed through Chapter 1.  However, it is important because it provides the contextual underpinning for XP.  Understanding the definition and philosophy of XP  provides a foundation for beginning to explore XP in more depth next week.

Programming Notes:  Next week we will begin Section One: Exploring XP.  Currently, I am planning to re-read two chapters per week even though they are relatively short. Please get a copy of Extreme Programing  Explained, Second Edition (this link will support the blog and podcast), read along and comment!   


Categories: Process Management

Python: Regex ‚Äď matching foreign characters/unicode letters

Mark Needham - Sat, 06/18/2016 - 08:38

I’ve been back in the land of screen scrapping this week extracting data from the Game of Thrones wiki and needed to write a regular expression to pull out characters and actors.

Here are some examples of the format of the data:

Peter Dinklage as Tyrion Lannister
Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)
Filip Lozińá as Young Nobleman
Morgan C. Jones as a Braavosi captain
Adewale Akinnuoye-Agbaje as Malko

So the pattern is:

<actor> as <character>

optionally followed by some other text that we’re not interested in.

The output I want to get is:

Peter Dinklage, Tyrion Lannister
Daniel Naprous, Oznak zo Pahl
Filip Lozińá, Young Nobleman
Morgan C. Jones, a Braavosi captain
Adewale Akinnuoye-Agbaje, Malko

I started using the ‘split’ command on the word ‘as’ but that broke down when I realised some of the characters had the letters ‘as’ in the middle of their name. So regex it is!

This was my first attempt:

import re
strings = [
    "Peter Dinklage as Tyrion Lannister",
    "Filip Lozińá as Young Nobleman",
    "Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)",
    "Morgan C. Jones as a Braavosi captain",
    "Adewale Akinnuoye-Agbaje as Malko"
regex = "([A-Za-z\-'\. ]*) as ([A-Za-z\-'\. ]*)"
for string in strings:
    print string
    match = re.match( regex, string)
    if match is not None:
        print match.groups()
        print "FAIL"
	print ""
Peter Dinklage as Tyrion Lannister
('Peter Dinklage', 'Tyrion Lannister')
Filip Lozińá as Young Nobleman
Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)
('Daniel Naprous', 'Oznak zo Pahl')
Morgan C. Jones as a Braavosi captain
('Morgan C. Jones', 'a Braavosi captain')
Adewale Akinnuoye-Agbaje as Malko
('Adewale Akinnuoye-Agbaje', 'Malko')

It works for 4 of the 5 scenarios but now for Filip Lozińá. The ‘ńá’ character causes the issue so we need to be able to match foreign characters which the current charset I defined in the regex doesn’t capture.

I came across this Stack Overflow post which said that in some regex libraries you can use ‘\p{L}’ to match all letters. I gave that a try:

regex = "([\p{L}\-'\. ]*) as ([\p{L}\-'\. ]*)"

And then re-ran the script:

Peter Dinklage as Tyrion Lannister
Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)
Filip Lozińá as Young Nobleman
Morgan C. Jones as a Braavosi captain
Adewale Akinnuoye-Agbaje as Malko

Hmmm, not sure if I did it wrong or if that isn’t available in Python. I’ll assume the latter but feel free to correct me in the comments and I’ll update the post.

I went search again and found this post which suggested another approach:

You can construct a new character class:


instead of \w. Translated into English, it means “Any character that is not a non-alphanumeric character ([^\W] is the same as \w), but that is also not a digit and not an underscore”.

Let’s try plugging that in:

regex = "([A-Za-z\-'\.^\W\d_ ]*) as ([A-Za-z\-'\.^\W\d_ ]*)"
Peter Dinklage as Tyrion Lannister
('Peter Dinklage', 'Tyrion Lannister')
Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)
('Daniel Naprous as Oznak zo Pahl(credited', 'Stunt Performer)')
Filip Lozińá as Young Nobleman
('Filip Lozi\xc4\x87', 'Young Nobleman')
Morgan C. Jones as a Braavosi captain
('Morgan C. Jones', 'a Braavosi captain')
Adewale Akinnuoye-Agbaje as Malko
('Adewale Akinnuoye-Agbaje', 'Malko')

So that’s fixed Filip but now Daniel Naprous is being incorrectly parsed.

For Attempt #4 I decided to try excluding what I don’t want instead:

regex = "([^0-9\(]*) as ([^0-9\(]*)"
Peter Dinklage as Tyrion Lannister
('Peter Dinklage', 'Tyrion Lannister')
Daniel Naprous as Oznak zo Pahl(credited as Stunt Performer)
('Daniel Naprous', 'Oznak zo Pahl')
Filip Lozińá as Young Nobleman
('Filip Lozi\xc4\x87', 'Young Nobleman')
Morgan C. Jones as a Braavosi captain
('Morgan C. Jones', 'a Braavosi captain')
Adewale Akinnuoye-Agbaje as Malko
('Adewale Akinnuoye-Agbaje', 'Malko')

That does the job but has exposed my lack of regex skillz. If you know a better way let me know in the comments.

Categories: Programming

Stuff The Internet Says On Scalability For June 17th, 2016

Hey, it's HighScalability time:

You've seen the Netflix Death Star microservices map. Here's a map of microbes conversing on your skin.


If you like this sort of Stuff then please support me on Patreon.
  • 4281: # of unread articles in my HackerNews feed; 23%: of all corporate cash is held by Microsoft, Apple, Google; 400 million: number of new servers needed by 2020; ~25,740TB: storage Backblaze adds per month; 3 bits: IBM stores per memory cell; 488 million: faked comments by China per year; 90%: revenue Spotify makes fron 30% of users; 780 million: miles of Tesla driving data; 4 days: median time to binge watch a season on Netlix; $33: cost of Nike Air Max; $50 billion: amount Apple has paid out to app developers; $270 million: amount Line makes from selling stickers; 4,600: # of trees Apple will plant aorund the Spaceship; 200 million: Google photos users; $1.8 billion: Series F round for Snapchat; 3x: capacity of the roadway with driverless cars; 138%: growth in Alibaba's cloud; 

  • Quotable Quotes:
    • @swardley: By this time next year, AMZN should be comfortably worth more than IBM + HPQ + HPE + CISCO + VMW + ORCL + NetApp + EMC combined.
    • @pcalcado: The #serverless revolution, or as we call it in my hometown: "JavaScript folks finally find out about CGI"
    • @kellan: Those patents you filed at Y! that would "only ever be used defensively"? Up for sale
    • @mipsytipsy: oooo, the term FaaS (Functions as a Service) is WAY better than #serverless.  Accurate, doesn't irritate me at all!
    • Ian Eyberg:  We need a new paradigm. We really need to be deploying software not the systems the software lives on. We need to be configuring software not the system that they are on. This is a new way of thinking.
    • @amernetflix: 2 Million #PacketsPerSecond on a #aws public #cloudinstance. 
    • @antirez: /me hopes that because at Redis Conf there were already folks telling me “don’t bother too much with Docker, we are moving away”.
    • @swardley: Asked "Do I not like Docker?" - I like Docker but I view containers as an important but for most, ultimately invisible subsystem ...
    • @jeffjarvis: Before AMP, 51% of WaPost users returned in 7 days; after AMP that's up to 63% says David Merrell at #io16
    • @HackerNewsOnion: How LinkedIn Scaled To Billions Of Unread Messages
    • @nzben: 90% Domain Driven Design 10% Switch Statements
    • @igrigorik: 0.3s latency improvement → £8M revenue gain for thetrainline(.com): http://bit.ly/1NiFzdP  - great case study.
    • @thegrugq: A monk asked Satoshi: “Why do you not sign a challenge msg?” Satoshi answered: “signing proves nothing!” The monk was englightened
    • #bitkoans
    • @johnrobb: Stanford:  Apple could manufacture its products in the US with robotic assembly at same price as China now
    • @Khanoisseur: Amazon changes prices 2.5 million times a day. Wal-Mart and Best Buy change prices 50,000 times over entire month.
    • @Khanoisseur: Amazon pushes code live every 12 seconds and can test a feature on 5000 users by turning it on for just 45 seconds
    • southpolesteve: I have one hard example I can share. We had a node service that was running on ec2 and cost ~$2500/mo. Moved the code directly over to lambda. Now ~$400/mo.
    • Keith Chen~ The behavioural economist at UCLA said users are willing to accept surge pricing increases as high as 9.9 times the normal price of a ride if their smartphone's battery is close to dying.
    • @Bill_Gross: With its always on cellular connection, every 10 hours Tesla gets a MILLION miles of additional data
    • @timallenwagner: Nice example of converting batch system to real-time serverless analytics!
    • @danielbryantuk: "In one study 35% of catastrophic failures were caused by what I call 'dev laziness' " @caitie #qconnewyork
    • @dechampsgu~ "Improving Anything but The Bottleneck is Close to Meaningless" @ziobrando  #dddx
    • @heinrichhartman: "We [#SQLite] don't compete against Oracle, we compete against fopen(3)" Richard Hipp on @changelog https://changelog.com/201/  (47:00)
    • @Pinboard: Any complex website is an interdependent ballet of dozens of mutually supporting services. You can’t reduce it to a word like “up” or “down”
    • @kevinmarks: “The web is already decentralized,” Mr. Berners-Lee said. “The problem is the dominance of one search engine…
    • CHARITY.WTF: I’ve seen what happens when application developers think they don’t have to care about the skills associated with operations engineering.  When they forget that no matter how pretty the abstractions are, you’re still dealing with dusty old concepts like “persistent state” and “queries” and “unavailability” and so forth, or when they literally just think they can throw money at a service to make it go faster because that’s totally how services work.
    • @beaucronin: A dollar-an-hour EC2 instance today would have been at or near the top of Top500 in the early/mid 90s
    • @cloud_opinion: Is this the real ops? / Is this just devops? / Caught in a deploy / No escape from techdebt / Open your eyes / Look up to the cloud and see #serverles
    • lostcolony: In context, I'm pretty sure it was just saying "Unlike computation, bandwidth, and memory size, we haven't seen much improvement in latency, and even if we focused on it, we have a very clear limit we can't get past".
    • People have said a lot more Stuff. Read the full article to see what've they said.

  • You know all those elaborate A/B testing and new feature testing systems built into websites so companies can gather data and learn more about how their product works in the real world? It's not just for websites. Tesla Tests Self-Driving Functions with Secret Updates to Its Customers’ Cars. It appears by deploying actual cars with real drivers Tesla has the Big Data advantage when it comes to creating self-driving cars. Though like much of the modern world it's damn spooky: Tesla can pull down data from the sensors inside its customers’ vehicles to see how people are driving and the road and traffic conditions they experience. It uses that data to test the effectiveness of new self-driving features. The company even secretly tests new autonomous software by remotely installing it on customer vehicles so it can react to real road and traffic conditions, without controlling the vehicle.

  • Can you be a Libertarian and use the government as an amplification attack on an enemy? The Stunning and Expected End of Gawker.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture