Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

Ending support for Android Market on Android 2.1 and lower

Android Developers Blog - Tue, 06/20/2017 - 19:00
Posted by Maximilian Ruppaner, Software Engineer on Google Play

On June 30, 2017, Google will be ending support for the Android Market app on Android 2.1 Eclair and older devices. When this change happens, users on these devices will no longer be able to access, or install other apps from, the Android Market. The change will happen without a notification on the device, due to technical restrictions in the original Android Market app.

It has been 7 years since Android 2.1 Eclair launched. Most app developers are no longer supporting these Android versions in their apps given these devices now account for only a small number of installs.

We will still be supporting later versions of Android Market for as long as feasible. Google Play, the replacement for Android Market, is available on Android 2.2 and above.

Categories: Programming

Code Health: Too Many Comments on Your Code Reviews?

Google Testing Blog - Tue, 06/20/2017 - 18:20
This is another post in our Code Health series. A version of this post originally appeared in Google bathrooms worldwide as a Google Testing on the Toilet episode. You can download a printer-friendly version to display in your office.

By Tom O'Neill


Code reviews can slow down an individual code change, but they’re also an opportunity to improve your code and learn from another intelligent, experienced engineer. How can you get the most out of them?Aim to get most of your changes approved in the first round of review, with only minor comments. If your code reviews frequently require multiple rounds of comments, these tips can save you time.

Spend your reviewers’ time wisely—it’s a limited resource. If they’re catching issues that you could easily have caught yourself, you’re lowering the overall productivity of your team.Before you send out the code review:
  • Re-evaluate your code: Don’t just send the review out as soon as the tests pass. Step back and try to rethink the whole thing—can the design be cleaned up? Especially if it’s late in the day, see if a better approach occurs to you the next morning. Although this step might slow down an individual code change, it will result long-term in greater average throughput.
  • Consider an informal design discussion: If there’s something you’re not sure about, pair program, talk face-to-face, or send an early diff and ask for a “pre-review” of the overall design.
  • Self-review the change: Try to look at the code as critically as possible from the standpoint of someone who doesn’t know anything about it. Your code review tool can give you a radically different view of your code than the IDE. This can easily save you a round trip.
  • Make the diff easy to understand: Multiple changes at once make the code harder to review. When you self-review, look for simple changes that reduce the size of the diff. For example, save significant refactoring or formatting changes for another code review.
  • Don’t hide important info in the submit message: Put it in the code as well. Someone reading the code later is unlikely to look at the submit message.
When you’re addressing code review comments:
  • Re-evaluate your code after addressing non-trivial comments: Take a step back and really look at the code with fresh eyes. Once you’ve made one set of changes, you can often find additional improvements that are enabled or suggested by those changes. Just as with any refactoring, it may take several steps to reach the best design.
  • Understand why the reviewer made each comment: If you don’t understand the reasoning behind a comment, don’t just make the change—seek out the reviewer and learn something new.
  • Answer the reviewer’s questions in the code: Don’t just reply—make the code easier to understand (e.g., improve a variable name, change a boolean to an enum) or add a comment. Someone else is going to have the same question later on.
Categories: Testing & QA

Commenting Policy and How to Get Answers to Your Questions

Mike Cohn's Blog - Tue, 06/20/2017 - 17:00

Here is a short reminder about the Commenting Policy for this site. You can read the full policy here or via the link in the footer on every page.

A lot of the value of this blog comes from the discussion in the comments, and so it is important that all comments be related to the original post. While I appreciate anyone who takes the time to leave a comment, if a comment is not related to the original post, I won't reply and may delete the comment.

Do not hijack a post to an entirely different question. Please see if your question has been addressed by other posts. If not, you can submit it at http://www.mountaingoatsoftware.com/ask.

I try to address the questions and topics suggested there through this blog and my weekly tips emails.

I'm sorry, but I cannot reply by email to questions that are highly specific to your team. If your question or topic is one that I think is of general interest to others here, I will do my best to address it in a blog or weekly tip.

Brotli Compression in Google Display Ads

Google Code Blog - Tue, 06/20/2017 - 17:00
Posted by Michael Burns, Software Engineer, Publisher Tagging & Ads Latency Team

Our goal is to help publishers monetize their content and build sustainable businesses through advertising products that allow sites to load as fast as possible to minimize impact to user experience.

Almost two years ago, our compression team announced a new compression algorithm called Brotli. Today, we are happy to announce that the Brotli compression algorithm is now being used to compress Google Display Ads whenever possible. In our experiments, we see data savings of 15% in aggregate over standard gzip compression, and in some instances, a savings of over 40%! This reduces the amount of data sent to end users by tens of thousands of gigabytes every day! This also results in faster page loads and less battery consumption.

We hope results like this will encourage wider adoption and will advance web standards such as Brotli compression.

Categories: Programming

Quote of the Month June 2017

From the Editor of Methods & Tools - Tue, 06/20/2017 - 08:44
Avoid using defect tracking systems during the iteration. Fix bugs discovered in the work underway for the iteration immediately. If it takes a lot of work, create a task in the Spring backlog. However, there is usually no need to log this bug in the defect tracking system. Logging it would only create another work […]

SE-Radio Episode 294: Asaf Yigal on Machine Learning in Log Analysis

Asaf Yigal talks with SE Radio’s Edaena Salinas about machine learning in log analysis. The discussion starts with an overview of the structure of logs and what information they can contain. Asaf discusses what the log analysis process looks like without machine learning — and the role of humans in this – before moving on […]
Categories: Programming

Defining “Scaling” Agile, Part 6: Creating the Agile Organization

We might start to think about agile approaches as a project change. However, if you want to “scale” agile, the entire culture changes. Here is a list of the series and how everything changes the organization’s culture:

All of these “scaling” ideas require change.

Agile organizations become very good at small changes all the time. They are adaptive and resilient. They understand how change works and they embrace it. (Big changes are quite difficult for almost everyone. Small changes tend to be much easier.)

Here is a picture of the Satir change model. We start off in Old Status Quo with some level of performance. Some Foreign Element arrives and we have uneven performance while we are in Chaos, searching for that Transforming Idea. We discover that TI and practice and integrate that idea into our work until we reach a New Status Quo, hopefully at a higher level of performance.

The change model works for each human and for the entire organization. In fact, I don’t see how you can have an agile organization until people become comfortable with small and large changes on a regular basis. This is one of the reasons I say we should take an agile approach to agile. (See Agile is Not a Silver Bullet.)

Do you need to be a fully adaptive and resilient organization? Only you can answer that question. Maybe you start from teams and the project portfolio. Maybe you look for incentives that don’t create optimization “up.” (I need a new word for this! Do you have suggestions for me?? Please comment if so.)

You do not have to have a fully agile organization to recognize benefits at the team and project portfolio levels. Agile is Not for Everyone.

Decide how much change your organization needs to be successful. Practice change, as often as you can. That will help you more than an agile label will.

One last comment: I’m not sure “scaling” is the right way to think about this. For me, the scaling idea still has roots in silo thinking, not flow thinking. That could be just me. (Grin!) I wish I had a better way to explain it.

My ideas of scaling agile are about creating a culture based on our ability to change:

  • How do we treat each other? Do we/can we see each other as an intrinsic part of a whole?
  • What do we discuss? Do we discuss “failures” or learning opportunities?
  • What do we reward? How do we move to rewarding non-silo work? That is, work to deliver products (in either projects or programs) or other work that rewards collaboration and flow efficiency.

I hope you enjoyed this series. Please do comment on any of the posts. I am curious about what you think. I will learn from your comments.

Categories: Project Management

Quote of the Day

Herding Cats - Glen Alleman - Mon, 06/19/2017 - 03:55

Under the current imperfect administration of the Universe, most new ideas are false, so most ideas for improvement make matters worse

Categories: Project Management

SPaMCAST 447 – Product Owners and The Business Analyst with Angela Wick

SPaMCAST Logo

http://www.spamcast.net

Listen Now
Subscribe on iTunes
Check out the podcast on Google Play Music

 

SPaMCAST 447 features our interview with Angela Wick on the role of the Product Owner and Business Analyst in Agile efforts. These two roles are critically important for delivering value in an Agile environment. Angela provides a fresh take on the Product Owner role and the Product Owner’s relationship to other roles Agile teams.

Angela is the founder of BA-Squared, LLC, a training and consulting practice.  She is passionate about modernizing requirements practices and helping organizations collaborate on a Product Vision aligned to strategy and guiding them to a meaningful backlog and iterations that keep the customer and organizational value top of mind.

She trains, coaches and teaches organizations on Product Ownership and Agile BA!

Email: Angela@BA-Squared.Com

Web: http://www.ba-squared.com

LinkedIn: https://www.linkedin.com/in/angelawickcbap

Twitter: https://twitter.com/WickAng


This is not first time the SPaMCAST has featured essays and conversations on the role of product owners ( for exampleSPaMCAST 430 and SPaMCAST 325).

 

Re-Read Saturday News

Chapter 9 continues the third section of Holacracy, Evolution Installed: Living Holacracy.   Holacracy: The New Management System for a Rapidly Changing World by Brian J. Robertson was published by Henry Holt and Company in 2015.  This week’s chapter is titled If You’re Not Ready To Adopt: Moving Toward Holacracy.  In this chapter Robertson softens his if-you-can’t-do-it-all-don’t-do-anything approach.  

This chapter begins with a story of Robertson being asked how they can move forward in a limited manner.  The person had just intently listened to a talk on Holacracy. The person explained that they could see the value, but did not have to power to change the organization or even their department.  Robertson’s knee jerk response was that you could not use parts; however, the response felt wrong.  So he reached out the larger community of practitioners to gather their field observations for how they handled scenarios in which everything could not be implemented.  Whether the story is apocryphal or not matters less than that this chapter softens the all-or-nothing stance stated earlier in the book.  

 

Catch up on the all of the Holacracy entries:

Week 1:  Logistics and Introduction

Week 2: Evolving Organization

Week 3: Distribution Authority

Week 4: Organizational Structure

Week 5: Governance

Week 6: Operations

Week 7: Facilitating Governance

Week 8: Strategy and Dynamic Control

Week 9 Adopting Holacracy

Week 10: Moving Toward Holacracy

 

In approximately three weeks we will begin the next book in our Re-read series,  The Science of Successful Organizational Change. Remember to use the link to buy a copy in order to support the podcast and blog. The reread will be led by Steven Adams.   I am looking forward to sitting on the other side of the table during the next re-read! Visit the Software Process and Measurement Cast blog to participate in this and previous re-reads.

 

A Call To Action

If you got a new idea this week while listening to the podcast, please give the SPaMCAST a short, honest review in iTunes, Stitcher or wherever you are listening.  If you leave a review please send a copy to spamcastinfo@gmail.com.  Reviews help guide people to the cast!

 

Next SPaMCAST

SPaMCAST 448 will feature our essay on uncertainty. Al Pittampalli said, “uncertainty and complexity produce anxiety we wish to escape”. Dealing with uncertainty is part of nearly everything we do.  

The second column will feature Steve Tendon talking about Tame The Flow: Hyper-Productive Knowledge-Work Performance, The TameFlow Approach and Its Application to Scrum and Kanban published J Ross (buy a copy here). We tackle Chapter 18.  

Our third column will be from Jeremy Berriaul.t. Jeremy discusses leading in  QA.  Jeremy  blogs at https://jberria.wordpress.com/  

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

 


Categories: Process Management

SPaMCAST 447 - Product Owners and The Business Analyst with Angela Wick

Software Process and Measurement Cast - Sun, 06/18/2017 - 22:00

SPaMCAST 447 features our interview with Angela Wick on the role of the Product Owner and Business Analyst in Agile efforts. These two roles are critically important for delivering value in an Agile environment. Angela provides a fresh take on the Product Owner role and the Product Owner's relationship to other roles Agile teams.

Angela is the founder of BA-Squared, LLC, a training and consulting practice.  She is passionate about modernizing requirements practices and helping organizations collaborate on a Product Vision aligned to strategy and guiding them to a meaningful backlog and iterations that keep the customer and organizational value top of mind.

She trains, coaches and teaches organizations on Product Ownership and Agile BA!

Email: Angela@BA-Squared.Com

Web: http://www.ba-squared.com

LinkedIn: https://www.linkedin.com/in/angelawickcbap

Twitter: https://twitter.com/WickAng


This is not the first time the SPaMCAST has featured essays and conversations on the role of product owners ( for exampleSPaMCAST 430 and SPaMCAST 325).

 

Re-Read Saturday News

Chapter 9 continues the third section of Holacracy, Evolution Installed: Living Holacracy.   Holacracy: The New Management System for a Rapidly Changing World by Brian J. Robertson was published by Henry Holt and Company in 2015.  This week's chapter is titled If You're Not Ready To Adopt: Moving Toward Holacracy.  In this chapter Robertson softens his if-you-can't-do-it-all-don't-do-anything approach.  

This chapter begins with a story of Robertson being asked how they can move forward in a limited manner.  The person had just intently listened to a talk on Holacracy. The person explained that they could see the value, but did not have to power to change the organization or even their department.  Robertson’s knee jerk response was that you could not use parts; however, the response felt wrong.  So he reached out the larger community of practitioners to gather their field observations for how they handled scenarios in which everything could not be implemented.  Whether the story is apocryphal or not matters less than that this chapter softens the all-or-nothing stance stated earlier in the book.  

 

Catch up on the all of the Holacracy entries:

Week 1:  Logistics and Introduction

Week 2: Evolving Organization

Week 3: Distribution Authority

Week 4: Organizational Structure

Week 5: Governance

Week 6: Operations

Week 7: Facilitating Governance

Week 8: Strategy and Dynamic Control

Week 9 Adopting Holacracy

Week 10: Moving Toward Holacracy

 

In approximately three weeks we will begin the next book in our Re-read series,  The Science of Successful Organizational Change. Remember to use the link to buy a copy in order to support the podcast and blog. The reread will be led by Steven Adams.   I am looking forward to sitting on the other side of the table during the next re-read! Visit the Software Process and Measurement Cast blog to participate in this and previous re-reads.

 

A Call To Action

If you got a new idea this week while listening to the podcast, please give the SPaMCAST a short, honest review in iTunes, Stitcher or wherever you are listening.  If you leave a review please send a copy to spamcastinfo@gmail.com.  Reviews help guide people to the cast!

 

Next SPaMCAST

SPaMCAST 448 will feature our essay on uncertainty. Al Pittampalli said, “uncertainty and complexity produce anxiety we wish to escape”. Dealing with uncertainty is part of nearly everything we do.  

The second column will feature Steve Tendon talking about Tame The Flow: Hyper-Productive Knowledge-Work Performance, The TameFlow Approach and Its Application to Scrum and Kanban published J Ross (buy a copy here). We tackle Chapter 18.  

Our third column will be from Jeremy Berriaul.t. Jeremy discusses leading in  QA.  Jeremy  blogs at https://jberria.wordpress.com/  

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

 

Categories: Process Management

Holacracy: Re-read Week 10, Chapter 9: If You’re Not Ready To Adopt: Moving Toward Holacracy

Book Cover

Holacracy

 

In approximately three weeks we will begin the next book is The Science of Successful Organizational Change. Remember to use the link to buy a copy in order to support the podcast and blog. The reread will be led by Steven Adams.   I am looking forward to sitting on the other side of the table during the next re-read!

Chapter 9 continues the third section of Holacracy, Evolution Installed: Living Holacracy.  This week’s chapter is titled If You’re Not Ready To Adopt: Moving Toward Holacracy.  In this chapter Robertson softens his if-you-can’t-do-it-all,-don’t-do-anything approach.  

This chapter begins with a story of Robertson being asked by very someone that had just intently listened to a talk on Holacracy. The person intently explained that they could see the value but did not have to power to change an organization or even their department.  Robertson’s knee jerk response was that you could not use parts; however, the response felt wrong.  So he reached out the larger community of practitioners to gather their field observations for how they handled scenarios in which everything could not be implemented.  Whether the story is apocryphal or not matters less than that this chapter softens the all or nothing stance stated earlier in the book.

Robertson suggests four ways that parts of Holacracy can be leveraged.  The are:

  1. Start using the language of Holacracy. Language is both a reflection of culture and a tool to generate culture.  For example, shifting from using words like ‘problems’ and ‘solutions’ to ‘tensions’ and ‘processing’ shifts the blame-game dialog to a dialog of continual improvement.  Other examples include using providing proposal instead or just identifying problems.  A final example is to stop talking about people and focus discussions on roles. The idea is that if you change the words used in the organization the culture will follow.
  2. Rewrite your role descriptions.  Writing or rewriting role descriptions changes the focus to the set of activities needed to deliver value rather than mushy job descriptions.  Once written roles then evolve to address changes in the environment more easily than modifying job description with often is mired in bureaucratic rules. 
  3. Work on your organization not just in it.  Own the part of the business you can and engage in governance (a Holacracy word) to make changes that will help people work together to deliver value. Change the part of the organization that is within your span of control.  Foster an entrepreneurial mindset and evangelize that mindset to colleagues in an effort to evolve the organization.  
  4. Streamline your meetings. The meeting structure for tactical meetings, with its focus on building a dynamic agenda, training issues and processing one issue at a time, have value outside of Holacracy.  Robertson also suggests that the check in and closing rounds are useful focusing mechanisms in all meetings.  Another meeting technique that I have borrowed is the integrated decision-making process (described on page 72).  I have found the process useful for meetings that have to get through a number of emotionally charged decisions in an efficient manner (such as a project review board).

Chapter 9 answers, for a second time, the question of whether you can do parts of Holacracy by saying you can’t do Holacracy without going all in, but you can borrow parts and perhaps start the change.

Remember to buy a copy of Holacracy (use the link to help support and defray the costs of the Software Process and Measurement Cast blog and podcast).

Previous Entries in the re-read:

Week 1:  Logistics and Introduction

Week 2: Evolving Organization

Week 3: Distribution Authority

Week 4: Organization Structure

Week 5: Governance

Week 6: Operations

Week 7: Facilitating Governance

Week 8: Strategy and Dynamic Control

Week 9:Adopting Holacracy

 


Categories: Process Management

The Smell of Dysfunction and Corrective Action

Herding Cats - Glen Alleman - Sat, 06/17/2017 - 17:57

When we hear about some dysfunction in our work, a product, a system - Root Cause Analysis can provide the corrective action needed to remove the dysfunction and most importantly prevent the dysfunction from returning.

There are Eight Questions needed to gain insight into the source of the problem, before conjecturing any fix. In the Appollo Root Cause Analysis method, we have both Events (Actions) and Conditions, so these Eight need to be restated

  1. What was the harm from the event or the condition that created the primary effect?
  2. What was the significance of this effect? How is this measured? 
  3. What was the set-up for the event or the condition? In Apollo, the cause and the effect are the same things in an infinite chain.  My cause is your effect, which creates a  new cause for a new effect.
  4. What triggered the event? Either an Action or a Condition. Find out, write that down, record the source of the observation and the connection in the chain of cause and effect.
  5. What made the harm as severe as it was? Assess - an after action report - what damage was done. 
  6. What kept the harm from being a lot worse? Look for preventative measures that are already in place.
  7. What should be learned from the event? This is the basis of another method, the Phoenix method of Root Cause Analysis, where Lessons Learned are captured and used to prevent or correct future problems.
  8. What should be done about the event? This is the Corrective Action Plan (CAP) for the Primary Effect. In Apollo, this is defined by removing one of more cause and effect paths.

Here's a framework for applying Apollo

Root cause analysis master plan from Glen Alleman

When you hear someting like Estimates are the smell of Dysfunction you'll now know that is complete utter malarky, since not cause has been stated and not corrective action has been devised that can be tested to prevent or correct that undefined smell. 

So don't let anyone get away with making a unsubstianted statement about some problem they have observed and some suggested correction to that problem. It's intellectual laziness to do make those statements. And intellectual laziness to not challenge them. 

Categories: Project Management

Performance-Based Project Management®

Herding Cats - Glen Alleman - Sat, 06/17/2017 - 14:00

Successfully managing projects, no matter the domain, the size, the development or engineering method starts with a set of Principles, Practices, and Processes.

Here are those Principles and Practices. The Principles are Immutable. The Practices are universal. The Processes need to be tailored to your domain.

When hearing about some new, supposed innovative way for delivering value in exchange for money, ask does that idea fit into any known set of Principles and Practices of project success?

Principles and Practices of Performance-Based Project Management® from Glen Alleman Related articles Applying the Right Ideas to the Wrong Problem Capabilities Based Planning Information Technology Estimating Quality
Categories: Project Management

Semantic Time support now available on the Awareness APIs

Android Developers Blog - Fri, 06/16/2017 - 18:00
Posted by Ritesh Nayak M, Product Manager

Last year at I/O we launched the Awareness API, a simple yet powerful API that let developers use signals such as Location, Weather, Time and User Activity to build contextually relevant app experiences.

Available via Google Play services, the Awareness API offers two ways to take advantage of context signals within your app. The Snapshot API lets your app request information about the user's current context, while the Fence API lets your app react to changes in user's context, and when it matches a certain set of conditions. For example, "tell me whenever the user is walking and their headphone is plugged in".

Until now, you could specify a time fence on the Awareness APIs but were restricted to using absolute/canonical representation of time. Based on developer feedback, we realized that the flexibility of the API in regards to building time fences did not support higher level abstractions people use when they think and talk about time. "This weekend", "on the next holiday", "after sunset", are all very common and colloquial ways of expressing time. That's why we're adding Semantic time support to these APIs starting today

For e.g., if you were building a fitness app and wanted a way to prompt users everyday morning to start their routine, or if you're a reading app that wants to turn on night mode after dusk; you would have to query a 3p API for sunrise/sunset information at the user location and then write up an Awareness fence with those canonical time values. With our latest update, you can use our TIME_INSTANT_SUNRISE and TIME_INSTANT_SUNSET constants and let the platform manage all the complexity for you.

Let's look at an example. Suppose you're building a fitness app which prompts users on Tuesday, and Thursday around sunrise to begin their morning work out. You can set up this triggering using the following lines of code.

// A sun-state-based fence that is TRUE only on Tuesday and Thursday during Sunrise 
AwarenessFence.and(
    TimeFence.aroundTimeInstant(TimeFence.TIME_INSTANT_SUNRISE,
            -10 * ONE_MINUTE_MILLIS, 5 * ONE_MINUTE_MILLIS),
    AwarenessFence.or(
        TimeFence.inIntervalOfDay(TimeFence.DAY_OF_WEEK_TUESDAY,
                0, ONE_DAY_MILLIS),
        TimeFence.inIntervalOfDay(TimeFence.DAY_OF_WEEK_THURSDAY,
                0, ONE_DAY_MILLIS)));

One of our favorite semantic time features is public holidays. Every country and regions within it have different holidays. Assume you were a local hiking & adventure app that wants to show users activities they can indulge in on a holiday that falls on a Friday or a Monday. You can use a combination of Days and Holiday flags to identify this state for all your users around the world. You can do this with just 3 lines of code and have this work in any part of the world.

// A local-time fence that is TRUE only on public holidays in the
// device locale that fall on Fridays or Mondays.
AwarenessFence.and(
    TimeFence.inTimeInterval(TimeFence.TIME_INTERVAL_HOLIDAY),
    AwarenessFence.or(
        TimeFence.inIntervalOfDay(TimeFence.DAY_OF_WEEK_FRIDAY,
                9 * ONE_HOUR_MILLIS, 11 * ONE_HOUR_MILLIS),
        TimeFence.inIntervalOfDay(TimeFence.DAY_OF_WEEK_MONDAY,
                9 * ONE_HOUR_MILLIS, 11 * ONE_HOUR_MILLIS)));

In both example cases, Awareness does the heavy lifting of localizing time and holidays based on the device locale settings.

We're excited to see what problems you'll solve using this powerful API. Please join our mailing list to get updates about this and other Context APIs at Google.

.blogimage img { width: 100%; }
Categories: Programming

The Five Laws of Software Estimating are Wrong

Herding Cats - Glen Alleman - Fri, 06/16/2017 - 17:11

There's a blog post from a few years back that has resurfaced The 5 Laws of Software Estimates. It's one of those posts that's heavy on opinion and light on principles. Time to revisit this ill-informed idea on why, what, and how we need to use estimates to make decisions in the presence of uncertainty. 

Law of Software Estimating Fact of Software Estimating Estimates are waste

A waste for whom?

Developers think they are waste, but it's not their money.

To those paying the developers, estimates provide actionable information needed to make decisions:

  • Can we afford to develop this feature?
  • If we develop this feature when will it start earning its keep for the business
  • What tradeoffs are possible in cost and schedule for this feature compared to other features?
Estimates are non-transferrable 

Its claimed Software estimates are not fungible.

Estimates are transferrable if you keep track of what you develop and codify those Features for future reference.

I work in a domain where a Central Repiository is kept of all work performed and used to estimate other projects. This is called Reference Class Forecasting and done in all mature project management domains.

This is basic business process improvement and part of any credible estimating process.

Estimates are Wrong

 Estimates are NOT Guesses

This is a fundamental category error in knowledge. Assigning a meaning to a term that is incorrect.

Either a lack of knowledge about estimating or a willful lack of knowledge about estimating. I suspect that later since every High School Statistics class introduces the notion of an estimate and the two attributes of that estimate

  • Precision - what precision is needed to make your decision
  • Accuracy - what accuracy is needed to make your decision

Define these before you start estimating, not afterward. 

We need to know the cost for the product is "at our below $1,200,000 with an 80% confidence would be a statement a CIO might make to a development team.

Or we need to know you can produce that product for that cost "on or before" the end of the 2nd quarter.

Estimates are Temporary

Yes they are, that's why the Estimate to Complete and the Estimate at Completion are continually emerging with new data, better models, empirical data from delivered.

This is obvious to any mature estimating organization.

Estimates are Necessary

Correct Businesses cannot make decisions about whether or not to build software without having some idea of the cost and time involved.

So why even introduce the first 4 as LAWs, rather state the first 4 as dysfunctions that need to be removed to increase the Value of estimates.    

So before you go off and Stop Estimating, ask yourself - or better confirm that those paying you for your work - that there is no need to know how much it will cost to produce the Value asked for or no need to know when that Value will arrive at some degree of precision and accuracy?

If there is no need, then estimates likely don't need to be made.

If there is a need for all the right business reasons, then ignore the first 4 Laws, and learn how to make estimates in the presence of uncertainty for those whose money you are spending.

Related articles Information Technology Estimating Quality Humpty Dumpty and #NoEstimates Five Immutable Principles of Project Success and Estimating Why Guessing is not Estimating and Estimating is not Guessing Capabilities Based Planning Architecture -Center ERP Systems in the Manufacturing Domain IT Risk Management Making Conjectures Without Testable Outcomes What's the Smell of Dysfunction?
Categories: Project Management

scikit-learn: Random forests – Feature Importance

Mark Needham - Fri, 06/16/2017 - 06:55

As I mentioned in a blog post a couple of weeks ago, I’ve been playing around with the Kaggle House Prices competition and the most recent thing I tried was training a random forest regressor.

Unfortunately, although it gave me better results locally it got a worse score on the unseen data, which I figured meant I’d overfitted the model.

I wasn’t really sure how to work out if that theory was true or not, but by chance I was reading Chris Albon’s blog and found a post where he explains how to inspect the importance of every feature in a random forest. Just what I needed!

Stealing from Chris’ post I wrote the following code to work out the feature importance for my dataset:

Prerequisites
import numpy as np
import pandas as pd

from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split

# We'll use this library to make the display pretty
from tabulate import tabulate
Load Data
train = pd.read_csv('train.csv')

# the model can only handle numeric values so filter out the rest
data = train.select_dtypes(include=[np.number]).interpolate().dropna()
Split train/test sets
y = train.SalePrice
X = data.drop(["SalePrice", "Id"], axis=1)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42, test_size=.33)
Train model
clf = RandomForestRegressor(n_jobs=2, n_estimators=1000)
model = clf.fit(X_train, y_train)
Feature Importance
headers = ["name", "score"]
values = sorted(zip(X_train.columns, model.feature_importances_), key=lambda x: x[1] * -1)
print(tabulate(values, headers, tablefmt="plain"))
name                 score
OverallQual    0.553829
GrLivArea      0.131
BsmtFinSF1     0.0374779
TotalBsmtSF    0.0372076
1stFlrSF       0.0321814
GarageCars     0.0226189
GarageArea     0.0215719
LotArea        0.0214979
YearBuilt      0.0184556
2ndFlrSF       0.0127248
YearRemodAdd   0.0126581
WoodDeckSF     0.0108077
OpenPorchSF    0.00945239
LotFrontage    0.00873811
TotRmsAbvGrd   0.00803121
GarageYrBlt    0.00760442
BsmtUnfSF      0.00715158
MasVnrArea     0.00680341
ScreenPorch    0.00618797
Fireplaces     0.00521741
OverallCond    0.00487722
MoSold         0.00461165
MSSubClass     0.00458496
BedroomAbvGr   0.00253031
FullBath       0.0024245
YrSold         0.00211638
HalfBath       0.0014954
KitchenAbvGr   0.00140786
BsmtFullBath   0.00137335
BsmtFinSF2     0.00107147
EnclosedPorch  0.000951266
3SsnPorch      0.000501238
PoolArea       0.000261668
LowQualFinSF   0.000241304
BsmtHalfBath   0.000179506
MiscVal        0.000154799

So OverallQual is quite a good predictor but then there’s a steep fall to GrLivArea before things really tail off after WoodDeckSF.

I think this is telling us that a lot of these features aren’t useful at all and can be removed from the model. There are also a bunch of categorical/factor variables that have been stripped out of the model but might be predictive of the house price.

These are the next things I’m going to explore:

  • Make the categorical variables numeric (perhaps by using one hot encoding for some of them)
  • Remove the most predictive features and build a model that only uses the other features

The post scikit-learn: Random forests – Feature Importance appeared first on Mark Needham.

Categories: Programming

Adaptability Requires Comfort With Uncertainty

Making decisions is exhausting. It involves perception and analysis, and most of all: taking responsibility.
Mental load and the worry cache, Seth Godin, June 15, 2017

In most organizations, software development and maintenance is not a solitary activity.  Even a relatively simple enhancement require the involvement of multiple roles to identify, translate and implement an idea.  Teams are the tool used to effectively gather and coordinate all the needed roles in one place. While it is easy to perceive the team as a monolithic unit, every individual on a team is a decision-making machine.  Each person will have a different tolerance for uncertainty and ambiguity which will affect how they make decisions.

In 1983, Geert Hofstede published a paper titled National Cultures in Four Dimensions.  The paper introduced the idea of the Uncertainty Avoidance Index (UAI) to explain behavioral differences between nations. The theories espoused by Dr. Hofstede illustrate that different cultures react to uncertainty differently. Organizations and the teams inside those organizations also have different tolerances for uncertainty.  While the factors that influence how avoidant of uncertainty any organization or team are probably far more complex than national cultures, what is important is that we understand that some organizations and teams will avoid uncertainty more vociferously than others. In software development, how work is done is often a reflection of differing approaches and needs to reduce the anxiety generated by uncertainty.

Software development organizations that are highly avoidant of uncertainty exhibit many of the following attributes:

  1. Structure and rules focus. Development processes are spelled out at a high level of granularity and are often augmented by informal norms and rules.  Changing the process is formally evaluated and approved.
  2. Roles and individuals are conflated.  An individual plays a specific role and is expected to have a high degree of expertise in the role which makes them required participants for decisions that impact the role.
  3. Consensus decision making is used to increase the knowledge base for any specific decision while distributing responsibility.  Consensus decision making often increases the required energy to generate a specific decision. 
  4. Life is hectic.  Significant amounts of energy are required to continuously monitor the flow of work and the decisions being made that could possibly affect the work being performed. Excuse me while I check email and Slack on my iPhone during a meeting.  

Organizations and teams whose work and products impact health or safety often all into this category. The people that work in this environment must be comfortable with this form of structure. Organizations that have less tolerance for uncertainty will gravitate toward frameworks and methods that provide them a level of structure that reduces uncertainty.  DSDM, SAFe and other more structured approaches are often used in organizations in which uncertainty generates anxiety.

The reaction any team or organization has to uncertainty will be different, which is why there is no one perfect framework or method for delivering value. Highly structured approaches will appeal to some organizations while less structured approaches will appeal to others.  While these are just two poles on a continuum, my observations is that most organizations tend to live closer to one end than the center. That makes using the wrong approach in an organization tantamount two teams playing football together with one using the rules from the National Football League and the other playing by the rules of the Canadian Football League.  The result would be lots of tension.

 


Categories: Process Management

Android Things Developer Preview 4.1

Android Developers Blog - Thu, 06/15/2017 - 22:41
Posted by Wayne Piekarski, Developer Advocate for IoT

Today, we're releasing a new Developer Preview 4.1 (DP4.1) of Android Things, with updates for new supported hardware and bug fixes to the platform. Android Things is Google's platform to enable Android Developers to create Internet of Things (IoT) devices, and seamlessly scale from prototype to production.

New hardware

A new Pico i.MX6UL revision B board has been released, which supports many common external peripherals from partners such as Adafruit and Pimoroni. There were some prototype Pico i.MX6UL boards made available to some early beta testers, and these are not compatible with DP4.1.

Improvements

DP4.1 also includes some performance improvements since DP4, such as boot time optimizations that improve the startup time of i.MX7D based hardware. This Developer Preview also includes a version of Google Play Services, specifically optimized for IoT devices. This new IoT variant is a lot smaller and optimized for use with Android Things, and requires the use of play-services 11.0.0 or later in your build.gradle. For more information about the supported features in the IoT variant of Google Play Services, see the information page.

Google I/O

Android Things had a large presence at Google I/O this year, with 7 talks covering different aspects of Android Things for developers, and these are available as videos in a playlist for those who could not attend:

What’s New In Google’s IoT Platform? Ubiquitous Computing at Google Bringing Device Production to Everyone With Android Things From Prototype to Production Devices with Android Things Developing for Android Things Using Android Studio Security for IoT on Android Things Using Google Cloud and TensorFlow on Android Things Building for Enterprise IoT Using Android Things and Google Cloud Platform

Google I/O also had a codelab area, where attendees could sit down and test out Android Things development with some simple guided training guides. These codelabs are available for anyone to try at https://codelabs.developers.google.com/?cat=IoT

Feedback

Thank you to all the developers who submitted feedback for the previous developer previews. Please continue sending us your feedback by filing bug reports and feature requests, and asking any questions on stackoverflow. To download images for DP4.1, visit the Android Things download page and find the changes in the release notes. You can also join Google's IoT Developers Community on Google+, a great resource to get updates and discuss ideas, with over 5,600 members.

.parent { display: flex; justify-content: space-around; align-items: center; width: 100% flex-wrap: wrap; } .child1 { width: 30%; display: flex; justify-content: center; align-items: center; } .child1 img { object-fit: contain; width: 150px; margin: 0; border: 0; padding: 0; } .child2 { width: 70%; text-align: left; padding: 0 0 0 5px; }
Categories: Programming

Welcome to your New Home on Android TV

Android Developers Blog - Thu, 06/15/2017 - 18:43
.image img { width: 100%; padding: 10px 0; margin: 0; border: 0; } Posted by Paul Saxman, Android Devices and Media Developer Relations Lead

Android TV brings rich app experiences and entertainment to the biggest screen in your house, and with Android O, we’re making it even easier for users to access content from their favorite apps. We’ve built a new, content-centric home screen experience for Android TV, and we're bringing the Google Assistant to the platform as well. These features put content that users want to access a few clicks, or spoken words, away.

The New Android TV Home Screen

The new Android TV home screen organizes video content into channels and programs in a way that’s familiar to TV viewers. Each Android TV app can publish multiple channels, which are represented as rows of programs on the home screen. Apps add relevant programs on each channel, and update these programs and channels as users access content or when new content is available. To help engage users, programs can include a video preview, which is automatically played when a user focuses on a program. Users can configure which channels they wish to see on the home screen, and the ordering of channels, so the themes and shows they’re interested in are quick and easy to access.

In addition to channels for you app, the top of the new Android TV home screen includes a quick launch bar for users' favorite apps, and a special Watch Next channel. This channel contains programs based on the viewing habits of the user.

The APIs for creating and maintaining channels and programs are part of the TvProvider APIs, which are distributed as an Android Support Library module with Android O. To get started using these APIs, visit the Android O Developer Preview site for an overview, and try out the Android TV Channels and Programs codelab for a first-hand experience building an Android TV app for Android O.

Later this year, Nexus Players will receive the new Android TV home experience as an OTA update. If you wish build and test apps for the new interface today, however, you can use the Android TV emulator or Nexus Player device images that are part of the latest Android O Developer Preview.

The Google Assistant on Android TV

The Google Assistant on Android TV, coming later this year, will allow users to quickly find and access content using their voice. Because the Assistant is context-aware, it can help users narrow down what content to play. Users will also be able access the Assistant to control playback, even while a video or music is playing. And since the Assistant can control compatible smart home devices, a simple voice request can dim the lights to create an ideal movie viewing environment. When the Google Assistant comes to Android TV, it will launch in the US on Android devices running M, N, and O.

We're looking forward to seeing how developers take advantage of the new Android TV home screen. We welcome feedback, so please visit the Android TV Developer Community on G+ to share you thoughts and ideas!

Categories: Programming

Defining “Scaling” Agile, Part 5: Agile Management

One of the challenges I see in organizations is how managers can use agile approaches. One of the biggest problems is that the entire organization is organized for resource efficiency (think silos of functional experts). Agile approaches use flow efficiency. Thinking in flow efficiency changes everything.

Many people in organizations believe that dividing up the work among specialists will get the work done faster. That’s the case in manufacturing. (Think piece work.) Manufacturing processes use resource efficiency to reasonable effect. However, manufacturing does not account for innovation or learning. (Design of manufacturing processes is innovative and requires learning. That’s why manufacturing processes often prototype (iterate on their work) when they develop the process.)

Organizations who need to optimize for innovation or learning realize that the people work collaboratively. Collaborative work—including management work—demands flow efficiency instead of resource efficiency.

Flow efficiency helps people optimize “up” for lack of a better term. (If you have not yet read This is Lean: Resolving the Efficiency Paradox, I recommend you  do so.)

Once you start to think about flow efficiency, you might start to think about throughput (and maybe cycle time and cumulative flow).  That changes the data the managers need to make decisions.

Now, it doesn’t matter what anyone’s utilization is. What matters is the Cost of Delay (or Cost of Delay/Duration). It might even matter where the bottlenecks are and who can unwedge those bottlenecks.

An organization might still need to track the run rate for a project, program, or even a workgroup such as Customer Support. Run rate might not mean as much when you can measure revenue, customer acquisition and retention, and how often you can get the customer to return and buy more. (The more often you deliver value, the more you can acquire customers who pay.)

One manager learned about flow efficiency. She was managing the team working on the “most important” project in the company. She decided to stop tracking utilization. She told her team, “I want to track your throughput as a team. I want to know what I can do improve the flow of work through your team. Please bring me your impediments to flow and I’ll address them.”

The team told her about build time (too slow), insufficient test automation (not enough and too slow). She built the case using Cost of Delay/Duration to get other teams to help this team reduce build time and increase test automation.

The teams together took eight weeks to make what they called infrastructure improvements. After the first two of those eight weeks, the team finished twice as many stories (2 instead of 1) as they had before all the improvements started. The team continued to increase their throughput. By the end of the eight weeks, the team was able to finish anywhere from 8-12 stories. The team continued their now-higher level of throughput. After three months, the organization had attracted many more customers. They decided to move to a subscription model for their product, and recognize much more revenue.

Yes, a team’s ability to deliver value fast created feedback loops for management: product management, project portfolio management.

(I’m still reading about agile-useful metrics, so I’m sure there is more here.)

If managers worry about flow efficiency, they ask the teams, “What can I do to help your throughput?” The managers manage the project portfolio. Work flows through teams, not through people.

That changes what managers do. Top-level managers (and maybe product managers) define the strategy and talk to customers to see what customers need so they can refine the strategy. Top managers also consider new options for entirely new products—changes in strategy.

Middle managers plan and replan the project portfolio to implement the strategy. First-level managers remove impediments to collaboration.

Let me summarize a little:

Agile managers have a very different role than more traditional managers. They manage differently. Agile managers might use the two pillars of lean: respect for people and continuous improvement as a basis for their management style. To me, that means building relationships with each person the manager “manages,”  the willingness to question all assumptions, and to look at the whole: what does it take for us to succeed as an organization. The idea of one function succeeding instead of all of us vanishes.

Flow efficiency changes the corporate culture: managers change what they discuss. Managers change what they reward. Managers change how they treat people and how they expect people to be treated, because it’s not about the individual “resource.” It’s about the system of work.

The posts so far aside from this post:

I’m quite sure this post is not perfect, but I have other writing to do. I expect to have one more summary post.

Categories: Project Management