Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

SPaMCAST 286 – Brian Wernham, Agile Project Management for Government

Software Process and Measurement Cast - Sun, 04/20/2014 - 22:00

Listen to the Software Process and Measurement Cast 286. SPaMCAST 286 features our interview with Brian Wernham, author of Agile Project Management for Government. Agile government does not have to be an oxymoron.

Brian Wernham has more than 30 years of experience in adaptive change program leadership.  He is an independent consultant and works in both the public and private sector.  He has extensive international experience in the USA, UK, Canada, Hong Kong, Germany and offshore development in Bangalore.

By the time that the term ‘Agile leadership’ was first coined, Brian had already been successfully leading iterative, adaptive projects for over 10 years on both sides of the Atlantic.  He works as a hands-on program director and has real-world implementation expertise together with a comprehensive understanding of the related international research.  He has consulted for major strategic international organizations such as Deloitte, PwC, Gartner Group, the National Audit Office in London and Seer Technologies in North Carolina. His comprehensive public sector experience includes the Department for International Development (DFID), the World Bank, the United Nations (Geneva), and local government authorities.

Brian is a Fellow of the Association for Project Management, a Fellow of the BCS and has a MBA from Henley Management College.  He applies adaptive planning approaches as an offshore Yachtmaster and as a keen off-piste skier.  He is currently consulting for the UK Government in London.

Read Brian’s blog, visit his website and of course buy the book!

Get in touch with us anytime or leave a comment here on the blog. Help support the SPaMCAST by reviewing and rating it on iTunes. It helps people find the cast. Like us on Facebook while you’re at it.

Next week we will feature our essay on Scrumban. Scrumban is the combination of agile (Scrum) and lean (Kanban) concepts that can be used to manage projects.

Upcoming Events

StarEast

I will be speaking at the StarEast Conference May 4th – 9th in Orlando, Florida.  I will be presenting a talk titled, The Impact of Cognitive Biases on Test and Project Teams. Follow the link for more information on StarEast.

An ITMPI Webinar!

On June 3 I will be presenting the webinar titled “Rescuing a Troubled Project With Agile.” The webinar will demonstrate how Agile can be used to rescue troubled projects.  Your will learn how to recognize that a project is in trouble and how the discipline, focus, and transparency of Agile can promote recovery. Register now!

Upcoming DCG Webinars:

May 22 11:30 EDT – Agile User Stories
June 19 11:30 EDT – How To Split User Stories
July 24 11:30 EDT - The Impact of Cognitive Bias On Teams

Check these out at www.davidconsultinggroup.com

I look forward to seeing or hearing all SPaMCAST readers and listeners at all of these great events! 

 The Software Process and Measurement Cast has a sponsor.

As many you know I do at least one webinar for the IT Metrics and Productivity Institute (ITMPI) every year. The ITMPI provides a great service to the IT profession. ITMPI's mission is to pull together the expertise and educational efforts of the world's leading IT thought leaders and to create a single online destination where IT practitioners and executives can meet all of their educational and professional development needs. The ITMPI offers a premium membership that gives members unlimited free access to 400 PDU accredited webinar recordings, and waives the PDU processing fees on all live and recorded webinars. The Software Process and Measurement Cast some support if you sign up here. All the revenue our sponsorship generates goes for bandwidth, hosting and new cool equipment to create more and better content for you. Support the SPaMCAST and learn from the ITMPI.

 Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: "This book will prove that software projects should not be a tedious process, neither for you or your team." Support SPaMCAST by buying the book here.

Available in English and Chinese. 

Categories: Process Management

We Can Know the Business Value of What We Build?

Herding Cats - Glen Alleman - Sun, 04/20/2014 - 01:58

In a recent post titled #NoEstimates - Really? there was an interesting comment.

Clearly, the business value of any feature or project can not be known with much certainty in advance of it being implemented. Still, for the purpose of keeping the analysis simple for now, let’s table this issue for a bit.

This is not the case in a governance based organization or a Capabilities Based Planning organization, where the "valuation" of the resulting product, service, or purchased or built product is part of the planning process.

It's a "build to valuation"

Knowing - to some probabilistic level of confidence - what business value or mission fulfillment the project or product will produce is the core of any decision making process. Knowing the cost of the value is about making decisions, analysis of alterntaives, or assessing the trade space

With the "estimated" value and the "estimated" cost for that value, a decision can be made in what is called "analysis of alternatives" in our software intensive domain.

Only by having both estimates - value and  cost of acheiving  that value, their most likely numbers and the probabilistic range of those numbers (measured usually in dollars), can we make that "analysis of alternatives," or "trade space" needed to Govern both the business and the project and products that enable the business to meet its goals.

&

So there are popular myths around the estimating of cost and value discussion, and a few that are just flat out bogus:

  • We can't know the value of our outcomes, until they are done.
    • Start with a Balanced Scorecard strategy approach, where you define the needed value of any work before starting, determine the estimated cost of achieving that value for the busines or mission and do the math of ROI = (Value - Cost) / Cost.
  • No requirement can be confirmed to be correct until the software is complete and put to work.
    • Read any - my favorite The Requirements Engineering Handbook - requirements management book to see if this makes any sense at all.
    • This assumes - the myth that is - that those developing the software really don't know what done looks like in any units of measure meanigful to the decision makers. Requirements definition is the role of Systems Engineering. So if the developers can't know, maybe they need to the Systems Engineers. Don't have Systems Engineers? Now that's a different problem.
    • All the Capabilities Based Planned and Requirements Management processes are based on assessing the value of those requirements before the product or service arrives.
    • Assessing that value is part of all Systems Engineering process using Measures of Effectiveness and Measures of Performance.
    • During the project's execution, Technical Performance Measures and their supporting Quantifiable Backup Data are used to assue that what is being built is meeting the needs of the customer.
  • Making Estimates is a waste of time when we don;t know what the requirements are.
    • This is true.
    • But if you don't know what the requirements are, someone has to pay to find out. This someone is usually the customer. This is the basis of Agile, and it works.
      • But someone has to pay. 
    • Even in the realm where the requirements are vague, there is hopefully some  notion of what Capabilities are needed from the project or product.
    • What are you going to do with the results of the project or the product when it arrives?
    • How much are you willing to pay for those capabilities?
    • Don't have the answers to some level of confidence? - Just set fire to the money now and save all the effort.
  Related articles The "Real" Root Cause of IT Project Failure Resources for Moving Beyond the "Estimating Fallacy" 5 Questions That Need Answers for Project Success Why Johnny Can't Estimate or the Dystopia of Poor Estimating Practices
Categories: Project Management

Teams and Overly Self-interested Behavior

Overly Self-interested Behavior Is Bad For Teams!

Overly Self-interested Behavior Is Bad For Teams!

Hand Drawn Chart Saturday

Teams are an important concept in most IT organizations, regardless of their development philosophy. Philosophies like Agile may put more emphasis on teams, but even in organizations that do not embrace Agile philosophies, teams are important. Dan Ariely in his Ted Talk, “What Makes Us Feel Good About Our Work” suggested that overly self-interested, cynical behavior can negatively impact organizations by reducing their ability to communicate and innovate. The same problem can occur, albeit on a small scale, at the team level. In a recent presentation, a fellow Agile coach described a team engaged in overly self-interested behavior. He described a scenario in an organization that cuts the bottom 10% of all groups annually and stated vision that IT should maximize the value it deliverers to its customers. After losing a popular team member the previous year, the team had decided to make sure that his replacement was given the worst assignments in order to ensure he stayed on the low end of the performance scale in the coming year. Their goal was to ensure that the core team stayed intact during the next review cycle. In their mind, keeping the core team together ensured that they would deliver more value to their internal customers. The behavior of the team attempted to circumvent the idea that adding new and more highly qualified personnel would lead to improved performance. Viewed from the point of view of organizational policy, the whole team was acting in an overly self-interested behavior manner, but from the point of view of core team they are acting rationally and within their interpretation of the rules as seen through IT’s vision of value delivery. The team did not believe that their behavior was at odds with the behavior the organization wanted to incent.

What we perceive as overly self-interested at a team level is based on collective cognitive biases. Biases are powerful psychological filters that affect how both individuals and teams perceive the world around themselves and then guide how they behave. Biases reflect shortcuts in how we interpret and react to stimulus. In many cases, these reactions are valuable, however they can also cause problems (as many shortcuts occasionally can). Understanding how biases impact how individuals and teams perceive the world around them can help teams make better decisions and therefore deliver value more effectively. In the example, the core team had decided to protect itself by defining the new person as an outsider, which allowed them to hold them at arm’s length. In-group favoritism is a typical team level bias that causes teams to favor those inside the team’s boundaries over those perceived to be on the outside. This type of bias negatively affect how outsiders are perceived to perform. Negative perceptions of outsiders will effect whether we listen to their ideas and whether we trust them to deliver value.

Teams need to break overly self-interested patterns of behavior by striving ensure that they are pursuing a goal that is greater than team’s own self-interest. At a project level, product owners need to clearly identify and communicate the overall value proposition, so the team has something greater than themselves to pursue. When behavior goes off track coaching can help to identify issues that the team can’t see and diagnosis themselves. However, in many cases overly self-interested behavior at the team level can be a reflection of poor philosophies at the organization level. Organizational philosophies, like decimation or objectives that foster individual competition rarely support intergroup communication and innovation.


Categories: Process Management

Neo4j: Cypher – Creating a time tree down to the day

Mark Needham - Sat, 04/19/2014 - 22:15

Michael recently wrote a blog post showing how to create a time tree representing time down to the second using Neo4j’s Cypher query language, something I built on top of for a side project I’m working on.

The domain I want to model is RSVPs to meetup invites – I want to understand how much in advance people respond and how likely they are to drop out at a later stage.

For this problem I only need to measure time down to the day so my task is a bit easier than Michael’s.

After a bit of fiddling around with leap years I believe the following query will create a time tree representing all the days from 2011 – 2014, which covers the time the London Neo4j meetup has been running:

WITH range(2011, 2014) AS years, range(1,12) as months
FOREACH(year IN years | 
  MERGE (y:Year {year: year})
  FOREACH(month IN months | 
    CREATE (m:Month {month: month})
    MERGE (y)-[:HAS_MONTH]->(m)
    FOREACH(day IN (CASE 
                      WHEN month IN [1,3,5,7,8,10,12] THEN range(1,31) 
                      WHEN month = 2 THEN 
                        CASE
                          WHEN year % 4 <> 0 THEN range(1,28)
                          WHEN year % 100 <> 0 THEN range(1,29)
                          WHEN year % 400 <> 0 THEN range(1,29)
                          ELSE range(1,28)
                        END
                      ELSE range(1,30)
                    END) |      
      CREATE (d:Day {day: day})
      MERGE (m)-[:HAS_DAY]->(d))))

The next step is to link adjacent days together so that we can easily traverse between adjacent days without needing to go back up and down the tree. For example we should have something like this:

(jan31)-[:NEXT]->(feb1)-[:NEXT]->(feb2)

We can build this by first collecting all the ‘day’ nodes in date order like so:

MATCH (year:Year)-[:HAS_MONTH]->(month)-[:HAS_DAY]->(day)
WITH year,month,day
ORDER BY year.year, month.month, day.day
WITH collect(day) as days
RETURN days

And then iterating over adjacent nodes to create the ‘NEXT’ relationship:

MATCH (year:Year)-[:HAS_MONTH]->(month)-[:HAS_DAY]->(day)
WITH year,month,day
ORDER BY year.year, month.month, day.day
WITH collect(day) as days
FOREACH(i in RANGE(0, length(days)-2) | 
    FOREACH(day1 in [days[i]] | 
        FOREACH(day2 in [days[i+1]] | 
            CREATE UNIQUE (day1)-[:NEXT]->(day2))))

Now if we want to find the previous 5 days from the 1st February 2014 we could write the following query:

MATCH (y:Year {year: 2014})-[:HAS_MONTH]->(m:Month {month: 2})-[:HAS_DAY]->(:Day {day: 1})<-[:NEXT*0..5]-(day)
RETURN y,m,day
2014 04 19 22 14 04

If we want to we can create the time tree and then connect the day nodes all in one query by using ‘WITH *’ like so:

WITH range(2011, 2014) AS years, range(1,12) as months
FOREACH(year IN years | 
  MERGE (y:Year {year: year})
  FOREACH(month IN months | 
    CREATE (m:Month {month: month})
    MERGE (y)-[:HAS_MONTH]->(m)
    FOREACH(day IN (CASE 
                      WHEN month IN [1,3,5,7,8,10,12] THEN range(1,31) 
                      WHEN month = 2 THEN 
                        CASE
                          WHEN year % 4 <> 0 THEN range(1,28)
                          WHEN year % 100 <> 0 THEN range(1,29)
                          WHEN year % 400 <> 0 THEN range(1,29)
                          ELSE range(1,28)
                        END
                      ELSE range(1,30)
                    END) |      
      CREATE (d:Day {day: day})
      MERGE (m)-[:HAS_DAY]->(d))))
 
WITH *
 
MATCH (year:Year)-[:HAS_MONTH]->(month)-[:HAS_DAY]->(day)
WITH year,month,day
ORDER BY year.year, month.month, day.day
WITH collect(day) as days
FOREACH(i in RANGE(0, length(days)-2) | 
    FOREACH(day1 in [days[i]] | 
        FOREACH(day2 in [days[i+1]] | 
            CREATE UNIQUE (day1)-[:NEXT]->(day2))))

Now I need to connect the RSVP events to the tree!

Categories: Programming

Neo4j 2.0.1: Cypher – Concatenating an empty collection / Type mismatch: expected Integer, Collection<Integer> or Collection<Collection<Integer>> but was Collection<Any>

Mark Needham - Sat, 04/19/2014 - 20:51

Last weekend I was playing around with some collections using Neo4j’s Cypher query language and I wanted to concatenate two collections.

This was easy enough when both collections contained values…

$ RETURN [1,2,3,4] + [5,6,7];
==> +---------------------+
==> | [1,2,3,4] + [5,6,7] |
==> +---------------------+
==> | [1,2,3,4,5,6,7]     |
==> +---------------------+
==> 1 row

…but I ended up with the following exception when I tried to concatenate with an empty collection:

$ RETURN [1,2,3,4] + [];
==> SyntaxException: Type mismatch: expected Integer, Collection<Integer> or Collection<Collection<Integer>> but was Collection<Any> (line 1, column 20)
==> "RETURN [1,2,3,4] + []"
==>                     ^

I figured there was probably some strange type coercion going on for the empty collection and came up with the following work around using the RANGE function:

$ RETURN [1,2,3,4] + RANGE(0,-1);
==> +-------------------------+
==> | [1,2,3,4] + RANGE(0,-1) |
==> +-------------------------+
==> | [1,2,3,4]               |
==> +-------------------------+
==> 1 row

While writing this up I decided to check if it behaved the same way in the recently released 2.0.2 and was pleasantly surprised to see that the work around is no longer necessary:

$ RETURN [1,2,3,4] + [];
==> +----------------+
==> | [1,2,3,4] + [] |
==> +----------------+
==> | [1,2,3,4]      |
==> +----------------+
==> 1 row

So if you’re seeing the same issue get yourself upgraded!

Categories: Programming

10 Big Ideas from Mindset: The New Psychology of Success

My parents taught me early on to focus on growth over greatness.

The idea was that while natural ability can take you only so far, it’s things like curiosity, challenges, continuous learning, the power of persistence, taking risks, etc. that would take you further.

They also taught me that if I worried about whether I was naturally good, that I would give up on things where I didn’t start off so great.

It was great advice, even if it wasn’t scientific.

But there is science.

In fact, there’s a lot of science about how choosing a growth mindset over a fixed mindset help people to become the best in their field.  A growth mindset is what actually creates better parents, teachers, coaches, and CEOs.   A growth mindset creates better students, better artists, and even better geniuses.

Why?

Because people with a growth mindset embrace the challenges, struggles, criticisms, and setbacks as a source of growth.

And that’s how they rise above any limitation of “natural” ability.

Teaching, learning, and continuous growth takes them further than relying on talent or fear of taking risks where they might look bad or might not start off so great.

Carol S. Dweck, Ph.D.  wrote an outstanding book on how our mindsets shape us and how they can limit or enable us to realize our potential.

I wrote up my take aways using a “10 Big Ideas from …” style:

10 Big Ideas from Mindset: The New Psychology of Success

I think you'll enjoy the insights and I think you’ll appreciate how you can apply them to work and life.

Categories: Architecture, Programming

Get Up And Code 50: Fitness Life Change With Miguel Carrasco

Making the Complex Simple - John Sonmez - Sat, 04/19/2014 - 15:00

I finally got a chance to interview Miguel Carrasco. I’ve been watching as he has successfully launched a new business around his passion for fitness. Miguel has an awesome perspective on life and is helping many people achieve their fitness goals. Best part, Miguel just recently left his life as a developer to pursue this […]

The post Get Up And Code 50: Fitness Life Change With Miguel Carrasco appeared first on Simple Programmer.

Categories: Programming

Neo4j: Cypher – Creating relationships between a collection of nodes / Invalid input ‘[‘:

Mark Needham - Sat, 04/19/2014 - 07:33

When working with graphs we’ll frequently find ourselves wanting to create relationships between collections of nodes.

A common example of this would be creating a linked list of days so that we can quickly traverse across a time tree. Let’s say we start with just 3 days:

MERGE (day1:Day {day:1 })
MERGE (day2:Day {day:2 })
MERGE (day3:Day {day:3 })
RETURN day1, day2, day3

And we want to create a ‘NEXT’ relationship between adjacent days:

(day1)-[:NEXT]->(day2)-[:NEXT]->(day3)

The most obvious way to do this would be to collect the days into an ordered collection and iterate over them using FOREACH, creating a relationship between adjacent nodes:

MATCH (day:Day)
WITH day
ORDER BY day.day
WITH COLLECT(day) AS days
FOREACH(i in RANGE(0, length(days)-2) | 
  CREATE UNIQUE (days[i])-[:NEXT]->(days[i+1]))

Unfortunately this isn’t valid syntax:

Invalid input '[': expected an identifier character, node labels, a property map, whitespace, ')' or a relationship pattern (line 6, column 32)
"            CREATE UNIQUE (days[i])-[:NEXT]->(days[i+1]))"
                                ^

It doesn’t seem to like us using array indices where we specify the node identifier.

However, we can work around that by putting days[i] and days[i+1] into single item arrays and using nested FOREACH loops on those, something Michael Hunger showed me last year and I forgot all about!

MATCH (day:Day)
WITH day
ORDER BY day.day
WITH COLLECT(day) AS days
FOREACH(i in RANGE(0, length(days)-2) | 
  FOREACH(day1 in [days[i]] | 
    FOREACH(day2 in [days[i+1]] | 
      CREATE UNIQUE (day1)-[:NEXT]->(day2))))

Now if we do a query to get back all the days we’ll see they’re connected:

2014 04 19 07 32 37
Categories: Programming

Brainstorming . . . Well Maybe Not

Crowds are susceptible to groupthink.

Crowds are susceptible to groupthink.

Brainstorming has been a staple in the business world since its creation in the mid-1940s.  Other forces have combined to reinforce the technique such as the mania over crowd sourcing and consensus management styles.  The problem is that the data shows that the technique is not as effective as we all believe, and in some cases can actually be unproductive – such as when groupthink occurs.  There are better ways to generate innovative ideas.

Brainstorming is a process that through free association generates ideas to find a solution or conclusion for a specific problem.  There are a core set of tenants that define brainstorming.  It is generally a group activity that includes a focus on generating as many ideas as possible (quantity), which includes welcoming unusual ideas, combining and improving ideas and the avoidance of criticism.  Great stuff!   However, research[1] has recently questioned whether the process is as effective and efficient as the popularity of the method would suggest.  The research suggests that the reliance on groups and a lack of debate and criticism causes the technique to be less effective at generating creative ideas than other techniques.

Why do you and I care about this topic?  Developing new ideas and innovative solutions is an integral part of software development, enhancement and maintenance.  Using the most effective techniques, or at least knowing what the most effective techniques, is of more just of academic interest.

Groupthink happens when the desire for harmony in a group overrides a realistic appraisal of alternatives.  The after effects of a groupthink gone wrong might include the passive aggressive comment: “I knew this wouldn’t work but did not say anything.”  “Idea gravity wells” and the forbearance of criticism help to create the problem.

Brainstorming and other group techniques for generating ideas leverage the diversity of thought a well-defined group[2] can create. Groups that allow a single powerful individual to dominate can create an idea gravity well that curtails innovation or green-field thinking as new ideas can’t escape the status quo. The problem is that conclusions that have been influenced by groupthink may not hold up in the bright light of the marketplace.

The second criticism is the lack of immediate feedback.  Forbearing debating and criticizing ideas, like a critique in art school, avoids exploring the depths of an idea and deciding quickly what is relevant.  Critiquing ideas makes sure the good parts are honed and built upon and the bad stuff either refined or sloughed off.   Debate, discussion and criticism create a platform to provide immediate feedback, allowing a group to reshape the idea being examined rather than building on a poor base.

I have been experimenting with and honing a process that attacks two of the major criticisms of brainstorming.  The process begins by making sure you have the right team (see the well-formed team footnote).  I strongly suggest using video if you have remote participants.  Make sure you have someone that will act as the scribe and someone that will also act as the moderator for the session.  Second, provide the “team” with the topic being explored and all background information before the planned session.   Each team member is responsible for reviewing the data and developing a list of their best ideas . . . individually.  Generating the initial set of ideas helps to avoid the possibility of groupthink or idea gravity well occurring directly out of the box.  Note: The list of ideas from each person is the price of admission; lack of preparation should bar a potential participant from the session.

The ideation session begins with a round-robin presentation of ideas from each participant with discussion, debate, criticism and refinement (no fisticuffs or off-topic criticisms).  The round-robin presentation generally continues until everyone has put at least one idea on the table or until the initial set of ideas are used up.  I allow and encourage participants to switch to more of free-for-all for laying out and discussing ideas as soon as everyone has completed laying out at least one idea.  Get one idea on the table from each person makes sure all of the participants understand that they have permission to participate.

I allow the process to continue until team members’ lose the will to live, run out of ideas or reach a consensus on an idea or idea set.  I believe that the facilitator or moderator should push a team at least slightly past what is comfortable in order to generate potentially extraordinary results.  Pushing past what is comfortable is one of the reasons I ask interviewees on my podcast for two ideas to solve the problem presented at the end of the interview rather than just one, which would be far more comfortable (note off the record: many interviewees have indicated that one response to the “two things” question would be far easier).

My wife suggests that after the session is complete, give everyone twenty four hours to chew on the topic and then reconvene to make sure that reflection has not provided tweaks to make the solution or solution set better.  Remember that not everyone thinks at the same rate as everyone else.

Why do we think brainstorming works?  We all have experience with the process.  It is comfortable, it is safe and in many cases it generates good ideas . . . just not the best ideas possible.  To brainstorm or not to brainstorm, that is the question, perhaps not exactly Shakespeare, but relevant nevertheless.  As we gain a better understanding of what works and why, when it comes to generating innovative ideas isn’t it time to put aside preconceived notions based on hearsay?  The data from academia suggests that what we know is based on just that — hearsay.  Don’t let data get in the way, you might say; we believe what we believe.  This week I announced to a group that brainstorming was not as effective as other techniques in terms of generating innovative ideas.  You would have thought that I had insulted their children.  Everyone has a story about successes using brainstorming.  Those successes may well have helped us get out of tight situations, but because of those successes it is hard to acknowledge that there are better ways to generate solutions and new ideas.  This is true whether we are discussing idea generation, software development or Marigolds and Moonflowers (something absolutely unrelated). Brainstorming may still have a place at the innovation table, but it should no longer sit at the head of the table.  There are better ways to generate creative solutions and now is not the time to leave creative ideas on the table . . . maybe it is never time to leave creative ideas on the table!

[1] There are several sources of research on brainstorming including:
http://www.cpsb.com/resources/downloads/public/302-Brainstorm.pdf
http://www.cbsnews.com/8301-505125_162-30640906/the-case-against-brainstorming/

http://www.tcp-uk.co.uk/creative_articles_nemeth.htm

[2] Well-formed means that only the minimum number of people should participate and that those that participate include a range of experience and backgrounds.


Categories: Process Management

Why Projects Fail, No Matter the Domain

Herding Cats - Glen Alleman - Fri, 04/18/2014 - 23:07

There are endless reasons for why projects fail. Some are correct, some are bogus, some are down right nonsense. Into this list I'd like at add my opinion, informed by hands on experience on software intensives programs in defense, energy, bio-pharma, commercial products, and enterprise IT.

It starts and ends, with the

Failure to know what DONE looks like in units of measure meaningful to the decision makers

The starting point for the description of DONE is the clear and concise statement of the Needed Capabilities for the resulting project that produce value for the stakeholders. These capabilities are stated in Measures of Effectiveness.

These capabilities state what the results from the project will do for the business or mission when they are available. The planned availability date is part of their capability. Only then come the requirements, then planning for the delivery of the technical and operational components that enable these requirements. Then several more enablers of projects success are needed.

All these activities, outcomes, processes, methods are encapsulated in the paradigm of Systems Engineering. The picture below are the notional elements of Systems Engineering. This picture is from the FAA Systems Engineering Handbook. This handbook is for the National Airspace System (NAS), a software intensive program to upgrade the exiting software, hardware, and processes. Other agencies and business have similar pictures.

SE Elements

  • Synthesis - Transforms requirements into physical solutions.
  • Functional Analyses - describes the functional characteristics that are used to derive the products or services of the system resulting from the project
  • Requirements Management - identifies and manages the requirements that describe the desires capabilities of the project
  • Interface Management - identifies and manages the interactions between components of the system or interactions with external systems
  • Integrated Technical Planning - Plans the projects efforts and products
  • Speciality Engineering - Analyzes the system, requirements, functions, solutions, and/or interfaces using specialized skills and tools. Assists in the derivation of requirements, synthesis of solutions, selection of alternatives, and validation and verification of requirements.
  • Integrity of Analyses - Ensures that the analyses provide the required level of fidelity and accuracy.
  • Lifecycle Engineering - Identifies and manages requirements for system lifecycle attributes, including real estate management, deployment and transition, integrated logistics support, sustainment / technology evolution, and disposal.
  • Risk Management - Identifies, analyzes, and manages the uncertainties of achieving program requirements by developing strategies to reduce the severity or likelihood of those uncertainties.
  • Trade Studies - assists decision making by analyuzing selecting the best-balanced solutions to match requirements that enable the capabilities
  • Configuration Management - Establishes and maintains consistency and manages change in the system performance, functional, and physical attributes.
  • Verification and Validation - Determines if system requirements are correct. Determines that the solution meets the validated requirements.

What Does All This Mean?

When we hear our customers don't know what they want. Or, our customers don't know the target budget for the work they are asking us to do for them. What they are saying is I don't know what DONE looks like in an tangible way that can be used to measure the performance of the project.

So one of two things has to happen when this is the case...

  1. The customer has to spend money to find out what they actually want. This is done all the time on our larger enterprise, defense, and space programs. It's paying to explore. It's establishing the credibility of the capabilities by spending money to do so
  2. Prepare for project failure.

So when we here that famous - infamous actually - phrase let's explore. Ask a serious question, and demand a serious answer ...

Who's pay for us to explore to discover what we should have already know? 

Don't get an answer? Run away from that project, it's going to be a failed project.

 

Categories: Project Management

Unicode

Eric.Weblog() - Eric Sink - Fri, 04/18/2014 - 19:00

(This entry is part of a series. The audience: SQL Server developers. The topic: SQLite on mobile devices.)

Well, Actually

First, go read this blog entry by Miguel de Icaza. Right now. I'll wait.

Welcome back. Now let me apologize. I don't want to be a pedantic jerk who quibbles about minor details. But the topic here is Unicode, so there really is no other way.

All Unicode, all the time

The relevant difference with SQLite is easy to describe:

  • In the world of Windows and SQL Server, you have all kinds of possible code pages.

  • In SQLite, everything is Unicode.

But if you don't have much context on these issues, I haven't really told you much. Let's go further.

But let's not go too far

I don't want to rewrite articles that have already been written quite well. So you should probably also go read this blog entry by Joel Spolsky. I'll be here when you get back.

OK, now let's get started

SQL Server has two basic ways you store text:

  • You can use the char/varchar types, which can be used with one of several collations, each of which implies a specific code page, which implies a specific character encoding.

  • Or you can use the nchar/nvarchar types (note the extra 'n'), which are Unicode.

SQLite has no such distinction. All text in SQLite is Unicode.

What the heck is Unicode again?

It's a character set: a collection of characters, each with a number that can be used to refer to it.

More specifically, it's the only character set which is [trying to be] complete. If you choose any character set or encoding which is not Unicode, there will be characters you cannot use.

And what's an encoding?

Saying that SQLite uses Unicode doesn't tell you how the text is actually represented. Unicode is not an encoding. It is more abstract than that. There are lots of different ways of representing Unicode as bytes.

Microsoft's sad history with Unicode

In the Windows and SQL Server world, there is a long history of encoding Unicode in 16-bit-ish ways. Originally, this was UCS-2, a simple encoding which represents each character as a 16-bit number. But then the Unicode consortium realized that 16 bits are not enough, so they expanded the space to 32 bits. This left Microsoft in an awkward spot. UCS-2 is a fundamentally defective encoding of Unicode, since there are many characters in Unicode that simply cannot be represented.

If the goal of Unicode is to be complete, it is reasonable to say that, well actually, UCS-2 is not Unicode.

The conceptual replacement for UCS-2 is to use 32 bits for every character. This encoding is called UCS-4 or UTF-32. But now the wasted space for storing a simple English string is getting out of hand. Switching the complete works of Arthur Conan Doyle from ASCII (which is also an encoding) to UCS-4 would take four times as much space.

Gradually, the world seems to be adopting UTF-8 as the most popular Unicode encoding. This is a variable width encoding. Sometimes a single character is represented with just one byte. Sometimes it needs more. That's very unfortunate, but the only fixed width alternative is UCS-4, which is also very unfortunate. Choose which problem you prefer, but keep in mind that almost everybody has chosen to accept the problems of UTF-8.

But Microsoft has so much history with UCS-2 that transitioning everything to UTF-8 would be really hard. So they have been moving from UCS-2 to UTF-16, which is basically a variable width encoding built around a 16-bit unit instead of an 8-bit unit. UTF-16 is approximately the worst correct way of representing Unicode, unless you have invested billions of dollars in the fundamentally broken UCS-2, in which case UTF-16 is a pretty awesome way out of the mess you ended up in.

Just remember that if you're going out tonight to a club for pedantic nerds and you want to impress someone, you've got to keep the terminology straight:

  • Unicode is an abstraction, not an encoding, not a code page, not a data format, and not a font.

  • Saying your text is Unicode says nothing about how it is represented. It might be UTF-8. It might UTF-16. It might be code point numbers handwritten on Post-It notes stuck on the wall. All of these are valid representations of Unicode.

  • If you ever say anything to suggest that you think Unicode is 16 bits per character, you will be identified as clueless.

  • If you say that your text is stored in Unicode, you are not entirely incorrect, but people will wonder about whether you really know the difference between Unicode and the encodings of same.

SQLite

SQLite always uses Unicode to represent text.

(Hopefully you are now screaming at me saying, "Yeah, but which encoding?!?")

The best answer to this question is: SQLite uses UTF-8. Forget about everything else.

A more correct answer to this question is: SQLite uses UTF-8 but also supports UTF-16.

Either way, there is no distinction between char and nchar. There is no way to save storage space in a column by realizing that you only use lower case english characters so it's safe to use char instead of nchar. There are no code pages. There is no way to move your Shift JIS data into SQLite without converting it to Unicode (or storing it as blobs, I suppose).

Summary

Microsoft has done a lot of things right, but its history with Unicode is very unfortunate. And it's not entirely their fault. They adopted Unicode early and it changed underneath them.

With respect to its emphasis on UTF-8, SQLite is far more typical of most non-Microsoft software today.

 

Stuff The Internet Says On Scalability For April 18th, 2014

Hey, it's HighScalability time:


Scaling to the top of "Bun Mountain" in Hong Kong
  • 44 trillion gigabytes: size of the digital universe by 2020; 6 Times: we have six times more "stuff" than the generation before us.
  • Quotable Quotes:
    • Facebook: Our warehouse stores upwards of 300 PB of Hive data, with an incoming daily rate of about 600 TB.
    • @windley: The problem with the Internet of Things is right now it’s more like the CompuServe of Things
    • Chip Overclock: If you want to eventually generate revenue, you must first optimize for developer productivity; everything else is negotiable.
    • @igrigorik: if you are gzipping your images.. you're doing it wrong: http://bit.ly/1pb8JhK  - check your server configs! and your CDN... :)
    • Seth Lloyd: The arrow of time is an arrow of increasing correlations.
    • @kitmacgillivray: When will Google enable / require all android apps to have full deep search integration so all content is visible to the engine?
    • Neal Ford: Yesterday's best practices become tomorrow's anti-patterns.
    • Rüdiger Möller: just made a quick sum up of concurrency related issues we had in a 7-15 developer team soft realtime application (clustered inmemory data grid + GUI front end). 95% of threads created are not to improve throughput but to avoid blocking (e.g. IO, don't block messaging receiver thread, avoid blocking the event thread in a GUI app, ..).
    • Ansible: When done correctly, automation tools are giving them time back -- and helping out of this problem of needing to wear many hats.

  • Amazon and Google are in an epic battle to dominate the cloud—and Amazon may already have won: If Amazon’s entire public cloud were a single computer, it would have five times more capacity than those of its next biggest 14 competitors—including Google—combined. Every day, one-third of people who use the internet visit a site or use a service running on Amazon’s cloud.

  • What books would you select to help sustain or rebuild civilization? Here's Neal Stephenson’s list. He was too shy to do so, but I would certainly add his books to my list. 

  • 12-Step Program for Scaling Web Applications on PostgreSQL. Great write up of lessons learned by wanelo.com that they used to sustain 10s of thousand of concurrent users at 3K req/sec. Highly detailed. 1) Add more cache, 2) Optimize SQL, 3) Upgrade Hardware and RAM, 4) Scale reads by replication, 5) Use more appropriate tools, 6) Move write heave table out, 7) Tune Postures and your File System, 8) Buffer and serialize frequent updates, 9) Optimize DB Schema, 10) Shard busy tables vertically, 11) Wrap busy tables with services, 12) Shard services backend horizontally.

  • Is this a soap opera? It turns out Google and not Facebook is buying Titan Aerospace, makers of high flying solar powered drones. Google's fiber network would make a great backbone network for a drone and loon powered wireless network, completely routing around the telcos.

  • Building Carousel, Part I: How we made our networked mobile app feel fast and local. Dropbox changes to an eventualy consistent / optimistic replication syncing model to make their app "feel fast, responsive, and local, even though the data on which users operate is ultimately backed by the Dropbox servers."  Lesson: don’t block the user! Instead of requiring changes to be propagated to the server synchronously. Local and remote photos are treated as equivalent objects. Actions take effect immediately locally then work there way out globally. Changes are used to stay consistent. A fast hash is used to tell what photos have not been backed up to dropbox. Remote operations happen asynchronously.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...

Categories: Architecture

EVM World Coming Soon

Herding Cats - Glen Alleman - Fri, 04/18/2014 - 15:04

Speaking on Techncial Performance Measures and conducting workshop on same topic with Mr. Kranz and Tom Coonce of IDA.

Categories: Project Management

Hanging Out

NOOP.NL - Jurgen Appelo - Fri, 04/18/2014 - 10:21
Hanging out...

This week I spent more time hangout out with people, which means less time is available for my blog. Sorry about that! (Also, I have a book to finish…)

The post Hanging Out appeared first on NOOP.NL.

Categories: Project Management

Mind Mapping: Brainstorming and Other Uses

Mind maps can be useful for note taking.

Mind maps can be useful for note taking.

Mind mapping is a technique for mapping information. A basic mind map typically emanates from a central topic with subdivisions branching out from that topic. The process for mind mapping has few basic rules and suggestions for constructing and formatting mind maps, which makes them highly flexible. Mind maps have a wide variety of uses based on one central theme: learning. The uses of mind maps include:

  1. Note taking: Most lectures tend to follow a more linear outline with relationships and linkages between topics inferred. Standard note taking is generally a reflection of how the lecturer thinks rather than how the note taker thinks. Mind mapping helps the note taker to capture the branches of the topic and then to visualize the linkages. Structuring notes based on how the note taker thinks makes memory recollection easier. Note: I occasionally use this technique to restructure standard notes as a means reinforce my memory.
  2. Planning: Few plans are linear. Mind maps are useful tools for planning and visualizing program-level backlogs. The branching attributes of the map provide a tool to show how functionality breaks down and then visualize the linkages (dependencies) between the entries. While every story should be independent, story and task independence is generally a goal rather than a fait accompli in many organizations. A second use for mind maps in the planning category is as a tool to capture sprint planning results. The sprint goal serves as the central theme with stories radiating from the theme. Activities and task branch from stories. Relationships can be added to show predecessors and successors (or as a trigger for re-planning).
  3. Research: Using a mind map a tool in research is very similar to how mind mapping is used in note taking, with a few subtle differences.  The first is that the mind mapper is generally gathering data from multiple sources while looking for gaps or unnoticed relationships as data is acquired. I often use mind maps as tools for gathering and reorganizing information that I collect (the ability to reorganize data is a strength for most tool-based mind mapping solutions). Many tools support clipping URLs and information for bibliographical entries.       The real power of using mind maps as a research tool is the ability to visualize the data collected, which generally makes gaps obvious and can be useful when looking for relationships between branches of the research.
  4. Presentation Tool: In the entry Mind Mapping: An Introduction, I recounted the story of Ed Yourdon using a mind map to direct his presentation. I have developed mind maps as a precursor to building a classic PowerPoint presentation. When developing or giving a presentation from a mind map, the flow of information reflects how you have visualized and reflected it on the map.
  5. Organizing thoughts: This is my favorite use for a mind map. I begin the organization process by generating my central theme and then using the theme as a hub add each separate item I can think of based on that theme. I generally do this on paper without worrying about spelling or whether one or two items are duplicates. The goal is get everything that is known around the central theme. A starburst pattern will be generated. A simple example:

1Untitled

Once the starburst is created the map can be mined to establish major subdivisions and to indicate area where more research is needed. Walk through each entry and gather related items together. From the items you gather together, the name of the subdivision will emerge. For example, in our mind mapping mind map, when I gathered note taking, planning, research and other together the major subdivision titled “Uses” emerged.

2Untitled

Every Daily Process Thought essay begins using this type of mind map. Many people use the term brainstorming for this type of mind mapping activity.

Mind maps are tools for visualizing data. Seeing your thoughts put into patterns that represent how you think makes it easier to remember the ideas and concepts being mapped. Mind maps also help the mapper see gaps in the data or to jog creative thoughts by exposing relationships that do not jump out when processed linearly. Mind mapping is an extremely flexible tool, therefore there are an enormous number of uses. There is a saying that “if the only tool you have is a hammer, everything will look like a nail.” Given the varied number of uses for mind maps, perhaps they might be an information-visualization Swiss Army knife.


Categories: Process Management

DevArt - Winner Announced

Google Code Blog - Thu, 04/17/2014 - 17:00
By Emma Turpin, DevArt Lead at Google Creative Lab and Paul Kinlan, Developer Advocate

Mapping a dream as it navigates through your brain using G+APIs. Exploring metamorphosis through storytelling in the form of a poetic adventure with Chrome Apps and Compute API. Travelling through a playful giant map that explores fantasy and reality on a huge scale using Map API. Creating music through the touch of your finger on a simple piece of wood using Android.

These are just a sample of the hundreds of projects we received after inviting the developer community to express themselves creatively as part of DevArt. We were looking for a unique idea which mixes art and code and pushes the boundaries, to be featured in the Barbican's Digital Revolution exhibition, opening this summer in London and from there touring the rest of the world.

And the winner is … a duo Cyril Diagne & Béatrice Lartigue from France. Cyril and Beatrice’s project, Les métamorphoses de Mr. Kalia, is an interactive poetic adventure around the theme of metamorphosis in the human body. It invites gallery visitors to personify Mr. Kalia as he goes through many surrealistic changes [video] [project page on DevArt site]. The piece conveys feelings related to change, evolution and adaptation. Mr. Kalia is brought to life through the use of a skeleton tracking technology, and uses Chrome apps and Google Compute Engine.


Cyril and Béatrice’s installation will sit alongside three of the world’s finest interactive artists who are also creating installations for DevArt: Karsten Schmidt, Zach Lieberman, and the duo Varvara Guljajeva and Mar Canet. The Digital Revolution Exhibition will be opening in London on 3 July with tickets available online here.

We were overwhelmed by all of the amazing ideas we saw, a testament to the creativity that’s possible with code. Watch this space - DevArt at the Digital Revolution exhibition at the Barbican opens in July!

Paul Kinlan is a Developer Advocate in the UK on the Chrome team specialising on mobile. He lives in Liverpool and loves trying to progress the city's tech community from places like DoES Liverpool hack-space.


Posted by Louis Gray, Googler
Categories: Programming

Cross Platform Mobile Development (iOS, Android, WP8)

Making the Complex Simple - John Sonmez - Thu, 04/17/2014 - 16:00

What is your strategy for cross platform mobile development? It is a hard question to answer, because there are so many choices out there. In thie video, I talk about what I think about cross platform mobile development, define your options and give you my recommendation. So, if you are thinking about developing for iOS, […]

The post Cross Platform Mobile Development (iOS, Android, WP8) appeared first on Simple Programmer.

Categories: Programming

How to start your own requirements company – Part I

Software Requirements Blog - Seilevel.com - Thu, 04/17/2014 - 12:30
In 2000, I worked for a .com during the .com boom. I left my safe, secure job with Schlumberger, the biggest oil services company at the time, to try to make my fortune in the wild wild west of internet companies. After my new company went public, the stock price kept climbing and I was […]

How to start your own requirements company – Part I is a post from: http://requirements.seilevel.com/blog

Categories: Requirements

We're Asking the Wrong Question

Herding Cats - Glen Alleman - Thu, 04/17/2014 - 04:15

The right question for any suggested improvement, change, or suggesting we stop doing that need to produce an answer to ...

Does this suggestion increase the probability of project success?

This means tracing the suggestion to the outcome of the project being better, faster, cheaper, or some other tangible measure of improvement. 

What does project success look like?

The delivery of agreed-upon capability within established resource constriants, e.g. funding, schedule, facilities.

Five factors are used to assess the success

  • Capabilities - do we know what capabilities will be needed, when they'll be needed,  for what cost, to meet the business plan, competitive market position, or any other assessment of success?
    • It's the capabilities that enable the customer to do valuable things for their customers, themselves, or their citizens.
  • Requirements - do we know what technical and operational requirements must be in place to fulfill the needed capabilities?
    • It's the capabilty to do something of value that the customer bought.
  • Resources - what resources will be needed to deliver these capabilities, fulfilled by the requirements within the expected cost and time needed to meet the business or mission goals for the Return on Investment?
    • ROI = (Value - Cost) / Cost
    • As a business or funding agency, this ROI is a primary assessment of success.
    • Did we get what we planned to get for the money we thought we would spend on the day we thought we would? All within the expected probabilistic confidence range?
  • Execution - do we have a means of measuring progress toward producing value for the customer?
    • This value needs to be available at a point in time to meet the plan for the business or the mission
    • This time and cost of the available capabilities is an essential need for any business.
    • Without a goal such as this there is no real crticality to the project. If the project is not critical, then any approach can be viable. The probability of success is really not an interesting question to ask.

So Now What?

When we hear about something new, anything new, how can we test it that suggestion against business needs, mission needs, governance, or strategy. This starts with establishing the domain in which the suggestion might possibly work. Then proposing the framework in which it has worked or might work. Then a proposed way to assess the possible benefits of performing the sugegsted solution.

Categories: Project Management

Mind Mapping: Some Guidelines

 

These are suggestions, not law.

These are suggestions, not law.

Mind mapping is a technique for mapping information using color, pictures, symbols and, most importantly, a branching structure emanating from a central concept. Tony Buzan in his book, The Mind Map Book, adds four characteristics to the definition that further define mind maps. They are:

  • The topic is the central image
  • Main themes radiate from the center
  • Branches are represented by a key image or word
  • Branches are connected

I would add a fifth characteristic. Topics that are less important tend to be placed farther from the center image.

These five characteristics can be mined for rules to draw a mind map.

  • Use images.  Images engage visual and linguistic learning styles which encourage memory. Buzan suggests that the central image should be an image. When hand drawn, the use of an image provides a strong anchor for the mind map. Drawing the image adds another layer of involvement in the map through physical drawing.
  • Use single words or a short phrase when you can’t use images. When using phrases make them as short as possible.
  • Color can provide added meaning to the images and words.  For example, use of the color red conveys danger or urgency (depending on context) while green tend to portray growth.  Frankly, I am colorblind (pretty close to total), so I use color sparingly as combinations I find pleasing, but others find befuddling (that is why plaid is my favorite color – think about it). Use color to draw attention, show linkage between items or highlight items that are helpful to you. While some think of mind maps as art, I think they are a tool that just happens to look cool.
  • Weighting can be used to show the relative importance of each topic. Two types of weighting can be used.  Varying font or image size is one way to convey importance. The larger the font the more important the entry. Second, the mind mapper can vary the width of the branch. This form of weighting can be difficult or unavailable if you are using a computer-based tool for mind mapping.  The goal of weighting is to draw attention to specific features to increase memory retention or to increase readability.  Remember not all entries or subdivisions are created equal.

Here is an example of the use of images, words, colors and weighting.

1Untitled

  • Crosslinking is a way to show relationships between items that are not within the same branch.  For example in the figure below, a relationship is shown between the categories of processes and common problems.2Untitled

Linkages allow the mind mapper to layer in nuances that might not be observable without repeating entries.

I consider these rules to be important, but not absolute.  I have created many mind maps whose branches include links to websites or whole paragraphs of notes.  I broke the “rules” because breaking them provided more value to me than not breaking them.  The only two hard and fast rules are:

  1. The core topic and branch structure.
  2. All the other rules are guidelines.

The rules and guidelines for mind mapping exist to help the mind mapper get the most value from the map possible.  The map should engage as many senses and learning styles as possible to get the job done.  Tomorrow we tackle different uses of mind maps.


Categories: Process Management