Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

Blood Sweat & Code

DevHawk - Harry Pierson - Wed, 04/02/2014 - 20:17

CNNToday was a *HUGE* thrill as I got to present in the keynote at //build! I’ll have more on the specifics of Brokered WinRT Components later after my session, but apparently quite a big deal was made of my shirt. I ended up as an internet meme and on the CNN Live Blog!

A long, long time ago (back when I wrote Photoshop Plugins for Mac long before I joined Microsoft), I had a Metrowerks CodeWarrior t-shirt with the “Blood Sweat & Code” slogan on the back. I loved that slogan, but lost the shirt somewhere along the way. So a few months ago, I decided to make a new one – but this time with the purple and blue of Visual Studio’s brand instead of CodeWarrior yellow. When I got a chance to be a part of the //build keynote today, I figured it was a good wardrobe choice.

For those who want one of their own, I posted the design on Zazzle.

Categories: Architecture, Programming

The *Real* Test Driven Development

Google Testing Blog - Wed, 04/02/2014 - 16:49

Update: As noticed in the comments, the date of the post was not a mere coincidence :)

by Kaue Silveira

Here at Google, we invest heavily in development productivity research. In fact, our TDD research group now occupies nearly an entire building of the Googleplex. The group has been working hard to minimize the development cycle time, and we’d like to share some of the amazing progress they’ve made.

The Concept

In the ways of old, it used to be that people wrote tests for their existing code. This was changed by TDD (Test-driven Development), where one would write the test first and then write the code to satisfy it. The TDD research group didn’t think this was enough and wanted to elevate the humble test to the next level. We are pleased to announce the Real TDD, our latest innovation in the Program Synthesis field, where you write only the tests and have the computer write the code for you!

The following graph shows how the number of tests created by a small feature team grew since they started using this tool towards the end of 2013. Over the last 2 quarters, more than 89% of this team’s production code was written by the tool!

See it in action:

Test written by a Software Engineer:

class LinkGeneratorTest(googletest.TestCase):

def setUp(self):
self.generator = link_generator.LinkGenerator()

def testGetLinkFromIDs(self):
expected = ('https://frontend.google.com/advancedSearchResults?'
's.op=ALL&s.r0.field=ID&s.r0.val=1288585+1310696+1346270+')
actual = self.generator.GetLinkFromIDs(set((1346270, 1310696, 1288585)))
self.assertEqual(expected, actual)

Code created by our tool:

import urllib

class LinkGenerator(object):

_URL = (
'https://frontend.google.com/advancedSearchResults?'
's.op=ALL&s.r0.field=ID&s.r0.val=')

def GetLinkFromIDs(self, ids):
result = []
for id in sorted(ids):
result.append('%s ' % id)
return self._URL + urllib.quote_plus(''.join(result))

Note that the tool is smart enough to not generate the obvious implementation of returning a constant string, but instead it correctly abstracts and generalizes the relation between inputs and outputs. It becomes smarter at every use and it’s behaving more and more like a human programmer every day. We once saw a comment in the generated code that said "I need some coffee".

How does it work?

We’ve trained the Google Brain with billions of lines of open-source software to learn about coding patterns and how product code correlates with test code. Its accuracy is further improved by using Type Inference to infer types from code and the Girard-Reynolds Isomorphism to infer code from types.

The tool runs every time your unit test is saved, and it uses the learned model to guide a backtracking search for a code snippet that satisfies all assertions in the test. It provides sub-second responses for 99.5% of the cases (as shown in the following graph), thanks to millions of pre-computed assertion-snippet pairs stored in Spanner for global low-latency access.



How can I use it?

We will offer a free (rate-limited) service that everyone can use, once we have sorted out the legal issues regarding the possibility of mixing code snippets originating from open-source projects with different licenses (e.g., GPL-licensed tests will simply refuse to pass BSD-licensed code snippets). If you would like to try our alpha release before the public launch, leave us a comment!

Categories: Testing & QA

Management 3.0 Workout: One-Day Workshop (Part 1)

NOOP.NL - Jurgen Appelo - Wed, 04/02/2014 - 14:37
One-Day Workshop Part 1

While I am frantically working to finish my new book (with three copy editors and fifty proof readers) I’m also developing ideas for the new one-day workshop that is part of the global book tour.

The post Management 3.0 Workout: One-Day Workshop (Part 1) appeared first on NOOP.NL.

Categories: Project Management

Requirements for Business Analytics follow-on with more ideas

Software Requirements Blog - Seilevel.com - Wed, 04/02/2014 - 12:28
I recently gave a webinar on Requirements for Analytics Projects and had a few interesting questions come up during it. I thought I’d share some of the Q&A here, since we didn’t get to cover them all in the webinar. You can listen to the webinar here. Here are some of the questions and answers. […]

Requirements for Business Analytics follow-on with more ideas is a post from: http://requirements.seilevel.com/blog

Categories: Requirements

Learning Styles and Communication Problems

Include differences (learning styles, that is)

Include differences (learning styles, that is)

The team that completes a project will be different from the one that began the project. Each person on the team will have a range of individual experiences, and presumably, they will learn from these experiences. A mismatch of learning styles can result in communication problems. Communication problems act as a filter on what each individual learns by blocking or altering what the learner perceives.

Learning styles reflect an individual’s preference for how they will learn. In many cases individuals mirror their own learning style when share with others. Most, if not all, teams I have been associated with over my career have been comprised of individuals with different learning styles. This means that to effectively communicate and transmit knowledge, each team member must understand the learning styles of their team members (this is another reason why stable teams generally have higher levels of performance).

An example of the impact when team members do not understand each other’s learning style can be seen in a team I recently observed.  The team is a relatively new team and is distributed, with most interactions occurring via teleconference, which complicates bonding. Most team members have not had time to adjust to each other’s learning style; therefore members use their own learning patterns as a default when interacting. For example, one team member follows the logical/Lawyer learning style. When presenting information they build a case – fact by fact – in great detail. No one else on the team leverages this as their primary learning style. The great level of detail and the slow (but relentless) build to the conclusion leads to frustration and disengagement. On the same team, another member is verbal learner/Talker.  This person needs learns by hearing, and in many cases, by vocalizing each point.  This person presents information in the same manner as they learn, talking it through (with lots of Keynote slides … with no pictures). In both cases because the members are not aware of the learning styles of other team members communication is inefficient (and my observation is that it can be ineffective).

Teams that are centrally located generally recognize learning style mismatches based on visual and empathetic feedback and can self-correct (assuming that team members actually pay attention when they get together). Distributed teams generally need to take a more active approach to learning each other’s preferences. I recommend the following approach which can be used as a team building exercise or as a retrospective exercise:

  1. Before the exercise, create a couple of canned scenarios.  For example:
    1. Scenario One:  Pass status information about a trouble task, including a plea for help.
    2. Scenario Two:  Build consensus for a design decision.
  2. Have each team member identify their primary and secondary learning style.
  3. Share these styles with the team.
  4. Once all team members have shared their style, have each team member select a method that is not their primary or secondary style and have them convey the information needed in complete each scenario. (Allow 5 minutes for preparation).

The goal of exposing the team to other types of learning styles is to push each person outside their comfort zone.  This serves multiple purposes. First, the process helps to build empathy. The process also reinforces the awareness that, on a diverse team, all messages need to be shared in a variety of ways so that multiple learning styles can easily absorb the information.  Finally, by learning and becoming sensitive to other learning styles, individual team members will expand their ability to recognize nuances in communications that lay outside their normal learning style. This will ultimately increase the effectiveness and efficiency of the team.


Categories: Process Management

Start Your Journey of Innovation and Build a Culture of Great Innovation

In todays world, the mantra is innovate or die.

You’re either climbing ahead or falling backward … there’s no hanging out in the middle.

And some folks are actually leap frogging ahead.

Disruptive innovation is keeping everybody on their toes.

Whether you are re-imagining you or your company, or you are driving innovation in your process, product, or capabilities, there are skills you can learn to be a lot more effective in your innovation efforts.

It’s a crazy world where a One-Man Band can write an app, serve it up on the Cloud, and change the world.  It’s also a strange world where a little idea can be a big shot heard round the world.   It’s a scary thing for businesses when a handful of developers can spin up a new service in the Cloud and instantly make a business obsolete.

What can you hold on to in this crazy world?   What can you latch on to, if you want to rise above the noise, and instead of getting washed out by a wave, be the one that makes the waves?

There are several things, but I’ll boil them down to this:

  1. Use your customer as the North Star (and remember that some customers are better for you than others)
  2. Share and scale your unique value to the world
  3. Adapt or die

What happens to a super successful business or a super effective person when the landscape changes under their feet?

It depends on how they adapt Smile

Nature favors the flexible.  Darwin taught us that.

You have to get your bold on, and embrace innovation as your shiny sword to do battle against challenge and change, but most importantly, to create the change that serves you, and those you serve.

I’m taking a fresh look at innovation, as well as going back through hard, expensive lessons I’ve learned in the past.  Whatever doesn’t kill us makes us stronger, so my battle scars are a healthy reminder of the lessons I’ve learned on how we can use innovation to leap frog ahead, as well as change the playing field (heck with changing the game, change the field and be the disruptor.)

Believe it or not, Peter Drucker was a wealth of wisdom when it comes to innovation.  Many of you know him as the wise and wonderful professor of business and guru of management.   But when you read through a lot of his work, he was incredibly insightful and pragmatic when it comes to creating a culture of innovation.

Innovation Nuggets to Get You Started on Your Innovation Journey

I’ve got a ton of innovation books, but one that I’m really liking lately is Out Think: How Innovative Leaders Drive Exceptional Outcomes, by G. Shawn Hunter.   I’ve been sharing some nuggets from the book, and it’s been reminding me what it takes to build a culture of innovation.

If you want to start your innovation journey, and create a culture of innovation, here are a few posts to help you on your way:

3 Key Questions to Challenge Yourself to Innovate

3 Keys for a Successful Innovation

A Superior Product is Not Built from It’s Features

Beware of Benchmarking Your Way to Mediocrity

Energized Differentiation Separates Brands from the Pack

High-Leverage Strategies for Innovation

How Great Leaders Build a Culture of Innovation and Change

Incremental Changes or Disruptive Innovation?

Innovate in Your Approach

Innovation Life Cycle

Innovation, Quantification, and Orchestration

The Innovative Team: Unleashing Creative Potential for Breakthrough Results

The Role of Process in Driving Reliable Innovation

Great Innovation Quotes to Inspire the Art of the Possible

If you need to remind yourself what innovation feels like or what’s possible, be sure to soak up some powerful words of wisdom:

Innovation Quotes

In my Innovation Quotes, I’ve also included a special section to light up what Bill Gates, Steve Jobs, and Walt Disney teach us about building a culture of innovation.

Let’s boldly go where we have not gone before.

Categories: Architecture, Programming

The Mullet Cloud Selection Pattern

In a recent thread on Hacker News one of the commenters mentioned that they use Digital Ocean for personal stuff, but use AWS for business.

This DO for personal and AWS for business split has become popular enough that we can now give it a name: the Mullet Cloud Selection Pattern - business on the front and party on the back.

Providers like DO are cheap and the lightweight composable container model has an aesthetic appeal to developers. Even though it seems like much of the VM infrastructure has to be reinvented for containers, the industry often follows the lead of developer preference.

The mullet is dead. Long live the mullet! Developers are ever restless, always eager to move onto something new. 

Categories: Architecture

Sponsored Post: Layer, The Factory, Airseed, Uber, ScaleOut Software, Couchbase, Tokutek, Booking, MongoDB, BlueStripe, AiScaler, Aerospike, LogicMonitor, AppDynamics, ManageEngine, Site24x7

Who's Hiring?
  • We’re looking for talented and driven engineers who are passionate about scalability and reliability to help us build Layer, the open communications layer for the Internet. Layer enables app developers to easily build secure, scalable messaging, voice and video features into any app. For more information and our full list of openings, please visit: https://layer.com/jobs

  • The Factory is seeking a collaborative, talented and entrepreneurial Sr. Front End Engineer. Backed by the co-founder of Skype+Rdio and led by Rdio's founding team, we're changing the way products are built and companies get launched (think incubator/accelerator without the nagging outside influence or funding/timing constraints). Our goal is to build a platform to better launch startups and opensource what we do along the way.  http://www.thefactory.com/pdfs/sr_frontend.pdf

  • Airseed -- a Google Ventures backed, developer platform that powers single sign-on authentication, rich consumer data, and analytics -- is hiring lead backend and fullstack engineers (employees #4, 5, 6). More info here: https://www.airseed.com/jobs

  • Join the team that scales Uber supply globally! Our supply engineering team is responsible for prototyping, building, and maintaining the partner-facing platform. We're looking for experienced back-end developers who care about developing highly scalable services. Apply at https://www.uber.com/jobs/4810.

  • We need awesome people @ Booking.com - We want YOU! Come design next generation interfaces, solve critical scalability problems, and hack on one of the largest Perl codebases. Apply: http://www.booking.com/jobs.en-us.html

  • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here.

  • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here.
Fun and Informative Events
  • The Biggest MongoDB Event Ever Is On. Will You Be There? Join us in New York City June 23-25 for MongoDB World! The conference lineup includes Amazon CTO Werner Vogels and Cloudera Co-Founder Mike Olson for keynote addresses.  You’ll walk away with everything you need to know to build and manage modern applications. Register before April 4 to take advantage of super early bird pricing.
  • April 3 Webinar: The BlueKai Playbook for Scaling to 10 Trillion Transactions a Month. As the industry’s largest online data exchange, BlueKai knows a thing or two about pushing the limits of scale. Find out how they are processing up to 10 trillion transactions per month from Vice President of Data Delivery, Ted Wallace. Register today.
Cool Products and Services
  • As one of the fastest growing VoIP services in the world Viber has replaced MongoDB with Couchbase Server, supporting 100,000+ operations per second in the short term and 1,000,000+ operations per second in the long term for their third generation architecture.  See the full story on the Viber switch.

  • Do Continuous MapReduce on Live Data? ScaleOut Software's hServer was built to let you hold your daily business data in-memory, update it as it changes, and concurrently run continuous MapReduce tasks on it to analyze it in real-time. We call this "stateful" analysis. To learn more check out hServer.

  • LogicMonitor is the cloud-based IT performance monitoring solution that enables companies to easily and cost-effectively monitor their entire IT infrastructure stack – storage, servers, networks, applications, virtualization, and websites – from the cloud. No firewall changes needed - start monitoring in only 15 minutes utilizing customized dashboards, trending graphs & alerting.

  • BlueStripe FactFinder Express is the ultimate tool for server monitoring and solving performance problems. Monitor URL response times and see if the problem is the application, a back-end call, a disk, or OS resources.

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Cloud deployable. Free instant trial, no sign-up required.  http://aiscaler.com/

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Categories: Architecture

2-Minute Models Part II: Decision Tables Video

Software Requirements Blog - Seilevel.com - Tue, 04/01/2014 - 12:45
This is the second part of a two-part video, describing how to use and create the Decision Table model. http://www.youtube.com/watch?v=z1Yx1wO07XQ&feature=c4-overview&list=UUkQ_qa34JnQPDjwIPx0HxxA 2-Minute Models Part II: Decision Tables Video is a post from: http://requirements.seilevel.com/blog

2-Minute Models Part II: Decision Tables Video is a post from: http://requirements.seilevel.com/blog

Categories: Requirements

Introducing the LAFABLE Process for Scaling Agile

Mike Cohn's Blog - Tue, 04/01/2014 - 09:00

Over the last year or so, scaling agile and Scrum have become very popular topics. A variety of scaling approaches have been promoted—and some of them were even tested on real teams before the marketing machinery spun up to promote them.

You've undoubtedly heard of a few of these scaling frameworks.

Today, Mountain Goat Software is proud to announce its own scaling framework: LAFABLE or the Large Agile Framework Appropriate for Big, Lumbering Enterprises.

Building on the popularity and success of Scrum, LAFABLE projects employ a variation of Scrum’s “sprints.” But to slow the project down appropriately for an organization that wants to pretend to be agile rather than actually being agile, LAFABLE refers to these as “Strolls” rather than “Sprints.”

Under the coaching of a StrollMaster (preferably a Certified StrollMaster), LAFABLE projects include three very distinct phases: Planning, Strolling, and Stabilization. Although this may sound very waterfall-like, it’s not, because we say so.

Complete information on LAFABLE can be found at www.LAFABLE.com.

Introducing the LAFABLE Process for Scaling Agile

Mike Cohn's Blog - Tue, 04/01/2014 - 09:00

Over the last year or so, scaling agile and Scrum have become very popular topics. A variety of scaling approaches have been promoted—and some of them were even tested on real teams before the marketing machinery spun up to promote them.

You've undoubtedly heard of a few of these scaling frameworks.

Today, Mountain Goat Software is proud to announce its own scaling framework: LAFABLE or the Large Agile Framework Appropriate for Big, Lumbering Enterprises.

Building on the popularity and success of Scrum, LAFABLE projects employ a variation of Scrum’s “sprints.” But to slow the project down appropriately for an organization that wants to pretend to be agile rather than actually being agile, LAFABLE refers to these as “Strolls” rather than “Sprints.”

Under the coaching of a StrollMaster (preferably a Certified StrollMaster), LAFABLE projects include three very distinct phases: Planning, Strolling, and Stabilization. Although this may sound very waterfall-like, it’s not, because we say so.

Complete information on LAFABLE can be found at www.LAFABLE.com.

C# Eye for the APL guy

Phil Trelford's Array - Tue, 04/01/2014 - 08:08

A Programming Language (APL) was invented in the swinging 60s, first appearing in 1964. C#, an OOP language, first appeared in the year 2000, to coincide with Britney’s Oops!… I did it again world tour.

Adding numbers

APL C# 2 + 2
using System;

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine(2 + 2);
    }    
}

 

Note: C# may get a REPL in Visual Studio when project Roslyn is finally released.

Min/Max

APL C# 11 ⌊ 20
75.6 ⌈ 87.3
11 28 52 14 ⌈ 20
Math.Min(11, 20);
Math.Max(75.6, 87.3);
(new[] { 11, 28, 52, 14 })
    .Select(x => Math.Min(x, 20));

Note: in C# math operations are preceded by Math and followed by dot.

Counting Costs

APL C# Costs ← 10.4 11.5 10.8 24 16.9
+/ Costs
var costs = 
    new[] {10.4,11.5,10.8,24,16.9};
costs.Sum();

Computing Costs

APL C#

Price ← 5.2 11.5 3.6 4 8.45
Qty ← 2 1 3 6 2
Costs ← Price × Qty

var prices = 
    new[] {5.2,11.5,3.6,4,8.45};
var qtys =
    new[] {2,1,3,6,2};
var costs =
    prices
        .Zip(qtys, (price, qty) =>
        new {Price=price, Qty=qty})
        .Select(pair =>
        pair.Price * pair.Qty);

Summary

C# has the clear lead on the number of characters to express problems. That said if you’re not paid by the number of characters you write you may want to Try APL.

Categories: Programming

Learning Styles and Teams

Proceed with caution!

Proceed with caution!

A team is a collection of individuals. This fact is important to remember because how each individual consumes and synthesizes information is as varied as the number of team members. However there is a finite set of learning styles to take into account. Learning styles not only impact how individuals absorb and remember information, but how they share information with others. While there are several models of learning styles, I have found the Seven Learning Styles to be useful in multicultural IT teams.  Here is my interpretation of the Seven Learning Styles:

Visual – The Diagramer absorbs information from pictures.  This is the person that builds diagrams or draws pictures to understand a concept. Adherents of mind maps tend to fall in to the visual category. Walk around your department and look at how whiteboards are being used. In a meeting the person that jumps up and starts drawing when they begin to explain a concept is generally a visual learner.

Aural – The Musician needs to hear the information they are processing. Pitch, pace and rhythm tend to important components in how this type of learner processes information. When the aural learner talks about concepts, they often combine sound references into the descriptions. For example, when attorney Johnny Cochran famously intoned “If the gloves don’t fit, you must acquit” he was evoking aural techniques that helped make the point sticky.

Verbal – The Talker needs to talk though the content they are trying to absorb. In many cases the dialog can occur internally. For example, I tend to game plan certain meeting scenarios beforehand by running sample conversations through my mind so I can anticipate how they will sound.

Physical – The Builder builds models as a means of building an understanding of a concept. Experimentation is a form of physical learning you often find in an IT department. Physical learners build something that is tangible so they can develop knowledge.  If the learners we were discussing were rocket scientists they might build model rockets rather than drawing pictures of rockets.  If we talking about programmers we would expect them to create executable code rather than models or diagrams. True prototypes (throwaway proof of concepts) are means of hands-on learning. Physical learners in non-physical situations will use tactile words to describe concepts. I recently talked to a database modeler that described the model symmetry of the model he was working on.

Logical – The lawyer builds knowledge by assembling facts and assertions into logical arguments that can be evaluated. The process that the Lawyer follows tends to build very solid bases of knowledge that are hard to challenge and disrupt.  Logical learners because they tend to move from point to point it is more difficult for them to make large jumps that do not follow from point to point. To paraphrase Socrates: All programmers are human, Joe is a programmer, therefore Joe is human.

Social – The Grouper prefers learning in group settings. The critical component for the social learner is other people.  The interaction with others is an important part of processing. Interaction in groups includes verbal and non-verbal communication and emotional support. Do you remember the person when you were at University that always organized the group study sessions? They probably fell into this category.

Solitary – The Introvert learns best by themselves.  This is the type of person that takes the book home over the weekend and just figures it out.

Learning styles are not mutually exclusive.  Each person usually has a predominate style and one or more secondary styles. I tend to the visual, but often augment pictures with physical experiments (whether writing code or brewing beer). The individuals that make up a team will have a mixture of learning styles. Each person’s learning style influences not only how they acquire knowledge, how they store knowledge and also influence how they can retrieve that data.  For example, music or sounds are a tool for aural learners (the musician) to gather information and then retrieve it.  Many of us use have used mnemonics to memorize facts. When I was young I learned to play a piano.  When I was learning to read music, my teach taught me the mnemonic, “every good boy does fine.” These are the notes on the treble cleft. Teams need to work together to accommodate and validate different learning styles. When team members are not aware of how others on the team learn they can often talk past one anther, which could reduce knowledge-transfer effectiveness.


Categories: Process Management

Slicing Work Into Small Pieces

Herding Cats - Glen Alleman - Mon, 03/31/2014 - 19:32

One of the suggestions in #NoEstimates is the slicing of work - either Stories or any word needed to indicate an agile projects chunking of the work - into small pieces. This of course doesn't actually address the issue of producing and Estimate at Completion for the project. An estimate needed by those funding or authorizing the spending of funds to know how much and when.

But slicing is a process of reducing the exposure to uncertainty to a manageable size.  It's the next level down's answer to what's the value at risk? Make in small and reduce the value at risk of not showing up on time and on budget. Slicing answers the question, that has been around for some time.

How long are you willing to wait till you find out you are late (or over budget, or it doesn't work as planned)?

The answer to this  - how long - question varies according to the domain, value at risk, and other factors usually associated with risk tolerance. But it is a question that must be answered periodically (month;y for us) Recently this notion of slicing has been put forth as part of the solution to the estimating problem, which of course it's not. Since the size of the work chunks only reducing the uncertainty of the variance. Both the aleatory (irreducible uncertainty) and epistemic (reducible uncertainty) will be less when the exposure to the uncertainty is smaller. Beneficial to the project for sure. But the total all in cost and schedule are related to the slicing size only by the cumulative variance of the parts.

It many be interesting to know, that slicing is part of ANSI-748 Earned Value Management assessment by the Defense Contract Management Agency (DCMA). DCMA is the DOD agency that validates the Earned Value Management System's 32 Guidelines. DCMA performs a 14-point assessment of the Integrated Master Schedule (integrated because it is connected to the cost based) and the Performance Measurement Baseline (PMB) (the time phased planned budget for all the work).

DCMA Check 8 looks for high duration activities. These are know to cause issues with the exposure to programmatic risk for the program. The 44 day number represents 2 working months. The work then passes through one accounting period (monthly submission of the Integrated Program Management Report). At the end of each accounting period an assessment of Percent Complete is used to calculate the Budgeted Cost of Work Performed (BCWP) - the earned value for the tasks, works packages, and control accounts (funding buckets) for the program. 44 days may sound long for an agile software development project, but 44 days is short on a multi-year, many millions and likley billions of dollars of defesnse work.

Screen Shot 2014-03-30 at 4.18.49 PM

Since the agile comminity is  fond of saying there is nothing new here, while suggesting their ideas are new and unique, the above clip is from the DCMA guide, long used in our defense program management paradigm.

So it is worth repeating the principle of asking how long you are willing to wait before you find out something. The rule of thumb is to sample the status of the thing you are controlling are a rate twice your needed to control or determine a value. This is called the Nyquist rate from single processing. Signal processing is where I grew up writing software for Fast Fourier Transforms, Finite Impulse Response Filter, Kalman Filters for particle physics data streaming off the particle accelerator. When I didn't have an orginal idea to finish my PhD studies, I switched to writing the same software for radar signals intelligence, and electroninc warfare systems. Same principles work for any control system including a management control system.

Just as an aside in the control systems paradigm, there is discussion about monitoring and decision making from the information gathered from the monitoring. This is an Open Loop control systems. Without a planned value to seek, the SET POINT if you're using the room thermostat analogy, monitoring the value provide no value, since you don't know what you are seeking the system to do. Just monitoring is Open Loop, you've got numbers from the system the room temperature or the number of stories produced, but no target to control against.

To have a closed loop system, you need a SET POINT, a steering target, a goal, a desired outcome. Then the monitoring - sampling - can produce a variance, a difference between goal and actual - by which you cn take action. Raise or lower the temperature, speed up or sloe down the car, speed up or slow down the production of software outputs. Yes you can go too fast, the down stream user can't take the results and by the time they can, the requirements may have changed. This is the Closed Loop Control

Open Closed Loop

Categories: Project Management

Updates to Google BigQuery following Cloud Platform Live

Google Code Blog - Mon, 03/31/2014 - 17:00
Author PhotoBy Felipe Hoffa, Cloud Platform team

Cross-posted from the Google Cloud Platform Blog
Editor's note: This post is a follow-up to the announcements we made on March 25th at Google Cloud Platform Live.

Last Tuesday we announced an exciting set of changes to Google BigQuery making your experience easier, faster and more powerful. In addition to new features and improvements like table wildcard functions, views, and parallel exports, BigQuery now features increased streaming capacity, lower pricing, and more.


1000x increase in streaming capacity

Last September we announced the ability to stream data into BigQuery for instant analysis, with an ingestion limit of 100 rows per second. While developers have enjoyed and exploited this capability, they've asked for more capacity. You now can stream up to 100,000 rows per second, per table into BigQuery - 1,000 times more than before.

For a great demonstration of the power of streaming data into BigQuery, check out the live demo from the keynote at Cloud Platform Live.

Users often partition their big tables into smaller units for data lifecycle and optimization purposes. For example, instead of having yearly tables, they could be split into monthly or even daily sets. BigQuery now offers table wildcard functions to help easily query tables that match common parameters.

The downside of partitioning tables is writing queries that need to access multiple tables. This would be easier if there was a way to tell BigQuery "process all the tables between March 3rd and March 25th" or "read every table which names start with an 'a'". You can do this with this release.

TABLE_DATE_RANGE() queries all tables that overlap with a time range (based on the table names), while TABLE_QUERY() accepts regular expressions to select the tables to analyze.

For more information, see the documentation and syntax for table wildcard functions.

Improved SQL support and table views

BigQuery has adopted SQL as its query language because it's one of the most well known, simple and powerful ways to analyze data. Nevertheless BigQuery used to impose some restrictions on traditional SQL-92, like having to write multiple sub-queries instead of simpler multi-joins. Not anymore, now BigQuery supports multi-join and CROSS JOIN, and improves its SQL capabilities with more flexible alias support, fewer ORDER BY restrictions, more window functions, smarter PARTITION BY, and more.

A notable new feature is the ability to save queries as views, and use them as building blocks for more complex queries. To define a view, you can use the browser tool to save a query, the API, or the newest version of the BigQuery command-line tool (by downloading the Google Cloud SDK).

User-defined metadata

Now you can annotate each dataset, table, and field with descriptions that are displayed within BigQuery. This way people you share your datasets with will have an easier time identifying them.

JSON parsing functions

BigQuery is optimized for structured data: before loading data into BigQuery, you should first define a table with the right columns. This is not always easy, as JSON schemas might be flexible and in constant flux. BigQuery now lets you store JSON encoded objects into string fields, and you can use the JSON_EXTRACT and JSON_EXTRACT_SCALAR functions to easily parse them later using JSONPath-like expressions.

For example:
SELECT json_extract_scalar(
"{'book': {
'category':'fiction',
'title':'Harry Potter'}}",
"$.book.category");


Fast parallel exports

BigQuery is a great place to store all your data and have it ready for instant analysis using SQL queries. But sometimes SQL is not enough, and you might want to analyze your data with external tools. That's why we developed the new fast parallel exports: With this feature, you can define how many workers will be consuming the data, and BigQuery exports the data to multiple files optimized for the available number of workers.

Check the exporting data documentation, or stay tuned for the upcoming Hadoop connector to BigQuery documentation.

Massive price reductions

At Cloud Platform live, we announced a massive price reduction: Storage costs are going down 68%, from 8 cents per gigabyte per month to only 2.6, while querying costs are going down 85%, from 3.5 cents per gigabyte to only 0.5. Previously announced streaming costs are now reduced by 90%. And finally, we announced the ability to purchase reserved processing capacity, for even cheaper prices and the ability to precisely predict costs. And you always have the option to burst using on-demand capacity.

I want to take this space to celebrate the latest open source community contributions to the BigQuery ecosystem. R has its own connector to BigQuery (and a tutorial), as Python pandas too (check out the video we made with Pearson). Ruby developers are now able to use BigQuery with an ActiveRecord connector, and send all their logs with fluentd. Thanks all, and keep surprising us!

Felipe Hoffa is part of the Cloud Platform Team. He'd love to see the world's data accessible for everyone in BigQuery.

Posted by Louis Gray, Googler
Categories: Programming

How WhatsApp Grew to Nearly 500 Million Users, 11,000 cores, and 70 Million Messages a Second

When we last visited WhatsApp they’d just been acquired by Facebook for $19 billion. We learned about their early architecture, which centered around a maniacal focus on optimizing Erlang into handling 2 million connections a server, working on All The Phones, and making users happy through simplicity.

Two years later traffic has grown 10x. How did WhatsApp make that jump to the next level of scalability?

Rick Reed tells us in a talk he gave at the Erlang Factory: That's 'Billion' with a 'B': Scaling to the next level at WhatsApp (slides), which revealed some eye popping WhatsApp stats:

What has hundreds of nodes, thousands of cores, hundreds of terabytes of RAM, and hopes to serve the billions of smartphones that will soon be a reality around the globe? The Erlang/FreeBSD-based server infrastructure at WhatsApp. We've faced many challenges in meeting the ever-growing demand for our messaging services, but as we continue to push the envelope on size (>8000 cores) and speed (>70M Erlang messages per second) of our serving system.

What are some of the most notable changes from two years ago?

  • Obviously much bigger in every dimension, except the number of engineers. More boxes, more datacenters, more memory, more users, and more scale problems. Handling this level of growth with so few engineers is what Rick is most proud of: 40 million users per engineer. This is part of the win of the cloud. Their engineers work on their software. The network, hardware, and datacenter is handled by someone else.

  • They’ve gone away from trying to support as many connections per box as possible because of the need to have enough head room to handle the overall increased load on each box. Though their general strategy of keeping down management overhead by getting really big boxes and running efficiently on SMP machines, remains the same.

  • Transience is nice. With multimedia, pictures, text, voice, video all being part of their architecture now, not having to store all these assets for the long term simplifies the system greatly. The architecture can revolve around throughput, caching, and partitioning.

  • Erlang is its own world. Listening to the talk it became clear how much of everything you do is in the world view of Erlang, which can be quite disorienting. Though in the end it’s a distributed system and all the issues are the same as in any other distributed system.

  • Mnesia, the Erlang database, seemed to be a big source of problem at their scale. It made me wonder if some other database might be more appropriate and if the need to stay within the Erlang family of solutions can be a bit blinding?

  • Lots of problems related to scale as you might imagine. Problems with flapping connections, queues getting so long they delay high priority operations, flapping of timers, code that worked just fine at one traffic level breaking badly at higher traffic levels, high priority messages not getting serviced under high load, operations blocking other operations in unexpected ways, failures causing resources issues, and so on. These things just happen and have to be worked through no matter what system you are using.

  • I remain stunned and amazed at Rick’s ability to track down and fix problems. Quite impressive.

Rick always gives a good talk. He’s very generous with specific details that obviously derive directly from issues experienced in production. Here’s my gloss on his talk…

Stats
Categories: Architecture

What to Do When You Don’t Feel Like Writing and You Have Nothing to Say

Making the Complex Simple - John Sonmez - Mon, 03/31/2014 - 16:00

I spend a lot of time doing two things: blogging and telling other developers the benefits of doing things like starting their own blog. (Occasionally I squeeze in a little bit of time to code as well. And my wife says I spend too much time answering emails and checking my phone—she wanted me to […]

The post What to Do When You Don’t Feel Like Writing and You Have Nothing to Say appeared first on Simple Programmer.

Categories: Programming

Personal Branding

NOOP.NL - Jurgen Appelo - Mon, 03/31/2014 - 10:27
Personal Brand

I call myself a Creative Networker. It is the most accurate and succinct description I could think of that wraps all the projects I am involved in. My business cards and personal website explain that I’m a writer, speaker, trainer, entrepreneur, illustrator, manager, blogger, reader, dreamer, leader, and freethinker. (I seem to have forgotten bragger in that list.) If I had to summarize my personal brand with three words I would choose creative, smart, and funny. But I would easily agree that brands are better defined by their observers, not by their owners. My readers might prefer to describe me as weird, blunt, and smug. It is all in the eye of the beholder.

The post Personal Branding appeared first on NOOP.NL.

Categories: Project Management

SPaMCAST 283 – User Stories Pure and Simple

Software Process and Measurement Cast - Sun, 03/30/2014 - 22:00

Listen to the Software Process and Measurement Cast 283. The SPaMCAST 283 features our essay on user stories.  A user story is a brief, simple requirement statement from the user perspective. User stories are narratives describing who is interacting with the application; how they are interacting with the application and the benefit they derive from that interaction.

If you would like to read the original blog entries that formed the basis of this essay they can be found at:

User Stories
Cards
Epics, Themes and Issues
Problems
Same as Use Cases and Traditional Requirements?

Get in touch with us anytime or leave a comment here on the blog. Help support the SPaMCAST by reviewing and rating it on iTunes. It helps people find the cast. Like us on Facebook while you’re at it.

Next week we will feature our interview with Evan Leybourn author of Directing the Agile Organization. We discussed agile business management. Agile is not just for IT anymore!

Upcoming Events

QAIQuest 2014

I will be facilitating a ½ Day tutorial titled Make Integration and Acceptance Testing Truly Agile. The tutorial will wrestle with the flow of testing in Agile projects and will include lots of practical advice and exercises. Remember that Agile testing is not waterfall done quickly. I will also be around for the conference and look forward to meeting and talking with SPaMCAST readers and listeners.  More confernce information   ALSO I HAVE A DISCOUNT CODE…. Email me at spamcastinfo@gmail.com or call 440.668.5717 for the code.

StarEast

I will be speaking at the StarEast Conference May 4th – 9th in Orlando, Florida.  I will be presenting a talk titled, The Impact of Cognitive Biases on Test and Project Teams. Follow the link for more information on StarEast. ALSO I HAVE A DISCOUNT CODE…. Email me at spamcastinfo@gmail.com or call 440.668.5717 for the code.

I look forward to seeing all SPaMCAST readers and listeners at all of these great events!

The Software Process and Measurement Cast has a sponsor.

As many you know I do at least one webinar for the IT Metrics and Productivity Institute (ITMPI) every year. The ITMPI provides a great service to the IT profession. ITMPI's mission is to pull together the expertise and educational efforts of the world's leading IT thought leaders and to create a single online destination where IT practitioners and executives can meet all of their educational and professional development needs. The ITMPI offers a premium membership that gives members unlimited free access to 400 PDU accredited webinar recordings, and waives the PDU processing fees on all live and recorded webinars. The Software Process and Measurement Cast some support if you sign up here. All the revenue our sponsorship generates goes for bandwidth, hosting and new cool equipment to create more and better content for you. Support the SPaMCAST and learn from the ITMPI.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: "This book will prove that software projects should not be a tedious process, neither for you or your team." Support SPaMCAST by buying the book here.

Available in English and Chinese.

Categories: Process Management

SPaMCAST 283 – User Stories Pure and Simple

www.spamcast.net

http://www.spamcast.net

Listen to the Software Process and Measurement Cast 283.  The SPaMCAST 283 features our essay on user stories.  A user story is a brief, simple requirement statement from the user’s perspective. User stories are narratives describing who is interacting with the application, how they are interacting with the application and the benefit they derive from that interaction.

If you would  like to read the original blog entries that formed the basis of this essay they can be found at:
User Stories
Cards
Epics, Themes and Issues
Problems
Same as Use Cases and Traditional Requirements?

Get in touch with us anytime or leave a comment here on the blog. Help support the SPaMCAST by reviewing and rating it on iTunes. It helps people find the cast. Like us on Facebook while you’re at it.

Next week we will feature our interview with Evan Leybourn author of Directing the Agile Organization. We discussed Agile business management. Agile is not just for IT anymore!

Upcoming Events

QAIQuest 2014

I will be facilitating a ½ Day tutorial titled Make Integration and Acceptance Testing Truly Agile. The tutorial will wrestle with the flow of testing in Agile projects and will include lots of practical advice and exercises. Remember that Agile testing is not waterfall done quickly. I will also be around for the conference and look forward to meeting and talking with SPaMCAST readers and listeners.  More confernce information   ALSO I HAVE A DISCOUNT CODE…. Email me at spamcastinfo@gmail.com or call 440.668.5717 for the code.

StarEast

I will be speaking at the StarEast Conference May 4th – 9th in Orlando, Florida.  I will be presenting a talk titled, The Impact of Cognitive Biases on Test and Project Teams. Follow the link for more information on StarEast. ALSO I HAVE A DISCOUNT CODE…. Email me at spamcastinfo@gmail.com or call 440.668.5717 for the code.

I look forward to seeing all SPaMCAST readers and listeners at all of these great events!

The Software Process and Measurement Cast has a sponsor.

As many you know I do at least one webinar for the IT Metrics and Productivity Institute (ITMPI) every year. The ITMPI provides a great service to the IT profession. ITMPI’s mission is to pull together the expertise and educational efforts of the world’s leading IT thought leaders and to create a single online destination where IT practitioners and executives can meet all of their educational and professional development needs. The ITMPI offers a premium membership that gives members unlimited free access to 400 PDU accredited webinar recordings, and waives the PDU processing fees on all live and recorded webinars. The Software Process and Measurement Cast some support if you sign up here. All the revenue our sponsorship generates goes for bandwidth, hosting and new cool equipment to create more and better content for you. Support the SPaMCAST and learn from the ITMPI.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.


Categories: Process Management