Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

Equality for All Agile Team Members?

Mike Cohn's Blog - Tue, 08/05/2014 - 15:00

"Liberté, égalité, fraternité" is the national motto of France, and originated during the French Revolution. And while freedom, equality, and brotherhood are great ideals for a country, I'm not sure about one of them for agile teams: Equality.

I'm frequently asked if agile means that everyone is equal on an agile team. The feeling is that self-organization means everyone should be equal--including that junior intern that started yesterday.

Fortunately, self-organization does not require everyone to be equal. In fact, self-organization requires the opposite: there must be differences between the agents who are self-organizing.

In her Ph.D. dissertation on self-organization, Glenda Eoyang wrote that self-organization requires three things: a container, differences and transforming exchanges. This is known as the CDE model, for “Containers, Differences, and Exchanges.

Differences are necessary for self-organization to occur because otherwise, any organization that emerged would be random. There would be no advantage to one agent (person) doing something rather than someone else if all agents were identical. So, differences are necessary. And, fortunately, when dealing with humans, differences abound.

So, no, becoming good at agile does not require everyone on a team to be treated equally. Each team member should get the respect he or she deserves. When an experienced team member with a track record of giving solid advice says something will be difficult, others should consider that opinion.

When that junior intern we hired yesterday says the same, team members should listen politely but give only minor credence to the opinion.

Another meaning of equality could mean that all team members do the same work – meaning everyone in agile becomes a generalist and we have no room for specialists. That’s an equality myth that I’ll address next week.

In the comments section below, let me know what you think, and how your team works. Is everyone equal on your team?

Azure: Virtual Machine, Machine Learning, IoT Event Ingestion, Mobile, SQL, Redis, SDK Improvements

ScottGu's Blog - Scott Guthrie - Tue, 08/05/2014 - 07:28

This past month we’ve released a number of great enhancements to Microsoft Azure.  These include:

  • Virtual Machines: Preview Portal Support as well as SharePoint Farm Creation
  • Machine Learning: Public preview of the new Azure Machine Learning service
  • Event Hub: Public preview of new Azure Event Ingestion Service
  • Mobile Services: General Availability of .NET support, SignalR support
  • Notification Hubs: Price Reductions and New Features
  • SQL Database: New Geo-Restore, Geo-Replication and Auditing support
  • Redis Cache: Larger Cache Sizes
  • Storage: Support for Zone Redundant Storage
  • SDK: Tons of great VS and SDK improvements

All of these improvements are now available to use immediately (note that some features are still in preview).  Below are more details about them: Virtual Machines: Support in the new Azure Preview portal

We previewed the new Azure Preview Portal at the //Build conference earlier this year.  It brings together all of your Azure resources in a single management portal, and makes it easy to build cloud applications on the Azure platform using our new Azure Resource Manager (which enables you to manage multiple Azure resources as a single application).  The initial preview of the portal supported Web Sites, SQL Databases, Storage, and Visual Studio Online resources.

This past month we’ve extended the preview portal to also now support Virtual Machines.  You can create standalone VMs using the portal, or group multiple VMs (and PaaS services) together into a Resource Group and manage them as a single logical entity. You can use the preview portal to get deep insights into billing and monitoring of these resources, and customize the portal to view the data however you want.  If you are an existing Azure customer you can start using the new portal today: http://portal.azure.com.

Below is a screen-shot of the new portal in action.  The service dashboard showing service/region health can be seen in the top-left of the portal, along with billing data about my subscriptions – both make it really easy for you to see the health and usage of your services in Azure.  In the screen-shot below I have a single VM running named “scottguvstest” – and clicking the tile for it displays a “blade” of additional details about it to the right – including integrated performance monitoring usage data:

image

The initial “blade” for a VM provides a summary view of common metrics about it.  You can click any of the titles to get even more detailed information as well. 

For example, below I’ve clicked the CPU monitoring title in my VM, which brought up a Metric blade with even more details about CPU utilization over the last few days.  I’ve then clicked the “Add Alert” command within it to setup an automatic alert that will trigger (and send an email to me) any time the CPU of the VM goes above 95%:

image

In the screen-shot below, I’ve clicked the “Usage” tile within the VM blade, which displays details about the different VM sizes available – and what each VM size provides in terms of CPU, memory, disk IOPS and other capabilities.  Changing the size of the VM being used is as simple as clicking another of the pricing tiles within the portal – no redeployment of the VM required:

image

SharePoint Farm support via the Azure Gallery

Built-into the Azure Preview Portal is a new “Azure Gallery” that provides an easy way to deploy a wide variety of VM images and online services.  VM images in the Azure Gallery include Windows Server, SQL Server, SharePoint Server, Ubuntu, Oracle, Baracuda images. 

Last month, we also enabled a new “SharePoint Server Farm” gallery item.  It enables you to easily configure and deploy a highly-available SharePoint Server Farm consisting of multiple VM images (databases, web servers, domain controllers, etc) in only minutes.  It provides the easiest way to create and configure SharePoint farms anywhere:

image

Over the next few months you’ll see even more items show up in the gallery – enabling a variety of additional new scenarios.  Try out the ones in the gallery today by visiting the new Azure portal: http://portal.azure.com/

Machine Learning: Preview of new Machine Learning Service for Azure

Last month we delivered the public preview of our new Microsoft Azure Machine Learning service, a game changing service that enables your applications and systems to significantly improve your organization’s understanding across vast amounts of data. Azure Machine Learning (Azure ML) is a fully managed cloud service with no software to install, no hardware to manage, and no OS versions or development environments to grapple with. Armed with nothing but a browser, data scientists can log into Azure and start developing Machine Learning models from any location, and from any device.

ML Studio, an integrated development environment for Machine Learning, lets you set up experiments as simple data flow graphs, with an easy to use drag, drop and connect paradigm. Data scientists can use it to avoid programming a large number of common tasks, allowing them to focus on experiment design and iteration. A collection of best of breed algorithms developed by Microsoft Research comes built-in, as is support for custom R code – and over 350 open source R packages can be used securely within Azure ML today.

image

Azure ML also makes it simple to create production deployments at scale in the cloud. Pre-trained Machine Learning models can be incorporated into a scoring workflow and, with a few clicks, a new cloud-hosted REST API can be created.

Azure ML makes the incredible potential of Machine Learning accessible both to startups and large enterprises. Startups are now able to immediately apply machine learning to their applications. Larger enterprises are able to unleash the latent value in their big data to generate significantly more revenue and efficiencies. Above all, the speed of iteration and experimentation that is now possible will allow for rapid innovation and pave the way for intelligence in cloud-connected devices all around us. Getting Started

Getting started with the Azure Machine Learning Service is easy.  Within the current Azure Portal simply choose New->Data Services->Machine Learning to create your first ML service today:

image

Subscribe to the Machine Learning Team Blog to learn more about the Azure Machine Learning service.  And visit our Azure Machine Learning documentation center to watch videos and explore tutorials on how to get started immediately.

Event Hub: Preview of new Azure Event Ingestion Service

Today’s connected world is defined by big data.  Big data may originate from connected cars and thermostats that produce telemetry data every few minutes, application performance counters that generate events every second or mobile apps that capture telemetry for every user’s individual action. The rapid proliferation of connected devices raises challenges due to the variety of platforms and protocols involved.  Connecting these disparate data sources while handling the scale of the aggregate stream is a significant challenge. 

I’m happy to announce the public preview of a significant new Azure service: Event Hub. Event Hub is a highly scalable pub-sub ingestor capable of elastic scale to handle millions of events per second from millions of connected devices so that you can process and analyze the massive amounts of data produced by your connected devices and applications. With this new service, we now provide an easy way for you to provision capacity for ingesting events from a variety of sources, and over a variety of protocols in a secure manner. Event Hub supports a variety of partitioning modes to enable parallelism and scale in your downstream processing tier while preserving the order of events on a per device basis. Creating an Event Hub

You can easily create a new instance of Event Hub from the Azure Management Portal by clicking New->App Services->Service Bus->Event Hub. During the Preview, Event Hub service is available in a limited number of regions (East US 2, West Europe, Southeast Asia) and requires that you first create a new Service Bus Namespace:

image Learn More

Try out the new Event Hub service and give us your feedback! For more information, visit the links below:

Mobile Services: General Availability of .NET Support, SignalR and Offline Sync

A few months ago I announced a preview of Mobile Services with .NET backend support. Today I am excited to announce the general availability of the Mobile Services .NET offering, which makes it an incredibly attractive choice for developers building mobile facing backend APIs using .NET.  Using Mobile Services you can now:

  • Quickly add a fully featured backend to your iOS, Android, Windows, Windows Phone, HTML or cross-platform Xamarin, Sencha, or PhoneGap app, leveraging ASP.NET Web API, Mobile Services, and corresponding Mobile Services client SDKs.
  • Publish any existing ASP.NET Web API to Azure and have Mobile Services monitor and manage your Web API controllers for you.
  • Take advantage of built-in mobile capabilities like push notifications, real-time notifications with SignalR, enterprise sign-on with Azure Active Directory, social auth, offline data sync for occasionally connected scenariosYou can also take full advantage of Web API features like OData controllers, and 3rd party Web API-based frameworks like Breeze.
  • Have your mobile app’s users login via Azure Active Directory and securely access enterprise assets such as SharePoint and Office 365. In addition, we've also enabled seamless connectivity to on-premises assets, so you can reach databases and web services that are not exposed to the Internet and behind your company’s firewall.
  • Build, test, and debug your Mobile Services .NET backend using Visual Studio running locally on your machine or remotely in Azure.

You can learn more about Mobile Services .NET from this blog post, and the Mobile Services documentation center. Real-time Push with Mobile Services and SignalR

We recently released an update to our Mobile Services .NET backend support which enables you to use ASP.NET SignalR for real-time, bi-directional communications with your mobile applications. SignalR will use WebSockets under the covers when it's available, and fallback to other “techniques” (i.e. HTTP hacks) when it isn't. Regardless of the mode, your application code stays the same.

The SignalR integration with Azure Mobile Services includes:

  • Turnkey Web API Integration: Send messages to your connected SignalR applications from any Web API controller or scheduled job – we automatically give you access to SignalR Hubs from the ApiServices context.
  • Unified Authentication: Protect your SignalR Hubs the same way you protect any of your Mobile Service Web API controllers using a simple AuthorizeLevel attribute.
  • Automatic Scale-out: When scaling out your Azure Mobile Service using multiple front-ends, we automatically scale out SignalR using Azure Service Bus as the backplane for sync’ing between the front-ends. You don’t need to do anything to scale your SignalR Hubs.

Learn more about the SignalR capability in Mobile Services from Henrik’s blog. Mobile Services Offline Sync support for Xamarin and native iOS apps

I've blogged earlier about the new Offline Sync feature in Mobile Services, which provides a lightweight, cross-platform way for applications to work with data even when they are offline / disconnected from the network. At that time we released Offline Sync support for Windows Phone and Windows Store apps.

Today we are also introducing a preview of Mobile Services Offline Sync for native iOS apps, as well as Xamarin.iOS, and Xamarin.Android. Mobile Services Accelerators

I’m pleased to also introduce our new Mobile Services Accelerators, which are feature complete sample apps that demonstrate how to leverage the new enterprise features of the Mobile Services platform in an end-to-end scenario. We will have two accelerator apps for you today, available as a source code, as well as published in the app store.

These apps leverage the Mobile Services .NET backend support to handle authenticating employees with Azure Active Directory, store data securely, working with data offline, as well as get reminders via push notifications. We hope you will find these apps useful for your teams as a reference material. Stay tuned, as more accelerators are coming! Notification Hubs: Price reductions and new features

The Azure Notification Hubs service enables large scale cross platform push notifications from any server backend running on-premise or in the cloud.  It supports a variety of mobile devices including iOS, Android, Windows, Kindle Fire, and Nokia X. I am excited to announce several great updates to Azure Notification Hubs today:

  • Price reduction. We are reducing the Notification Hubs price by up to 40x to accommodate a wider range of customer scenarios. With the new price (effective September 1st), customers can send 1 million mobile push notifications per month for free, and pay $1 per additional million pushes using our new Basic tier. Visit the Notification Hubs pricing page for more details.
  • Scheduled Push. You can now use Notification Hubs to schedule individual and broadcast push notifications at certain times of the day. For example, you can use this feature to schedule announcements to be delivered in the morning to your customers.  We include support to enable this no matter which time zone your customers are in.
  • Bulk Registration management. You can now send bulk jobs to create, update or export millions of mobile device registrations at a time with a single API call. This is useful if you are moving from an old push notification system to Notification Hubs, or to import user segments from a 3rd party analytics system.

You can learn more about Azure Notification Hubs at the developer center.

SQL Databases: New Geo-Restore, Geo-Replication and Auditing support

In April 2014, we first previewed our new SQL Database service tiers: Basic, Standard, and Premium. Today, I’m excited to announce the addition of more features to the preview:

  • Geo-restore: Designed for emergency data recovery when you need it most, geo-restore allows you to recover a database to any Azure region. Geo-restore uses geo-redundant Azure blob storage for automatic database backups and is available for Basic, Standard, and Premium databases in the Windows Azure Management Portal and REST APIs.
  • Geo-replication: You can now configure your SQL Databases to use our built-in geo-replication support that enables you to setup an asynchronously replicated secondary SQL Database that can be failed over to in the event of disaster.  Geo-replication is available for Standard and Premium databases, and can be configured via the Windows Azure Management portal and REST APIs. You can get more information about Azure SQL Database Business Continuity and geo-replication here and here.
  • Auditing: Our new auditing capability tracks and logs events that occur in your database and provides dashboard views and reports that enables you to get insights into these events. You can use auditing to streamline compliance-related activities, gain knowledge about what is happening in your database, and to identify trends, discrepancies and anomalies. Audit events are also written to an audit log which is stored in a user-designated Azure storage account.  Auditing is now available for all Basic, Standard, and Premium databases.

You can learn even more about these new features here.

Redis Cache: Large Cache Sizes, Six New Regions, Redis MaxMemory Policy Support

This past May, we launched the public preview of the new Azure Redis Cache service. This cache service gives you the ability to use a secure, dedicated Redis cache, managed as a service by Microsoft. Using the new Cache service, you get to leverage the rich feature set and ecosystem provided by Redis, and reliable hosting and monitoring from Microsoft.

Last month we updated the service with the following features:

  • Support for larger cache sizes. We now support the following sizes: 250 MB, 1 GB, 2.5 GB, 6 GB, 13 GB and 26 GB. 
  • Support for six new Azure Regions. The full list of supported regions can be found in the Azure Regions page.
  • Support for configuring Redis MaxMemory policy

For more information on the Azure Redis Cache, check out this blog post: Lap around Azure Redis Cache. Storage: Support for Zone Redundant Storage

We are happy to introduce a new Azure Storage account offering: Zone Redundant Storage (ZRS).

ZRS replicates your data across 2 to 3 facilities either within a single Azure region or across two Azure regions. If your storage account has ZRS enabled, then your data is durable even in the case where one of the datacenter facilities hosting your data suffers a catastrophic issue. ZRS is also more cost efficient than the existing Global Redundant Storage (GRS) offering we have today.

You can create a ZRS storage account by simply choosing the ZRS option under the replication dropdown in the Azure Management Portal.

image

You can find more information on pricing for ZRS at http://azure.microsoft.com/en-us/pricing/details/storage/. Azure SDK: WebSites, Mobile, Virtual Machines, Storage and Cloud Service Enhancements

Earlier today we released the Update 3 release of Visual Studio 2013 as well as the new Azure SDK 2.4 release.  These updates contain a ton of great new features that make it even easier to build solutions in the cloud using Azure.  Today’s updates include:

Visual Studio Update 3

  • Websites: Publish WebJobs from Console or Web projects.
  • Mobile Services: Create a Dev/Test environment in the cloud when creating Mobile Services projects. Use the Push Notification Wizard with .NET Mobile Services.
  • Notification Hubs: View and manage device registrations.

Azure SDK 2.4

  • Virtual Machines: Remote debug 32-bit Virtual Machines. Configure Virtual Machines, including installation & configuration of dynamic extensions (e.g. anti-malware, Puppet, Chef and custom script). Create Virtual Machine snapshots of the disk state.
  • Storage: View Storage activity logs for diagnostics. Provision Read-Access Geo-redundant Storage from Visual Studio.
  • Cloud Services: Emulator Express is the default option for new projects (Full Emulator is deprecated). Configure new networking capabilities in the service model.

You can learn all about the updates from the Azure team’s SDK announcement blog post.

Summary

This most recent release of Azure includes a bunch of great features that enable you to build even better cloud solutions.  If you don’t already have a Azure account, you can sign-up for a free trial and start using all of the above features today.  Then visit the Azure Developer Center to learn more about how to build apps with it.

Hope this helps,

Scott

P.S. In addition to blogging, I am also now using Twitter for quick updates and to share links. Follow me at: twitter.com/scottgu

Categories: Architecture, Programming

Seven Deadly Sins of Metrics Programs: Envy

Definition of done

Don’t trip the runner next to you just to win.

The results of software measurement can be held up as badge of honor. It is not uncommon for a CIO, department manager, project manager or even technical lead to hold up the performance on their projects in front others, engendering envy from other projects. Envy is a feeling of discontent and resentment aroused by and in conjunction with desire for the possessions or qualities of another. Measurement is a spotlight that can focus other’s envy if the situation is right. That can occur when  bonuses are tied to measurement and when the assignment and staffing of projects is driven by unknown factors. There are two major types of metrics-based envy: one must be addressed at the personnel level and the second must be addressed organizationally.

Envy can be caused when the metrics of projects managed by others in your peer group (real or perceived) are held up as examples to be emulated.  The active component of envy at this level is triggered by a social comparison that threatens a person’s self image, and can be exacerbated when the attributes that impact performance are outside of the team’s control. The type or complexity of the work coming to a team is generally negotiable. Teams that get the really tough problem will generally not have the highest productivity even though they may solved an intractable business problem. Envy generated by this type of problem translates into a variety of harmful behaviors. In benign cases, we might just pass it off as office politics (which everybody loves, not), or in a worst case scenario could generate a self destructive spiral of negative behavior which is not helpful to anyone.  Typical envy-driven behaviors to watch for include the loss of will, poor communication, withdrawal and hiding.  While the amateur psychologist in me would be happy to pontificate on the personal side of envy, I am self aware enough to know that I shouldn’t.  If you have fallen into the trap of envy, get professional help. If you are a manager of a person that is falling into this hole, get them help or get them out of the organization.

The other category of triggers are organizational.  These are the triggers that as managers, we have more control over and have the obligation to address.  As leaders we have a chance to mold the organizational culture to be supportive of efficiency and effectiveness.  Cultures and environments can facilitate and foster both good and bad behaviors.  Cultures that support an atmosphere of individual competition above collaboration can create an atmosphere where envy will flourish. This will act as a feedback loop to further deepen silos and the possibility of envy. For example, Sid may feel that Joe always gets the best recruits and he is powerless to change the equation (for whatever reason), therefore he can’t compete.  Envy may cause him to focus on stealing Joe’s recruits rather than coaching his own. Thisculture can disrupt communication and collaboration and create silos. In this type of environment positive behaviors, such as displaying measurement data, can act as feedback loop to deepen the competitive culture rather than generating collaboration and communication.  Typical behaviors generated by envy triggered by organizational issues include those noted earlier and outright sabotage of projects and careers (tripping the runner next to you so you can win), and just as bad, the pursuit of individual goals at the expense of the overall business goals.

Measurement programs can take the lead in developing a culture where teams can perform, be recognized for that performance and then share the lessons that delivered that performance when it is truly special. An important way to understand what type of performance really should be held up and emulated is based on the work of W. Edward Deming. In Deming’s seminal work Out of the Crisis, he suggested that only variation caused by by special causes should be specifically reviewed rather than normal or common cause performance. Understanding and using the concepts of common and special cause of variation as tools in your analysis will help ground your message in a reality that focuses on where specific performance is different enough to be studied. Common cause variation is generated by outcomes that are within the capability of the system.  Whereas special cause outcomes represent performance outside the normal capacity of the system. In every case, performance outside of the norm, should be studied and where positive, held up for others to emulate. By focusing your spotlight on these outcomes you have the opportunity to identify new cutting edge ideas and ideas that should be avoided.  Another technique for fostering collaboration (an environment where envy is less likely to take root) is to invite all parties to participate in the analysis of measurement data using tools such as a WIKI. The measurement group should provide the first wave of analysis, then let the stakeholders participate in shaping the final analysis, using the crowd sourcing techniques made famous by Jimmy Wales and Wikipedia.  Getting everyone involved creates a learning environment that uses measurement not only as tool to generate information, but also as a tool to shape the environment and channel the corporate culture.

Measurement and measurement programs don’t cause the sin of envy.  People and organizational cultures foster this sin in equal measure. Done correctly, measurement programs can act as a tool to tame the excess that lead to this sin. However the corollary is also true.  Done incorrectly or poorly, measurement ceases to be a positive tool and becomes part of the problem.  Measurement that fosters transparency and collaboration will help an organization communicate, grow and improve.


Categories: Process Management

What Do We Mean When We Say "Agile Community?"

Herding Cats - Glen Alleman - Mon, 08/04/2014 - 17:29

In a recent Skype conversation around agile, estimating, Little's Law and the #NoEstimate hashtag, the term agile community was used. My first reaction was whose agile community? The community of sole contributors? The community of $1B weapons systems and all in between?

My thoughts go back to the presentation below. There is a spectrum of project management processes built around agile. My experience starts with 5. Literally 5, since I have time in that aircraft. My software development management experience goes all the way to the end. And aircraft experience to 25.

Paradigm of agile project management from Glen Alleman So when I hear the agile community which one is it? The recent Agile Conference looks like it was attended by sole contributors or small team members. But there are other agile communities where I work

And guideance for deploying agile

So Now Back To The Core Issues

If you're a sole contributor and have a customer sitting near by, estimating you cost, schedule, and technology outcomes is likely of little value. If you're in the other end, say the flight avionics systems for the 777, then the level of rigor, formality, reporting is different. Both use agile. Not all in the same way, but both write code using the principles of agile development

12 agile principlesNo credble management process would or should object to these principles and practices. Do to so means Doing Stupid Things on Purpose. So many of the motivators for not doing something are actually bad management. Let's not estimate, because estimates are misused is my favorite DDSTOP example.

Here's an example of how to connect the dots between these principles and practices in a more formal business management process - in this case Earned Value Management. 

Ev+agile=success (final v2) from Glen Alleman

So when we hear the agile community and those representing the agile community which community is that? 

There is a crass American term used in our domain.

When you see dysfunction, see something you don't understand, or see something that is counter to your paradigm - Follow the Money.

This is the basis of microeconomics of writing software for money. What is considered a waste or even evil in one domain is a critical success factor in another domain.

Ask some simple questions to establish this domain:

  • What's the Value At Risk? 
  • Are you subject to any governance process?
  • Is this project considered mission critical in any way? Project's that are not mission critical have little need to be successful on time, on budget, with the needed capabilities.

In The End

Can we have any meanigful disucssion about any topic in the absence of a domain and context? Especially when that topic is driven by Value at Risk, Governance, and business processes?

I'd say it is incumbent on those making a suggestion for example

No Estimates

To show in what domain this statement is applicable, how we would recognize its applicability outside the the domain of those making the suggestion, how we could test the suggestion to see if it is applicable, and most important what are the conditions that allow the suggestion to work in those domains? 

  • How can decisions be made in the absence of knowing the impact of those decisions? This is a violation of the principles of microeconomics.
  • How can we spend other peoples money and not inform them about the probability of the total cost, delivery date, and confidence in delivering the needed capabilities?
  • How can we plan for needed capacity in the absence of knowing what Done looks like in some unit of measure meaningful to the decision maker - not just the solution provider?
Related articles Agile and the Federal Government Agile as a Systems Engineering Paradigm Can Enterprise Agile Be Bottom Up? What Software Domain Do You Work In? More #NoEstimates Using Agile Effectively in DoD Environments Is Your Organization Ready for Agile? Domain and Context Are King, Then Comes Process and Experience
Categories: Project Management

Tumblr: Hashing Your Way to Handling 23,000 Blog Requests per Second

This is a guest post by Michael Schenck, SRE Staff Engineer at Tumblr.

At Tumblr, blogs (or Tumblelog) are one of our most highly trafficked faces on the internet.  One of the most convenient aspects of tumblelogs is their highly cacheable nature, which is fantastic because of the high views/post ratio the Tumblr network offers our users.  That said, it's not entirely trivial to scale out the perimeter proxy tier, let alone the caching tier, necessary for serving all of those requests.

This article describes the architecture of the portion of our perimeter responsible for blogs serving, one of our more highly trafficked perimeter end-points.

Here's how we do it.

Stats
Categories: Architecture

I Hate Astronomers

Making the Complex Simple - John Sonmez - Mon, 08/04/2014 - 16:00

The majority is not always right. If you get nothing else from this post, but it has helped remind you of this fact, then I’ll feel that I have done my job. The society we live in today is more connected than ever. It is easier and easier to share ideas and communicate. Right now […]

The post I Hate Astronomers appeared first on Simple Programmer.

Categories: Programming

The Use and Misuse of Little's Law and Central Limit Theorem in Agile

Herding Cats - Glen Alleman - Mon, 08/04/2014 - 14:36

6a00d834527c1469e20120a8a4a808970b-800wiLittle's Law and the Central Limit Theorem are used many times in agile software discussions when speaking about flow processes based on Deming's principles. 

These discussions usually start by quoting something from a summary of Little's Law or the Central Limit Theorem. 

A critical element of both Little's Law and the CLT is the notion of Identical Independently Distributed (IID) random variables. These variables are the arrival rate to a service - stories selected from the backlog for development, or someone arriving in line at the bank to make a deposit of a check.

Let's start with some math. In probability theory, the central limit theorem (CLT) says,

  • Given certain conditions (we'll define these next), the arithmetic mean (the average) of a sufficiently large number (this needs to be defined) of samples of independent random variables, each with a well-defined expected value and well-defined variance, will be approximately normally  (Gaussian) distributed. 
  • There are some important ideas here, Independent random variables are the first important idea. That is the variables are not related to each other.
  • As well these random variables must be Identically distributed. That is, they are all drawn from the same underlying probability distribution, and this assumes a large - usually infinite - population of random variables.

Using a grocery store cehck-out line or bank teller window example, Little’s law gives the relation between the mean number of customers in the system, E(L), and the mean transit time through the system, E(S), and the average number of customers entering the system per unit time, λ, as E(L) = λE(S).

An Actual Example In Preparation for Developing Software From The Story Queue

Let's pretend we're in line at the grocery store. We'll call the check-out line the resource and the people lining up at the check-out line the customers. If the clerk manning the check-out station is busy checking out customers, a queue will form in the line waiting to check out.

The population of customers that can use the store is usually finite, but this makes the problem harder, so let's assume for the moment the population of customers is infinite. The number of check-out lines can be one or many, but we'll want to assume they are identical in their services for the moment as well. Let's define the capacity of the store as the number of people that can wait in line, plus the person being served by the clerk. In most stores there will be a finite number of people in the queue at check-out, but again if this number is infinite to makes the analysis easier.

We need another simplifying assumption. The distribution of the amount of time each customer stays at check-out (once they arrive) is Independent and Identically  Distributed (IID). As well the probability that a customer will arrive at check-out is also an IID variable. This distribution is usually taken to be exponential. This means the longer you wait, the more likely it is someone will show up at the check-out stand ready to check out.

These are critical assumptions for what follows about Little's Law. If the above conditions are not met, Little's Law is not applicable to the problem being described.

So let's have a quick summary of Little's Law:

Mean number of people in line at check-out = Arrival Rate of customers Ă— Check-Out Time

This law can be applied if some other conditions exist:

  • The mean number of people arriving at the check-out line equals the mean number of people being checked out,.
  • This means - over a long period of time - the mean number of people arriving in line approximately equals the mean number of people leaving the store after check-out. 
  • In math terms:
    • The number of arrivals N over a long period of time T defines the arrival rate.
    • Mean Arrival Rate = Total Arrivals / Total Time = N/T.

Now for Software Development

Instead of assuming if Little's Law can be applied to software development, let's first ask are the conditions right to apply the law:

  • Is the arrival rate at the server an Identical Independently Distributed random variable?
  • Is the service time for each piece of work an Identical Independently Distributed random variable?

This means, do the jobs - stories - arriving at the service - development (or some other process) - behave like IID variables. That is they have no knowledge of each other and are indistinguishable from each other and when serviced, they can not be distinguished from the other work serviced(meaning developed, tested, installed, etc.)

Let's look at an actual project, a simple one. We want to fly to the moon for the first time, land, and come home.

Gantt ChartTo do this we need to do some work, in a specific order, with dependencies between this work in order to produce some outcomes that enable us to fly to the moon and back - and live to tell about it.

Doing work on a development project is not the same as work arriving in a queue of a service, it is a network of dependencies, with interconnections, and most importantly - most critical actually - the duration of the work, the time spent in the service - are not independent, identically distributed random variables. A network of work looks like this (notionally).

Random IMS

So does Little's Law apply? Nope!

  • The work arriving at the service is not an Independent Identically Distributed random variable.
  • The service time is not an Independent Identically Distributed random variable.
  • The dependencies between work may itself by a random variable of unknown distribution, behaviing dynamically depending on the prior work, the follow-on work and other external conditions.

These networks of work are called Stochastic Networks and are not subject to Little's Law. The proicess in the Little's Law condition can be stochastic, but there has to be independence between the work elements and they have to be identically distirbuted probability distributions.

Production queues of parts going down an assembly line are. Cards being pulled from the the tray in a Kanban furniture manufacturing system are. The notional Kanban system in agile development are - but If and Only IF (IFF) the work pulled from the wall is independent from all other work, and the probability distribution of the durtaion of that work independently distributed from the actual work as well.

If you can find a project where all the features are independent from each other, there work efforts are identical, independently distributed random variables, and you'll be able to apply Little's Law.

The End

Little's Law applies to software development work that looks like productioni flow - like the assembly line at Toyota, or the office furniture production line we designed at a factory in Idaho.

But those types of software projects must be intentionally designed to have no dependencies between the work performed, have the duration of the work in the service cycle (development) have no dependency on the prior work or the work. 

This is the condition of Identical Independent Distribution (IID) needed for Little's Law as well as the Central Limit Theorem. So befoe anyone says Little's Law applies to software development, they need to show these conditions exist.

One Final Observation

The slicing proposed by some in the agile community might create the conditions for Little's Law to work. But the effort to slice the stories into equal sized - or at least Independent Identically Distributed work sizes for the entire project duration seems like a lot of work. Especially when there are much easier ways to estimating the total work, total duration, and total cost. 

But since this slicing paradigm appears to be anecdotal and untested across a wide variety of projects, domains, and sizes, the population sample size condition is unlikely to be met as well. 

More research, based on actual analysis needs to be done, then that research reviewed and tested, before the notion of mathematically slicing has much use outside of anecdotal examples.

Resources (of many)

Related articles Central limit theorem animation Four Critical Elements of Project Success Everything is a Random Variable Is fine grained decomposition of work the same as "Not Estimating?" Making Estimates For Your Project Require Discipline, Skill, and Experience
Categories: Project Management

SPaMCAST 301- Technical Debt Essay

www.spamcast.net

http://www.spamcast.net

Listen to the Software Process and Measurement Cast.

Software Process and Measurement Cast number 301 features our essay on technical debt. Technical debt is the work not done or the shortcuts taken when delivering a product. We all take shortcuts, but at what cost? The essay begins:

Technical debt is a term coined by Ward Cunningham to represent the work not done or the shortcuts taken when delivering a product. In almost every circumstance there are multiple paths than can be taken to deliver a functional product.  For example, when documenting the code you are writing there is a difference between explaining exactly what the code does in detail and being terse and a bit oblique (I can hear the rationalization, “they can just read the code”). The code runs, but if there is ever a problem it will take longer to diagnose the problem. Whether fixing a defect or rewriting the code, if there is a delay caused by figuring out the code, that represents the ‘debt’ of technical debt.  Technical debt is applied to software, but the phrase can be extended to any deliverable or product.  The work that is not done may or may not be fixed in the future.  Until the technical debt is paid back, the debt accrues interest.  Whether or not that interest is important depends on your situation.

Listen to the rest on the Software Process and Measurement Cast 301

 

Next

Software Process and Measurement Cast number 302 will our interview with Larry Maccherone of Rally Software. We talked about Agile and metrics.  Can you combine Agile and metrics without creating an oxymoron?

Upcoming Events

I will be presenting at the International Conference on Software Quality and Test Management in San Diego, CA on October 1.  I have a great discount code!!!! Contact me if you are interested!

I will be presenting at the North East Quality Council 60th Conference October 21st and 22nd in Springfield, MA.

More on all of these great events in the near future! I look forward to seeing all SPaMCAST readers and listeners that attend these great events!

The Software Process and Measurement Cast has a sponsor.

As many you know I do at least one webinar for the IT Metrics and Productivity Institute (ITMPI) every year. The ITMPI provides a great service to the IT profession. ITMPI’s mission is to pull together the expertise and educational efforts of the world’s leading IT thought leaders and to create a single online destination where IT practitioners and executives can meet all of their educational and professional development needs. The ITMPI offers a premium membership that gives members unlimited free access to 400 PDU accredited webinar recordings, and waives the PDU processing fees on all live and recorded webinars. The Software Process and Measurement Cast some support if you sign up here. All the revenue our sponsorship generates goes for bandwidth, hosting and new cool equipment to create more and better content for you. Support the SPaMCAST and learn from the ITMPI.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.

 

 


Categories: Process Management

SPaMCAST 301- Technical Debt Essay

Software Process and Measurement Cast - Sun, 08/03/2014 - 22:00

Software Process and Measurement Cast number 301 features our essay on technical debt. Technical debt is the work not done or the shortcuts taken when delivering a product. We all take shortcuts, but at what cost? The essay begins:

Technical debt is a term coined by Ward Cunningham to represent the work not done or the shortcuts taken when delivering a product. In almost every circumstance there are multiple paths than can be taken to deliver a functional product.  For example, when documenting the code you are writing there is a difference between explaining exactly what the code does in detail and being terse and a bit oblique (I can hear the rationalization, “they can just read the code”). The code runs, but if there is ever a problem it will take longer to diagnose the problem. Whether fixing a defect or rewriting the code, if there is a delay caused by figuring out the code, that represents the 'debt' of technical debt.  Technical debt is applied to software, but the phrase can be extended to any deliverable or product.  The work that is not done may or may not be fixed in the future.  Until the technical debt is paid back, the debt accrues interest.  Whether or not that interest is important depends on your situation.

Listen to the rest on the Software Process and Measurement Cast 301

 

Next

Software Process and Measurement Cast number 302 will our interview with Larry Maccherone of Rally Software. We talked about Agile and metrics.  Can you combine Agile and metrics without creating an oxymoron?

Upcoming Events

I will be presenting at the International Conference on Software Quality and Test Management in San Diego, CA on October 1.  I have a great discount code!!!! Contact me if you are interested!

I will be presenting at the North East Quality Council 60th Conference October 21st and 22nd in Springfield, MA.

More on all of these great events in the near future! I look forward to seeing all SPaMCAST readers and listeners that attend these great events!

The Software Process and Measurement Cast has a sponsor.

As many you know I do at least one webinar for the IT Metrics and Productivity Institute (ITMPI) every year. The ITMPI provides a great service to the IT profession. ITMPI’s mission is to pull together the expertise and educational efforts of the world’s leading IT thought leaders and to create a single online destination where IT practitioners and executives can meet all of their educational and professional development needs. The ITMPI offers a premium membership that gives members unlimited free access to 400 PDU accredited webinar recordings, and waives the PDU processing fees on all live and recorded webinars. The Software Process and Measurement Cast some support if you sign up here. All the revenue our sponsorship generates goes for bandwidth, hosting and new cool equipment to create more and better content for you. Support the SPaMCAST and learn from the ITMPI.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.

Categories: Process Management

How Employees Lost Empathy for their Work, for the Customer, and for the Final Product

“In most cases being a good boss means hiring talented people and then getting out of their way.” -- Tina Fey

The Digital Revolution marked the beginning of the Information Age.

The Information Age, or Digital Age, or New Media Age, is a shift away from the industrial revolution to an economy based on information computerization.  Some would say, along with this shift, we are now in a Knowledge Economy or a Digital Economy. 

This opens the door to new ways of working and a new world of work to generate new business value and customer impact.

But what did the Industrial Age do to employees and what paradigms could limit us in this new world?

In the book The Future of Management, Gary Hamel walks through how industrialization and large Enterprises have created a disconnect between employees and their customers, their final product, and the big financial picture.  And in the process, he argues, this had led to disengaged employees, crippled innovation, and inflexible organizations.

If you don’t know Gary Hamel, he’s been ranked the #1 influential business thinker by the Wall Street Journal.

According to Hamel, what we traded for scale and efficiencies created gaps between workers and employees and gaps between employees and their customers, the product, the financial impact, and … a diminished sense of responsibility for quality and efficiency.

Maybe We Have Managers Because We Have Employees

Do managers exist because employees do?

Via The Future of Management:

“Here's a thought.  Maybe we need 'managers' because we have 'employees.'  (Be patient, this is not as tautological as it sounds.)  Think about the way computers are dependent on software.  PCs aren't smart enough to write their own operating instructions, and they sit idle until a user sets them to work.  Perhaps the same is true for employees.”

Did We Manufacture a Need for Managers?

When we manufactured employees, did we manufacture a need for managers?

Via The Future of Management:

“Earlier, I talked about the invention of 'the employee.' What happened in this process, at the dawn of the 20th century?  How did work life change as individuals left their farms and workshops to be absorbed into large-scale organizations?  In manufacturing employees, did we manufacture a need for managers as well?  I think so.  If we understood how this came about, we will gain clues into how we might learn to manage without managers -- or, at least, with a lot fewer of them.”

Disconnected from the Customer

As the size and scale of industrial organizations grew, so did the disconnect between employees and their final customers.

Via The Future of Management:

“In pre-industrial times, farmers and artisans enjoyed an intimate relationship with their customers.  The feedback they received each day from their patrons was timely and unfiltered.  Yet as industrial organizations grew in size and scale, millions of employees found themselves disconnected from the final customer.  Robbed of direct feedback, they were compelled to rely on others who were closer to the customer to calibrate the effectiveness of their efforts and to tell them how they could better please their clients.”

A Diminished Sense of Responsibility for Producer Quality and Efficiency

Without a connection to the customer, employees lose empathy for their work, for the customer, and for the final product.

Via The Future of Management:

“As companies divided themselves into departments and functions, employees also became disconnected from the final product.  As tasks became narrower and more specialized, employees lost their emotional bond with the end product.  The result? A diminished sense of responsibility for producer quality and efficiency.  No longer were workers product craftsmen, now they were cogs in an industrial machine over which they had little control.”

Employees No Longer Have a System Wide View of the Production Process

It’s hard to make changes to the system when you no longer have a system wide view.

Via The Future of Management:

“Size and scale also separate employees from their coworkers.  Working in semi-isolated departments, they no longer had a system wide view of the production process.  If that system was suboptimal, they had no way of knowing it and now way of correcting it.”

The Gap Widens Between Workers and Owners

People at the top don’t hear from the people at the bottom.

Via The Future of Management:

“Industrialization also enlarged the gulf between workers and owners.  While a 19th-century apprentice would have had the ear of the proprietor, most 20th-century employees reported to low-level supervisors.  In a large enterprise a junior employee could work for decades and never have the chance to speak one-on-one with someone empowered to make important policy decisions.”

The Scoreboard is Contrived

Scoreboards tell employees how they are doing their jobs, but not how the company is doing overall.

Via The Future of Management:

“In addition, growing operational complexity fractured the information that was available to employees.  In a small proprietorship, the financial scoreboard was simple and real time; there was little mystery about how the firm was doing.  In a big industrial company, employees had a scoreboard but it was contrived.  It told workers how they were doing their jobs, but little about how the company was doing overall.  With no more than a knothole view of the company's financial model, and only a sliver of responsibility for results, it was difficult for an employee to feel a genuine burden for the company's performance.”

Industrialization Disconnects Employees from Their Own Creativity

Standardizing jobs and processes limits innovation in the jobs and processes.  They are at odds.

Via The Future of Management:

“Finally, and worst of all, industrialization disconnected employees from their own creativity.  In the industrial world, work methods and procedures were defined by experts and, once defined, were not easily altered.  No matter how creative an employee might be, the scope for exercising that gift was severely truncated.”

The Pursuit of Scale and Efficiency Advantages Disconnected Workers from Their Essential Inputs

With the disconnect between employees and their inputs, there was a natural need for the management class.

Via The Future of Management:

“To put it simply, the pursuit of scale and efficiency advantages disconnected workers from the essential inputs that had, in earlier times, allowed them to be (largely) self-managing -- and in so doing, it made the growth on an expansive managerial class inevitable.”

Employees Don’t Lack Wisdom and Experience

Employees don’t lack wisdom and experience.  They just lack information and context.

Via The Future of Management:

“To a large extent, employees need managers for the same reason 13-year-olds need parents: they are incapable of self-regulation.  Adolescents, with their hormone-addled brains and limited lie experience, lack the discernment to make consistently wise choices.  Employees on the other hand, aren't short of wisdom and experience, but they do lack information and context -- since they are so often disconnected from customers, associates, end products, owners, and the big financial picture.  Deprived of the ability to exercise control from within, employees must accept control from above.  The result: disaffection.  It turns out that employees enjoy being treated like 13-year-olds even less than 13-year-olds.”

Disengaged Employees, Hamstrung Innovation, and Inflexible Organizations

What is the result of all this disconnect?   Stifled innovation, rigid organizations, and disinterested employees.

Via The Future of Management:

“Disengaged employees.  Hamstrung innovation.  Inflexible organizations.  Although we are living in a new century, we are still plagued by the side effects of a management model that invented roughly a hundred years ago.  Yet history doesn't have to be destiny -- not if you are willing to go back and reassess the time-forgotten choices that so many others still take for granted.  With the benefit of hindsight, you can ask: How have circumstances changed? Are new approaches possible? Must we be bound by the shackles of the past?  These are essential questions for every management innovator.”

Does history have to be destiny?

We’re writing new chapters of history each and every day.

In all of my experience, where I’ve seen productivity thrive, people shine, and innovation unleashed, it’s when employees are connected with customers, they are empowered and encouraged to make changes to processes and products, and they are part of a learning organization with rapid feedback loops.

You Might Also Like

Ability to Execute

Business Value Generation is the New Bottleneck

Customer-Connected Development

How We Adhered to the Agile Manifesto on the Microsoft patterns & practices Team

Why So Many Ideas Die or Don’t Get Adopted

Categories: Architecture, Programming

Free ebook: Building Cloud Apps with Microsoft Azure

ScottGu's Blog - Scott Guthrie - Sun, 08/03/2014 - 03:20

9780735695658f Last week MS Press published a free ebook based on the Building Real-World Apps using Azure talks I gave at the NDC and TechEd conferences.  The talks + book walks through a patterns-based approach to building real world cloud solutions, and help make it easier to understand how to be successful with cloud development. Videos of the Talks You can watch a video recording of the talks I gave here:

 Part 1: Building Real World Cloud Apps with Azure

 Part 2: Building Real World Cloud Apps with Azure

eBook Downloads

You can now download a completely free PDF, Mobi or ePub version of the ebook based on the talks using the links below:

Download the PDF (6.35 MB)  

Download the EPUB file (12.3 MB)  

Download the Mobi for Kindle file (22.7 MB)

Hope this helps,

Scott

Categories: Architecture, Programming

Seven Deadly Sins of Metrics Programs: Pride

Can't see forest for the trees

Can’t see forest for the trees

The first deadly sin is pride. In the cannon of deadly sins, pride is the sin from which all other spring. In the world of metrics programs, the sin of pride is when a metrics program settles on a single metric that is used to reflect the value or well-being of a project, a group or organization. Examples of abound of metrics programs that fixated on cost or productivity to the exclusion of a broader palette of metrics and work attributes.  Most metrics professionals quickly learn that one metric cannot be used for all projects.  If you can’t easily answer the question, “Does this relate?” each time you use a metric and for each metric you use, the information generated through measurement and analysis will provide little or no value.  The goal is to understand the differences between groups of work so that when comparisons are made, you can discern what is driving the difference (or even if there is a difference).  Comparing package implementations, hardware intensive projects or custom development is rational only if you understand that there will be differences and what those differences mean.   The bottom line is that rarely does a single metric deliver that deep level of understanding that generates value from measurement.

Another example of the single metric syndrome generated by the sin of pride occurs when an organization uses a single metric to value performance in a contractual arrangement.  While entire contracts are rarely stipulated on a single metric, it easy for a single metric to be given disproportional weight due to the framers’ lack of understanding or a disconnect between the framers and the people that administer the contract.  Poor understanding of the relationship between the numbers and the concepts they represent is akin to failure in punctuation when writing.  The resulting meaning can be garbled as the contract is negotiated, implemented and managed.  We won’t get into an existential argument over whether something is a sin if it is inadvertent; the result is the same. Garbled concepts can lead to a single metric focus which once discovered will beg to be taken advantage of.  This usually causes an overemphasis on a specific portion of the value chain, such as productivity being emphasized over time-to-market, quality or cost.


Categories: Process Management

Code Golf

Phil Trelford's Array - Sat, 08/02/2014 - 19:22

Last month Grant Crofton popped down from Leeds to the F#unctional Londoners meetup at Skills Matter to run a fun hands on code golf session. The idea of code golf is to implement a specific algorithm in the fewest possible characters. This is not the kind of thing you should be doing in enterprise code, but it is fun, and an interesting way of exploring features in a programming language.

On the night we attempted condensed versions of FizzBuzz and 99 bottles of beer, with Ross and I winning the first challenge and Simon & Adrian the second.

FizzBuzz Score99 Bottles Score

Thanks again to Grant for a really fun session.

F# FizzBuzz

A while back I strived to squeeze an F# implementation of FizzBuzz into a tweet, and with white space removed it weighs in at 104 characters (excluding line breaks):

for n=1 to 100 do 
 printfn"%s"
  (match n%3,n%5 with 0,0->"FizzBuzz"|0,_->"Fizz"|_,0->"Buzz"|_,_->string n)

The implementation, although quite clear, requires a fair number of characters for the pattern matching portion.

After some head scratching we came up with the idea of using a lookup to select the string to display:

N %3 N % 5 Index Output 0 0 0 N >0 0 1 “Fizz” 0 >0 2 “Buzz” >0 >0 3 “FizzBuzz”

This took the implementation down to 89 characters (without line breaks):

for i=1 to 100 do
 printfn"%s"["FizzBuzz";"Buzz";"Fizz";string i].[sign(i%5)*2+sign(i%3)]

Another trick is to abuse the sign function, to get 1 if the result of the modulus is above 0 and 0 otherwise.

The lookup trick can be used in other languages, and here’s a few examples, just for fun.

VB.Net FizzBuzz

VB.Net has a reputation for being a little verbose, but using the lookup trick it was possible to it get down to 96 characters (excluding line breaks):

For i=1 To 100:
Console.WriteLine({i,"Fizz","Buzz","FizzBuzz"}(-(i Mod 3=0)-(i Mod 5=0)*2))
:Next

In VB.Net true values translate to –1 and false to 0. This allowed me to simply negate the result of i % N = 0 to compute an index.

Python FizzBuzz

Using a similar trick in Python, where true translates to 1 and 0 to false, I was able to get to a very respectable 79 characters (excluding line breaks):

for x in range(1,101):
 print([x,"Fizz","Buzz","FizzBuzz"][(x%3==0)+2*(x%5==0)])

Python’s simple print function also helped to keep the character count down.

Amanda FizzBuzz

Amanda is a variant of David Turner’s quintessential functional programming language Miranda. Amanda runs on Windows, and is used for teaching FP at some universities.

Using a list comprehension it was possible to squeeze in to a mere 67 characters:

[["FizzBuzz","Buzz","Fizz",itoa x]!(min[1,x%3]+min[1,x%5]*2)|x<-[1..100]]

Note: this is cheating a little as we are not explicitly writing to the console.

APL FizzBuzz

APL is a very mature language, dating back to the 1960s, and is still used commercially today. It also wins hands down in code golf with just 54 characters:

⍪{'FizzBuzz' 'Buzz' 'Fizz'⍵[1+(2×1⌊5|⍵)+1⌊3|⍵]}¨⍳100

APL is particularly strong at matrix processing and provides single character representations for common operations:

Notation Name Meaning ⍳B Index generator Creates a list from 1 to B ¨ Each for each loop ⍪ Table Produces a one column matrix ⌊B Floor Greatest integer less than or equal to B

Try APL in your browser now.

Challenge

Can you beat the APL implementation of FizzBuzz?

Have fun!

Categories: Programming

Seven Deadly Sins of Metrics Programs

3216110254_15d4a00bf2_b

In Christianity, the seven deadly sins are the root of all other sins. This concept has been used as an analogy for the ills or risks for many professions.  The analogy fits as well for software metrics; focusing attention on the behaviors that could sap your program’s integrity, effectiveness and lifespan. Here we will look at the deadly sins from the point of view of a person or group that is creating or managing a metrics program. As with many things in life, forewarned is forearmed, and knowledge is a step towards avoidance.

Here are the seven deadly sins of metrics programs:

  • Pride – Believing that a single number/metric is more important than any other factor.
  • Envy – Instituting measures that facilitate the insatiable desire for another team’s people, tools or applications.
  • Wrath – Using measures to create friction between groups or teams.
  • Sloth – Unwillingness to act on or care about the measures you create.
  • Greed – Allowing metrics to be used as a tool to game the system for gain.
  • Gluttony – Application of an excess of metrics.
  • Lust – Pursuit of the number rather than the business goal.

All of the deadly sins have an impact on the value a metrics program can deliver.  Whether anyone sin is more detrimental than another is often a reflection of where a metrics program is in it’s life cycle. For instance, pride, the belief that one number is more important than all other factors, is more detrimental than sloth or a lack of motivation as a program begins whereas sloth becomes more of an issue as a program matures.  These are two very different issues with two very different impacts, however neither should be sneezed at if you value the long-term health of a metrics program. Pride can lead to overestimating your capabilities and sloth can lead to not using those you have in the end self-knowledge is the greatest antidote.

Over the next few days we will visit the seven deadly sins of metrics!


Categories: Process Management

Can Enterprise Agile Be Bottom Up?

Herding Cats - Glen Alleman - Fri, 08/01/2014 - 20:48

Much of the objection to SAFe comes from its seemingly Top Down paradigm. Many agile voices object that this approach is not agile, in the way they define agile - individual teams making their own decisions about what to do with their customer.

The domain of this bottom up approach is usually not well defined, other than the classic eXtreme Programming or the Agile Spectrum of  where Co-Hacking is on the left, where the developers live by the pure agile manifesto. 

But what happens when agile is applied to an enterprise development effort. One where the business needs define the capabilities that are not emergent, but rather they are needed to fulfill the business strategy or the mission of the organization. Then another paradigm emerges. One where higher order questions, frameworks, framing assumptions, governance, and other externalities trump the needs of the individual team.

Here's one approach that has served us well over time. 

Critical Success Factors for ERP from Glen Alleman   So when we hear about critisms of SAFe like:
  • The structure and teaching of SAFe is relentlessly top down. All the important stuff is figured out up in Portfolio. All the features are figured out up in Program. Now you guys Build and Test that.

The  statement is a bit off, since it's the Capabilities that are defined by the business. These capabilities are then turned into requirements, which may in fact emerge, which themselves are turned into working software. Starting with the capabilities, an enterprise software development effort means re-looking at the agile manifesto statements.

  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

    • It certaintly is the highest priority to satisfy the customer with valuable software.
    • The delivery in the enterprise needed to be planned at the absorption rate and business rhythm. 
    • The units of measure of value need to be connected to the business strategy. This is usually to role  of Balanced Scorecard, which defines the value in the 4 perspectives of the business. 
      • Financial
      • Customer
      • Internal business processes
      • Learning and Growth
      • Here's how to develop your BSC for enterprise software development.
    • Releasing software in the enterprise means integrating with the business processes, customer processes, product release process - all defined by the rhythm of how the business works
    • Imagine updating the point of sale terminals across 10,000 users on a daily basis, adding new features, changing possibly for the better how they work.
    • The training, Tier 1 support, transaction formatting, and other enterprise actiities woudl alos have to change.
  • Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

    • This is one of those untested statements.
    • Imagine changing how ICD-10 claims are processed late in the development cycle.
    • Other enterprise systems are based on ICD-10 formats, now they're changing as we approach "go live." 
    • We need look no further than the ACA roll out to see how nonsensical this is.
  • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

    • For the enterprise, the business rhythm defines the frequency of software release. 
    • For example, the flight navigation systems for an aircraft have many other processes beyond just writing code. 
    • Full Information Assurance, DO-178 validation, Digital Flight Bag integration, PCI
    • For health care SP-600 for HIPPA verification and validation.
    • Or for the IT Enterprise ISACA.
    • Externalities drive business rhythm. Having the developers decide this rhythm ignores these externalities. If there are no externalities, then the project is likely perfect for the agile manifesto.
  • Business people and developers must work together daily throughout the project.

    • This works when the business people don't have day jobs.
    • If they have day jobs, some proxy of the needed capabilities, and requirements is needed.
    • This is the role of documentation and models. Both developed concurrently with developers and business people.
  • Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

    • Can't go wrong here.
    • Remember thought, trust but verify.
    • The value at risk needs to be considered. What happens if what the developers say turns out not to be the case. How much are you willing to risk that the trust was misplaced?
  • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

    • Back to the availability issue. 
    • Geography is another issue. Many teams are distributed.
    • Face to Face needs a proxy.
  • Working software is the primary measure of progress.

    • Correct, this is basis of Earned Value Management's Physical Percent Complete.
    • Working needs a definition. Measures of Effectiveness, Measures of Performance, Key Performance Parameters, Technical Performance Measures are all mechanisms to assess the workingness of the software in the enterprise domain.
  • Agile processes promote sustainable development. 

    • All work processes should be sustainable, nit just agile.
    • Knowing the capacity for work is the starting point.
    • This means not only knowing the capacity, but planning the work around this capacity with the needed margin to provide for sustainability.
  • The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

    • Yes, good business and engineering practice.
    • Requires capacity planning
    • Cost, Schedule, and Technical margin.
    • Measures of performance to make steering adjustment and needs a control system to connect the dots between the desired outcome, and actual performance, and the corrective actions needed to sustain this needed performance.
  • Continuous attention to technical excellence and good design enhances agility.

    • This is a platitude that is used in all engineering domains.
    • rarely if ever are product build with low technical excellence and good design practices.
    • Exactly how this enhances agility is untested outside of anecdotes.
    • But certainly changeability in response to external factors is measurable as robustness of the design.
  • Simplicity - the art of maximizing the amount of work not done - is essential.

    • Another platitude, without any actionable suggestions.
    • But what are the units of measure of simplicity.
    • Everyone should read Synthesis of Form, Christopher Alexander.
    • Then the practice for enterprise projects, The Art of Systems Architecting, 2nd Edition, Mark W. Maier and Eberhardt Rechtin.
    • And followed by Systems Engineering: Coping with Complexity, Richard Stevens, et al.
    • These three books address the platitude and replace it with actionable steps to maximize value produced with minimal work.
  • The best architectures, requirements, and designs emerge from self-organizing teams.

    • In the enterprise domain this is a non-starter.
    • Architecture is the glue to holds the systems integration together.
    • Emerging architectures means, all the other systems in the enterprise will be constantly changing as the architecture of the system under development changes.
    • Those self-organizing teams may or may not have the skills, experience, and even ability to be  systems architects.
    • Read the systems engineering books above. Join www.incose.org to learn how complex enterprise systems are defined, developed, deployed, and maintained.
  • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

    • This is standard good engineering practice, not unique to agile.

So What Does All This Mean?

Without a domain, hard to assess the applicability and appropriateness of much of anything.

What this really means of Scaled Agile Framework is the place to start for the enterprise.

Related articles Agile Requires Discipline, In Fact Successful Projects Require Discipline Agile and the Federal Government The Agile Architecture Mind Map Capabilities Based Planning and Development Agile in the enterprise: To succeed, avoid the fundamentalists How To Create Confusion About Root Causes Agile Paradigm The Myth of Incremental Development An Agile Estimating Story Agile as a Systems Engineering Paradigm
Categories: Project Management

Stuff The Internet Says On Scalability For August 1st, 2014

Hey, it's HighScalability time:


From Systems Performance: Enterprise and the Cloud.
  • Quotable Quotes:
    • @shanselman: Wife: "How was your day?" Me: "I'm using Grunt to automate NuGet creation for AngularJS." Wife: "But will that scale?" Me: "Well played."
    • John Hagel: winners in the concentrating parts of the economy are increasingly determined by the ability to connect with, and build strong relationships with, the participants who are operating in fragmenting parts of the economy.
    • Jack Clark: This means that although Amazon still grew at a respectable rate, its actual revenues were clipped by the heightened competition. This is what happens when you sell goods with deflationary pricing, it seems.

  • Taxi app Hailo on Scaling micro-services Architecture on AWS: Micro-services + Containers + Scheduling on AWS will be a dominant architecture pattern in the next few years.

  • Netflix. Revisiting 1 Million Writes per second. How will Cassandra perform on AWS's new instance types? There's no big reveal so you'll have to decide for yourself. Good discussion on reddit and Hacker News.

  • TrueTime in Google's Spanner was one of its most buzzworthy innovations. Who doesn't like atomic clocks as a way to time-stamp transactions anywhere in the world? What if you don't have spare atomic clocks? Hybrid Logical Clocks: HLC captures the causality relationship like LC, and enables easy identification of consistent snapshots in distributed systems. Dually, HLC can be used in lieu of PT clocks since it maintains its logical clock to be always close to the PT clock.

  • Vertical integration for the win. Apple is building out their own CDN with many terabits of capacity. Capable of handling traffic bursts from software downloads. Or maybe something else? More from Dan Rayburn in Apple’s CDN Now Live: Has Paid Deals With ISPs, Massive Capacity In Place

  • Clouds make a lot of money on their network pricing. Chris Swan explores this marketing magic in Cloud Price Wars – What about the network?: there haven’t been any major shifts in network pricing.

  • Scaling with Microservices and Vertical Decomposition: The architecture of otto.de is based on the concept of vertical decomposition: the whole system is vertically split into several loosely coupled applications. Every “vertical” is responsible for a single business domain such as “Order”, “Search & Navigation”, “Product”, etc. It has its own presentation layer, persistence layer and a separate database. From the development perspective, every vertical is implemented by exactly one team and no code is shared between the different systems.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Playing around with Yo

Xebia Blog - Fri, 08/01/2014 - 13:09

Yo has been quite a bit in the news lately. Mainly because it got a lot of investment, which surprised and shocked some people because it all seems too simple. Yo only allows you to send a Yo notification to other users. However, it has a lot of potential to become big while staying that simple.

Screenshot 2014-08-01 14.23.48

After reading Why A Stupid App Like Yo May Have Billion-Dollar Platform Potential a few days ago I felt it was time to play around a bit with Yo and it's API.

I came up with 3 very simple use cases that should be simple to solve with Yo:

  • Get a Yo when it's time to have lunch when I'm at work
  • Get a Yo when I forgot to check out from my parking App
  • Get a Yo when a new blog post is published here on the Xebia Blog

Time to register a couple of Yo developer usernames. The Yo API is, just like Yo itself, very simple. You register a username from which you want to receive notifications at http://yoapi.justyo.co after which you'll receive an API token for that username. Once you have that, people can subscribe to that username with the Yo app. Then you can send a Yo to either all subscribers or to a single subscribe with a simple POST request and the token. All this is explained at https://medium.com/@YoAppStatus/e7f2f0ec5c3c in more detail.

Let's start with our lunch notifications. I created the TIME2LUNCH username for this. I want my notifications at 12:00pm, since that's the time I go to lunch. So all I need is some service that sends the POST request every day at 12:00pm (for now I don't care about getting one in the weekend as well and I only care about my own time zone, Central European Time). Using NodeJS, it's just a single line of code to do such a request:

require('request')
  .post('http://api.justyo.co/yoall/', 
    {
      form: {api_token: 'my_secret_api_token'}
    }
  );

Now we need to have a scheduler that executes it every day at 12:00pm. Luckily Heroku has a scheduler that can do exactly that:

Screenshot 2014-07-31 17.43.51

So after deploying our javascript file with a single line of code and creating our scheduled job we will receive our daily Yo from TIME2LUNCH. Not bad for a first attempt.

Usually my co-workers will remind me when it's time to go to lunch so let's do something that's actually a bit less useless.

To park my car near the office I always use the Parkmobile service. At the end of the day I have to check out to avoid paying too much. Unfortunately it's an easy thing to forget. Parkmobile knows this and can send sms alerts at a specific time or after parking a certain amount of hours. Unfortunately they charge € 0.25 per sms. They also send e-mails, but they're easier to miss. It would be nice to get a Yo instead, for free of course.

What we need is to send the Yo POST request each time we receive the Parkmobile e-mails. Sounds like we might be able to use IFTTT (if this then that) to accomplish this. When browsing for channels and recipes on IFTTT I saw that they already support Yo as a channel. I thought I was gonna be done fast. Unfortunately it's only possible to use Yo as a trigger (if Yo then that) and not as an action (if this then Yo). So we need another solution here. I couldn't find a way to send a cURL request directly from IFTTT, but when Googling for a solution I found a webhook project: https://github.com/captn3m0/ifttt-webhook. The ifttt-webhook works by acting as a WordPress site, which is something that can act as an action of IFTTT. It then allows us to send a POST request to a specific URL. Not exactly the POST requests that are accepted by the Yo API though. But we already made some NodeJS code to send a Yo request so I'm sure we can add some code to accept a request from the ifttt-webhook and then pass it on to something that Yo does understand.

If we follow the instructions on the Github page and set our username to our Yo username and use our API token as password, then the webhook will send a POST request with a JSON body that looks something like this:

{ user: 'MYUSERNAME', pass: 'ab1234-1234-abcd-1234-abcd1234abcd', title: '' }

We can handle that in NodeJS like this:

var express = require('express');
var bodyParser = require('body-parser')
var app = express();
var request = require('request');

app.use(bodyParser.json());
app.post('/api/yo', function (req, res) {
  var user = req.body.user;
  var apiToken = req.body.pass;
  request.post('http://api.justyo.co/yo/',
    {
      form: {
        api_token: apiToken,
        username: user
      }
    });
});

var port = Number(process.env.PORT || 5000);
app.listen(port, function() {
  console.log('Listening on ' + port);
});

This is just a simple express web server that listens for POST calls on /api/yo and then uses the user and pass fields from the body to send a POST request to the Yo API.

This is deployed at http://youser.herokuapp.com/ so everyone can use it as a IFTTT to Yo action.

We can now create our IFTTT recipe. Creating the this step is easy. I receive the e-mails from Parkmobile in my Gmail and their e-mail address is norepy@parkmobile.com. So the rule becomes to trigger each time when I receive an email from them. Then in the that step I activate the WordPress channel with the Yo username and api token and in the body I set the http://youser.herokuapp.com/api/yo URL.

Here is the recipe:

Screenshot 2014-07-31 18.32.12

The last use case I had is to send a Yo each time a new blog post was posted on this blog. For that I registered the XEBIABLOG username (so make sure to subscribe to that in your Yo app if you want to get Yo'd as well for each new blog post).

Since this blog has an RSS feed, I figured I could poll that once in a while to check for new posts. We also already used the Heroku scheduler, so we might as well use that again. I found a little node library called feed-read that makes reading RSS feeds easy. So here is our little app that runs every hour:

var feed = require("feed-read");
var request = require('request');
var ONE_HOUR = 60 * 60 * 1000;

feed("http://blog.xebia.com/feed/", function(err, articles) {
  if (err) throw err;

  var lastArticle = articles[0];
  if ((new Date() - lastArticle.published) < ONE_HOUR) {
    console.log('Sending Yo for ' + lastArticle.title);
    request.post('http://api.justyo.co/yoall/',
      {
        form: {
          api_token: 'my_secret_token'
        }
      });
  }
});

We now have completed our 3 little use cases. Not the most useful things but nice non nonetheless. When looking back on them, we can imagine a couple of improvements. For example for the TIME2LUNCH it would be possible to make a little service where people could register and set their timezone at which they want to receive their notification. We could create a little database that store Yo usernames and the zone. But at this moment it's not possible to verify that USERX is really USERX. Yo doesn't support third party authentication like Facebook and Twitter have with OAuth. Perhaps that's something they will add in the future to make platform more useable for user specific notifications.

13 Business Models for Book Authors

NOOP.NL - Jurgen Appelo - Fri, 08/01/2014 - 11:03
Business Models for Book Authors

It’s such a great time to write books!

The publishing world is transforming rapidly and smart authors–with a sense of business–will thrive on opportunities that didn’t exist until a few years ago.

It appears that almost one-third of all e-books sold on Amazon are published by indie authors. This is great news for everyone who loves freedom and independence (including yours truly). On the other hand, income for book writers seems to be declining steadily. This will only get worse now that Amazon has started offering subscriptions to read unlimited numbers of e-books, thereby moving in the same direction as Netflix (for movies) and Spotify (for music), further reducing the income for authors along the way.

The post 13 Business Models for Book Authors appeared first on NOOP.NL.

Categories: Project Management

Learn How UX Design can Make Your App More Successful

Android Developers Blog - Fri, 08/01/2014 - 02:39

By Nazmul Idris, a Developer Advocate at Google who's passionate about Android and UX design

As a mobile developer, how do you create 5-star apps that your users will not just download, but love to use every single day? How do you get your app noticed, and how do you drive engagement? One way is to focus on excellence in design — from visual and interaction design to user research, in other words: UX design.

If you’re new to the world of UX design but want to embrace it to improve your apps, we've created a new online course just for you. The UX Design for Mobile Developers course teaches you how to put your designer hat on, in addition to your developer hat, as you think about your apps' ideal user and how to meet their needs.

The course is divided into a series of lessons, each of which gives you practical takeaways that you can apply immediately to start seeing the benefits of good UX design.

Without jargon or buzzwords, the course teaches you where you should focus your attention, to bring in new users, keep existing users engaged, and increase your app's ratings. You'll learn how to optimize your app, rather than optimizing login/signup forms, and how to use low-resolution wireframing.

After you take the course, you'll "level up" from being an excellent developer to becoming an excellent design-minded developer.

Check out the video below to get a taste of what the course is like, and click through this short deck for an overview of the learning plan.

The full course materials — all the videos, quizzes, and forums — are available for free for all students by selecting “View Courseware”. Personalized ongoing feedback and guidance from Coaches is also available to anyone who chooses to enroll in Udacity’s guided program.

If that’s not enough, for even more about UX design from a developer's perspective, check out our YouTube UXD series, on the AndroidDevelopers channel: http://bit.ly/uxdplaylist.


Android Developers
at Udacity

Join the discussion on
+Android Developers


Categories: Programming

Agile 2014: My Retrospective

photo

The Agile 2014 Conference is much like a symphony with many individual movements that form a whole, like the Agile movement in general. 4 of the 5 days included 15 planned tracks with at least 4 presentations per track (Friday has a much smaller number of presentations). The variety of presentations and topics reflected the range of related, but different movements classified as Agile. Topics ranged from the social sciences of psychology and sociology targeted at coaching teams and organizational change to highly technically topics such as pair programming and DevOps.  The conference attracted 3,000+ attendees and speakers. For anyone interested in Agile this is conference that not allows but almost insists on an Agile immersion experience.

Several attendees that have attended in years past suggested that this year’s attendees reflect a subtle shift. There were more decision-makers than developers/practitioners in attendance. The theme of scaled Agile was also more prevalent than in past years.

Many of the early Agile thinkers were in attendance and involved not only in presenting, but shaping the conference. None of the early thinkers have lost their particular fire. For example, one speaker from this cohort publicly lamented that sponsors were monetizing the conference. Ken Schwaber even suggested turning the branded lanyards over to hide the branding. From my perspective, picking on the vendors is an easy target that could have negative consequences. The vibrant vendor community present was a positive and provided the sponsorship needed to hold great special events that made the conference more than just a fire hose of information. Agile 2014 was also a social experience.

If continual presentations and interactive workshops are not to your liking, there were plenty of less structured events. The conference had constant hallways talks, lean coffees every morning, Open Jams, coaching clinics and space provided for one-on-one conversations. There was no shortage of ideas, nor any shortage of space to discuss those ideas outside of the formal agenda.

This conference is large enough to hit some of your important topics one day and then to explore topics outside of your comfort zone the next and then have time to switch back and forth on the third, fourth or fifth days. I overhead folks discussing an interesting strategy; spending a day attending sessions that were exactly opposite of those you would typically attend to make sure you were exposed to new ideas.

Conversations with in the hall suggest that attendees that have been part of the Agile movement for more than a few years feel that innovation experimentation by Agile practitioners is slowing.  That the cycle of inspecting how Agile practices are performed by organizations and teams are being codified and in some cases being practiced as cannon and orthodoxy.  Hallway conversations suggest that the hardening of the rules that some groups are exhibiting is being noticed.  Leading voices are beginning to reemphasize learning and experimentation. One of the reflections of the slowing in evolution of Agile  is a suggestion from a few people that I chatted with that unless you are new or in the business of Agile consulting that attending the conference every two years and reading blogs is enough to keep current. I am not convinced that this suggestion is 100% accurate yet but it maybe soon unless innovation accelerates again.

There were very few negatives.  One downside worth mentioning was that many speakers that I went to see ran out of time. It seemed almost a badge of honor for speakers to announce they were out of time even though they were not done with the material they intended to cover. This included one of the two keynote speakers. Many of the offending parties I would consider professional speakers and know better. Not the end of the world, the ones I really want to know the rest of the story I will track down and interview on the podcast.

As Ernest Hemingway said, “It is good to have an end to journey toward; but it is the journey that matters, in the end.” Agile 2014 is part of my journey of enlightenment.


Categories: Process Management