Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
The story of driving the golden spike to symbolize the completion of the transcontinental railroad is famous in the US. What is not so well known is the story of how it also foreshadowed changes to come as an early version of both the Internet and the Internet of Things. But that was 1869, how can that possibly be?Telegraph as Internet
Iâ€™ll be in Ireland and the UK next week presenting at several events. Below are details on the talks Iâ€™ll be doing if you want to come along and hear them:Dublin: Monday Dec 2nd
Iâ€™m doing two separate free events in Dublin on Monday:
There is no content overlap between the two talks â€“ so feel free to attend both if you want to!London: Wed Dec 4th and 5th
Iâ€™m presenting at the NDC London Conference on Dec 4th and Dec 5th as well. This is a great developer conference being hosted in the UK for the first time. It has a great line up of speakers attending.
Iâ€™m presenting 2 separate two-part talks:
Hope to see some of you there!
P.S. In addition to blogging, I am also now using Twitter for quick updates and to share links. Follow me at: twitter.com/scottgu
Smaller user stories have significant benefits. So, why wouldnâ€™t we break down all stories?Â The law of diminishing returns is alive and well in the realm of story splitting. Generally, splitting stories is not a simple process, and therefore requires time, effort and mental capital. At some point the value from splitting a story falls below cost (both the effort and the opportunity cost of not doing something else). Size and testability are the leading filters for right sizing stories.
Stories must be small enough to be completed during a single iteration if we are going to deliver potentially releasable software at the end of each sprint. The story should be sized so that it does not require the whole team to complete. As described in Splitting User Stories: Why, small stories increase the flow of work through the team and when things go wrong, a single story wonâ€™t block the entire sprint. What is the right size for a story?Â It is my observation that all teams develop a norm that they find useful.Â As a rule of thumb I suggest that each story should be split until they can be completed in 2 to 3 days and require no more than 1/3 of the team at any one time.
Stories need to be testable within the sprintâ€™s time box.Â A reflection of the time it will take to develop and test any story is the number of acceptance criteria needed to prove a story is complete. A rule of thumb I typically suggest is that a story that requires more than 5 to 7 acceptance criteria should be split.Â A corollary, if groups of acceptance criteria are unrelated then the story should be split. For example, I recently reviewed a story whose acceptance criteria required that employees were able to review their time accounting data and that supervisors were able to review the time that had been entered.Â We split the story into two stories, the first for employee time review and the other story for the supervisorâ€™s review.
As I am splitting stories, I useÂ Bill Wakeâ€™s acronym INVEST as a framework when deciding if the split I have made makes sense.Â INVEST stands for:
When a story is split apply the implicit questions embedded in INVEST to make sure the result still makes sense.Â Can the stories standalone?Â Are they still valuable (if not, can part be jettisoned)? Are the resulting parts small enough to be estimated and completed in a sprint?Â Finally can we prove that we have solved the business problem that each story represents? If answer to all of INVEST questions are positive, we are done splitting and we can move on to the next story or, better yet, start developing.
If any of these items interest you there's a full description of each sponsor below. Please click to read more...
In order to solve real problems, maybe governments shouldÂ notÂ write yet another report.
One of my favorite articles ever about the Dutch governmentÂ (sorry, no link) provided an overview of the number of reports the government was producing about reducing the size of the government. No less than two pages (newspaper-style) were needed to list all the committees, workgroups, and task forces that had been trying to get a grip on the size of the government
One of the common problems when building data heavy applications is that names of things in the domain are often named differently depending on which system you get the data from.
This means that we’ll typically end up running the data from different sources through a normalisation process to ensure that we have consistent naming in the database:
I’ve recently started linking the football stadium a match was played in to the match in my football graph but unfortunately different match compilers use different spellings or even names for the same stadium.
My first instinct was to write a normalisation layer but instead I decided to store the stadium names as they were but separately create an ‘alias’ relationship back in to the correct spelling.
The model looks like this:
I add a ‘Stadium’ label to all the stadiums which means that if we do a query to find the number of games at each stadium we’ll get back games at the misspelled versions too:
MATCH (stadium:Stadium)<-[:in_stadium]-(game) RETURN stadium.name, count(game) AS games ORDER BY games DESC
==> +-------------------------------+ ==> | stadium.name | games | ==> +-------------------------------+ ==> | "Craven Cottage" | 57 | ==> | "Villa Park" | 57 | ==> | "Anfield" | 57 | ==> | "Stamford Bridge" | 57 | ==> | "Britannia Stadium" | 57 | ==> | "Emirates Stadium" | 57 | ==> | "Etihad Stadium" | 57 | ==> | "Stadium of Light" | 57 | ==> | "Old Trafford" | 57 | ==> | "The Hawthorns" | 57 | ==> | "White Hart Lane" | 57 | ==> | "Goodison Park" | 57 | ==> | "DW Stadium" | 39 | ==> | "Molineux Stadium" | 38 | ==> | "Liberty Stadium" | 38 | ==> | "Ewood Park" | 38 | ==> | "Carrow Road" | 38 | ==> | "Reebok Stadium" | 38 | ==> | "Loftus Road Stadium" | 37 | ==> | "St James' Park" | 34 | ==> | "Upton Park" | 33 | ==> | "Bloomfield Road" | 19 | ==> | "Madejski Stadium" | 19 | ==> | "St. James' Park" | 19 | ==> | "St Andrews Stadium" | 19 | ==> | "St. Mary's Stadium" | 19 | ==> | "The DW Stadium" | 18 | ==> | "Boleyn Ground" | 5 | ==> | "St. James Park" | 4 | ==> | "Loftus Road" | 1 | ==> +-------------------------------+ ==> 30 rows
We only want to show the proper spellings of stadiums which we can do with the following query:
MATCH (stadium:Stadium)<-[:alias_of*0..1]-()<-[:in_stadium]-(game) WHERE NOT(stadium-[:alias_of]->()) return stadium.name, count(game) AS games ORDER BY games DESC
Here we get all the stadiums, then get any incoming aliases and find the games played in those stadiums. We then filter out any stadium which has an outgoing ‘alias_of’ relationship because that indicates that the node isn’t the original node for the stadium.
If we run the query we’ll get the expected result:
==> +-------------------------------+ ==> | stadium.name | games | ==> +-------------------------------+ ==> | "St James' Park" | 57 | ==> | "Craven Cottage" | 57 | ==> | "Villa Park" | 57 | ==> | "Anfield" | 57 | ==> | "Stamford Bridge" | 57 | ==> | "Britannia Stadium" | 57 | ==> | "Emirates Stadium" | 57 | ==> | "Etihad Stadium" | 57 | ==> | "Stadium of Light" | 57 | ==> | "Old Trafford" | 57 | ==> | "The Hawthorns" | 57 | ==> | "White Hart Lane" | 57 | ==> | "The DW Stadium" | 57 | ==> | "Goodison Park" | 57 | ==> | "Loftus Road Stadium" | 38 | ==> | "Molineux Stadium" | 38 | ==> | "Upton Park" | 38 | ==> | "Liberty Stadium" | 38 | ==> | "Ewood Park" | 38 | ==> | "Carrow Road" | 38 | ==> | "Reebok Stadium" | 38 | ==> | "Bloomfield Road" | 19 | ==> | "Madejski Stadium" | 19 | ==> | "St Andrews Stadium" | 19 | ==> | "St. Mary's Stadium" | 19 | ==> +-------------------------------+ ==> 25 rows
The nice thing about this approach is that I only need to collect aliases in one place and I can just import the data as is from the source.
On the other hand it does add some complexity to queries as you need to take aliases into account each time.
I’d love to hear your thoughts on what you think of this approach.
When initially constructed, a user story can represent a concept that the team canâ€™t wrap their mind around and canâ€™t finish in an individual iteration. Therefore the user story needs to be broken down into smaller chunks.Â A user story (See User Stories, October 28, 2013) is a brief, simple requirement statement from the users perspective. The classic format of a user story is â€śAs a <persona>, I want to <goal> so that <benefit>.â€ť Large user stories have three main side problems that can be addressed by breaking them down.
Smaller stories are easier to understand, estimate and implement. When the team understands a story they will be better able to deliver the value inherent in the user story.Â The ability to understand is critical to reducing the possibility of surprises when they are developing the solution.Â Surprises, when they do happen, will have less of an impact on the team if the stories are small enough so the whole team is not stuck.Â Breaking down larger stories increases the through put and therefore increases the amount of value delivered.
The remainder of the week will be used to explore how to split stories.
Spring is right around the corner and it’s going to be fairly busy for me! I’m very exciting to take my talk arond the United States. If you’re getting started with mobile test automation, this is a great talk to attend. I’ve worked with dozens of customers to help them avoid typical pitfalls and get them on the right track. Mobile test automation is an exciting journey that will make a BIG difference for the quality of your mobile apps. But it is a true software project by itself that needs careful planning, dedication and some best practices to follow. This is what my talk is all about!
If you attend “Top Best Practices for Successful Mobile Test Automation“, you will learn:
This is my current engagements for the begining of the year. If you hear of any other great software testing conferences to attend, let me know!
In my first management role, I “managed” one person.Â My managee didn’t need much management. He guided me into how to manage him more than I managed him. He saved me from making too many mistakes. It was great practice for me.
Later in my management career, I managed a “team” of 15 testers. They were not a team. They were a group. I’m not sure why my management insisted on referring to them as a team, but my managers did. My role was to match-make testers with projects.
I didn’t think of my role as allocating people to projects, because I kept people on projects long-term. I didn’t fall for the management myth that I could move people like they were chess pieces. That’s why I thought of my role as match-making. I managed the project portfolio for the test group, so I could match-make.
What else did I do with this group?
I’m sure there was more, but that’s what I remember now.
I recently met a smart CTO of a company with about 100 engineers. He said he wanted a flat organization. Makes sense to me. Then he said this, “Every engineering manager should be able to manage about 15-20 engineers, and the projects that they work on, too.”
Oh, boy. You will notice that I was not managing projects in my list above. I was hiring like mad, in addition to the above management responsibilities, and I was full. I could not do any more. If you’d asked Mark, I bet he would have said I was past full. I did all my phone screens after dinner. When I needed to, I also wrote reports after dinner. I had no time during the day.
Could I have done any useful project management? No. Could I have managed any more people? No. Certainly not up to 20.Â Why? Because I needed time to meet with everyone, some people each week.
Why could I manage 14 people? Because I was an experienced manager by this time. I’d been practicing. First with one person. Then with three or four. Then seven, eight. When I had nine people in my group, I realized I had to move to biweekly one-on-ones for some people. I asked the people who were more senior if they minded. No, it was okay. But if they were more junior and needed coaching? It would have been a disaster.
And that is the topic of this month’s management myth: You Can Manage Any Number of People as a Manager. If you don’t care how well you manage, you cannot manage any number of people and do a great job.Â It’s the same principle as code or projects. If you don’t care how good the code is, you can write as much as you want. If it doesn’t have to work, it doesn’t matter. If you don’t care how good your project management is, you can manage as many as you want.
I’m not talking about micromanagement. I’m talking about providing coaching if the other person wants it, a learning environment for the people, a place for people to learn, a trusting relationship with each person every week. That’s it. I expected the people in my group to spend the rest of their time learning on their own and being responsible.
But because we were hiring, and because I had responsibilities across the organization, we were all busy. If I hadn’t made time for our one-on-ones, I could have gone for weeks, not seeing people. That would have been Wrong.
If you are managing more than nine people as a manager, rethink what you can do. If you are not having one-on-ones every week or every other week, what else are you doing?
Management is not about micromanagement. It’s about creating an environment in which everyone can do their best work. If you are too busy to do that, are you really managing?
InfiniSQL is the specific "Infinitely Scalable RDBMS" to which the title refers. It is free software, and instructions for getting, building, running and testing it are available in the guide. Benchmarking shows that an InfiniSQL cluster can handle over 500,000 complex transactions per second with over 100,000 simultaneous connections, all on twelve small servers. The methods used to test are documented, and the code is all available so that any practitioner can achieve similar results. There are two main characteristics which make InfiniSQL extraordinary:
There is a useful observation about the world that is often applied to software development called the Pareto principle or Pareto’s law.
This principle suggests that in many situations 80% of the results come from 20% of the causes.
For example, Pareto had realized that 80% of the land in Italy, during his time, was owned by 20% of the population.
Since then, many people, including Pareto himself, have applied the same observation of this 80-20 rule to many other areas of life from economics, to business and even software development.
But, I’ve found that we rely on Pareto’s law just a little bit too much.
The biggest problem I have with Pareto’s law is that is applied way too often to too many situations. In many cases, especially in software development, Pareto’s law becomes a self-fulfilling prophecyâ€”the more you look for Pareto’s law, the more you magically seem to find it.
None of this is to say that Pareto’s law isn’t a real thingâ€”of course it is. If you go and take a look at real hard numbers about distributions of things, you’ll find Pareto’s law all over the place in your statistical data. But, at the same time, if you go and look for the number 13, you’ll find an alarming number of occurrences of that two digit number above all others as well.
It is very tempting to force things that don’t quite fit generalizations into those generalizations. How often do we use the phrases “always” and “never?” How often do we fudge the data just a little bit so that it fits into that nice 80-20 distribution? 82% is close enough to 80 right? And of course 17.5% is close enough to just call it 20 after all.
Not only can you take just about any piece of data and make it fit into Pareto’s law by changing what you are measuring a little bit and fudging the numbers just a little if they are close enough, but you can also take just about any problem domain and easily, unconsciously, find the data points which will fit nicely into Pareto’s law. There is a good chance you are doing thisâ€”we all are. I do it myself all the time, but most of the time I am not aware of it.
I’ve found myself spouting off generalizations about data involving Pareto’s law without really having enough evidence to back up what I am saying. It is really easy to assume that some data will fit into Pareto’s law, because deep down inside I know I can make it fit if I have to.Seeing the world through Pareto colored glasses
You might think there is no harm in fudging numbers a bit and finding more and more places to apply Pareto’s law. But, looking at the world and blindly assuming all data falls into a distribution of 20 percent of the causes being responsible for 80 percent of the effects, is like walking around with blinders on your eyesâ€”you are only seeing part of reality and even the reality you are seeing tends to be a bit distorted.
Again, this doesn’t mean that Pareto’s law isn’t correct a large amount of the time, but it means that when you are just assuming that any data that appears to obey this law will, or worse yet, that all data MUST obey this law, you are severely limiting your perspective and restricting your options to those that already fit your preconceived ideals.
Sometimes I wish I had never heard of Pareto’s law, so that I wouldn’t be subject to this bias.
Let me give you a bit of a more concrete example.
Suppose you blindly assume that 80% of your application’s performance bottleneck comes from 20% of your code. In that case, you might be right, but you might also be wrong. It is entirely feasible that there are some parts of your code that contribute more or less to the performance of the application. It is also pretty likely that there are some bottlenecks or portions of code that heavily impact the performance of your application. But, if you go in with the assumption that the ratio is 80-20, you may spend an inordinate amount of time looking for a magical 20% that doesn’t exist instead of applying a more practical method of looking for what the actual performance problems are and then fixing them in order of impact.
The same applies for bugs or testing. If we blindly assume that 20% of the code generates 80% of the bugs, or that 20% of our tests test 80% of our system, we are making pretty large conclusions about how our software works that may or may not be correct. What happens when you fix all the bugs caused by the 20% of code that generates 80% of them? Does a new section of code now magically produce 80% of the bugs? If 20% of your test cases test 80% of your code, can’t you just create those ones? Why create another 80% to only test another 20%? And if you did follow that advice, then wouldn’t you have the situation where 100% of your tests tested 80% of your code?
The problem is when you start applying and assuming that Pareto’s law applies blindly, you start making all kinds of incorrect assumptions about data and only see what you expect.So, was Pareto wrong?
In short, no. He wasn’t wrong. Pareto’s principle is a thing. In general, in many cases, it is useful to observe that a small amount of causes are responsible for a majority of effects.
But, it is not useful to apply this pattern everywhere you can. The observation of the data should guide the conclusion and not the other way around.
I find it more useful, especially in software development, to ask the question “is it possible to find a small thing that will have a great effect?”
A good book on this very subject is The 4 Hour Chef. Although I don’t always agree with Tim Ferris, he is definitely the master of doing more with less and talks frequently about concepts like minimum effective dosages.
In other words, given a particular situation, can I find a small thing I can do, or change I can make, that will give me the biggest bang for my buck?
Sometimes the answer is actually “no.” Sometimes, no matter how hard we try, we just can’t find a minority that influences the majority. Sometimes the bugs are truly evenly distributed throughout the system. Sometimes the contributions of team members are fairly equal. One team member is not always responsible for 80% of the results.
And let’s not forget about synergy. Which basically is when 1 + 1 is equal to 3 or more. Sometimes the combination of things together makes the whole and separating out the parts at all greatly reduces the function.
For example: eggs, sugar, flour and butter can be used to make cake, and you could say that 80% of the “cakiness” comes from 20% of the ingredients, but if you leave one of those ingredients out, you’ll quickly find that 100% of them are necessary and it doesn’t even make sense to try and figure out which ingredient is most important, because alone each ingredient functions much differently than they do all together.
In software development this is especially true. Often in complex systems all kinds of interactions between different components of a system combine together to create a particular performance profile or bug distribution. Hunting for the magical 20% in many cases is as futile as saying eggs are responsible for 80% of the “cakiness” of cake.
The post Not Everything Is 80-20, Donâ€™t Blindly Follow Paretoâ€™s Law appeared first on Simple Programmer.
This is my third experiment with 15-minute interviews using Hangouts on Air.
I keep experimenting with 15-minute interviews. I like the concept, and I want to learn how to do these interviews better. This third time Maarten Volders was willing to help me along my learning curve.
In the video you can see that I have a new animated leader and closing animation. (Sound will be added next time.) You might also notice that I bought a better camera (a Logitech HD Pro C920)
â€śUntil weâ€™re educating every kid in a fantastic way, until every inner city is cleaned up, there is no shortage of things to do.â€ť â€“ Bill Gates
Bill Gates was my original inspiration for joining Microsoft. Here was a guy with all the money in the world that showed up everyday, and put in more hours than most people I know, to change the world.
He had ruthless focus on empowering people and building a better world.
Technology just happens to be his way.
Related to this, a funny thing happened a few weeks back. I was meeting a colleague I hadnâ€™t seen in a while. While I was waiting, I noticed a poster on his wall.
It was a poster of lessons learned from Bill Gates. It was 25 lessons learned from Bill Gates.
Lesson #18 caught my eye:Lesson #18. The toughest feedback to hear, is the feedback you need the most.
You get better by listening to your toughest critics. Your greatest source of growth can come from the people that will tell you what you need to hear, not just what you want to hear. Bill says, â€śYour most unhappy customers are your greatest source of learning.â€ť Bill also says, â€śYouâ€™ve got to want to be in this incredible feedback loop where you get the world-class people to tell you what youâ€™re doing wrong.â€ť
I started to read a few more of the lessons. #25 also caught my eye:25. Go digital.
Connect people, process, and technology. Create a digital landscape or a virtual world to reduce friction and to create new possibilities. Bill says, â€śOne of the wonderful things about the information highway is that virtual equity is far easier to achieve than real-world equityâ€¦We are all created equal in the virtual world and we can use this equality to help address some of the sociological problems that society has yet to solve in the physical world.â€ť
I had to ask my friend where he got the poster from. He told me from my site:
How funny was that?
My friend had formatted the poster so well, I didnâ€™t recognize my original post from long ago.
Anyway, I did a quick formatting sweep of my post, Lessons Learned from Bill Gates.
Hopefully, the lessons are easier to read now, and better emphasize the insight that Gates has shared with the world over his lifetime.
BTW â€“- if you have any favorite lessons from Bill Gates, feel free to share them.You Might Also Like
I recently spent two weeks in India. India is a land of extreme contrasts. Â As extreme as any of the examples is the Dharavi slum which sits within spitting distance of world-class architecture.Â As a business man marveling at Incredible India while sitting in an executive lounge, riding in the back seat of a car with a driver whisking me to appointments, eating at tables Â set aside for dignitaries and even overseen by random strangers while touring the slum it is hard to understand India. Â Being a visitor with handlers providing orthodox interpretations of the sights and sounds reinforces the fishbowl mentality and reduces the chances of deep understanding.
Software developers and other IT specialties can trap themselves into a fishbowl by latching onto a single set of ideas and then reinforcing those ideas by allowing gatekeepers of orthodoxy to constrain how they interpret what they read and hear. There are times within the IT community that ideas take on a life of their own.Â For example, the 1990â€™s marked the high water mark for CMMI and the concepts that revolved around the model.Â Adherents of the CMMI model became almost religious in their zeal to protect their boundaries, like the adherents of Object Orientation and Case.Â Â Each new community created its own fishbowl to supplant the last. The fishbowl ensured that the next new â€śthingâ€ť went unnoticed by those in the fishbowl.
Many Agile adherents have begun to build fishbowls of rules around frameworks, like Scrum or Kanban, suggesting that only a single orthodoxy exists. They are abandoning the culture of improvement and experimentation that spawned the Agile movement.Â To be truly small â€śaâ€ť agile, we must continuously look for ways to escape our fishbowl, to learn, to grow and to change.