Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator&page=8' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

Some Moral Hazards In Software Development


Too big to fail?

Moral hazards occur when the potential outcome of taking a risk is disassociated from who will bear the cost of a risk. ¬†Moral hazard is often caused by information asymmetry; the risk taker has more information than the person or organization that will bear the cost of a risk. Even though we assume in many cases perfect information or harp on the need for communication, information asymmetry is a common occurrence. Too big to fail is a form of moral hazard in which the organization may take larger risks with the potential the larger returns because they know they will not be allowed to fail. Another example of a scenario that can lead to a moral hazard is the principal‚Äďagent relationship. In the principal-agent relationship, the agent generally has more information than the principal. In this scenario, the principal can not spend all of his or her time monitoring the agent so that they have complementary or the same amount of information. There has to be a relationship of trust. In some scenarios, the agent might have the incentive to act inappropriately by taking more risk than the principal might deem appropriate. For example, the project manager position can often be construed as an agent-principal relationship. There are times that I’ve seen where a project manager who is behind will curtail testing so that they can catch up and deliver on time. This is a moral hazard.

There are numerous moral hazards that can occur in the software development and maintenance environments. Some of the scenarios that generate the potential for significant moral hazards are:

Too Important to Fail Projects: Information technology is littered with projects that are too important to fail. Participants on projects that are too big or too important to fail could assume that someone will step in if the project gets into trouble. For example, I have personally been involved with bank mergers with announced cutover dates multiple times in my career. ¬†If those dates were not met everything from the organization’s stock to the management team would have been sunk. I saw evidence of the impact of a missed date when a merger cutover had to be postponed due to an external event; the stock of both organizations plunged 50% in two days and the CIO and his staff were gone in less than 30 days. On more than one occasion decisions were made to beg for more resources or cut corners to make the dates when times got tough.

Software Teams Insulated From Business Risk: In many organizations, requirements are developed and provided by the business.  The requirements or user stories are provided to a development team as a transaction.  Once the team obtains the requirements they begin the process of development.  Because requirements are viewed as a transaction the team falls into a principal-agent trap where there is asymmetry in information.  The team can’t easily link their development knowledge with business risk.

Specialization: Separating some types of related work can generate the potential for moral hazard. In many organizations, development and maintenance functions are staffed separately.  The software is developed and then tossed over the wall to the maintenance personnel. In scenarios in which developers are not responsible for fixing defects, they may take shortcuts which benefit development, but not maintenance.  A similar argument can be made for separating planning and estimating from development functions.

The financial crisis of the last decade can be traced at least partially to information asymmetry and moral hazard. Innovations or continuous changes can impact information asymmetry.  Concepts such as Scrum can level the information playing field by embedding the business in day-to-day project decision making.  However, when non-business proxy product owners are substituted we end up back in the same position of potential moral hazard.

Next: Impact of Agile on Moral Hazards

Categories: Process Management

The Broad Range of Domains of Agile

Herding Cats - Glen Alleman - Thu, 07/07/2016 - 17:18

I work on Agile software development programs. Most everyone in Boulder Colorado works on Agile development programs. We meet once a month or so for coffee and talk about Agile. We have formal MeetUps hosted by local vendors - Rally, Scaled Agile, and other agile development shops.

The range of projects at our morning coffee clutch go from one man shows to multi-billion dollar space flight programs and back again. There are times when we get pissed off at each other for all the right reasons - this is the big ended littled ended argument of Gulliver's Travels that was actually an argument about the bit order in microprocessors when the 32 bit machines started with floating point calculations - Holy Wars and a Plea for Peace. When I was working on embedded systems and we had to choose a 32 bit machine for our new signal processing system, we got in huge fights about how the floating points software was going to be moved over to the floating point hardware.

So the question is what are the shared principles across a broad range of projects, business and technical domains. Here's some background on the domain I work and the application of Agile in that domain. I'm speaking at the Boulder Agile Meetup July 27th if anyone is interested in hearing about Agile in Government.

Here's some background on Agile in the domain I work

Typically these are also Software Intensive System of Systems. Here's some background on those

So when we hear something about exploring new approaches, ask first - in what domain have you tested that idea? By what Principles can that new idea be accepted into a domain outside your domain? Is there any evidence that your new idea can work outside your personal experience? How would I test your idea before spending my customer's money? What would be those test cases?

Categories: Project Management

The Misuse Hofstadter's Law

Herding Cats - Glen Alleman - Thu, 07/07/2016 - 02:35

The misuse of Hofstadter's Law is common in many agile development domains. The quote is ...

It always takes longer than you expect, even when you take into account Hofstadter's Law

This quote is misused to suggest that estimating can't be done. On page 152 of Gödel, Escher, Bach: an Eternal Golden Braid, Hofstadter explains the context and meaning of Hofstadter's Law.

He's speaking about the development of a Chess playing program - and doing so from the perspective of 1978 style software development. The game playing programs use a look ahead tree with branches of the moves and countermoves. The art of the program is to avoid exploring every branch of the look ahead tree down to the terminal nodes. In chess - actual chess, people - not the computer - have the skill to know what branches to look down and what branches to not look down. 

In the early days (before 1978) people used to estimate that it would be ten years until the computer was a world champion, But after ten years (1988) it was still estimated that day was still ten years away. 

This notion is part of the recursive Hofstadter's Law which is what the whole book is about. The principle of Recursion and Unpredictability is described at the bottom of page 152. 

For a set to be recursively enumerable (the condition to traverse the look ahead tree for all position moves), means it can be generated from a set of starting points (axioms), by the repeated application of rules of inference. Thus, the set grows and grows, each new element being compounded somehow out of previous elements, in a sort of mathematical snowball. But this is the essence of recursion - something being defined in terms of simpler versions of itself, instead of explicitly. 

Recursive enumeration is a process in which new things emerge from old things by fixed rules. There seem to be many surprises in such processes ...

So if you work on the development of recursive enumeration based software designs, then yes - estimating when you'll have your program working is likely going to be hard. Or if you work on the development of software that has no stated Capabilities, no Product Roadmap, no Release Plan, no Product Owner or Customer that may have even the slightest notion of what Done Looks like in units of measure meaningful to the decision makers, then probably you can apply Hofstadter's Law. Yourdan calls this type of project A Death March Project - good luck with that.

If not, then DO NOT fall prey to the misuse of Hofstadter's Law by those likely to not have actually read Hofstadter's book, nor have the skills and experience to understand the processes needed to produce credible estimates.

So once again, time to call BS, when quotes are misused

Related articles Agile Software Development in the DOD Risk Management is How Adults Manage Projects I Think You'll Find It's a Bit More Complicated Than That Architecture -Center ERP Systems in the Manufacturing Domain IT Risk Management Why Guessing is not Estimating and Estimating is not Guessing Making Conjectures Without Testable Outcomes Deadlines Always Matter Two Books in the Spectrum of Software Development Seven Principles of Agile Software Development in the US DOD
Categories: Project Management

Machine Learning Driven Programming: A New Programming for a New World


If Google were created from scratch today, much of it would be learned, not coded. Around 10% of Google's 25,000 developers are proficient in ML; it should be 100% -- Jeff Dean

Like the weather, everybody complains about programming, but nobody does anything about it. That’s changing and like an unexpected storm the change comes from an unexpected direction: Machine Learning / Deep Learning.

I know, you are tired of hearing about Deep Learning. Who isn’t by now? But programming has been stuck in a rut for a very long time and it's time we do something about it.

Lots of silly little programming wars continue to be fought that decide nothing. Functions vs objects; this language vs that language; this public cloud vs that public cloud vs this private cloud vs that ‘fill in the blank’; REST vs unrest; this byte level encoding vs some different one; this framework vs that framework; this methodology vs that methodology; bare metal vs containers vs VMs vs unikernels; monoliths vs microservices vs nanoservices; eventually consistent vs transactional; mutable vs immutable; DevOps vs NoOps vs SysOps; scale-up vs scale-out; centralized vs decentralized; single threaded vs massively parallel; sync vs async. And so on ad infinitum.

It’s all pretty much the same shite different day. We are just creating different ways of calling functions that we humans still have to write. The real power would be in getting a machine to write the functions. And that’s what Machine Learning can do, write functions for us. Machine Learning might just might be some different kind of shite for a different day.

Machine Learning Driven Programming
Categories: Architecture

Moral Licensing and Process Change

 Offering a feast of carrot sticks instead of chocolate chip cookies as a reward for committing to diet is probably not going to be well received.

Offering a feast of carrot sticks instead of chocolate chip cookies as a reward for committing to diet is probably not going to be well received.

Moral licensing is when¬†doing something that improves or strengthens our positive self-image makes us worry less about the implications of another ‚Äúimmoral‚ÄĚ behavior. This makes it more likely that we make ‚Äúimmoral‚ÄĚ choices. When discussing software development the word ‘immoral’ could be construed as over the top.¬† A better term might be overly convenient, ad-hoc or perhaps undisciplined.¬† An¬†understanding¬†of moral licensing is important. ¬†As change agents we are often have to understand¬†behavior so we can help teams and organizations deliver more value.¬†¬†For example¬†in some cases, it might be possible to confuse moral licensing with passive aggressive behavior. ¬†The remedies these different kinds of behavior are very different. Moral licensing is a real problem that that can disrupt progress in an agile transformation (or any other behavior change).

How many times have¬†you heard ‚ÄúI will reach out to the business and solicit a product owner for our next sprint; however for now . . .‚ÄĚ And then watched as a proxy product owner was assigned rather than someone from the business.¬†You can substitute, ‚Äúwe will start doing stand-up meetings tomorrow; however those meeting begin with¬†management assigning tasks‚ÄĚ or any number of long litany of good choices followed by a poor process choice. If we think of this a form of unconscious accounting, making the right choice provides positive capital, making it possible to spend some of that capital.¬† This is the same behavior you may have experienced the last time you made a New Year‚Äôs Resolution.¬† Have you ever committed to losing weight, to exercise or sleep more in the New Year but before starting went to one more big party or pulled one last all-nighter? ¬†Moral licensing is a bit of mental gymnastics that provides the mental cover for those choices.¬† Any act or promise to act¬†that is perceived to be ‚Äúgood,‚Ä̬† ‚Äúdisciplined,‚ÄĚ or ‚Äúmoral‚ÄĚ can license can illicit the feeling that a reward for being so righteous is deserved.

Wrestling with moral licensing in Agile or process changes begins with removing the good/ bad labels so that behavior/reward cycle is not triggered.  A second potential solution is to provide a set of choices that are all acceptable to help guide progress. This option can seem a bit manipulative if the pallet of choices are shallow or condescending.  Offering a feast of carrot sticks instead of chocolate chip cookies as a reward for committing to diet is probably not going to be well received.  A third choice for muting the possible impact of moral licensing is coaching. Coaches can help bring the choices that are being made out into the light so that practitioners consider their choices and the value of each.  In the end, we would like those we are helping to transform to not have to make choices between doing a stand-up or not doing a stand-up or between having a product owner from the business or a casual stand-in from the development team. 

Categories: Process Management

Mathematical Definitions - Again Misused to Convey Unsubstantiated Information

Herding Cats - Glen Alleman - Tue, 07/05/2016 - 21:58

1280px-Galileos_Dialogue_Title_PageIt's popular in the #NoEstimates community to claim Forecasting is not Estimating. Using English Dictionaries, they build a case using logical like this. It's been repeating nearly continuously since the start of the discussion about how to make decisions in the presence of uncertanty without estimating. 

Salviati: If you have a number of uniform sized tasks & know velocity you can predict ttc (Time to Complete) Simplicio responds: Indeed...gives us much stronger forecasting ability than estimation 

Let's use the Oxford Dictionary of Mathematics, not the High School English Dictionary.

It ain't so much the things we don't know that gets us in trouble. It's the things we know that ain't so - Artemus Ward


  • Point Estimate -¬†A single value of a statistic derived from a sample that is taken as an approximation for the population value of that statistic. For example, the sample mean , x , is a point estimate¬†of the population mean, őľ
  • Interval Estimate -¬†An inference made about the range of values within which a population parameter will lie, drawn from values obtained from a random sample.
  • Interval Estimate -¬†An interval within which a parameter under study (such as a relative risk ) is stated to lie with a particular degree of confidence, likelihood, or probability based on an analysis of a study or multiple studies. See also confidence interval.
  • Maximum Likelihood Estimate -¬†The value for an unknown parameter in a model that maximizes the probability of obtaining exactly the data that were observed.
  • Estimation is used to calculate an unknown value.
  • Estimation is the calculated approximation of a result.¬†


  • Forecast -¬†Also referred to as prediction. In econometrics, a point forecast is the expected value of the variable of interest, or the dependent variable, conditional on the given values of the exogenous and predetermined variables. An interval forecast is the confidence interval for the point¬†forecast.
  • Forecast -¬†To predict or estimate a future event or trend.
  • Forecast -¬†A statistical synthesis of probabilities and expert opinion that attempts to define an outcome either in terms of numbers or actual courses of action.
  • Sale Forecast -¬†An estimate of future sales volumes and revenue. It is usually based on past trends and takes into account current and future directions.
  • Forecasting -¬†prediction of future events.
  • Forecasting is the process of making a forecast or prediction. The terms forecast and prediction are often used interchangeably but sometimes forecasts are distinguished from predictions in that forecasts often provide explanations of the pathways to an outcome.
  • Forecasting -¬†to calculate or predict (some future event or condition) usually as a result of study and analysis of available pertinent data predict.

Estimating is about the past, present, and future outcomes of a process, a model, or some external observation. We can estimate the number leaves that have fallen and need to be raked come fall, to size how many bags have to be bought at Home Depot (past). We need to estimate the number of people on the Pearl Street Mall right now to assess how long the walk will be to Starbucks from our parking spot (present). We can estimate the number of minutes we'll need to reach our favorite parking spot at Denver International Airport from the office parking lot (future).

Forecasting is Estimating about the future. The weather forecast for tomorrow is 70 degrees and a 30% chance of rain in northern Boulder County (where we live). With this forecast, we can estimate what time we need to tee off to have a high probability of getting all 18 holes of golf on our course in before the rain starts.

Forecasting is estimating about the future

Related articles What does it mean when we say 80% confidence in a number? Taxonomy of Logical Fallacies All Project Numbers are Random Numbers - Act Accordingly How We Make Decisions is as Important as What We Decide. Who Builds a House without Drawings? Managing Projects By The Numbers Critical Success Factors of IT Forecasting Do The Math Humpty Dumpty and #NoEstimates
Categories: Project Management

Sponsored Post: Cassandra Summit, Gusto, LaunchDarkly, Awake Networks, Kinsta, Aerospike, InMemory.Net, VividCortex, MemSQL, Scalyr, AiScaler, AppDynamics, ManageEngine, Site24x7

Who's Hiring?
  • IT Security Engineering. At Gusto we are on a mission to create a world where work empowers a better life. As Gusto's IT Security Engineer you'll shape the future of IT security and compliance. We're looking for a strong IT technical lead to manage security audits and write and implement controls. You'll also focus on our employee, network, and endpoint posture. As Gusto's first IT Security Engineer, you will be able to build the security organization with direct impact to protecting PII and ePHI. Read more and apply here.

  • Awake Networks is an early stage network security and analytics startup that processes, analyzes, and stores billions of events at network speed. We help security teams respond to intrusions with super-human  efficiency and provide macroscopic and microscopic insight into the networks they defend. We're looking for folks that are excited about building systems that handle scale in a constrained environment. We have many open-ended problems to solve around stream-processing, distributed systems, machine learning, query processing, data modeling, and much more! Please check out our jobs page to learn more.

  • Software Engineer (DevOps). You are one of those rare engineers who loves to tinker with distributed systems at high scale. You know how to build these from scratch, and how to take a system that has reached a scalability limit and break through that barrier to new heights. You are a hands on doer, a code doctor, who loves to get something done the right way. You love designing clean APIs, data models, code structures and system architectures, but retain the humility to learn from others who see things differently. Apply to AppDynamics

  • Software Engineer (C++). You will be responsible for building everything from proof-of-concepts and usability prototypes to deployment- quality code. You should have at least 1+ years of experience developing C++ libraries and APIs, and be comfortable with daily code submissions, delivering projects in short time frames, multi-tasking, handling interrupts, and collaborating with team members. Apply to AppDynamics
Fun and Informative Events
  • Join database experts from companies like Apple, ING, Instagram, Netflix, and many more to hear about how Apache Cassandra changes how they build, deploy, and scale at Cassandra Summit 2016. This September in San Jose, California is your chance to network, get certified, and trained on the leading NoSQL, distributed database with an exclusive 20% off with  promo code - Academy20. Learn more at CassandraSummit.org

  • NoSQL Databases & Docker Containers: From Development to Deployment. What is Docker and why is it important to Developers, Admins and DevOps when they are using a NoSQL database? Find out in this on-demand webinar by Alvin Richards, VP of Product at Aerospike, the enterprise-grade NoSQL database. The video includes a demo showcasing the core Docker components (Machine, Engine, Swarm and Compose) and integration with Aerospike. See how much simpler Docker can make building and deploying multi-node, Aerospike-based applications!  
Cool Products and Services
  • Dev teams are using LaunchDarkly’s Feature Flags as a Service to get unprecedented control over feature launches. LaunchDarkly allows you to cleanly separate code deployment from rollout. We make it super easy to enable functionality for whoever you want, whenever you want. See how it works.

  • Kinsta provides high speed, automatically scalable managed WordPress hosting services for businesses large and small. All servers run on Google Cloud and all individual sites are completely compartmentalized using the latest LXD technology. All sites include powerful SSH access and tools like Git and WP-CLI are available out-of-the-box.

  • Scalyr is a lightning-fast log management and operational data platform.  It's a tool (actually, multiple tools) that your entire team will love.  Get visibility into your production issues without juggling multiple tabs and different services -- all of your logs, server metrics and alerts are in your browser and at your fingertips. .  Loved and used by teams at Codecademy, ReturnPath, Grab, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net

  • VividCortex measures your database servers’ work (queries), not just global counters. If you’re not monitoring query performance at a deep level, you’re missing opportunities to boost availability, turbocharge performance, ship better code faster, and ultimately delight more customers. VividCortex is a next-generation SaaS platform that helps you find and eliminate database performance problems at scale.

  • MemSQL provides a distributed in-memory database for high value data. It's designed to handle extreme data ingest and store the data for real-time, streaming and historical analysis using SQL. MemSQL also cost effectively supports both application and ad-hoc queries concurrently across all data. Start a free 30 day trial here: http://www.memsql.com/

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Also available on Amazon Web Services. Free instant trial, 2 hours of FREE deployment support, no sign-up required. http://aiscaler.com

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below...

Categories: Architecture

Reflecting on a Work Anniversary

I’ve been the technical editor for agileconnection.com for the past five years. It popped up to my LinkedIn network. Several people congratulated me on my work anniversary.

I have learned many things in the past five years:

  • Sometimes, people need “permission” to write what they feel. (They’re concerned they will be too bold, too loud, too something.)
  • Some people need help finding the “right”¬†structure for their writing. Sometimes, that structure is about how to find the time to write. Sometimes, that’s the article structure.
  • Some people need help learning what and how people read on the web.

The biggest thing I have learned is this:

If I tell people the results I need, they will then deliver those results.

It’s the same way on your projects, too. Tell people the results you want. I bet they will deliver those results, if at all possible.

I have asked these questions:

  • Can you tell a story here to illustrate your point?
  • Can you expand this bullet to tell the story? I am sure there is something quite interesting here.
  • I’m confused. Passive voice does that to me. Can you make this active?

I have more questions up my sleeve, and that’s fine.¬†Notice that I don’t “criticize” the writing. I don’t like criticism. I much prefer knowing what to do to improve. I bet you do, too. That’s why¬†I ask for what I want.

If you use agile and have a story to tell, I’m interested. Let me help you publish your story. Send me an email.

If you would like to write better, let me know if you would like to be a part of my next non-fiction writing workshop. I have a wait list for the August workshop. I’ll definitely run it again.

I thank you for all your good wishes, and I do hope I can continue this (part-time) gig. It’s quite fun!

Categories: Project Management

The 2 Things You Need to Do in Daily Scrums to End Complaints

Mike Cohn's Blog - Tue, 07/05/2016 - 15:00

You have undoubtedly had team members complain at some point about the length of your daily scrums. Join the club.

I want to share two extremely simple things you can do to put an end to most of those complaints. (I can’t promise getting rid of all complaints—some people will always complain.)

Because of the Scrum guideline that a daily scrum is not to be used for problem solving, many Scrum Masters will note the problem during the daily scrum and then discuss it immediately afterwards.

For example, if two hot issues are mentioned during the daily scrum, the Scrum Master might stop discussion of those topics. The Scrum Master will then bring the two topics back up after everyone has first had a chance to address the three questions of the daily scrum.

This leads to the team being there for longer than the standard timebox of 15 minutes for a daily scrum.

The first thing you should do is to end every daily scrum by announcing how long the meeting took. Do this right after everyone has addressed the three questions and before switching into problem-solving mode.

You might, for example, announce, “Thanks everyone. That took twelve minutes.”

But then, remind everyone of any problems or issues that were brought up. Suggest that those who are needed stay to discuss or resolve them. If possible, facilitate the team splitting into more than one subgroup if more than one issue needs to be discussed.

Then, do the second thing that helps end complaints about the length of the daily scrum: Announce that those who are not needed to resolve any of the issues being discussed can leave.

By taking these two actions:

1. Calling the daily scrum officially over and announcing how many minutes it took; and,

2. Telling team members who are not needed for further discussions that they can leave

You will help team members from feeling that the daily scrum is exceeding its 15-minute timebox.

What Do You Think?

What have you done to eliminate complaints about the daily scrum? How have you helped your team see the value of this meeting? Please share your thoughts in the comments below.

July 4th 2016

Herding Cats - Glen Alleman - Mon, 07/04/2016 - 15:09


Categories: Project Management


NOOP.NL - Jurgen Appelo - Mon, 07/04/2016 - 11:50

I recently enjoyed a chat with Doug Kirkpatrick, one of the advocates of self-management, and I had a great meeting with Harvard Business Review author Ed Batista, who is working on a book about self-coaching.

The words that such authors and experts use fascinate me. As a complexity thinker, I am already well aware of the concept of self-organization. And in the last year or so, I’ve had more than a few discussions with people about the topics of self-development and self-education.

This made me think.

The most frequently asked questions in business are all about changing other people. And management, coaching, organization, development and education are, by default, things we do unto others. The prefix self appears to be the special case.


“Be the change you wish you see” is a famous quote, usually attributed to Gandhi. Gandhi felt the need to emphasize that changes in the world must start with ourselves. It seems he also noticed that most people mainly seek to change others.

Perhaps this is one reason why I’ve never been very interested in doing any coaching or consulting. As I always say, I have too many problems of my own, so I don’t have time to deal with those of others.

However, I love to write and speak about my problems and how I’ve tried to solve them. And with our company Happy Melly, my team and I seek to be the change that we wish to see in others. What we do is mainly self-management, self-coaching, and self-development. The result is a never-ending stream of experiments that, hopefully, are an inspiration to many. So they can change themselves too.

photo (c) 2008, Philo Nordlund, Creative Commons 2.0

The post Self-Change appeared first on NOOP.NL.

Categories: Project Management

SPaMCAST 401 ‚Äď Listening, Quality, Testing and Contract Closure, Developers and Testing



Listen Now

Subscribe on iTunes                   Check out the podcast on Google Play Music

The Software Process and Measurement Cast 401 features three columns!  We begin with our essay on listening.  Every time we answer the phone, interact with a co-worker or even turn on the television we need to hear and interpret the messages that are being sent. Our complicated business and life environments impact how we listen through the situations we face. Listening is important. Like reading, it is fundamental to almost every activity needed to build, enhance or maintain a product; therefore, learning and understanding how to listen, and as importantly how not to listen, are table stakes for getting anything done!

Jon M. Quigley’s second column discusses the topic of cost, quality, testing and contract closure. All of the parts of a product have to fit together for everyone to feel comfortable and pay the bill!

Jeremy Berriault and the QA Corner anchor the cast.  I asked Jeremy to talk about whether developers should test.  (Don’t tell anyone, but the answer is HECK YES.)

Re-Read Saturday News

This week we continue the Re-read Saturday of  Kent Beck’s XP Explained, Second Edition with a discussion of Chapters 4 and 5.  We do a deep dive into values and principles.  Values and principles are the basis for all of the practices we will explore as we read.  Use the link to XP Explained in the show notes when you buy your copy to read along to support both the blog and podcast. Visit the Software Process and Measurement Blog (www.tcagley.wordpress.com) to catch up on past installments of Re-Read Saturday.


The next Software Process and Measurement Cast will feature our interview with Ulises Torres.  Ulises and I talked about how his firm, Intellego, has leveraged Agile and the CMMI to improve quality, increase customer satisfaction and business.  

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: ‚ÄúThis book will prove that software projects should not be a tedious process, for you or your team.‚ÄĚ Support SPaMCAST by buying the book here. Available in English and Chinese.

Categories: Process Management

SPaMCAST 401 - Listening, Quality, Testing and Contract Closure, Developers and Testing

Software Process and Measurement Cast - Sun, 07/03/2016 - 22:00

The Software Process and Measurement Cast 401 features three columns!  We begin with our essay on listening.  Every time we answer the phone, interact with a co-worker or even turn on the television we need to hear and interpret the messages that are being sent. Our complicated business and life environments impact how we listen through the situations we face. Listening is important. Like reading, it is fundamental to almost every activity needed to build, enhance or maintain a product; therefore, learning and understanding how to listen, and as importantly how not to listen, are table stakes for getting anything done!

Jon M. Quigley’s second column discusses the topic of cost, quality, testing and contract closure. All of the parts of a product have to fit together for everyone to feel comfortable and pay the bill!

Jeremy Berriault and the QA Corner anchor the cast.  I asked Jeremy to talk about whether developers should test.  (Don’t tell anyone, but the answer is HECK YES.)


Re-Read Saturday News

This week we continue the Re-read Saturday of  Kent Beck’s XP Explained, Second Edition with a discussion of Chapters 4 and 5.  We do a deep dive into values and principles.  Values and principles are the basis for all of the practices we will explore as we read.  Use the link to XP Explained in the show notes when you buy your copy to read along to support both the blog and podcast. Visit the Software Process and Measurement Blog (www.tcagley.wordpress.com) to catch up on past installments of Re-Read Saturday.


The next Software Process and Measurement Cast will feature our interview with Ulises Torres.  Ulises and I talked about how his firm, Intellego, has leveraged Agile and the CMMI to improve quality, increase customer satisfaction and business.  

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: ‚ÄúThis book will prove that software projects should not be a tedious process, for you or your team.‚ÄĚ Support SPaMCAST by buying the book here. Available in English and Chinese.


Categories: Process Management

Looking for developers to join my group

I’ve recently moved to a new position as CTO of a stealth mode company – so while I can’t say much about the company over the internet I am looking for few developers to join our team. The positions are located in Herzeliya Israel

Below are profiles for senior backend and senior frontend developers

Please contact me for more details (you can also ping my twitter or via linkedin)

Senior Backend

Position Summary:  

We are searching for a Senior Backend Developer to join our fast-growing tech start up and perform as a valuable team player in our backend team.

What will you need to succeed?

  • 7+ years of experience in a developer role for software products
  • B.Sc.\B.A. in computer science ‚Äď An advantage.
  • Strong problem solving and coding skills ‚Äď A must.
  • Experience with Scala/Clojure/Haskell or other functional language ‚Äď A must.
  • Polyglot developer – ¬†A plus
  • Experience with Spark, Kafka an advantage.
  • Experience with Docker, Mesos, Kubernetes an advantage,
  • Experience with ElasticSearch/SolR, Neo4J/Titan an advantage.
  • Experience with working in an agile SCRUM/Kanban methodology ‚Äď An advantage.
  • Good understanding of Distributed systems theory (CAP, FLP, Lamport Clocks, CRDTs, Paxos etc.) A big plus
Senior Frontend

Position Summary:  

We are searching for a Senior Front-end Developer to join our fast-growing tech start up and perform as a valuable team player in our front-end team.

What will you need to succeed?

  • 5+ years of experience in a developer role for software products
  • B.Sc.\B.A. in computer science ‚Äď An advantage.
  • Strong problem solving and coding skills ‚Äď A must.
  • Extensive experience with javascript, specifically NodeJS ¬†¬†‚Äď A must.
  • Experience with html, css a must
  • Experience with sass/less – an advantage.
  • Good knowledge of ReactJS – a must.
  • Good knowledge of Redux – an advantage.
  • Experience working with non-relational databases (MongoDB\Cassandra\Couchbase) ‚Äď An advantage.
  • Experience with working in an agile SCRUM/Kanban methodology ‚Äď An advantage.


For both positions Рyou’ll most enjoy our company culture if you’re:

  • Self motivated, team player, action and results oriented
  • Eagerness to learn
  • Well organized with good communication and reporting skills
  • Has an ability to successfully work under tight project deadlines
  • Willing and able to meet a challenge head-on, solve problems independently and make things happen.Open minded, flexible and thrive in a highly dynamic, fast-paced, ever-changing environment.
Categories: Architecture

Verbal Aikido for Product Managers

Xebia Blog - Sun, 07/03/2016 - 13:41
"Well eh ok, I guess so" mumbled the student in the training exercise where he was practicing how to say no to feature gluttony. I decided to give the class an additional exercise to awaken their inner diplomat. “Diplomacy is the art of telling people to go to hell in such a way that they

Extreme Programming Explained, Second Edition: Week 3


XP Explained

This week we continue with the building blocks of Extreme Programming by tackling chapters four and five in Kent Beck’s Extreme Programing Explained, Second Edition (2005). Chapters four and five provide a deep dive into values and principles.  Why, when we talk about frameworks or methodologies, do we spend so much time on values and principles?  Simply put, values and principles provide a rationale for the practices that are the nuts and bolts of a methodology. Without an understanding of why we do something, it will be difficult to hold on to a practice or to understand when and why to tweak a practice when things get tough.
Chapter 4: Values

Values represent a judgment of what is important. What we think is important directly impacts how we behave. XP‚Äôs values guide practitioners actions and to reduce waste through the coordination of activity. Beck points out that the difference between what is perceived as being valuable and what really is¬†valuable ¬†creates waste. The five values of XP are designed to focus thought and effort on what’s important to the team.¬† The five values of XP are:

  1. Communication

Communication is important for creating a sense of team and effective cooperation.  Communication also spreads knowledge, which keeps everyone on the same path and improves the team’s capability to deliver value. Communication is the most important of the five values.

  1. Simplicity

What is the simplest thing that could possibly work? A focus on simplicity allows a team to deliver quickly and get feedback so they can stay on track and build trust.  Simplicity exists in context; what is simple in one situation is not simple in another.  Knowledge and communication help define what is simple.

  1. Feedback

In business, software development or a family road trip, absolutes that define direction, requirements or architecture remain don’t valid for long. Change is inevitable. Without feedback, any project or effort will go off track. Beck suggests that teams use feedback to get closer to our goals rather than expecting instant perfection. Simplicity makes it easier to get feedback.

  1. Courage

Courage is taking effective action in the face of fear. Software development is an activity that is intrinsically scary.  Every time a developer, tester or business analysts begins work, they are pitting their wits against a problem that stretches their capabilities.  Without courage, it is hard to push the limits and tackle problems that we might not be able to surmount.  Lack of courage causes paralysis; however, blind courage yields recklessness.  Courage needs to be balanced with other values, such as feedback and communication, to provide balance.

  1. Respect

Software development and XP are team sports. Team members must care about each other and what they’re doing or XP won’t work. Respect reflects a judgment of value on a team no one is intrinsically worth more than any other team member regardless of the roles that are played.

All five of the values in XP are interrelated and provide reinforcement.   Beck points out that teams and organizations can choose other values or even add to the set. However, values need to be held across the team or organization.  Maintaining multiple sets of values simultaneously will cause conflict and waste.  Values provide the direction that helps teams and organizations coordinate their activities and achieve more than a bunch of individuals.


Chapter 5: Principles

Values, like high-level goals, are abstract.   They provide general guidance but are too abstract to guide behavior directly.  Principles act as a bridge between tactical behavior and the high-level vision of behavior built into values.  XP includes 14 principles:

  1. Humanity

Software development is a people-based activity.¬† Beck states that processes and practices of software development need to ‚Äúaddress human needs, acknowledge human frailty, and leverage human strengths.‚Ä̬† Humanity also requires finding the balance between the needs of individuals and needs of the team (sounds like something from Star Trek).

  1. Economics

Software development and maintenance must have value, meet business goals and needs or no one will pay for it.

  1. Mutual benefit

Every activity should benefit all concerned, i.e. win-win solutions both in the future and now.

  1. Self-similarity

Patterns are a tool that humans have always used for efficiency and safety.  Most cognitive biases (shortcuts) are a reflection of the principle of self-similarity.  We find a shape or pattern that works and then use it as a place to start.  Software development is no different.

  1. Improvement

Perfect is a verb not an adjective. Beck suggests, “Find a starting place, get started, and improve from there.” Retrospectives are an improvement opportunity.

  1. Diversity

Teams rarely are effective if they have a homogenous set of skills, attitudes, and perspectives.  Teams with a diversity of capabilities are able see the problems and pitfalls required to solve and to implement solutions. Teams need people with a variety of skills and perspectives to reach maximum effectiveness.

  1. Reflection

Excellence requires that teams not only execute, but think about how work is done and why they are working. Reflection can be done at any time and include both reviewing data as well as thinking.

  1. Flow

Deliver a steady flow of valuable software.  Problems are more effectively solved incrementally versus a big bang approach.  I have always suggested that if a team is having trouble meeting its goals that they take smaller bites (shorten the increment they are using and move closer to a continuous flow of delivery).

  1. Opportunity

To reach excellence, problems need to be turned into opportunities for learning and improvement.  Problems are often addressed as survival events, which rarely translate into increased capabilities.  Think of turning lemons into lemonade.

  1. Redundancy

The critical, difficult problems and software development should be solved in several different ways. Said differently, create scenarios where you can test your options and choose the best one.

  1. Failure

Experiment and learn so that you can get the data needed to decide which option will better provide a path for success.  Failure provides value it if it imparts knowledge (assuming we failed because we did not know better).

  1. Quality

One of the things I learned from working with Capers Jones was that quality saves time, money and builds trust.  Focusing on quality does not mean a team should not try new things, fail sometimes, learn, and get better.

  1. Baby steps

Incremental delivery of value generates feedback, is less dangerous and creates momentum. From a practical point of view, baby steps reduce the amount of overhead required to make any change.

  1. Accept responsibility

Responsibility can only be accepted, not assigned. When responsibility and authority are not aligned people often act in a passive aggressive manner.  The classic example is the activity of estimating. If the people doing the work are assigned an estimate they are far less likely to act as if they own the estimate.

The principles provide a framework to understand the practices that are introduced in Chapter 6.  Understanding values and principles allow the practitioners of XP to understand how they can apply and tailor the practices to meet their specific contexts.


Previously on Re-read Saturday: Extreme Programing Explained, Second Edition (2005) on Re-read Saturday:

Extreme Programming Explained: Embrace Change Second Edition Week 1, Preface and Chapter 1

Week 2, Chapters 2 Р3 

Categories: Process Management

More tips and tricks for running Gatling in Docker containers

Agile Testing - Grig Gheorghiu - Fri, 07/01/2016 - 18:11
This post is a continuation of my previous one on "Running Gatling tests in Docker containers via Jenkins". As I continued to set up Jenkins jobs to run Gatling tests, I found the need to separate those tests for different environments - development, staging and production. The initial example I showed contained a single setup, which is not suitable for multiple environments.

Here is my updated Gatling directory structure


Note that I created a separate directory under simulations for each environment (development, staging, production), each with its own simulation files.

I also created a data directory under user-files, because that is the default location for CSV files used by Gatling feeders.

Most importantly, I created a separate configuration directory (staging, production) under gatling/conf, each directory containing its own customized gatling.conf file. I started by copying the gatling-defaults.conf file from GitHub to gatling/conf/staging/gatling.conf and gatling/conf/production/gatling.conf respectively.

Here is what I customized in staging/gatling.conf:

mute = true # When set to true, don't ask for simulation name nor run description
simulations = user-files/simulations/staging

I customized production/gatling.conf in a similar way:

mute = true # When set to true, don't ask for simulation name nor run description
simulations = user-files/simulations/production

Setting mute to true is important because without it, running Gatling in a Docker container was segfaulting while waiting for user input for the simulation ID:

Select simulation id (default is 'gatlingsimulation'). Accepted characters are a-z, A-Z, 0-9, - and _ Exception in thread "main" java.lang.NullPointerException at io.gatling.app.Selection$Selector.loop$1(Selection.scala:127) at io.gatling.app.Selection$Selector.askSimulationId(Selection.scala:135) at io.gatling.app.Selection$Selector.selection(Selection.scala:50) at io.gatling.app.Selection$.apply(Selection.scala:33) at io.gatling.app.Gatling.runIfNecessary(Gatling.scala:75) at io.gatling.app.Gatling.start(Gatling.scala:65) at io.gatling.app.Gatling$.start(Gatling.scala:57) at io.gatling.app.Gatling$.fromArgs(Gatling.scala:49) at io.gatling.app.Gatling$.main(Gatling.scala:43) at io.gatling.app.Gatling.main(Gatling.scala)

The other customization was to point the simulations attribute to the specific staging or production sub-directories.
Since the CSV files containing URLs to be load tested are also environment-specific, I modified the Simulation.scala files to take this into account. I also added 2 JAVA_OPTS variables that can be passed at runtime for HTTP basic authentication. Here is the new Crawl object (compare with the one from my previous post):
object Crawl {  val feeder = csv("staging-urls.csv").random
  val userName = System.getProperty("username")  val userPass = System.getProperty("password")
  val crawl = exec(feed(feeder)    .exec(http("${loc}")    .get("${loc}").basicAuth(userName, userPass)    ))}
One more thing is needed: to make Gatling use a specific configuration file instead of its default one, which is conf/gatling.conf. To do that, I set GATLING_CONF as an ENV variable in the Dockerfile, so it can be passed as a 'docker run' command line parameter. Here is the Dockerfile:
# Gatling is a highly capable load testing tool.## Documentation: http://gatling.io/docs/2.2.2/# Cheat sheet: http://gatling.io/#/cheat-sheet/2.2.2
FROM java:8-jdk-alpine
MAINTAINER Denis Vazhenin
# working directory for gatlingWORKDIR /opt
# gating versionENV GATLING_VERSION 2.2.2
# create directory for gatling installRUN mkdir -p gatling
# install gatlingRUN apk add --update wget && \  mkdir -p /tmp/downloads && \  wget -q -O /tmp/downloads/gatling-$GATLING_VERSION.zip \  https://repo1.maven.org/maven2/io/gatling/highcharts/gatling-charts-highcharts-bundle/$GATLING_VERSION/gatling-charts-highcharts-bundle-$GATLING_VERSION-bundle.zip && \  mkdir -p /tmp/archive && cd /tmp/archive && \  unzip /tmp/downloads/gatling-$GATLING_VERSION.zip && \  mv /tmp/archive/gatling-charts-highcharts-bundle-$GATLING_VERSION/* /opt/gatling/
# change context to gatling directoryWORKDIR  /opt/gatling
# set directories below to be mountable from hostVOLUME ["/opt/gatling/conf", "/opt/gatling/results", "/opt/gatling/user-files"]
# set environment variablesENV PATH /opt/gatling/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binENV GATLING_HOME /opt/gatlingENV GATLING_CONF /opt/gatling/confENV JAVA_OPTS ""
ENTRYPOINT ["gatling.sh"]
Finally, here is how I invoke 'docker run' to tie everything together:
docker run --rm -v ${WORKSPACE}/gatling/conf:/opt/gatling/conf -v ${WORKSPACE}/gatling/user-files:/opt/gatling/user-files -v ${WORKSPACE}/gatling/results:/opt/gatling/results -e GATLING_CONF="/opt/gatling/conf/staging" -e JAVA_OPTS="-Dusers=$USERS -Dduration=$DURATION -Dusername=myusername -Dpassword=mypass" /PATH/TO/DOCKER/REGISTRY/gatling
Note the GATLING_CONF parameter passed with -e with the value of /opt/gatling/conf/staging. Also note the username and password JAVA_OPTS parameters.

Happy load testing!

Building for Billions

Google Code Blog - Fri, 07/01/2016 - 17:37

Originally posted on Android Developers blog

Posted by Sam Dutton, Ankur Kotwal, Developer Advocates; Liz Yepsen, Program Manager


These are common warnings for many smartphone users around the world.

To build products that work for billions of users, developers must address key challenges: limited or intermittent connectivity, device compatibility, varying screen sizes, high data costs, short-lived batteries. We first presented developers.google.com/billionsand related Android and Web resources at Google I/O last month, and today you can watch the video presentations about Android or the Web.

These best practices can help developers reach billions by delivering exceptional performance across a range of connections, data plans, and devices. g.co/dev/billions will help you:

Seamlessly transition between slow, intermediate, and offline environments

Your users move from place to place, from speedy wireless to patchy or expensive data. Manage these transitions by storing data, queueing requests, optimizing image handling, and performing core functions entirely offline.

Provide the right content for the right context

Keep context in mind - how and where do your users consume your content? Selecting text and media that works well across different viewport sizes, keeping text short (for scrolling on the go), providing a simple UI that doesn’t distract from content, and removing redundant content can all increase perception of your app’s quality while giving real performance gains like reduced data transfer. Once these practices are in place, localization options can grow audience reach and increase engagement.

Optimize for mobile hardware

Ensure your app or Web content is served and runs well for your widest possible addressable market, covering all actively used OS versions, while still following best practices, by testing on virtual or actual devices in target markets. Native Android apps should set minimum and target SDKs. Also, remember low cost phones have smaller amounts of RAM; apps should therefore adjust usage accordingly and minimize background running. For in-depth information on minimizing APK size, check out this series of Medium posts. On the Web, optimize JavaScript CPU usage, avoid raster image rendering, and minimize resource requests. Find out more here.

Reduce battery consumption

Low cost phones usually have shorter battery life. Users are sensitive to battery consumption levels and excessive consumption can lead to a high uninstall rate or avoidance of your site. Benchmark your battery usage against sessions on other pages or apps, or using tools such as Battery Historian, and avoid long-running processes which drain batteries.

Conserve data usage

Whatever you‚Äôre building, conserve data usage in three simple steps: understand loading requirements, reduce the amount of data required for interaction, and streamline navigation so users get what they want quickly. Conserving data on behalf of your users (and with native apps, offering configurable network usage) helps retain data-sensitive users -- especially those on prepaid plans or contracts with limited data -- as even ‚Äúunlimited‚ÄĚ plans can become expensive when roaming or if unexpected fees are applied.

Have another insight, or a success launching in low-connectivity conditions or on low-cost devices? Let us know on our G+ post.

Categories: Programming

Stuff The Internet Says On Scalability For July 1st, 2016

Hey, it's HighScalability time:

If you can't explain it with Legos then you don't really understand it.


If you like this sort of Stuff then please support me on Patreon.

  • 700 trillion: more pixels in Google's Satellite Map; 9,000km: length of new undersea internet cable from Oregon to Japan; 60 terabits per second: that undersea internet cable again; 12%: global average connection speed increase; 76%: WeChat users who spend more than 100RMB ($15) per month; 5 liters: per day pay in beer for Pyramid workers;  680: number of rubber bands it takes to explode a watermelon; 1,000: new Amazon services this year; $15 billion: amount Uber has raised; 7 million: # of feather on on each bird in Piper; 5.8 million: square-feet in Tesla Gigafactory; 2x: full-duplex chip could double phone-network data capacity; 

  • Quotable Quotes:
    • @hyc_symas: A shame everyone is implementing on top of HTTP today. Contemporary "protocol design" is a sick joke.
    • @f3ew: Wehkamp lost dev and accept environments 5 days before launch. Shit happens.  48 hours to recovery. #devopsdays
    • Greg Linden: Ultimately, [serverless computing] this is a good thing, making compute more efficient by allowing more overlapping workloads and making it easier to move compute around. But it does seem like compute on demand could cannibalize renting VMs.
    • @viktorklang: What if we started doing only single-core chips with massive eDRAM on-package and PCI-E peer-writes MPI between? (Micro-blade machines?)
    • Robert Graham: Programmers fetishize editors. Real programmers, those who produce a lot of code, fetishize debuggers
    • @jasonhand: "Systems are becoming more ephemeral and we have to find a way to deal with it" (regarding monitoring) - @adrianco #monitorama
    • @aphyr: Queues *can* improve fault tolerance, but this only happens if they don't lose your messages. The only one I know of that doesn't is Kafka.
    • Tom Simon: There are, accordingly, two ways of reading books; but infinitely many ways to divide up the act of reading into two classes.
    • Puppet: High-performing IT organizations deploy 200 times more frequently than low performers, with 2,555 times faster lead times.
    • @benzobot: “The system scaled with the number of engineers - more engineers, more metrics.” #monitorama
    • fizx: You seem to have installation confused with administration. Off the top of my head you forgot security, monitoring, logging config, backups, handling common production issues such as splitbrains, write multiplication, garbage collection snafus, upgrades between versions with questionably compatible internal apis.
    • @Mark_J_Perry: This has to be one of the most remarkable achievements ever: Global Poverty Fell Below 10% for 1st Time in 2015
    • ewams: If you are a services company, he is right, you should be focusing on outcomes. But, if you can't tell me in 2-3 sentences what problem you are solving and how it benefits the customer you are doing it wrong.
    • @retrohack3r: Dance like nobody is watching. Encrypt like everyone is.
    • @GundersenMarius: ES6 + HTTP/2 + Service Workers + Bloom-filter = efficient module loading without bundlin mariusgundersen.net/module-pusher/ 
    • @timperrett: Distributed systems are about protocols, not implementations. Forget languages, protocols are everything.
    • steveblank: What’s holding large companies back?...companies bought into the false premise that they exist to maximize shareholder value – which said “keep the stock price high.” As a consequence, corporations used metrics like return on net assets (RONA), return on capital deployed, and internal rate of return (IRR) to measure efficiency. These metrics make it difficult for a company that wants to invest in long-term innovation.
    • Greg Linden: Like a lot of things at Amazon, this went through many stages of wrong before we got it right. As I remember it, this went through some unpleasant, heavyweight, and inefficient RPC (esp. CORBA) and pub-sub architectures before an unsanctioned skunkworks project built iquitos for lightweight http-based microservices 
    • @rawrthis:  "We all die." Except my legacy stack. That crap will live forever. #devopsdays
    • Jim Handy: The industry already has more than enough DRAM wafer capacity for the foreseeable future. Why is this happening?  The answer is relatively simple: the gigabytes per wafer on a DRAM wafer are growing faster than the market’s demand for gigabytes.
    • SwellJoe: A container doesn't consume complexity and emit order. The complexity is still in there; you still have to build your containers in a way that is replicable, reliable, and automatable. I'm not necessarily saying configuration management is the only way to address that complexity, but it does have to be addressed in a container-based environment.

  • Deep Learning desires data, so if you want to build an AI that learns how to program this is how you would go about it, you would bring all the open source code into your giant, voracious, data crunching maw. Making open source data more available: Today, [GitHub] we're delighted to announce that, in collaboration with Google, we are releasing a collection of additional BigQuery tables to expand on the GitHub Archive...This 3TB+ dataset comprises the largest released source of GitHub activity to date. It contains activity data for more than 2.8 million open source GitHub repositories including more than 145 million unique commits, over 2 billion different file paths.

  • In almost all cases, the single-threaded implementation outperforms all the others, sometimes by an order of magnitude, despite the distributed systems using 16-128 cores. Scalability! But at what COST? The paper is, at its heart, a criticism of how the performance of current research systems are evaluated. The authors focus on the field of graph processing, but their arguments extend to most distributed computation research where performance is a key factor. They observe that most systems are currently evaluated in terms of their scalability, how their performance changes as more compute resources are used, but that this metric is often both useless and misleading.

  • hinkley: The older I get the more I feel like we're an accelerate version of the fashion industry. At least in fashion you can make an excuse that the design is thirty years old and most people don't remember the last time we did this. With software it's every six or seven. It's hard not to judge my peers for having such short memories. We were in the midst of one of these upheavals when I first started, and so I learned programming in that environment. It also means I have one more cycle than most people near my age. Now it all looks the same to me, and I understand those people who wanted to be more conservative. In fact I probably owe some people an apology.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Innovation and Magical Thinking

Detroit Tigers

I like baseball.  I can’t tell you the number of times I have spent the afternoon listening to a game and hearing the announcers expound on batting averages and on-base percentages as I puttered around the house. As someone with a background in quantitative analysis, I understand that the chances of a game-winning grand slam in the bottom of the ninth inning by a player that has not hit a home run in the major league are small.  However, in my mind’s eye, I can see the event happening and even believe that because I am listening that it will occur. This example is one form of magical thinking.  Magical thinking occurs when we attribute a causal or synchronistic relationship between actions or events that can’t be justified by reason and observation.  The current business environment means that innovation and magical thinking are often intertwined. Innovation without the ground game of implementation and continuous improvement is magical thinking.

Innovation, by definition, represents a substantial deviation from the thought processes of the past.  For example, Scrum and Extreme Programming (XP) are seen as innovations when compared to waterfall software development. Both are considered revolutionary, and when (and often if) adopted required substantial organizational transformations.  Organizational change is rarely easy; therefore, innovations are rarely adopted whole but rather reflect step-wise transformations.  Kent Beck in Extreme Programing Explained, Second Edition (2005) exposed the values of improvement and baby steps.  Once an innovation is identified it need to be implemented and then improved upon.  Believing that an innovation in its own right will change or save any organization is simply magical thinking.  Any innovation is only useful if it is USED and delivers value.

Even today, organizations consider Scrum, XP, and other Agile frameworks to be innovations under the ‚Äúif it is new to me, it is new‚ÄĚ mantra.¬† Regardless of the difficulty defending that definition of innovation, any organizations that think that they can ‚Äúadopt‚ÄĚ Agile without addressing implementation will up-end the status quo causing disruptions and dislocations. The Agile transformation fails because the bridge between innovation and value is defined by some sort of magical thinking.¬† Several years ago I was having a drink with a friend on Broadway in New York City.¬† The friend was describing a new innovative development framework his firm was adopting.¬† When I asked how it was going to be implemented, I was told that the CEO had mandated its use AND because it was so cool everybody would want to use it.¬† That was the whole implementation plan. Simply put, he had fallen prey to magical thinking. Within six months both the framework and he were not with the firm. Agile thinking has reinforced the idea that getting quickly started, generating feedback and then reacting to the feedback reduces risk and generates value quickly. Unless you have the luxury to implement an innovative idea, concept or product in a new organization without a past, just hoping that something will happen won‚Äôt generate change.¬†

In 1932 Frank Whittle was granted a patent for the turbojet.  Using the current state and predictability attributes noted in Innovation and Change, the turbojet was certainly not business as usual and could not be predicted from past events.  Whittle’s work was an innovation; however, due to testing and development issues, the RAF dismissed the idea prior to World War 2. The introduction of the jet fell to others.  Innovation did not translate to competitive advantage because of a failure in implementation. Innovation is an important step on the path to competitive advantage; however, it is simply a step. Unless innovation is combined with an implementation, we are just dealing with magical thinking.

Categories: Process Management