Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

Four Attributes of Empathic People


Happy Thanksgiving

Empathy is defined as understanding what another person is experiencing from their frame of reference. Translating that definition into action requires more than just an understanding. People that are empathic exhibit four basic attributes.  A person being empathetic must:

  • See the world as others see it.  Put yourself in another person’s shoe.  More importantly, you must be able to put aside your own filters.  This why it is often easier to be empathetic towards people that are like you or have similar experiences (we will revisit this potential dark side later in this series).
  • Be nonjudgmental.  Judging another person’s situation discounts the other person’s experience. Judging another’s experiences generates emotional distance.
  • Develop an understanding of yourself to understand others.  In order to be empathetic you need to understand your own feelings and biases so that you can put them aside.
  • Have the ability communicate our understanding of another’s feelings.  Communication includes listening, acknowledging what is being said and building a connection.  Using statements such as “I understand and I have had a similar experience” or “ Tell me more about what happened” (or how it made you feel).

Like all models, the four attributes that people that exhibit empathy are a simplification of human phycology.  The four attributes provide a  path to becoming more empathetic (even though the devil is in the details).

Note to my readers in the United States, Happy Thanksgiving. I hope you enjoy your holiday and have time to reflect on the aspects of your life that make you thankful. Holidays like Thanksgiving are a great time to reflect on the past and plan for the future. Using holidays as a retrospective is one way to develop an understanding of yourself so that you can understand others. 

Categories: Process Management


Herding Cats - Glen Alleman - 12 hours 27 min ago

Reflect upon your present blessings of which every man has many - not on your past misfortunes, of which all men have - Charles Dickens

Categories: Project Management

Estimating Software Intensive Systems

Herding Cats - Glen Alleman - 14 hours 49 min ago

Estimating SW Intensive SystemsEstimation and the measurements that result are critical success factors for Sofware Intensive Systems. Software development projects exist to design, build, and test products and to provide services from those software systems to customers. These products and there services include hardware, software, the documents describing their use and maintenance, and the operational data needed to provide business value to the customers who paid for that value.

Some may argue the differences between a project, product, or service. That is irrelevant for the discussion here. Some may argue there are many types of way to develop products or services. That is irrelevant here as well. Some projects deliver services on a continuous basis. Some projects have finite period of performance. Each needs to be answer the questions below.

But just to clear the air there are some differences between a system and a product. The system usually implies a unique collection of hardware and software, which a single organization owns and operates. The system of processing ACA eligibility applications is an example I worked on. This system was operated by a State agency in support of the governmental policies of providing affordable health care access to its citizens. The $42,000,000 a year budget for this system includes design, development, test, operations, and support of the hardware and software for a state government health care agency.

The products used to deploy the system included database engines, web servers, security products, development products, testing and integration products, help desk products and similar purpose built or COTS products integrated into a system.

Good Estimates Are One Key To Success

Estimates provide information to the decision makers to assess the impact of those decisions on future outcomes of the project, product, or system. Estimates provide information to make decisions, define feasible performance objectives, and substantiate the credibility of plans, revise designs and improve future estimates and processes.

The big question is, if the activities listed below are needed for the success of the project, how can they be answered in the absence of making an estimate?

Product Size, Performance, and Quality?

  • Evaluate feasibility of requirements
  • Analyze alternative product designs
  • Determine the required capacity and speed of hardware components
  • Evaluate product performance - accuracy, speed, reliability, availability and other ...ilities
  • Quantify resources needed to develop, deploy, and support a product, service, or system
  • Provide technical baselines for tracking and controlling cost,m schedule, and technical performance

Project Effort, Cost, and Schedule?

  • Determine project feasibility in terms of cost and schedule
  • Identify  and assess project risks
  • Negotiate achievable commitments
  • Prepare realistic plans and budgets
  • Evaluate business value, the cost of achieving that value and the cost - benefits of that value
  • Provide cost and schedule baselines for tracking and controlling the business and technical processes

Process Capability and Performance

  • Predict resources consumption and efficiency of those resources
  • Establish norms for expected performance
  • Identify opportunities for improvement of technical, cost, and schedule parameters

Anyone conjecturing the answers to these questions can be obtained without estimating needs to show exactly that can happen. 

Related articles Estimates Critical Success Factors of IT Forecasting Humpty Dumpty and #NoEstimates Complex Project Management Estimating Software-Intensive Systems Essential Reading List for Managing Other People's Money Can Enterprise Agile Be Bottom Up? Intentional Disregard for Good Engineering Practices?
Categories: Project Management

Should I Buy a Small House?

In this episode, I talk about buying a small house. Full transcript: John:               Hey, John Sonmez from simpleprogrammer.com. I got a question about investing in property. I did a couple of videos on property investment since I do some real estate investment and this email comes from John and John says, “Hi John, myself and […]

The post Should I Buy a Small House? appeared first on Simple Programmer.

Categories: Programming

Scheduling containers and more with Nomad

Xebia Blog - 19 hours 59 min ago

Specifically for the Dutch Docker Day on the 20th of November, HashiCorp released version 0.2.0 of Nomad which has some awesome features such as service discovery by integrating with Consul, the system scheduler and restart policies.  HashiCorp worked hard to release version 0.2.0 on 18th of November and we pushed ourselves to release a self-paced, hands-on workshop. If you would like to explore and play with these latest features of Nomad, go check out the workshop over at http://workshops.nauts.io.

In this blog post (or as I experienced it: roller coaster ride), you will catch a glimpse of the work that went into creating the workshop.

Last friday, November the 20th, was the first edition of the Dutch Docker Day where I helped prepare a workshop about "scheduling containers and more with Nomad". It was a great experience where attendees got to play with the new features included in 0.2.0, which nearly didn't make it into the workshop.


When HashiCorp released Nomad during their HashiConf event at the end of September, I was really excited as they always produce high quality tools with great user experience. As soon as the binary was available I downloaded it and tried to set up a cluster to see how it compared to some of it's competitors. The first release already had a lot of potential but also a lot of problems. For instance: when a container failed, Nomad would report it dead, but take no action; restart policies were still but a dream.

There were a lot of awesome features in store for the future of Nomad: integration with Consul, system jobs, batch jobs, restart policies, etc. Imagine all possible integrations with other HashiCorp tools! I was sold. So when I was asked to prepare a workshop for the Dutch Docker Day I jumped at the opportunity to get better acquainted with Nomad. The idea was that the attendees of the workshop, since it was a pretty new product and had some quirks, would go on an explorative journey into the far reaches of the scheduler and together find it's treasures and dark secrets.

Time went by and the workshop was taking shape nicely. We have a nice setup with a cluster of machines that automatically bootstrap the Nomad cluster and set up it's basic configuration. We were told that there would be a new version released before the Dutch Docker Day but nothing appeared, until the day before the event. I was both excited and terrified! The HashiCorp team worked long hours to get the new release of Nomad done in time for the Dutch Docker Day so Armon Dadgar, the CTO of HashiCorp and original creator of Nomad, could present the new features during his talk. This of course is a great thing, except for the fact that the workshop was entirely aimed at 0.1.2 and we had none of these new features incorporated into our Vagrant box. Were we going to throw all our work overboard and just start over, the night before the event?

“Immediately following the initial release of Nomad 0.1, we knew we wanted to get Nomad 0.2 and all its enhancements into the hands of our users by Dutch Docker Day. The team put in a huge effort over the course of a month and a half to get all the new features done and to a polish people expect of HashiCorp products. Leading up to the release we ran into one crazy bug after another (surprisingly all related to serialization). After intense debugging we got it to the fit and polish we wanted the night before at 3 AM! Once the website docs were updated and the blog post written, we released Nomad 0.2. The experience was very exciting but also exhausting! We are very happy with how it turned out and hope the users are too!„

- Alex Dadgar, HashiCorp Engineer working on Nomad

It took until late in the evening to get an updated Vagrant box with a bootstrapped Consul cluster and the new Nomad version, in order to showcase the auto discovery feature and Consul integration that 0.2.0 added. However, the slides for the workshop were still referencing the problems we encountered when trying out the 0.1.0 and 0.1.2 release, so all the slides and statements we had made about things not working or being released in the future had to be aligned with the fixes and improvements that came with the new release. After some hours of hectic editing during the morning of the event, the slides were finally updated and showcased all the glorious new features!

Nomad simplifies operations by supporting blue/green deployments, automatically handling machine failures, and providing a single workflow to deploy applications.

The new features they added in this release and the amount of fixes and improvements is staggering. In order to discover services there is no longer a need for extra tools such as Registrator, and your services are now automatically registered and deregistered as soon as they are started and stopped (which I first thought was a bug, cause I wasn't used to Nomad actually restarting my dead containers). The system scheduler is another feature I've been missing in other schedulers for a while, as it makes it possible to easily schedule services (such as Consul or Sysdig) on all of the eligible nodes in the cluster.

Feature 0.1.2 0.2.0 Service scheduler Schedule a long lived job. Y Y Batch scheduler Schedule batch workloads. Y Y System scheduler Schedule a long lived job on every eligible node. N Y Service discovery Discover launched services in Consul. N Y Restart policies If and how to restart a service when it fails. N Y Distinct host constraint Ensure that Task Groups are running on distinct clients. N Y Raw exec driver Run exec jobs without jailing them. N Y Rkt driver A driver for running containers with Rkt. N Y External artifacts Download external artifacts to execute for Exec and Raw exec drivers. N Y   And numerous fixes/improvements were added to 0.2.0

If you would like to follow the self-paced workshop by yourself, you can find the slides, machines and scripts for the workshop at http://workshops.nauts.io together with the other workshops of the event. Please let me know your experiences, so the workshop can be improved over time!

I would like to thank the HashiCorp team for their amazing work on the 0.2.0 release, the speed at which they have added so many great new features and improved the stability is incredible.

It was a lot of fun preparing the workshop for the Dutch Docker Day. Working with bleeding edge technologies is always a great way to really get to know it's inner workings and quirks, and I would recommend it to anyone, just be prepared to do some last-minute work ; )

Agile at Scale - A Reading List (Update)

Herding Cats - Glen Alleman - Wed, 11/25/2015 - 22:29

I'm working two programs where Agile at Scale as the development paradigm. When we start an engagement using other peoples money, in this case the money belongs to a sovereign, we make sure everyone is on the same page. When Agile at Scale is applied, it is usually applied on programs that have tripped the FAR 34.2/DFARS 234.2 levels for Earned Value Management. This means $20M programs are self assessed and $100M and above are validated by the DCMA (Defense Contract Management Agency).

While these programs are applying Agile, usually Scrum, they are also subject to EIA-748-C compliance and a list of other DID's (Data Item Description) and other procurement, development, testing, and operational guidelines . These means there are multiple constraints on how the progress to plan is reported to the customer - the sovereign. These programs are not 5 guys at the same table as their customer exploring what will be needed for mission success when they're done. These programs are not everyone's cup of tea, but agile is a powerful tool in the right hands of Software Intensive System of Systems for Mission Critical programs. Programs that MUST, deliver the needed Capabilities, at the Need Time, for the Planned Cost, within the planned Margins for cost, schedule, and technical performance.

One place to start to improve the probability that we're all on the same page- is this reading list. This is not an exhaustive list, and it is ever growing. But it's a start. It;s hoped this list is the basis of a shared understanding that while Agile is a near universal principle, there are practices that must be tailored to specific domains. And one's experience in one domain may or may not be applicable to other domains. 

Like it says in the Scrum Guide. 

Scrum (n): A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value.

And since Scrum is an agile software development framework, Scrum is a framework not a methodology. Scrum of Scrums, Agile At Scale, especially Agile at Scale inside EIA-748-C prorgams has much different needs than 5 people sitting at the same table with their customer with an emerging set of requirements where the needed capabilities will also emerge.

Related articles Slicing Work Into Small Pieces Agile Software Development in the DOD Technical Performance Measures What Do We Mean When We Say "Agile Community?" Can Enterprise Agile Be Bottom Up? How To Measure Anything
Categories: Project Management

Debunking: The Five Laws of Software Estimating

Herding Cats - Glen Alleman - Wed, 11/25/2015 - 17:52

The Five Laws of Software Estimating contains several Bunk ideas. let's look at each of the Five

1. Estimates are a Waste

Time spent on estimates is time that isn’t spent delivering value. It’s a zero-sum game when it comes to how much time developers have to get work done – worse if estimates are being requested urgently and interrupting developers who would otherwise be “in the zone” getting things done. If your average developer is spending 2-4 hours per 40-hour week on estimates, that’s a 5-10% loss in productivity, assuming they were otherwise able to be productive the entire time. It’s even worse if the developer in question is part-time, or is only able to spend part of their work week writing software.

The estimates aren't for the developers, they're for those paying the developers. 

Estimates can help developers plan work sequencing, needed staffing and other resources, but to say it takes away from their development effort reeks of hubris. 

Writing software for money, requires money to be provided, sales revenue to be acquired to offset that cost. This is a business discussion not a coders point of view. 

2. Estimates are Non-Transferable

Software estimates are not fungible, mainly as a corollary to the fact that team members are not fungible. This means one individual’s estimate can’t be used to predict how long it might take another individual to complete a task.

This is the basis of Reference Class Forecasting. Write down what the estimates were when you made them, classify the estimates by the categories of software, put those in a sprad sheet and use then again when you encounter a similar class of software. This is standard software engineering practice in any mature organization. CMMI ML 2 and 3 process areas. 

3. Estimates are Wrong

Estimates aren’t promises. They’re guesses, and generally the larger the scope and the further in the future the activity being estimated is, the greater the potential error.

This is one of those naive statements that ignores the mathematics of estimating.

Estimate can be guesses, that'd be really bad estimating. Don't do stupid things on purpose. Read Estimating Software Intensive Systems, and learn how to estimate, then you won't be saying naive things like estimates are guesses

4. Estimates are Temporary

Estimates are perishable. They have a relatively short shelf-life. A developer might initially estimate that a certain feature will take a week to develop, before the project has started. Three months into the project, a lot has been learned and decided, and that same feature might now take a few hours, or a month, or it might have been dropped from the project altogether due to changes in priorities or direction. In any case, the estimate is of little or perhaps even negative value since so much has potentially changed since it was created.

Yes, estimates are temporary. They are updated periodically with new information. This is standard estimating process.

Reference Class Forecasting updates the reference classes when new data is available. 

5. Estimate are Necessary

Despite the first four Laws of Estimates, estimates are often necessary. Businesses cannot make decisions about whether or not to build software without having some idea of the cost and time involved.

Yes they are. Making decisions in the presence of uncertainty means making estimates of the outcomes and impacts of those decisions on future.

So before applying any of these ideas, ask a s simple question and get the answer.

What's the value at risk for NOT estimating the impact of your decision?


Related articles Sources for Reference Class Forecasting Should I Be Estimating My Work? Managing Your Project With Dilbert Advice - Not!
Categories: Project Management

How To Hire A Tech Team

Making the Complex Simple - John Sonmez - Wed, 11/25/2015 - 14:00

As with any career, there comes a time when you have to take on more responsibility. Usually, that’s because you’ve become an established, experienced member of the team, or you’ve reached a point of respect where your seniors look to you for guiding their hiring decisions. Ultimately, they trust you. At this point, you might […]

The post How To Hire A Tech Team appeared first on Simple Programmer.

Categories: Programming

Software architecture diagrams should be maps of your source code

Coding the Architecture - Simon Brown - Wed, 11/25/2015 - 09:23

If you've ever worked on a codebase that's more than just a sample application, you'll know that understanding and navigating the code can be tricky, certainly until you familiarise yourself with the key structures within it. Once you have a shared vocabulary that you can use to describe those key structures, creating some diagrams to describe them is easy. And if those structures are hierarchical, your diagrams become maps that you can use to navigate the codebase.

Software architecture diagrams are maps of your code

If you open up something like Google Maps on your smartphone and do a search for Jersey, it will zoom into Jersey. This is great if you want to know what's inside Jersey and what the various place names are, but if you've never heard of Jersey it's completely useless. What you then need to do is pinch-to-zoom-out to get back to the map of Europe, which puts Jersey in context. Diagrams of our software should be the same. Sometimes, as developers, we want the zoomed-in view of the code and at other times, depending on who we are talking to for example, we need a zoomed-out view.

Software architecture diagrams are maps of your code

A feature that has been built into Structurizr is that you can link components on a component diagram to code-level elements, which provides that final level of navigation from diagrams to code. You can try this yourself on the software architecture diagrams for the Spring PetClinic application.

Whatever tooling you use to create software architecture diagrams though, make sure that your diagrams reflect real structures in the code and that the mapping between diagrams and code is simple. My FREE The Art of Visualising Software Architecture ebook has more information on this topic.

Categories: Architecture

Defining Empathy

Empathy is the ability to place yourself in another person’s shoes.

Empathy is the ability to place yourself in another person’s shoes.

Coaching is a key tool to help individuals and teams reach peak performance. One of the key attributes of a good coach is empathy. I was recently drawn into a discussion about the role of empathy in coaching. Critical to the understanding the role that empathy plays in coaching is understanding the definition of empathy. One of the most difficult parts of the discussion was agreeing on a common definition .

One of the first definitions proffered during our discussion was that empathy represented the ability to understand others feelings and emotions. On reflection it was decided that this definition only goes part of the way. Empathy is a deeper concept that subsumes mere understanding, requiring more of a cognitive connection between he coach and coachee. The level of understanding requires both parties understand not only the emotion or behavior, but have understanding of the basis for the emotion or behavior without taking on the emotion or accepting the emotion. In the end we arrived at a more precise definition, empathy is the capacity/ability to understand or feel what another person is experiencing from their frame of reference.  Put more succinctly, Chris Nurre on SPaMCAST 362 stated,

The ability to put yourself in another person’s shoes is critical for a good coach because it helps the coach to accept the coachee on his or her own terms.  Empathy is useful to:

  • Help to build trust.
  • Build a pathway that enables communication.
  • Disarm the coachee’s self-interest.
  • Facilitate the development of altruism.
  • Encourage bigger picture point of view

Empathy is also a valuable commodity at a team level.  Empathy between team members allows teams to create bonds of trust in a similar manner as between individuals. Without empathy between members, teams will be not be cohesive and will pursue their individual self-interest.  A coach must be able to put themselves into the shoes of the team and team members they interact with in order to help those they are coaching to build self-knowledge.  At the same time, team members must have the enough empathy for their fellow team members to hold a team together so they can pursue a single goal. 

Next few entries will include discussions of :

  1. Attributes and techniques of an empathic coach.
  2. Learning empathy.
  3. Too much empathy.

Categories: Process Management

Sponsored Post: StatusPage.io, iStreamPlanet, Redis Labs, Jut.io, SignalFx, InMemory.Net, VividCortex, MemSQL, Scalyr, AiScaler, AppDynamics, ManageEngine, Site24x7

Who's Hiring?
  • Senior Devops Engineer - StatusPage.io is looking for a senior devops engineer to help us in making the internet more transparent around downtime. Your mission: help us create a fast, scalable infrastructure that can be deployed to quickly and reliably.

  • As a Networking & Systems Software Engineer at iStreamPlanet you’ll be driving the design and implementation of a high-throughput video distribution system. Our cloud-based approach to video streaming requires terabytes of high-definition video routed throughout the world. You will work in a highly-collaborative, agile environment that thrives on success and eats big challenges for lunch. Please apply here.

  • As a Scalable Storage Software Engineer at iStreamPlanet you’ll be driving the design and implementation of numerous storage systems including software services, analytics and video archival. Our cloud-based approach to world-wide video streaming requires performant, scalable, and reliable storage and processing of data. You will work on small, collaborative teams to solve big problems, where you can see the impact of your work on the business. Please apply here.

  • At Scalyr, we're analyzing multi-gigabyte server logs in a fraction of a second. That requires serious innovation in every part of the technology stack, from frontend to backend. Help us push the envelope on low-latency browser applications, high-speed data processing, and reliable distributed systems. Help extract meaningful data from live servers and present it to users in meaningful ways. At Scalyr, you’ll learn new things, and invent a few of your own. Learn more and apply.

  • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here.

  • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here.
Fun and Informative Events
  • Your event could be here. How cool is that?
Cool Products and Services
  • Real-time correlation across your logs, metrics and events.  Jut.io just released its operations data hub into beta and we are already streaming in billions of log, metric and event data points each day. Using our streaming analytics platform, you can get real-time monitoring of your application performance, deep troubleshooting, and even product analytics. We allow you to easily aggregate logs and metrics by micro-service, calculate percentiles and moving window averages, forecast anomalies, and create interactive views for your whole organization. Try it for free, at any scale.

  • Turn chaotic logs and metrics into actionable data. Scalyr replaces all your tools for monitoring and analyzing logs and system metrics. Imagine being able to pinpoint and resolve operations issues without juggling multiple tools and tabs. Get visibility into your production systems: log aggregation, server metrics, monitoring, intelligent alerting, dashboards, and more. Trusted by companies like Codecademy and InsideSales. Learn more and get started with an easy 2-minute setup. Or see how Scalyr is different if you're looking for a Splunk alternative or Sumo Logic alternative.

  • SignalFx: just launched an advanced monitoring platform for modern applications that's already processing 10s of billions of data points per day. SignalFx lets you create custom analytics pipelines on metrics data collected from thousands or more sources to create meaningful aggregations--such as percentiles, moving averages and growth rates--within seconds of receiving data. Start a free 30-day trial!

  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net

  • VividCortex goes beyond monitoring and measures the system's work on your servers, providing unparalleled insight and query-level analysis. This unique approach ultimately enables your team to work more effectively, ship more often, and delight more customers.

  • MemSQL provides a distributed in-memory database for high value data. It's designed to handle extreme data ingest and store the data for real-time, streaming and historical analysis using SQL. MemSQL also cost effectively supports both application and ad-hoc queries concurrently across all data. Start a free 30 day trial here: http://www.memsql.com/

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Also available on Amazon Web Services. Free instant trial, 2 hours of FREE deployment support, no sign-up required. http://aiscaler.com

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Categories: Architecture

jq: Cannot iterate over number / string and number cannot be added

Mark Needham - Tue, 11/24/2015 - 01:12

In my continued parsing of meetup.com’s JSON API I wanted to extract some information from the following JSON file:

$ head -n40 data/members/18313232.json
    "status": "active",
    "city": "London",
    "name": ". .",
    "other_services": {},
    "country": "gb",
    "topics": [],
    "lon": -0.13,
    "joined": 1438866605000,
    "id": 92951932,
    "state": "17",
    "link": "http://www.meetup.com/members/92951932",
    "photo": {
      "thumb_link": "http://photos1.meetupstatic.com/photos/member/8/d/6/b/thumb_250896203.jpeg",
      "photo_id": 250896203,
      "highres_link": "http://photos1.meetupstatic.com/photos/member/8/d/6/b/highres_250896203.jpeg",
      "photo_link": "http://photos1.meetupstatic.com/photos/member/8/d/6/b/member_250896203.jpeg"
    "lat": 51.49,
    "visited": 1446745707000,
    "self": {
      "common": {}
    "status": "active",
    "city": "London",
    "name": "Abdelkader Idryssy",
    "other_services": {},
    "country": "gb",
    "topics": [
        "name": "Weekend Adventures",
        "urlkey": "weekend-adventures",
        "id": 16438
        "name": "Community Building",
        "urlkey": "community-building",

In particular I want to extract the member’s id, name, join date and the ids of topics they’re interested in. I started with the following jq query to try and extract those attributes:

$ jq -r '.[] | [.id, .name, .joined, (.topics[] | .id | join(";"))] | @csv' data/members/18313232.json
Cannot iterate over number (16438)

Annoyingly this treats topic ids on an individual basis rather than as an array as I wanted. I tweaked the query to the following with no luck:

$ jq -r '.[] | [.id, .name, .joined, (.topics[].id | join(";"))] | @csv' data/members/18313232.json
Cannot iterate over number (16438)

As a guess I decided to wrap ‘.topics[].id’ in an array literal to see if it had any impact:

$ jq -r '.[] | [.id, .name, .joined, ([.topics[].id] | join(";"))] | @csv' data/members/18313232.json
92951932,". .",1438866605000,""
jq: error (at data/members/18313232.json:60013): string ("") and number (16438) cannot be added

Woot! A different error message at least and this one seems to be due to a type mismatch between the string we want to end up with and the array of numbers that we currently have.

We can cast our way to victory with the ‘tostring’ function:

$ jq -r '.[] | [.id, .name, .joined, ([.topics[].id | tostring] | join(";"))] | @csv' data/members/18313232.json
92951932,". .",1438866605000,""
193866304,"Abdelkader Idryssy",1445195325000,"16438;20727;15401;9760;20246;20923;3336;2767;242;58259;4417;1789;10454;20274;10232;563;25375;16433;15187;17635;26273;21808;933;7789;23884;16212;144477;15322;21067;3833;108403;20221;1201;182;15083;9696;4377;15360;18296;15121;17703;10161;1322;3880;18333;3485;15585;44584;18692;21681"
28643052,"Abhishek Chanda",1439688955000,"646052;520302;15167;563;65735;537492;646072;537502;24959;1025832;8599;31197;24410;26118;10579;1064;189;48471;16216;18062;33089;107633;46831;20479;1423042;86258;21441;3833;21681;188;9696;58162;20398;113032;18060;29971;55324;30928;15261;58259;638;16475;27591;10107;242;109595;10470;26384;72514;1461192"
39523062,"Adam Kinder-Jones",1438677474000,"70576;21549;3833;42277;164111;21522;93380;48471;15167;189;563;25435;87614;9696;18062;58162;10579;21681;19882;108403;128595;15582;7029"
194119823,"Adam Lewis",1444867994000,"10209"
14847001,"Adam Rogers",1422917313000,""
87709042,"Adele Green",1436867381000,"15167;18062;102811;9696;30928;18060;78565;189;7029;48471;127567;10579;58162;563;3833;16216;21441;37708;209821;15401;59325;31792;21836;21900;984862;15720;17703;96823;4422;85951;87614;37428;2260;827;121802;19672;38660;84325;118991;135612;10464;1454542;17936;21549;21520;17628;148303;20398;66339;29661"
11497937,"Adrian Bridgett",1421067940000,"30372;15046;25375;638;498;933;374;27591;18062;18060;15167;10581;16438;15672;1998;1273;713;26333;15099;15117;4422;15892;242;142180;563;31197;20479;1502;131178;15018;43256;58259;1201;7319;15940;223;8652;66493;15029;18528;23274;9696;128595;21681;17558;50709;113737"
14151190,"adrian lawrence",1437142198000,"7029;78565;659;85951;15582;48471;9696;128595;563;10579;3833;101960;16137;1973;78566;206;223;21441;16216;108403;21681;186;1998;15731;17703;15043;16613;17885;53531;48375;16615;19646;62326;49954;933;22268;19243;37381;102811;30928;455;10358;73511;127567;106382;16573;36229;781;23981;1954"
183557824,"Adrien Pujol",1421882756000,"108403;563;9696;21681;188;24410;1064;32743;124668;15472;21123;1486432;1500742;87614;46831;1454542;46810;166000;126177;110474"
Categories: Programming

Android Studio 2.0 Preview

Android Developers Blog - Tue, 11/24/2015 - 00:52

Posted by, Jamal Eason, Product Manager, Android

One the most requested features we receive is to make app builds and deployment faster in Android Studio. Today at the Android Developer Summit, we’re announcing a preview of Android Studio 2.0 featuring Instant Run that will dramatically improve your development workflow. With Android Studio 2.0, we are also including a preview of a new GPU Profiler.

All these updates are available now in the canary release channel, so we can get your feedback. Since this initial release is a preview, you may want to download and run an additional copy of Android Studio in parallel with your current version.

New Features in Android Studio 2.0
Instant Run: Faster Build & Deploy

Android Studio’s instant run feature allows you to to quickly see your changes running on your device or emulator.

Getting started is easy. If you create a new project with Android Studio 2.0 then your projects are already setup. If you have a pre-existing app open Settings/Preferences, the go to Build, Execution, Deployment → Instant Run. Click on Enable Instant Run... This will ensure you have the correct gradle plugin for your project to work with Instant Run.

Enable Instant Run for Android Studio projects

Select Run as normal and Android Studio will perform normal compilation, packaging and install steps and run your app on your device or emulator. After you make edits to your source code or resources, pressing Run again will deploy your changes directly into the running app.

New Run & Stop Actions in Android Studio for Instant Run

For a more detailed guide setup and try Instant Run, click here.

GPU Profiler

Profiling your OpenGL ES Android code is now even easier with the GPU Profiler in Android Studio. The tool is in early preview, but is very powerful and not only shows details about the GL State and Commands, you can record entire sessions and walk through the GL Framebuffer and Textures as your app is running OpenGL ES Code.

Android Studio GPU Profiler

To get started, first download the GPU Debugging Tools package from the Android Studio SDK Manager. Click here for more details about the GPU Profiler tool and how to set up your Android app project for profiling.

Whats Next

This is just a taste of some of the bigger updates in this latest release of Android Studio. We'll be going through the full release in more detail at the Android Developer Summit (livestreamed on Monday and Tuesday). Over the next few weeks, we'll be showing how to take advantage of even more features in Android Studio 2.0, so be sure to check back in.

If you're interested in more Android deep technical content, we will be streaming over 16 hours of content from the inaugural Android Developer Summit over the next two days, and together with Codelabs, all of this content will be available online after the Summit concludes.

Android Studio 2.0 is available today on the Android Studio canary channel. Let us know what you think of these new features by connecting with the Android Studio development team on Google+.

Categories: Programming

Example Mapping - Steering the conversation

Xebia Blog - Mon, 11/23/2015 - 19:14

People who are familiar with BDD and ATDD already know how useful the three amigos (product owner, tester and developer) session is for talking about what the system under development is supposed to do. But somehow these refinement sessions seem to drain the group's energy. One of the problems I see is not having a clear structure for conversations.

Example Mapping is a simple technique that can steer the conversation into breaking down any product backlog items within 30 minutes.

The Three Amigos

Example Mapping is best used in so called Three Amigo Sessions. The purpose of this session is to create a common understanding of the requirements and a shared vocabulary across product owner and the rest of the team. During this session the product owner shares every user story by explaining the need for change in a product. It is essential that the conversation has multiple points of view. Testers and developers identify missing requirements or edge cases and are addressed by describing accepted behaviour, before a feature is considered ready for development.

In order to help you steer the conversations, here is a list of guidelines for Three Amigos Sessions:

  • Empathy: Make sure the team has the capability to help each other understand the requirements. Without empathy and the room for it, you are lost.
  • Common understanding of the domain: Make sure that the team uses the same vocabulary (digital of physical) and speaks the same domain language.
  • Think big, but act small: Make sure all user stories are small and ready enough to make impact
  • Rules and examples: Make sure every user story explains the specification with rules and scenarios / examples.
Example mapping

Basic ingredients for Example Mapping are curiosity and a pack of post-it notes containing the following colours:

  • Yellow for defining user story
  • Red for defining questions
  • Blue for defining rules
  • Green for defining examples

Using the following steps can help you steer the conversations towards accepted behaviour of the system under development:

  1. Understanding the problem
    Let the product owner start by writing down the user story on a yellow post-it note and have him explain the need for change in the product. The product owner should help the team understand the problem.
  2. Challenge the problem by asking questions
    Once the team has understood the problem, the team challenges the problem by asking questions. Collect all the questions by writing them down starting with "What if ... " on red post-it notes. Place them on the right side of the user story (yellow) post-it note. We will treat this post-it note as a ticket for a specific and focussed discussion.
  3. Identifying rules
    The key here is to identify rules for every given answer (steered from the red question post-it notes). Extract rules from the answers and write them down on a blue post-it note. Place them below the user story (yellow) post-it note. This basically describes the acceptance criteria of a user story. Make sure that every rule can be discussed separately. The single responsibility principle and separation of concerns should be applied.
  4. Describing situations with examples
    Once you have collected all the important rules of the user story, you collect all interesting situations / examples by writing them down on a green post-it note. Place them below the rule (blue) post-it note. Make sure that the team talks about examples focussed on one single rule. Steer the discussion by asking questions like: Have we reached the boundaries of the rule? What happens when the rule fails?
An example


Here in the given example above, the product owners requires a free shipping process. She wrote it down on a yellow post-it note. After collecting and answering questions, two rules were discussed and refined on blue post-it notes; shopping cart limit and the appearance of the free shipping banner on product pages. All further discussions were steered towards the appropriate rule. Two examples in the shopping cart limit were defined and one example for the free shipping banner on a green post-it notes. Besides steering the team in rule based discussions, the team also gets a clear overview of the scope for the first iteration of the requirement.

Getting everyone on the same page is the key to success here. Try it a couple of times and let me know how it went.


Your New Technical Skills

One of the struggles a developer faces when moving up the ladder is how to keep their technical skills.

If they are used to being a high-performing, individual contributor, and a technical go-to resource, this is especially challenging.


Because the job is different, now.

It’s no longer about how awesome your developer skills are.  Now it’s about bringing out the best from the people you manage, and hopefully *lead.*  Your job is now about creating a high-performing team.   It’s about growing more leaders.  It’s about being the oil and the glue.  The oil so that the team can work effectively, as friction-free as possible, and the glue, so that all the work connects together.

There’s a good book called What Got You Here, Won’t Get You There, by Marshall Goldsmith.  The book title sort of says it all, but the big idea is that if you take on a new management role, but continue to perform like an individual contributor, or at a lower level, don’t expect to be successful.

The irony is that most people will quickly default to doing what they do best, which is what got them to where they are.   But now the rules have changed, and they don’t adapt.  And as the saying goes, adapt or die.  It’s how a lot of careers end.

But not you.

While you will want to keep up your skills that got you to where you are, the real challenge is about adding new ones.   And, at first blush, they might just seem like “soft skills”, while you are used to learning “technical skills.”   Well, treat these at your new technical skills to learn.

Your new technical skills are:

  1. Building EQ (Emotional Intelligence) in teams
  2. Building High-Performance Teams
  3. Putting vision/mission/values in place
  4. Putting the execution model in place
  5. Directing and inspiring as appropriate – situational leadership – per employee
  6. Creating and leverage leadership opportunities and teachable moments
  7. Creating the right decision frameworks and flows and empowerment models
  8. Building a better business
  9. And doing thought-work in the space for the industry

I’ll leave this list at 9, so that it doesn’t become a 10 Top Skills to Learn to Advance Your Career post.

Emotional Intelligence as a Technical Skill

If you wonder how Emotional Intelligence can be a technical skill, I wish I could show you all the Mind Maps, the taxonomies, the techniques, the hard-core debates over the underlying principles, patterns, and practices, that I have seen many developers dive into over the years.

The good news is that Emotional Intelligence is a skill you can build.  I’ve seen many developers become first time managers and then work on their Emotional Intelligence skills and everything changes.  They become a better manager.  They become more influential.  They read a room better and know how to adapt themselves more effectively in any situation.  They know how to manage their emotions.  And they know how to inspire and delight others, instead of tick them off.

Along the lines of Emotional Intelligence, I should add Financial Intelligence to the mix.  So many developers and technologists would be more effective in the business arena, if they mastered the basics of Financial Intelligence.  There is actually a book called Financial Intelligence for IT Professionals.   It breaks down the basics of how to think in financial terms.   Innovation doesn’t fund itself.  Cool projects don’t fund themselves.  Technology is all fun and games until the money runs out.  But if you can show how technology helps the business, all of a sudden instead of being a cost or overhead, you are now part of the value chain, or at least the business can appreciate what you bring to the table.

Building High-Performance Teams as a Technical Skill

Building High-Performance Teams takes a lot of know-how.  It helps if you are already well grounded in how to ship stuff.  It really helps if you have some basic project management skills and you know how to see how the parts of the project come together as a whole.  It especially helps if you have a strong background in Agile methodologies like Kanban, Scrum, XP, etc.  While you don’t need to create Kanbans, its certainly helps if you get the idea of visualizing the workflow and reducing open work.  And, while you may not need to do Scrum per se, it helps if you get the idea behind a Product Backlog, a Sprint Backlog, and Sprints.  And while you may not need to do XP, it helps if you get the idea of sustainable pace, test-driven development, pairing, collective ownership, and an on-site customer. 

But the real key to building high-performance teams is actually about trust. 

Not trust as in “I trust that you’ll do that.”  

No.  It’s vulnerability-based trust, as in “I’ve got your back.”   This is what enables individuals on a team to go out on a limb, to try more, to do more, to become more.

Otherwise, they everybody has to watch out for their own backs, and they spend their days making sure they don’t get pushed off the boat or hanging from a limb, while somebody saws it off.   (See 10 Things Great Managers Do.)

And nothing beats a self-organizing team, where people sign-up for work (vs. get assigned work), where people play their position well, and help others play theirs.

Vision, Mission, Values as a Technical Skill

Vision, mission, and values are actually some of the greatest technical skills you can master, for yourself and for any people or teams you might lead, now or in the future.   So many people mix up vision and mission.

Here’s the deal:

Mission is the job.

Vision is where you want to go, now that you know what the job is.

And Values are what you express in actions in terms of what you reward.  Notice how I said actions, not words.  Too many people and teams say they value one thing, but their actions value another.

It’s one thing to go off and craft a vision, mission, and values that you want everybody to adhere to.  It’s another thing to co-create the future with a team, and create your vision, mission, and values, with everybody’s fingerprints on it.  But that’s how you get buy-in.   And getting buy-in, usually involves dealing with conflict (which is a whole other set of technical skills you can master.)  

When a leader can express a vision, mission, and values with clarity, they can inspire the people around them, bring out the best in people, create a high-performance culture, and accelerate results.

Execution as a Technical Skill

This is where the rubber meets the road.  There are so many great books on how to execute with skill.  One of my favorites is Flawless Execution.  And of the most insightful books on creating an effective execution model is Managing the Design Factory.

The main thing to master here is to be able to easily create a release schedule that optimizes resources and people, while flowing value to customers and stakeholders.

I know that’s boiling a lot down, but that’s the point.  To master execution, you need to be able to easily think about the challenges you are up against:  not enough time, not enough resources, not enough budget, not enough clarity, not enough customers, etc.

It’s a powerful thing when you can turn chaos into clarity and get the train leaving the station in a reliable way.

It’s hard to beat smart people shipping on a cadence, if they are always learning and always improving.

Situational Leadership as a Technical Skill

Sadly, this is one of the most common mistakes of new managers.  Seasoned ones, too.  They treat everybody on the team the same.  And they usually default to whatever they learned.   They either focus on motivating or they focus on directing.  And directing to the extreme, very quickly becomes micro-managing.

The big idea of Situational Leadership is to consider whether each person needs direction or motivation, or both.  

If you try to motivate somebody who is really looking for direction, you will both be frustrated.  Similarly, if you try to direct somebody who really is looking for motivation, it’s a quick spiral down.

There are many very good books on Situational Leadership and how to apply it in the real world.

Decision Making as a Technical Skill

This is where a lot of blood-shed happens.   This is where conflict thrives or dies.   Decision making is the bread-and-butter of today’s knowledge worker.  That’s what makes insight so valuable in a Digital Economy.  After all, what do you use the insight for?  To make better decisions.

It’s one thing for you to just make decisions.

But the real key here is how to create simple ways to deal with conflict and how to make better decisions as a group.   This includes how to avoid the pitfalls of groupthink.  It includes the ability to leverage the wisdom of the crowds.  It also includes the ability to influence and persuade with skill.  It includes the ability to balance connection with conviction.  It includes the ability to balance your Conflict Management Style with the Conflict Management Style of others.

Business as a Technical Skill

Business can be hard-core.   This isn’t so obvious if you deal with mediocre business people.  But when you interact with serious business leaders, you quickly understand how complicated, and technical, running a business and changing a business really is.

At the most fundamental level, the purpose of a business is to create a customer.

But even who you choose to serve as your “customer” is a strategic choice.

You can learn a lot about business by studying some of the great business challenges in the book, Case Interview Secrets, which is written by a former McKinsey consultant.

You can also learn a lot about business by studying which KPIs and business outcomes matter, in each industry, and by each business function.

It also helps to be able to quickly know how to whiteboard a value chain and be able to use some simple tools like SWOT analysis.  If you can really internalize Michael Porter’s mental models and toolset, then you will be ahead of many people in the business world.

Thoughtwork as a Technical Skill

There are many books and guides on how to be a leader in your field.   One of my favorites is, Lead the Field, by Earl Nightingale.  It’s an oldie, but goodie.

The real key is to be able to master ideation.  You need to be able to come up with ideas.   Probably the best technique I learned was long ago.   I simply set an idea quota.   In the book, ThinkerToys, by Michael Michalko, I learned that Thomas Edison set a quote to think up new ideas.  Success really is a numbers game.   Anyway, I started by writing one idea per note in my little yellow sticky pad.  The first week, I had a handful of ideas.   But once my mind was cleared by writing my ideas down, I was soon filling up multiple yellow sticky pads per week.

I very quickly went from having an innovation challenge to having an execution challenge.

So then I went back to the drawing board and focused on mastering execution as a technical skill Winking smile

Hopefully, if you are worried about how to keep growing your skills as you climb your corporate ladder, this will give you some food for thought.

Categories: Architecture, Programming

How Wistia Handles Millions of Requests Per Hour and Processes Rich Video Analytics

This is a guest repost from Christophe Limpalair of his interview with Max Schnur, Web Developer at  Wistia.

Wistia is video hosting for business. They offer video analytics like heatmaps, and they give you the ability to add calls to action, for example. I was really interested in learning how all the different components work and how they’re able to stream so much video content, so that’s what this episode focuses on.

What does Wistia’s stack look like?

As you will see, Wistia is made up of different parts. Here are some of the technologies powering these different parts:

What scale are you running at?
Categories: Architecture

Software Development Conferences Forecast November 2015

From the Editor of Methods & Tools - Mon, 11/23/2015 - 17:14
Here is a list of software development related conferences and events on Agile project management ( Scrum, Lean, Kanban), software testing and software quality, software architecture, programming (Java, .NET, JavaScript, Ruby, Python, PHP), DevOps and databases (NoSQL, MySQL, etc.) that will take place in the coming weeks and that have media partnerships with the Methods […]

Typical Mistakes Even Experienced Developers Make In WordPress

Making the Complex Simple - John Sonmez - Mon, 11/23/2015 - 14:00

We all know and love WordPress, which is why it’s used for 4.7% of all websites on the internet. One would think that developers would be way above making any mistakes on such a huge platform by now. And one would be wrong. Despite using the platform for more than a decade and watching it […]

The post Typical Mistakes Even Experienced Developers Make In WordPress appeared first on Simple Programmer.

Categories: Programming

SPaMCAST 369 – The Stand-Up Meeting, #NoEstimates, More on Mastery

Software Process and Measurement Cast - Sun, 11/22/2015 - 23:00

The Software Process and Measurement Cast 369 features our essay on stand-up meetings.  Stand-up meetings are a combination of tactical planning on steroids and the perfect start to a great day. Stand-up meetings are one the easiest Agile practices to adopt and often one the easiest to mess up.

Also, this week features we have Kim Pries and his Software Sensei column.  Kim discusses what it takes to move toward mastery. Mastery implies more than just being good at any particular task. The Software Sensei provides a path forward.

Gene Hughson brings the first of his discussions on the topic of #NoEstimates from his Form Follows Function blog! Specifically Gene provided a more detail and background on his essay #NoEstimates – Questions, Answers, and Credibility.

Call to Action!

Review the SPaMCAST on iTunes, Stitcher or your favorite podcatcher/player and then share the review! Help your friends find the Software Process and Measurement Cast. After all, friends help friends find great podcasts!

Re-Read Saturday News

Remember that the Re-Read Saturday of The Mythical Man-Month is winding down. THis week we are running a quick poll to identify the next book  on the Software Process and Measurement Blog.  What would you like the next re-read book to be?

Upcoming Events

Details on 2016 Conferences that include QAI Quest and StarEast to name a few in a few weeks.


The next Software Process and Measurement Cast will feature a discussion with Steve Tendon discussing more of his new book Tame The Flow, and more. (Subject to change due to the Thanksgiving holiday in the States).

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects

should not be a tedious process, neither

for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

Categories: Process Management

SPaMCAST 369 – The Stand-Up Meeting, #NoEstimates, More on Mastery



Listen Now

Subscribe on iTunes

The Software Process and Measurement Cast 369 features our essay on stand-up meetings.  Stand-up meetings are a combination of tactical planning on steroids and the perfect start to a great day. Stand-up meetings are one the easiest Agile practices to adopt and often one the easiest to mess up.

Also, this week features we have Kim Pries and his Software Sensei column.  Kim discusses what it takes to move toward mastery. Mastery implies more than just being good at any particular task. The Software Sensei provides a path forward.

Gene Hughson brings the first of his discussions on the topic of #NoEstimates from his Form Follows Function blog! Specifically Gene provided a more detail and background on his essay #NoEstimates – Questions, Answers, and Credibility.

Call to Action!

Review the SPaMCAST on iTunes, Stitcher or your favorite podcatcher/player and then share the review! Help your friends find the Software Process and Measurement Cast. After all, friends help friends find great podcasts!

Re-Read Saturday News

Remember that the Re-Read Saturday of The Mythical Man-Month is winding down. THis week we are running a quick poll to identify the next book  on the Software Process and Measurement Blog.  What would you like the next re-read book to be?

Upcoming Events

Details on 2016 Conferences that include QAI Quest and StarEast to name a few in a few weeks.


The next Software Process and Measurement Cast will feature a discussion with Steve Tendon discussing more of his new book Tame The Flow, and more. (Subject to change due to the Thanksgiving holiday in the States).

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects

should not be a tedious process, neither

for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

Categories: Process Management