Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

Stuff The Internet Says On Scalability For February 10th, 2017

Hey, it's HighScalability time:


It was a game of drones.
If you like this sort of Stuff then please support me on Patreon.
  • Half a trillion: Apple’s cash machine; 4,000-5,000: collected data points per adult in US; 10 million: gallons of gas UPS saves turning right; 2.27: Tesla 0-60 time; 40: complex steps to phone security; $2.3 billion: VR/AR investment in 2016; 18%: small players make up public cloud services market; 500°C: first chip to survive on Venus; 5 billion: ever notes; 375,000: images from The Metropolitan Museum of Art in public domain; 18 million: queries per minute against Facebook's Beringei database; 159: jobs per immigrant founder; 2.5 miles: whales breach for stronger signal; 10,000x: computers faster in 2035; 

  • Quotable Quotes: 
    • @martin_casado: Chinese factory replaces 90% of human workers with robots. Production rises by 250%, defects drop by 80%
    • Jure Leskovec: It’s [trolling] a spiral of negativity. Just one person waking up cranky can create a spark and, because of discussion context and voting, these sparks can spiral out into cascades of bad behavior. Bad conversations lead to bad conversations. People who get down-voted come back more, comment more and comment even worse.
    • sudhirj: The first concrete thing I learnt is this - implement pull first, it works 100% of the time, but may be inefficient with regards to time. Then implement push, it works 99% of the time but is much faster. But always have both running.
    • Tom Randall: California’s goal is considerable, but it’s dwarfed by Tesla’s ambition to single-handedly deliver 15 gigawatt hours 1 of battery storage a year by the 2020s—enough to provide several nuclear power plants–worth of electricity to the grid during peak hours of demand
    • @aphyr: Like I can't show that it's 100% correct, but so far I haven't found a way to break 3.4.0. Opens up a bunch of new use cases for MongoDB.
    • Azethoth666: The coming fast non-volatile memory architectures will be interesting. Everything will be in memory, but it will not go away. The infection cycle will have to clean up after itself or remain in the super fast volatile memory parts.
    • StorageMojo: In five years the specter of AWS cloud dominance will be a distant memory. The potential cloud market is enormous and we are, in effect, where the computer industry was in 1965. AWS will be successful, just not dominant. No tears for AWS.
    • @johnrobb: ~ 'Bots make public conversation a synthetic conversation. This makes it very difficult to know what consensus looks like.
    • W. Daniel Hillis: One day when I was having lunch with Richard Feynman, I mentioned to him that I was planning to start a company to build a parallel computer with a million processors. His reaction was unequivocal, “That is positively the dopiest idea I ever heard.”
    • @supershabam: Every database is a message bus if you try hard enough
    • mlechha: Boltzmann machines are a stochastic version of the Hopfield network. The training algorithm simply tries to minimize the KL divergence between the network activity and real data. So it was quite surprising when it turned out that the algorithm needed a "dream phase" as they call it. Francis Crick was inspired by this and proposed a theory of sleep.
    • @benjammingh: OH "Docker is Latin for a fire consisting predominantly of tires
    • UweSchmidt: "Real" bitcoining doesn't use services like coinbase; the coins are on your computer which you have to secure yourself. At least this is what you get told in cryptocurrency forums when one of the exchanges get hacked.
    • @axleyjc: 'Think of your System as a "Set of annotated request trees"' to manage microservice complexity @adrianco @ExpediaEng
    • @happy_roman: VW CEO on Tesla: "We'll win in the end, because of our abilities to scale & spread production."
    • aaron bell: Whichever cloud provider you pick based on your needs and their specific offering, I beg of you‚Ää—‚Ääplease don’t try hybrid
    • zebra9978: Kubernetes introduces a lot of upfront complexity with little benefit sometimes. For example, kargo is failing with Flannel, but works with Calico (and so on and so forth). Bare metal deployments with kubernetes are a big pain because the load balancer setups have not been built for it - most kubernetes configs depend on cloud based load balancers (like ELB). In fact, the code for bare metal load balancer integration has not been fully written for kubernetes.
    • a13n: This is huge. 87-99% shared code between iOS and Android. Someday companies as big as Instagram won't need to have entire separate product teams for separate platforms.
    • David Rosenthal: I've always said that the chief threat to digital preservation is economic; digital information being very vulnerable to interruptions in the money supply. 
    • YZF: There are no channels [in C++], there are no lightweight/green threads, there's no standard HTTP library, no standard crypto libraries, no standard test framework. For certain classes of applications this makes Go significantly more productive and significantly less bug/error prone. Not to mention compile times.
    • jdwyah: Kinesis firehose to S3 and then query with Athena is pretty great. I've been very happy with the combo.
    • mcherm: Your example from RethinkDB really struck home to me. The idea that superior technology might lose out due to poor marketing or (in this case) a system that is optimized for the real world rather than being optimized for benchmarks really disturbs me.
    • Aras Pranckevińćius: Moral of the story is: code that used to do something with five things years ago might turn out to be problematic when it has to deal with a hundred. And then a thousand. And a million. And a hundred billion. Kinda obvious, isn’t it?
    • kordless: I've come to a hypothesis that technology's purpose is to gently erode the concept of "self"
    • Microsoft: Close to a year ago we reset and focused on how we would actually get Git to scale to a single repo that could hold the entire Windows codebase (include estimates of growth and history) and support all the developers and build machines.
    • XorNot: I've run extensive benchmarks of Hadoop/HBase in Docker containers, and there is no performance difference. There is no stability difference (oh a node might crash? Welcome to thing which happens every day across a 300 machine cluster). Any clustered database setup should recover from failed nodes. Any regular relational database should be pretty close to automated failover with replicated backups and an alert email. Containerization doesn't make this better or worse, but it helps a lot with testing and deployment.
    • Dan Luu: When I was at Google, someone told me a story about a time that “they” completed a big optimization push only to find that measured page load times increased. When they dug into the data, they found that the reason load times had increased was that they got a lot more traffic from Africa after doing the optimizations. The team’s product went from being unusable for people with slow connections to usable, which caused so many users with slow connections to start using the product that load times actually increased.

  • There's a quintessential Silicon Valley moment in The Founder, a movie about the more interesting than expected McDonald's origin story. Brothers Mac and Dick McDonald kicked around from startup to startup. Nothing stuck. Drive-ins ruled the day, but were ripe for disruption. They were slow, used lots of servers, had too many options, attracted the wrong user base, and often delivered the wrong results. Metrics told them users mostly bought burgers, fries, and milkshakes. So the brothers decided to completely rethink how burgers were made and sold. What they came up with disrupted the food industry: a serverless drive-in based on a new low latency pipeline for making burgers called the Speedy System. An order was delivered within 30 seconds of being made; metrics helped control the latency distribution. Here's a short vignette showing how it was done. You'll love it. They traced the exact dimensions of the kitchen and conducted numerous simulations to figure out the optimal configuration. Users walk up to the window to order, so no servers. The API was narrow, only a few items could be ordered. Waste was reduced because utensils were done away with using innovative packaging design. Automation and a proprietary tool chain delivered a consistent product experience. And as often happens in Silicon Valley the founders were out maneuvered. While the McDonald brothers innovated the tech, Ray Kroc innovated the business model. Guess who ended up with everything? 

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Why I Use a Paper Kanban Board

My most recent post about how to Visualize Your Work So You Can Say No showing a couple of different kanbans was quite popular. Several people ask me how I use my personal kanban.

I use paper. Here’s why I don’t use a tool:

  • I am too likely to put too much into a tool. I put all this week’s work, next week’s work, next month’s and next year’s work, even though I’m not going to think about anything that far out. Paper helps me contain my To Do list.
  • When I collaborate with others, they want to break down the stories (large as they may be) into tasks. No!! I can’t take tasks. I need to see the value. See my post about From Tasks to Stories with Value.
  • I change my board, depending on what’s going on. I often have a week-based kanban because I retrospect at the end of the week. I often—not always—have a “today” column.

This is what my board looks like this week. it might look like this for a while because I’m trying to finish a book. (I have several more books planned, so yes, I will have a bunch of work “in progress” for the next several months/rest of the year.)

I have several chapters in This Week. I have two chapters in “Today:” That helps me focus on the work I want to finish this week and today.¬†As a technical editor for and as a shepherd for XP 2017, I have work in “Waiting to Discuss.” I¬†will discuss other people’s writing.

Earlier this week, I had interactions with a potential client, so that work is now in Waiting for Feedback. Sometimes, I have book chapters there, if I need to discuss what the heck goes in there and doesn’t go in a chapter.

I haven’t finished much yet this week. I am quite close on two chapters, which I expect to finish today. My acceptance criteria is ready for my editor to read. I do not expect them to be done as in publishable. I’ll do that after I receive editorial feedback.

Could I do this on an electronic board? Of course.

However, I limit my WIP by staying with paper. I can’t add any more to the paper.

Should I have WIP limits? Maybe. If I worked on a project, I would definitely have WIP limits. However, the fact that I use paper limits what I can add to my board. If I notice I have work building up in any of the Waiting columns, I can ask myself: What can I do to move those puppies to Done before I take something off the Today or To Do columns?

I’ve been using personal kanban inside one-week iterations since I read Benson’s and Barry’s book, Personal Kanban.¬†(See my book review,¬†Book Review: Personal Kanban: Mapping Work | Navigating Life.

I recommend it for a job search. (See Manage Your Job Search and  Personal Kanban for Your Job Hunt.

You should use whatever you want as a tool. Me, I’m sticking with paper for now. I don’t measure my cycle time or lead time, which are good reasons to use an electronic board. I also don’t measure my cumulative flow, which is another reason to use a board.

I do recommend that until you know what your flow is, you use paper. And, if you realize you need to change your flow, return to paper until you really understand your flow. You don’t need a gazillion columns, which is easy to do in a tool. Use the fewest number of columns that help you achieve control over your work nad provide you the information you need.

Question for you: Do you want to see a parking lot board? I have images of these in Manage Your Project Portfolio, but you might want to see my parking lot board. Let me know.

Categories: Project Management

Migrate a complete Git Repository from TFS to VSTS

Xebia Blog - Fri, 02/10/2017 - 11:10
I am migrating some Git repositories from a TFS server to VSTS. Moving a Git repo including all the history is very simple fo course. Since Git is a Distributed Source Control repository. But I always forget the syntax of moving the “complete” Git repository at once. With all the branches, tags etc. So here

Android Wear 2.0 is here with new hardware features!

Android Developers Blog - Fri, 02/10/2017 - 01:16

Posted by Hoi Lam, Lead Developer Advocate, Android Wear

Today, we are releasing the final SDK for Android Wear 2.0. In this release, we have added support for the new hardware features announced yesterday. If you have not done so already, it really is time to publish your apps so as to not miss the consumer hardware launch tomorrow.
Throughout the developer preview program, you have given us a lot of constructive feedback as well as bug reports. Thank you again!

Android Wear 2.0 recap

Android Wear 2.0 is our biggest update since we launched Wear in 2014, with numerous platform and developer enhancements. Some of the highlights include:
  • Material Design for Android Wear - A new system user interface and design guidelines, featuring a darker colour palette, vertical layout and visual components such as the WearableRecyclerView and WearableNavigationDrawer. We have also enhanced notifications on the watch with the new MessagingStyle rich notification style and inline actions.
  • Watch Face Complications - Complications are areas of the watch face that display information other than time. Apps can supply data to supported watch faces by creating a ComplicationProviderService, and watch faces can render this data in a style that suits the watch face design.
  • Standalone Android Wear apps and iOS support - Apps can now be downloaded directly to Wear devices via an on-watch Google Play Store. In addition, these apps can access the internet directly without relying on phone apps. This means that apps can now run on Android Wear devices that are paired to iOS devices.
New hardware supportThe first two watches with Android Wear 2.0 give users more ways to interact with their smartwatches. In the final SDK, we have added API support for physical button locations and rotary input. At present, developers will need the new LG Watch Style or LG Watch Sport to test these new functionalities; however, we are working to add these new hardware features to the emulator. Stay tuned for updates! The SDK also includes a few other final bug fixes, such as support for more than three items in the Wearable Action Drawer.
App review changesNow that Android Wear 2.0 is live, we'll soon update the Android Wear App Quality review process with two important changes. First, enhancing your phone app notifications for Android Wear will no longer be sufficient for passing the review. Second, it will soon be required that you upload a watch APK that's compatible with Android Wear 2.0. Only apps that pass these criteria will receive badging in Play Store on the phone and be eligible for top charts for Android Wear apps. These changes will ensure a more consistent experience for users and allow us to streamline the review process for you.
The journey doesn't stop here! The Android Wear 2.0 developer preview lasted longer than we originally planned, but we think that the extra time has paid off in a big way. Thank you once again for your input and patience. You helped us achieve a higher quality bar than we could have achieved on our own.

We have integrated the Android Wear 2.0 Developer Preview documentation into the main Wear developer documentation site and we continue to maintain links to the factory images for the preview devices. For developer related bugs, please continue to file developer bug reports or post comments in our Android Wear Developers community.

From the Android Wear team: Thank you again for your feedback and support!

Categories: Programming

Impact of Leadership Types on Adopting and Staying Agile


How does an organization‚Äôs leadership style affect the adoption of Agile. The focus of the question oft-times begins as a question about teams, which I generally steer to a discussion of the tendency of the organization or at least the senior leadership. The organization‚Äôs leadership culture (usually the same as the senior leaders’) are a leading indicator of whether Agile can take root and grow. There are numerous leadership styles, some are more conducive to adopting and keeping organizations Agile. If we consider ten of the more prevalent leadership styles, there are some that are conducive to Agile and some that are downright hostile. ¬†¬†

The styles with the most filled in Harvey Balls are typically the most conducive to adopting and sustaining Agile.

Leadership Styles

Leadership Styles

The worst of the bunch is autocratic leadership. It tends to follow a path that can be best charted as ‘my way or the highway,’ which is an anathema to self-organization and the flexible structure of Agile. ¬†Alternately democratic/participative, servant and transformational leadership tend to be the best of the bunch because they¬†clash less with overall agile values.

A more in-depth evaluation of the first four follows.

Autocratic Leadership

Autocratic leadership is a form of leadership in which a leader exerts high levels of power over his or her employees or team members. Team members (loosely used) are given few opportunities to make suggestions or decisions even in scenarios where it would be advantageous to the team or organization for making suggestions. (Listen to Gene Hughson SPaMCAST 430 describe in more detail why this form of leadership is dangerous.) Agile principles are designed to empower the team to make decisions and to foster collaboration.  Agile principles will always be at odds with autocratic leadership.  In this environment Agile generally, will not flourish and will only take hold in areas that are cut off from autocratic managers.

Bureaucratic Leadership

Bureaucratic leaders explicitly follow the book or the process. This style of work makes sense in some specific areas, for example pilots reviewing the preflight checklist or handling of hazardous chemicals. However, in most cases developing, enhancing and maintain software does not fall into this category. ¬†I use the qualifier, ‚Äúmost‚ÄĚ in a cautious manner, there are lifecycle activities that can be improved by some level bureaucratic management. For example, the enforcement of code check-in before implementation, safety and security testing of software and other operational activities (many of these activities can and should be automated to enforce the process). ¬†In most cases, bureaucratic leadership will smother innovation. Implementing concepts such as backlogs with their emergent qualities or continuous re-planning will be difficult under bureaucratic leadership. ¬†Bureaucratic leadership is bad for Agile, but not quite as bad as autocratic leadership.

Charismatic Leadership

Charismatic leadership injects huge doses of enthusiasm into his or her team and is very energetic in driving others forward. If the leader is committed to Agile then this form of leadership is decidedly a plus (thus the mostly filled in Harvey Ball).  The downside of this style is that charismatic leadership tends to foster a cult of personality in which the leader and his or her passions are more important than the team. Further, this form of leadership puts programs, such as an Agile transformation, at risk when the leader changes position.  When the leader leaves, the program can no longer draw on the charisma for energy.  I have seen programs of all types falter when the charismatic leader that championed and lead the implementation is promoted or takes a position outside of the organization.  As a change agent, use this type of leadership to your advantage but recognize the risk and develop other leaders to fill the gap when the leader moves on.

Democratic Leadership or Participative Leadership

Democratic/participative leadership invites other members of the team to contribute to the decision-making process. This form of leadership recognizes that team members bring knowledge, expertise and other points-of-view to the decision-making process. Team members are empowered, therefore they are motivated to work toward the team’s goals. This form of leadership is very conducive to adopting and embracing the Agile principles.  The downside of this type of leadership is that when a crisis occurs the decision-making process can be slower than autocratic forms of leadership.

Almost every organization wants to adopt or perhaps experiment with Agile.  Agile principles and many of the common leadership practices aren’t compatible.  A number of years ago I observed a leader that pushed his teams to adopt Agile (Scrumban in this case), only to turn up every morning at stand-up meetings to assign tasks to each person on each team.  The problem was not that he and the teams did not understand Agile but rather his management style was at odds the how Agile really works.

Next  РThe Next Six Leadership Styles

Categories: Process Management

Welcome New Host

We’re pleased to welcome¬†Edaena Salinas to the SE Radio team. Edaena is¬†a software engineer at Microsoft where she has worked on front-end and back-end development for web applications. In 2017, she started working in the Knowledge and Technologies group at Microsoft Research. Her technical¬†interests include software engineering, web development, artificial intelligence, testing, and DevOps. She […]
Categories: Programming

Visualize Your Work So You Can Say No

Most people I know—even the people supposedly using agile—have too much work to do. You have project work. You have support work, formal for customer support or sales, and informal for your colleagues. You have reports to write or file, time cards to fill out, or other periodic events.

You need a way to say no to more work.

I wrote an article for Better Software, which is now online. See Saying No to More Work. (You need to register for the site to see the article. No charge for registration.)

One person wanted to see the kanban boards I referred to in the article. I am happy to show them to you here.

These are two potential kanban boards. The one on the left is the basic personal kanban board. Note that there are no WIP (Work in Progress) limits (yet) on this board. I would add WIP limits. Especially if I wanted to convince my manager I was doing too much work.

On the right,  you can see a disaster of a personal kanban board. There are many items To Do, three in Progress and a total of six stuck in various Waiting states. Yes, I had a board that looked like this many years ago. Then, I made a picture on a piece of paper and explained to my boss I was just one person. What did he want me to do and not do?

Now, given what I know, I would add WIP limits to each column.

If you want to see the project portfolio images for how I start at the organization level: the calendar view and kanban view, see¬†Manage Your Project Portfolio at the Prags. At the end of the blurb, there’s a link to the quick start guide, which has just two of the images in the book. (I included many possible ways to visualize the project portfolio in this edition of the book.)

Here’s the key idea: Don’t take on more work than you can complete in a reasonable amount of time. Don’t multitask. Instead, see what your work is and how you might discuss it with your manager.

Categories: Project Management

Android Things Developer Preview 2

Android Developers Blog - Thu, 02/09/2017 - 18:01

Posted by Wayne Piekarski, Developer Advocate for IoT

Today we are releasing Developer Preview 2 (DP2) for Android Things, bringing new features and bug fixes to the platform. We are committed to providing regular updates to developers, and aim to have new preview releases approximately every 6-8 weeks. Android Things is a comprehensive solution to building Internet of Things (IoT) products with the power of Android. Now any Android developer can quickly build a smart device using Android APIs and Google services, while staying highly secure with updates direct from Google. It includes familiar tools such as Android Studio, the Android Software Development Kit (SDK), Google Play Services, and Google Cloud Platform. Android Things supports a System-on-Module (SoM) architecture, where a core computing module can be initially used with development boards and then easily scaled to large production runs with custom designs, while continuing to use the same Board Support Package (BSP) from Google.
New features and bug fixes
Thanks to great developer feedback from our Developer Preview 1, we have now added support for USB Audio to the Hardware Abstraction Layer (HAL) for Intel Edison and Raspberry Pi 3. NXP Pico already contains direct support for audio on device. We have also resolved many bugs related to Peripheral I/O (PIO). Other feature requests such as Bluetooth support are known issues, and the team is actively working to fix these. We have added support for the Intel Joule platform, which offers the most computing power in our lineup to date.
Native I/O and user drivers There are many developers who use native C or C++ code to develop IoT software, and Android Things supports the standard Android NDK. We have now released a library to provide native access to the Peripheral API (PIO), so developers can easily use their existing native code. The documentation explains the new API, and the sample provides a demonstration of how to use it.
An important new feature that was made available with Android Things DP1 was support for user drivers. Developers can create a user driver in their APK, and then bind it to the framework. For example, your driver code could read a GPIO pin and trigger a regular Android KeyEvent, or read in an external GPS via a serial port and feed this into the Android location APIs. This allows any application to inject hardware events into the framework, without customizing the Linux kernel or HAL. We maintain a repository of user drivers for a variety of common hardware interfaces such as sensors, buttons, and displays. Developers are also able to create their own drivers and share them with the community.
TensorFlow for Android Things One of the most interesting features of Android Things is the ability to easily deploy machine learning and computer vision. We have created a highly requested sample that shows how to use TensorFlow on Android Things devices. This sample demonstrates accessing the camera, performing object recognition and image classification, and speaking out the results using text-to-speech (TTS). An early-access TensorFlow inference library prebuilt for ARM and x86 is provided for you to easily add TensorFlow to any Android app with just a single line in your build.gradle file.

TensorFlow sample identifying a dog's breed (American Staffordshire terrier)  on a Raspberry Pi 3 with camera
Feedback Thank you to all the developers who submitted feedback for the previous developer preview. Please continue to send us your feedback by filing bug reports and feature requests, and ask any questions on stackoverflow. To download images for Developer Preview 2, visit the Android Things download page, and find the changes in the release notes. You can also join Google's IoT Developers Community on Google+, a great resource to keep up to date and discuss ideas, with over 2900 new members.

Categories: Programming

Quote of the Month February 2017

From the Editor of Methods & Tools - Thu, 02/09/2017 - 16:40
One of the most important expectations to set early on is about involvement in the process. Many project stakeholders accustomed to traditional-style development view their role in a software development project as akin to dropping a car off for service: You tell someone what you need done and come back at appointed time to pick […]

Running Windows Containers on Azure Service Fabric - Part II

Xebia Blog - Thu, 02/09/2017 - 12:39
The previous post showed how you can create an unsecure Service Fabric test cluster in Azure, and how to run a Windows Container on it. In this follow up post, I'll show you what's going on inside the cluster, using the Docker command line. Knowledge about this can be very useful when troubleshooting. Verify your container

Running Windows Containers on Azure Service Fabric

Xebia Blog - Thu, 02/09/2017 - 12:37
Background Since the release of Service Fabric runtime version 5.4.145, Microsoft added a (preview) feature to run Windows Containers on Windows Server 2016. The Linux version already supported this for a while. This post explains why Containers are useful and how to get it to work. What is Service Fabric? Most companies have a lot of applications

Copy & paste driven development

Actively Lazy - Wed, 02/08/2017 - 22:40

Software development is rife with copy & paste: all of us resort to copy and paste coding sometimes. We know we probably shouldn’t, but we do
it anyway. It’s like the industry’s dirty little secret: we mainly jucopy-paste-naileditst copy and paste code from the internet or from somewhere else in the code base then bash it till it works.

But maybe, just maybe, the fact that we all rely on this from time to time should be telling us something?

The good

Sometimes copy & paste coding can be a good thing. A while ago I was pairing with someone where we did what I would call “search and replace coding”. I love to code golf. In tools like Eclipse, Intellij or Resharper there is always an optimal way to make each change, letting the tools do as much of the typing as possible. So it was with fascination recently that a colleague showed me an interesting code golf using search & replace.

The change was around extending an existing class with a load of new fields. We had a basic class, with a couple of sample fields, and a test that verified something simple about the class – say that it could be serialized to JSON successfully. We needed to add a boatload of new fields to the class (don’t ask). This involved five separate tasks which, at a macro level, had a lot in common:

  • Adding each field
  • Initialising each field in the constructor
  • Modifying the test to setup sample values for each field
  • Modifying the test to pass the sample value for each field to the constructor
  • Writing an assert that each field had successfully serialized and deserialized

My colleague demonstrated that we could write the list of fields once. Then, copy & paste, search & replace – we have a list of parameters for the constructor. By carefully crafting the search term and a suitable replacement term you can do some limited meta-programming. Taking the list of fields and transforming it into the actual lines of code you need in each instance. The list of fields replaced one way gives you a list of parameters to the constructor; with a different replacement you get the field declarations in the test; with another replacement you get assert statements.

I found this a very interesting way to write software. It definitely optimized the number of key presses required to type the code in. The long, boring list of field names only had to be typed in once; after that merely carefully crafting a search & replace regex to do the lifting for us. But it demonstrates an underlying truth: we had five separate changes to make, which accepted a parameter. If we could have actually meta-programmed this, we would have passed in the list of fields to some meta-programming code which would output the desired lines of code.

While it’s very¬†clever that we could use search & replace meta-programming for this, it feels like the tools are lacking somehow.

The bad

Everyone copies code from stackoverflow from time to time. Hopefully you do it sensibly, using it as a starting point for your own production code. Working through what you’ve just pasted in to understand it, experimenting with it, modifying it, making it fit for your purpose. Rather than just blindly copy & pasting random code from the internet. I mean, who’d just randomly run code from the internet?

Stackoverflow is great. It’s an amazing resource for programmers. While learning WPF I got the sense that as a technology it couldn’t have taken off without something like stackoverflow. The technology is so opaque, so hard to learn. It took me many, many months of copy & pasting xaml from stackoverflow until I started to really understand how it worked. This is a technology that is not trivial to understand. Without being able to just copy & paste code from the internet, the technology would take a lot longer to learn.

But often, we’re lazy, we try it. It works. Woohoo, next problem. I’ve written before about voodoo programming, but the problem is that it’s easy to¬†think you’ve understood what code is doing. If you didn’t actually have to invent those lines, to reason through to them – maybe you don’t understand. Maybe you only think you know what the code’s doing? Maybe there’s some horrific bug you’re not aware of yet?

The ugly

Almost every time I’ve seen TDD done at any scale, when it comes to writing a new test, the first question to ask is: “which existing test is this most like?” Yup, where can I go copy & paste. I’m so lazy, I don’t want to have to invent a whole test setup on my own. I’ll just borrow somebody else’s homework. We’ve all done it. It seems to be an unwritten rule of TDD. By the time you’re on to the third or fourth test in a given file, I guarantee the temptation to just copy & paste to make the fifth test is¬†incredibly strong.

The trouble with this is all sorts of weird test artefacts get copied forwards. You start off with a simple test, with some simple setup. Say an empty bank account. Then to test non-zero balance you need an account with a transaction. Then to test balance summing you need an account with two transactions. Each test so far is building on the last, so you’ve just copy & pasted the previous one as a starting point. The fourth test is inserting a new transaction, so you just copy the third test (with two existing transactions, unnecessary for this test). Test five¬†is that a withdrawal can’t go below zero, so you copy & paste the previous test and set the amount to be a large negative value. Test six is an overdraft test, so you copy & paste the previous test but change the account setup to include an overdraft so the balance can go negative. By the time you copy & paste test seven, your¬†starting point¬†is an account with an overdraft, with two transactions where a third transaction is added. Test seven is about adding a standing order. None of this noise is necessary.

This might seem a ridiculous example, but I see this time and again in real code. Sometimes it’s not even me that’s done it. This noise accumulates as you read down a test file. The tests at the bottom have all sorts of weird artefacts that were only relevant to one test half way up the file. It means fixing tests and changing behaviours becomes a real problem. If I change, say, how overdrafts are defined in my example above – I might have half a dozen tests to change, only one of which even mentions overdrafts. But they’ve inherited that setup because the tests were just copy & pasted around. Not only does this make tests hard to read and understand it makes them hard to maintain.

We all do it. It seems a pretty accepted part of how TDD happens in the wild. And yet, it clearly isn’t right. With discipline, we can keep our tests clean. Yes, when we’re all being conscientious developers we start writing our tests from scratch each time. But most of the time we’re busy or lazy or whatever.


What do these three things have in common, besides copy & paste? In each example we’re using copy & paste to save time. To find the most efficient path through the work we’re doing. Nobody is doing it out of malice or stupidity. Laziness? Almost certainly – but the good kind. The kind of laziness that encourages elegant solutions.

But copy & paste isn’t an elegant solution. It’s a crappy solution to a more fundamental problem: our tools are deficient. Really what we’re working around is the fact that our tools don’t let us express what we really want.

What if we could write our tests in a higher level language? “A test with a bank account with two transactions”. Sure, there are internal & external DSLs you can use to do this. But typically the cost in setting up the DSL isn’t worth the hassle for¬†unit tests. It would completely ruin the flow of TDD. Does that just mean we haven’t found a good way of doing it yet? Is there a way we could more fluidly express the intent of the test, filling in the gaps as we go?

Instead of copy & pasting code from the internet, could our tools get smarter? Could we take some of the amazing machine learning stuff that’s going on and apply it to software development? I tried playing with an IntelliJ plugin recently that promises just that. Unfortunately it’s pretty buggy at the minute and doesn’t really work. But the¬†idea is incredibly attractive. I like the idea of being able to express intent instead of mindlessly typing in the nuts & bolts.

Finally, instead of doing search & replace coding, wouldn’t it be great if we could actually meta-program? If we could actually write code that would write code for us? Not just code generators, but something that can generate small sections of code. I had a very limited go at this some time ago with rescripter – but it turns out its very hard to write a decent meta-programming tool, that anyone except the author can understand. But I think the idea still has merit: too often I find cases where I can describe the intent of my change very succinctly, but implementing it will involve far more typing than I’d like.

Categories: Programming, Testing & QA

G Suite Developer Sessions at Google Cloud Next 2017

Google Code Blog - Wed, 02/08/2017 - 19:18
Originally posted on the G Suite Developers Blog

Posted by Wesley Chun (@wescpy), Developer Advocate, G Suite

There are over 200 sessions happening next month at Google Cloud's Next 2017 conferencein San Francisco... so many choices! Along with content geared towards Google Cloud Platform, this year features the addition of G Suite so all 3 pillars of cloud computing (IaaS, PaaS, SaaS) are represented!

There are already thousands of developers including Independent Software Vendors (ISVs) creating solutions to help schools and enterprises running the G Suite collaboration and productivity suite (formerly Google Apps). If you're thinking about becoming one, consider building applications that extend, enhance, and integrate G Suite apps and data with other mission critical systems to help businesses and educational institutions succeed.

Looking for inspiration? Here's a preview of some of the sessions that current and potential G Suite developers should consider:

The first is Automating internal processes using Apps Script and APIs for Docs editors. Not only will you hear directly from senior engineers on the Google Sheets & Google Slides REST API teams, but you'll also find out how existing customers are already doing so! Can't wait to get started with these APIs? Here are the intro blog post & video for the latest Google Sheets API as well as the intro blog post & video for the Google Slides API. Part of the talk also covers Google Apps Script, the Javascript-in-the-cloud solution that gives developers programmatic access to authorized G Suite data along with the ability to connect to other Google and external services.

If that's not enough Apps Script for you, or you're new to that technology, swing by to hear its Product Manager give you an introduction in his talk, Using Google Apps Script to automate G Suite. If you haven't heard of Apps Script before, you'll be wondering why you haven't until now! If you want a headstart, here's a quick intro video to give you an idea of what you can do with it!

Did you know that Apps Script also powers "add-ons" which extend the functionality of Google Docs, Sheets, and Forms? Then come to "Building G Suite add-ons with Google Apps Script". Learn how you can leverage the power of Apps Script to build custom add-ons for your business, or monetize by making them available in the G Suite Marketplace where administrators or employees can install your add-ons for their organizations.

In addition to Apps Script apps, all your Google Docs, Sheets, and Slides documents live in Google Drive. But did you know that Drive is not just for individual file storage? Hear directly from a Drive Product Manager on how you can, "[Use] Drive APIs to customize Google Drive." With the Drive API and Team Drives, you can extend what Drive can do for your organization. One example from the most recent Google I/O tells the story of how WhatsApp used the Drive API to back up all your conversations! To get started with your own Drive API integration, check out this blog post and short video. Confused by when you should use Google Drive or Google Cloud Storage? I've got an app, err video, for that too! :-)

Not a software engineer but still code as part of your profession? Want to build a custom app for your department or line of business without having to worry about IT overhead? You may have heard about Google App Maker, our low-code development tool that does exactly that. Curious to learn more about it? Hear directly from its Product Manager lead in his talk entitled, "Citizen developers building low-code apps with App Maker on G Suite."

All of these talks are just waiting for you at Next, the best place to get your feet wet developing for G Suite, and of course, the Google Cloud Platform. Start by checking out the session schedule. Next will also offer many opportunities to meet and interact with industry peers along with representatives from all over Google who love the cloud. Register today and see you in San Francisco!

Categories: Programming

From Tasks to Stories with Value

I’m almost at the end of the January Practical Product Owner workshop. One of the participants has a problem I’ve seen before. They have a backlog of work, and it’s all tasks. Not a story in sight.

I understand how that happens. Here are some ways I’ve seen the tasks-not-stories problem occur:

  • The technical people see the work they want to accomplish. They create a list of tasks to get there: design database, create infrastructure, fix these typos in the UI, and more.
  • Or, someone (such as an architect) is in charge of breaking down work, not team members. That person creates tasks.
  • Marketing or sales (or someone not in the product development team) says something like this: “I want a drop-down menu,” or a radio button or another¬†report. They don’t remember to explain who the value is for, or why they want that value. Pretty soon, the idea of value is gone altogether, and only tasks remain.
  • Teams start to create stories, and the stories are so large, they create tasks to cover the stories. Pretty soon, they stop creating stories at all. They only create tasks.

Here’s a gotcha: When teams measure story points as opposed to features, they often feel pressure from management to do more points. (See Who’s Playing Agile Schedule Games?)

Your customers don’t buy points. They buy completed features.

Something clicked for the participant last week. He saw the feature chart, which explains how scope expands during the project and what the team(s) delivers.

This chart shows the features complete, added, and remaining to do. Because it measures features—what customers buy and use—there’s no confusion about work done or not done. Plenty of work might be done. But, if the work doesn’t add to a feature, the work is inventory (or possibly waste).

Here’s one value of this chart: Until tasks add up to features, you don’t count them.

My participant couldn’t articulate the problems before. The chart helped him see and discuss:

  • Tasks—by themselves—might not add up to a feature. He wants features.
  • When he counts features, he can describe what’s in a feature set—a collection of features that you might call an epic or a theme.
  • He can explain why he wants just this small feature, and not necessarily all of a feature set for now.

The chart helped him see how to separate stories and count them. He is moving from tasks and technical stories to features, real user stories.

I use this chart with cumulative flow diagrams and the product backlog burnup chart to see where our work is and how much progress we make over time for a given feature set.

I recommend you build and count features (stories). The smaller you can make a story, the faster you can get feedback and see the value in it.

If you’re interested in this workshop, I have just announced the May 2017 dates. See Practical Product Owner Workshop: Deliver What Your Customers Value and Need.

Categories: Project Management

In-memory noSQL DBMS Client in Big Data Cluster

This is guest post by Sergei Sheinin, creator of the 2DX Web UI Database Cluster Framework, a low latency big data cluster with in-memory noSQL DBMS Web Browser client.

When I began working in the field of data management the disconnect between rigid structure of relational database tables and free form of documents managed by end users and their businesses stood out as a technical and managerial hurdle. On the one hand there were strict definitions of normalized relational database models and unstructured document formats on the other. Often the users in charge of changing document structures held organizational responsibilities far removed from database modeling or programming. On one occasion I was involved in a project where call center operators made on the fly decisions to update a document structure based on phone conversations with customers. Such updates had to be streamed into a relational back-end creating havoc in database structure and build of table columns.

In seeking a permanent solution I researched merits of Entity-Attribute-Value database schema and its applications. This technique proved successful in enabling front end users to modify relational-bound documents through performing updates to structure described in their metadata. However application of EAV raised its own issues, for example accommodation of updated document metadata at times required changes to definitions of the relational tables, attention of developers due to complexity of application layer in client-server interoperability, rapidly growing fact tables and performance of multiple join statements in select queries...

Categories: Architecture

Running Windows Containers on Azure Service Fabric ‚Äď Part II

Xebia Blog - Wed, 02/08/2017 - 09:27
The previous post showed how you can create an unsecure Service Fabric test cluster in Azure, and how to run a Windows Container on it. In this follow up post, I’ll show you what’s going on inside the cluster, using the Docker command line. Knowledge about this can be very useful when troubleshooting. Verify your container

Running Windows Containers on Azure Service Fabric

Xebia Blog - Wed, 02/08/2017 - 09:17
Background Since the release of Service Fabric runtime version 5.4.145, Microsoft added a (preview) feature to run Windows Containers on Windows Server 2016. The Linux version already supported this for a while. This post explains why Containers are useful and how to get it to work. What is Service Fabric? Most companies have a lot of

Types of Influence and Their Usefulness for Coaches


Coaches and change agents use many types of influence to help teams and organizations perform better as they lead.  Influence can be applied through a number of highly nuanced approaches. And like many activities, when you find success with one it is easy to fall into a trap of thinking that that approach will always work.  While sports analogies are often overdone, I will add one more to the pile before swearing them off (for this essay at least).  The Super Bowl, the pinnacle of US Football, was recently played and featured a come from behind victory. The New England Patriots won the game despite having many of their top receivers sidelined due to injury. If the Patriots had only one approach to the game based on that set of receivers they would have been blown out. A good coach will be able to leverage different forms of influence based on the context they find themselves face or be able to recognize when dangerous forms of influence are being used.  Recently I ran across a list of 7 approaches to influencing teams or organizations. Some of these approaches can be useful for coaches and some are harmful. The 7 forms of influence, some good and some bad, include:

  1. ¬†¬†¬†¬†¬†Eminence-based influence ‚Äď Seniority and experience add gravity to the words and suggestions made by a coach that relies on eminence-based influence. ¬†In the late 1990‚Äôs I had an interviewer suggest that my prematurely graying hair added a degree of eminence that I could leverage in consulting (I did not take that job).
  2. ¬†¬†¬†¬†¬†Eloquence-based influence ‚Äď Sartorial elegance and verbal eloquence are powerful tools for influencing the behavior of a team or organization. ¬†Polished style and stirring oratory is can be used to champion or support a cause. ¬†Eloquence is often very useful when sharing and shaping a vision for transformation, but somewhat less useful at the team level unless it conforms to the team‚Äôs culture.¬†
  3.      Evidence-based influence РData combined coupled with quantitative and other feedback approaches can provide valuable feedback for a team to evaluate their approach and behavior.  In transformations and team-level coaching, evidence often is used in combination with other forms of influence.
  4. ¬†¬†¬†¬†¬†Vehemence-based influence ‚Äď Brow-beating and bullying (intimidation) is a method of influence often practiced by poor or untrained coaches. ¬†The form of influence seldom has a lasting impact and causes passive aggressive behavior in the short run and a backlash when bully leaves or loses power.
  5. ¬†¬†¬†¬†¬†Providence-based influence ‚Äď Expecting Karma or some form of external intervention to change the trajectory of the team is a common approach for coaches that are out of their depth (whether based on skill, training or experience is not material). ¬†A coach needs to avoid assuming that an outside force will intervene to change the direction of the team or organization.
  6. ¬†¬†¬†¬†¬†Fear-based influence ‚Äď From time immemorial fear has been used as a tool to influence behavior. Often fear is generated by a lack of knowledge of the future because of risk or an impending event. Mergers or similar evens often generate fear that some leaders harness to guide behavior. ¬†In times of great peril, fear can be a powerful and valuable motivator; however, most of the time it will generate severe negative side effects. ¬†Use the fear or threats at your own great peril.¬†
  7. ¬†¬†¬†¬†¬†Confidence-based influence ‚Äď Confidence-based influence is based on trust and the perception that the team or organizations can rely on the advice and leadership being offered. ¬†This type of influence is similar to eminence-based influence (and perhaps supported by evidence). ¬†The downside of confidence-based influence is that if the influencer leaves, often so does his or her influence. ¬†Use this form of influence when needed, but make sure it is not focused on a single person.

Influence is key to leadership and change. There are lots of ways to generate influence some are better than others. Use multiple types of influence, but stay away from fear, bullying and don’t start a cult of personality. Influence used correctly is the grease that gets things done, but when used incorrectly leads to failure.


Categories: Process Management

SE-Radio Episode 281: James Whittaker on Career Strategy

Edaena Salinas talks with James Whittaker about Career Strategy in the technology field. James is a Distinguished Technical Evangelist at Microsoft. Throughout his career, he specialized in Software Testing, Security, and Storytelling. He is the author of ‚ÄúHow Google Tests Software‚ÄĚ and the viral blog post ‚ÄúWhy I left Google‚ÄĚ. Topics include: Career Management, the […]
Categories: Programming

Introducing Google Developers India: A Local Youtube Channel for India’s Mobile Development Revolution

Google Code Blog - Tue, 02/07/2017 - 21:36
Posted By Peter Lubbers, Senior Program Manager

Today, we're launching the Google Developers India channel: a brand new Youtube channel tailored for Indian Developers. The channel will include original content like interviews with local experts, developer spotlights, technical tutorials, and complete Android courses to help you be a successful developer.

Why India?

By 2018, India will have the largest developer base in the world with over 4 million developers. Our initiative to train 2 million Indian developers, along with the tremendous popularity of mobile development in the country and the desire to build better mobile apps, will be best catered by an India-specific developers channel featuring Indian developers, influencers, and experts.

Here is a taste of what's to come in 2017:
  • Tech Interviews: Advice from India's top developers, influencers and tech experts.
  • Developer Stories: Inspirational stories of Indian developers.
  • DevShow India: A weekly show that will keep new and seasoned developers updated on all the news, trainings, and API's from Google.
  • Skilled to Scaled: A real life developer journey that takes us from the germination of an idea for an app, all the way to monetizing it on Google Play.
So what's next?

The channel is live now. Click hereto check it out.

Categories: Programming