Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

First Steps in gRPC Bindings for React Native

Xebia Blog - Wed, 02/22/2017 - 13:54

When you want to use gRPC in your React Native app there is no official support yet, but that shouldn’t stop you! In this post I’ll show you how we designed an implementation with type safety in mind and successfully called a service remotely from React Native on Android. Read more

The post First Steps in gRPC Bindings for React Native appeared first on Xebia Blog.

Big Bang, Incrementalism, or Somewhere In Between

Line Segment Shutdown

Line Segment Shutdown

When making any significant change to a team or organization, deciding whether to take a big bang or incremental approach is important. Β Both of these approaches–and hybrids in between–can work. Β Big Bang and incremental approaches mark the two ends of a continuum of how organizations make a change. Β The decision is almost never straightforward and organizations often struggle with how they should approach change. Β The decision process begins by defining Big Bang and incremental implementation approaches in between the two ends so they can be compared.

Big Bang Implementations

A big bang adoption of a process or system is an instant changeover, a “one-and-done” type of approach in which everyone associated with a new system or process switches over in mass at a specific point in time. Β For example, most of the bank mergers I participated in were big bangs. Β The systems were all cut over on a specific date (lots of pizza and coffee was required for the cutover weekend) and the next business day all of the branches and ATMs began the day using a single system.

Big bangs are always the culmination of a lot of specific activities including planning, coordination, software changes, data conversions, and reviews. Β All of these activities are focused on making the Big Bang successful. Β Individually, the steps have little to no value if the final step fails.

Big Bang changes are sometimes equated to β€œbet the business” scenarios: if the change doesn’t Β work, everything needs to be backed out or significant business impact will ensue.

Incrementalism / Incremental Approach

An incremental approach focuses on defining identifying and implementing specific pieces of work. Β These pieces are generally smaller standalone pieces of work that progress an organization toward an overall goal but not generally all parts of one specific cohesive project. Β For example, quality or process programs often use a continuous process improvement model in which practitioners identify changes or improvements which are then captured as part of a backlog and prioritized for implementation. This type of work is sometimes called continuous process improvement. In this scenario, lots of individual pieces of work accumulate over time to deliver a big benefit. Incremental changes generate a fast feedback loop which delivers enhanced learning. The small changes typically found in incremental approaches are useful for experimentation.

In Between or Phased Implementations

The term phased adoption can have alternate meanings. Β The first (and in 2017 the most common meeting) is to break implementation into smaller pieces so that the organization has use of functionality sooner. Β This is closer to an incremental approach (next) than a big bang. Β Phased approaches break a bigger project into smaller projects so the adoption will happen in several steps. After each step, the system is a little nearer to be fully adopted. Phased differs from incremental generally in scope and the types of work in the backlog. Β For example, in bank mergers, one phase might be to convert checking accounts and trust accounts in another phase. Β In an Agile adoption, a phased approach might be to transform one team after another in a serial fashion.

The second possible use of the term is the famed waterfall approach in which analysis is completed before design all the way to implementation. This approach is far less common than it was in the late 20th century before the Agile movement, however, make sure you check by asking how the word phased is being used.

Which implementation approach makes the most sense will always depend on context. Β The right choice requires understanding the goal of the change, resources available to make the change and above all else the organization’s culture. Β The choice is not as stark as Big Bang (everything at once) or incrementalism (lots of continuous little changes) although these are the choices most often considered.

 

In the next entry in this theme, we will explore the pros of the Big bang approach.


Categories: Process Management

Friction in Software

Actively Lazy - Tue, 02/21/2017 - 21:39

Friction can be a very powerful force when building software. The things that are made easier or harder can dramatically influence how we work. I’d like to discuss three areas where I’ve seen friction at work: dependency injection, code reviews and technology selection.

DI Frameworks

A few years ago a colleague and I discussed this and came to the conclusion that the reason most DI frameworks suck (I’m looking in particular at you, Spring) is that they make adding new dependenciesΒ so damned easy! There’s absolutely no friction. Maybe a little XML (shudder) or just a tiny little attribute. It’s so easy!

So when we started a new, greenfield project, we decided to put our theory to the test and introduced just a little bit of friction to dependency injection. I’ve written before about the basic scheme we adopted and the AOP endpoint it reached. But the end result was, I believe, very successful. After a couple of years of development we still had of the order of only 10-20 dependencies. The friction we’d introduced was light (add a couple of lines to a single class), but it was sufficient to act as a constant reminder not to just add a new dependency because it wasΒ easy.

Code Reviews

I was reminded of this recently when discussing code reviews. I have mixed feelings about code reviews: I’ve seen them work well, and it is better to have code reviews than not to have them; but it’s better still to pair program. But not all teams, not all developers, like pair programming – so code reviews exist. The trouble with code reviews is they can provide a form of friction.

If you & I are pairing on a piece of work, we will discuss the various trade-offs as we go: do we spend time on this, do we refactor that, etc etc. The constant judgements about what warrants attention and what can be left for another day are verbalised and agreed. In general I find the code written while pairing is high in quality but also remains tightly focused on task. The long rambling refactors I’ve been guilty of in the past disappear and the lazy “quick hacks” we all try and explain away to ourselves, aren’t so easy to gloss over when pairing.

But code reviews exist outside of this dynamic. In the cold light of the following day, someone uninvolved reviews your work and passes judgement on whether they think it’s up to scratch. It’s easy to see why this becomes combative:Β rather than being collaborative it can be seen as a judgement being passed, on not only the code but the author, too.

When reviewing code it is easy to set a very high bar, higher than you might set for yourself and higher than you might have agreed when pairing. Now, does this mean the comments aren’t valid? Absolutely not! You’re right, there is a test case missing here, although my change is unrelated, I should have added the missing test case. And you’re right this code is a mess; it was a mess before I was here and made a simple edit; but you’re right, I should have tidied it up. Everyone should practice code gardening.

These are all perfectly valid comments. But they create a form ofΒ friction. When I worked on a team that relied on these code reviews you knew you were going to get comments: so you kept the commit small, so as to minimize the diff. A small diff minimizes the amount of extra tests you could be asked to write. A small diff keeps most of the existing mess out of the review, so you won’t be asked to start refactoring.

Now, this seems dysfunctional: we’re deliberately trying to optimize a smooth passage through the review process, instead of optimizing for code quality. Worse than this though was what never happened: refactoring commits. Looking back I realise that the only code reviews I saw (as both reviewer and reviewee) were for feature changes. There were never any code reviews submitted for purely technical debt reduction. Sure, there’d be some individual commits in amongst the feature changes. But never any dedicated, multi-commit sessions, whose sole aim was to improve the code base. Which was a shame, because like any legacy code base, there was scope for improvement.

Comparing this to teams that don’t do code reviews, whereΒ I’ve tended to see more effort on reducing technical debt.Β Without fearing an endless cycle of review comments, developers are free to embark on refactoring efforts (that may or may not even work out!) – but at least they can try. Instead, code reviews provide a form of friction that might actually hurt code quality in the long run.

Technology Selection

I was talking to another colleague recently who is convinced that Hibernate is still the best way to get data in and out of a relational database. I can’t really work out how to persuade people they’re wrong – surely using Hibernate is enough to persuade you? Especially in a large, legacy code base – the pain that Hibernate causes is obvious. Yet plenty of people still believe in Hibernate. There are even people that still believe in Spring. Whether or not they still believe in the tooth fairy is unclear.

But I think technology selection is another area where friction is important. When contemplating moving away from something well-known and well used in industry like Spring or Hibernate there is a lot of friction. There are new technologies to learn, new approaches to understand and new risks to manage. This all adds friction, so sometimes it’s easiest just to stick with what we know. Sometimes it really is the right choice – the technology you have expertise in is the one you’ll be most productive in immediately. But there are longer term questions too, which are much harder to answer: will the team eventually be more productive using technology X than technology Y?

Friction in software is a powerful process: we’re very lazy creatures, constantly trying to optimise. Anything that slows us down or gets in our way quickly gets side-stepped or worked around. We can use this knowledge as a tool to guide developer behaviour; but sometimes we need to be aware of how friction can change behaviours for the worse as well.


Categories: Programming, Testing & QA

Build flexible layouts with FlexboxLayout

Android Developers Blog - Tue, 02/21/2017 - 19:48
Posted by Takeshi Hagikura, Developer Programs Engineer

At Google I/O last year we announced ConstraintLayout, which enables you to build complex layouts while maintaining a flat view hierarchy. It is also fully supported in Android Studio's Visual Layout Editor.

At the same time, we open sourced FlexboxLayout to bring the same functionalities of the CSS Flexible Layout module to Android. Here are some cases where FlexboxLayout is particularly effective.

FlexboxLayout can be interpreted as an advanced LinearLayout because both layouts align their child views sequentially. The significant difference between LinearLayout and FlexboxLayout is that FlexboxLayout has a feature for wrapping.

That means if you add the flexWrap="wrap" attribute, FlexboxLayout puts a view to a new line if there is not enough space left in the current line as shown in the picture below.


One layout for various screen sizes With that characteristic in mind, let's take a case where you want to put views sequentially but have them move to new lines if the available space changes (due to a device factor, orientation changes or the window resizing in the multi-window mode).


Nexus5X portrait


Nexus5X landscape

Pixel C with multi window mode enabled, divider line on the left.

Pixel C with multi window mode enabled, divider line on the middle.

Pixel C with multi window mode enabled, divider line on the right.
You would need to define multiple DP-bucket layouts (such as layout-600dp, layout-720dp, layout-1020dp) to handle various screen sizes with traditional layouts such as LinearLayout or RelativeLayout. But the dialog above is built with a single FlexboxLayout.

The technique used in the example is setting the flexWrap="wrap" as explained above,

<com .google.android.flexbox.flexboxlayout 
     android:layout_width="match_parent" 
     android:layout_height="wrap_content" 
     app:flexwrap="wrap">
then you can get the following layout where child views are aligned to a new line instead of overflowing its parent.




Another technique I'd like to highlight is setting the layout_flexGrow attribute to an individual child. This helps improve the look of the final layout when free space is left over. The layout_flexGrow attribute works similar to the layout_weight attribute in LinearLayout. That means FlexboxLayout will distribute the remaining space according to the layout_flexGrow value set to each child in the same line.

In the example below, it assumes each child has the layout_flexGrow attribute set to 1, so free space will be evenly distributed to each of them.
 <android .support.design.widget.TextInputLayout
     android:layout_width="100dp"
     android:layout_height="wrap_content" 
     app:layout_flexgrow="1">



You can check out the complete layout xml file in the GitHub repository.
RecyclerView integration  Another advantage of FlexboxLayout is that it can be integrated with RecyclerView. With the latest release of the alpha version the new FlexboxLayoutManager extends RecyclerView.LayoutManager, now you can make use of the Flexbox functionalities in a scrollable container in much more memory-efficient way.

Note that you can still achieve a scrollable Flexbox container with FlexboxLayout wrapped with ScrollView. But, you will be likely to experience jankiness or even an OutOfMemoryError if the number of items contained in the layout is large, as FlexboxLayout doesn't take view recycling into account for the views that go off the screen as the user scrolls.

(If you would like to learn more about the RecyclerView in details, you can check out the videos from the Android UI toolkit team such as 1, 2)

A real world example where the RecyclerView integration is useful is for apps like the Google Photos app or News apps, both expect large number of items while needing to handle various width of items.

One example is found in the demo application in the FlexboxLayout repository. As you can see in the repository, each image shown in RecyclerView has a different width. But by setting the flexWrap setting to wrap,

FlexboxLayoutManager layoutManager = new FlexboxLayoutManager();
layoutManager.setFlexWrap(FlexWrap.WRAP);
and setting the flexGrow (as you can see, you can configure the attributes through FlexboxLayoutManager and FlexboxLayoutManager.LayoutParams for child attributes instead of configuring it from xml) attribute to a positive value for each child,
void bindTo(Drawable drawable) {
  mImageView.setImageDrawable(drawable);
  ViewGroup.LayoutParams lp = mImageView.getLayoutParams();
  if (lp instanceof FlexboxLayoutManager.LayoutParams) {
    FlexboxLayoutManager.LayoutParams flexboxLp = 
        (FlexboxLayoutManager.LayoutParams) mImageView.getLayoutParams();
    flexboxLp.setFlexGrow(1.0f);
  }
}
you can see every image fits within the layout nicely regardless of the screen orientation.



If you would like to see complete FlexboxLayout example, you can check:

What's next? Check out the full documentation for other attributes to build flexible layouts tailored for your needs. We're very open to hear your feedback, if you find any issues or feature requests, please file an issue on the GitHub repository.



Categories: Programming

Quote of the Day

Herding Cats - Glen Alleman - Tue, 02/21/2017 - 05:49

β€œI am not much given to regret, so I puzzled over this one a while. Should have taken much more statistics in college, I think.” β€” Max Levchin, Paypal Co-founder, Slide Founder

Perhaps anyone conjecturing that decisionsΒ can be made in the presence of uncertaintyΒ may want to call Β their High School Probability and Statistics teacher and find out what they missed

Categories: Project Management

Scaling @ HelloFresh: API Gateway

HelloFresh keeps growing every single day: our product is always improving, new ideas are popping up from everywhere, our supply chain is being completely automated. All of this is simply amazing us, but of course this constant growth brings many technical challenges.

Today I’d like to take you on a small journey that we went through to accomplish a big migration in our infrastructure that would allow us to move forward in a faster, more dynamic, and more secure way.

The Challenge

We’ve recently built an API Gateway, and now we had the complex challenge of moving our main (monolithic) API behind itβ€Š—β€Šideally without downtime. This would enable us to create more microservices and easily hook them into our infrastructure without much effort.

The Architecture
Categories: Architecture

Design by contract using GraphQL

Xebia Blog - Mon, 02/20/2017 - 17:20

When interfacing between systems it is good practice to think about the interface design prior to developing the systems. GraphQL can be a useful tool to write down these design decisions using its schema definition language. Even when you are not using GraphQL itself in production. GraphQL’s schema can be used to generate a mock […]

The post Design by contract using GraphQL appeared first on Xebia Blog.

Software Development Linkopedia February 2017

From the Editor of Methods & Tools - Mon, 02/20/2017 - 11:08
Here is our monthly selection of knowledge on programming, software testing and project management. This month you will find some interesting information and opinions about being a better leader and software manager, component teams, UX & Agile, writing less code, handling incidents, preventing Scrum to fail, test automation and lean principles. Text: How to be […]

Neo4j: Analysing a CSV file using LOAD CSV and Cypher

Mark Needham - Sun, 02/19/2017 - 23:39

Last week we ran our first online meetup for several years and I wanted to wanted to analyse the stats that YouTube lets you download for an event.

The file I downloaded looked like this:

$ cat ~/Downloads/youtube_stats_pW9boJoUxO0.csv 
Video IDs:, pW9boJoUxO0, Start time:, Wed Feb 15 08:57:55 2017, End time:, Wed Feb 15 10:03:10 2017
Playbacks, Peak concurrent viewers, Total view time (hours), Average session length (minutes)
348, 112, 97.125, 16.7456896552, 

Country code, AR, AT, BE, BR, BY, CA, CH, CL, CR, CZ, DE, DK, EC, EE, ES, FI, FR, GB, HU, IE, IL, IN, IT, LB, LU, LV, MY, NL, NO, NZ, PK, PL, QA, RO, RS, RU, SE, TR, US, VN, ZA
Playbacks, 2, 2, 1, 14, 1, 10, 2, 1, 1, 1, 27, 1, 1, 1, 3, 1, 25, 54, 1, 4, 6, 8, 1, 1, 1, 1, 1, 23, 1, 1, 1, 1, 1, 1, 2, 6, 22, 1, 114, 1, 1
Peak concurrent viewers, 2, 1, 1, 4, 1, 5, 1, 1, 0, 0, 11, 1, 1, 1, 2, 1, 6, 25, 1, 3, 3, 2, 1, 1, 1, 1, 1, 9, 1, 1, 0, 1, 0, 1, 1, 3, 7, 0, 44, 1, 0
Total view time (hours), 1.075, 0.0166666666667, 0.175, 2.58333333333, 0.00833333333333, 3.01666666667, 0.858333333333, 0.0583333333333, 0.0, 0.0, 8.69166666667, 0.8, 0.0166666666667, 0.0583333333333, 0.966666666667, 0.0166666666667, 4.20833333333, 20.8333333333, 0.00833333333333, 1.39166666667, 1.75, 0.766666666667, 0.00833333333333, 0.15, 0.0333333333333, 1.05833333333, 0.0333333333333, 7.36666666667, 0.0583333333333, 0.916666666667, 0.0, 0.00833333333333, 0.0, 0.00833333333333, 0.4, 1.10833333333, 5.28333333333, 0.0, 32.7333333333, 0.658333333333, 0.0
Average session length (minutes), 32.25, 0.5, 10.5, 11.0714285714, 0.5, 18.1, 25.75, 3.5, 0.0, 0.0, 19.3148148148, 48.0, 1.0, 3.5, 19.3333333333, 1.0, 10.1, 23.1481481481, 0.5, 20.875, 17.5, 5.75, 0.5, 9.0, 2.0, 63.5, 2.0, 19.2173913043, 3.5, 55.0, 0.0, 0.5, 0.0, 0.5, 12.0, 11.0833333333, 14.4090909091, 0.0, 17.2280701754, 39.5, 0.0

I want to look at the country specific stats so the first 4 lines aren’t interesting to me:

$ tail -n+5 youtube_stats_pW9boJoUxO0.csv > youtube.csv

I then put the youtube.csv file into the import directory of Neo4j and wrote the following query to return a row representing each country and its score for each of the metrics:

load csv with headers from "file:///youtube.csv" AS row
WITH [key in keys(row) where key <> "Country code"] AS keys, row, row["Country code"] AS heading
UNWIND keys AS key
RETURN key AS country, heading AS key, row[key] AS value

╒═════════╀═══════════╀═══════╕
β”‚"country"β”‚"key"      β”‚"value"β”‚
β•žβ•β•β•β•β•β•β•β•β•β•ͺ═══════════β•ͺ═══════║
β”‚" SE"    β”‚"Playbacks"β”‚"22"   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚" GB"    β”‚"Playbacks"β”‚"54"   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚" FR"    β”‚"Playbacks"β”‚"25"   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚" RS"    β”‚"Playbacks"β”‚"2"    β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚" LV"    β”‚"Playbacks"β”‚"1"    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”˜

Now I want to create a node representing each country and create a property for each of the metrics. Since the property names are going to be dynamic I’ll make use of the APOC library which I drop into my plugins directory. I then tweaked the query to create the nodes:

load csv with headers from "https://dl.dropboxusercontent.com/u/14493611/youtube.csv" AS row
WITH [key in keys(row) where key <> "Country code"] AS keys, row, row["Country code"] AS heading
UNWIND keys AS key
WITH key AS country, heading AS key, row[key] AS value
MERGE (c:Country {name: replace(country, " ", "")})
WITH *
CALL apoc.create.setProperty(c, key, toInteger(value))
YIELD node
RETURN COUNT(*)

We can now see which country provided the most viewers:

MATCH (n:Country) 
RETURN n.name, n.Playbacks AS playbacks, n.`Total view time (hours)` AS viewTimeInHours, n.`Peak concurrent viewers` AS peakConcViewers, n.`Average session length (minutes)` AS aveSessionMins
ORDER BY playbacks DESC
LIMIT 10

╒════════╀═══════════╀═════════════════╀═════════════════╀════════════════╕
β”‚"n.name"β”‚"playbacks"β”‚"viewTimeInHours"β”‚"peakConcViewers"β”‚"aveSessionMins"β”‚
β•žβ•β•β•β•β•β•β•β•β•ͺ═══════════β•ͺ═════════════════β•ͺ═════════════════β•ͺ════════════════║
β”‚"US"    β”‚"114"      β”‚"32"             β”‚"44"             β”‚"17"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"GB"    β”‚"54"       β”‚"20"             β”‚"25"             β”‚"23"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"DE"    β”‚"27"       β”‚"8"              β”‚"11"             β”‚"19"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"FR"    β”‚"25"       β”‚"4"              β”‚"6"              β”‚"10"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"NL"    β”‚"23"       β”‚"7"              β”‚"9"              β”‚"19"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"SE"    β”‚"22"       β”‚"5"              β”‚"7"              β”‚"14"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"BR"    β”‚"14"       β”‚"2"              β”‚"4"              β”‚"11"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"CA"    β”‚"10"       β”‚"3"              β”‚"5"              β”‚"18"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"IN"    β”‚"8"        β”‚"0"              β”‚"2"              β”‚"5"             β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"IL"    β”‚"6"        β”‚"1"              β”‚"3"              β”‚"17"            β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

The United States in first unsurprisingly followed by the UK, Germany, and France. We ran the meetup at 5pm UK time so it was a friendly enough time for this side of the globe but not so friendly for Asia or Australia so it’s not too surprising we don’t see anybody from there!

For my last trick I wanted to see the full names of the countries so I downloaded the 2 digit codes for each country along with their full name.

I then updated my graph:

load csv with headers from "file:///countries.csv" AS row
MATCH (c:Country {name: row.Code})
SET c.fullName = row.Name;

Now let’s re-run our query and show the country fullnames instead:

MATCH (n:Country) 
RETURN n.fullName, n.Playbacks AS playbacks, n.`Total view time (hours)` AS viewTimeInHours, n.`Peak concurrent viewers` AS peakConcViewers, n.`Average session length (minutes)` AS aveSessionMins
ORDER BY playbacks DESC
LIMIT 10

╒════════════════╀═══════════╀═════════════════╀═════════════════╀════════════════╕
β”‚"n.fullName"    β”‚"playbacks"β”‚"viewTimeInHours"β”‚"peakConcViewers"β”‚"aveSessionMins"β”‚
β•žβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•ͺ═══════════β•ͺ═════════════════β•ͺ═════════════════β•ͺ════════════════║
β”‚"United States" β”‚"114"      β”‚"32"             β”‚"44"             β”‚"17"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"United Kingdom"β”‚"54"       β”‚"20"             β”‚"25"             β”‚"23"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"Germany"       β”‚"27"       β”‚"8"              β”‚"11"             β”‚"19"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"France"        β”‚"25"       β”‚"4"              β”‚"6"              β”‚"10"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"Netherlands"   β”‚"23"       β”‚"7"              β”‚"9"              β”‚"19"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"Sweden"        β”‚"22"       β”‚"5"              β”‚"7"              β”‚"14"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"Brazil"        β”‚"14"       β”‚"2"              β”‚"4"              β”‚"11"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"Canada"        β”‚"10"       β”‚"3"              β”‚"5"              β”‚"18"            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"India"         β”‚"8"        β”‚"0"              β”‚"2"              β”‚"5"             β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚"Israel"        β”‚"6"        β”‚"1"              β”‚"3"              β”‚"17"            β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

And that’s the end of my analysis with no relationships in sight!

The post Neo4j: Analysing a CSV file using LOAD CSV and Cypher appeared first on Mark Needham.

Categories: Programming

SPaMCAST 431- Andrew Neitlich, Leadership is Core a Requirement

SPaMCAST Logo

http://www.spamcast.net

Listen Now
Subscribe on iTunes
Check out the podcast on Google Play Music

The Software Process and Measurement Cast 431 features our interview with Β Andrew Neitlich on leadership. Β We discussed whether leadership can be learned and if tech leadership is different than other kinds of leadership. Β Leadership is a core requirement for making all teams, Agile or not, effective!

Andrew’s bio:

Andrew Neitlich is the founder and director of the Center for Executive Coaching (http://centerforexecutivecoaching.com), a leader in training and certifying executive and leadership coaches. He also leads his own executive coaching practice, with an emphasis on working technical leaders that sometimes get frustrated with engaging their teams and having more impact when they communicate. Andrew is the author of Coach!, Elegant Leadership, and Guerrilla Marketing for a Bulletproof Career. He received his MBA from Harvard Business School, and lives in Sarasota, Florida.

Re-Read Saturday News

This week we tackle Chapter 4 in Carol Dweck’s Mindset: The New Psychology of Success (buy your copy and read along). Β In Chapter 4, Dweck hits a home run by reflecting on how mindsets translate into action in the sports arena (thus the sports allusions). Β Sports stories are one the most used metaphors in a business environment. Β I bet that you can’t you to go to two meetings in any corporate environment without hearing a project likened to the exploits of sports teams or athletes. This an easy metaphor theme because most everyone has been exposed to some form of sports or at least a story about sports before they take a job. In Chapter 4, Dr. Dweck, scores (I can’t help myself) by using the exploits of athletes and sports teams to further illustrate the differences and impact mindsets deliver.

Every week we discuss a chapter then consider the implications of what we have β€œread” from the point of view of someone pursuing an organizational transformation and also how to use the material when coaching teams. Β 

Remember to buy a copy of Carol Dweck’s Mindset and read along!

Visit the Software Process and Measurement Cast blog to participate in this and previous re-reads.

Feedback on SPaMCAST 428 with Mark Bojeun. Β Dan Β Stafford wrote to Mark and said, β€œ Great talk Mark, insightful as ever. Β Open and honest communication is such an important tenet.” Β Β (Listen Now)

Do you have thoughts and comments you would like to share? Β Email us at spamcastinfo@gmail.com

Next SPaMCAST

In the Software Process and Measurement Cast, we will feature an essay on the impact of leadership types on adopting and sustaining Agile. Β Leadership style has a direct impact on an organization’s ability to adopt and sustain Agile. Β Some leadership styles are more supportive and others evoke more of a Β response that is epitomized by locking feral cats and dogs in a room (nobody wins).

We will also have columns from Jeremy Berriault,Β who brings his QA Corner to the cast. Β Visit Jeremy’s new blog at https://jberria.wordpress.com/ Β Next, we will have a column from The Software Sensei, Kim Pries. Reach out to Kim on LinkedIn. Last, but not least, Jon M Quigley brings his column, the Alpha and Omega of Product Development, to the cast. Β One of the places you can find Jon is at Value Transformation LLC.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: β€œThis book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

 


Categories: Process Management

SPaMCAST 431- Andrew Neitlich, Leadership is Core a Requirement

Software Process and Measurement Cast - Sun, 02/19/2017 - 23:00

The Software Process and Measurement Cast 431 features our interview with Β Andrew Neitlich on leadership. Β We discussed whether leadership can be learned and if tech leadership is different than other kinds of leadership. Β Leadership is a core requirement for making all teams, Agile or not, effective!

Andrew’s bio:

Andrew Neitlich is the founder and director of the Center for Executive Coaching (http://centerforexecutivecoaching.com), a leader in training and certifying executive and leadership coaches. He also leads his own executive coaching practice, with an emphasis on working technical leaders that sometimes get frustrated with engaging their teams and having more impact when they communicate. Andrew is the author of Coach!, Elegant Leadership, and Guerrilla Marketing for a Bulletproof Career. He received his MBA from Harvard Business School, and lives in Sarasota, Florida.

Re-Read Saturday News

This week we tackle Chapter 4 in Carol Dweck’s Mindset: The New Psychology of Success (buy your copy and read along). Β In Chapter 4, Dweck hits a home run by reflecting on how mindsets translate into action in the sports arena (thus the sports allusions). Β Sports stories are one the most used metaphors in a business environment. Β I bet that you can’t you to go to two meetings in any corporate environment without hearing a project likened to the exploits of sports teams or athletes. This an easy metaphor theme because most everyone has been exposed to some form of sports or at least a story about sports before they take a job. In Chapter 4, Dr. Dweck, scores (I can't help myself) by using the exploits of athletes and sports teams to further illustrate the differences and impact mindsets deliver.

Every week we discuss a chapter then consider the implications of what we have β€œread” from the point of view of someone pursuing an organizational transformation and also how to use the material when coaching teams. Β 

Remember to buy a copy of Carol Dweck’s Mindset and read along!

Visit the Software Process and Measurement Cast blog to participate in this and previous re-reads.

Feedback on SPaMCAST 428 with Mark Bojeun. Β Dan Β Stafford wrote to Mark and said, β€œ Great talk Mark, insightful as ever. Β Open and honest communication is such an important tenet.” Β Β (Listen Now)

Do you have thoughts and comments you would like to share? Β Email us at spamcastinfo@gmail.com

Next SPaMCAST

In the Software Process and Measurement Cast, we will feature an essay on the impact of leadership types on adopting and sustaining Agile. Β Leadership style has a direct impact on an organization’s ability to adopt and sustain Agile. Β Some leadership styles are more supportive and others evoke more of a Β response that is epitomized by locking feral cats and dogs in a room (nobody wins).

We will also have columns from Jeremy Berriault,Β who brings his QA Corner to the cast. Β Visit Jeremy's new blog at https://jberria.wordpress.com/ Β Next, we will have a column from The Software Sensei, Kim Pries. Reach out to Kim on LinkedIn. Last, but not least, Jon M Quigley brings his column, the Alpha and Omega of Product Development, to the cast. Β One of the places you can find Jon is at Value Transformation LLC.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: β€œThis book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

Β 

Categories: Process Management

Stuff The Internet Says On Scalability For February 17th, 2017

Hey, it's HighScalability time:

 

Gorgeous satellite images of a thawing Greenland (NASA).
If you like this sort of Stuff then please support me on Patreon.
  • 1 cubic millimeter: computer with deep-Learning; 1,600: data on nearby stars; 40M: users for largest Parse app; 58x: Tensorflow 1.0 speedup on 64 gpus; 46%: ecommerce controlled by Amazon; 60%: IT growth in public cloud; 200 TB: one tv episode; 

  • Quotable Quotes:
    • @krishnan: Serverless will not be around in 5 years. It will be AI coding AI coding Ai....... Serverless or not doesn't matter #RunForrestRun
    • user5994461: Amazon: Create usual services and sell them. Google: Make unique products that push the boundaries of what was previously thought possible. Amazon: Don't care about inefficiencies and usage. Inefficiencies can be handled by charging more to the clients, usage doesn't matter because the users are mostly the clients and they don't feel their pain. Google: Had to make all their core technologies efficient, performant, scalable and maintainable or they couldn't sustain their business.
    • Hans Rosling: To me, the impressive thing is that people succeed at all.
    • @littleidea: Google Spanner didn't beat CAP, just mitigated the hell out of P
    • @jordw: Cloud Spanner is a very well-engineered CP database that is also very good at being available.
    • Cade Metz: The AI Threat Isn’t Skynet. It’s the End of the Middle Class
    • hosh: Four years ago, I determined that while development work might seem to be near the top of the food chain, there will at some point where my work will be replaced by AIs.
    • mi100hael:  I found Go's "simplicity" to be limiting and frustrating when it came to building production applications. Things like the weird split between functions returning errors but occasionally panicking, lack of inheritance, and poor dependency management through github links make Go a poor choice for applications within a business setting. 
    • @NathanTippy: New #Java web server clearing 1 million HTTP requests per second on 4 core box.  Can run in < 100MB of memory.
    • @kellabyte: It doesn’t matter what the founder or developer of a database tells you. It’s about the true peopeties it guarantees.
    • @swardley: Private cloud starting to drop, public cloud a three horse race - AWS 1st, MSFT 2nd, GooG 3rd ... sensible stuff 
    • @ollekullberg: Kullberg's law: when we increase the size of a microservice we increase the benefit of static typing for this microservice.
    • @swardley: ... it's not lack of engineering capability or finance or market or marketing or branding, the real story of cloud is executive failure.
    • katied: Trophic cascade is a process that starts at the top of a food chain and works its way to the bottom of it. So, even though as predators wolves survive by taking life, they also have the ability to create it.
    • @swardley: Cloud wars in IaaS - oh, please. War was well over in 2012, yes there will be price cuts as constraints are reduced but there is no battle.
    • @HenryR: 1. CAP has always said only one thing: that there is always a particular network failure that forces you to give up either C or A. 2. It has nothing at all to do with how likely that failure mode is. The failure is system-specific. 
    • throwawaydbfif: The movement from ownership to renting on the web is absolutely terrifying to me. Within the span of a few years we've gone from owning our technology to renting it out from a big players for monthly fees that we cannot completely predict or control.
    • computerex: People use cloud computing because it already is massively impractical to run your own servers. Hardware is hard to run and scale on your own and experiences economies of scale. This principle is seen everywhere and can hardly be viewed as something controversial. 
    • stuckagain: You did not ever own your own globally consistent, massively scalable, replicated database. The fact that you can now rent one by the hour is strictly an improvement for you, if you need that kind of thing
    • tedd4u: Aurora is very cool but won't help you much after you vertically scale your master and still need more write capacity. With Cloud Spanner you get horizontal write scalability out of the box. Critical difference.
    • @koivimik: REST != CRUD via HTTP #microXchg @olivergierke
    • Linus: It's almost boring how well our process works. All the really stressful times for me have been about process. They haven't been about code. When code doesn't work, that can actually be exciting ... Process problems are a pain in the ass. You never, ever want to have process problems ... That's when people start getting really angry at each other.
    • @littleidea: Almost every task run under Borg contains a built-in HTTP server that publishes information about the health of the task...
    • W. Daniel Hillis: For Richard [Feynman], figuring out these problems was a kind of a game. He always started by asking very basic questions like, “What is the simplest example?” or “How can you tell if the answer is right?” He asked questions until he reduced the problem to some essential puzzle that he thought he would be able to solve.
    • @ewolff: "Every hackathon uses Lambda. They build really complicated, production-ready systems in 12h" @adrianco at @microXchg
    • Daniel Bryant: The term "microservices" itself will probably disappear in the future, but the new architectural style of functional decomposition is here to stay.
    • @rbranson: The NoSQL movement might be a disappointment, but emerging from the rubble is the log-based (i.e. Kafka) model that actually works.
    • Chip Overclock: Surprisingly, GPS satellites actually know nothing about position. What they know about is time.
    • @codinghorror: I look at my old blog posts and think... there was a time when I believed 24GB was a lot of RAM
    • vidarh: Depending on your workloads, DO servers can come out cheaper or more expensive than AWS, but bandwidth at DO is so much cheaper than AWS that for bandwidth intensive stuff I can't serve entirely out of Europe (where Hetzner is vastly cheaper than DO again), DO is often a much cheaper alternative. Sometimes we use it as a cost-cutting do-it-yourself CDN in front of AWS for clients that insist on S3 for storage (and again where we can't just cache everything in Europe for latency reasons). For bandwidth heavy applications, you can pay for significant numbers of Droplets from the AWS bandwidth savings alone.
    • lobster_johnson: we use Google Container Engine (hosted Kubernetes), with Salt for the non-GKE VMs. This is needed because K8s is not mature enough to host all the things. In particular, stateful sets are still in beta. 
    • anonymous: The overall impact [algorithms] will be utopia or the end of the human race; there is no middle ground foreseeable. I suspect utopia given that we have survived at least one existential crisis (nuclear) in the past and that our track record toward peace, although slow, is solid.
    • keenio: In conclusion, the TCO is probably significantly lower for Kinesis. So is the risk. And in most projects, risk-adjusted TCO should be the final arbiter.
    • Adem Efe Gencer: the weekly [Bitcoin] mining power of a single miner has never exceeded the 30% of the overall mining power in 2016. Morever, in the second half of the year, the highest mining power has consistently been under the 20% range.
    • David Rosenthal: The security downside of Postel's Law is even more fundamental. The law requires the receiver to accept, and do something sensible with, malformed input. Doing something sensible will almost certainly provide an attacker with the opportunity to make the receiver do something bad.
    • douche: That's pretty much the way it has always been. You can go back at least to the Civil War and find politics has had more to do with procurement than performance of the weapon systems in question.
    • Jonathan Suen: While the brain and the Internet clearly operate using very different mechanisms, both use simple local rules that give rise to global stability. I was initially surprised that biological neural networks utilized the same algorithms as their engineered counterparts, but, as we learned, the requirements for efficiency, robustness, and simplicity are common to both living organisms and the networks we have built.
    • Bruce Johnson: Code reviews set the tone for the entire company that everything we do should be open to scrutiny from others, and that such scrutiny should be a welcome part of your workflow rather than viewed as threatening.
    • codingmyway: I think some miners are against any increase because it will lower fees. Without a blocksize limit fees tend to zero, which is fine while there is the block reward but they still want to milk the congestion fees. To say they are pro segwit or pro unlimited is bluffing. They are pro status quo and congestion and high fees.
    • edejong: Many engineers I have worked with like to throw around terms like: "CQRS", "Event sourcing", "no schema's", "document-based storage", "denormalize everything" and more. However, when pushed, I often see that they lack a basic understanding of DBMSes, and fill up this gap by basically running away from it. For 95% of the jobs, a simple, non-replicated (but backed-up) DBMS will do just fine.
    • adamu__: If China were to shut down bitcoin mining, my understanding is that the worst case scenario is much more dire. The network only adjusts the 'difficulty' relative to current network hash power every 2,016 blocks. Depending on the severity of the overall hash power reduction, new block discovery might slow down significantly. This would also delay a recalculation of the new difficulty accommodating the reduction in hash power. The network could be severely throttled for weeks.
    • boulos: Slightly off-topic, but EC2 doesn't really scale independently if you compare it to GCE. We let you combine 24 vcpus with 39 GB of RAM, 3 partitions of Local SSD and a few GPUs, all independently (though the ratio of RAM to vcpu is currently bounded between .9 and 6.5).
    • Veratyr: Personally, I settled with colocation. I pay $60/mo + $2k one-off for the initial hardware + say $150/5y/4TB HDD, which, for 80TB of storage over 5y comes out to a total of ~$88/mo, or $0.001/GBmo. 

  • Now this is object oriented programming. New software for increasingly flexible factory processes: new software that allows each individual component to tell the machine what has to be done. By breaking away from central production planning, factories can achieve unprecedented agility and flexibility... Everything would go much faster if production and the requisite machines were not rigidly set by a control program, but if every component itself knew the best way for it to be moved quickly through the process chain. 

  • Relax. Videos from TensorFlow Dev Summit 2017 are now available. Also, Learn TensorFlow and deep learning, without a Ph.D. Also also, Deep Learning book.

  • Google is Introducing Cloud Spanner: a global database service for mission-critical applications. It will be interesting to see if Spanner, as a unique hard to duplicate feature, becomes a Google Cloud differentiator. Will it make the delta between the clouds significant enough that developers choose Google? Quizlet, already running on GCP, really likes Spanner, but it's not a drop in replacement for MySQL. Like with NoSQL there's special care and feeding to make it work, but that's the sacrifice high QPS requires. Performance: "Cloud Spanner queries have higher latency at low throughputs compared with a virtual machine running MySQL. Spanner's scalability, however, means that a high-capacity cluster can easily handle workloads that stretch our MySQL infrastructure." And p90s are consistently lower than 50 ms. Cost: "For very small or low-throughput databases Cloud Spanner is overkill [min ~$8,000/yr]...Cloud Spanner comparable or slightly cheaper based on the performance in our testing."  With Spanner hitting the market maybe that will help CockroachDB? Some older articles: Spanner - It's About Programmers Building Apps Using SQL Semantics At NoSQL ScaleGoogle Spanner's Most Surprising Revelation: NoSQL Is Out And NewSQL Is InF1 And Spanner Holistically ComparedHow Google Invented An Amazing Datacenter Network Only They Could Create

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Mindset: The New Psychology of Success: Re-Read Week 4, Chapter 4 – Sports: The Mindset of a Champion

Mindset Book Cover

Today we rush into Chapter 4 in Carol Dweck’s Mindset: The New Psychology of Success (buy your copy and read along). Β In Chapter 4, Dweck hits a home run by reflecting on how mindsets translate into action in the sports arena (thus the sports allusions). Β Sports stories are one the most used metaphors in a business environment. Β I bet that you can’t you to go to two meetings in any corporate environment without hearing a project likened to the exploits of sports teams or athletes. This an easy metaphor theme because most everyone has been exposed to some form of sports or at least a story about sports before they take a job. In Chapter 4, Dr. Dweck, scores (I can’t help myself) by using the exploits of athletes and sports teams to further illustrate the differences and impact mindsets deliver.

Chapter 4 begins with the story of Billy Beane. Mr. Beane was a baseball player that was drafted and came through the minor leagues being viewed as a β€œnatural” (someone that has all the talent in the world and will be a star). Β Unfortunately, at the time, he had a fixed mindset, and when completion got tough he was not able to deal with the fact talent alone would not carry him forward. Β The realization that he had to shift his mindset later led to the concepts discussed in the book and movie, Moneyball. Β The statistical analysis Beane helped pioneer focused on identifying players with attributes, including many of the attributes we would recognize as a growth mindset, rather than raw talent.Β 

Chapter 4 goes on to highlight a wide range of other legendary athletes that illustrate that hard work and discipline are very powerful tools. All of the highlighted athletes became legends not just because they had awesome natural talent, but also because their mindset allowed them to see to benefit from learning from setbacks and hard work. It might be possible to make the argument that someone with a growth mindset can’t help learning from their experiences. Β Something it took Billy Beane a long time to learn. Β The top line takeaway of this chapter is that success is rarely just a reflection of ability.

An important reflection in the chapter is that humans seem to have a natural bias toward the belief that great athletes (and by extension other forms of genius) shouldn’t require effort to become the top of their field. This flies (another sports’ metaphor) in the face of the evidence that most β€œnaturals” have only gotten to the top through discipline and learning from setbacks. Β Dweck provides several examples ranging from Babe Ruth, Maury Wills, Michael Jordan, Tiger Woods, and Wilma Rudolph. A quote from Malcolm Gladwell is used to drive the point home, β€œPeople prize natural endowment over earned ability.”

Adversity is a feedback mechanism. Β One of the descriptions of the behavior of Agile teams is that they pursue feedback (sometimes called running toward feedback) so that the team can learn how to change its behavior to deliver more value and customer satisfaction. Without a mindset that seeks feedback from even minor adversities, when you hit a major roadblock you may not be able to deal with it.

A person’s character (related to effort and discipline) is at least a partial reflection of the mindset the person has adopted. Β The examples provided by Dr. Dweck in this chapter illustrate how the two mindsets help shape character. Β Over and over, athletes with growth mindsets are better atΒ dealing with adversity because they see setbacks as a tool to learn. In scenarios where top athletes don’t have a growth mindset, Dweck provides examples of how their inability to deal with negative feedback leads to failure. Β Two athletes used to highlight the problems of fixed mindset were Pedro Martinez (baseball pitcher who imploded in playoffs against the New York Yankees) and John McEnroe (just one tantrum from YouTube). Β While both athletes had their moment in the sun, I think we could ask just how good could they be if they had a growth mindset. Dweck contrasts the character of these two athletes with the character of Pete Sampras. Β When asked how he could rally back to win after being at match point and down two sets (men’s championship tennis is typically the best of 5 sets) indicated he used adversity he faced overcome to remind himself about what possible and what he was capable of achieving. Β With champions, when the going gets tough the tough get going. The examples in the book cherry pick, the most easily recognizableΒ stars however even if mindsets don’t explain all variance in performance, mindset explain a large proportion. Β 

In this chapter the point that how we deal with feedback and whether we seek out feedback loops is a predictor of eventual success. Β People with a fixed mindset eschew feedback and view setbacks as being victimized by outside sources. Β When you are a victim it easy to decide that you do not need to take responsibility for your ability and control of your motivation.

Chapter 4 – From a Coach’s Perspective

Transforming a group or organization requires understanding the overall cultural mindset of the organization being changed. Β Organizations often have a bias towards one mindset or the other. Β For example, very early in my career, I worked for a firm that manufactured and sold junior ready to wear. At the time the organization had become the largest firm in the world in its category. They had achieved over a decade of phenomenal year-over-year sales increases. I recently found a transcript from one of their major sales meetings as things began to stagnate (they later collapsed). Β It was amazing to see the evidence of how the firm rejected feedback from clients and successes described with β€œI” words rather than words that were inclusive of the whole team needed to deliver value to the clients and customers. Β Words are an indicator of mindset both at an individual level and at an organizational level. Β As you consider a path for a transformation, listen to how is performance is described in the organization. If these conversations don’t happen in the natural course of an assessment, illicit stories about performance and projects. Β The words that are used will give you a strong indication of mindset. Β For example, does the word β€œI” get used or the word β€œwe” when describing a project? Β Do people describe their identity though outcomes or effort (If I win I’ll be somebodyΒ if I lose I’ll be nobody)?

Identifying mindsets can be an important change management tool. Β Change leaders can use mindsets toΒ filter people whoΒ can be trailblazers and/or early adopters for the concepts being put forward in the transformation.

At the team level, listening is also applicable. Β Use techniques like storytelling to elicit the markers of mindset. Β I use this technique occasionally in retrospectives or when asked to help a team deal with behaviors that are impacting a project. Β In the short term having a grasp on someone’s mindset is useful for knowing how to relate to the person and how they will react in certain situations. For a leader knowing a person’s mindset can be used to help a team member to accept the work that fits their temperament. Β In the longer run, knowing which mindset a person emulates can help a coach to provide mentoring and training that can shift that mindset (if needed).

Previous Entries of the re-read of Mindset:

 


Categories: Process Management

Debug TensorFlow Models with tfdbg

Google Code Blog - Fri, 02/17/2017 - 23:32
Posted by Shanqing Cai, Software Engineer, Tools and Infrastructure.

We are excited to share TensorFlow Debugger (tfdbg), a tool that makes debugging of machine learning models (ML) in TensorFlow easier.
TensorFlow, Google's open-source ML library, is based on dataflow graphs. A typical TensorFlow ML program consists of two separate stages:
  1. Setting up the ML model as a dataflow graph by using the library's Python API,
  2. Training or performing inference on the graph by using the Session.run()method.
If errors and bugs occur during the second stage (i.e., the TensorFlow runtime), they are difficult to debug.

To understand why that is the case, note that to standard Python debuggers, the Session.run() call is effectively a single statement and does not exposes the running graph's internal structure (nodes and their connections) and state (output arrays or tensors of the nodes). Lower-level debuggers such as gdb cannot organize stack frames and variable values in a way relevant to TensorFlow graph operations. A specialized runtime debugger has been among the most frequently raised feature requests from TensorFlow users.

tfdbg addresses this runtime debugging need. Let's see tfdbg in action with a short snippet of code that sets up and runs a simple TensorFlow graph to fit a simple linear equation through gradient descent.

import numpy as np
import tensorflow as tf
import tensorflow.python.debug as tf_debug
xs = np.linspace(-0.5, 0.49, 100)
x = tf.placeholder(tf.float32, shape=[None], name="x")
y = tf.placeholder(tf.float32, shape=[None], name="y")
k = tf.Variable([0.0], name="k")
y_hat = tf.multiply(k, x, name="y_hat")
sse = tf.reduce_sum((y - y_hat) * (y - y_hat), name="sse")
train_op = tf.train.GradientDescentOptimizer(learning_rate=0.02).minimize(sse)

sess = tf.Session()
sess.run(tf.global_variables_initializer())

sess = tf_debug.LocalCLIDebugWrapperSession(sess)
for _ in range(10):
sess.run(train_op, feed_dict={x: xs, y: 42 * xs})

As the highlighted line in this example shows, the session object is wrapped as a class for debugging (LocalCLIDebugWrapperSession), so the calling the run() method will launch the command-line interface (CLI) of tfdbg. Using mouse clicks or commands, you can proceed through the successive run calls, inspect the graph's nodes and their attributes, visualize the complete history of the execution of all relevant nodes in the graph through the list of intermediate tensors. By using the invoke_stepper command, you can let the Session.run() call execute in the "stepper mode", in which you can step to nodes of your choice, observe and modify their outputs, followed by further stepping actions, in a way analogous to debugging procedural languages (e.g., in gdb or pdb).

A class of frequently encountered issue in developing TensorFlow ML models is the appearance of bad numerical values (infinities and NaNs) due to overflow, division by zero, log of zero, etc. In large TensorFlow graphs, finding the source of such nodes can be tedious and time-consuming. With the help of tfdbg CLI and its conditional breakpoint support, you can quickly identify the culprit node. The video below demonstrates how to debug infinity/NaN issues in a neural network with tfdbg:

A screencast of the TensorFlow Debugger in action, from this tutorial.

Compared with alternative debugging options such as Print Ops, tfdbg requires fewer lines of code change, provides more comprehensive coverage of the graphs, and offers a more interactive debugging experience. It will speed up your model development and debugging workflows. It offers additional features such as offline debugging of dumped tensors from server environments and integration with tf.contrib.learn. To get started, please visit this documentation. This research paperlays out the design of tfdbg in greater detail.

The minimum required TensorFlow version for tfdbgis 0.12.1. To report bugs, please open issues on TensorFlow's GitHub Issues Page. For general usage help, please post questions on StackOverflow using the tag tensorflow.
Acknowledgements
This project would not be possible without the help and feedback from members of the Google TensorFlow Core/API Team and the Applied Machine Intelligence Team.





Categories: Programming

Quote of the Day

Herding Cats - Glen Alleman - Fri, 02/17/2017 - 22:58

The real world is fraught with risk.
Forecasting with empirical data - the #NoEstimates approach to estimating - ignores this fact. In the NE version ofΒ forecasting there is no place for uncertainty, which is why No Estimates advocates assert Forecasting is not estimating. In this version of Estimating without calling it Estimating,Β the past behaviour represents the future behaviour, with not adjustments for reducible and irreducible uncertainties that create risk to the project's success.
When we try to plan, knowing we cannot predict the future precisely, the mismatch between the planning and the real world, creates confusion. This confusion creates biases and distortions, resulting in padded estimates and undue optimism as well as misuse and even dysfunction ofΒ the management in the presence of uncertainty.
There needs to be an approach to planning and forecasting that is connected with reality, an approach that acknowledges uncertainty from the beginning and continues toΒ manage in the presence of that uncertanty throughout the life of the project.
This approach mandates estimating all our work that not de minimis. This work is driven by uncertainty - reducible and irreducible.
This uncertainty creates risk and Risk Management is How Adults Manage Projects - Tim Lister.

Practical Risk Assessment for Project Management, Stephen Grey, Wiley Series in Software Engineering Practice, 1995. Nothing has changed since these words. We still live, operate, and manageΒ in the presence of uncertainty. Here's how we manage in our software intensive system of systems domain. Your domain may be different, but the principles of managing in the presence of uncertainty are the same.

Here's the now familiarΒ briefingΒ for how we management in the presence of uncertainty in our Software Intensive System of Systems domain.

Managing in the presence of uncertainty from Glen Alleman

Β 

Related articles The Flaw of Empirical Data Used to Make Decisions About the Future The Flaw of Averages and Not Estimating Herding Cats: Decision Making On Software Development Projects Monte Carlo Simulation of Project Performance
Categories: Project Management

Decision Making On Software Development Projects

Herding Cats - Glen Alleman - Fri, 02/17/2017 - 21:38

Decision Making on Software Development Projects
Is Both Simple and Complex at the Same Time

All projects operate in the presence of uncertainty.

Real-world decision making is performed under uncertainty.

Decision makers must make decisions which best incorporate these uncertainties.

These uncertainties come in two forms – reducible (I can do something about it) and irreducible (I can't do anything about it).

Irreducible (Aleatory) uncertainty is the natural variability of the processes and technology on the project. These are stochastic, they may be nonstationary, and are due to chance from the underlying statistical distributions.

Aleatory uncertainties are modeled as random variables described by statistical distributions Β (Triangle is a common one when the actual distribution is not known)Β 
In aleatory uncertainty, decision makers make assumptions about the distribution's descriptive statistics - the Mean and Variance are needed along with the Most Likely (Mode) at a minimum. The shape of the curve is needed as well.

Irreducible uncertainty can only be dealt with margin. Cost margin, schedule margin, technical margin.

Reducible (Epistemic) uncertainty is subjective, with the subjectivity coming fromΒ lack of knowledge.

ThisΒ lack of knowledge comes from the probabilistic nondeterministic behavior of the system or the environment.

Reducible uncertainty is addressed with redundancy, experiments, prototypes, models, measures, empiricalΒ data to reveal knowledge about the underlying probabilities of the process.

All Uncertainty Creates Risk.

Reducible risk requires estimating the probability distribution of the occurrence.

Irreducible risk requires estimating the statistical distributionΒ of the naturally occurring processes.

Risk Management is How Adults Manage projects – Tim Lister.

Risk management requires estimating.

Adult management of projects requires estimating.

Not Estimating means not managing as an Adult.

  1. Β β€œTreatment of aleatory and epistemic uncertainty in performance assessments for complex systems,”J. C. Helton and D. E. Burmaster, Reliability Engineering and System Safety, vol. 54, no. 2-3, pp. 91–94, 1996.
  2. β€œUncertainty quantification using evidence theory,” W. L. Oberkampf, in Proceedings from the Advanced Simulation & Computing Workshop, Albuquerque, NM, USA, 2005.
  3. β€œTreatment of uncertainty in performance assessments for complex systems,”J. C. Helton, Risk Analysis, vol. 14, no. 4, pp. 483–511, 1994.
  4. S. N. Rai, D. Krewski, and S. Bartlett, β€œA general framework for the analysis of uncertainty and variability in risk assessment,” Human and Ecological Risk Assessment, vol. 2, no. 4, pp. 972–989, 1996.
  5. β€œUnderstanding uncertainty,”W. D. Rowe, Risk Analysis, vol. 14, no. 5, pp. 743–750, 1994.
  6. Possibility Theory: An Approach to Computerized Processing of Uncertainty, D. Dubois and H. Prade, Plenum Press, New York, NY, USA, 1988.
  7. β€œJudgment under uncertainty: heuristics and biases,”A. Tversky and D. Kahneman, Science, vol. 185, no. 4157, pp. 1124–1131, 1974.
  8. Methods for representing uncertainty.Β A literature review, Enrico Zio and Nicola Pedroni,Β Foundation for an Industrial Safety Culture, Toulouse, France
  9. Estimating Software-Intensive Systems, Richard Stutzke, Addison Wesley
Related articles Making Decisions in the Presence of Uncertainty Managing in the Presence of Uncertainty Herding Cats: Cone of Uncertainty - Part Cinq (Updated) Herding Cats: Managing Uncertainty, Risk, Threat, and Opportunity Some More Background on Probability, Needed for Estimating Estimating is Risk Management IT Risk Management Risk Management is How Adults Manage Projects
Categories: Project Management

Quote of the Day

Herding Cats - Glen Alleman - Fri, 02/17/2017 - 21:32

People behave the way they are managed
- A Wise old Navy Captain as told by LCMDR NicholasΒ Pisano

Categories: Project Management

Why don’t monitoring tools monitor changes?

Xebia Blog - Fri, 02/17/2017 - 12:38

Changes in applications or IT infrastructure can lead to application downtime. This not only hits your revenue, it also has a negative impact on your reputation. Everybody in IT understands the importance of having the right monitoring solutions in place. From an infrastructure – to a business perspective, we rely on monitoring tools to get […]

The post Why don’t monitoring tools monitor changes? appeared first on Xebia Blog.

The 8 Leading Attributes of Trust, Leadership’s Most Important Tool

26303029941_b4376df9c4_k

Leaders require trustΒ between them and those they lead to be effective. Trust is not a simple attribute like hair color. Β Trust is a synthesis of several attributes. None of the attributes that impact trust are fixed at birth. As humans, we learn the attributes that generate trust based on the environments we are exposed to and hone them based on effort and importance we place on these characteristics. The 8 most important characteristics that shape trust in software development and Agile environments are:

  1. Competence – Trust is difficult in scenarios where there areΒ significant mismatches in the combination of skills and experiences each individual brings to an endeavor. I recently observed a team in which a new, highly trained but inexperienced tester joined a Scrum team. There was not trust until the person had gained experience by interacting with the team for several sprints.
  2. Truthfulness – Trustworthy leaders will know and share the truth. Β Deception, even when ‘harmless’ or even beneficial, will reduce the credibility of every statement going forward.
  3. Act as they think – Words, feelings, and beliefs match actions of the leader. Β A popular adage is that if a leader β€œis going to talk the talk, they’ve got to walk the walk” in order to be trusted.Β 
  4. Integrity – Trustworthy leaders take responsibility for their actions and work and make sure that the work of other is attributed correctly. Β A leader with integrity will link themselves to a set of moral and ethical principles that are known to the team and organization.
  5. Reliability – Trustworthy leaders say what they will do and do what they say they will do. Β A corollary is that trustworthy leaders also do what they say they will do when they say they will do it.
  6. Loyalty – Trustworthy leaders are loyal to their people and organization (but not stupidly loyal). Β All too often untrustworthy leaders will throw someone under the bus when issues are exposed or talk negatively about someone when they are not present. Β Showing loyalty towards others is a prerequisite for receiving trust from others.
  7. Accountability – Leaders build trust by recognizing, admitting and accepting responsibility for their own mistakes.
  8. Just – A trustworthy leader is just to those on their team and with those outside their team. The actions of a just leader are predictable and measured rather than erraticΒ and extreme.Β 

There are other attributes of trust; however, in a collaborative software development environment using Agile, these are often the most important contributors to trust in a leader. Β A point that is often missed when discussing Agile or other decentralized frameworks is that leadership is dynamic. Therefore trust is important for anyone that might take up the mantle of leadership to foster. Β I often recommend carving out time in retrospectives to examine trust to ensure that building trust is not seen as something that happens naturally or in random team building exercises.

 


Categories: Process Management

Fifty Great Project Management Blogs

Herding Cats - Glen Alleman - Thu, 02/16/2017 - 19:06

Screen Shot 2017-02-16 at 11.03.57 AMOn Line PM Course has listed the 50 Greatest Project Management Blogs in alphabetical order

Categories: Project Management