Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Architecture

AngularJS directives for c3.js

Gridshore - Tue, 08/19/2014 - 10:19

For one of my projects we wanted to create some nice charts. Feels like something you often want but do not do because it takes to much time. This time we really needed it. We had a look at D3.js library, a very nice library but so many options and a lot to do yourself. Than we found c3.js, check the blog post by Roberto: Creating charts with C3.js. Since I do a lot with AngularJS, I wanted to integrate these c3.js charts with AngularJS. I already wrote a piece about the integration. Now I went one step further by creating a set of AngularJS directives.

You can read the blog on the trifork blog:

http://blog.trifork.com/2014/08/19/angularjs-directives-for-c3-js-chart-library/

The post AngularJS directives for c3.js appeared first on Gridshore.

Categories: Architecture, Programming

1 Aerospike server X 1 Amazon EC2 instance = 1 Million TPS for just $1.68/hour

This a guest post by Anshu Prateek, Tech Lead, DevOps at Aerospike and Rajkumar Iyer, Member of the Technical Staff at Aerospike.

Cloud infrastructure services like Amazon EC2 have proven their worth with wild success. The ease of scaling up resources, spinning them up as and when needed and paying by unit of time has unleashed developer creativity, but virtualized environments are not widely considered as the place to run high performance applications and databases.

Cloud providers however have come a long way in their offerings and need a second review of their performance capabilities. After showing 1 Million TPS on Aerospike on bare metal servers, we decided to investigate cloud performance and in the process, bust the myth that cloud != high performance.

We examined a variety of Amazon instances and just discovered the recipe for processing 1 Million TPS in RAM on 1 Aerospike server on a single C3.8xlarge instance - for just $1.68/hr !!!

According to internetlivestats.com, there are 7.5k new tweets per second, 45k google searches per second and 2.3 Million emails sent per second. What would you build if you could process 1 Million database transactions per second for just $1.68/hr?

Categories: Architecture

Little's Law in 3D

Xebia Blog - Sun, 08/17/2014 - 16:21

The much used relation between average cycle time, average total work and input rate (or throughput) is known as Little's Law. It is often used to argue that it is a good thing to work on less items at the same time (as a team or as an individual) and thus lowering the average cycle time. In this blog I will discuss the less known generalisation of Little's Law giving an almost unlimited number of additional relation. The only limit is your imagination.

I will show relations for the average 'Total Operational Cost in the system' and for the average 'Just-in-Timeness'.

First I will describe some rather straightforward generalisations and in the third part some more complex variations on Little's Law.

Little's Law Variations

As I showed in the previous blogs (Applying Little's Law in Agile Games and Why Little's Law Works...Always) Little's Law in fact states that measuring the total area from left-to-right equals summing it from top-to-bottom.

Once we realise this, it is easy to see some straightforward generalisations which are well-known. I'll mention them here briefly without ging into too much details.

Subsystem

new doc 8_1 Suppose a system that consists of 1 or more subsystems, e.g. in a kanban system consisting of 3 columns we can identify the subsystems corresponding to:

  1. first column (e.g. 'New') in 'red',
  2. second column (e.g. 'Doing') in 'yellow',
  3. third column (e.g. 'Done') in 'green'

See the figure on the right.

By colouring the subsystems different from each other we see immediately that Little's Law applies to the system as a whole as well as to every subsystem ('red' and 'yellow' area).

Note: for the average input rate consider only the rows that have the corresponding color, i.e. for the input rate of the column 'Doing' consider only the rows that have a yellow color; in this case the average input rate equals 8/3 items per round (entering the 'Doing' column). Likewise for the 'New' column.

Work Item Type

new doc 9_1Until now I assumed only 1 type of work items. In practise teams deal with more than one different work item types. Examples include class of service lanes, user stories, and production incidents. Again, by colouring the various work item type differently we see that Little's Law applies to each individual work item type.

In the example on the right, we have coloured user stories ('yellow') and production incidents ('red'). Again, Little's Law applies to both the red and yellow areas separately.

Doing the math we se that for 'user stories' (yellow area):

  • Average number in the system (N) = (6+5+4)/3 = 5 user stories,
  • Average input rate (\lambda\lambda = 6/3 = 2 user stories per round,
  • Average waiting time (W) = (3+3+3+3+2+1)/6 = 15/6 = 5/2 rounds.

As expected, the average number in the system equals the average input rate times the average waiting time.

The same calculation can be made for the production incidents which I leave as an exercise to the reader.

Expedite Items

new doc 10_1 Finally, consider items that enter and spend time in an 'expedite' lane. In Kanban an expedite lane is used for items that need special priority. Usually the policy for handling such items are that (a) there can be at most 1 such item in the system at any time, (b) the team stop working on anything but on this item so that it is completed as fast as possible, (c) they have priority over anything else, and (d) they may violate any WiP limits.

Colouring any work items blue that spend time in the expedite lane we can apply Little's Law to the expedite lane as well.

An example of the colouring is shown in the figure on the right. I leave the calculation to the reader.

3D


We can even further extend Little's Law. Until now I have considered only 'flat' areas.

The extension is that we can give each cell a certain height. See the figure to the right. A variation on Little's Law follows once we realise that measuring the volume from left-to-right is the same as calculating it from top-to-bottom. Instead of measuring areas we measure volumes instead.

The only catch here is that in order to write down Little's Law we need to give a sensible interpretation to the 'horizontal' sum of the numbers and a sensible interpretation to the 'vertical' sum of the numbers. In case of a height of '1' these are just 'Waiting Time' (W) and 'Number of items in the system' (N) respectively.

A more detailed, precise, and mathematical formulation can be found in the paper by Little himself: see section 3.2 in [Lit11].

Some Applications of 3D-Little's Law

Value

As a warming-up exercise consider as the height the (business) value of an item. Call this value 'V'. Every work item will have its own specific value.
new doc 12_1

 

 

 \overline{\mathrm{Value}} = \lambda \overline{V W} \overline{\mathrm{Value}} = \lambda \overline{V W}

The interpretation of this relation is that the 'average (business) value of unfinished work in the system at any time' is equal to the average input rate multiplied by the 'average of the product of cycle time and value'.

Teams may ant to minimise this while at the same time maximising the value output rate.

Total Operational Cost

As the next example let's take as the height for the cells a sequence of numbers 1, 2, 3, .... An example is shown in the figures below. What are the interpretations in this case?

Suppose we have a work item that has an operational cost of 1 per day. Then the sequence 1, 2, 3, ... gives the total cost to date. At day 3, the total cost is 3 times 1 which is the third number in the sequence.

new doc 12_2The 'vertical' sum is just the 'Total Cost of unfinished work in the system.

For the interpretation of the 'horizontal' sum we need to add the numbers. For a work item that is in the system for 'n' days, the total is 1+2+3+...+n1+2+3+...+n which equals 1/2 n (n+1)1/2 n (n+1). For 3 days this gives 1+2+3=1/2 * 3 * 4 = 61+2+3=1/2 * 3 * 4 = 6. Thus, the interpretation of the 'horizontal' sum is 1/2 W (W+1)1/2 W (W+1) in which 'W' represents the waiting time of the item.

Putting this together gives an additional Little's Law of the form:

 \overline{\mathrm{Cost}} = \frac{1}{2} \lambda C \overline{W(W + 1)} \overline{\mathrm{Cost}} = \frac{1}{2} \lambda C \overline{W(W + 1)}

where 'C' is the operational cost rate of a work item and \lambda\lambda is the (average) input rate. If instead of rounds in a game, the 'Total Cost in the system' is measured at a time interval 'T' the formula slightly changes into

 \overline{\mathrm{Cost}} = \frac{1}{2} \lambda C \overline{W\left(W + T\right)} \overline{\mathrm{Cost}} = \frac{1}{2} \lambda C \overline{W\left(W + T\right)}

Teams may want to minimise this which gives an interesting optimisation problem is different work item types have different associated operational cost rates. How should the capacity of the be divided over the work items? This is a topic for another blog.

Just-in-Time

For a slightly more odd relation consider items that have a deadline associated with them. Denote the date and time of the deadline by 'D'. As the height choose the number of time units before or after the deadline the item is completed. Further, call 'T' the time that the team has taken up to work on the item. Then the team finishes work on this item at time  T + W T + W , where 'W' represent the cycle time of the work item.

new doc 12_4

In the picture on the left a work item is shown that is finished 2 days before the deadline. Notice that the height decreases as the deadline is approached. Since it is finished 2 time units before the deadline, the just-in-timeness is 2 at the completion time.

new doc 12_3

The picture on the left shows a work item one time unit after the deadline and has an associated just-in-timeness of 1.

 

 \overline{\mathrm{Just-in-Time}} = \frac{1}{2} \lambda \overline{|T+W-D|(|T+W-D| + 1)} \overline{\mathrm{Just-in-Time}} = \frac{1}{2} \lambda \overline{|T+W-D|(|T+W-D| + 1)}

This example sounds like a very exotic one and not very useful. A team might want to look at what the best time is to start working on an item so as to minimise the above variable.

Conclusion

From our 'playing around' with the size of areas and volumes and realising that counting it in different ways (left-to-right and top-to-bottom) should give the same result I have been able to derive a new set of relations.

In this blog I have rederived well-known variations on Little's Law regarding subsystems and work items types. In addition I have derived new relations for the 'Average Total Operational Cost', 'Average Value', and 'Average Just-in-Timeness'.

Together with the familiar Little's Law these give rise to interesting optimisation problems and may lead to practical guidelines for teams to create even more value.

I'm curious to hear about the variations that you can come up with! Let me know by posting them here.

References

[Lit11] John D.C. Little, "Little’s Law as Viewed on Its 50th Anniversary", 2011, Operations Research, Vol. 59 , No 3, pp. 536-549, https://www.informs.org/content/download/255808/2414681/file/little_paper.pdf

 

 

Managing OpenStack security groups from the command line

Agile Testing - Grig Gheorghiu - Fri, 08/15/2014 - 20:47
I had an issue today where I couldn't connect to a particular OpenStack instance on port 443. I decided to inspect the security group it belongs (let's call it myapp) to from the command line:

# nova secgroup-list-rules myapp
+-------------+-----------+---------+------------+--------------+
| IP Protocol | From Port | To Port | IP Range   | Source Group |
+-------------+-----------+---------+------------+--------------+
| tcp         | 80        | 80      | 0.0.0.0/0  |              |
| tcp         | 443       | 443     | 0.0.0.0/24 |              |
+-------------+-----------+---------+------------+--------------+

Note that the IP range for port 443 is wrong. It should be all IPs and not a /24 network.

I proceeded to delete the wrong rule:

# nova secgroup-delete-rule myapp tcp 443 443 0.0.0.0/24                                                               
+-------------+-----------+---------+------------+--------------+
| IP Protocol | From Port | To Port | IP Range   | Source Group |
+-------------+-----------+---------+------------+--------------+
| tcp         | 443       | 443     | 0.0.0.0/24 |              |
+-------------+-----------+---------+------------+--------------+


Then I added back the correct rule:
 # nova secgroup-add-rule myapp tcp 443 443 0.0.0.0/0                                                                   +-------------+-----------+---------+-----------+--------------+| IP Protocol | From Port | To Port | IP Range  | Source Group |+-------------+-----------+---------+-----------+--------------+| tcp         | 443       | 443     | 0.0.0.0/0 |              |+-------------+-----------+---------+------------+--------------+
Finally, I verified that the rules are now correct:
# nova secgroup-list-rules myapp                                                                                       +-------------+-----------+---------+-----------+--------------+| IP Protocol | From Port | To Port | IP Range  | Source Group |+-------------+-----------+---------+-----------+--------------+| tcp         | 443       | 443     | 0.0.0.0/0 |              || tcp         | 80        | 80      | 0.0.0.0/0 |              |+-------------+-----------+---------+-----------+--------------+
Of course, the real test was to see if I could now hit port 443 on my instance, and indeed I was able to.

Creativity, INC. – by Ed Catmull

Gridshore - Fri, 08/15/2014 - 20:39


A colleague of mine, Ronald Vonk, recommended this book to me. It is a book by one of the founders of Pixar, you know from all those fantastic computer animated movies. At pixar they created an continuous changing environment where creativity should excel. It is a very interesting read if you are interested in management books that are not to heavy on theories. Ed explains very well and entertaining how they went from a small company with a vision to a huge company with a vision.

Without to much spoilers, you really need to reed the book yourself, I want to mention a few things that I remembered after reading the book.

The team is more important than the idea’s or the talent of the separate people. Take care of the team, make sure they function well and give them responsibility. Make them feel proud when they finished what they wanted to create. Always put people first.

This is something I ran into in my normal working life as well. I do think you have to enable the teams to adept and to stay as a good team. The challenge is to get others in to learn and later on replace team members or start their own team.

We would never make a film that way again. It is the managements job to take the long view, to intervene and protect our people from their willingness to pursue excellence at all costs. Not to do so would be irresponsible.

This was a remark after delivering a movie under great time stress. They pulled through, but at a cost.

Braintrust – Group of people giving feedback and ideas for improvements on a certain idea. Important is that the feedback is meant to improve the idea, not to bully the person(s) the idea originated from. It is very important that everybody is open to the feedback and not defensive. In the end it is not the braintrust that makes a decision, it is the person in charge for the product. Still this group of people is kind of the first user and therefore the feedback should not be taken to lightly.

This was something I had a long thought about, my conclusion was that I am not really good at this. I often do feel that my ideas are my babies that need to be defended. First persuade me I am wrong, o sorry, an idea that someone had was not the best.

I did not want to become a manager, I just wanted to be one of the boys and do research. When we became bigger I realised I became more important and new people did not see me as a peer or one of the guys. I realised things were starting to get hidden from me. It is no problem as long as you trust people will tell someone else that will tell the most important things to me again.

Couldn’t agree more.

You can have this very nice polished finely tuned locomotive. People think that being the driver of the train is giving them power. They feel that driving the train in the end is shaping the company. The truth is, it’s not. Driving the train does not set it’s course. The real job is laying the track.

This was an eye opener a well, something you know but is hard to put into words.

At pixar they do not have contracts. They feel that employment contracts both hurt the employer as well as the employee. If someone had a problem with the company, there wasn’t much point in complaining because they were under contract. If someone didn’t perform well, on the other hand, there was no point in confronting them about it; their contract simply wouldn’t be renewed, which might be the first time they heard about their need to improve. The whole system discouraged and devaluated day-to-day communication and was culturally dysfunctional. But since everybody was used to it, they were blind to the problem.

This is a long one, have thought about it for a while. I think for now I would be to scared to do this in a company, still I like the idea.

What is the point of hiring smart people if you don’t empower them to fix what’s broken? Often to much time is lost in making sure no mistakes will be made. Often however, it just takes a few days to find solutions for mistakes.

Keeps coming back to the same point, a manager is a facilitator, nothing more nothing less. It is a very important role, just like all the others. Think about it, it is the team, the complete team.

The post Creativity, INC. – by Ed Catmull appeared first on Gridshore.

Categories: Architecture, Programming

Stuff The Internet Says On Scalability For August 15th, 2014

Hey, it's HighScalability time:


Somehow this seems quite appropriate. (via John Bredehoft)
  • 75 acres: Pizza eaten in US daily; 270TB: Backblaze storage pod; 14nm: Intel extends Moore's Law
  • Quotable Quotes
    • discreteevent: The dream of reuse has made a mess of many systems.
    • David Crawley: Don't think of Moore's Law in terms of technology; think of it in terms of economics and you get much greater understanding. The limits of Moore's Law is not driven by current technology. The limits of Moore's Law are really a matter of cost.
    • Simon Brown: If you can't build a monolith, what makes you think microservices are the answer?
    • smileysteve: The net result is that you should be able to transmit QPSK at 32GBd in 2 polarizations in maybe 80 waves in each direction. 2bits x 2 polarizations x 32G ~128Gb/s per wave or nearly 11Tb/s for 1 fiber. If this cable has 6 strands, then it could easily meet the target transmission capacity [60TB].
    • Eric Brumer: Highly efficient code is actually memory efficient code.

  • How to be a cloud optimist. Tell yourself: an instance is half full, it's not half empty; Downtime is temporary; Failures aren't your fault.

  • Mother Earth, Motherboard by Neal Stephenson. Goes without saying it's gorgeously written. The topic: The hacker tourist ventures forth across the wide and wondrous meatspace of three continents, chronicling the laying of the longest wire on Earth. < Related to Google Invests In $300M Submarine Cable To Improve Connection Between Japan And The US.

  • IBM compares virtual machines and against Linux containers: Our results show that containers result in equal or better performance than VM in almost all cases. Both VMs and containers require tuning to support I/O-intensive applications.

  • Does Psychohistory begin with BigData? Of a crude kind, perhaps. Google uses BigQuery to uncover patterns of world history: What’s even more amazing is that this analysis is not the result of a massive custom-built parallel application built by a team of specialized HPC programmers and requiring a dedicated cluster to run on: in stark contrast, it is the result of a single line of SQL code (plus a second line to create the initial “view”). All of the complex parallelism, data management, and IO optimization is handled transparently by Google BigQuery. Imagine that – a single line of SQL performing 2.5 million correlations in just 2.5 minutes to uncover the underlying patterns of global society.

  • Fabian Giesen with an deep perspective on how communication has evolved to use a similar pattern. Networks all the way down (part2): anything we would call a computer these days is in fact, for all practical purposes, a heterogeneous cluster made up of various specialized smaller computers, all connected using various networks that go by different names and are specified in different standards, yet are all suspiciously similar at the architecture level; a fractal of switched, packet-based networks of heterogeneous nodes that make up what we call a single “computer”. It means that all the network security problems that plague inter-computer networking also exist within computers themselves. Implementations may change substantially over time, the interfaces – protocols, to stay within our networking terminology – stay mostly constant over large time scales, warts and all.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Hamsterdb: An Analytical Embedded Key-value Store

 

In this post, I’d like to introduce you to hamsterdb, an Apache 2-licensed, embedded analytical key-value database library similar to Google's leveldb and Oracle's BerkeleyDB.

hamsterdb is not a new contender in this niche. In fact, hamsterdb has been around for over 9 years. In this time, it has dramatically grown, and the focus has shifted from a pure key-value store to an analytical database offering functionality similar to a column store database. 

hamsterdb is single-threaded and non-distributed, and users usually link it directly into their applications. hamsterdb offers a unique (at least, as far as I know) implementation of Transactions, as well as other unique features similar to column store databases, making it a natural fit for analytical workloads. It can be used natively from C/C++ and has bindings for Erlang, Python, Java, .NET, and even Ada. It is used in embedded devices and on-premise applications with millions of deployments, as well as serving in cloud instances for caching and indexing.

hamsterdb has a unique feature in the key-value niche: it understands schema information. While most databases do not know or care what kind of keys are inserted, hamsterdb supports key types for binary keys...

Categories: Architecture

Success Articles for Work and Life

"Success consists of going from failure to failure without loss of enthusiasm." -- Winston Churchill

I now have more than 300 articles on the topic of Success to help you get your game on in work and life:

Success Articles

That’s a whole lot of success strategies and insights right at your fingertips. (And it includes the genius from a wide variety of sources including  Scott Adams, Tony Robbins, Bruce Lee, Zig Ziglar, and more.)

Success is a hot topic. 

Success has always been a hot topic, but it seems to be growing in popularity.  I suspect it’s because so many people are being tested in so many new ways and competition is fierce.

But What is Success? (I tried to answer that using Zig Ziglar’s frame for success.)

For another perspective, see Success Defined (It includes definitions of success from Stephen Covey and John Maxwell.)

At the end of the day, the most important definition of success, is the one that you apply to you and your life.

People can make or break themselves based on how they define success for their life.

Some people define success as another day above ground, but for others they have a very high, and very strict bar that only a few mere mortals can ever achieve.

That said, everybody is looking for an edge.   And, I think our best edge is always our inner edge.

As my one mentor put it, “the fastest thing you can change in any situation is yourself.”  And as we all know, nature favors the flexible.  Our ability to adapt and respond to our changing environment is the backbone of success.   Otherwise, success is fleeting, and it has a funny way of eluding or evading us.

I picked a few of my favorite articles on success.  These ones are a little different by design.  Here they are:

Scott Adam’s (Dilbert) Success Formula

It’s the Pebble in Your Shoe

The Wolves Within

Personal Leadership Helps Renew You

The Power of Personal Leadership

Tony Robbins on the 7 Traits of Success

The Way of Success

The future is definitely uncertain.  I’m certain of that.   But I’m also certain that life’s better with skill and that the right success strategies under your belt can make or break you in work and life.

And the good news for us is that success leaves clues.

So make like a student and study.

Categories: Architecture, Programming

The Easy Way of Building a Growing Startup Architecture Using HAProxy, PHP, Redis and MySQL to Handle 1 Billion Requests a Week

This Case Study is a guest post written by Antoni Orfin, Co-Founder and Software Architect at Octivi

In the post I'll show you the way we developed quite simple architecture based on HAProxy, PHP, Redis and MySQL that seamlessly handles approx 1 billion requests every week. There’ll be also a note of the possible ways of further scaling it out and pointed uncommon patterns, that are specific for this project.

Stats:
Categories: Architecture

The AngularJS Promise DSL

Xebia Blog - Mon, 08/11/2014 - 10:21

As promised in my previous post, I just pushed the first version of our "Angular Promise DSL" to Github. It extends AngularJS's $q promises with a number of helpful methods to create cleaner applications.

The project is a V1, it may be a bit rough around the edges in terms of practical applicability and documentation, but that's why it's open source now.

The repository is at https://github.com/fwielstra/ngPromiseDsl and licensed as MIT. It's the first OS project I've created, so bear with me. I am accepting pull requests and issues, of course.

Questions? Ask them on the issues page, ask me via Twitter (@frwielstra) or send me an e-mail. I'd offer you to come by my office too... if I had one.

Introduction to big data presentation

I presented big data to Amdocs’ product group last week. One of the sessions I did was recorded so I might be able to add here later. Meanwhile you can check out the slides.

Note that trying to keep the slide visual I put some of the information is in the slide notes and not on the slides themselves.

Big data Overview from Arnon Rotem-Gal-Oz

Categories: Architecture

Are You Doing Agile Results?

If you already use Agile Results as your personal results system, you have a big advantage.

Why?

Because most people are running around, scrambling through a laundry list of too many things to do, a lack of clarity around what the end result or outcomes should be, and a lack of clarity around what the high-value things to focus on are.  They are using their worst energy for their most important things.  They are spending too much time on the things that don’t matter and not enough time on the things that do.   They are feeling at their worst, when they need to feel at their best, and they are struggling to keep up with the pace of change.

I created Agile Results to deal with the chaos in work and life, as a way to rise above the noise, and to easily leverage the most powerful habits and practices for getting better results in work and life.

Agile Results, in a nutshell, is a simple system for mastering productivity and time management, while at the same time, achieving more impact, realizing your potential, and feeling more fulfillment.

I wrote about the system in the book Getting Results the Agile Way.  It’s been a best seller in time management.

How Does Agile Results Work?

Agile Results works by combining proven practices for productivity, time management, psychology, project management, and some of the best lessons learned on high-performance.   And it’s been tested for more than a decade under extreme scenarios and a variety of conditions from individuals to large teams.

Work-Life balance is baked into the system, but more importantly Agile Results helps you live your values wherever you are, play to your strengths, and rapidly learn how to improve your results in an situation.  When you spend more time in your values, you naturally tap into your skills and abilities that help bring out your best.

The simplest way to think of Agile Results is that it helps you direct your attention and apply your effort on the things that count.  By spending more time on high-value activities and by getting intentional about your outcomes, you dramatically improve your ability to get better results.

But none of that matters if you aren’t using Agile Results.

How Can You Start Using Agile Results?

Start simple.

Simply ask yourself, “What are the 3 wins, results, or outcomes that I want for today?.”   Consider the demands you have on your plate, the time and energy you’ve got, and the opportunities you have for today, and write those 3 things down.

That’s it.   You’re doing Agile Results.

Of course, there’s more, but that’s the single most important thing you can do to immediately gain clarity, regain your focus, and spend your time and energy on the most valuable things.

Now, let’s assume this is the only post you ever read on Agile Results.   Let’s take a fast walkthrough of how you could use the system on a regular basis to radically and rapidly improve your results on an ongoing basis.

How I Do Agile Results? 


Here’s a summary of how I do Agile Results.

I create a new monthly list at the start of each month that lists out all the things that I think I need to do, and I bubble up 3 of my best things I could achieve or must get done to the top.   I look at it at the start of the week, and any time I’m worried if I’m missing something.  This entire process takes me anywhere from 10-20 minutes a month.

I create a weekly list at the start of the week, and I look at it at the start of each day, as input to my 3 target wins or outcomes for the day, and any time I’m worried if I’m missing anything.   This tends to take me 5-10 minutes at the start of the week.

I barely have to ever look at my lists – it’s the act of writing things down that gives me quick focus on what’s important.   I’m careful not to put a bunch of minutia in my lists, because then I’d train my brain to stop focusing on what’s important, and I would become forgetful and distracted.  Instead, it’s simple scaffolding.

Each day, I write a simple list of what’s on my mind and things I think I need to achieve.   Next, I step back and ask myself, “What are the 3 things I want to accomplish today?”, and I write those down.   (This tends to take me 5 minutes or less.  When I first started it took me about 10.)

Each Friday, I take the time to think through three things going well and three things to improve.   I take what I learn as input into how I can simplify work and life, and how I can improve my results with less effort and more effectiveness.   This takes me 10-20 minutes each Friday.

How Can You Adopt Agile Results?

Use it to plan your day, your week, and your month.

Here is a simple recipe for adopting Agile Results and using it to get better results in work and life:

  1. Add a recurring appointment on your calendar for Monday mornings.  Call it Monday Vision.   Add this text to the body of the reminder: “What are your 3 wins for this week?”
  2. Add a recurring appointment on your calendar to pop up every day in the morning.  Call it Daily Wins.  Add this text to the body of the reminder: “What are your 3 wins for today?”
  3. Add a recurring appointment on your calendar to pop up every Friday mid-morning.  Call it Friday Reflection.  Add this text to the body of your reminder:  What are 3 things going well?  What are 3 things to improve?”
  4. On the last day of the month, make a full list of everything you care about for the next month.   Alphabetize the list.  Identify the 3 most important things that you want to accomplish for the month, and put those at the top of the list.   Call this list  Monthly Results for Month XYZ.  (Note – Alphabetizing your list helps you name your list better and sort your list better.  It’s hard to refer to something important you have to do if you don’t even have a name for it.  If naming the things on your list and sorting them is too much to do, you don’t need to.  It’s just an additional tip that helps you get even more effective and more efficient.)
  5. On Monday of each week, when you wake up, make a full list of everything you care about accomplishing for the week.  Alphabetize the list.  Identify the 3 most important things you want to accomplish and add that to the top of the list.  (Again, if you don’t want to alphabetize then don’t.)
  6. On Wednesdays, in the morning, review the three things you want to accomplish for the week to see if anything matters that you should have spent time on or completed by now.  Readjust your priorities and focus as appropriate.  Remember that the purpose of having the list of your most important outcomes for the week isn’t to get good at predicting what’s important.  It’s to help you focus and to help you make better decisions about what to spend time on throughout the week.  If something better comes along, then at least you can make a conscious decision to trade up and focus on that.  Keep trading up.   And when you look back on Friday, you’ll know whether you are getting better at trading up or if you are just getting randomize or focusing on the short-term but hurting the long term.
  7. On Fridays,  in the morning, do your Friday Reflection.  As part of the exercise, check against your weekly outcomes and your monthly outcomes that you want to accomplish.  If you aren’t effective for the week, don’t ask “why not,” ask “how to.”   Ask how can you bite off better things and how can you make better choices throughout the week.  Just focus on little behavior changes, and this will add up over time.  You’ll get better and better as you go, as long as you keep learning and changing your approach.   That’s the Agile Way.

There are lots of success stories by other people who have used Agile Results.   Everybody from presidents of companies to people in the trenches, to doctors and teachers, to teams and leaders, as well as single parents and social workers.

But none of that matters if it’s not your story.

Work on your success story and just start getting better results, right here, right now.

What are the three most important things you really want to accomplish or achieve today?

Categories: Architecture, Programming

Stuff The Internet Says On Scalability For August 8th, 2014

Hey, it's HighScalability time:


Physicists reveal the scaling behaviour of exotic giant molecules.
  • 5 billion: Transistors Intel manufactures each second; 396M: WeChat active users
  • Quotable Quotes:
    • @BenedictEvans: Every hour or so, Apple ships phones with something around 2x more transistors than were in all the PCs on earth in 1995.
    • @robgomes: New client. Had one of their employees tune an ORM-generated query. Reduced CPU by 99.999%, IO by 99.996%.  Server now idle.
    • @pbailis: As a hardware-oriented systems builder, I'd pay attention to, say, ~100 ns RTTs via on-chip photonic interconnects
    • @CompSciFact: "Fancy algorithms are buggier than simple ones, and they're much harder to implement." -- Rob Pike's rule No. 4
    • @LusciousPear: I'm probably doing in Google what would have taken 5-8 engineers on AWS.
    • C. Michael Holloway, NASA: To a first approximation, we can say that accidents are almost always the result of incorrect estimates of the likelihood of one or more things.
    • Stephen O'Grady: More specific to containers specifically, however, is the steady erosion in the importance of the operating system. 

  • Wait, I thought mobile meant making single purpose apps? Mobile meant tearing down the portal cathedrals built by giants of the past. Then Why aren’t App Constellations working?: The App Constellation strategy works when you have a core resource which can be shared across multiple apps. 

  • Decentralization: I Want to Believe. The irony is mobile loves centralization, not p2p. Mobile IP addresses change all the time and you can't run a server on a phone. The assumption that people want decentralization has been disproven. Centralized services have won. People just want a service that works. The implementation doesn't matter that much. Good discussion on HackerNews and on Reddit.

  • Myth: It takes less money to start a startup these days. Sort of.  Why the Structural Changes to the VC Industry Matter: It turns out that, while it is in fact cheaper to get started and enter the market, it also requires more money for the breakout companies to win the market. Ultimately, today’s winners have a chance to be a lot bigger. But winning requires more money for geographic expansion, full-stack support of multiple new disciplines, and product expansion. And these companies have to do all of this while staying private for a much longer period of time; the median for money raised by companies prior to IPO has doubled in the past five years. 

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Extending AngularJS services with the Decorate method

Xebia Blog - Fri, 08/08/2014 - 12:00

Many large Angular applications tend to see a lot of repetition - same API endpoint, same method of dealing with and transforming data, etcetera. One technique you can use to at least alleviate that is using AngularJS's decorate method, which allows you to extend, adjust or even fully replace any existing service.

As you'll see in this post, using this allows you to modify and extend the framework you build your app in, which will lead to a cleaner, more legible codebase, written in a more functional style (the what, not the how).

Update 11/8: The follow-up is now live, along with the GitHub repository.

A feature not often used when developing AngularJS applications is the $provide service, which is the primary service used to register components with the $injector. More commonly, a developer would use methods like $provide.service() or $provide.factory to do so, but those are merely utility methods defined in the $provide service and exposed via angular.module().

The main reasons to use $provide over the service() and factory() methods is to configure the service before it's instantiated, for example. While there may be more advanced use-cases for using $provide, I haven't yet encountered them while developing regular applications and I'm sure they won't occur often.

One of the methods listed at the very bottom of the $provide documentation is the decorate() method. It doesn't look like much (it's at the bottom, after all), but its documentation hints that it's very powerful:

"A service decorator intercepts the creation of a service, allowing it to override or modify the behaviour of the service. The object returned by the decorator may be the original service, or a new service object which replaces or wraps and delegates to the original service."

Nothing to add there. You can use decorate() to change, add to, or completely replace the behaviour of services without having to edit its code. This can be done on any code not your own - core AngularJS services, but also third-party libraries. It's the equivalent of overriding methods in OO languages or monkey-patching in the more dynamic languages.

“Isn’t that evil?”, I hear you ask. As with every programming-related question, the only correct answer is: it depends. I’m going to give a few practical examples of when I believe using decorate() is appropriate. In a future blog post, I'll expand on this example, showing how relatively simple code can positively influence your entire application architecture.

Here’s a practical example, a neat follow-up on my previous blog about angular promises: decorating $q to add methods to the promise object. The promise API itself defines only one method: then(). $q adds a few simple methods to that like catch() and finally(), but for your own application you can add a few more.

If you’ve been working with promises for a little while in your AngularJS application, you’ve probably noticed some operations are pretty common; assigning the promise result to the scope (or any object), logging the result in the console, or calling some other method. Using decorate(), we can add methods to the promise object to simplify those. Here's a bit of code from my previous post; we'll add a method to $q to remove the need for a callback:

CustomerService.getCustomer(currentCustomer)
 .then(CartService.getCart)
 .then(function(cart) {
   $scope.cart = cart;
 })
 .catch($log.error);

First, we’ll need to do some boilerplate: we create a function that adds our methods to the promise object, and then we replace all the default promise methods. Note that the decorating function will also apply itself to the given promise.then method again, so that our customisations aren’t lost further down a promise chain:

angular.module('ngPromiseDsl', [])
  .config(function ($provide) {
    $provide.decorator('$q', function ($delegate, $location) {

      // decorating method

      function decoratePromise(promise) {
        var then = promise.then;

        // Overwrite promise.then. Note that $q's custom methods (.catch and .finally) are implemented by using .then themselves, so they're covered too.

        promise.then = function (thenFn, errFn, notifyFn) {
          return decoratePromise(then(thenFn, errFn, notifyFn));
        };

        return promise;
      }

      // wrap and overwrite $q's deferred object methods

      var defer = $delegate.defer,
        when = $delegate.when,
        reject = $delegate.reject,
        all = $delegate.all;

      $delegate.defer = function () {
        var deferred = defer();
        decoratePromise(deferred.promise);
        return deferred;
      };

      $delegate.when = function () {
        return decoratePromise(when.apply(this, arguments));
      };

      $delegate.reject = function () {
        return decoratePromise(reject.apply(this, arguments));
      };

      $delegate.all = function () {
        return decoratePromise(all.apply(this, arguments));
      };

      return $delegate;

    });
  });

With that boilerplate in place, we can now start adding methods. As I mentioned earlier, one of the most common uses of a then() function is to set the result onto the scope (or some other object). This is a fairly trivial operation, and it’s pretty straightforward to add it to the promise object using our decorator, too:

function decoratePromise(promise) {
  var then = promise.then;

  promise.then = function (thenFn, errFn, notifyFn) {
    return decoratePromise(then(thenFn, errFn, notifyFn));
  };

  // assigns the value given to .then on promise resolution to the given object under the given varName
  promise.thenSet = function (obj, varName) {
    return promise.then(function (value) {
      obj[varName] = value;
      return value;
    });
  };

  return promise;
}

That’s all. Put this .config block in your application's module definition, or create a new module and add a dependency to it, and you can use it throughout your application. Here's the same piece of code, now with our new thenSet method:

CustomerService.getCustomer(currentCustomer)
  .then(CartService.getCart)
  .thenSet($scope, 'cart')
  .catch($log.error);

This particular example can be extended in a multitude ways to add useful utilities to promises. In my current project we’ve added a number of methods to the promise object, which allows us to reduce the number of callback definitions in our controllers and thus create cleaner, more legible code.

 

Replacing custom callbacks with named methods allows for a more functional programming style, and allows readers to read and write code as a list of “whats”, instead of “hows” - and it's also fully asynchronous.

Extending $q is just the start though: Every angular service can be extended for various purposes - add performance monitoring and logging to $http, set common prefixes or fixed properties on $resource urls or template paths, you name it. Leave a remark in the comments about how you've used decorate() to create a better application.

Stay tuned for an upcoming post where I release a small open source project that extends angular’s promise objects with a number of helpful methods to perform common tasks.

Further reading, resources, helpful links:

The Principles of Modern Management

Are your management practices long in the tooth?

I think I was lucky that early on, I worked in environments that shook things up and rattled the cage in pursuit of more customer impact, employee engagement, and better organizational performance.

In one of the environments, a manufacturing plant, the management team flipped the typical pyramid of the management hierarchy upside down to reflect that the management team is there to empower and support the production line.

And when I was on the Microsoft patterns & practices team, we had an interesting mix of venture capitalist type management coupled with some early grandmasters of the Agile movement.   More than just Agile teams, we had an Agile management culture that encouraged a customer-connected approach to product development, complete with self-organizing, multi-disciplinary teams, empowered people, a focus on execution excellence, and a fierce focus on being a rapid learning machine. 

We thrived on change.

We also had a relentless focus on innovation.  Not just in our product, but in our process.  If we didn’t innovate in our process, then we got pushed out of market by becoming too slow, too expensive, or by lacking the quality experience that customers have come to expect.

But not everybody knows what a great environment for helping people thrive and do great things for the world, looks like.

While a lot of people in software or in manufacturing have gotten a taste of Agile and Lean practices, there are many more businesses that don’t know what a modern learning machine of people and processes that operate at a higher-level looks like. 

Many, many businesses and people are still operating and looking at the world through the lens of old world management principles.

In the book The Future of Management, Gary Hamel walks through the principles upon which modern management is based.

The Principles of Modern Management

Hamel gives us a nice way to frame looking at the modern management principles, by looking at their application, and their intended goal.

Via The Future of Management:

Principle Application Goal Standardization Minimize variances from standards around inputs, outputs, and work methods. Cultivate economies of scale, manufacturing efficiency, reliability, and quality. Specialization (of tasks and functions) Group like activities together in modular organizational units. Reduce complexity and accelerate learning. Goal alignment Establish clear objectives through a cascade of subsidiary goals and supporting metrics. Ensure that individual efforts are congruent with top-down goals. Hierarchy Create a pyramid of authority based on a limited span of control. Maintain control over a broad scope of operations. Planning and control Forecast demand, budget resources, and schedule tasks, then track and correct deviations from plan. Establish regularity and predictability in operations; conformance to plans. Extrinsic rewards Provide financial rewards to individuals and teams for achieving specified outcomes. Motivate effort and ensure compliance with policies and standards. What are the Principles Upon Which Your Management Beliefs are Based?

Most people aren’t aware of the principles behind the management beliefs that they practice or preach.  But before coming up with new ones, it helps to know what current management thinking is rooted in.

Via The Future of Management:

“Have you ever asked yourself, what are the deepest principles upon which your management beliefs are based? Probably not.  Few executives, in my experience, have given much thought to the foundational principles that underlie their views on how to organize and manage.  In that sense, they are as unaware of their management DNA as they are of their biological DNA.  So before we set off in search of new management principles, we need to take a moment to understand the principles that comprise our current management genome, and how those tenets may limit organizational performance.”

A Small Nucleus of Core Principles

It really comes down to a handful of core principles.  These principles serve as the backbone for much of today’s management philosophy.

Via The Future of Management:

“These practices and processes of modern management have been built around a small nucleus of core principles: standardization, specialization, hierarchy, alignment, planning, and control, and the use of extrinsic rewards to shape human behavior.”

How To Maximize Operational Efficiency and Reliability in Large-Scale Organizations

It’s not by chance that the early management thinkers came to the same conclusions.  They were working on the same problems in a similar context.  Of course, the challenge now is that the context has changed, and the early management principles are often like fish out of water.

Via The Future of Management:

“These principles were elucidated early in the 20th century by a small band of pioneering management thinkers -- individuals like Henri Fayol, Lyndall Urwick, Luther Gullick, and Max Weber. While each of these theorists had a slightly different take on the philosophical foundations of modern management, they all agreed on the principles just enumerated. This concordance is hardly surprising, since they were all focusing on the same problem: how to maximize operational efficiency and reliability in large-scale organizations. Nearly 100 years on, this is still the only problem that modern management is fully competent to address.”

If your management philosophy and guiding principles are nothing more than a set of hand me downs from previous generations, it might be time for a re-think.

You Might Also Like

Elizabeth Edersheim on Management Lessons of a Lifelong Student

How Employees Lost Empathy for their Work, for the Customer, and for the Final Product

No Slack = No Innovation

The Drag of Old Mental Models on Innovation and Change

The New Competitive Landscape

The New Realities that Call for New Organizational and Management Capabilities

Who’s Managing Your Company

Categories: Architecture, Programming

A Valuable Sprint Review (a.k.a. Demo): How To

Xebia Blog - Thu, 08/07/2014 - 10:05

A valuable Sprint Review (from now on in this blog referred to as Demo) can be built in three steps. It starts during the Sprint planning session with agreeing on and understanding the user stories on the Sprint backlog. Then, during the Sprint, the team constantly exchanges ideas and results of the realisation of the Story. Finally, during the demo itself, the Product Owner and the rest of the team demo the stories to the stakeholders to display the value delivered and open up for feedback.

Planning for a good demo

During the planning session, it is imperative that the Product Owner and the rest of the team understand the stories that will be picked up. This sounds obvious, but it happens often that this is not the case. Stories might be too technical so the Product Owner is disconnected or stories are so high level that it is hard to determine what needs to be done.

Make sure stories are formulated from the perspective of an end-user of the functionality that will be delivered. This could be an actual user, a system that picks up whatever result is created or any other manifestation of who or what will use the result of the story.

Also take care of getting the acceptance criteria clear. This way it will be clear to developers what to build, to testers what to test for and designers what to design. It will help the Product Owner to have a better idea what is in and what might have to be defined in a new/other user story.

It is important that everyone understands the context in which the story ‘lives’. What part of the system is touched (end-to-end is preferred but not always possible), which parties are affected by the change, what prerequisites are needed, etc.

Building for a great demo

When during the creation of the value of each story the whole team is in constant contact about intermediate results and decisions taken, everyone will be able to add to the value and be aware of what the result of the story will be. It is very important that the whole team is including the Product Owner. When the PO sees the intermediate results, she or he can already create an image of what the result will be like. Also, the PO can contact stakeholders that might have an opinion of what is created and, when needed, adjust the end result to match expectations.

Delivering an valuable demo

In the demo, the Product Owner should present to the stakeholders the value of each user story that has been delivered. So, per story, explain what has changed from the perspective of the end-user and have the rest of the team show this. Also, when stories are not done, explain which (sub-)functionality is not yet finished. Make sure to ask for feedback from the end-user or other stakeholders on what is demonstrated.

Conclusion

The value of the demo depends largely on the cooperation of the entire team. When the Product Owner and the rest of the team work together on understanding what will be delivered and help each other to get the most value from each story delivered the demo will be focused, valuable and fun.

Scrumban

Xebia Blog - Wed, 08/06/2014 - 19:19

Scrum has become THE revolution in the world of software development. The main philosophy behind scrum is accepting that a problem cannot be fully understood or defined at start; scrum has the focus on maximizing the team's ability to deliver quickly and respond to emerging requirements. It came as truly refreshing in the time when projects were ruled by procedure and MS-project planning. Because of scrum:

  1. Projects can deliver what the customer needs, not just what he thought he wanted.
  2. Teams are efficient. They work as a unit to reach a common goal.
  3. We have better project roles (like a product owner and scrum master), ceremonies (like daily stand-ups, grooming) and a scrumboard.

But the central question is: "are we there yet"? And the answer is: "No!". We can optimize scrum by mixing it with kanban, which leads to scrumban.

A kanban introduction for scrummers

Instead of scrum, which is a software development framework in the widest sense of the term, kanban is a method. It, or instance, does not define ceremonies and project roles. There are two main principles in kanban I would like to highlight:

  1. Each column on the kanban board represents a step in the workflow. So, instead of the lanes 'todo', 'inprogress' and 'done' like in scrum, you have 'defining', 'developing', 'testing' and 'deploying'. That is a more full-stack view; a task has a wider lifecycle. This concept is also called 'from concept to cash'; from user research and strategic planning to data center operations and product support.
  2. Another principle of Kanban is that it limits WIP (work in progress). An example of a WIP limit is limiting the number of cards allowed in each column. The advantage is that it reveals bottlenecks dynamically. Because of the WIP, Kanban is a pull mechanism. For instance, a tester can only pickup a next work item if there are items available in de done-column of development-lane and when the WIP limit of the test-lane isn't exceeded.

After all kanban is incredibly simple, but at the same time incredibly powerful.

What's wrong with scrum?

  1. The reason why we went to scrum is because we did not want the waterfall approach anymore. But, in fact each sprint in scrum has become a mini waterfall. In each sprint teams plan, try to design, develop and test. At the end the product owner reviews the completed work and decides which of the stories are shippable and ready for production. Those sprints can result in a staccato flow, which can be exhausted. With kanban we can make sprints more agile and the goal is to have a more continuous flow. In comparison with how to run a marathon? You don't make sprints of 200 meters, but rather with a constant rate.
  2. Scrum is a push mechanism and therefore 'pushes' the quality out of your product. When a sprint backlog is defined, the team is asked for a commitment. Whatever happened, a team must satisfy its commitment and at the end of the sprint the product owner must say 'decharge', else the team has failed. No team wants to publicly fail, so most of the time, at the end of the sprint, teams take shortcuts to satisfy the deadline. Those shortcuts are killing for quality! Asking for commitment is like not trusting the intrinsic motivation of the team. The correct commitment is visible during each standup. During a standup team members have to tell each other what they've done the day before. If they are working too long on a story, another team member will rebel. That is the real commitment.
  3. One of the reasons why we do scrum is that it is better to start immediately instead of doing an estimation and a feasibility study upfront, because almost always after the study is completed, the project will be executed. The estimation at the start is not reliable and the feasibility study is just a waste of time. Isn't that the same mistake we make with scrum with the grooming and ready sessions that causes a lot of overhead? The first overhead during grooming is that we give an estimation with relative precision. It is in a developer's nature to argue about the story points; is it 3, 5, 8 or maybe 1 points? And that is a waste. You should only talk about the story sizes large, medium or small. Making a more precise estimation is just a waste of time, because there are too many external factors. Second, with the grooming we do a mini feasibility study. With a team we will think about a direction of the solution, which is fine. But most of the time it takes two or three sprints before it is realized in the sprint. And with all the weekends of beer in between we've already forgot the solutions. So one smart guy says: 'yes, lets document it', but that is an inefficient way for the real problem: there is too much time between the grooming and the realization.

Scrumban: the board of kanban

Untitled

A scrumban board

The first column in a scrumban board is reserved for the backlog, where stories are ordered by their respective priority. There are several rules for the kanban backlog:

  1. It is the product owner's responsibility to fill this lane with stories, and keep it steadily supplied. The product owner must avoid creating or analyzing too many stories, because this is a waste and it corrupts with the Just-In-Time principle of scrumban. Therefore the scrumban board has a WIP-limit of 5 backlog stories.
  2. Assure the necessary level of analysis before starting development. Each story must be analyzed with a minimum effort. That should be done in the Weekly Time Box (WTB), which will be discussed later on.
  3. The backlog should be event-driven with an order point.
  4. Prioritization-on-demand. The ideal work planning process should always provide the team with best things to work on next, no more and no less.

Next to the backlog-column is the tasking-column, in which there should always be at least one story that is tasked (a minimum WIP-limit). If this isn't the case the team will task after the standup to satisfy this condition. A story is picked up from the backlog and is tasked by the team. Tip: put the tasked cards at the back of the story cards. The next columns are the realization columns. Each team is free to add, remove or change necessary columns so it suits the business. In the realization columns there should be a maximum number of stories that are worked on. If the maximum limit has not been reached, a story can be pulled from the tasking column and unfolded on the 'to implement' lane. Now the team can work on the tasks of the story. Each task that is implemented can be moved to the 'ready' lane. If all of the tasks are done for a story, the story can be moved to the next lane. When the story and tasks are ready, the cards can be moved to the right bottom of the board, so there is a new horizontal lane available for the next story.

Scrumban: the ceremonies of scrum

With scrumban we only have two types of meetings: the daily standup and the weekly timeblock. The Weeky Timeblock is a recurring meeting used for multiple purposes. It should be set up in the middle of the week. The big advantage of the weekly timeblock is that developers are distracted from their work only once a week (instead of the various of meetings with scrum).

The Weekly Timeblock contains three parts. First there is a demo of the work done. Second, there is a retrospective on the development process of the last week. Third, the team should have a preview of upcoming workitems. The team try to understand the intent of each item and provide feedback. The only size-indication a team has to make is if the story is small, medium or large. Avoid using poker cards/story points, which are too fine-grained and are to vague.

Conclusion

Scrumban is a mix of the scrum ceremonies and the kanban method. With scrumban we

  1. Introduce the weekly timeblock (WTB). The weekly timeblock should be around 4 hours and there are no more meetings
  2. Have a wider lifecycle of a story: 'from concept to cash'.
  3. Change the scrumboard to  company flows and avoid the push principle of a sprint but using a pull mechanism.

Tokutek White Paper: A Comparison of Log-Structured Merge (LSM) and Fractal Tree Indexing

What data structure does your database use? It's not something the typical database user spends much time pondering. Since data structure is destiny, what data structure your database uses is a key point to consider in your selection process.

We know CouchDB uses a modified B+ tree. We've learned a lot fascinating details over the years about the use of Log-structured merge-trees in Cassandra, HBase and LevelDB. So B+ trees and LSMs seem familiar by now.

What may not be so familiar is Tokutek's Fractal Tree Indexing technology that is supposed to be even better than B+ trees and LSMs.

As a comparison between Fractal Tree Indexing and LSMs, Bradley Kuszmaul, Chief Architect at Tokutek, has written a detailed paper, a must read for the algorithmically inclined or someone interested in database internals: A Comparison of Log-Structured Merge (LSM) and Fractal Tree Indexing.

Here's a quick intro to Fractal Tree (FT) indexes:

Categories: Architecture

Kanban should be the default choice for DevOps teams

Xebia Blog - Wed, 08/06/2014 - 14:59

We had a successful workshop on DevOpsDays 2014. Our main point was that Kanban should be the default choice for DevOps teams. The presentation can be downloaded here.

DevOpsDays 2014 was a success

On the 19th, 20th & 21st of June 2014 the second edition of DevOpsDays Amsterdam was held in Pakhuis De Zwijger in Amsterdam. This year I was asked to teach a course there on Kanban for DevOps. At the 2013 edition I also gave a presentation about this subject and it was nice to be invited back to this great event.

With the Open Source mindset in mind I teamed up with Maarten Hoppen en Bas van Oudenaarde. Our message: Kanban should be the default choice for DevOps teams. 

The response to this workshop was very positive and because we received a lot of great feedback I thought I’d share the slide deck. The presentation assumes you are working in an environment where DevOps might work or is already being implemented. 

Main points of the presentation

DevOps is about Culture, Automation, Measurement and Sharing (CAMS). These four values require a way of working that looks past existing processes, handovers and role descriptions. 

The Kanban Method is about looking at your organization in a different way. From a point of:

  • Sustainability: by shaping demand and limiting Work in Progress
  • Service-Orientation: by creating an SLA based on past results and data
  • Survivability: create an improvement mindset in the organization to respond to rapidly changing environments

The three different ways the Kanban Method makes you look at your organization makes it an extremely powerful solution for DevOps.

If you are interested to learn more about the workshop, check out the slides here:

http://www.slideshare.net/jsonnevelt/kanban-bootcamp-devopsdays-2014

Sponsored Post: Apple, Gawker, FoundationDB, Monitis, Cie Games, BattleCry, Surge, Cloudant, CopperEgg, Logentries, Couchbase, MongoDB, BlueStripe, AiScaler, Aerospike, AppDynamics, ManageEngine, Site24x7

Who's Hiring?
  • Apple has multiple openings. Changing the world is all in a day's work at Apple. Imagine what you could do here. 
    • Software Developer in Test. The iOS Systems team is looking for a Quality Assurance engineer. In this role you will be expected to work hand-in-hand with the software engineering team to find and diagnose software defects. The ideal candidate will also seek out ways to further automate all aspects of our existing process. This is a highly technical role and requires in-depth knowledge of both white-box and black-box testing methodologies. Please apply here
    • Senior Software Engineer -iOS Systems. Do you love building highly scalable, distributed web applications? Does the idea of a fast-paced environment make your heart leap? Do you want your technical abilities to be challenged every day, and for your work to make a difference in the lives of millions of people? If so, the iOS Systems Carrier Services team is looking for a talented software engineer who is not afraid to share knowledge, think outside the box, and question assumptions. Please apply here.
    • Software Engineering Manager, IS&T WWDR Dev Systems. The WWDR development team is seeking a hands-on engineering manager with a passion for building large-scale, high-performance applications. The successful candidate will be collaborating with Worldwide Developer Relations (WWDR) and various engineering teams throughout Apple. You will lead a team of talented engineers to define and build large-scale web services and applications. Please apply here.
    • C++ Senior Developer and Architect- Maps. The Maps Team is looking for a senior developer and architect to support and grow some of the core backend services that support Apple Map's Front End Services. Ideal candidate would have experience with system architecture, as well as the design, implementation, and testing of individual components but also be comfortable with multiple scripting languages. Please apply here.

  • Systems & Networking Lead at Gawker. We are looking for someone to take the initiative on the lowest layers of the Kinja platform. All the way down to power and up through hardware, networking, load-balancing, provisioning and base-configuration. The goal for this quarter is a roughly 30% capacity expansion, and the goal for next quarter will be a rolling CentOS7 upgrade as well as to planning/quoting/pitching our 2015 footprint and budget. For the full job spec and to apply, click here: http://grnh.se/t8rfbw

  • Cie Games, small indie developer and publisher in LA, is looking for rock star Senior Game Java programmers to join our team! We need devs with extensive experience building scalable server-side code for games or commercial-quality applications that are rich in functionality. We offer competitive comp, great benefits, interesting projects, and exceptional growth opportunities. Check us out at http://www.ciegames.com/careers.

  • BattleCry, the newest ZeniMax studio in Austin, is seeking a qualified Front End Web Engineer to help create and maintain our web presence for AAA online games. This includes the game accounts web site, enhancing the studio website, our web and mobile- based storefront, and front end for support tools. http://jobs.zenimax.com/requisitions/view/540

  • FoundationDB is seeking outstanding developers to join our growing team and help us build the next generation of transactional database technology. You will work with a team of exceptional engineers with backgrounds from top CS programs and successful startups. We don’t just write software. We build our own simulations, test tools, and even languages to write better software. We are well-funded, offer competitive salaries and option grants. Interested? You can learn more here.

  • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here.

  • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here.
Fun and Informative Events
  • OmniTI has a reputation for scalable web applications and architectures, but we still lean on our friends and peers to see how things can be done better. Surge started as the brainchild of our employees wanting to bring the best and brightest in Web Operations to our own backyard. Now in its fifth year, Surge has become the conference on scalability and performance. Early Bird rate in effect until 7/24!
Cool Products and Services
  • Couchbase, MongoDB and DataStax: Compared. Find out which database delivers great read/write latency while scaling well with both read-intensive and balanced workloads. Get the initial results here: http://info.couchbase.com/2014-Benchmark-Showdown-Results-LP.html.

  • Now track your log activities with Log Monitor and be on the safe side! Monitor any type of log file and proactively define potential issues that could hurt your business' performance. Detect your log changes for: Error messages, Server connection failures, DNS errors, Potential malicious activity, and much more. Improve your systems and behaviour with Log Monitor.

  • The NoSQL "Family Tree" from Cloudant explains the NoSQL product landscape using an infographic. The highlights: NoSQL arose from "Big Data" (before it was called "Big Data"); NoSQL is not "One Size Fits All"; Vendor-driven versus Community-driven NoSQL.  Create a free Cloudant account and start the NoSQL goodness

  • Finally, log management and analytics can be easy, accessible across your team, and provide deep insights into data that matters across the business - from development, to operations, to business analytics. Create your free Logentries account here.

  • CopperEgg. Simple, Affordable Cloud Monitoring. CopperEgg gives you instant visibility into all of your cloud-hosted servers and applications. Cloud monitoring has never been so easy: lightweight, elastic monitoring; root cause analysis; data visualization; smart alerts. Get Started Now.

  • Whitepaper Clarifies ACID Support in Aerospike. In our latest whitepaper, author and Aerospike VP of Engineering & Operations, Srini Srinivasan, defines ACID support in Aerospike, and explains how Aerospike maintains high consistency by using techniques to reduce the possibility of partitions.  Read the whitepaper: http://www.aerospike.com/docs/architecture/assets/AerospikeACIDSupport.pdf.

  • BlueStripe FactFinder Express is the ultimate tool for server monitoring and solving performance problems. Monitor URL response times and see if the problem is the application, a back-end call, a disk, or OS resources.

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Cloud deployable. Free instant trial, no sign-up required.  http://aiscaler.com/

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Categories: Architecture