Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator/categories/7&page=1' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Architecture
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

Start with Needs and Wants

“The purpose of a business is to create a customer.” – Peter Drucker

So many people start with solutions, and then wonder where the customers are.

It’s the proverbial, “When all you have is a hammer, everything looks like a nail.”

The truth is, if all you have is a hammer, then get better at finding nails.  And while you are looking for those nails, get better at expanding your toolbox.

If you want to be a better Entrepreneur or a trend hunter or a product manager or a visionary leader, then start with needs and wants.  It will help you quickly cut through the overwhelm and overload of ideas, trends, and insights to get to the ideas that matter.

Some say the most valuable thing in the world is ideas.  Many others say that coming up with ideas is not the problem.  The problem is execution.  The truth here is that so many ideas fail because they didn’t create a customer or raving fans.  They didn’t address relevant pains, needs, and desired outcomes.  Instead, they solve problems that nobody has or create things that nobody wants (unless it’s free), besides the creator, and that’s how you end up in the mad scientist syndrome.  Or, ideas die because they were not presented in a way that speaks to the needs and wants, and so you end up a brilliant, misunderstood genius.

Start Viewing the World Through the Lens of Human Needs and Wants

Here is some good insight and timeless truths on how to find trends that matter and how to create ideas that do, too from the 5 Trends for 2016 report by Trendwatching.com.

Via 5 Trends for 2016:

“Trends emerge as innovators address consumers’ basic needs and wants in novel ways.
As trend watchers, that’s why we look for clusters of innovations which are defining (and redefining) customer expectations.

Start by asking why customers might embrace you using a channel. Next, challenge whether existing channels really satisfy the deep needs and wants of your customers. Could you create any new ones? Finally, imagine entirely new contexts you could leverage (perhaps even those that customers aren’t yet consciously aware of).

As long as the onslaught of technological change continues, we’ll keep shouting this mantra from the rooftops: stop viewing the world through the lens of technology, and start viewing technology through the lens of basic human needs and wants.

Put another way: all those tech trends you’re obsessed with are fine, but can you use them to deliver something people actually want?”

Start with Scenarios to Validate Customer Pains, Needs, and Desired Outcomes

A scenario is simply a story told from the customer's point of view that explains their situation and what they want to achieve.

They are a great tool for validating ideas, capturing ideas, and sharing ideas.  What makes them so powerful is that they are a story told in the Voice-of-the-Customer (VOC).  The Current State story captures the pains and needs.  The Desired Future State captures the vision of the desired outcomes.  Here is an example:

Current State
As a product manager, I'm struggling to keep up with changing customer behavior and band perception is eroding.  Competition from new market entrants is creating additional challenges as we face new innovations, lower prices, and better overall customer experiences.

Desired Future State
By tapping into the vast amounts of information from social media, we gain deep customer insight.  We find new opportunities to better understand customer preferences and perceptions of the brand.  We combine social data with internal market data to gain deeper insights into brand awareness and profitable customer segments.  Employees are better able to share ideas, connect with each other, connect with customers, and connect with partners to bring new ideas to market.  We are able to pair up with the key influencers in social media to help reshape the story and perception of our brand.

Customer Wants and Needs are the Breeding Ground of Innovation

Makes total sense right?   But how often do you see anybody ever do this?  That’s the real gap.

Instead, we see hammers not even looking for nails, but trying to sell hammers.

But maybe people want drills?  No, they don’t want to by drills or drill-bits.  They want to buy holes.  And when you create that kind of clarity, you start to get resourceful and you can create ideas and solutions in a way that’s connected to what actually counts.

You Might Also Like

6 Steps for Enterprise Architecture as Strategy

10 High-Value Activities in the Enterprise

Agile Methodology in Microsoft patterns & practices

Customer-Connected Engineering

How To Turn IT into an Asset Rather than a Liability

Scenario-Driven Value Realization

Why So Many Ideas Die

Categories: Architecture, Programming

Hoe om te gaan met Start-ups

Xebia Blog - Fri, 01/29/2016 - 16:15

Dit is een vraag die regelmatig door mijn hoofd speelt. In ieder geval moeten we stoppen met het continu romantiseren van deze initiatieven en als corporate Nederland nou eens echt mee gaan spelen.

Maar hoe?

Grofweg zijn er twee strategieën als corporate: opkopen of zelf beter doen! Klinkt simpel, maar is toch best wel complex. Waarschijnlijk is de beste strategie om een mix te kiezen van beide, waarbij je maximaal je eigen corporate kracht gebruikt (ja, die heb je), en tegelijkertijd volledig de kracht van start-up innovatie kunt gebruiken.

Deze post verkent de mogelijkheden en je moet vooral verder lezen, als ook jij wilt weten hoe jij de digitalisering van de 21ste eeuw wilt overleven.

Waarom moet ik hier iets mee?

Eigenlijk hoef ik deze alinea natuurlijk niet meer te schrijven toch? De gemiddelde leeftijd van bedrijven neemt af.
avg age fortune 500
Dit komt mede doordat de digitalisering van producten en diensten de entree barrières in veel markten steeds lager maken. Er is dus meer concurrentie en daarom moet je beter je best doen om relevant te blijven.
Ten tweede, start-ups zijn hot! Iedereen wil voor een start-up werken en dus gaat het talent daar ook heen. Talent uit een toch al (te) krappe pool. Daarom moet je meer dan voorheen innoveren, omdat je anders de “war on talent” verliest.
Als laatste is er natuurlijk veel te winnen met digitale innovatie. De snelheid waarmee bedrijven tegenwoordig via digitale producten en diensten winstgevend kunnen worden is ongelofelijk, dus doe je het goed, dan doe je mee.

Wat zijn mijn mogelijkheden?

Er zijn eigenlijk maar twee manieren om met start-ups om te gaan. De eerste is om simpelweg een aandeel te nemen in een start-up of een veelbelovende start-up over te nemen. De andere is om zelf te gaan innoveren vanuit de eigen kracht in de organisatie.

De voordelen van aandelen en overnames is natuurlijk de snelle winst. De huid wordt vaak niet goedkoop verkocht, maar dan heb je ook wat. Helemaal mooi is het, de start-up actief is in een segment of markt, waar je zelf met je brand niet in wilt of kunt zitten (incumbent inertia). De nieuwe aanwinst is dan complementair aan de bestaande business. Bijvoorbeeld een grote bank, die een start-up overneemt die gericht is op het verkopen van kortlopend krediet aan midden- en kleinbedrijf.

Het nadeel is natuurlijk dat het moeilijk is om de bestaande assets over te hevelen. Bovendien wordt het nieuwe bijna nooit echt een onderdeel van de staande organisatie en misschien wil je dat ook helemaal niet. De kans bestaat namelijk dat de overgenomen start-up te veel beïnvloed wordt door het moeder bedrijf en daarom meegetrokken wordt door de zwaartekracht van bureaucratie en contraproductieve bestaande corporate gedragspatronen.

Daarbij komt, dat een succesvolle start-up vanzelf onderhevig wordt aan aanvallen van weer andere start-ups. De “kannibalen mindset” moet er in blijven! Facebook heeft daarom altijd gezegd, als wij ons eigen model niet disrupten, doet iemand anders dat wel. Misschien is het waar wat de CEO van Intel Andy Grove eens zei: “only the paranoid survive”.

Zelf innovatiever worden is natuurlijk ook een optie. Dat is echter behoorlijk complex. Meestal wordt innovatie binnen de corporate nog in een lab-setting geïsoleerd. Niet dat dit fout is, maar start-ups doen natuurlijk zoiets niet he. De start-up is namelijk het lab!

Het is grappig dat het lijkt alsof start-ups altijd nieuwe markten met nieuwe producten proberen te bereiken en dat we dit doorgaans bestempelen als “echte” innovatie. In een corporate setting worden namelijk alle product-marktcombinaties in een portfolio gezet (bijvoorbeeld een BCG-matrix) en gaat het om balans tussen huidige business en nieuwe business en de juiste cashflow verhoudingen.
Nou is het leuke dat start-ups maling hebben aan jouw portfolio en dus in elk kwadrant concurreren, zij het business die voor jou in de volwassenheid zit, of in de groei of in je lab-fase. Start-ups zijn simpelweg in de meerderheid en opereren los van elkaar op verschillende fronten. Dit betekent dat feitelijk iedereen in de corporate setting onder vuur ligt door start-ups en dus dat ook iedereen ongeacht de rol in het portfolio moet leren innoveren. Een voorbeeld van hoe waardevol het is om deze mindset te adopteren is het verhaal van deze jongeman uit 1973, hij werkte voor Kodak.

Je hele bedrijf veranderen is natuurlijk een ontzettende klus en als alternatief zou je ook kunnen kiezen om hetzelfde effect te creëren als bij een overname; het bewust lanceren van eigen start-ups voldoende los opgezet van de moederorganisatie om te versnellen. Deze eigen start-ups moeten directe concurrenten worden van de huidige business en zo succesvol worden dat iig een deel van de bestaande eigen en concurrerende business daar naartoe stroomt. Grote corporates kunnen zich op die manier meer en meer omvormen tot een netwerk van start-up nodes, waarbij het moeder bedrijf ondersteund en strategische complementaire nodes aankoopt waar nodig. Hoe zo’n node er uit ziet en hoe je dit organiseert is voer voor een volgende post.

Mocht je niet kunnen wachten en dit eerder willen weten dan kun je natuurlijk altijd bellen voor een gesprek.

Tinder: How does one of the largest recommendation engines decide who you'll see next?

We've heard a lot about the Netflix recommendation algorithm for movies, how Amazon matches you with stuff, and Google's infamous PageRank for search. How about Tinder? It turns out Tinder has a surprisingly thoughtful recommendation system for matching people.

This is from an extensive profile, Mr. (Swipe) Right?, on Tinder founder Sean Rad:

Categories: Architecture

Tinder: How does one of the largest recommendation engines decide who you'll see next?

We've heard a lot about the Netflix recommendation algorithm for movies, how Amazon matches you with stuff, and Google's infamous PageRank for search. How about Tinder? It turns out Tinder has a surprisingly thoughtful recommendation system for matching people.

This is from an extensive profile, Mr. (Swipe) Right?, on Tinder founder Sean Rad:

Categories: Architecture

Backlog ordering done right!

Xebia Blog - Wed, 01/27/2016 - 11:00

Various methods exist for helping product owners to decide which backlog item to start first. That this pays off to do so (more or less) right has been shown in blogs of Maurits Rijk and Jeff Sutherland.

These approaches to ordering backlog items all assume that items once picked up by the team are finished according to the motto: 'Stop starting, start finishing'. An example of a well-known algorithm for ordering is Weighted Shortest Job First (WSJF).

For items that may be interrupted, this results not in the best scheduling possible. Items that usually are interrupted by other items include story map slices, (large) epics, themes, Marketable Features and possibly more.

In this blog I'll show what scheduling is more optimal and how it works.

ruitenwisser_dots

Weighted Shortest Job First (WSJF)

In WSJF scheduling of work, i.e. product backlog items, is based on both the effort and (business) value of the item. The effort may be stated in duration, story points, or hours of work. The business value may be calculated using Cost of Delay or as is prescribed by SAFe.

When effort and value are known for the backlog items, each item can be represented by a dot. See the picture to the right.
The proper scheduling is obtained by sweeping the dashed line from the bottom right to the upper left (like a windshield wiper).

 

ruitenwisserIn practice both the value and effort are not precisely known but estimated. This means that product owners will treat dots that are 'close' to each other the same. The picture to the left shows this process. All green sectors have the same ROI (business value divided by effort) and have roughly the same value for their WSJF.

Product owners will probably schedule items according to: green cells from left-to-right. Then consider the next 'row' of cells from left-to-right.

 

Other Scheduling Rules

It is known at least since the 1950's (and probably earlier) that WSJF is the most optimal scheduling mechanism if both value and size are known. The additional condition is that preemption, i.e. interruption of the work, is not allowed.

If either of these 3 conditions (known value, known size, no preemption) is not valid, WSJF is not the best mechanism and other scheduling rules are more optimal. Other mechanisms are (for a more comprehensive overview and background see e.g. Table 3.1, page 146 in [Kle1976]):

No preemption allowed

  1. no value, no effort: FIFO
  2. only effort: SJF / SEPT
  3. only value: on value
  4. effort & value: WSJF / SEPT/C
  5. Story map slices: WSJF (no preemption)

FIFO = First in, First out
SEPT = Shortest Expected Processing Time
SJF = Shortest Job First
C = Cost

Examples: (a) user stories on the sprint backlog: WSJF, (b) production incidents: FIFO or SJF, (c) story map slices that represent a minimal marketable feature (or short Feature). Leaving out a single user story from a Feature creates no business value (that's why it is a minimal marketable feature) and starting such a slice also means completing it before starting anything else. These are scheduled using WSJF. (d) User stories that are part of Feature; they represent no value by themselves, but all are necessary to complete the Feature they belong to. Schedule these according to SJF.

Preemption allowed

  1. no value: SIRPT (SIJF)
  2. effort & value: SIRPT/C or WSIJF (preemption)
  3. SIRPT = Shortest Imminent Remaining Processing Time

SIRPT/C = Shortest Imminent Remaining Processing Time, weighted by Cost
SIJF = Shortest Imminent Job First
WSIJF = Weighted Shortest Imminent Job First

The 'official' naming for WSIJF is SIRPT/C. In this blog I'll use Weighted Shortest Imminent Job First, or WSIJF.

Examples: (a) story map slices that contain more than one Feature (minimal marketable feature). We call these Feature Sets. These are scheduled using WSIJF, (b) (Large) Epics that consist of more than 1 Feature Set, or epics that are located at the top-right of the windshield-wiper-diagram. The latter are usually split in smaller one containing most value for less effort. Use WSIJF.

Summary

  • User Story (e.g. on sprint backlog and not part of a Feature): WSJF
  • User Story (part of a Feature): SJF
  • Feature: WSJF
  • Feature Set: WSIJF
  • Epics, Story Maps: WSIJF
Weighted Shortest Imminent Job First (WSIJF)

Mathematically, WSIJF is not as simple to calculate as is WSJF. Perhaps in another blog I'll explain this formula too, but in this blog I'll just describe what WSIJF does in words and show how it affects the diagram with colored sections.

WSIJF: Work that is very likely to finish in the next periods, has large priority

What does this mean?

Remember that WSIJF only applies to work that is allowed to be preempted in favour of other work. Preemption happens at certain points in time. Familiar examples are Sprints, Releases (Go live events), or Product Increments as used in the SAFe framework.

The priority calculation takes into account:

  • the probability (or chance) that the work is completed in the next periods,
  • if completed in the next periods, the expected duration, and
  • the amount of time already spent.

ruitenwisser_dots_wsijfExample. Consider a Scrum team that has a cadence of 2-week sprints and time remaining to the next release is 3 sprints. For every item on the backlog determine the chance for completing it in the next sprint and if completed, divide by the expected duration. Likewise for completing the same it in the next 2 and 3 sprints. For each item you'll get 3 numbers. The value divided by the maximum of these is the priority of the backlog item.

Qualitatively, the effect of WSIJF is that items with large effort get less priority and items with smaller effort get larger priority. This is depicted in the diagram to the right.

Example: Quantifying WSIJF

In the previous paragraph I described the basics of WSIJF and only qualitatively indicated its effect. In order to make this concrete, let's consider large epics that have been estimated using T-shirt sizes. Since WSIJF affects the sizing part and to less extent the value part, I'll not consider the value in this case. In a subtle manner value also plays a role, but for the purpose of this blog I'll not discuss it here.

Teams are free to define T-shirt sizes as they like. In this blog, the following 5 T-shirt sizes are used:

  • ­XS ~ < 1 Sprint
  • S ~ 1 – 2 Sprints
  • M ~ 3 – 4 Sprints
  • L ~ 5 – 8 Sprints
  • XL ~ > 8 Sprints

Items of size XL take around 8 sprints, so typically 4 months. These are very large items.

probabilitiesOf course, estimates are just what they are: estimates. Items may take less or more sprints to complete. In fact, T-shirt sizes correspond to probability distributions: an 'M'-sized item has a probability to complete earlier than 3 sprints or may take longer than 4 sprints. For these distributions I'll take:

  • ­XS ~ < 1 Sprint (85% probability to complete within 1 Sprint)
  • ­S ~ 1 – 2 Sprints (85% probability to complete within 3 Sprints)
  • ­M ~ 3 – 4 Sprints (85% probability to complete within 6 Sprints)
  • ­L ~ 5 – 8 Sprints (85% probability to complete within 11 Sprints)
  • ­XL ~ > 8 Sprints (85% probability to complete within 16 Sprints)

As can be seen from the picture, the larger the size of the item the more uncertainty in completing it in the next period.

Note: for the probability distribution, the Wald or Inverse Gaussian distribution has been used.

Based on these distributions, we can calculate the priorities according to WSIJF. These are summarized in the following table:

wsijf_table

Column 2 specifies the probability to complete an item in the next period, here the next 4 sprints. In the case of an 'M' this is 50%.
Column 3 shows that, if the item is completed, what the expected duration will be. For an 'M' sized item this is 3.22 Sprints.
Column 4 contains the calculated priority as 'value of column 2' divided by 'value of column 3'.
The last column shows the value as calculated using SJF.

The table shows that items of size 'S' have the same priority value in both the SIJF and SJF schemes. Items larger than 'S' actually have a much lower priority as compared to SJF.

Note: there are slight modifications to the table when considering various period lengths and taking into account the time already spent on items. This additional complexity I'll leave for a future blog.

In practice product owners only have the estimated effort and value at hand. When ordering the backlog according to the colored sections shown earlier in this blog, it is easiest to use a modified version of this picture:

ruitenwisser_done_right

Schedule the work items according to the diagram above, using the original value and effort estimates: green cells from left to right, then the next row from left to right.

Conclusion

Most used backlog prioritization mechanisms are based on some variation of ROI (value divided by effort). While this is the most optimal scheduling for items for which preemption is not allowed, it is not the best way to schedule items that are allowed to be preempted.

As a guide line:

  • Use WSJF (Weighted Shortest Job First) for (smaller) work items where preemption is not allowed, such as individual user stories with (real) business value on the sprint backlog and Features (minimal marketable features, e.g. slices in a story map).
  • Use SJF (Shortest Job First) for user stories within a Feature.
  • Use WSIJF (Weighted Shortest Imminent Job First) for larger epics and collections of Features (Feature Set), according to the table above, or more qualitatively using the modified sector chart.
References

[Kle1976] Queueing Systems, Vol. 2: Computer Applications, Version 2, Leonard Kleinrock, 1976

[Rij2011] A simulation to show the importance of backlog prioritisation, Maurits Rijk, June 2011, https://maurits.wordpress.com/2011/06/08/a-simulation-to-show-the-importance-of-backlog-prioritization/

[Sut2011] Why a Good Product Owner Will Increase Revenue at Least 20%, Jeff Sutherland, June 2011, https://www.scruminc.com/why-product-owner-will-increase-revenue/

Design of a Modern Cache

This is a guest post by Benjamin Manes, who did engineery things for Google and is now doing engineery things for a new load documentation startup, LoadDocs.

Caching is a common approach for improving performance, yet most implementations use strictly classical techniques. In this article we will explore the modern methods used by Caffeine, an open-source Java caching library, that yield high hit rates and excellent concurrency. These ideas can be translated to your favorite language and hopefully some readers will be inspired to do just that.

Eviction Policy

A cache’s eviction policy tries to predict which entries are most likely to be used again in the near future, thereby maximizing the hit ratio. The Least Recently Used (LRU) policy is perhaps the most popular due to its simplicity, good runtime performance, and a decent hit rate in common workloads. Its ability to predict the future is limited to the history of the entries residing in the cache, preferring to give the last access the highest priority by guessing that it is the most likely to be reused again soon...

Categories: Architecture

Demonstration of the Exactness of Little's Law

Xebia Blog - Sat, 01/23/2016 - 11:00

Day 18

Little's Law is a powerful tool that relates the amount the work a team is doing and the average lead time of each work item. Basically there are two main applications involving either 1) the input rate of work entering the team, or 2) the throughput of work completed.

In previous posts (Applying Little's Law in agile gamesWhy Little's law works...always) I already explained that Little's Law is exact and hardly has any assumptions, other than work entering the team (or system).

This post demonstrates this by calculating Little Law at every project day while playing GetKanban.

The video below clearly shows that Little's Law holds exactly at every project day. For both the input rate and throughput versions. Throughput is based on the subclass of 'completed' items.

E.g. on the yellow post-it the product of lambda and W equals N on every project day.

http://blog.xebia.com/wp-content/uploads/2016/01/LittlesLaw_540p.mp4

 

IMG_1436

The set-up is that we run the GetKanban game from day 9 through day 24. The video will show on the right hand side the board and charts whereas the left hand side shows the so-called 'sample path' and Little's Law calculation for both input rate (yellow post-it) and throughput (green post-it).

Sample Path. The horizontal axis shows the project day running from 9 till 24. The vertical axis shows the work item: each row represents a item on the board.

The black boxes mark the days that the work in on the board. For example, item 8 was in the system on project day 9 and completed at the end of project day 12 where it was deployed.

The collection of all black boxes is called a 'Sample Path'.

 

 

IMG_1433 copyLittle's Law. The average number of items in the system (N) is show on top. This is an average over the project days. Here W denotes the average lead-time of the items. This is an average taken over all work items.

Input rate: on the yellow post-it the Greek lambda indicates the average number of work per day entering the system.

Throughput: the green post-it indicates the average work per day completed. This is indicated by the Greek mu.

Note: the numbers on the green post-it are obtained by considering only the subclass of work that is completed (the red boxes).

References

[Rij2014a] http://blog.xebia.com/why-littles-law-works-always/

[Rij2014b] http://blog.xebia.com/applying-littles-law-in-agile-games/

Stuff The Internet Says On Scalability For January 22nd, 2016

Hey, it's HighScalability time:


The Imaginary Kingdom of Aurullia. A completely computer generated fractal. Stunning and unnerving.

 

If you like this Stuff then please consider supporting me on Patreon.
  • 42,000: drones from China securing the South China Sea; 1 billion: WhatsApp active users; 2⁻¹²²: odds of a two GUIDs with 122 random bits colliding; 25,000 to 70,000: memory chip errors per billion hours per megabit; 81,500: calories in a human body; 62: people as wealthy as half of world's population; 1.66 million: App Economy jobs in the US; 521 years: half-life of DNA; 0.000012%: air passenger fatalities; $1B: Microsoft free cloud resources for nonprofits; 4000-7000+: BBC stats collected per second; $1 billion: Google's cost to taste Apple's pie;

  • Quotable Quotes:
    • @mcclure111: 1995: Every object in your home has a clock & it is blinking 12:00 / 2025: Every object in your home has a IP address & the password is Admin
    • @notch: Coming soon to npm: tirefire.js, an asynchronous framework for implementing helper classes for reinventing the wheel. Based on promises.
    • @ayetempleton: Fun fact: You are MORE likely to win a million or more dollars in the #powerball lottery than to lose an #AWS #S3 object in a given year.
    • @viktorklang: IMO biggest lie in performance work: constant factors don't matter in Big-Oh.
    • Flavien Boucher: We all came to the conclusion that Docker is adding a complexity layer compare to a virtual machine approach, and this complexity will be for the deployment, development and build.
    • @Frances_Coppola: Uber is a cab cartel. And AirBNB is wealthy - though its suppliers aren't. They are simply firms with apps.
    • Susan Sontag: The method especially appeals to people handicapped by a ruthless work ethic – Germans, Japanese and Americans. Using a camera appeases the anxiety which the work driven feel about not working when they are on vacation and supposed to be having fun. They have something to do that is like a friendly imitation of work: they can take pictures.
    • @SachaNauta: "It's never been easier to be a billionaire and never been harder to be a millionaire" @profgalloway #DLD16
    • @Techmeme: Google Play saw 100% more downloads than iOS App Store, but Apple generated 75% more revenue 
    • Ryan Shea: we’ve concluded that 8MB blocks are simply too large to be considered safe for the network at this point in time, considering the current global bandwidth levels.
    • @RichRogersHDS: "In the old world you spent 30% of your time building a great service & 70% shouting about it. In the new world, that inverts." - Jeff Bezos
    • @thetinot: When you have an SDN, yes, networking throughput does grow on trees. Why @googlecloud is faster than #AWS and #Azure 
    • @GOettingerEU: Digital tech has contributed to around 1/3 of EU GDP growth in over the past decade and I believe this number will continue to grow #wef16
    • @COLRICHARDKEMP: More women fly F16s in Israel than drive cars in Saudi Arabia. KA. 
    • @JoshZumbrun: The total collapse in shopping mall construction
    • @jeffjarvis: 44 million people saw NY Fashion Show content on Instagram last year says Instagram's Marne Levine. Attn: Conde & Hearst!  #DLD16
    • @HackerNewsOnion: Developer Accused Of Unreadable Code Refuses To Comment
    • Lloyds online banking: in a 60-second period: 12,900 people visit its website, 400 bills are paid, 1,500 customers log onto the mobile app, 350 transfers are made and 3,000+ logins
    • @bdha: 2013: DevOps 2014: Docker 2015: Containers 2016: Unikernels 2017: Threads 2018: Syscalls 2019: Inodes
    • hacknat: Two things need to happen to make unikernels attractive. A new Hypervisor needs to get made, one that is just as extensible as an OS around the isolated primitives. It should also have something extra too (like the ability to fine tune resource management better than an OS can). Secondly a user friendly mechanism like Docker needs to happen.

  • It's a winner take all world, but not everywhere. Brian Brushwood on Cordkillers with an insightful breakdown of how the new diversified market for TV content has actually become far less of a winner take all system. We have more good content than ever. Gone are the days of Mash when everyone watched the same show at the same time. Is it bad that actors are making less? No. We are seeing the destruction of the tournament, as explained in the book Freakonomics, is the idea that those at the very top make all the money, those at the bottom of the pyramid make next to nothing. And the winners only have to win by a nose to reap all the rewards, the don't even need to win on merit. This is an inefficient system. Now we are reaching an artistically efficient system. If you have a story to tell and no budget you can tell it on YouTube. This is the democratization of talent. It's inconvenient for those who used to be at the top. What we have now is more working actors producing more content than ever.  And since a lot of this content does not have to pander to advertisers to get made the content is more diverse and more interesting than ever as well.

  • The RAMCloud Storage System: RAMCloud combines low-latency, large scale, and durability. Using state of the art networking with kernel bypass, RAMCloud expects small reads to complete in less that 10µs on a cluster of 10,000 nodes. This is 50 – 1,000 times faster that storage systems commonly in use.

  • All Change Please. Adrian Colyer makes the case that we are transitioning to a new part of the technology cycle that promises great change. Networking: 40Gbps and 100Gbps ethernet. Memory: battery backed RAM; 3D XPoint, MRAM, MeRAM, etc. Storage: NVRAM and fast PCIe. Processing: GPUs; integrated on processor FPGAs; hardware transactional memory. This is the question: What happens when you combine fast RDMA networks with ample persistent memory, hardware transactions, enhanced cache management support and super-fast storage arrays? It’s a whole new set of design trade-offs that will impact the OS, file systems, data stores, stream processing, graph processing, deep learning and more. And this is before we’ve even introduced integration with on-board FPGAs, and advances in GPUs…

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Why does Unikernel Systems Joining Docker Make A Lot of Sense?

Unikernel Systems Joins Docker. Now this is an interesting match. The themes are security and low overhead, though they do seem to solve the same sort of problem.

So, what's going on?

In FLOSS WEEKLY 302 Open Mirage, starting at about 10 minutes in, there are a series of possible clues. Dr. Anil Madhavapeddy, former CTO of Unikernel Systems, explains their motivation behind the creation of unikernels. And it's a huge and exciting vision...

Categories: Architecture

Building An Infinitely Scaleable Online Recording Campaign For David Guetta

This is a guest repost of an interview posted by Ryan S. Brown that originally appeared on serverlesscode.com. It continues our exploration of building systems on top of Lambda.

Paging David Guetta fans: this week we have an interview with the team that built the site behind his latest ad campaign. On the site, fans can record themselves singing along to his single, “This One’s For You” and build an album cover to go with it.

Under the hood, the site is built on Lambda, API Gateway, and CloudFront. Social campaigns tend to be pretty spiky – when there’s a lot of press a stampede of users can bring infrastructure to a crawl if you’re not ready for it. The team at parall.ax chose Lambda because there are no long-lived servers, and they could offload all the work of scaling their app up and down with demand to Amazon.

James Hall from parall.ax is going to tell us how they built an internationalized app that can handle any level of demand from nothing in just six weeks.

The Interview
Categories: Architecture

Achieve The Unthinkable using Hyper-Sprints

Xebia Blog - Wed, 01/20/2016 - 13:17
 33177940

2015-06-25 AMSTERDAM - Wereldkampioen sprint Dafne Schippers poseert naast de Nuna 7S van het Nuon Solar Team. De atlete neemt het in Olympisch Stadion op tegen het Nuon Solar Team, de wereldkampioen zonneracen. Projecten zoals Nuna en Forze worden door Hardware Scrum coaches van Xebia begeleid.

In my opinion, the best indicator how "agile" teams actually are, is their sprint length.  The theory says 2-4 weeks. To be honest, as an agile coach, this doesn’t feel agile all the time.

Like I wrote in one of my previous posts, in my opinion the ultimate form of agility is nature. Nature’s sprint length seems to vary from billions of years how the universe is created to a fraction of a second how matter is formed.

Of course, it’s nonsense stating we could end up in sprints of just a few nano-seconds.  But on the other hand, we see our society is speeding up dramatically. Where a service or product could take years before it went to market a couple of years ago, now it can be a matter of days, even hours.  Think about the development of disruptive apps and technology like Uber and 3D-printing.

In these disruptive examples a sprint length of 2 weeks can be a light year.  Even in Scrum we can be trapped in our patterns here. Why don’t we experiment with shorter sprint lengths?  All agile rituals are relative in time; during build parties and hackathons I often use sprints of only 30 or 60 minutes; 5 mins for planning, 45 mins for the sprint, 5 mins for the review/demo, 5 mins for the retrospective.  Combined with a fun party atmosphere and competition, this creates a hyper-productive environment.

Try some hyper sprinting next to your regular sprints. You’ll be surprised how ultra-productive and fun they are. For example, it enables your team to build a car in just an afternoon. Enjoy!

Automated deployment of Docker Universal Control Plane with Terraform and Ansible

Xebia Blog - Tue, 01/19/2016 - 20:06

You got into the Docker Universal Control Plane beta and you are ready to get going, and then you see a list of manual commands to set it up. As you don't want to do anything manually, this guide will help you setup DUCP in a few minutes by using just a couple of variables. If you don't know what DUCP is, you can read the post I made earlier. The setup is based on one controller, and a configurable amount of replicas which will automatically join the controller to form a cluster. There a few requirements we need to address to make this work, like setting the external (public) IP while running the installer and passing the controller's certificate fingerprint to the replicas during setup. We will use Terraform to spin up the instances, and Ansible to provision the instances and let them connect to each other.

Prerequisites

Updated 2016-01-25: v0.7 has been released, and no Docker Hub account is needed anymore because the images are moved to public namespace. This is updated in the 'master' branch on the Github repository. If you still want to try v0.6, you can checkout tag 'v0.6'!

Before you start cloning a repository and executing commands, let's go over the prerequisites. You will need:

  • Access to the DUCP beta (during installation you will need to login with a Docker Hub account which is added to the 'dockerorca' organization, tested with v0.5v0.6 and v0.7. Read update notice above for more information.)
  • An active Amazon Web Services and/or Google Cloud Platform account to create resources
  • Terraform (tested with v0.6.8 and v0.6.9)
  • Ansible (tested with 1.9.4 and 2.0.0.1)

Step 1: Clone the repository

CiscoCloud's terraform.py is used as Ansible dynamic discovery to find our Terraform provisioned instances, so --recursive is needed to also get the Git submodule.

git clone --recursive https://github.com/nautsio/ducp-terraform-ansible
cd ducp-terraform-ansible

Step 2.1: AWS specific instructions

These are the AWS specific instructions, if you want to use Google Cloud Platform, skip to step 2.2.

For the AWS based setup, you will be creating an aws_security_group with the rules in place for HTTPS (443) en SSH (22). With Terraform you can easily specify what we need by specifying ingress and egress configurations to your security group. By specifying 'self = true' the rule will be applied to the all resources in that to be created security group. In the single aws_instance for the ducp_controller we use the lookup function to get the right AMI from the list specified in vars.tf. Inside each aws_instance we can reference the created security group by using "${aws_security_group.ducp.name}". This is really easy and it keeps the file generic. To configure the amount of instances for ducp-replica we are using the count parameter. To identify each instance to our Ansible setup, we specify a name by using the tag parameter. Because we use the count parameter, we can generate a name by using a predefined string (ducp-replica) and add the index of the count to it. You can achieve this by using the concat function like so: "${concat("ducp-replica",count.index)}". The sshUser parameter in the tags block is used by Ansible to connect to the instances. The AMIs are configured inside vars.tf and by specifying a region, the correct AMI will be selected from the list.

variable "amis" {
    default = {
        ap-northeast-1 = "ami-46c4f128"
        ap-southeast-1 = "ami-e378bb80"
        ap-southeast-2 = "ami-67b8e304"
        eu-central-1   = "ami-46afb32a"
        eu-west-1      = "ami-2213b151"
        sa-east-1      = "ami-e0be398c"
        us-east-1      = "ami-4a085f20"
        us-west-1      = "ami-fdf09b9d"
        us-west-2      = "ami-244d5345"
    }
}

The list of AMIs

    ami = "${lookup(var.amis, var.region)}"

The lookup function

Let's configure the variables so you can use it to create the instances. Inside the repository you will find a terraform.tfvars.example file. You can copy or move this file to terraform.tfvars so that Terraform will pick it up during a plan or apply.

cd aws
cp terraform.tfvars.example terraform.tfvars

Open terraform.tfvars with your favorite text editor so you can set up all variables to get the instances up and running.

  • region can be selected from available regions
  • access_key and secret_key can be obtained through the console
  • key_name is the name of the key pair to use for the instances
  • replica_count defines the amount of replicas you want

The file could look like the following:

region = "eu-central-1"
access_key = "string_obtained_from_console"
secret_key = "string_obtained_from_console"
key_name = "my_secret_key"
replica_count = "2"

By executing terraform apply you can create the instances, let's do that now. Your command should finish with:

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

 Step 2.2: Google Cloud Platform specific instructions

In GCP, it's a bit easier to set everything up. Because there are no images/AMI's per region, we can use a disk image with a static name. And because the google_compute_instance has a name variable, you can use the same count trick as we did on AWS, but this time without the metadata. By classifying the nodes with the tag https-server, it will automatically open port 443 in the firewall. Because you can specify the user that should be created with your chosen key, setting the ssh_user is needed to connect with Ansible later on.

Let's setup our Google Cloud Platform variables.

cd gce
cp terraform.tfvars.example terraform.tfvars

Open terraform.tfvars with your favorite text editor so you can set up all variables to get the instances up and running.

The file could look like the following:

project = "my_gce_project"
credentials = "/path/to/file.json"
region = "europe-west1"
zone = "europe-west1-b"
ssh_user = "myawesomeusername"
replica_count = "2"

By executing terraform apply you can create the instances, let's do that now. Your command should finish with:

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Step 3: Running Ansible

Instances should all be there, now let's install the controller and add a replica. This setup uses terraform.py to retrieve the created instances (and IP addresses) based on the terraform.tfstate file. To make this work you need to specify the location of the tfstate file by settings TERRAFORM_STATE_ROOT to the current directory. Then you specify the script to lookup the inventory (-i) and the site.yml where you can assign the roles to the hosts.

There are two roles that will be applied to all hosts, called common and extip. Inside common everything is set up to get Docker running on the hosts, so it configures the apt repo, installs the docker-engine package and finally installs the docker-py package because Ansible needs this to use Docker. Inside extip, there are two shell commands to lookup external IP addresses. Because if you want to access DUCP on the external IP, it should be present inside the certificate that DUCP generates. Because the external IP addresses are not found on GCP instances and I wanted a generic approach where you would only need one command to provision both AWS and GCP I chose to look them up, and eventually register the variable extip with the one that was correctly looked up on the instances. Second reason to use the external IP, is that all the replicas need the external IP of the controller to register themselves. By passing the --url parameter to the join command, you can specify to what controller it should register.

--url https://"{{ hostvars['ducp-controller']['extip'] }}"

The extip variable used by replica

This also counts for the certificates fingerprint, the replica should provide the fingerprint of the controllers certificate to register successfully. We can access that variable the same way: "{{ hostvars['ducp-controller']['ducp_controller_fingerprint'].stdout }}. It specifies .stdout to only use the stdout part of the command to get the fingerprint, because it also registers exitcode and so.

To supply external variables, you can inject vars.yml through --extra-vars. Let's setup the vars.yml by copying the example file to vars.yml and configure it.

cd ../ansible
cp vars.yml.example vars.yml

As stated before, the installer will login to the Docker Hub and download images that live under the dockerorca organization. Your account needs to be added to this organization to let it succeed. Fill in your Docker Hub account details in vars.yml, and choose an admin user and admin password for your installation. If you use ssh-agent to store your SSH private keys, you can proceed with the ansible-playbook command, else you can specify your private-key file by adding --private-key <priv_key_location> to the list of arguments.

Let's run Ansible to set up DUCP. You need to change to the directory where the terraform.tfstate file resides, or change TERRAFORM_STATE_ROOT accordingly.

cd ../{gce,aws}
TERRAFORM_STATE_ROOT=. ansible-playbook -i ../terraform.py/terraform.py \
                       ../ansible/site.yml \
                       --extra-vars "@../ansible/vars.yml"

If all went well, you should see something like:

PLAY RECAP *********************************************************************
ducp-controller            : ok=13   changed=9    unreachable=0    failed=0   
ducp-replica0              : ok=12   changed=8    unreachable=0    failed=0   
ducp-replica1              : ok=12   changed=8    unreachable=0    failed=0

To check out our brand new DUCP installation, run the following command to extract the IP addresses from the created instances:

TERRAFORM_STATE_ROOT=. ../terraform.py/terraform.py --hostfile

Copy the IP address listed in front of ducp-controller and open up a web browser. Prefix the address with https://<ip> and now you can login with your chosen username and password.

ducp login

Let me emphasise that this is not a production ready setup, but can definitely help if you want to try out DUCP and maybe build a production ready version from this setup. If you want support for other platforms, please file an issue on Github or submit a pull request. I'll be more than happy to look into it for you.

Sponsored Post: Netflix, Macmillan, Aerospike, TrueSight Pulse, LaunchDarkly, Robinhood, StatusPage.io, Redis Labs, InMemory.Net, VividCortex, MemSQL, Scalyr, AiScaler, AppDynamics, ManageEngine, Site24x7

Who's Hiring?
  • Manager - Site Reliability Engineering: Lead and grow the the front door SRE team in charge of keeping Netflix up and running. You are an expert of operational best practices and can work with stakeholders to positively move the needle on availability. Find details on the position here: https://jobs.netflix.com/jobs/398

  • Macmillan Learning, a premier e-learning institute, is looking for VP of DevOps to manage the DevOps teams based in New York and Austin. This is a very exciting team as the company is committed to fully transitioning to the Cloud, using a DevOps approach, with focus on CI/CD, and using technologies like Chef/Puppet/Docker, etc. Please apply here.

  • DevOps Engineer at Robinhood. We are looking for an Operations Engineer to take responsibility for our development and production environments deployed across multiple AWS regions. Top candidates will have several years experience as a Systems Administrator, Ops Engineer, or SRE at a massive scale. Please apply here.

  • Senior Service Reliability Engineer (SRE): Drive improvements to help reduce both time-to-detect and time-to-resolve while concurrently improving availability through service team engagement.  Ability to analyze and triage production issues on a web-scale system a plus. Find details on the position here: https://jobs.netflix.com/jobs/434

  • Manager - Performance Engineering: Lead the world-class performance team in charge of both optimizing the Netflix cloud stack and developing the performance observability capabilities which 3rd party vendors fail to provide.  Expert on both systems and web-scale application stack performance optimization. Find details on the position here https://jobs.netflix.com/jobs/860482

  • Senior Devops Engineer - StatusPage.io is looking for a senior devops engineer to help us in making the internet more transparent around downtime. Your mission: help us create a fast, scalable infrastructure that can be deployed to quickly and reliably.

  • Software Engineer (DevOps). You are one of those rare engineers who loves to tinker with distributed systems at high scale. You know how to build these from scratch, and how to take a system that has reached a scalability limit and break through that barrier to new heights. You are a hands on doer, a code doctor, who loves to get something done the right way. You love designing clean APIs, data models, code structures and system architectures, but retain the humility to learn from others who see things differently. Apply to AppDynamics

  • Software Engineer (C++). You will be responsible for building everything from proof-of-concepts and usability prototypes to deployment- quality code. You should have at least 1+ years of experience developing C++ libraries and APIs, and be comfortable with daily code submissions, delivering projects in short time frames, multi-tasking, handling interrupts, and collaborating with team members. Apply to AppDynamics
Fun and Informative Events
  • Aerospike, the high-performance NoSQL database, hosts a 1-hour live webinar on January 28 at 1PM PST / 4 PM EST on the topic of "From Development to Deployment" with Docker and Aerospike. This session will cover what Docker is and why it's important to Developers, Admins and DevOps when using Aerospike; it features an interactive demo showcasing the core Docker components and explaining how Aerospike makes developing & deploying multi-container applications simpler. Please click here to register.

  • Your event could be here. How cool is that?
Cool Products and Services
  • Dev teams are using LaunchDarkly’s Feature Flags as a Service to get unprecedented control over feature launches. LaunchDarkly allows you to cleanly separate code deployment from rollout. We make it super easy to enable functionality for whoever you want, whenever you want. See how it works.

  • TrueSight Pulse is SaaS IT performance monitoring with one-second resolution, visualization and alerting. Monitor on-prem, cloud, VMs and containers with custom dashboards and alert on any metric. Start your free trial with no code or credit card.

  • Turn chaotic logs and metrics into actionable data. Scalyr is a tool your entire team will love. Get visibility into your production issues without juggling multiple tools and tabs. Loved and used by teams at Codecademy, ReturnPath, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net

  • VividCortex measures your database servers’ work (queries), not just global counters. If you’re not monitoring query performance at a deep level, you’re missing opportunities to boost availability, turbocharge performance, ship better code faster, and ultimately delight more customers. VividCortex is a next-generation SaaS platform that helps you find and eliminate database performance problems at scale.

  • MemSQL provides a distributed in-memory database for high value data. It's designed to handle extreme data ingest and store the data for real-time, streaming and historical analysis using SQL. MemSQL also cost effectively supports both application and ad-hoc queries concurrently across all data. Start a free 30 day trial here: http://www.memsql.com/

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Also available on Amazon Web Services. Free instant trial, 2 hours of FREE deployment support, no sign-up required. http://aiscaler.com

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below...

Categories: Architecture

Agile Results for 2016

Agile Results is the personal productivity system for high-performance.

Agile Results is a “whole person” approach to personal productivity. It combines proven practices for mind, body, and emotions. It helps you realize your potential the agile way.  Best of all, it helps you make the most of what you’ve got to achieve higher levels of performance with less time, less effort, and more impact.

Agile Results helps you achieve rapid results by focusing on outcomes over activities, spending more time in your strengths, focusing on high-value activities, and using your best energy for your best results.

If you want to use Agile Results, it’s simple. I’ll show you how to get started, right, here, right now. If you already know Agile Results, then this will simply be a refresher.

Write Three Things Down

The way to get started with Agile Results is simple. Write three things down that you want to achieve today. Just ask yourself, “What are your Three Wins that you want to achieve today?”

For me, today, I want to achieve the following:

  1. I want to get agreement on a shared model across a few of our teams.
  2. I want to create a prototype for business model innovation.
  3. I want to create a distilled view of CEO concerns for a Mobile-First, Cloud-First world.

In my mind, I might just remember: shared model, business model innovation, and CEO. I’ll be focused on the outcomes, which are effectively agreement on a model, innovation in business models for a Mobile-First, Cloud-First world, and a clear representation of top CEO pains, needs, and desired outcomes.

Even if I throw away what I write down, or lose it, the value is in the brief moment I spent to prioritize and visualize the results that I want to achieve. 

This little vision will stick with me as a guide throughout my day.

Think in Three Wins

Writing these three items down, helps me focus. It helps me prioritize based on value. It also helps me create a simple vision for my day.

Plus, thinking in Three Wins adds the fun factor.

And, better yet, if somebody asks me tomorrow what my Three Wins were for yesterday, I should be able to tell a story that goes like this: I created rapport and a shared view with our partner teams, I created a working information model for business model innovation for a mobile-first cloud-first world, and I created a simplified view of the key priorities for CEOs in a Mobile-First, Cloud-First world.

When you can articulate the value you create, to yourself and others, it helps provide a sense of progress, and a story of impact.  Progress is actually one of the keys to workplace happiness, and even happiness in life.

In a very pragmatic way, by practicing your Three Wins, you are practicing how to identify and create value.  You are learning what is actually valued, by yourself and others, by the system that you are in.

And value is the ultimate short-cut.  Once you know what value is, you can shave off a lot of waste.

The big idea here is that it’s not your laundry list of To-Dos, activities, and reminders -- it’s your Three Wins or Three Outcomes or Three Results.

Use Your Best Energy for Your Best Results

Some people wonder why only Three Wins?  There is a lot of science behind the Rule of 3, but I find it better to look at how the Rule of 3 has stood the test of time.  The military uses it.  Marketing uses it.  You probably find yourself using it when you chunk things up into threes.

But don’t I have a bazillion things to do?

Yes. But can I do a bazillion things today? No. But what I can do is spend my best energy, on the best things, my best way.

That’s the best I can do.

But that’s actually a lot. When you focus on high-value outcomes and you really focus your time, attention, and energy on those high-value outcomes, you achieve a lot. And you learn a lot.

Will I get distracted? Sure. But I’ll use my Three Wins to get back on track.

Will I get randomized and will new things land on my plate? Of course, it’s the real-world. But I have Three Wins top of mind that I can prioritize against. I can see if I’m trading up for higher-value, higher-priorities, or if I’m simply getting randomized and focusing on lower-value distractions.

Will I still have a laundry list of To-Do items? I will. But, at the top of that list, I’ll have Three Wins that are my “tests for success” for the day, that I can keep going back to, and that will help me prioritize my list of actions, reminders, and To-Dos.

20-Minute Sprints

I’ll use 20-Minute Sprints to achieve most of my results. It will help me make meaningful progress on things, keep a fast pace, stay engaged with what I’m working on, and to use my best energy.

Whether it’s an ultradian rhythms, or just a natural breaking point, 20-Minute Sprints help with focus.

We aren’t very good at focusing if we need to focus “until we are done.” But we are a lot better at focusing if we have a finish line in site. Plus, with what I’m learning about vision, I wonder if spending more than 20-Minutes is where we start to fatigue our eye muscles, and don’t even know it.

Note that I primarily talk about 20-Minute Sprints as timeboxing, after all, that’s what it is, but I think it’s more helpful to use a specific number. I remember that 40-Hour Work Week was a good practice from Extreme Programming before it became Sustainable Pace. Once it became Sustainable Pace, then teams started doing the 70 or 80 hour work week, which is not only ineffective, it does more harm than good.

Net net – start with 20-Minute Sprints. If you find another timebox works better for you, than by all means use it, but there does seem to be something special about 20-Minute Sprints for paving your work through work.

If you’re wondering, what if you can’t complete your task in a 20-Minute Sprint? You do another sprint.

All the 20-Minute Sprint does is give you a simple timebox to focus and prioritize your time, attention, and energy, as well as to remind you to take brain breaks. And, the 20-Minute deadline also helps you sustain a faster pace (more like a “sprint” vs. a “job” or “walk”).

Just Start

I could say so much more, but I’d rather you just start doing Agile Results.

Go ahead and take a moment to think about your Three Wins for today, and go ahead and write them down.

Teach a friend, family member, or colleague Agile Results.  Spread the word.

Help more people bring out their best, even in their toughest situations.

A little clarity creates a lot of courage, and that goes a long when it comes to making big impact.

You Might Also Like

10 Big Ideas from Getting Results the Agile Way

10 Personal Productivity Tools from Agile Results

What Life is Like with Agile Results

Categories: Architecture, Programming

Productivity Power Magazine

image

"Productivity is being able to do things that you were never able to do before." -- Franz Kafka

One of my experiments over the weekend was to do a fast roundup of my productivity articles.

Here it is -- Productivity Power Magazine:

Productivity Power Magazine

I wanted to create a profound knowledge base of principles, patterns, and practices for productivity.  I also wanted to make it fast, really fast, to be able to go through all of my productivity articles that I’ve created for Sources of Insight and on MSDN. 

I also wanted it to be more visual, I wanted thumbnails of each articles, so that I could flip through very quickly.

After looking at a few options, I tried Flipboard.  It’s a simple way to create personal magazines, and world class publications like The New York Times, PEOPLE Magazine, Fast Company and Vanity Fair use Flipboard.

Productivity Power Magazine (A Flipboard Experiment)

Here is my first Flipboard experiment to create Productivity Power Magazine:

Productivity Power Magazine

I think you’ll find Productivity Power Magazine a very fast way to go through all of my productivity articles.  You get to see everything and a glance, scroll through a visual list, and then dive into the ones you want to read.  If you care about productivity, this might be your productivity paradise.

Note that I take a “whole person” approach to productivity, with a focus on well-being.  I draw from positive psychology, sports psychology, project management practices, and a wide variety of sources to help you achieve high-performance.  Ultimately, it’s a patterns and practices approach to productivity to help you think, feel, and do your best, while enjoying the journey.

Some Challenges with Productivity Power Magazine

Flipboard is a fast way to roundup and share articles for a theme.

I do like Flipboard.  I did run into some issues though while creating my Productivity Power Magazine: 1)  I wasn’t able to figure out how to create a simpler URL for the landing page, 2)  I wasn’t able to swap out images if I didn’t like what was in the original article 3) I couldn’t add an image if the article was missing one, 4) I couldn’t easily re-sequence the flow of articles in the magazine, and 5) I can’t get my editorial comments to appear.  It seems like all of my write ups are in the tool, but don’t show on the page.

That said, I don’t know a faster, simpler, better way to create a catalog of all of my productivity articles at a glance.  What’s nice is that I can go across multiple sources, so it’s a powerful way to round up articles and package them for a specific theme, such as productivity in this case.

I can also see how I can use Flilpboard for doing research on the Web, alone or with a team of people, since you can invite people to contribute to your Flipboard.   You can also make Flipboards private, so you can choose which ones you share.

Take Productivity Power Magazine for a spin and let me know how it goes.

Categories: Architecture, Programming

Use Google For Throughput, Amazon And Azure For Low Latency

Which cloud should you use? It may depend on what you need to do with it. What Zach Bjornson needs to do is process large amounts scientific data as fast as possible, which means reading data into memory as fast as possible. So, he made benchmark using Google's new multi-cloud PerfKitBenchmarker, to figure out which cloud was best for the job.

The results are in a very detailed article: AWS S3 vs Google Cloud vs Azure: Cloud Storage Performance. Feel free to datamine the results for more insights, but overall his conclusions are:

Categories: Architecture

DevNexus 2016 in Atlanta, GA

Coding the Architecture - Simon Brown - Mon, 01/18/2016 - 13:33

I'm pleased to say I'll be in the United States next month for the DevNexus 2016 conference that is taking place in Atlanta, GA. In addition to a number of talks about software architecture, I'll also be running my popular "The Art of Visualising Software Architecture" workshop. Related to the (free) book with the same name, this hands-on workshop is about improving communication and specifically software architecture diagrams. We'll talk about UML and some anti-patterns of "boxes and lines" diagrams, but the real focus is on my "C4 model" and some lightweight techniques for communicating software architecture. The agenda and slides for this 1-day workshop are available online. I hope you'll be able to join me.

Categories: Architecture

Stuff The Internet Says On Scalability For January 15th, 2016

Hey, it's HighScalability time:


Space walk from 2001: A Space Odyssey? Nope. A base jump from the CN Tower in Toronto.

 

If you like this Stuff then please consider supporting me on Patreon.
  • 13.5TB: open data from Yahoo for machine learning; 1+ exabytes: data stored in the cloud; 13: reasons autonomous cars should have steering wheels; 3,000: kilowatt-hours of energy generated by the solar bike path; 10TB: helium-filled hard disk; $224 Billion: 2016 gadget spending in US; 85: free ebooks; 17%: Azure price drop on some VMs; 20.5: tons of explosives detonated on Mythbusters; 20 Billion: Apple’s App Store Sales; 70%: Global Internet traffic goes through Northern Virginia; 12: photos showing the beauty of symmetry; 

  • Quotable Quote:
    • @WhatTheFFacts: Scaling Earth's 'life' to 46 years, the industrial revolution began 1 minute ago -- In that time we've destroyed half the world's forests.
    • David Brin: The apotheosis of Darth Vader was truly disgusting. Saving one demigod—a good demigod, his son—wiped away all his guilt from slaughtering billions of normal people.
    • Brian Brazil: In today’s world, having a 1:1 coupling between machines and services is becoming less common. We no longer have the webserver machine, we have one machine which hosts one part of the webserver service. 
    • @iamxavier: "Snapchat is said to have 7 billion mobile video views vs Facebook’s 8 bil.The kicker: Fb has 15x Snapchat’s users."
    • Charlie Stross: Do you want to know the real reason George R. R. Martin's next book is late? it's because keeping track of that much complexity and so many characters and situations is hard work, and he's not getting any younger. 
    • @raju: Unicorn-Size Losses: @Uber lost $671.4 million in 2014 & $987.2 million in the first half of 2015
    • @ValaAfshar: 3.8 trillion photos were taken in all of human history until mid-2011. 1 trillion photos were taken in 2015 alone
    • @ascendantlogic: 2010: Rewrite all the ruby apps with javascript 2012: Rewrite all the javascript apps with Go 2014: Rewrite all the Go apps with Rust
    • @kylebrussell: “Virtual reality was tried in the 90s!” Yeah, with screens that had 7.9% of the Oculus Rift CV1 resolution
    • @kevinmarks: #socosy2016 @BobMankoff: people don't like novelty - they like a little novelty in a cocoon of familiarity, that they could have thought of
    • @toddhoffious: The problem nature has solved is efficient variable length headers. Silicon doesn't like them for networks, or messaging protocols. DNA FTW.
    • @jaykreps: I'm loving the price war between cloud providers, cheap compute enables pretty much everything else in technology. 
    • The Confidence Game: Transition is the confidence game’s great ally, because transition breeds uncertainty. There’s nothing a con artist likes better than exploiting the sense of unease we feel when it appears that the world as we know it is about to change.
    • @somic: will 2016 be the year of customer-defined allocation strategies for aws spot fleet? (for example, through a call to aws lambda)
    • beachstartup: i run an infrastructure startup. the rule of thumb is once you hit $20-99k/month, you can cut your AWS bill in half somewhere else. sites in this phase generally only use about 20% of the features of aws.
    • @fart: the most important part of DevOps to me is “kissing the data elf”
    • @destroytoday: In comparison, @ProductHunt drove 1/4 the traffic of Hacker News, but brought in 700+ new users compared to only 20 from HN.
    • @aphyr~ Man, if people knew even a *tenth* of the f*cked up shit tech company execs have tried to pull... Folks are *awfully* polite on twitter.
    • @eric_analytics: It took Uber five years to get to a billion rides, and its Chinese rival just did it in one
    • lowpro: Being a 19 year old college student with many friends in high school, I can say snapchat is the most popular social network, followed by Instagram then Twitter, and lastly Facebook. If something is happening, people will snap and tweet about it, Instagram and Facebook are reserved for bigger events that are worth mentioning, snapchat and Twitter are for more day to day activities and therefore get used much more often.
    • Thaddeus Metz: The good, the true, and the beautiful give meaning to life when we transcend our animal nature by using our rational nature to realize states of affairs that would be appreciated from a universal perspective.
    • Reed Hastings: We realized we learned best by getting in the market and then learning, even if we’re less than perfect. Brazil is the best example. We started [there] four years ago. At first it was very slow growth, but because we were in the market talking to our members who had issues with the service, we could get those things fixed, and we learned faster.

  • Why has Bitcoin failed? From Mike Hearn: it has failed because the community has failed. What was meant to be a new, decentralised form of money that lacked “systemically important institutions” and “too big to fail” has become something even worse: a system completely controlled by just a handful of people. Worse still, the network is on the brink of technical collapse. The mechanisms that should have prevented this outcome have broken down, and as a result there’s no longer much reason to think Bitcoin can actually be better than the existing financial system.

  • Lessons learned on the path to production. From Docker CEO: 1) IaaS is too low; 2) PaaS is too high: Devs do not adopt locked down platforms; 3) End to end matters: Devs care about deployment, ops cares about app lifecycle and origin; 4) Build management, orchestration, & more in a way that enables portability; 5) Build for resilience, not zero defects; 6) If you do 5 right, agility + control

  • Is this the Tesla of database systems? No Compromises: Distributed Transactions with Consistency, Availability, and Performance: FaRMville transactions are processed by FaRM – the Fast Remote Memory system that we first looked at last year. A 90 machine FaRM cluster achieved 4.5 million TPC-C ‘new order’ transactions per second with a 99th percentile latency of 1.9ms. If you’re prepared to run at ‘only’ 4M tps, you can cut that latency in half. Oh, and it can recover from failure in about 60ms. 

  • Uber tells the story behind the design and implementation of their scalable datastore using MySQL. Uber took that path of many others in writing an entire layer on top of MySQL to create the database that best fits their use case. Uber wanted: to be able to linearly add capacity by adding more servers; write availability; a way of notifying downstream dependencies; secondary indexes; operation trust in the system, as it contains mission-critical trip data. They looked at Cassandra, Riak, and MongoDB, etc. Features alone did not decide their choice. What did?: "the decision ultimately came down to operational trust in the system we’d use."  If you are Uber this is a good reason that may not seem as important to those without accountability. Uber's design is inspired by Friendfeed, and the focus on the operational side inspired by Pinterest.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Judo Strategy

Xebia Blog - Thu, 01/14/2016 - 23:00

In the age of disruption incumbents with decades of history get swept away by startups at an alarming rate. What allows these fledgling companies to succeed? What is their strategy and how can you defend against such an opponent?

I found that that the similarities between Judo and Business strategy allow us to apply the Judo principles to become a Product Samurai.

There are people who learn by reading (carry on doing so), people who learn by listening and observing (see the video registration) and people who learn by experiencing (sign up here). With that out of our way, let’s get started.

Enter the dojo

The dojo is a place where Judoka’s come to practice. One of the key things that make Judo great is that it can be learned, and it can be taught. To illustrate just how powerful it is and what the three core principles are I asked my 9-year-old daughter to give a demonstration.

The three principles of Judo Strategy

Jigorō Kanō founded Judo after having studied many different styles. Upon reaching a deeper level of understanding of what works and what doesn’t he created his philosophy based on the rule of “maximum effect with minimal effort”. What contributed to this rule became part of Judo, what did not contribute was discarded.

The operation of disruptors is not much different. With no past to hang on to, startups are free to pivot and take technology, processes and tools from the best offerings and combine them to a new technique. If they do not operate on the principle of “maximum effect with minimal effort” they will die. This is a luxury that incumbents have but usually not leverage. Typically the incumbents have the capital, equipment, skills, market knowledge, channels and people, still choose not to leverage.

Movement

The first principle in Judo is movement. Before we get in close with our opponent we seek to grab him in a favorable position. Maybe we can grip a sleeve, or arm and catch the opponent off guard. As a disruptor I also seek for uncontested ground. A head on attack on the core product of an established player is usually met with great resistance, as a disruptor I cannot hope to win that battle.

Variables in the Lean Canvas

Figure: Variables in the Lean Canvas

So to seek uncontested ground, means launching under the radar. This will go against the advice of your marketing director who will tell you to make “as much noise as possible”. This will indeed attract more people, but also tell your opponent exactly what you are doing. So have your marketing align with your ability to execute. Why go multi-lingual when you can only serve local customers? Why do a nation wide campaign when you are still running a pilot in a single city? Why pinpoint the shortcomings in your competitor’s product when you cannot outpace them in development (yet)?

At some point contact is inevitable, but a good disruptor will have chosen a vantage point. By the time Dell became noticed they had a well-oiled distribution network in place and were able to scale quickly, whereas the competition still relied on brick and mortar partners. Digital media like nu.nl came out of nowhere and by the time traditional newspapers had noticed them it was not only hard to catch up, but their opponent had defined where the battle would take place: in the land of Apps, away from the strengths of traditional media. There is still a chance for incumbents, and that is to out innovate your opponent and to do so you have absorb some disruption DNA yourself.

Balance

The second principle is balance. Once we have gripped our opponent we are trying to destroy their balance whilst strengthening our own. Ironically this usually means keeping your enemies close.

Puppy ploy

Figure: Puppy ploy

Perhaps the best possible example of this is the “puppy ploy” that Apple pulled with the music industry in the early days of iTunes. By emphasizing that downloadable music was not really their business (it only represented 1 or 2%) and was highly unprofitable (due to illegal sources like Napster and Kazaa) did Apple obtain the position it still holds today, and became the dominant music platform. As history repeats itself, a small company from Finland did the same thing with streaming, rather than owning the music. If you close your eyes you can hear them pitch: “it’s only a few percent of your business, and it’s like radio, does radio effect your business?”.

A little bit closer to home, I’ve seen first hand how a major eCommerce platform brokered deals with numerous brands. Sure, there is a kickback involved, but it prevents your competitors from opening their own store. By now they have become the dominant player, and their partners have come to rely on their digital partner to run the shop for them. It’s a classical example of keeping your enemies so close, that they cannot leverage their strength.

Leverage 

My favourite category of throwing techniques (Nage Waza) are the Sutemi, or in English “sacrifice throws”. In these techniques you typically sacrifice your position so you can leverage the power of your opponent.

Basically it means: go after sunk cost. Observe your opponent and learn where he has invested. Virgin Air does not fly out of major airports, therewith circumventing the enormous investment that other airlines have made. Has your opponent invested in a warehouse to support building a one-day delivery service? Make delivery cost free! Is it a platform battle? Open source it and make money from running services on top of it.

Does it hurt? of course it does! This is why the first thing you learn in Judo is fall breaking (ukemi waza). The question is not if you will fall, but if you can get back up quickly enough. Now this is not a plea for polka style pivoting startup behavior. You still need a strategy and stick to your product vision, but be prepared to sacrifice in order to reach that.

I once ran a Customer Journey mapping workshop at Al Jazeera. Though we focused on Apps, the real question was: “what is the heart of the brand” How can we be a better news agency than ABC, BBC, CNN etc.? By creating better articles? by providing more in-depth news? Turned out we could send photographers where they could not. They had invested in different areas and by creating a photo driven news experience they would be hindered by sunk cost.

If you manage to take the battle to uncontested grounds and have destroyed your opponent’s balance, his strength will work against him. It took Coca Cola 15 years to respond to the larger Pepsi bottle due its investment in their iconic bottle. By the time they could and wanted to produce larger bottles, Pepsi had become the second largest brand. WhatsApp reached a 10 billion messages in 2 years; you don’t have Coca Cola’s luxury anymore.

Awesome machines that you need to let go of

Figure: Awesome machines that you need to let go of

Why did Internet only news agencies like nu.nl scored a dominant position in the mobile space? Because the incumbents were too reluctant to cannibalize their investments in dead tree technology.

Key take away 

Judo can be learned and so can these innovation practices. We have labeled the collection of these practices Continuous Innovation. Adopting these practices means adopting the DNA of a disruptor.

It’s a relentless search to find unmet market needs, operating under the radar until you find market-fit. You can apply typical Lean startup techniques like Wizard of Oz, landing pages or product bootcamp.

Following through fast means scalable architecture principles and an organization that can respond to change. As an incumbent, watch out for disruptors that destroy your balance; typically by running a nice of your business for you that will become strategic in the future.

Finally: if you are not prepared to sacrifice your current product for one that addresses the customers need better someone else will do it for you.

 

This blog is part of the Product Samurai series. Sign up here to stay informed of the upcoming book: The Product Manager's Guide to Continuous Innovation.

The Product Manager's guide to Continuous Innovation

 

Live Video Streaming At Facebook Scale

With 1.49 billion monthly active users, operating at Facebook scale is far from trivial. Facebook's new live video streaming services present a fascinating use case for designing streaming service in global distribution and massive scale.
Categories: Architecture