Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator/categories/7&page=2' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Architecture
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

Codifying the rules used to organise your code

Coding the Architecture - Simon Brown - Tue, 03/29/2016 - 09:23

Regular readers will already know about Structurizr - a set of open source libraries to create a software architecture model as code, plus a SaaS product to visualise those models. Having created and helped create a number of models with Structurizr now, I've noticed an interesting side-effect. In the absence of architectural information being present in the code, the power of using something like Structurizr to define a software architecture model using code is in extracting information algorithmically, by codifying the rules that you've ultimately used to structure your codebase.

Let me give you an example. Imagine you're building a web-MVC web application in Java, C#, etc and you have a tens or hundreds of controller classes, each of which uses a number of other components to implement some functionality. Drawing a single diagram to visualise the static structure of the entire web application is a bad idea because it shows too much information. A better approach is to create one view per vertical slice, where there could be one vertical slice per web controller. This results in smaller, simpler diagrams like this.

A component diagram based upon a web controller

So far so good, and this is relatively easy to do using static analysis techniques. But you'll notice this diagram includes an "Authenticated User", which isn't part of the code itself. This raises the question of how the user ends up getting included on the diagram. There are a number of options:

  • Manually add the correct type of user on a case by case basis.
  • Match the controller's URL routing to the permitted user role (this is likely specified in configuration somewhere) and use this information to choose the appropriate type of user for each controller.
  • Add machine-readable metadata (e.g. Java Annotations, C# Attributes) to specify the user type (there are some Structurizr annotations I've created to help with this).
  • Codify the rules you've used to organise the controllers in your codebase.

The ability to codify the rules you've used to organise the controllers in your codebase obviously depends on how much thought you've put into doing this. For example, did you dump all of these controller classes into a single package or namespace without giving it much thought at all? Or perhaps you took Martin Fowler's advice and modularised further, creating one package/namespace per functional area or aggregate root, for example. Another possibility is that you grouped controllers together based upon whether unauthenticated users, authenticated users or other software systems are using them. Organising your code well provides you with another angle to extract architectural information, because you can codify rules such as, "the Anonymous User uses all controllers in the com.mycompany.mywebapp.unsecured package/namespace".

With hindsight this is fairly obvious, but we often don't put enough thought into how we organise our code, possibly because we perceive that it doesn't actually matter that much and modern IDEs provide powerful ways to navigate large and/or complex codebases. Trying to codify the rules used to organise a codebase certainly gets you thinking, and often refactoring too.

Categories: Architecture

How we implemented the video player in Mail.Ru Cloud

We’ve recently added video streaming service to Mail.Ru Cloud. Development started with contemplating the new feature as an all-purpose “Swiss Army knife” that would both play files of any format and work on any device with the Cloud available. Video content uploaded to the Cloud mostly falls into one of the two categories: “movies/series” and “users’ videos”. The latter are the videos that users shoot with their phones and cameras, and these videos are most versatile in terms of formats and codecs. For many reasons, it is often a problem to watch these videos on other end-user devices without prior normalization: a required codec is missing, or the file size is too big to download, or whatever.

In this article, I’ll go into detail to explain how video playback works in Mail.Ru Cloud, and how we made the Cloud player “omnivorous” and ensured support on a maximum number of end-user devices.

Storing and Caching: two approaches
Categories: Architecture

Personal Empowerment All-Stars: Jack Canfield, Ken Blanchard, and Stephen Covey at Microsoft

“You can, you should, and if you’re brave enough to start, you will.”  — Stephen King

One of the best things at Microsoft is the chance to meet extraordinary people.

Jack Canfield, Ken Blanchard, and Stephen Covey are a few that top my list.

They are personal empowerment all-stars.

As I was re-writing my posts on lessons learned from Jack Canfield, Ken Blanchard, and Stephen Covey, I noticed what they share in common.

What do Jack Canfield, Ken Blanchard and Stephen Covey have in common?

Their work has a heavy emphasis on personal-empowerment, positivity, and people.

I thought it would be interesting to write a narrative about lessons learned from each, to supplement my bullet point write ups.

Here we go …

Jack Canfield at Microsoft

Jack Canfield is all about taking full responsibility for everything that happens in your life.  And he starts with self-talk.  He says it’s not what people say or do, it’s what you say to yourself.  For example, it’s not what Jack says to Laura, it’s what Laura says to Laura.

From a personal empowerment standpoint, Jack reminds us that we have control over three responses: 1) what we say or do, 2) our thoughts, 3) the images in our head.  Jack is a big believer in the power of visualization and he reminds us that’s how athletes perform at greater levels — they see things in their minds, to guide what they can do with their bodies.

Jack shares a very simple formula for success.  Jack’s success formula is Event + Response = Outcome.  If you want to change the outcome, then change your response.  It sounds simple, but it’s empowering.

Jack Canfield also reminded us that we are the creative force in our life and to get out of victimism:

“You are not the victim of your circumstances–You are the creative force of your life.”

Grow your circle of influence and make tremendous impact.

Read more at Lessons Learned from Jack Canfield.

Ken Blanchard at Microsoft

Ken Blanchard is really about accentuating the positive.  So much of the world focuses on what’s wrong, but he wants to focus on what’s right, so we can do more of that.

Ken has an incremental model of leadership that starts with you and expands from there: you, your team, your organization.  The idea is that you can’t lead others effectively, if you can’t even lead yourself.

Ken’s model for leadership is really an adaptive model, that’s focused on the greater good, and it starts by helping everybody get an “A.”  Leaders that apply one style to all team members, aren’t very effective.  Ken suggests that leaders apply the right styles depending on what individuals need.  Ken’s 4 leadership styles are:

  1. Directive
  2. Coaching
  3. Supportive
  4. Delegating.

Perhaps, the most profound statement that Ken made is that “leadership is love.”  He said that leadership includes “loving your mission”, “loving your cusotmers”, “loving your people”, and “loving yourself — enough to get out of the way so others can be magnificent.”

Read more at Lessons Learned from Ken Blanchard.

Stephen Covey at Microsoft

Stephen Covey was really about personal effectiveness, realizing your potential, and leaving a legacy.

Covey really emphasized a whole-person approach: Body, Mind, Heart, Spirit.  His point was that if you take one of the four parts of your nature away, then you’re treating a person like a “thing” you control and manage.

Covey also emphasized the importance of a personal mission.  It gives meaning to your work and it helps you channel all of your efforts as you live and lead your legacy.  He also suggested writing your personal mission down and visualizing it to imprint it on your subconscious.

The other key to realizing your potential is finding your voice.  Use all of you, your best way, in your unique way, for your best results.  That’s how you differentiate and add value for yourself and others.

And, of course, Stephen Covey reminded us of the 7 Habits of Highly Effective People:

  1. Be proactive.
  2. Begin with the end in mind.
  3. Put first things first.
  4. Think win-win.
  5. Seek first to understand, then to be understood.
  6. Synergize.
  7. Sharpen the saw.

Habits 1,2,and 3 are the foundation for private victories and integrity.  Habits 4, 5, and 6 are the keys to public victories.

Read more at Lessons Learned from Stephen Covey.

All-in-all, I have to say that while individually each of these personal empowerment all-stars has great wisdom and insight for personal effectiveness, leadership, and success, they are actually “better together.”

Each day in the halls of Microsoft, I find myself reflecting on their one-liner reminders, whether it’s Covey’s “Put first things first,” or Canfield’s “You are the creative force of your life”, or Blanchard’s “None of us is as smart as all of us.”

You Might Also Like

How Tos for Personal Effectiveness at a Glance

Personal Effectiveness at Microsoft

Personal Effectiveness Toolbox

Categories: Architecture, Programming

Compile-Time Evaluation in Scala with macros

Xebia Blog - Sun, 03/27/2016 - 15:20
Many 'compiled' languages used to have a strict separation between what happens at 'compile-time' and what happens at 'run-time'. This distinction is starting to fade: JIT compilation moves more of the compile phase to run-time, while conversely various kinds of optimizations do 'run-time' work at compile time. Powerful type systems allow the expression things previously

Common Sense Agile Scaling (Part 1: Intro)

Xebia Blog - Sat, 03/26/2016 - 10:05
Agile is all about running experiments and see what works for you. Inspect and adapt. Grow. This also applies to scaling your agile organization. There is no out of the box scaling framework that will fit your organization perfectly and instantly. Experiment, and combine what you need from as many agile models and frameworks as

Stuff The Internet Says On Scalability For March 25th, 2016


Did you know there's a field called computational aesthetics? Neither did I. It's cool though.

 

If you like this sort of Stuff then please consider offering your support on Patreon.
  • 51%: of billion-dollar startups founded by immigrants; 2.8 billion: Twitter metric ingestion service writes per minute; 1 billion: Urban Airship push notifications a day; 1.5 billion: Slack messages sent per month; 35 million: server nodes in the world; 10: more regions will be added to Google Cloud;  697 million: WeChat active monthly users; 

  • Quotable Quotes:
    • Dark Territory: When officials in the Air Force or the NSA neglected to let Microsoft (or Cisco, Google, Intel, or any number of other firms) know about vulnerabilities in its software, when they left a hole unplugged so they could exploit the vulnerability in a Russian, Chinese, Iranian, or some other adversary’s computer system, they also left American citizens open to the same exploitations—whether by wayward intelligence agencies or by cyber criminals, foreign spies, or terrorists who happened to learn about the unplugged hole, too. 
    • @xaprb: If you adopt a microservices architecture with 1000x more things to monitor, you should not expect your monitoring cost to stay the same.
    • The Swrve Monetization Report 2016: almost half of all the revenue generated in mobile gaming comes from just 0.19 percent of users.
    • Nassim Taleb: Now some empiricism. Consider that almost all tech companies "in the tails" were not started by "funding". Take companies you are familiar with: Microsoft, Apple, Google, Facebook. These companies started with risk-taking. Funding came in small amounts, way later.
    • @leegomes: In a big shift, Google says a go-anywhere self-driving car might not be ready for 30 years.
    • Google’s Eric Schmidt: Machine learning will be basis of ‘every huge IPO’ in five years.
    • @brendangregg: "Memory bandwidth is the number one issue we see today" Denis at Facebook
    • @ogrisel: PostgreSQL 9.6 will support parallel aggregation! TPC-H Q1 @ 100GB benchmark shows linear scaling up to 30 workers 
    • @sarah_edo: The hardest part of being a developer isn't the code, it's learning that the entire internet is put together with peanut butter and goblins.
    • @beaucronin: "Cryptocurrencies are an emergent property of the Internet – almost a fifth protocol"
    • Thomas Frey: We are moving toward an era of megaprojects. We’ll finish the Pan-American Highway with a 25-mile bridge over the Darien Gap in Panama. 
    • @samphippen: “Do you expect me to talk?” “No Mr. Bond, I expect you to be willing to relocate to san francisco"
    • @brendanbaker: Outside of the core people, who actually know what they're doing, AI is talked about like gamification was three years ago.
    • @RichRogersHDS: Did you know? The collective noun for a group of programmers is a merge-conflict." - @omervk
    • @jbeda: This is how you know Google is serious about cloud. Real money on real facilities. 
    • Farhad Manjoo: The lesson so far in the on-demand world is that Uber is the exception, not the norm. Uber, but for Uber — and not much else.
    • @DKThomp: Airbnb woulda made a killing in 1900: One third of urban families used to make 10%+ of their income from "lodgers" 
    • @AstroKatie: "We can make 'smart drones'!" "Your chatbot became a Nazi in like a day." "OK good point."
    • @adrianco: I agree GCP are setup for next gen apps, think they are missing out on where most of the $ are being spent in the short term.
    • @EdwardTufte: Like book publishers and Silicon Valley, the further the distance from content production, the greater the money. 
    • Biz Carson: Slack grew from 80 to 385 employees in 14 months
    • Chip Overclock®: One of those things is being evidence-based. Don't guess. Test. Measure. Look and see. Ask. If you can avoid guessing, do so.

  • Impressive demo of the new smaller, less dorky looking Meta augmented reality headset. Here's a hands on report. The development kit is $949. This most likely will be the new app store level opportunity so it might be smart to get on it now. The Gold Rush phase is still in the future. The uses are obvious to anyone who reads Science Fiction. This is a TED talk, so of course no details on performance, etc. What are the backend infrastructure opportunities? Hopefully they'll keep all that open instead of building another walled garden.

  • Is artificial intelligence ready to rule the world? IMHO: No. You would need a large training set. The problem is we have so few good examples of ruling the world successfully. You could create an artificial world in VR with a simulated world to generate training data, but that's just another spin on in the long history of Utopian thinking. We should probably learn to govern ourselves first before we pitch it over to an AI.

  • "It's better to have a media strategy than a security strategy." That's Greg Ferro commenting in an episode of Network Break on Home Depot's paltry $19.5 million fine for their massive 2014 data breach. Why pay for security when there's no downside? It's not like people stopped shopping at Home Depot. 

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Refactoring to Microservices - Introducing a Process Manager

Xebia Blog - Fri, 03/25/2016 - 14:15
A while ago I described the first part of our journey to refactor a monolith to microservices (see here). While this was a useful first step, a lot can be improved. I was inspired by Greg Young's course at Skills Matter, see CQRS/DDD course. Because I think it’s useful to reflect on the steps you

What does Etsy's architecture look like today?

This is a guest post by Christophe Limpalair based on an interview (video) he did with Jon Cowie, Staff Operations Engineer and Breaksmith @ Etsy.

Etsy has been a fascinating platform to watch, and study, as they transitioned from a new platform to a stable and well-established e-commerce engine. That shift required a lot of cultural change, but the end result is striking.

In case you haven't seen it already, there's a post from 2012 that outlines their growth and shift. But what has happened since then? Are they still innovating? How are engineering decisions made, and how does this shape their engineering culture? These are questions we explored with Jon Cowie, a Staff Operations Engineer at Etsy, and the author of Customizing Chef, in a new podcast episode.

What does Etsy's architecture look like nowadays?
Categories: Architecture

To Compress or Not to Compress, that was Uber's Question

Uber faced a challenge. They store a lot of trip data. A trip is represented as a 20K blob of JSON. It doesn't sound like much, but at Uber's growth rate saving several KB per trip across hundreds of millions of trips per year would save a lot of space. Even Uber cares about being efficient with disk space, as long as performance doesn't suffer. 

This highlights a key difference between linear and hypergrowth. Growing linearly means the storage needs would remain manageable.  At hypergrowth Uber calculated when storing raw JSON, 32 TB of storage would last than than 3 years for 1 million trips, less than 1 year for 3 million trips, and less 4 months for 10 million trips.

Uber went about solving their problem in a very measured and methodical fashion: they tested the hell out of it. The goal of all their benchmarking was to find a solution that both yielded a small size and a short time to encode and decode.

The whole experience is described in loving detail in the article: How Uber Engineering Evaluated JSON Encoding and Compression Algorithms to Put the Squeeze on Trip Data. They came up with a matrix of 10 encoding protocols (Thrift, Protocol Buffers, Avro, MessagePack, etc) and 3 compression libaries (Snappy, zlib, Bzip2). The target environment was Python. Uber went to an IDL approach to define and verify their JSON protocol, so they ended up only considering IDL solutions. 

The conclusion: MessagePack with zlib.  Encoding time: 4231 ms. Decoding: 715 ms. There was a 78% reduction in size relative to the JSON zlib combination.

The result: 1 TB disk will now last almost a year (347 days), compared to a month (30 days) without compression. Uber now has enough space to last over 30 years compared to just under 1 year before. That's a huge win for a relatively simple change. Hopefully there's a common library handling all the messaging so this change could be completely transparent to all the developers. Uber also noted that smaller packet sizes mean less data transiting through the system which means less processing time which means less hardware is needed. Another big win.

Something to consider: don't use JSON for messaging. The compression/decompression times are still dog slow. If you are going to use an IDL, which every grown up project eventually moves to for reliability and security reasons, consider not using JSON for messaging. Go for a binary protocol from the start. The performance savings can be dramatic. A lot of the performance drain comes from serialization/deserialization churning through memory and that can be reduced greatly by not using text based protocols like JSON. JSON is convenient, but it's also hugely wasteful at scale. 

Categories: Architecture

To Compress or Not to Compress, that was Uber's Question

Uber faced a challenge. They store a lot of trip data. A trip is represented as a 20K blob of JSON. It doesn't sound like much, but at Uber's growth rate saving several KB per trip across hundreds of millions of trips per year would save a lot of space. Even Uber cares about being efficient with disk space, as long as performance doesn't suffer. 

This highlights a key difference between linear and hypergrowth. Growing linearly means the storage needs would remain manageable.  At hypergrowth Uber calculated when storing raw JSON, 32 TB of storage would last than than 3 years for 1 million trips, less than 1 year for 3 million trips, and less 4 months for 10 million trips.

Uber went about solving their problem in a very measured and methodical fashion: they tested the hell out of it. The goal of all their benchmarking was to find a solution that both yielded a small size and a short time to encode and decode.

The whole experience is described in loving detail in the article: How Uber Engineering Evaluated JSON Encoding and Compression Algorithms to Put the Squeeze on Trip Data. They came up with a matrix of 10 encoding protocols (Thrift, Protocol Buffers, Avro, MessagePack, etc) and 3 compression libaries (Snappy, zlib, Bzip2). The target environment was Python. Uber went to an IDL approach to define and verify their JSON protocol, so they ended up only considering IDL solutions. 

The conclusion: MessagePack with zlib.  Encoding time: 4231 ms. Decoding: 715 ms. There was a 78% reduction in size relative to the JSON zlib combination.

The result: 1 TB disk will now last almost a year (347 days), compared to a month (30 days) without compression. Uber now has enough space to last over 30 years compared to just under 1 year before. That's a huge win for a relatively simple change. Hopefully there's a common library handling all the messaging so this change could be completely transparent to all the developers. Uber also noted that smaller packet sizes mean less data transiting through the system which means less processing time which means less hardware is needed. Another big win.

Something to consider: don't use JSON for messaging. The compression/decompression times are still dog slow. If you are going to use an IDL, which every grown up project eventually moves to for reliability and security reasons, consider not using JSON for messaging. Go for a binary protocol from the start. The performance savings can be dramatic. A lot of the performance drain comes from serialization/deserialization churning through memory and that can be reduced greatly by not using text based protocols like JSON. JSON is convenient, but it's also hugely wasteful at scale. 

Categories: Architecture

Stuff The Internet Says On Scalability For March 18th, 2016


We come in peace. 5,000 years of battles mapped from Wikipedia. Maybe not.

 

If you like this sort of Stuff then please consider offering your support on Patreon.

 

  • 500 petabytes: data stored in Dropbox; 8.5 kB: amount of drum memory in an IBM 650; JavaScript: most popular programming language in the world (OMG); $20+ billion: Twitch in 2020; Two years: time it took to fill the Mediterranean; 

  • Quotable Quotes:
    • Dark Territory: The other bit of luck was that the Serbs had recently given their phone system a software upgrade. The Swiss company that sold them the software gave U.S. intelligence the security codes.
    • Alec Ross~ The principle political binary of the 20th century is left versus right. In the 21st century the principle political binary is open versus closed. The real tension both inside and outside countries are those that embrace more open economic, political and cultural systems versus those that are more closed. Looking forward to the next 20 years the states and societies that are more open are those that will compete and succeed more effectively in tomorrows industry.
    • @chrismaddern"Population size: 1. Facebook 2. China

Now Available: The Relaunch of Sources of Insight

image“Do your own thing on your own terms and get what you came here for.” — Oliver James

Long ago, I had spun off a focus on Personal Effectiveness to Sources of Insight.

I mentor a lot of teams and leaders for high-performance, and I needed a place to consolidate and share principles, patterns, and practices for personal effectiveness. In fact, the tag-line is:

Proven Practices for Personal Effectiveness

I’ve significantly revamped the user experience, the content, and the collections of resources on Sources of Insight to reflect the latest feedback that users have shared with me (thank you all.)

Sources of Insight is for people with a passion for more from work and life. The goal of Sources of Insight is to be your source of insight, inspiration, and impact to help you achieve more in work and life. Whether you are an achiever or a high-performer, or simply want to work on your personal effectiveness, Sources of Insight will help you accelerate your results.
Better Insights, Better Results

One of the slogans I use is “Better Insights, Better Results” because the big idea is to “empower people with skill for work and life.”

Sources of Insight is now a success library of more than 1,300 articles with a focus on helping you “create a smarter, more creative, more capable you.”

You can think of this as applying patterns & practices to work and life, as well as making “Agile for Life” real.

The idea here is to give you the tools and techniques that will help you rise above the noise, get a better vantage point, and make better decisions to change any situation you find yourself in, or to create better situations from the start.

Stand on the Shoulders of Giants

As part of building out Sources of Insight, I draw from great books, great people, and great quotes, to help you “Stand on the shoulders of giants.”

As my friend puts it, what I really do is help you “Run with the titans.”

I’m a big believer in finding the “best of the best” from various disciplines and experts around the world and synthesizing into actionable guidance.

I periodically have featured guests on Sources of Insight.  I try to find the people that are the best in the world at what they do, or that have some interesting insight that will help you think differently or make more impact.

Many of my guest posts are by best-selling authors, but I also include comedians, and others.  

Some of my guests include Al Ries – the father of brand positioning in the mind, Guy Kawasaki – who is all about empowering people, Gretchen Rubin – author of The Happiness Project, Jairek Robbins – author of Live It (and Tony Robbins’s son),  Jim Kouzes – author of The Leadership Challenge, Marie Forleo – some say she is a female Tony Robbins, Michael Michalko – author of Thinker Toys, and a former Disney imagineer, Rick Kirschner – author of Dealing with People You Can’t Stand, Tim Ferris – author of The 4-Hour Work Week, and many more.  (You can see featured guests at a glance on the Feature Guests page on Sources of Insight.)
Realize Your Potential

What makes the articles a bit different on Sources of Insight is they usually reflect problems that I am helping people with, so they are real-world scenarios and solutions.

In this way, Sources of Insight is more than a clearinghouse of the world’s best insight and action for work and life – it’s also a self-paced, virtual mentor where you can learn how to be YOUR best.

Some readers say that the key thing for them is that Sources of Insight helps you “realize your potential.”

As Ralph Waldo Emerson would say, “Make the most of yourself….for that is all there is of you.”

And I would add, nobody else is going to do it for you Winking smile
Popular Topics for Personal Effectiveness

On Sources of Insight, I cover key topics for personal effectiveness. The most popular topics are:

Personal Effectiveness

Emotional Intelligence

Leadership

Life Hacks

Motivation

Productivity

Personal Development

You can browse more topics including Confidence, Conflict, Influence, Intellectual Horsepower, Interpersonal Skills, Strengths, Time Management, and more on the Topics page at Sources of Insight.

Great Books, Great People, Great Quotes

I also have collections of Great Books, Great People, Great Quotes:

Great Books – These are hand-crafted indexes of interesting and insightful books that you can use to improve all aspects of your work and life.  I spend a lot of money on books every single month and I read a lot of books each and every week.  In fact, many of my blog posts are what I call “Book Nuggets” which are like the best needles in the haystack.  My Great Books collection reflects a heavy investment in my quest to find the best wisdom of the world that is spread across hundreds, and thousands of books. Some of my more popular collections of books include Business Books, Career Books, Leadership Books, Personal Development Books, and Productivity Books.

Great People – This is where I shared and scale my best lessons learned and key insights from all walks of life.  What I do is I try to compact and distill the best insights from various people into lessons learned.  You can think of it as “Greatness Distilled.”  Some people are famous and others are unsung heroes.  What I focus on is the interesting insights that you can use to get better at work and life.  Here are a few of my more popular great people pages:  Bruce LeeChalene Johnson, Oprah Winfrey, Stephen Covey, Tony Robbins, and Steve Jobs.

Great Quotes – This is my attempt to organize collections of the world’s best wisdom of the ages and modern sages at your fingertips.  The right words can spark the right ideas, or the right thinking or the right feeling or taking the right action.  The right words help us live better, and they help us do better, and they help us be better.  The right words help us build better vocabularies and better mental models and better ways of doing and being, living, and even breathing.  Here are some of the more popular quotes collections: Focus Quotes, Happiness Quotes, Inspirational Quotes, Leadership Quotes, Motivation Quotes, Personal Development Quotes, and Productivity Quotes.
Articles

As I mentioned early, Sources of Insight is a Success Library of more than 1,3000+ Articles for Personal Greatness.  You can start at the Articles page, 

The Articles page has a simple set of some of the best articles.  From there, you can also explore the Archives.  You can also browse Topics from there.

Some of the most popular articles include:  7 Habits of Happiness, 25 Inspirational Movies, 50 Life Hacks for Your Future Self, 101 of the Greatest Insights and Actions for Work and Life, 101 Questions that Empower You, How To Get Whatever You Want, How To Think Like Bill Gates, Inspirational Quotes, Lessons Learned from Bruce Lee, The Exponential Results Formula, You 2.0.

Books

On Sources of Insight, the Books page is where I share free eBooks as well as feature the books I author.  At this point, my main book featured is Getting Results the Agile Way, which is a personal results system for work and life. 

Getting Results the Agile Way is where I introduce my simple productivity system: Agile Results.

Agile Results is really a simple system for meaningful results.  It helps you create more moments that matter.  It also helps you work on the right things, at the right time, the right way, with the right energy, to amplify your influence and impact.

Most importantly, it gets you spend more time in your strengths, less time in your weaknesses, and it helps you give your best where you have your best to give.

In terms of free eBooks, on the Books page you can find the following free books:  You 2.0, 30 Days of Getting Results, Getting Started with Agile Results, and The Zen of Results.

Courses

On Sources of Insight, the Courses page is where I share training to help you realize your potential and bring out your best.

My favorite way to provide training is through what I call “Monthly Improvement Sprints” or “30 Day Improvement Sprints” or just “30 Day Sprints.”

They are effectively 30 Day Challenges where you practice a little each day, to get better over time.

I find 30 Day Challenges or 30 Day Sprints are a great way to build better habits, learn new things, and improve your skills and abilities at whatever you focus on.

On the Courses page, you’ll find 30 Days of Getting Results, which was my attempt to share the absolute best principles, patterns, and practices for personal productivity.

Best of all, it’s free.  It could well be the best self-paced training you ever take for high-performance and for mastering productivity, time management, and work-life balance.

Plus, it’s a powerful way to learn Agile Results in a simple way, with one mini-lesson each day, that includes an exercise to put it into practice.

Resources

On Sources of Insight, the Resources page is effectively a library of helpful resources at your fingertips.  Here are a few of the key resources:

Book Review – My Book Reviews are like mini movie trailers of books, where I include key highlights from the book as well as my key take aways.  I don’t really do book reviews, where I talk about pros and cons.  Instead, I look for the most interesting or the most insightful parts of the book and focus on those.  I always ask the question, “How can I use this?” and I apply those “Book Nuggets” and those key take aways to real world scenarios.

Cheat Sheets – Cheat Sheets put key information at your fingertips.  The only Cheat Sheet I have so far is a Blogging Resources Cheat Sheet.   It’s actually a very powerful Cheat Sheet though, if you happen to be a blogger.  I get asked a lot about blogging, everything from how to get started to how to create a successful blog.  People ask me what the connection of blogging is to Personal Effectiveness, and to me it’s simple:  Working on your blog, is working on your life.  By building a blog, you build a personal platform for learning and growth.   Blogging is still one of the most effective ways I know to focus on personal development, while giving your best where you have your best to give, and sharing your unique expertise with the world.  I plan to add some very special Cheat Sheets of hard-core knowledge, so this page is more of a placeholder for now.

Checklists – Checklists are a quick way to provides lists of “one-liner reminders.”  In general, I try to focus on creating actionable checklists that inspire and trigger the right thinking or the right actions.  Currently, I provide a Focus Checklist, Leadership Checklist, Time Management Checklist, and The Charge Checklist, which is a checklist I created based on the best-selling book, The Charge.

How Tos – How Tos are a great way to turn insight into action.  My most popular How  Tos include How To Achieve Any Goal, How To Avoid Breaking Under Pressure, How To Change Any Habit, How To Find Your Strengths, and How To Find Your Values.

Product Recommendations – This is my roundup of the best products I’ve used for personal development and improving personal effectiveness.  The big deal here is The Greatest Personal Development Gifts Ever.  Not only are these the personal development programs that have served me well, but they are the gifts that I give to friends and family to give them an edge in work and life.

Trends – This is where I share key trends each year.  If you’ve ever read one of my trends posts, you know that they are deep, and they help give you a big advantage when it comes to seeing the road ahead.  One of the most important personal effectiveness skills that you can build is anticipation.  The way to improve your anticipation is to learn how to identify, understand, and apply trends to create your future.  When you focus on trends, they also help you build your visionary leadership skills, and if there’s one thing this world needs more of, it’s visionary leadership.  Here is my Trends for 2016 post.  It is a really deep dive into what’s happening around the world, but it also provides you the balcony view at a glance.  Use this as your advantage to maneuver at work, to shape your business, and to shape yourself, with clarity, courage, and competence.  After all, 2016 is the year of the bold!

Personal Effectiveness Toolbox

I wonder if I saved the best for last?  The Personal Effectiveness Toolbox is, to this date, my greatest compilation of the greatest programs and tools to help you do more and achieve more in this lifetime.

On the Personal Effectiveness Toolbox page, I share all of the best tools that I have used over the years to exponentially improve my ability to get results and to amplify my impact.

These are some of the best programs that have helped me really understand influence and impact.

They have helped me create my own personal achievement systems.

They have helped me get over any limiting beliefs and master my mind.

They have helped me really understand emotional intelligence at a deeper level and learn real skills and techniques.

They have also been my greatest programs for personal development and improving personal effectiveness across mind, body, emotions, career, finance, relationships, and fun.

Change the World or Go Home

We have a little saying that we use in the halls at Microsoft:

Change the world or go home!

Every now and then, you can see a poster in the hall or on somebody’s wall of the Microsoft Blue Monster.

It was the work of Hugh MacLeod as you can recognize by his art – simplicity and elegance in action (and you can read the backstory at The Blue Monster.)

image

You have everything at your fingertips to be YOUR best and to realize your potential, the agile way.

Go ahead and change the world, your way.

Always remember to give your best, where you have YOUR best to give.

Thrive on.

Categories: Architecture, Programming

Jeff Dean on Large-Scale Deep Learning at Google


If you can’t understand what’s in information then it’s going to be very difficult to organize it.

 

This quote is from Jeff Dean, currently a Wizard, er, Fellow in Google’s Systems Infrastructure Group. It’s taken from his recent talk: Large-Scale Deep Learning for Intelligent Computer Systems.

Since AlphaGo vs Lee Se-dol, the modern version of John Henry’s fatal race against a steam hammer, has captivated the world, as has the generalized fear of an AI apocalypse, it seems like an excellent time to gloss Jeff’s talk. And if you think AlphaGo is good now, just wait until it reaches beta.

Jeff is referring, of course, to Google’s infamous motto: organize the world’s information and make it universally accessible and useful.

Historically we might associate ‘organizing’ with gathering, cleaning, storing, indexing, reporting, and searching data. All the stuff early Google mastered. With that mission accomplished Google has moved on to the next challenge.

Now organizing means understanding.

Some highlights from the talk for me:

  • Real neural networks are composed of hundreds of millions of parameters. The skill that Google has is in how to build and rapidly train these huge models on large interesting datasets, apply them to real problems, and then quickly deploy the models into production across a wide variery of different platforms (phones, sensors, clouds, etc.).

  • The reason neural networks didn’t take off in the 90s was a lack of computational power and a lack of large interesting data sets. You can see how Google’s natural love of algorithms combined with their vast infrastructure and ever enlarging datasets created a perfect storm for AI at Google.

  • A critical difference between Google and other companies is that when they started the Google Brain project in 2011, they didn’t keep their research in the ivory tower of a separate research arm of the company. The project team worked closely with other teams like Android, Gmail, and photos to actually improve those properties and solve hard problems. That’s rare and a good lesson for every company. Apply research by working with your people.

  • This idea is powerful: They’ve learned they can take a whole bunch of subsystems, some of which may be machine learned, and replace it with a much more general end-to-end machine learning piece. Often when you have lots of complicated subsystems there’s usually a lot of complicated code to stitch them all together. It’s nice if you can replace all that with data and very simple algorithms.

  • Machine learning will only get better, faster. A paraphrased quote from Jeff: The machine learning community moves really really fast. People publish a paper and within a week lots of research groups throughout the world have downloaded the paper, have read it, dissected it, understood it, implemented some extensions to it, and published their own extensions to it on arXiv.org. It’s different than a lot other parts of computer science where people would submit a paper, and six months later a conference would decide to accept it or not, and then it would come out in the conference proceeding three months later. By then it’s a year. Getting that time down from a year to a week is amazing.

  • Techniques can be combined in magical ways. The Translate Team wrote an app using computer vision that recognizes text in a viewfinder. It translates the text and then superimposes the translated text on the image itself. Another example is writing image captions. It combines image recognition with the Sequence-to-Sequence neural network. You can only imagine how all these modular components will be strung together in the future.

  • Models with impressive functionality are small enough run on Smartphones. For technology to disappear intelligence must move to the edge. It can’t be dependent on network umbilical cord connected to a remote cloud brain. Since TensorFlow models can run on a phone, that might just be possible.

  • If you’re not considering how to use deep neural nets to solve your data understanding problems, you almost certainly should be. This line is taken directly from the talk, but it’s truth is abundantly clear after you watch hard problem after hard problem made tractable using deep neural nets.

Jeff always gives great talks and this one is no exception. It’s straightforward, interesting, in-depth, and relatively easy to understand. If you are trying to get a handle on Deep Learning or just want to see what Google is up to, then it's a must see.

There’s not a lot of fluff in the talk. It’s packed. So I’m not sure how much value add this article will give you. So if you want to just watch the video I’ll understand.

As often happens with Google talks there’s this feeling you get that we’ve only been invited into the lobby of Willy Wonka’s Chocolate factory. In front of us is a locked door and we're not invited in. What’s beyond that door must be full of wonders. But even Willy Wonka’s lobby is interesting.

So let’s learn what Jeff has to say about the future…it’s fascinating...

What is Meant by Understanding?
Categories: Architecture

Sponsored Post: zanox Group, Varnish, LaunchDarkly, Swrve, Netflix, Aerospike, TrueSight Pulse, Redis Labs, InMemory.Net, VividCortex, MemSQL, Scalyr, AiScaler, AppDynamics, ManageEngine, Site24x7

Who's Hiring?
  • The zanox Group are looking for a Senior Architect. We're looking for someone smart and pragmatic to help our engineering teams build fast, scalable and reliable solutions for our industry leading affiliate marketing platform. The role will involve a healthy mixture of strategic thinking and hands-on work - there are no ivory towers here! Our stack is diverse and interesting. You can apply for the role in either London or Berlin.

  • Swrve -- In November we closed a $30m funding round, and we’re now expanding our engineering team based in Dublin (Ireland). Our mobile marketing platform is powered by 8bn+ events a day, processed in real time. We’re hiring intermediate and senior backend software developers to join the existing team of thirty engineers. Sound like fun? Come join us.

  • Senior Service Reliability Engineer (SRE): Drive improvements to help reduce both time-to-detect and time-to-resolve while concurrently improving availability through service team engagement.  Ability to analyze and triage production issues on a web-scale system a plus. Find details on the position here: https://jobs.netflix.com/jobs/434

  • Manager - Performance Engineering: Lead the world-class performance team in charge of both optimizing the Netflix cloud stack and developing the performance observability capabilities which 3rd party vendors fail to provide.  Expert on both systems and web-scale application stack performance optimization. Find details on the position here https://jobs.netflix.com/jobs/860482

  • Software Engineer (DevOps). You are one of those rare engineers who loves to tinker with distributed systems at high scale. You know how to build these from scratch, and how to take a system that has reached a scalability limit and break through that barrier to new heights. You are a hands on doer, a code doctor, who loves to get something done the right way. You love designing clean APIs, data models, code structures and system architectures, but retain the humility to learn from others who see things differently. Apply to AppDynamics

  • Software Engineer (C++). You will be responsible for building everything from proof-of-concepts and usability prototypes to deployment- quality code. You should have at least 1+ years of experience developing C++ libraries and APIs, and be comfortable with daily code submissions, delivering projects in short time frames, multi-tasking, handling interrupts, and collaborating with team members. Apply to AppDynamics
Fun and Informative Events
  • Varnish Summits are a worldwide event series where Varnish customers, partners, open source users and other enthusiasts come together to network and learn.  At the summits Varnish Software's experts and core developers do a deep dive into technical best practices and offer workshops for both new and advanced Varnish users.

  • Database Trends and Applications (DBTA) hosts a 1-hour live roundtable webcast on Thursday, April 7 at 11:00am PST / 2:00pm EST on the topic of "Leveraging Big Data with Hadoop, NoSQL and RDBMS". Presenters include Brian Bulkowski, CTO and Co-founder of Aerospike; Kevin Petrie, Sr. Director & Technology Evangelist at Attunity; and Reiner Kappenberger, Global Product Manager at Hewlett Packard Enterprise Security. Sign up here to reserve your seat!
Cool Products and Services
  • Dev teams are using LaunchDarkly’s Feature Flags as a Service to get unprecedented control over feature launches. LaunchDarkly allows you to cleanly separate code deployment from rollout. We make it super easy to enable functionality for whoever you want, whenever you want. See how it works.

  • TrueSight Pulse is SaaS IT performance monitoring with one-second resolution, visualization and alerting. Monitor on-prem, cloud, VMs and containers with custom dashboards and alert on any metric. Start your free trial with no code or credit card.

  • Turn chaotic logs and metrics into actionable data. Scalyr is a tool your entire team will love. Get visibility into your production issues without juggling multiple tools and tabs. Loved and used by teams at Codecademy, ReturnPath, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net

  • VividCortex measures your database servers’ work (queries), not just global counters. If you’re not monitoring query performance at a deep level, you’re missing opportunities to boost availability, turbocharge performance, ship better code faster, and ultimately delight more customers. VividCortex is a next-generation SaaS platform that helps you find and eliminate database performance problems at scale.

  • MemSQL provides a distributed in-memory database for high value data. It's designed to handle extreme data ingest and store the data for real-time, streaming and historical analysis using SQL. MemSQL also cost effectively supports both application and ad-hoc queries concurrently across all data. Start a free 30 day trial here: http://www.memsql.com/

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Also available on Amazon Web Services. Free instant trial, 2 hours of FREE deployment support, no sign-up required. http://aiscaler.com

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below...

Categories: Architecture

Snuggling Up to Papers We Love - What's Your Favorite Paper?


From a talk by @aysylu22 at QCon London on modern computer science applied to distributed systems in practice.

 

"CS research is timeless." Lessons learned are always pertinent. @aysylu22 #qconlondon

— Paula Walter (@paulacwalter) March 8, 2016

 

There has been a renaissance in the appreciation of computer science papers as a relevant source of wisdom for building today's complex systems. If you're having a problem there's likely some obscure paper written by a researcher twenty years ago that just might help. Which isn't to say there aren't problems with papers, but there's no doubt much of the technology we take for granted today had its start in a research paper. If you want to push the edge it helps to learn from primary research that has helped define the edge.

If you would like to share your love of papers, be proud, you are not alone:

What's Your Favorite Paper? 

If you ask your average person they'll have a favorite movie, book, song, or Marvel Universe character, but it's unlikely they'll have have a favorite paper. If you've made it this far that's probably not you.

My favorite paper of all time is without a doubt SEDA: An Architecture for Well-Conditioned, Scalable Internet Services. After programming real-time distributed systems for a long time I was looking to solve a complex work scheduling problem in a resource constrained embedded system. I stumbled upon this paper and it blew my mind. While I determined that the task scheduling latency of SEDA wouldn't be appropriate for my problem, the paper gave me a whole new way to look out how programs were structured and I used those insights on many later projects.

If you have another source of papers or a favorite paper please feel free to share.

Related Articles
Categories: Architecture

Don’t let your architecture become the new hardware

Yesterday I presented “Big data in the cloud – Welcome to cost oriented design” @ BDX2016 (slides are available here, and I think the video would be available soon).

One of the points I made that was that you need to continuously optimize the solution to handle costs (of course there can be and there are other drivers to that like performance, security etc.) – these optimizations may result in the need to change the architecture either by replacing a technology or redesigning parts of the solutions. I demonstrated this with a couple of such transformations we did (and doing, and will be doing in the future) @ AppsFlyer.

Ripping out a technology and implanting another in a running system sounds like a scary and risky move, but truly – architectures should be as agile and evolvable as any other part of your system. If you can be agile about adding features you should also be agile about your architecture. This is something we must be able to do

*Rant on*

Hardware vs. Software -the mere names tell you the difference. Hardware is rigid, fixed, Software is flexible, easy to change, soft … software.  lots of years ago when I worked on a system that developed both hardware and software components – the hardware guys always said they had to finalize designs before they start “cutting down the iron” – after that,  changes get expensive , hard, .. hardware.

Along came virtualization and then the cloud, hardware is probably  still rigid but for all intents and purposes, it is as flexible as you want it to be. You can change instance type to a bigger, stronger machine without effort,you can customize the vm sand you can add as many  more machines as you’d like – flexible, easy, soft… er… hardware

Software cannot be the bottleneck to change – I don’t want to sound too much like Trump but it is time to make software soft again!

*Rant off

By the way, this is not just a rant – I actually wrote about evolving architectures in the past (2010). You might want to read these posts to learn more:

Categories: Architecture

Digital Transformation is Ongoing

Digital Transformation is much broader than a technical play.  It’s a chance to reimagine your customer experience, how your employees work, and how you perform operations.

It’s also a chance to continuously create and capture value in new and innovative ways, and I don’t just mean with DevOps.

Your business isn’t static.  Neither is the world.  Neither is the market.  Neither is Digital Transformation.

Instead, Digital Transformation is a way to continuously evolve how you create and capture value in a mobile-first, cloud-first world.

In Digital Transformation Dr. Mark Baker shares how “digital” is more than just bits and bytes and there is always more Digital Transformation that can be done.

Digital Transformation is Bound by Business Decisions, Not Technical Ones

Your Digital Transformation should not be bounded by technical decisions.  Your Digital Business Transformation approach should be driven by your business decisions and your business design.  What matters is that the landscape is digital and that you have to design for new customer experiences, new ways of working, and new ways of performing operations in a mobile-first, cloud-first world.

Via Digital Transformation:

“Today, when we talk of digital transformation we mean restructuring an organization to use any and all information and network-based technologies that increase its competitiveness, in a way that, over a period of time, excludes and out-competes un-transformed organizations.  Of course, in a literal sense, when we walk literally about digital we mean something like expressing data as series of the digits 0 and 1 or using or storing data or information in the form of digital signals: digital TV, a digital recording or a digital computer system.

However, if we think about it that way the whole scope of our understanding and what we are thinking of achieving is quite limited and fairly technical.

In the bigger sense of digital we mean a road map that includes the full process of making a business or service so that every part is freely accessible at every level with bounds set by explicit management models, not by physical constraints.  Ultimately it means that all decisions become business or usage decisions, not technical ones.”

Example of First Generation Digital Transformation

There are always some basic things you can do to get in the game of Digital Transformation.  But that is just the start.  Baker shares an example using a library and how they performed their Digital Transformation.

Via Digital Transformation:

“It might be useful to give a simple example where, whose general principles apply to all digital projects.  The Bodleian Libraries are a collection of approximately 40 libraries that serve the University of Oxford in England.  One of the largest and most important libraries in the world, they hold 11 million printed items, 153 miles (246 kilometers) of shelving, including 3,224 bays with 95,000 shelf levels, and 600 map cabinets to hold 1.2 million maps and other items.

During the first generation of transformation I talked to senior librarians at the Bodleian, and the digital library projects that I was told of turned texts into bitmaps.  Information was still effectively siloed and not electronically searchable within books, but the advantage of digital transformation at that stage was that the physical master copies were protected and copies could be sent with manually controlled access over electronic network to authorized users anywhere in the world.”

Example of Second Generation Digital Transformation

Once you go digital, more opportunities open up for further transformation.  Baker continues the example of a library that undergoes Digital Transformation.

Via Digital Transformation:

“Later more advanced approaches, like Project Guttenberg, digitized the text into ASCII format so that catalogs of books were both digital and searchable as were the individual books.  Beyond that, projects like Google Books Library Project allowed the whole contents of all the books to become accessible to a single keyword search that could search all text across volumes.”

There is Always More Digital Transformation That Can Be Done

There is always more you can do and there are many stages to a full Digital Transformation.

Via Digital Transformation:

“Of course, going digital goes beyond digitizing content and a more advanced model would determine accessibility, access and usage rights and payments, not just in the local user community but worldwide.  In a project of that type any user would be able to do keyword searches across all the contents of a particular library and then usage and any payment would be determined for the specific books or documents they wanted access to, appropriate access would be granted and payment (if any) would be collected. If acquisition was performed on the same platform then requests for information, usage statistics, reader feedback and null-searches could be matched to the acquisition of new materials for the library, so as to better serve the users.

Ultimately even search goes further so that improved semantic search tools would allow search by meaning s well as by key words or phrases, as well as predictive analysis of future usage creating a proactive model, rather than a  reactive model where the available content is always out of date.

At each state the instigators might have expressed the view that they had ‘gone digital’ and at each stage there would have been much, much more that could be done.  This is, of course, just one specific instance of digital transformation related to libraries, but shows a simplified example of how there are many stages to a full transformation.”

Digital Transformation is not done when you are “transformed.” 

It’s a journey of continuous evolution.

You Might Also Like

Build Better Business Cases for Digital Initiatives

Business Value Generation is the New Bottleneck

Continuous Value Delivery the Agile Way

Digital Transformation is Business Transformation

Drive Business Transformation by Reimagining Your Customer Experience

Drive Business Transformation by Reimagining Your Operations

Dual-Speed IT Drives Digital Business Transformation

How Leaders are Building Digital Skills

How To Build a Roadmap for Your Digital Transformation

Microsoft Stories of Digital Business Transformation

Categories: Architecture, Programming

Stuff The Internet Says On Scalability For March 11th, 2016


The circle of life. Traffic flow through microservices at Netflix (Rob Young)

 

If you like this sort of Stuff then please consider offering your support on Patreon.
  • 400Gbps: DDoS attack; 50,000: frames per second Mythbusters films in HD; 3,900: pages Paul Klee’s Personal Notebooks; 1 terabit: satellites deliver in-flight Internet access at hundreds of megabits per second; 18%: overall mobile market revenue increase; 21 TB: amount of date the BBC writes daily to S3; $300 million: Snapchat revenue; 

  • Quotable Quotes:
    • Dark Territory:  Yes, he told them, the NORAD computer was supposed to be closed, but some officers wanted to work from home on the weekend, so they’d leave a port open.
    • @davefarley77: If heartbeat was a clock cycle, retrieving data from fastest SSD is equivalent to crossing whole of London on foot  @__Abigor__ #qconlondon
    • @fiddur: "Legacy is everything you wrote before lunch." - @russmiles #qconlondon
    • @BarryNL: Persistent memory could be the biggest change to computer architecture in 50 years. #qconlondon
    • @mpaluchowski: "You can tell which services are too big. That's the ones developers don't want to work with." #qconlondon @SteveGodwin
    • @danielbryantuk: "I'm not going to say how big microservices should be, but at the BBC we have converged on about 600 lines of Java" @SteveGodwin #qconlondon
    • Steve Kerr~ What we have to get back to is simple, simpl, simple. That's good enough. The leads to the spectacular. You can't try the spectacular without doing the simple first. Our guys are trying to make the spectacular plays when we just have to make the easy ones. If we don't get that cleaned up we're in big trouble.
    • Dark Territory: a disturbing thought smacked a few analysts inside NSA: Anything we’re doing to them, they can do to us.
    • @andyhedges: ~100k TPS with JDK SSL, then ~500k TPS with netty equivalent on same box. Netty fully uses the server's CPU resources too. #qconlondon
    • Paul Marks: Humanoid robots can’t outsource their brains to the cloud due to network latency
    • @manumarchal: O.5TB generated during each flight by jet engines sensors, used for optimising fuel consumption and accelerating repair #Iot #qconlondon
    • fhe: It's both exciting and eerie [AlphaGo]. It's like another intelligent species opening up a new way of looking at the world (at least for this very specific domain). and much to our surprise, it's a new way that's more powerful than ours.
    • @jaykreps: "Part of using Google's Cloud is convincing yourself that Google will invest 5+ years in really entering the market"
    • DEAN TAKAHASHI: With just 3 games, Supercell made $924M in profits on $2.3B in revenue in 2015.
    • @anne_e_currie: Even an anti-wrinkle cream liked my tweet about containers at #qconlondon. It's good to see #container appreciation has spread so wide.
    • @KingPrawnBalls: Failure is inevitable. What matters is that u learn from it. Never fail the same way twice! #qconlondon Josh Evans, Director Ops Eng Netflix
    • Quiizlet: Everyone involved unanimously picked GCP. It came down to this: we believe the core technology is better.
    • @KevlinHenney: "I have to change the word 'compassion' to 'derisking the people problem' when dealing with upper management." — @kkirk #QConLondon
    • People have said so much good stuff this week it can't all fit in the summary. Please read the whole post to see all the Quotable Quotes.

  • Strange to think the impact movies have had on national security policy. Dark Territory: The Secret History of Cyber War. Ronald Reagan after watching the movie WarGames asked if someone could hack the military. The answer: Yes, the problem is much worse than you think. Did anything happen? Nope. People didn't understand computers back then so they didn't think there was a threat (or opportunity in war). A stance that wouldn't change for over a decade. Admiral John "Mike" McConnell watched Sneakers and came up with a NSA mission statement from a soliloquy in the movie: The world isn’t run by weapons anymore, or energy, or money. It’s run by ones and zeroes, little bits of data. It’s all just electrons. . . . There’s a war out there, old friend, a world war. And it’s not about who’s got the most bullets. It’s about who controls the information: what we see and hear, how we work, what we think. It’s all about the information. 

  • Think about this: Amazon launched S3 on March 14, 2006 and with it they started the cloud revolution. That's just ten years ago! James Hamilton in A Decade of Innovation takes a little trip down memory lane. He lists year by year the major AWS product releases and it's impressive. Contributing to this speed may be how decisions are made: Another interesting aspect of AWS is how product or engineering debates are handled. These arguments come up frequently and are as actively debated at AWS as at any company. These decisions might even be argued with more fervor and conviction at AWS but its data that closes the debates and decisions are made remarkably quickly. At AWS instead of having a “strategy” and convincing customers that is what they really need, we deliver features we find useful ourselves and we invest quickly in services that customers adopt broadly. Good services become great services fast.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

The Simple Leads to the Spectacular

 

Steve Kerr, head coach of the record setting Golden State Warriors (my local Bay Area NBA basketball team), has this to say about what the team needs to do to get back on track (paraphrased):

What we have to get back to is simple, simple, simple. That's good enough. The simple leads to the spectacular. You can't try the spectacular without doing the simple first. Make the simple pass. Our guys are trying to make the spectacular plays when we just have to make the easy ones. If we don't get that cleaned up we're in big trouble. 

If you play the software game, doesn't this resonate somewhere deep down in your git repository?

If you don't like basketball or despise sports metaphors this is a good place to stop reading. The idea that "The simple leads to the spectacular" is probably the best TLDR of Keep it Simple Stupid I've ever heard.

Software development is fundamentally a team sport. It usually takes a while for this lesson to pound itself into the typical lone wolf developer brain. After experiencing a stack of failed projects I know it took an embarrassingly long time for me to notice this pattern. It's one of those truths that gradually reveals itself over time...

Categories: Architecture

Structurizr for .NET

Coding the Architecture - Simon Brown - Wed, 03/09/2016 - 16:35

The initial version of Structurizr was targeted at the Java ecosystem (see "Structurizr for Java"), for no other reason than it's what I'm most familiar with. Although this works for a good portion of the organisations that I visit when doing training/consulting, an equally sized portion use the Microsoft stack. For this reason, I've put together Structurizr for .NET, which is more or less a direct port of the Java version, with some automatically generated code from Swagger used as a starting point. It's by no means "feature complete" yet, especially since none of the component finder code (the part that extracts components automatically from a codebase) is present, but there's enough to create some basic diagrams. Here's some example code that creates a software model for the "Financial Risk System" case study that I use in my workshops.

It creates the following Context, Container and Component diagrams.

If you want to take a look or try it out, the source code can be found on GitHub and there's an initial version of the package on NuGet. Have fun!

Categories: Architecture