Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Architecture

Shneiderman's mantra

Coding the Architecture - Simon Brown - Thu, 01/08/2015 - 10:01

I attended a fantastic talk about big data visualisation at the YOW! 2014 conference in Sydney last month (slides), where Doug Talbott talked about how to understand and visualise large quantities of data. One of the things he mentioned was Shneiderman's mantra:

Overview first, zoom and filter, then details-on-demand

Leaving aside the thorny issue of how teams structure their software systems as code, one of the major problems I see teams having with software architecture is how to think about their systems. There are various ways to do this, including a number of view catalogs (e.g. logical view, design view, development view, etc) and I have my C4 model that focuses on the static structure of a software system. If you inherit an existing codebase and are asked to create a software architecture model though, where do you start? And how to people start understanding the model as quickly as possible so they can get on with their job?

Shneiderman's mantra fits really nicely with the C4 model because it's hierarchical.

Shneiderman's mantra and the C4 software architecture model

Overview first (context and container diagrams)

My starting point for understanding any software system is to draw a system context diagram. This helps me to understand the scope of the system, who is using it and what the key system dependencies are. It's usually quick to draw and quick to understand.

Next I'll open up the system and draw a diagram showing the containers (web applications, mobile apps, standalone applications, databases, file systems, message buses, etc) that make up the system. This shows the overall shape of the software system, how responsibilities have been distributed and the key technology choices that have been made.

Zoom and filter (component diagrams)

As developers, we often need more detail, so I'll then zoom into each (interesting) container in turn and show the "components" inside it. This is where I show how each application has been decomposed into components, services, modules, layers, etc, along with a brief note about key responsibilities and technology choices. If you're hand-drawing the diagrams, this part can get a little tedious, which is why I'm focussing on creating a software architecture model as code, and automating as much of this as possible.

Details on demand (class diagrams)

Optionally, I might progress deeper into the hierarchy to show the classes* that make up a particular component, service, module, layer, etc. Ultimately though, this detail resides in the code and, as software developers, we can get that on demand.

Understanding a large and/or complex software system

Next time you're asked to create an architecture model, understand an existing system, present an system overview, do some software archaeology, etc, my advice is to keep Shneiderman's mantra in mind. Start at the top and work into the detail, creating a story that gets deeper into the detail as it progresses. The C4 model is a great way to do this and if you'd like an introduction to it (with example diagrams), you can take a look at Simple Sketches for Diagramming Your Software Architecture on the new Voxxed website.

* this assumes an OO language like Java or C#, for example

Categories: Architecture

Habits, Dreams, and Goals

I’ve been talking to people in the halls about what they learned about goals from last year, and what they are going to do differently this year.   We’ve had chats about New Years Resolutions, habits, goals, and big dreams. (My theme is Dream Big for 2015.)

Here are a few of the insights that I’ve been sharing with people that really seems to create a lot clarity:

  1. Dream big first, then create your goals.  Too many people start with goals, but miss the dream that holds everything together.   The dream is the backdrop and it needs to inspire you and pull your forward.  Your dream needs to be actionable and believable, and it needs to reflect your passion and your purpose.
  2. There are three types of actions:  habits, goals, and inspired actions.   Habits can help support our goals and reach our dreams.   Goals are really the above and beyond that we set our sights on and help us funnel and focus our energy to reach meaningful milestones.   They take deliberate focus and intent.  You don’t randomly learn to play the violin with skill.  It takes goals.  Inspired actions are the flashes of insight and moments of brilliance.
  3. People mess up by focusing on goals, but not having any habits that support them.  For example, if I have an extreme fitness goal, but I have the ice-cream habit, I might not reach my goals.  Or, if I want to be an early bird, but I have the party-all-night long, or a I’m a late-night reader, that might not work out so well.  
  4. People mess up on their habits when they have no goals.  They might inch their way forward, but they can easily spend an entire year, and not actually have anything significant or meaningful for themselves, because they never took the chance to dream big, or set a goal they cared about.   So while they’ve made progress, they didn’t make any real pop.   Their life was slow and steady.  In some cases, this is great, if all they wanted.  But I also know people that feel like they wasted the year, because they didn’t do what they knew they were capable of, or wanted to achieve.
  5. People can build habits that help them reach new goals.   Some people I knew have built fantastic habits.  They put a strong foundation in place that helps them reach for more.  They grow better, faster, stronger, and more powerful.   In my own experience, I had some extreme fitness goals, but I started with a few healthy habits.  My best one is wake up, work out.  I just do it.  I do a 30 minute workout.   I don’t have to think about it, it’s just part of my day like brushing my teeth.  Since it’s a habit, I keep doing it, so I get better over time.  When I first started the workout, I sucked.  I repeated the same workout three times, but by the third time, I was on fire.   And, since it’s a habit, it’s there for me, as a staple in my day, and, in reality, the most empowering part of my day.  It boosts me and gives me energy that makes everything else in my day, way easier, much easier to deal with, and I can do things in half the time, or in some cases 10X.

Maybe the most important insight is that while you don’t need goals to make your habits effective, it’s really easy to spend a year, and then wonder where the year went, without the meaningful milestones to look back on.   That said, I’ve had a few years, where I simply focused on habits without specific goals, but I always had a vision for a better me, or a better future in mind (more like a direction than a destination.)

As I’ve taken friends and colleagues through some of my learnings over the holidays, regarding habits, dreams, and goals, I’ve had a few people say that I should put it all together and share it, since it might help more people add some clarity to setting and achieving their goals.

Here it is:

How Dreams, Goals, and Habits Fit Together

Enjoy, and Dream Big for 2015.

Categories: Architecture, Programming

The Ultimate Guide: 5 Methods for Debugging Production Servers at Scale

This a guest post by Alex Zhitnitsky, an engineer working at Takipi, who is on a mission to help Java and Scala developers solve bugs in production and rid the world of buggy software.

How to approach the production debugging conundrum?

All sorts of wild things happen when your code leaves the safe and warm development environment. Unlike the comfort of the debugger in your favorite IDE, when errors happen on a live server - you better come prepared. No more breakpoints, step over, or step into, and you can forget about adding that quick line of code to help you understand what just happened. In production, bad things happen first and then you have to figure out what exactly went wrong. To be able to debug in this kind of environment we first need to switch our debugging mindset to plan ahead. If you’re not prepared with good practices in advance, roaming around aimlessly through the logs wouldn’t be too effective.

And that’s not all. With high scalability architectures, enter high scalability errors. In many cases we find transactions that originate on one machine or microservice and break something on another. Together with Continuous Delivery practices and constant code changes, errors find their way to production with an increasing rate. The biggest problem we’re facing here is capturing the exact state which led to the error, what were the variable values, which thread are we in, and what was this piece of code even trying to do?

Let’s take a look at 5 methods that can help us answer just that. Distributed logging, advanced jstack techniques, BTrace and other custom JVM agents:

1. Distributed Logging
Categories: Architecture

Sponsored Post: Wikia, MemSQL, Campanja, Hypertable, Sprout Social, Scalyr, FoundationDB, AiScaler, Aerospike, AppDynamics, ManageEngine, Site24x7

Who's Hiring?
  • DevOps Engineer for Wikia. Wikia is the go-to place for fan content that is created entirely by fans! As a Quantcast Top 20 site with over 120 million monthly uniques we are tackling very interesting problems at a scale you won't find at many other places. We embrace a DevOps culture and are looking to expand our team with people that are excited about working with just about every piece of our stack. You'll also partner with our platform team as they break down the monolith and move towards service oriented architecture. Please apply here.

  • Engineer Manager - Platform. At Wikia we're tackling interesting problems at a scale you won't find at many other places. We're a Quantcast Top 20 site with over 120 million monthly uniques. 100% of the content on our 400,000+ communities is user generated. That combination of scale and UGC creates some pretty compelling challenges and on top of that we're working on moving away from a monolithic architecture and actively working on finding the best technologies to best suit each individual piece of our platform. We're currently in search of an experienced Engineer Manager to help drive this process. Please apply here.

  • Campanja is an Internet advertising optimization company born in the cloud and today we are one of the nordics bigger AWS consumers, the time has come for us to the embrace the next generation of cloud infrastructure. We believe in immutable infrastructure, container technology and micro services, we hope to use PaaS when we can get away with it but consume at the IaaS layer when we have to. Please apply here.

  • Performance and Scale EngineerSprout Social, will be like a physical trainer for the Sprout social media management platform: you will evaluate and make improvements to keep our large, diverse tech stack happy, healthy, and, most importantly, fast. You'll work up and down our back-end stack - from our RESTful API through to our myriad data systems and into the Java services and Hadoop clusters that feed them - searching for SPOFs, performance issues, and places where we can shore things up. Apply here.

  • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here.

  • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here.
Fun and Informative Events
  • Sign Up for New Aerospike Training Courses.  Aerospike now offers two certified training courses; Aerospike for Developers and Aerospike for Administrators & Operators, to help you get the most out of your deployment.  Find a training course near you. http://www.aerospike.com/aerospike-training/
Cool Products and Services
  • MemSQL provides a distributed in-memory database for high value data. It's designed to handle extreme data ingest and store the data for real-time, streaming and historical analysis using SQL. MemSQL also cost effectively supports both application and ad-hoc queries concurrently across all data. Start a free 30 day trial here: http://www.memsql.com/

  • Aerospike Hits 1M writes per second with 6x Fewer Servers than Cassandra. A new Google Compute Engine benchmark demonstrates how the Aerospike database hit 1 million writes per second with just 50 nodes - compared to Cassandra's 300 nodes. Read the benchmark: http://www.aerospike.com/blog/1m-wps-6x-fewer-servers-than-cassandra/

  • Hypertable Inc. Announces New UpTime Support Subscription Packages. The developer of Hypertable, an open-source, high-performance, massively scalable database, announces three new UpTime support subscription packages – Premium 24/7, Enterprise 24/7 and Basic. 24/7/365 support packages start at just $1995 per month for a ten node cluster -- $49.95 per machine, per month thereafter. For more information visit us on the Web at http://www.hypertable.com/. Connect with Hypertable: @hypertable--Blog.

  • FoundationDB 3.0. 3.0 makes the power of a multi-model, ACID transactional database available to a set of new connected device apps that are generating data at previously unheard of speed. It is the fastest, most scalable, transactional database in the cloud - A 32 machine cluster running on Amazon EC2 sustained more than 14M random operations per second.

  • Diagnose server issues from a single tab. The Scalyr log management tool replaces all your monitoring and analysis services with one, so you can pinpoint and resolve issues without juggling multiple tools and tabs. It's a universal tool for visibility into your production systems. Log aggregation, server metrics, monitoring, alerting, dashboards, and more. Not just “hosted grep” or “hosted graphs,” but enterprise-grade functionality with sane pricing and insane performance. Trusted by in-the-know companies like Codecademy – try it free! (See how Scalyr is different if you're looking for a Splunk alternative.)

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Cloud deployable. Free instant trial, no sign-up required.  http://aiscaler.com/

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Categories: Architecture

Resolving the conflict between software architecture and code

Coding the Architecture - Simon Brown - Tue, 01/06/2015 - 10:23

So, 2015 ... happy new year! 2014 was a busy year with workshops, conferences and consulting gigs in countries ranging from Iceland to Australia. I'd like to say a huge thank you to everybody who made 2014 so much fun.

Software architecture vs code

One of the things that I spent a good chunk of time on during 2014 was the conflict between software architecture and code. I've written about this before, but you will have seen this in action if the code for your software system doesn't reflect the architecture diagrams you have on the wall. If you've not seen it, my closing keynote from the ABB DevDay conference in Kraków, Poland last September provides a good summary of this.

What I'm really interested in is how we can solve this problem. And that's really where my focus is going to be this year, by taking my C4 software architecture model and representing it as code. I already have some experimental code and tooling that you can find at structurizr.com, but I'm going to be enhancing and expanding this over the coming weeks and months. I want to get people thinking about how to appropriately structure their codebase, understanding that there are different strategies for modularity and adopting, what George Fairbanks calls, an architecturally-evident coding style. I also want to provide tooling that helps people create software architecture models and keep them up to date, ideally based upon the real code and with as much automation as possible. To give you an example, here's a post about diagramming Spring MVC webapps.

I'll be posting updates on the blog, but if you want to hear me talk about this, I'll be at the following conferences over the next few months.

"Software Architecture for Developers" - now only $10

As a final note, my Software Architecture for Developers ebook is only $10 until the end of this week.

Software Architecture for Developers

All the best for 2015.

Categories: Architecture

Use 30 Days of Getting Results to Help You Reach Your Goals for 2015

imageSeveral Summers back, I used a 30 Day Improvement Sprint to share my best insights and best lessons learned on getting results.  I called it 30 Days of Getting Results:

30 Days of Getting Results

It’s timeless advice to help you be YOUR best.

The overall goal of the site was to help you master productivity, master time management, and achieve work life balance.   The idea was that by spending a little time each day, you would get back lots of time and energy and produce better results.  And we all need an edge in work and life.

Rise Above Productivity, Time Management, and Work-Life Balance Challenges

Here are the key things that I tried to help you with:

  • How to set yourself up for success on a daily basis
  • How to create a simple system your can use for getting great results in work and life
  • How to use proven practices to master time management, motivation, and personal productivity
  • How to embrace change and get better results in any situation
  • How to triple your personal productivity
  • How to focus and direct your attention with skill
  • How to use your strengths to create a powerful edge for getting results
  • How to change a habit and make it stick
  • How to achieve better work-life balance and spend more time doing the things you love

So if you’re struggling with any of the above, you might find just the piece of advice or the one or two ideas that help you find your breakthrough.

The Making of 30 Days of Getting Results

Behind the scenes, when I wrote each of the 30 days, I gave myself a 20-minute time limit (a 20-minute timebox for you in the know.)  I would then write as if writing to somebody where I only had a small window of time to help them as best I could to achieve better results, in any situation.

It might seem like the first few days start slow, but things pick up from there pretty fast.  Also, it’s self-paced so you can hop around to any particular day that you think you need the most.

I’ve had many people tell me that it was the course that they needed that helped them set and achieve better goals, while also helping them make new habits and break bad habits.   It’s also helped them find more energy as well as enjoy more of the things that they do.   It’s also helped them find ways to spend more time in their strengths and do what makes them come alive.

I will say that the user experience isn’t that great.   The site was a test and I didn’t want to spend a lot of time on the site design.    That said, it’s pretty straightforward.   When you go to the home page at 30 Days of Getting Results, you’ll see a brief intro and overview, and then you can dive in from there, by either starting with Day 1: Take a Tour of Agile Results, or  by clicking through the 30 Days on the left-hand side of the menu.

30 Days of Getting Results at a Glance

Here are all the days at a glance for your convenience:

  • Overview
  • 30 Days at a Glance
  • Day 1 – Take a Tour
  • Day 2 – Monday Vision
  • Day 3 – Daily Outcomes
  • Day 4 – Let Things Slough Off
  • Day 5 – Hot Spots
  • Day 6 – Friday Reflection
  • Day 7 – Setup Boundaries
  • Day 8 – Dump Your Brain
  • Day 9 – Prioritize Your Day
  • Day 10 – Feel Strong
  • Day 11 – Reduce Friction
  • Day 12 – Productivity Personas
  • Day 13 – Triage
  • Day 14 – Carve Out Time
  • Day 15 – Achieve a Peaceful Calm
  • Day 16 – Use Metaphors
  • Day 17 – Add Power Hours
  • Day 18 – Add Creative Hours
  • Day 19 – Who are You Doing it For?
  • Day 20 – Ask Better Questions
  • Day 21 — Carry the Good Forward
  • Day 22 – Design Your Day
  • Day 23 – Design Your Week
  • Day 24 – Bounce Back with Skill
  • Day 25 – Fix Time, Flex Scope
  • Day 26 – Solve Problems with Skill
  • Day 27 – Do Something Great
  • Day 28 – Find Your One Thing
  • Day 29 – Find Your Arena
  • Day 30 – Take It to the Next Level
  • The course is free.  Hopefully that doesn’t de-value it.   It has a lot of the lessons you would learn in some of the most advanced productivity and time management training.  

    The Structure of the Daily Lessons

    The structure of each day is the same.  It includes an outcome, a lesson, and an assignment.  And right up front, I include a relevant quote and picture.   Here is an example of Day 24 – Bounce Back with Skill:

    Quote: “Life is not about how fast you run, or how high you climb, but how well you bounce.” – Anonymous Your Outcome: Your Outcome: Bounce back with skill and roll with the punches.  Learn to draw from multiple sources of strength and energy, including your mind, body, emotions, and spirit. Lesson Welcome to day 24 of 30 Days of Getting Results, based on my book, Getting Results the Agile Way.  In day 23, you learned how to design your week with skill to get a fresh start, establish routines that support and renew you, and spend more time on the things that count for you.   Today, we learn how to bounce back with skill.  Bouncing back with skill helps us roll with the punches, refuel our bodies, and keep our spark alive.  It’s how we keep our engine going when the rest of us says “we can’t” and it’s how we “shut down” or “turn it off” so we can bounce back stronger.
    (For the rest of the lesson, see Day 24 – Bounce Back with Skill … ) Assignment

    Today’s Assignment

    1. Find one of your past victories in life and add that to your mental flip book of scenes to draw from when you need it most.
    2. Find one metaphor to help you to represent how you bounce back in life.
    3. Find one song or one saying to have in your mind that you can use as a one-liner reminder to take the right actions when it counts.  For example, one that some people like is “Stand strong when tested.”
    Get Started with 30 Days of Getting Results

    If you want to start off well for 2015, and you have big dreams and big goals in mind, then give 30 Days of Getting Results a try:

    30 Days of Getting Results

    If you want to take it slow and steady, then just try one lesson each day.  If you’re feeling gung-ho, then see how quickly you can make it through all 30 at your own pace.

    To help you stay on track, if you take the slow and steady route, build the habit by adding a simple reminder to your calendar in the morning to go and take the next lesson.    Do it Monday through Friday and take the weekends off.

    Enjoy and best wishes for your best year ever.

    Dream Big for 2015 (my personal theme for 2015)

     

     

     

  • Categories: Architecture, Programming

    Von Neumann had one piece of advice for us: not to originate anything.

    I don't know about you, but when I read about the exploits of people like John von Neumann, Alan Turing, J. Robert Oppenheimer, and Kurt Gödel in Turing's Cathedral: The Origins of the Digital Universe by George Dyson, I can't help but flash back to the Age of Heroes, where the names are different--Achilles, Odysseus, Agamemnon, and Ajax--but the larger than life story they lived is familiar. Dyson's book is the Iliad of our times, telling the story of great battles of the human mind: the atomic bomb, Turing machines, programmable computers, weather prediction, genetic-modeling, Monte Carlo simulation, and cellular automata.

    Which brings up another question I can't help but ponder: is it the age that makes the person or is it the person that makes the age? Do we have these kind of people today? Or can they only be forged in war?

    Anyway, I found this advice from John von Neumann, as told by Julian Bigelow, about how to go about building the MANIAC  computer. This advice still echoes down project management halls today:

    “Von Neumann had one piece of advice for us: not to originate anything.” This helped put the IAS project in the lead. “One of the reasons our group was successful, and got a big jump on others, was that we set up certain limited objectives, namely that we would not produce any new elementary components,” adds Bigelow. “We would try and use the ones which were available for standard communications purposes. We chose vacuum tubes which were in mass production, and very common types, so that we could hope to get reliable components, and not have to go into component research.”

    They did innovate on architecture by making it possible to store and run programs. Some interesting quotes from the book around that development:

    Categories: Architecture

    Peace of mind in a state of overload

    Xebia Blog - Sun, 01/04/2015 - 21:23

    This article is meant for knowledge workers that want to be more on top of things and feel secure that they haven’t forgotten about something, freeing their mind for the actual tasks at hand. It especially applies to those that are using or want to use SCRUM, a popular and formalized Agile methodology, in their day to day work.
    I got hooked on Agile back in 2005, while working for Db4o, and never looked back since. Improving the process per iteration and having a manageable amount of work per sprint gave me peace of mind and focus, enabling me to deliver the right software solutions to the stakeholders. When I got asked to join a tech startup in 2011 as its CTO I suddenly had a lot more to deal with: hiring and managing staff, managing costs, reporting to the board, applying for subsidies and making sure the books were kept in order. On top of this I still operated as SCRUM Master and technical lead within a SCRUM team.
    During this period one of my co-founders introduced me to Getting things done by David Allen. It took him only about 15 minutes to explain the basics and I got started with it straight away.

    You can optionally watch this presentation to go along with the article:

    Diverse responsibilities

    As knowledge workers, and more specifically consultants, we have a diversity of responsibilities. You have personal responsibilities, like planning a doctor’s appointment or picking up your kids from daycare, and you have responsibilities from your employer like ordering a backup disk, co-creating a course or preparing a presentation. Lastly you also have responsibilities from your client, like sprint tasks, meetings and organizing innovation days. Truly staying on top of all these responsibilities is tough, real tough!

    Agile

    For those of you that are not familiar with Agile methodologies. Agile is an iterative process. In each iteration a team commits to a finite amount of work. A typical iteration has a length of two weeks, which is its deadline. All stories, the units of work in an iteration, can be made actionable by the Agile team.

    GTD

    Getting Things Done takes a slightly different approach. There is a single inbox, comparable to the part of a backlog in SCRUM, that hasn’t been groomed yet. By regularly reviewing the inbox, it can be emptied by turning items into actionable tasks, high-level projects, calendar items, reference material or just trash. Actionable tasks can be extracted from a project, that will move it forward. Actionable tasks should be kept together to be able to prioritize them upon review.

    GTD Chart

    GTD Chart

    Please refer to the book Getting Things Done by David Allen in order to get the full explanation but below follows a quick overview.

    Quick overview of GTD Inbox

    The purpose of the GTD inbox is to collect anything new that might or might not require your attention. Whenever something pops up into your mind that you think requires your attention, either now or in the future, you collect it here. It doesn’t need to be organised. It doesn’t even need to be an attainable goal. As long as you get it off your mind and file it away for review, the inbox has done its job. Reviewing the inbox will empty it into one of the following categories.

    Trash

    You may find that a lot of the things you collect in your GTD inbox don’t really require you to take any action at all, so you can throw them away with impunity! Good riddance!

    File

    Many a time you get a piece of information that you don’t need immediately but would like to be able to reference in the future. These items go from your inbox into a file. This file can be physical, a folder structure on your computer system or something completely different.

    Calendar

    Though people have a tendency to put too many things in their calendar that really don’t need to be done at that exact time, inbox items that do have a specific time window move from there to your calendar.

    Waiting for

    If you delegate something you expect it to be done after a certain period of time. Though these dependencies are kept out of SCRUM for a good reason they are a part of everyday life. Move these from your inbox to your waiting for list so you can check in with whoever you delegated it to in order to be aware of their status.

    Someday maybe

    Colleague sent you an adorable youtube frolicking puppy compilation? Maybe you didn’t have time to watch it when it was brought to your attention but why not put it away to watch later? Items that you don’t need to do but would like to get around to eventually should be moved over here.

    Projects

    There are many things that people have on their to do list that they keep on staring at and get intimidated by for some reason. Often this pertains to items that are of a too high level to pick up straight away. This can range from Bring about world peace to Organise daughter’s birthday party. Both of these tasks really consist of multiple clearly actionable tasks even though one gets the impression the latter is easier to attain in one’s lifetime. Things that should be more clearly defined belong here. Review them and extract actionable tasks from them that move the projects closer to their goal.

    Next actions

    This is the work horse of the GTD system and is where you actually pick up tasks to do. You place well defined tasks here. If you need to call someone about a case, be sure to save this tasks along with the name of the person, their phone number and case reference to remove any impediments, like having to look up their number or the case reference. This way you make the barrier to get something done as low as possible.

    Obstacles

    Of course there are a few obstacles to overcome using these two methods side by side as a consultant.

    Since GTD encourages you to work on the most important thing at that time, you could be required to make private phone calls during consultancy hours or respond to a mail from a co-worker while getting ready for work. This causes work-time and private time to overlap. The obvious solution would be to track work intervals for each task. However, this takes a little bit of time and discipline so should ideally be automated.

    GTD requires you to review at least daily what should be done with stuff in your inbox and what the next most important actions are. This takes time. Is this personal time? Should it be billable? In my opinion working with GTD reduces stress and increases productivity and effectiveness. Thus working with GTD will make you a more valuable employee which is something your employer should invest in.

    While both GTD as well as Agile work with definitions of work, the priority of stories in SCRUM is determined by the Product Owner. How can this be incorporated into GTD? Well, even though the Product Owner is in charge of the priorities inside the sprint, he does not have a full overview of everything that needs doing in your life. Therefore you are the one that determines the order in which your tasks need to be performed. Within these tasks only the ones coming from your sprint have a relative order pre-determined by the PO.

    Improvements

    In my day to day usage of GTD I found that there are a few identifiable improvements.

    Due to work requirements I need to maintain multiple calendars. Since some GTD inbox items end up in your calendar this sometimes means having to create multiple items in your calendar, which causes needless overhead. It would be beneficial if this would be supported by software, so that a GTD inbox item can automatically be delegated to one or more calendars.

    When tasks are coming from external applications like Jira they have to be kept in sync. It would save time if this could be managed automatically.

    Lastly the question of ownership. Who owns a task? The assignee, the author or the company on who’s behalf it needs to be performed? I strongly believe that tasks belong to the author or organisation that the author wrote them for. If tasks have been delegated or synced from external systems they should be revocable by their owner. At the same hand an organisation should not have or control access to tasks a person authored without the author’s permission.

    Conclusion

    Unfortunately there is currently no software tool that would serve all my needs as outlined here. However the most essential properties of such a tool should be: multi-platform to ensure availability, tagging support to be able to categorise without having to split up the list and owned by you.

    Lightweight Quality Attribute Workshop

    Coding the Architecture - Simon Brown - Sun, 01/04/2015 - 21:21

    One of the core concepts in the Software Architecture for Developers course is that the Quality Attributes (non-functional requirements) need to be understood in order to provide foundations for a system's architecture. It's no good building a system that fulfills its user's functional requirements if these are delivered incorrectly. Consider the embedded software in a pacemaker. It may correctly analyse the rhythm of the patient's heart and conclude that a shock is required but if this is performed at the wrong time (possibly due to jitter in the response) then it may kill the patient.

    Discovering that critical quality attributes are not being met can require a complete system redesign e.g. modifying an asynchronous system to be synchronous. Therefore the early identification of key Quality Attributes is important to drive your design and in the selection of tools and technologies.

    However I've often had difficulties getting course attendees to identify specific attributes, as opposed to generic ones, for a case study. For example, most people will identify performance as important but struggle to go beyond this to consider trade-offs between, say, throughput and jitter.

    Therefore, in the last couple of courses, I have expanded the identification of Quality Attributes to include a very brief (and lightweight) Quality Attribute Workshop for our case study.

    The Software Engineering Institute has a description of how to perform a Quality Attribute Workshop which includes a full process and template set. While excellent (and a core part of their ATAM architecture evaluation process) this is too involved for a short training course. We therefore just performed the 'Identification of Architectural Drivers' and review steps.

    Importantly the SEI also provides a very useful tool for the identification of Quality Attributes - a taxonomy. This is not just a list of attributes with a detailed description, it actually breaks down attributes from the generic to the specific. Take, for example, the following diagram for performance:


    Performance Taxonomy (Performance Taxonomy Extracted from Barbacci, Mario; Klein, Mark; Longstaff, Thomas; & Weinstock, Charles. Quality Attributes (CMU/SEI-95-TR-021 ). Software Engineering Institute, Carnegie Mellon University, 1995.)


    The Quality Attributes are broken down under the 'Concerns' branch. For example, in the case study used the 'Response Window' is an important metric which needs analysis.

    The 'Factors' branch, lists properties of the system that can impact the concerns. In our case study the 'Arrival Pattern' and 'Execution Time' are both important factors that need to be considered.

    Lastly the 'Methods' branch lists tools/theories that can be used to analyse the concerns.

    This diagram is useful for identification as it encourages the reader to consider all the aspects of the attribute in question and the measurable specifics for it. Without this taxonomy it is common to hear comments such as "it has to run quick enough" but with the taxonomy the analysis becomes much more detailed and useful.

    However there is a danger, particularly with using a general, external taxonomy. My observation is that once provided with a taxonomy the participants tend to stick very closely to it and forget out the Quality Attributes NOT listed on it. For example the SEI list does not include Usability attributes or anything covering Internationalisation/Localisation. In response to this I'd suggest creating your own domain specific taxonomy. For example, if you work on retail websites you'll want more focus on usability and less on safety criticality.

    Conclusion

    I have found lightweight Quality Attribute Workshops to be a very effective way of identifying Quality Attributes in a short space of time, particularly if you use a Taxonomy to focus the participants. However you must be careful to not become blinkered by what it lists. Therefore I'd suggest you create your own taxonomy, specific to your domain.

    Categories: Architecture

    Lessons Learned from John Maxwell Revisited

    I did a major cleanup of my post on lessons learned from John Maxwell:

    Lessons Learned from John Maxwell

    It should be much easier to read now. 

    It was worth cleaning up because John Maxwell is one of the deepest thinkers in the leadership space.  He’s published more than 50 books on leadership and he lives and breathes leadership in business and in life.

    When I first started studying leadership long ago, John Maxwell’s definition of leadership was the most precise I found:

    “Leadership is influence.”

    As I began to dig through his work, I was amazed at the nuggets and gems and words of wisdom that he shared in so many books.  I started with Your Road Map for Success.   I think my next book was The 21 Irrefutable Laws of Leadership.   Ironically, I didn’t realize it was the same author until I started to notice on my shelf that I had a growing collection of leadership books, all by John Maxwell.

    It was like finding the leadership Sherpa.

    Sure enough, over the years, he continued to fill the shelves at Barnes & Nobles, with book after book on all the various nooks and crannies of leadership. 

    This was about the same time that I noticed how Edward de Bono had filled the shelves with books on thinking.  I realized that some people really share there life’s work as a rich library that is a timeless gift for the world.   I also realized that it really helps people stand out in their field or discipline when they contribute so many guides and guidance to the art and science of whatever their specific focus is.

    What I like about John Maxwell’s work is that it’s plain English and down to Earth.  He writes in a very conversational way, and you can actually see his own progress throughout his books.  In Your Road Map for Success, it’s a great example of how he doesn’t treat leadership as something that comes naturally.  He works hard at it, to build his own knowledge base of patterns, practices, ideas, concepts, and inspirational stories.

    While he’s created a wealth of wisdom to help advance the practice of leadership, I think perhaps his greatest contribution is The 21 Irrefutable Laws of Leadership.  It’s truly a work of art, and he does an amazing job of distilling down the principles that serve as the backbone of effective leadership.

    Categories: Architecture, Programming

    My Story of Personal Transformation

    Everybody has a story.   I thought I would share mine at Sources of Insight:

    My Story of Personal Transformation

    It’s the story of how I figured out how to do more of what makes me come alive, and how to share my unique value with the world.

    It’s a journey, but this story is a look backwards, and how it helped me shape my path forward.

    I included some of the key questions I asked, as well as some of the key resources I used to get a new lens on work and life. 

    Life can really be a game of chutes and ladders, depending on the questions you ask, the choices you make, and the actions you take.

    I think one of the biggest challenges we have in life, is a very personal one.  It’s the challenge of finding our voice.   It’s the challenge of finding our passion, our purpose, and our talents.  It’s the challenge of becoming all that we’re capable of.  And, it’s the challenge of how to make the most of what we’ve got, while helping others in our unique way.

    The other big challenge is avoiding regret, learning to live with regret, or learning how to live without regret.  What we regret the most, are the things we have a chance to change.  It’s our opportunities lost.  Or, to put it another way, we regret the things we didn’t do.   That can include things like not being true to ourselves, not expressing our feelings, not staying in touch with friends, or not letting ourselves be happier.

    The top regrets in life based on research are: education, career, romance, parenting, self-improvement, leisure, finance, health, friends, spirituality, and community.  Education is the top regret because it impacts so many areas of our life, and it’s within our control.

    The way I learned to write my story forward is to combine a combination of answering the following questions on an on-gong basis:

    1. Who do I want to be and what experiences do I want to create?
    2. Am I giving my best where you have my best to give?
    3. Am I  living my values?
    4. Am I saying “Yes” to opportunities?
    5. Am I sharing what I think and feel to the people in my life?

    Transformation is a journey of challenges and changes.  And that’s where our greatest growth comes from.

    Best wishes for your best year, ever.

    Categories: Architecture, Programming

    Stuff The Internet Says On Scalability For January 2nd, 2015

    Hey, it's HighScalability time:


    From Introduction to Metabolic Scaling Theory - From cells to ecosystems

     

    • 53 kilobytes: total amount of RAM in the world in 1953; 180-200 million: daily transactions at The Weather Channel; 
    • Quotable Quotes
      • Enquist, Brian: Life operates over 21 orders of magnitude in size - From Unicells to Whales and Giant Sequoias 
      • George Dyson: Digital computers translate between these two forms of information—structure and sequence—according to definite rules. Bits that are embodied as structure (varying in space, invariant across time) we perceive as memory, and bits that are embodied as sequence (varying in time, invariant across space) we perceive as code. Gates are the intersections where bits span both worlds at the moments of transition
      • : what is “scaling”? In its most elemental form, it simply refers to how systems respond when their sizes change
      • @muratdemirbas: Eventual consistency should not come to mean "Only God can judge me".
      • Raffi Krikorian: Every Problem is a Scaling Problem
      • The High-Interest Credit Card of Technical Debt: Experience has shown that the external world is rarely stable.
      • @Apcera: "#HybridCloud ROI isn’t there, & the complexity is huge." via @stevesi @Recode http://ow.ly/Gspxq  Time for a new solution in 2015. #PaaS
      • Nathan Bronson: I believe that to tackle big problems one must factor complexity into pieces that can each fit in someone’s brain, and that the key to such factoring is to create abstractions that hide complexity behind a simple mental model.

    • A prediction for the new year: algorithm profilers will be a hot new job category. Optical Illusions That Fool Google-Style Image Recognition Algorithms. SEO and HFT are a kind of profiling, but with the spread of algorithms through the consumption of the world by software, the hacking of all sorts of algorithms for advantage will become a permanent fixture of modern life. One more layer to the game.

    • Interesting idea from Brett Slatkin. Our approach to manufacturing is as quaint as punchcards: You'd turn in your punch cards and hope to get the output a week later — sooner if you were lucky...3D printing is slow. Even though laser printing can produce precision parts like rocket engines, it doesn't scale...To build cars, cell phones, and soda cans you need to produce high volumes quickly...What we need is a way to click a button and launch a manufacturing process.

    • If you need to optimize your Rails App for concurrency here's a good source: Heroku and Puma vs. Heroku and Unicorn. Puma was the winner, improving quality of service and reducing hosting costs. With Puman many fewer dynos were needed. The comment section has a vigorous debate.

    • The Current State of the Blockchain: Bitcoin, in its current state, cannot act as a major transaction network. Because blocks are current limited to be 1 MB in size, Bitcoin is limited to handle roughly 7 transactions per second. In comparison, thousands of credit card transactions happen per second across the world. < Good discussion on reddit. Also, The Blockchain is the New Database, Get Ready to Rewrite Everything

    Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

    Categories: Architecture

    Get Your Goals On

    “Another year over, And a new one just begun.” – John Lennon

    Ready to get your game on?

    January is a great time to focus on what you want out of this year.  As you close out last year, you can reflect on what went well and what things you could improve.   Focus on the growth.

    January is also a great time to build some momentum.  January and December are the bookends for your year.  It’s interesting how they are both a month apart and a year apart. 

    What you fill that year with, is your opportunity.

    If you’re having a hard time remembering what it means to dream big, I put together a collection of dream big quotes to rekindle your imagination.

    I’ve also put together a set of posts to help you create goals with skill:

    • 10 Reasons that Stop You from Reaching Your Goals - I see so many people who achieve their goals, and so many people who don’t.  I thought it would be helpful to nail down why so many people don’t achieve their goals, even when they have such good intentions.
    • Are You Living Your Dreams? - Here is a blurb I found from Dr. Lisa Christiansen that helps remind us to dream big, dream often, and live our dreams.
    • Change Your Strategy, Change Your Story, Change Your State - If you want to change your life, you have to change your strategy, you have to change your story, and you have to change your state.
    • Goal-Setting vs. Goal-Planning - Most people don’t step into what achieving their goal would actually take, so they get frustrated or disheartened when they bump into the first obstacles.   Worse, they usually don’t align their schedule and their habits or environment to help them.  They want their goals, they think about their goals, but they don’t put enough structure in place to support them when they need it most, especially if it’s a big habit change.  Don’t let this be you.
    • How Brian Tracy Sets Goals - Brian Tracy has an twelve-step goal-setting methodology that he’s taught to more than a million people. If you follow his approach … You will amaze yourself.  With his goal-setting methodology, he’s seen people transform.  They are astounded by what they start to accomplish.  They become a more powerful, positive, and effective person.  They feel like a winner every hour of the day.  They have a tremendous sense of personal control and direction.  They have more energy and enthusiasm.
    • How John Maxwell Sets Goals - John Maxwell is an internationally recognized leadership expert, speaker, and author.  And, one of his specialties is turning dreams into reality through a simple process of setting goals.
    • How Tony Robbins Sets Goals - This goal-setting approach is one of the most effective ways to motivate you from the inside out and move you to action, so if you have a case of the blahs, or if you want big changes in your life this might just be your answer.
    • How Tony Robbins Transformed His Life with Goals - Tony Robbins wanted to change his life with a passion.  He had hit rock bottom.  He was frustrated and feeling like a failure.  He was physically, emotionally, spiritually, and financially “broke.”  He was alone and almost 40 pounds over-weight.  He was living in a small, studio apartment where he had to wash his dishes in the bathtub, because there was no kitchen.  He wanted out.
    • The Power of Dreams - John Maxwell shares what he’s learned about the power of dreams to shape our goals, to shape our work, and to shape our lives.
    • The Real Price of Your Dreams - Tony Robbins walks through helping a young entrepreneur translate their dream of living like a billionaire into what their lifestyle might actually cost.

    One of the most helpful things I’ve found with goal setting, is to start with 3 dreams or 3 wins for the year.  I learned this while I was putting together Getting Results the Agile Way: A Personal Results System for Work and Life.  As my story goes, I got frustrated and bogged down by a heavy goal process and lost in creating SMART goals.  I finally stepped back and just asked myself, what are Three Wins or Three Outcomes that I want out of this year?  The first things that came to mind were 1) ship my book, 2) get to my fighting weight, and 3) take an epic adventure.

    It wasn’t scientific, but it was significant, and it was simple.  But most of all, it was empowering.

    In retrospect, it seems so obvious now, but what I was missing in my goals was the part that always needs to happen first:  Dream big.  We need to first put our dreams on the table because that’s where meaningful goals are born from.  It’s the dreams that make our goals a force to be reckoned with.  Really, goals are just a way to break our dreams down into chunks of change we can deal with, and to help guide us on our journey towards the end in mind.   That’s why we have to keep pushing our dreams beyond our limits.  That way we don’t try to push ourselves with our goals.  Instead, we pull ourselves with our dreams.

    If you want to know how to get started with Agile Results, before you get the book, you can use the Agile Results QuickStart guide.  You can use it to create your personal results system.   It’s a simple system, but a powerful one.  Individuals, teams, and leaders use it to bring out their best and to make the mot of what they’ve got.

    To give you a quick example, if you want to rise above the noise of your day, just take a quick pause, and write down Three Wins that you want out of today.   If you’re day is pretty tough, you might say, “great breakfast, great lunch, great dinner.”  We have those days.  Or, if you’re feeling pretty good, you might say, “ship feature X” or “clear my backlog” or “finish my presentation” or “win a raving fan”, etc. 

    It sounds simple but by having Three Wins to hold onto for today, it helps you focus.  It helps you prioritize.  And it helps you get back on track, when you get off track.  It also gives you a quick way to feel good about your achievements at the end of the day, because you can actually name them.  They are your private victories.

    So, if you want to practice Agile Results, just remember to think in threes: Three Wins for the Day, Three Wins for the Week, Three Wins for the Month, Three Wins for the Year.    It will help you funnel and focus your time and energy on meaningful results that matter.   And, you’ll build momentum a moment at a time, as you respond to challenges, exercises your choices, and drive your changes in work and in life.

    Categories: Architecture, Programming

    Linus: The whole "parallel computing is the future" is a bunch of crock.

    Linus Torvalds in his usual politically correct way made a typically understated statement about “pushing the whole parallelism snake-oil” that generated almost no response whatsoever.

    Well, not quite. His comment on Avoiding ping pong has generated hundreds of responses, both on the original post and on Reddit.

    The contention:

    The whole "let's parallelize" thing is a huge waste of everybody's time. There's this huge body of "knowledge" that parallel is somehow more efficient, and that whole huge body is pure and utter garbage. Big caches are efficient. Parallel stupid small cores without caches are horrible unless you have a very specific load that is hugely regular (ie graphics).

    Nobody is ever going to go backwards from where we are today. Those complex OoO [Out-of-order execution] cores aren't going away. Scaling isn't going to continue forever, and people want mobility, so the crazies talking about scaling to hundreds of cores are just that - crazy. Why give them an ounce of credibility?

    Where the hell do you envision that those magical parallel algorithms would be used?

    The only place where parallelism matters is in graphics or on the server side, where we already largely have it. Pushing it anywhere else is just pointless.

    So give up on parallelism already. It's not going to happen. End users are fine with roughly on the order of four cores, and you can't fit any more anyway without using too much energy to be practical in that space. And nobody sane would make the cores smaller and weaker in order to fit more of them - the only reason to make them smaller and weaker is because you want to go even further down in power use, so you'd still not have lots of those weak cores.

    Give it up. The whole "parallel computing is the future" is a bunch of crock.

    An interesting question to ponder on the cusp of a new year. What will programs look like in the future? Very different than they look today? Or pretty much the same?

    From the variety of replies to Linus it's obvious we are in no danger of arriving at consensus. There was the usual discussion of the differences between distributed, parallel, concurrent, and multithreading, with each succeeding explanation more confusing than the next. The general gist being that how you describe a problem in code is not how it has to run.  Which is why I was not surprised to see a mini-language war erupt. 

    The idea is parallelization is a problem only because of the old fashioned languages that are used. Use a better language and parallelization of the design can be separated from the runtime and it will all just magically work. There are echoes here of how datacenter architectures are now utilizing schedulers like Mesos to treat entire datacenters as a programmable fabric. 

    One of the more interesting issues raised in the comments was a confusion over what exactly is a server? Can a desktop machine that needs to run fast parallel builds be considered a server? An unsatisfying definition of a not-server may simply be a device that can comfortably run applications that aren't highly parallelized. 

    I pulled out some of the more representative comments from the threads for your enjoyment. The consensus? There is none, but it's quite an interesting discussion...

    Categories: Architecture

    10 technologies that impressed me in 2014

    Agile Testing - Grig Gheorghiu - Wed, 12/31/2014 - 07:26
    Some of these have been new to me, some are old friends that I came to appreicate more. In alphabetical order:

    1. Ansible
    2. Bitcoin
    3. Consul
    4. Docker
    5. Golang
    6. HAProxy
    7. nginx
    8. OpenStack
    9. Sysdig
    10. Varnish

    There’s Magic in the Air

    This is a follow up to Santa Lands on a Virgin Atlantic Plane with 4D Technology.  It turns out that you can measure the magic in the air.

    Here is the page that’s tracking the festive magic:

    There’s Magic in the Air

    Here is a snapshot of the page:

    image

     

    How are we measuring the festive magic?  Using cloud-based Dynamics social analytics software.

    Via the There’s Magic in the Air site:

    ”At Virgin Atlantic we love the festive season and so do lots of our customers.

    So this year, we wanted to see how much ‘festive magic’ our customers and employees are creating around the world.

    Partnering with our friends at Microsoft, we're using cloud based Dynamics social analytics software to calculate the volume and sentiment of social posts on platforms like Facebook and Twitter. We’re analyzing the posts for positive mentions of keywords like ‘Christmas,’ ‘magic’ and ‘reindeer.’ So we can understand how much of that festive magic is being generated every day until 7January, 2015.

    We're doing this in a totally anonymous way, and we're not storing any personal information. We just want to understand how much magic we're collectively creating during this period.”

    What a fun and festive way to show software in action, and use technology to light up the holiday season.

    You Might Also Like

    Santa Lands on a Virgin Atlantic Plane with 4D Technology

    The Microsoft Story

    The Mission of Microsoft Enterprise Services

    Categories: Architecture, Programming

    Productivity Books to Help You Know More, Be More, and Achieve More

    I’ve put together a roundup of the best productivity books that have helped me get better results in work and life.  It contains many of the same books that I recommend to people and teams that I mentor.  Here is the list:

    Productivity Books

    It has a lot of books.  To be productive, it takes a lot of skills, and a lot of self-mastery.  And, you never know which book is going to do it for you.

    I organize the productivity books in a scenario-based way:

    • Top 10 Best Productivity Books
    • Getting Started
    • Advanced Productivity
    • Action
    • Delegation
    • Drive, Energy, and Motivation
    • Focus
    • Goals
    • Learning and School
    • Procrastination
    • Teams and Organizations
    • To-Do Lists
    • Work-Life Balance

    At the end of the list, I included all the books in a simple, flat A-Z list so you can quickly check against your own productivity book collection to see if I have mentioned books that you don’t have or don’t know about.

    I’ve put this list of productivity books together to give you an unfair advantage.   Competition can be fierce.  Remember that the best person to always compete against is you—find ways to be better, faster, or cheaper, when it comes to making things happen.  It’s how you stay in the game, and it’s how you change your game.

    In fact, productivity is a backbone for surviving and thriving.  Yeah, it’s a big deal.

    Also, bear in mind that the big idea behind extreme productivity, or effective productivity is to focus on learning and growth.   If you have a growth mindset, you’ll win in the long-run, because you’ll get better over time, and you can compound your effort.  Also, learn to embrace the effort, and to love the work you do, or love the way you do it.  

    Your work is the ultimate self-expression, and the legacy you leave behind, can be an inspiration for yourself, as well as for others.

    My Productivity Books page will be a living catalog of the books I draw from, and it’s part of my bigger Great Books collection, where I share the world’s best books for insight and action on business, career, leadership, personal development, and more.

    I know there are a lot of productivity books on my list.   If I can only recommend one, I start people off with Getting Results the Agile Way.   I wrote it specifically to help people get better results in work and life with a simple system I developed over time in extreme scenarios.   It integrates proven practices for personal productivity, as well as positive psychology, project management, sports science, and more in terms of achieving high-performance with flow.  And, it’s easy to get started (Here is the Agile Results QuickStart.)

    I believe in the power of books to change lives.   Productivity is just one area, but it’s an area that impacts all the other major areas of our life.

    If you can master productivity, you can know more, be more, and achieve more.

    And, if you balance that with living your values, making an impact, and enjoying the journey, that is the key to living la vida buena.

    Categories: Architecture, Programming

    Santa Lands on a Virgin Atlantic Plane with 4D Technology

    Microsoft and Virgin help land Santa on top of a plane at 30,000 feet.  If you’ve been wondering where Santa’s been, he landed on top of a Virgin Atlantic plane and did a photo shoot with the passengers.

    Microsoft teamed up with Richard Branson and Virgin Atlantic to bring the magic of Christmas to life.  In the world’s first 4D experience in flight, Santa Claus appears to land on top of a Virgin Atlantic plane at 30,000 feet. 

    How’s that for some fancy flying with modern technology?!

    Each passenger was also given a Windows tablet so they could track Father Christmas and chat with him during the flight.

    Here’s the video of Santa landing on top of the plane and visiting with the passengers:

    Video: Santa Lands on Top of a Virgin Atlantic Plane at 30,000 Feet

    Here are a few scenes that show Santa in action …

    Here’s one of Santa’s reindeer peering down into the cabin from on top of the plane:

    image

    Here’s Santa peering down into the cabin from above the plane before he goes inside:

    image

    Santa sees somebody he recognizes:

    image

    Santa boards the plane and walks the cabin:

    image

    The kids are excited to see Santa:

    image

    image

     

    Adults are happy, too:

    image

    image

    Santa has time for some photo shoots:

    image

    image

    Santa leaves to get back to his sleigh on top of the plane:

    image

    Virgin Atlantic and Microsoft wish everybody a very, merry Christmas:

    image

    Here’s Richard Branson’s post on the story:

    Santa Lands on Virgin Atlantic Plane at 30,000 Feet

    Merry Christmas to all and to all a good night.

    JD

    You Might Also Like

    10 Big Ideas from a Christmas Carol

    25 Holiday Classic Movies and Lessons Learned

    Microsoft Cloud Case Studies at a Glance

    Categories: Architecture, Programming

    Speaking computers for more fun !

    Think Before Coding - Jérémie Chassaing - Wed, 12/24/2014 - 00:00

    I didn't try it on mono, but it should also work with some tweaking, see details here

    Xmas is a good time to surprise kids, and what's more fun than a talking computer ?!

    Hello world !

    Nothing's easier, and this kind of Hello World will appeal them to programming in a flash :

    #r "System.Speech"
    open System.Speech.Synthesis
    
    let synt = new SpeechSynthesizer()
    let say s = synt.Speak(s: string)
    
    say "Hello world !" 
    

    Of course, if you're french like me, it'll say this with an awful french accent - something like hélo ouorld !

    But you can select a different voice if available by providing hints:

    open System.Globalization
    let english = CultureInfo.GetCultureInfo("en-US")
    
    synt.SelectVoiceByHints(VoiceGender.NotSet, VoiceAge.NotSet, 1, english)
    say "Hello world !"
    

    Far better !

    Can you beat it ?

    Now, a talking fizz buzz, up to 100 ! Can you beat it ?

    [1 .. 100]
    |> List.map (fun n -> 
        match n%3, n%5 with
        | 0, 0 -> "FizzBuzz"
        | 0, _ -> "Fizz"
        | _, 0 -> "Buzz"
        | _ -> string n )
    |> List.iter say
    
    Even harder !

    Now with a recognizer, we can wait for voice user input.

    The problem with the Grammar API is that it's totally mutable and not really DSL oriented. Let's correct that :

    open System.Speech.Recognition
    
    type Grammar =
        | Phrase of text:string * result: string
        | Lst of Grammar list
        | Alt of Grammar list
        | Repeat of min: int * max: int * Grammar
    
    
    let rec build = function
        | Phrase (text, result) -> 
            // Just build the a single phrase
            GrammarBuilder(SemanticResultValue(text,result))
    
        | Lst grammars -> 
            // Append parts of grammars one after the other
            let builder = GrammarBuilder()
            grammars
            |> List.map build
            |> List.iter builder.Append
            builder
    
        | Alt alternatives -> 
            // Create alternatives
            let choices =
                alternatives
                |> List.map build 
                |> List.toArray
            GrammarBuilder(Choices())
    
        | Repeat(min, max, grammar) -> 
            // Repeat a part of the grammar
            GrammarBuilder(build grammar, min, max)
    

    This is not a full DSL for speach recognition, you can look at all the GrammarBuilder methods to add more possibilities.. Even here, I'll use only Phrase and Alt.

    Now, we need a recognizer and wire the grammar with functions that will be called when a part of the grammar is recognized or rejected. It is mandatory to set grammar's culture to the recognizer's culture. There's usually a single recognizer installed by default on your system and it uses installed system's culture. In my case, it'll be french.

    let recog = new SpeechRecognizer()
    
    let recognize grammar recognized rejected  = 
        let builder = build grammar
        builder.Culture <- recog.RecognizerInfo.Culture
        printfn "%A" recog.RecognizerInfo.Culture
        recog.LoadGrammar(Grammar builder)
        
        recog.SpeechRecognized |> Event.add (fun e -> recognized e.Result.Text (string e.Result.Semantics.Value))
        recog.SpeechRecognitionRejected |> Event.add (fun e -> rejected ())
        recog.Enabled
    

    We can then use this to create a little Christmass quizz thanks to the FSharp.Data FreeBase Type Provider !

    We'll use free base to find a list of Actors who plaid Santa in movies.

    For this, install the FSharp.Data NuGet:

    nuget install FSharp.Data -o packages -x

    The dll should be in .\packages\FSharp.Data\lib\net40\FSharp.Data.dll

    #r @"packages\FSharp.Data\lib\net40\FSharp.Data.dll"
    open FSharp.Data
    
    let fb =FreebaseData.GetDataContext()
    

    Let's build the grammar

    let santaActorsFilms =
        fb.``Arts and Entertainment``
          .Film
          .``Film characters``
          .IndividualsAZ.S
          .``Santa Claus``
          .``Portrayed in films``
        |> Seq.map (fun c -> c.Actor.Name, c.Film.Name)
        |> Seq.toList
    
    let santaActorsGrammar =
        santaActorsFilms
        |> List.map (fun (actor,film) -> Phrase(actor, film))
        |> Alt
    

    Here is the function to call when an actor is recognized.

    I tried to pass a discriminated union as a value, but even if the API uses an object, the documentation states that it has to be a bool, an int or a string. I used only strings here.

    let recognized text value =
        say (sprintf "True ! %s was Santa in %s" text value)
    

    Here is the function when the speech could not be matched with the grammar.

    It is also possible to get the audio of the text in this case. I decided to ignore it due to time constraints.

    let rejected () = say "No, Not a Santa !"
    

    Now, let's run it !!

    recognize santaActorsGrammar recognized rejected
    

    At this point the speech recognition configuration should appear if it's the first time you use it.

    Once done you should be able to try the quizz !

    If your OS culture is not english, don't hesitate to use a local accent for actor's name !

    Conlusion

    I hope you had fun with this API, and that you'll want to tweak it for your own demo !

    The full code - using FSharp.Formatting - is on my gist

    Happy Christmass !

    namespace System namespace System.Speech namespace System.Speech.Synthesis val synt : SpeechSynthesizer

    Full name: XMas fun.synt Multiple items
    type SpeechSynthesizer =
      new : unit -> SpeechSynthesizer
      member AddLexicon : uri:Uri * mediaType:string -> unit
      member Dispose : unit -> unit
      member GetCurrentlySpokenPrompt : unit -> Prompt
      member GetInstalledVoices : unit -> ReadOnlyCollection<InstalledVoice> + 1 overload
      member Pause : unit -> unit
      member Rate : int with get, set
      member RemoveLexicon : uri:Uri -> unit
      member Resume : unit -> unit
      member SelectVoice : name:string -> unit
      ...

    Full name: System.Speech.Synthesis.SpeechSynthesizer

    --------------------
    SpeechSynthesizer() : unit val say : s:string -> unit

    Full name: XMas fun.say val s : string SpeechSynthesizer.Speak(promptBuilder: PromptBuilder) : unit
    SpeechSynthesizer.Speak(prompt: Prompt) : unit
    SpeechSynthesizer.Speak(textToSpeak: string) : unit Multiple items
    val string : value:'T -> string

    Full name: Microsoft.FSharp.Core.Operators.string

    --------------------
    type string = System.String

    Full name: Microsoft.FSharp.Core.string namespace System.Globalization val english : CultureInfo

    Full name: XMas fun.english Multiple items
    type CultureInfo =
      new : name:string -> CultureInfo + 3 overloads
      member Calendar : Calendar
      member ClearCachedData : unit -> unit
      member Clone : unit -> obj
      member CompareInfo : CompareInfo
      member CultureTypes : CultureTypes
      member DateTimeFormat : DateTimeFormatInfo with get, set
      member DisplayName : string
      member EnglishName : string
      member Equals : value:obj -> bool
      ...

    Full name: System.Globalization.CultureInfo

    --------------------
    CultureInfo(name: string) : unit
    CultureInfo(culture: int) : unit
    CultureInfo(name: string, useUserOverride: bool) : unit
    CultureInfo(culture: int, useUserOverride: bool) : unit CultureInfo.GetCultureInfo(name: string) : CultureInfo
    CultureInfo.GetCultureInfo(culture: int) : CultureInfo
    CultureInfo.GetCultureInfo(name: string, altName: string) : CultureInfo SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender) : unit
    SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender, age: VoiceAge) : unit
    SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender, age: VoiceAge, voiceAlternate: int) : unit
    SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender, age: VoiceAge, voiceAlternate: int, culture: CultureInfo) : unit type VoiceGender =
      | NotSet = 0
      | Male = 1
      | Female = 2
      | Neutral = 3

    Full name: System.Speech.Synthesis.VoiceGender field VoiceGender.NotSet = 0 type VoiceAge =
      | NotSet = 0
      | Child = 10
      | Teen = 15
      | Adult = 30
      | Senior = 65

    Full name: System.Speech.Synthesis.VoiceAge field VoiceAge.NotSet = 0 Multiple items
    module List

    from Microsoft.FSharp.Collections

    --------------------
    type List<'T> =
      | ( [] )
      | ( :: ) of Head: 'T * Tail: 'T list
      interface IEnumerable
      interface IEnumerable<'T>
      member Head : 'T
      member IsEmpty : bool
      member Item : index:int -> 'T with get
      member Length : int
      member Tail : 'T list
      static member Cons : head:'T * tail:'T list -> 'T list
      static member Empty : 'T list

    Full name: Microsoft.FSharp.Collections.List<_> val map : mapping:('T -> 'U) -> list:'T list -> 'U list

    Full name: Microsoft.FSharp.Collections.List.map val n : int val iter : action:('T -> unit) -> list:'T list -> unit

    Full name: Microsoft.FSharp.Collections.List.iter namespace System.Speech.Recognition type Grammar =
      | Phrase of text: string * result: string
      | Lst of Grammar list
      | Alt of Grammar list
      | Repeat of min: int * max: int * Grammar

    Full name: XMas fun.Grammar union case Grammar.Phrase: text: string * result: string -> Grammar union case Grammar.Lst: Grammar list -> Grammar type 'T list = List<'T>

    Full name: Microsoft.FSharp.Collections.list<_> union case Grammar.Alt: Grammar list -> Grammar union case Grammar.Repeat: min: int * max: int * Grammar -> Grammar val min : e1:'T -> e2:'T -> 'T (requires comparison)

    Full name: Microsoft.FSharp.Core.Operators.min Multiple items
    val int : value:'T -> int (requires member op_Explicit)

    Full name: Microsoft.FSharp.Core.Operators.int

    --------------------
    type int = int32

    Full name: Microsoft.FSharp.Core.int

    --------------------
    type int<'Measure> = int

    Full name: Microsoft.FSharp.Core.int<_> val max : e1:'T -> e2:'T -> 'T (requires comparison)

    Full name: Microsoft.FSharp.Core.Operators.max val build : _arg1:Grammar -> GrammarBuilder

    Full name: XMas fun.build val text : string val result : string Multiple items
    type GrammarBuilder =
      new : unit -> GrammarBuilder + 7 overloads
      member Append : phrase:string -> unit + 7 overloads
      member AppendDictation : unit -> unit + 1 overload
      member AppendRuleReference : path:string -> unit + 1 overload
      member AppendWildcard : unit -> unit
      member Culture : CultureInfo with get, set
      member DebugShowPhrases : string
      static member Add : phrase:string * builder:GrammarBuilder -> GrammarBuilder + 4 overloads

    Full name: System.Speech.Recognition.GrammarBuilder

    --------------------
    GrammarBuilder() : unit
    GrammarBuilder(phrase: string) : unit
    GrammarBuilder(alternateChoices: Choices) : unit
    GrammarBuilder(key: SemanticResultKey) : unit
    GrammarBuilder(value: SemanticResultValue) : unit
    GrammarBuilder(phrase: string, subsetMatchingCriteria: SubsetMatchingMode) : unit
    GrammarBuilder(phrase: string, minRepeat: int, maxRepeat: int) : unit
    GrammarBuilder(builder: GrammarBuilder, minRepeat: int, maxRepeat: int) : unit Multiple items
    type SemanticResultValue =
      new : value:obj -> SemanticResultValue + 2 overloads
      member ToGrammarBuilder : unit -> GrammarBuilder

    Full name: System.Speech.Recognition.SemanticResultValue

    --------------------
    SemanticResultValue(value: obj) : unit
    SemanticResultValue(phrase: string, value: obj) : unit
    SemanticResultValue(builder: GrammarBuilder, value: obj) : unit val grammars : Grammar list val builder : GrammarBuilder GrammarBuilder.Append(value: SemanticResultValue) : unit
    GrammarBuilder.Append(key: SemanticResultKey) : unit
    GrammarBuilder.Append(alternateChoices: Choices) : unit
    GrammarBuilder.Append(builder: GrammarBuilder) : unit
    GrammarBuilder.Append(phrase: string) : unit
    GrammarBuilder.Append(phrase: string, subsetMatchingCriteria: SubsetMatchingMode) : unit
    GrammarBuilder.Append(builder: GrammarBuilder, minRepeat: int, maxRepeat: int) : unit
    GrammarBuilder.Append(phrase: string, minRepeat: int, maxRepeat: int) : unit val alternatives : Grammar list val choices : GrammarBuilder [] val toArray : list:'T list -> 'T []

    Full name: Microsoft.FSharp.Collections.List.toArray Multiple items
    type Choices =
      new : unit -> Choices + 2 overloads
      member Add : params phrases:string[] -> unit + 1 overload
      member ToGrammarBuilder : unit -> GrammarBuilder

    Full name: System.Speech.Recognition.Choices

    --------------------
    Choices() : unit
    Choices(params phrases: string []) : unit
    Choices(params alternateChoices: GrammarBuilder []) : unit val min : int val max : int val grammar : Grammar val recog : SpeechRecognizer

    Full name: XMas fun.recog Multiple items
    type SpeechRecognizer =
      new : unit -> SpeechRecognizer
      member AudioFormat : SpeechAudioFormatInfo
      member AudioLevel : int
      member AudioPosition : TimeSpan
      member AudioState : AudioState
      member Dispose : unit -> unit
      member EmulateRecognize : inputText:string -> RecognitionResult + 2 overloads
      member EmulateRecognizeAsync : inputText:string -> unit + 2 overloads
      member Enabled : bool with get, set
      member Grammars : ReadOnlyCollection<Grammar>
      ...

    Full name: System.Speech.Recognition.SpeechRecognizer

    --------------------
    SpeechRecognizer() : unit val recognize : grammar:Grammar -> recognized:(string -> string -> unit) -> rejected:(unit -> unit) -> bool

    Full name: XMas fun.recognize val recognized : (string -> string -> unit) val rejected : (unit -> unit) property GrammarBuilder.Culture: CultureInfo property SpeechRecognizer.RecognizerInfo: RecognizerInfo property RecognizerInfo.Culture: CultureInfo val printfn : format:Printf.TextWriterFormat<'T> -> 'T

    Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn SpeechRecognizer.LoadGrammar(grammar: Grammar) : unit event SpeechRecognizer.SpeechRecognized: IEvent<System.EventHandler<SpeechRecognizedEventArgs>,SpeechRecognizedEventArgs> Multiple items
    module Event

    from Microsoft.FSharp.Control

    --------------------
    type Event<'T> =
      new : unit -> Event<'T>
      member Trigger : arg:'T -> unit
      member Publish : IEvent<'T>

    Full name: Microsoft.FSharp.Control.Event<_>

    --------------------
    type Event<'Delegate,'Args (requires delegate and 'Delegate :> Delegate)> =
      new : unit -> Event<'Delegate,'Args>
      member Trigger : sender:obj * args:'Args -> unit
      member Publish : IEvent<'Delegate,'Args>

    Full name: Microsoft.FSharp.Control.Event<_,_>

    --------------------
    new : unit -> Event<'T>

    --------------------
    new : unit -> Event<'Delegate,'Args> val add : callback:('T -> unit) -> sourceEvent:IEvent<'Del,'T> -> unit (requires delegate and 'Del :> System.Delegate)

    Full name: Microsoft.FSharp.Control.Event.add val e : SpeechRecognizedEventArgs property RecognitionEventArgs.Result: RecognitionResult property RecognizedPhrase.Text: string property RecognizedPhrase.Semantics: SemanticValue property SemanticValue.Value: obj event SpeechRecognizer.SpeechRecognitionRejected: IEvent<System.EventHandler<SpeechRecognitionRejectedEventArgs>,SpeechRecognitionRejectedEventArgs> val e : SpeechRecognitionRejectedEventArgs property SpeechRecognizer.Enabled: bool namespace FSharp namespace FSharp.Data val fb : FreebaseData.ServiceTypes.FreebaseService

    Full name: XMas fun.fb type FreebaseData =
      static member GetDataContext : unit -> FreebaseService
      nested type ServiceTypes

    Full name: FSharp.Data.FreebaseData


    <summary>Typed representation of Freebase data. See http://www.freebase.com for terms and conditions.</summary>
    FreebaseData.GetDataContext() : FreebaseData.ServiceTypes.FreebaseService val santaActorsFilms : (string * string) list

    Full name: XMas fun.santaActorsFilms property FreebaseData.ServiceTypes.Film.Film.Film_characterDataIndividualsAZ.S: FreebaseData.ServiceTypes.Film.Film.Film_characterDataIndividualsAZ.Film_characterDataIndividualsIndexedS


    <summary>An indexing of specific named individuals of type &apos;Film character&apos; in the web data store</summary>
    module Seq

    from Microsoft.FSharp.Collections val map : mapping:('T -> 'U) -> source:seq<'T> -> seq<'U>

    Full name: Microsoft.FSharp.Collections.Seq.map val c : FreebaseData.ServiceTypes.Film.Film.PerformanceData property FreebaseData.ServiceTypes.Film.Film.PerformanceData.Actor: FreebaseData.ServiceTypes.Film.Film.ActorData


    <summary></summary>
    property Runtime.Freebase.IFreebaseObject.Name: string property FreebaseData.ServiceTypes.Film.Film.PerformanceData.Film: FreebaseData.ServiceTypes.Film.Film.FilmData


    <summary></summary>
    val toList : source:seq<'T> -> 'T list

    Full name: Microsoft.FSharp.Collections.Seq.toList val santaActorsGrammar : Grammar

    Full name: XMas fun.santaActorsGrammar val actor : string val film : string val recognized : text:string -> value:string -> unit

    Full name: XMas fun.recognized val value : string val sprintf : format:Printf.StringFormat<'T> -> 'T

    Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.sprintf val rejected : unit -> unit

    Full name: XMas fun.rejected
    Categories: Architecture, Requirements

    Sponsored Post: MemSQL, Campanja, Hypertable, Sprout Social, Scalyr, FoundationDB, AiScaler, Aerospike, AppDynamics, ManageEngine, Site24x7

    Who's Hiring?
    • DevOps Engineer for Wikia. Wikia is the go-to place for fan content that is created entirely by fans! As a Quantcast Top 20 site with over 120 million monthly uniques we are tackling very interesting problems at a scale you won't find at many other places. We embrace a DevOps culture and are looking to expand our team with people that are excited about working with just about every piece of our stack. You'll also partner with our platform team as they break down the monolith and move towards service oriented architecture. Please apply here.

    • Engineer Manager - Platform. At Wikia we're tackling interesting problems at a scale you won't find at many other places. We're a Quantcast Top 20 site with over 120 million monthly uniques. 100% of the content on our 400,000+ communities is user generated. That combination of scale and UGC creates some pretty compelling challenges and on top of that we're working on moving away from a monolithic architecture and actively working on finding the best technologies to best suit each individual piece of our platform. We're currently in search of an experienced Engineer Manager to help drive this process. Please apply here.

    • Campanja is an Internet advertising optimization company born in the cloud and today we are one of the nordics bigger AWS consumers, the time has come for us to the embrace the next generation of cloud infrastructure. We believe in immutable infrastructure, container technology and micro services, we hope to use PaaS when we can get away with it but consume at the IaaS layer when we have to. Please apply here.

    • Performance and Scale EngineerSprout Social, will be like a physical trainer for the Sprout social media management platform: you will evaluate and make improvements to keep our large, diverse tech stack happy, healthy, and, most importantly, fast. You'll work up and down our back-end stack - from our RESTful API through to our myriad data systems and into the Java services and Hadoop clusters that feed them - searching for SPOFs, performance issues, and places where we can shore things up. Apply here.

    • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here.

    • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here.
    Fun and Informative Events
    • Sign Up for New Aerospike Training Courses.  Aerospike now offers two certified training courses; Aerospike for Developers and Aerospike for Administrators & Operators, to help you get the most out of your deployment.  Find a training course near you. http://www.aerospike.com/aerospike-training/
    Cool Products and Services
    • MemSQL provides a distributed in-memory database for high value data. It's designed to handle extreme data ingest and store the data for real-time, streaming and historical analysis using SQL. MemSQL also cost effectively supports both application and ad-hoc queries concurrently across all data. Start a free 30 day trial here: http://www.memsql.com/

    • Aerospike Hits 1M writes per second with 6x Fewer Servers than Cassandra. A new Google Compute Engine benchmark demonstrates how the Aerospike database hit 1 million writes per second with just 50 nodes - compared to Cassandra's 300 nodes. Read the benchmark: http://www.aerospike.com/blog/1m-wps-6x-fewer-servers-than-cassandra/

    • Hypertable Inc. Announces New UpTime Support Subscription Packages. The developer of Hypertable, an open-source, high-performance, massively scalable database, announces three new UpTime support subscription packages – Premium 24/7, Enterprise 24/7 and Basic. 24/7/365 support packages start at just $1995 per month for a ten node cluster -- $49.95 per machine, per month thereafter. For more information visit us on the Web at http://www.hypertable.com/. Connect with Hypertable: @hypertable--Blog.

    • FoundationDB launches SQL Layer. SQL Layer is an ANSI SQL engine that stores its data in the FoundationDB Key-Value Store, inheriting its exceptional properties like automatic fault tolerance and scalability. It is best suited for operational (OLTP) applications with high concurrency. Users of the Key Value store will have free access to SQL Layer. SQL Layer is also open source, you can get started with it on GitHub as well.

    • Diagnose server issues from a single tab. The Scalyr log management tool replaces all your monitoring and analysis services with one, so you can pinpoint and resolve issues without juggling multiple tools and tabs. It's a universal tool for visibility into your production systems. Log aggregation, server metrics, monitoring, alerting, dashboards, and more. Not just “hosted grep” or “hosted graphs,” but enterprise-grade functionality with sane pricing and insane performance. Trusted by in-the-know companies like Codecademy – try it free!

    • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Cloud deployable. Free instant trial, no sign-up required.  http://aiscaler.com/

    • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

    • www.site24x7.com : Monitor End User Experience from a global monitoring network.

    If any of these items interest you there's a full description of each sponsor below. Please click to read more...

    Categories: Architecture