Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

Big, Small, Hot or Cold - Examples of Robust Data Pipelines from Stripe, Tapad, Etsy and Square

This is a guest repost by Pete Soderling, Founder at Hakka Labs, creating a community where software engineers come to grow.

In response to a recent post from MongoHQ entitled “You don’t have big data," I would generally agree with many of the author’s points.

However, regardless of whether you call it big data, small data, hot data or cold data - we are all in a position to admit that *more* data is here to stay - and that’s due to many different factors.

Perhaps primarily, as the article mentions, this is due to the decreasing cost of storage over time. Other factors include access to open APIs, the sheer volume of ever-increasing consumer activity online, as well as a plethora of other incentives that are developing (mostly) behind the scenes as companies “share” data with each other. (You know they do this, right?)

But one of the most important things I’ve learned over the past couple of years is that it’s crucial for forward thinking companies to start to design more robust data pipelines in order to collect, aggregate and process their ever-increasing volumes of data. The main reason for this is to be able to tee up the data in a consistent way for the seemingly-magical quant-like operations that infer relationships between the data that would have otherwise surely gone unnoticed - ingeniously described in the referenced article as correctly “determining the nature of needles from a needle-stack.”

But this raises the question - what are the characteristics of a well-designed data pipeline? Can’t you just throw all your data in Hadoop and call it a day?

As many engineers are discovering - the answer is a resounding "no!" We've rounded up four examples from smart engineers at Stripe, Tapad, Etsy & Square that show aspects of some real-world data pipelines you'll actually see in the wild.

How does Stripe do it?
Categories: Architecture

Done & Retrospectives in Scrum, Software Architecture in Methods & Tools Spring 2014 issue

From the Editor of Methods & Tools - Mon, 03/24/2014 - 16:05
Methods & Tools – the free e-magazine for software developers, testers and project managers – has just published its Spring 2014 issue that discusses how to be bone in Scrum; why, when and how to perform successful retrospectives; Why it is better to chop onions instead of layers for a improved software architecture. Methods & Tools Spring 2014 contains the following articles: * The Principles of Done * Doing Valuable Agile Retrospectives * Chop Onions Instead of Layers in Software Architecture * Ranorex – Automated Testing Tool for Desktop, Web & Mobile Applications 40 pages of ...

Why You Are “Stuck” In Your Career

Making the Complex Simple - John Sonmez - Mon, 03/24/2014 - 16:00

For software developers, there is no guidebook that tells you how to advance from one stage of your career to the next. Many programmers start out quickly in their careers, advancing up the ranks, only to find that they have hit the proverbial glass ceiling which caps their earnings and gives them the choice to […]

The post Why You Are “Stuck” In Your Career appeared first on Simple Programmer.

Categories: Programming

The Problem with Career Ladders

NOOP.NL - Jurgen Appelo - Mon, 03/24/2014 - 11:18
Career Ladders

I once had a small disagreement with the HR manager of an organization where I worked. She showed me the suggested career paths that software developers could have in our organization. To my horror I saw, if I remember well, that Software Testers could become Software Developers (the other way around was not expected), Software Developers might become Project Managers (not the other way around), and Project Managers could become Business Unit Managers (again, not the other way around). And, as could be expected from a hierarchically oriented organization, all the roads ultimately led to the Pope in Rome. Or, in our case, one top management position, which was theoretically possible, but practically unavailable.

The post The Problem with Career Ladders appeared first on NOOP.NL.

Categories: Project Management

SPaMCAST 282 – Ben Linders and Luis Gonçalves on Retrospectives

Software Process and Measurement Cast - Mon, 03/24/2014 - 03:00

Listen to the Software Process and Measurement Cast 282. In the SPaMCAST 282 we feature our interview with Ben Linders and Luis Gonçalves.  We discussed retrospectives and their great new book Getting Value out of Agile Retrospectives - A Toolbox of Retrospective Exercises. Retrospectives power the continuous improvement all projects and organizations need to deliver more value over time.

Luis Gonçalves is an Agile Coach, Co-Author, Speaker and a Blogger. He has been working in the software industry since 2003, being an Agile practitioner since 2007. Luis is the co-author of “Getting Value Out of Agile Retrospectives” and a co-founder of a MeetUp group in Munich called High Performing Teams.

He likes to write and share ideas with the world and this made him passionate blogger. You can follow his blog: http://lmsgoncalves.com. People can find Luis on twitter: @lgoncalves1979

Ben Linders is a Senior Consultant in Agile, Lean, Quality and Process Improvement, based in The Netherlands. Co-author of Getting Value out of Agile Retrospectives. As an advisor, coach and trainer he helps organizations by deploying effective software development and management practices. He focuses on continuous improvement, collaboration and communication, and professional development, to deliver business value to customers. Ben is an active member of several networks on Agile, Lean and Quality, and a frequent speaker and writer. He shares his experience in a bilingual blog (Dutch and English) and as an editor for Agile at InfoQ. You can find him on twitter: @BenLinders.

Get in touch with us anytime or leave a comment here on the blog. Help support the SPaMCAST by reviewing and rating it on iTunes. It helps people find the cast. Like us on Facebook while you’re at it.

Next week we will feature our essay on user stories.  A user story is a brief, simple requirement statement from the user perspective. User stories are narratives describing who is interacting with the application; how they are interacting with the application and the benefit they derive from that interaction.

Upcoming Events

ISMA 9

I will be attending the International Function Point Users Group conference and workshops in Madrid, Spain on March 27th with workshops on March 25th and 26th.

More information

QAIQuest 2014

I will be facilitating a ½ Day tutorial titled Make Integration and Acceptance Testing Truly Agile. The tutorial will wrestle with the flow of testing in Agile projects and will include lots of practical advice and exercises. Remember that Agile testing is not waterfall done quickly. I will also be around for the conference and look forward to meeting and talking with SPaMCAST readers and listeners.  More confernce information   ALSO I HAVE A DISCOUNT CODE…. Email me at spamcastinfo@gmail.com or call 440.668.5717 for the code.

StarEast

I will be speaking at the StarEast Conference May 4th – 9th in Orlando, Florida.  I will be presenting a talk titled, The Impact of Cognitive Biases on Test and Project Teams. Follow the link for more information on StarEast. ALSO I HAVE A DISCOUNT CODE…. Email me at spamcastinfo@gmail.com or call 440.668.5717 for the code.

I look forward to seeing all SPaMCAST readers and listeners at all of these great events!

The Software Process and Measurement Cast has a sponsor.

As many you know I do at least one webinar for the IT Metrics and Productivity Institute (ITMPI) every year. The ITMPI provides a great service to the IT profession. ITMPI's mission is to pull together the expertise and educational efforts of the world's leading IT thought leaders and to create a single online destination where IT practitioners and executives can meet all of their educational and professional development needs. The ITMPI offers a premium membership that gives members unlimited free access to 400 PDU accredited webinar recordings, and waives the PDU processing fees on all live and recorded webinars. The Software Process and Measurement Cast some support if you sign up here. All the revenue our sponsorship generates goes for bandwidth, hosting and new cool equipment to create more and better content for you. Support the SPaMCAST and learn from the ITMPI.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: "This book will prove that software projects should not be a tedious process, neither for you or your team." Support SPaMCAST by buying the book here. Available in English and Chinese.

Categories: Process Management

Join us at Game Developers Conference 2014!

Android Developers Blog - Mon, 03/24/2014 - 00:49

By Greg Hartrell, Google Play Games team

When we’re not guiding a tiny bird across a landscape of pipes on our phones, we’re getting ready for our biggest-ever Developer Day at this year’s Game Developers Conference in San Francisco.

On Tuesday 18 March, all the teams at Google dedicated to gaming will share their insights on the best ways to build games, grow audiences, engage players and make money.

Some of the session highlights include:

  • Growth Hacking with Play Games
  • Making Money on Google Play: Best Practices in Monetization
  • Grow Your Game Revenue with AdMob
  • From Players to Customers: Tracking Revenue with Google Analytics
  • Build Games that Scale in the Cloud
  • From Box2D to Liquid Fun: Just Add Water-like Particles!

And there’s a lot more, so check out the full Google Developer Day schedule on the GDC website, where you can also buy tickets. We hope to see you there, but if you can’t make the trip, don’t worry; all the talks will be livestreamed on YouTube, starting at 10:00AM PDT (5:00PM UTC).

Then from 19-21 March, meet the Google teams in person from AdMob, Analytics, and Cloud at the Google Education Center in the Moscone Center’s South Hall (booth 218), and you could win a Nexus 7.

Join the discussion on

+Android Developers
Categories: Programming

F# Type Providers

Phil Trelford's Array - Mon, 03/24/2014 - 00:14

Type Providers are a mature language feature of F# 3 that provide instant typed access, with code completion, to a wide variety of data sources, without requiring code generation. As a language feature, F# Type Providers are cross platform, meaning they can be accessed on Windows via .Net, or on Mac and Linux using the Mono runtime.

If you’re interested in using Type Providers pop down to Skills Matter this Thursday for a Type Provider Treasure Hunt with Tomas Petricek, or Creating Type Providers Hands On with Michael Newton on May 1st.

Much Text

CSV, JSON and XML Type Providers are all available from the community open source FSharp.Data project.

What’s the weather like in London now?

type Weather = JsonProvider< "weather.json" >
let url = "http://api.openweathermap.org/data/2.5/weather?units=metric&q="
let weather = Weather.Load(url+"London")
weather.Main.Temp

Nuget stats on FSharp.Data from an HTML table?

type NugetStats = HtmlProvider<"https://www.nuget.org/packages/FSharp.Data">
let rawStats = NugetStats().Tables.Table0

Many Database

Early type providers allowed access to SQL databases via a connection string, DBML and EDMX files or Entity framework.

The open source SqlClient provider lets you explore queries, stored procedures, user defined types and functions with intellisense.

Access to MS SQL Server, SQLite, PostgreSQL, Oracle and MySQL with full LINQ support is made possible with Ross McKinlay’s SQLProvider.

Connect to the cloud with Isaac Abraham’s Azure Storage Provider.

Or connect to a community-curated database of well-known people places and things with the Freebase provider:

let fb = FreebaseData.GetDataContext()
let astronomy = fb.Society.Celebrities.``Abused substances``

Very Language

Interop seamlessly with the R statistical programming language with the R Type Provider:

open RProvider.``base``
let v = R.c(1,2,3)

Or Matlab, or Java, or Python.

Or Brainfuck

Such Silly

It all started with Ross’s Choose Your Own Adventure Type Provider, and then there was MineSweeper and Battleships.

Now thanks to James you can enjoy FizzBuzz as a Type Provider too:


Have fun!

Categories: Programming

SPaMCAST 282 – Ben Linders and Luis Gonçalves on Retrospectives

www.spamcast.net

http://www.spamcast.net

Listen to the Software Process and Measurement Cast 282. In the SPaMCAST 282 we feature our interview with Ben Linders and Luis Gonçalves.  We discussed retrospectives and their great new book Getting Value out of Agile Retrospectives – A Toolbox of Retrospective Exercises. Retrospectives power the continuous improvement all projects and organizations need to deliver more value over time.

Luis Gonçalves is an Agile Coach, Co-Author, Speaker and a Blogger. He has been working in the software industry since 2003, being an Agile practitioner since 2007. Luis is the co-author of “Getting Value Out of Agile Retrospectives” and a co-founder of a MeetUp group in Munich called High Performing Teams.

He likes to write and share ideas with the world and this made him passionate blogger. You can follow his blog: http://lmsgoncalves.com. People can find Luis on twitter: @lgoncalves1979

Ben Linders is a Senior Consultant in Agile, Lean, Quality and Process Improvement, based in The Netherlands. Co-author of Getting Value out of Agile Retrospectives. As an advisor, coach and trainer he helps organizations by deploying effective software development and management practices. He focuses on continuous improvement, collaboration and communication, and professional development, to deliver business value to customers. Ben is an active member of several networks on Agile, Lean and Quality, and a frequent speaker and writer. He shares his experience in a bilingual blog (Dutch and English) and as an editor for Agile at InfoQ. You can find him on twitter: @BenLinders.

Get in touch with us anytime or leave a comment here on the blog. Help support the SPaMCAST by reviewing and rating it on iTunes. It helps people find the cast. Like us on Facebook while you’re at it.

Next week we will feature our essay on user stories.  A user story is a brief, simple requirement statement from the user perspective. User stories are narratives describing who is interacting with the application; how they are interacting with the application and the benefit they derive from that interaction.

Upcoming Events

ISMA 9

I will be attending the International Function Point Users Group conference and workshops in Madrid, Spain on March 27th with workshops on March 25th and 26th.

More information

QAIQuest 2014

I will be facilitating a ½ Day tutorial titled Make Integration and Acceptance Testing Truly Agile. The tutorial will wrestle with the flow of testing in Agile projects and will include lots of practical advice and exercises. Remember that Agile testing is not waterfall done quickly. I will also be around for the conference and look forward to meeting and talking with SPaMCAST readers and listeners.  More confernce information   ALSO I HAVE A DISCOUNT CODE…. Email me at spamcastinfo@gmail.com or call 440.668.5717 for the code.

StarEast

I will be speaking at the StarEast Conference May 4th – 9th in Orlando, Florida.  I will be presenting a talk titled, The Impact of Cognitive Biases on Test and Project Teams. Follow the link for more information on StarEast. ALSO I HAVE A DISCOUNT CODE…. Email me at spamcastinfo@gmail.com or call 440.668.5717 for the code.

I look forward to seeing all SPaMCAST readers and listeners at all of these great events!

The Software Process and Measurement Cast has a sponsor.

As many you know I do at least one webinar for the IT Metrics and Productivity Institute (ITMPI) every year. The ITMPI provides a great service to the IT profession. ITMPI’s mission is to pull together the expertise and educational efforts of the world’s leading IT thought leaders and to create a single online destination where IT practitioners and executives can meet all of their educational and professional development needs. The ITMPI offers a premium membership that gives members unlimited free access to 400 PDU accredited webinar recordings, and waives the PDU processing fees on all live and recorded webinars. The Software Process and Measurement Cast some support if you sign up here. All the revenue our sponsorship generates goes for bandwidth, hosting and new cool equipment to create more and better content for you. Support the SPaMCAST and learn from the ITMPI.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.


Categories: Process Management

Functional Programming in Java – Venkat Subramaniam: Book Review

Mark Needham - Sun, 03/23/2014 - 22:18

I picked up Venkat Subramaniam’s ‘Functional Programming in Java: Harnessing the Power of Java 8 Lambda Expressions‘ to learn a little bit more about Java 8 having struggled to find any online tutorials which did that.

A big chunk of the book focuses on lambdas, functional collection parameters and lazy evaluation which will be familiar to users of C#, Clojure, Scala, Haskell, Ruby, Python, F# or libraries like totallylazy and Guava.

Although I was able to race through the book quite quickly it was still interesting to see how Java 8 is going to reduce the amount of code we need to write to do simple operations on collections.

I wrote up my thoughts on lambda expressions instead of auto closeable, using group by on collections and sorting values in collections in previous blog posts.

I noticed a couple of subtle differences in the method names added to collection e.g. skip/limit are there instead of take/drop for grabbing a subset of said collection.

There are also methods such as ‘mapToInt’ and ‘mapToDouble’ where in other languages you’d just have a single ‘map’ and it would handle everything.

Over the last couple of years I’ve used totallylazy on Java projects to deal with collections and while I like the style of code it encourages you end up with a lot of code due to all the anonymous classes you have to create.

In Java 8 lambdas are a first class concept which should make using totallylazy even better.

In a previous blog post I showed how you’d go about sorted a collection of people by age. In Java 8 it would look like this:

List<Person> people = Arrays.asList(new Person("Paul", 24), new Person("Mark", 30), new Person("Will", 28));
people.stream().sorted(comparing(p -> p.getAge())).forEach(System.out::println)

I find the ‘comparing’ function that we have to use a bit unintuitive and this is what we’d have using totallylazy pre Java 8:

Sequence<Person> people = sequence(new Person("Paul", 24), new Person("Mark", 30), new Person("Will", 28));
 
people.sortBy(new Callable1<Person, Integer>() {
    @Override
    public Integer call(Person person) throws Exception {
        return person.getAge();
    }
});

Using Java 8 lambdas the code is much simplified:

Sequence<Person> people = sequence(new Person("Paul", 24), new Person("Mark", 30), new Person("Will", 28));
System.out.println(people.sortBy(Person::getAge));

If we use ‘forEach’ to print out each person individually we end up with the following:

Sequence<Person> people = sequence(new Person("Paul", 24), new Person("Mark", 30), new Person("Will", 28));
people.sortBy(Person::getAge).forEach((Consumer<? super Person>) System.out::println);

The compiler can’t work out whether we want to use the forEach method from totallylazy or from Iterable so we end up having to cast which is a bit nasty.

I haven’t yet tried converting the totallylazy code I’ve written but my thinking is that the real win of Java 8 will be making it easier to use libraries like totallylazy and Guava.

Overall the book describes Java 8′s features very well but if you’ve used any of the languages I mentioned at the top it will all be very familiar – finally Java has caught up with the rest!

Categories: Programming

Quote of the Day

Herding Cats - Glen Alleman - Sun, 03/23/2014 - 21:46

All the project world a non-linear, non-stationary stochastic process, driven by the exchange of cost, schedule, and technical capabilities - act accordinglyTech Cost Sched with MOE MOP TPM

This means:

  • No single point estimate can be credible in the absence of knowing the variance on that estimate and knowing something about the underlying statistical uncertainties that drive the probabilistic processes that produce those variances.
  • You can not asses the value of something, the decision to make or buy, or the decision to move foreword in search of the business benefit, without knowing the cost of that decision. Both the value and the cost need a probabilistic assessment.
  • All decisions must be risk adjusted if they are be be credible. Risk comes from uncertainty. Uncertainty is of two kinds - reducible and irreducible. 
  • Estimating the impact of your decisions is the role of management. Not estimating, or being unwilling to estimate, the cost, duration, and beneficial outcome of your work efforts is called labor.

 

Related articles Probabilistic Programming in Quantitative Finance Statistical Process Control The Basis of All Modern Control Systems Deterministic versus Stochastic Trends in Earned Value Management Data Quote of the Day - Correlation, Causation, and Statistics Models
Categories: Project Management

UK Tech Meets

Phil Trelford's Array - Sun, 03/23/2014 - 19:13

Passionate about programming, interested in meeting other like minded individuals?

A good place to start is a local meetup group.

.Net

Here’s my short list of active .Net/Mono related groups in the UK which feature some C#, F# or VB content (let me know if I’ve missed your group):

Group Members London Unity Usergroup 1074 Windows Phone User Group 850+ (on mailing list) F#unctional Londoners 742 Software East (Cambridge) 673 London .Net User Group 538 Canary Wharf Dot Net User Group 501 Cambridge Developer’s User Group 355 Norfolk Developers (Norwich) 325 Nottingham Programmers 255 Developer South Coast (Southampton) 241 Functional Bridgton 141 Cambridge DDD Nights 122 Brighton ALT.NET 49 Smart Devs (Hereford) 43 Functional South Coast (Southampton) 37 Craft Coders (Worcestershire) 34 Wales & West Windows Applications Group 25 NxtGen (Birmingham)   VBUG (Bristol)   Agile Yorkshire   Sheffield .Net User Group   Gloustershire .Net User Group   Dare Devs (Chesire)   IWDev (Isle of Wight)   Windows Phone (Bristol/Cardiff)   Windows Phone (Manchester)  

 

In the .Net arena, it seems gaming, windows phone, and functional programing are hot right now. Overall there’s a good spread of groups across the regions, with high concentrations in London, Cambridge and the South Coast.

Curiously the London C#/.Net focused user groups appear to be eclipsed by the London Java Community meetup which has close to 4000 members, and PHP London with nearly 3000 members.

Functional

Interest in functional programming has been growing steadily as have related groups in the capitol:

Group Members London Scala Users’ Group 1405 F#unctional Londoners 742 London Clojurians 540 London Haskell 423 London Erlang Uer Group 377

 

On the F# side, the F#unctional Londoners now meets every 2 weeks and has been growing at a rate of around 100 members every 6 months. Progress is not just confined to London, there’s F# user groups popping up across the globe

Data Science

Data Science and Machine Learning are another growth area, and the many machine learning based sessions have been very popular with the F#unctional Londoners.

Group Members Data Science London 2980 London Machine Learning 1009 Women in Data 380

Hackers

Hacker News is something of a phenomena, and hacker groups appear to be among the biggest:

Group Members Hacker News London 5771 Silicon Roudabout 5042 Hacks/Hackers London 2524 DevTank London 926

 

Get Involved

Like pizza, like beer, like learning, there’s a meetup group near you waiting :)

Categories: Programming

Why Are Requirements So Hard To Get Right?

photo (14)IT projects have been around in one form or another since the 1940’s. Looking back in the literature describing the history of IT, the topic of requirements in general and identification of requirements specifically have been top of mind since day one.  There are numerous definitions of ‘requirements,’ however at a high level requirements can be thought of as what the functionality developed by the project should ‘do’. Identifying requirements is difficult because it requires nearly a perfect storm of the correct process, involvement of the correct people for the business problem to be solved (before it is even defined) and an environment that is conducive to making all of the parts work together.  This confluence forms a set of three constraints that are overlaid on flowing time which ensures subtle changes are continually being introduced to all of the variables.  A breakdown of process, people or environment will reduce the effectiveness of the result or render it unusable.  The factors driving the people category are typically the most volatile and a seemingly least controllable of all of the variables within the requirements process.  This essay will focus on the ‘people’ category with subsequent essays focusing on process, environment and suggestions for solutions

People have a major impact on the vagaries of requirements.  All of the strengths and weaknesses that individuals and groups bring to the table will influence the final requirements product. A few of the more intractable contributors to requirements variance are:

  1. Lack of experience
  2. Human nature
  3. Communication
  4. Organizational politics

We’ll discuss the first two today and deal with points 3 and 4 on Monday.

Two types of experience are the most germane to this discussion.  The first is whether participants have knowledge of the problem space from a business perspective. Without that the requirements may not practically address the project needs. Knowledge of and experience in the problem space is critical for effectiveness. One technique that has been developed to mitigate this risk is to ensure access to business knowledge through co-location with a business partner.  This kind of access is a central tenet of the most of the Agile methods. The second category is experience with the requirements process itself. Without experience gathering, recording and managing the requirements process, it will be difficult to ensure information gathered is correct and that the results developed will be apt to be more costly than necessary and less valuable than needed.  Agile methods use coaching to reinforce this knowledge and experience, while other methods use training and processes.  The goal is the same in both cases: efficiency and effectiveness.

Human nature can act as tool to focus or redirect the requirement process.  Watching several requirements gathering systems has lead me to the conclusion that there is a natural tendency for groups to jump to defining the solution before defining the need.  This can lead to a number of communication issues.  Needs provide a locus for grounding the work, which focuses the solution.  It is important to remember in the grand scheme of life needs change before the solution changes, rather than the solution changing the need (even though in exceptional cases this can be true).

There are a number of tactical solutions for all of these issues, however the first step to solving requirements issues that are people-centric begins with recognizing that a problem exists.  One best practice that I would recommend, taking a page out of the Agile handbook, is to use coaches to support the people working on gathering and managing requirements.  The role of coaches is to be the voice of the people focused inward on the team and the work.  A coach observes how work is done, provides support and instruction in a consistent and focused manner when and where it is needed.  This role is different from that of project manager which is externally focused, interacting with outside parties and clearing external roadblocks.  While people are not the only factor driving the quality of requirements, they are a critical factor.  Pay attention to how people are being deployed, provide support and instruction and make darn sure the right people are in the right place at the right time.


Categories: Process Management

Get Up And Code 046: Checking Back With Jessica

Making the Complex Simple - John Sonmez - Sat, 03/22/2014 - 15:00

Jessica Engstrom was back on the podcast this episode talking about the diet and fitness plan Iris and I had set up for her. Stick around till the end, there is a bit of a surprise. Full transcription below: show John:               Hey everyone, welcome back to Get Up and CODE.  I am here with Jessica […]

The post Get Up And Code 046: Checking Back With Jessica appeared first on Simple Programmer.

Categories: Programming

Safe to Fail Experiments Are About Learning

Safe to fail experiments are about education.

Safe to fail experiments are about learning

In IT, ‘safe to fail’ describes activities that are used to generate knowledge. Additional terms that might be used are probes, spikes, R&D and prototypes. The outcome of a safe to fail experiment can be innovation, the information necessary to make decisions or to discover alternatives.  All these scenarios are appropriate for a safe to fail experiment however regardless of intent sometimes safe to fail  experiment are not safe nor are they appropriate.

Safe to fail experiments are appropriate in situations where learning is required to deliver a project. An experiment or a bit of prototyping might be needed to learn whether something is possible or if there is a more effective way to perform a repetitive task. For example, earlier in my career my team converted credit card portfolios from one provider to another. We used a standard process for most of the hard work. We were always looking for a way to make the process more efficient  using mock conversions as experiments. The goal was to find better processes and techniques.  These experiments did not always yield process changes, but they always added to our knowledge of what would and wouldn’t work. Another opportunity for safe to fail experiments is in situations where several possible alternatives exist. You need an experiment to determine which (if any) of the alternatives make sense. A test or an experiment is considered successful if it proves either that an alternative is a good choice OR if the results can be used to cross an alternative off the list. Organizations that judge results that can disqualify an alternative as a failure are apt sort out alternatives in the market place rather than before they go into production because experiments will not be safe. Finally, safe to fail experiments are useful tools in formal decision-making frameworks. For example, frameworks built to comply with the CMMI process area, Decision Analysis and Resolution (DAR), often leverage safe to fail experiments to generate data for formal, structured decision-making.

Safe to fail experiments are not always appropriate.  Many projects exist with a both fairly well understood solution and a hard deadline. In some portion of these projects the solution may not be perfectly optimal or the coolest possible solution, but it is doable in the timeframe. Experimenting to find a more optimal solution when taking the time and effort could cause you to miss the due date is not appropriate. In the credit card example mentioned earlier, we explored new ideas and optimization between projects to avoid experiments that would put the conversion at risk or cause the team to miss more of their weekend than was already necessary. Experiments that imperil the commitments a team has made or inhibits their ability to effectively deliver are not experiments and are safe to fail. Experimenting is also not appropriate where negative results are thought of as a black mark on a career. In this case straying from the tried and true approaches are generally a bad idea. This scenario tends to occur in highly politicized or overly competitive organizations. In this type of scenario if you can’t change the environment, I would suggest experimenting with other job opportunities. A third scenario is more questionable than absolutely inappropriate. In scenarios where there is little potential upside to the experiment experimentation is probably not a good idea. In most organizations, time and attention are always in short supply. Every team needs to judge what it needs to learn and ration resources that are in short supply when there is little or no perceived upside.

Safe to fail experiments are scenarios where an organization or team truly wants to sift through possible alternatives before using them in a project or in production. Safe to fail experimentation is an application of the scientific method (testing hypotheses) to generate knowledge and decisions so that teams and projects deliver more value.


Categories: Process Management

Stuff The Internet Says On Scalability For March 21st, 2014

Hey, it's HighScalability time:


Isaac Newton's College Notebook, such a slacker.
  • Quotable Quotes:
    • Chris Anderson: Petabytes allow us to say: ‘Correlation is enough.’
    • @adron: Back to writing the micro-services for my composite app with regionally distributed, highly available, key value crypto stores of unicorns!
    • DevOps Cafe: when the canary dies you don't buy a stronger canary.
    • Mark Pagel: Creativity, like evolution, is merely a series of thefts.
    • GitHub: Whoever has more capacity wins.
    • The Master: We balance probabilities and choose the most likely. It is the scientific use of the imagination.
    • Jonathan Ive: Steve, and I don’t recognize my friend in much of it. Yes, he had a surgically precise opinion. Yes, it could sting. Yes, he constantly questioned. ‘Is this good enough? Is this right?’ but he was so clever. His ideas were bold and magnificent. They could suck the air from the room. And when the ideas didn’t come, he decided to believe we would eventually make something great. And, oh, the joy of getting there!”
    • @joestump: Turns out the correct amount of RAM is always 16GB more than you have.

  • From a spec of sand does a pearl grow. In this case the oyster is Facebook, the irritant is PHP, and the pearl is Hack, a new improved version of PHP developed by Facebook. Hack now runs all of Facebook's front-end code. Lots of great comments on Hacker News and on Reddit with much lang on PHP hate. Realize Facebook is not trying to build the next perfect language. Historically they coded their web tier in PHP so they have a lot of code working in production. It would be insane to throw that all away. Over the years they've put a lot of work into making PHP more efficient. Hack continues that work by making PHP a better language, coupling PHP's convenient interpreted nature with static typing and many other features commonly found in modern programming languages. If PHP bothers you, Hack is not for you, but that's OK.

  • Dish Jump-Starts the Hopper With 8 Tuners and PS4 Support. Ultimate in local caching. Record all the things and then watch what you want later. Streaming access sucks. More local storage!

  • Cassandra Hits One Million Writes Per Second on Google Compute Engine on 300 VMs with 300 1TB Persistent Disk volumes, 15,000 clients, median latency of 10.3 ms and 95% completing under 23 ms. It took 1 hour and 10 minutes at a cost of $330 USD. All benchmark caveats apply. They wrote small records of 170 bytes. What happens when you write a large range of data sizes? When you are reading? Are you running queries? Are you backing up? Are indexes being updated? Is there contention? Etc, etc. Still, it's impressive.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...

Categories: Architecture

The Importance of Community: You Can’t Do It Alone

Making the Complex Simple - John Sonmez - Fri, 03/21/2014 - 16:00

I used to think I was a real hot shot. I used to think I could conquer the entire world all by myself with just an IDE and a mechanical keyboard. I couldn’t have been further from the truth. (Oh, and yes that really is a picture of me—don’t ask!) All of us are stronger […]

The post The Importance of Community: You Can’t Do It Alone appeared first on Simple Programmer.

Categories: Programming

USA and Poland, Where Do I Go?

NOOP.NL - Jurgen Appelo - Fri, 03/21/2014 - 15:17
United States

I have selected the USA and Poland as the next countries from the backlog for my global book tour. This means I am now waiting for your input on the best cities to travel to, for my brand new and exclusive one-day workshop.

Finland, Belgium, The Netherlands, Sweden, Germany, UK, and Bulgaria are already scheduled, and I’m happy to say that registrations are steadily trickling into my mailbox. But I can only schedule USA and Poland when you tell me which cities and venues to try!

The post USA and Poland, Where Do I Go? appeared first on NOOP.NL.

Categories: Project Management

Foq 1.6

Phil Trelford's Array - Fri, 03/21/2014 - 14:31

Foq is a mature open source .Net mocking library written in F#. It’s raison d'être is to enable F# as a unit testing language, for F#, C# or VB.Net code bases.

Foq is part a rich set of testing tools available to F# developers. Use Foq when unit testing in combination with staple .Net testing frameworks like NUnit and xUnit through to F# specific fluent assertion libraries like FsUnit and Unquote. Combine Foq with AutoFixture to simplify test setup even further.

Wow, Autofixture.AutoFoq + AutoFixture.xUnit + F# is just a mindblowing superfecta symphony. Thanks @xunit @ploeh @ptrelford @nikosbaxevanis

— Ruben Bartelink (@rbartelink) November 20, 2013

The name Foq is derived from Moq, with Foq providing a familiar fluent interface:

// Moq from C#
var mock = new Mock<ILoveThisFramework>();
mock.Setup(framework => framework.DownloadExists("2.0.0.0"))
   .Returns(true);
ILoveThisFramework lovable = mock.Object;

// Foq from F#
let lovable = 
   Mock<ILoveThisFramework>()
      .Setup(fun framework -> <@ framework.DownloadExists("2.0.0.0") @>)
      .Returns(true)
      .Create()

The main difference in the setups above is that Moq uses mutation where as Foq is thread safe, favouring method chaining to build an immutable representation of the type.

Foq takes advantage of F#’s code quotations and type inference for shorter setups:

// Mock succinctly with Foq
let lovable =
   Mock.With(fun (framework:ILoveThisFramework) ->
   <@ framework.DownloadExists("2.0.0.0") --> true @>)

Getting Started

Get the latest version of Foq from Nuget or simply include Foq.fs in your test project. An F# script packed with examples is included when you install Foq via Nuget.

Then take a peek at some of the many Foq testing examples on CodePlex:

    New features

Back in April 2013, Foq’s API stabilized at version 0.9, since then new features and bug fixes have been requested via CodePlex, StackOverflow and Twitter.

Many of the requests have come from people testing C# codebases from F# and requiring extensive support for:

Testing C# Code

Foq can be used to mock both interfaces and abstract classes. For abstract classes arguments can be passed to the base constructor when the mock is created:

let mock = Mock<AbstractBaseClassWithConstructorArgs>().Create(7, "seven")
Assert.AreEqual("seven", mock.Name)

In F# multiple return values are easily specified using tuples, in C# out parameters are required. To setup C# methods with out parameters using Foq, simply specify the out arguments as tuples as you would normally from F#:

// Calling a method with out arguments from F#
let (success, value) = dict.TryGetValue("Fred")
// Setting up a method with out arguments using Foq
Mock<IDictionary<string,int>>()
   .SetupByName("TryGetValue").Returns((true, 1))
   .Create()

Testing Events

In F# events are first class, meaning they can be created like objects and passed to functions. To setup an event on an interface:

// Arrange
let event = Event<_,_>() 
let instance = 
      Mock<INotifyPropertyChanged>() 
         .SetupEvent(fun x -> <@ x.PropertyChanged @>).Publishes(event.Publish) 
         .Create() 
// Act 
event.Trigger(instance, PropertyChangedEventArgs("Name"))

Non-standard events can be setup using the SetupEventByName method.

Testing F# Code

Foq provides helpers for quickly mocking a single member be it a property:

Mock<MyAbstractBaseClass>.Property(fun x -> <@ x.MyProperty @>).Returns(1)

or a method (note that arguments do not need to be specified):

Mock<GreetingTime>
    .Method(fun gt -> <@ gt.GetGreeting @>)
    .Returns("Good Night")

Foq can now mock F# specific members with curried arguments e.g.

Mock<ICurry>()
    .Setup(fun curry -> <@ curry.Fun (any()) (any()) @>)
    .Returns(true)
    .Create()

Finally Mock.With now supports return values via functions:

Mock<IFoo>.With(fun foo -> 
    <@ 
        foo.Bar(1) --> 1
        foo.Bar(any()) ---> fun () -> 2
    @>)

What next?

Try F# as a testing language for your .Net code bases and use Foq to mock F# and C# types with ease. Then try web testing with canopy, automated acceptance testing using NaturalSpec, TickSpec or SpecFlow or random testing via FsCheck.

Categories: Programming

Quote of the Day

Herding Cats - Glen Alleman - Fri, 03/21/2014 - 12:56

Once we've identified the problem, we've gone 80% of the way toward fixing it - Capture Manager, in our morning proposal stand up.

 

Categories: Project Management

The Anatomy of a User Story

Software Requirements Blog - Seilevel.com - Fri, 03/21/2014 - 12:36
User stories, a requirements format utilized in Agile software development, are typically formatted with the following template: As a (type of user), I want (some goal), so that (some reason). This template can be modified to suit a project’s need. In this post, we’ll show how user stories themselves can be broken down into tables […]

The Anatomy of a User Story is a post from: http://requirements.seilevel.com/blog

Categories: Requirements