Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

New D-Series of Azure VMs with 60% Faster CPUs, More Memory and Local SSD Disks

ScottGu's Blog - Scott Guthrie - 4 hours 51 min ago

Today I’m excited to announce that we just released a new set of VM sizes for Microsoft Azure. These VM sizes are now available to be used immediately by every Azure customer.

The new D-Series of VMs can be used with both Azure Virtual Machines and Azure Cloud Services.  In addition to offering faster vCPUs (approximately 60% faster than our A series) and more memory (up to 112 GB), the new VM sizes also all have a local SSD disk (up to 800 GB) to enable much faster IO reads and writes.

The new VM sizes available today include the following:

General Purpose D-Series VMs

Name vCores Memory (GB) Local SSD Disk (GB) Standard_D1 1 3.5 50 Standard_D2 2 7 100 Standard_D3 4 14 200 Standard_D4 8 28 400

 

High Memory D-Series VMs

Name vCores Memory (GB) Local SSD Disk (GB) Standard_D11 2 14 100 Standard_D12 4 28 200 Standard_D13 8 56 400 Standard_D14 16 112 800

For pricing information, please see Virtual Machine Pricing Details.

Local SSD Disk and SQL Server Buffer Pool Extensions

A temporary drive on the VMs (D:\ on Windows, /mnt or /mnt/resource on Linux) is mapped to the local SSDs exposed on the D-Service VMs, and provides a really good option for replicated storage workloads, like MongoDB, or for significantly increasing the performance of SQL Server 2014 by enabling its unique Buffer Pool Extensions (BPE) feature.

SQL Server 2014’s Buffer Pool Extensions allows you to extend the SQL Engine Buffer Pool with the memory of local SSD disks to significantly improve the performance of SQL workloads. The Buffer Pool is a global memory resource used to cache data pages for much faster read operations.  Without any code changes in your application, you can enable the buffer pool support with the SSDs of the D-Series VMs using a simple T-SQL query with just four lines:

ALTER SERVER CONFIGURATION
SET BUFFER POOL EXTENSION ON
SIZE = <size> [ KB | MB | GB ]
FILENAME = 'D:\SSDCACHE\EXAMPLE.BPE'

No code changes are required in your application, and all write operations will continue to be durably persisted in VM drives persisted in Azure Storage. More details on configuring and using BPE can be found here.

Start Using the D-Series VMs Today

You can start using the new D-Series VM sizes immediately.  They can be easily created and used via both the current Azure Management Portal as well as Preview Portal, as well as from the Azure management command-line/scripts/APIs.

To learn more about the D-Series please read this post which has even more details about them, as well as check out the Azure documentation center.

Hope this helps,

Scott

Categories: Architecture, Programming

How Facebook Makes Mobile Work at Scale for All Phones, on All Screens, on All Networks

When you find your mobile application that ran fine in the US is slow in other countries, how do you fix it? That’s a problem Facebook talks about in a couple of enlightening videos from the @scale conference. Since mobile is eating the world, this is the sort of thing you need to consider with your own apps.

In the US we may complain about our mobile networks, but that’s more #firstworldproblems talk than reality. Mobile networks in other countries can be much slower and cost a lot more. This is the conclusion from Chris Marra, Project Manager at Facebook, in a really interesting talk titled Developing Android Apps for Emerging Market.

Facebook found in the US there’s 70.6% 3G penetration with 280ms average latency. In India there’s 6.9% 3G penetration with 500ms latency. In Brazil there’s 38.6% 3G penetration with more than 850ms average latency.

Chris also talked about Facebook’s comprehensive research on who uses Facebook and what kind of phones they use. In summary they found not everyone is on a fast phone, not everyone has a large screen, and not everyone is on a fast network.

It turns out the typical phone used by Facebook users is from circa 2011, dual core, with less than 1GB of RAM. By designing for a high end phone Facebook found all their low end users, which is the typical user, had poor user experiences.

For the slow phone problem Facebook created a separate application that used lighter weight animations and other strategies to work on lower end phones. For the small screen problem Facebook designers made sure applications were functional at different screen sizes.

Facebook has moved to a product organization. A single vertical group is responsible for producing a particular product rather than having, for example, an Android team try to create all Android products. There’s also a horizontally focussed Android team trying to figure out best practices for Android, delving deep into the details of what makes a platform tick.

Each team is responsible for the end-to-end performance and reliability for their product. There are also core teams looking at and analyzing general performance problems and helping where needed to improve performance.

Both core teams and product teams are needed. The core team is really good at instrumentation and identifying problems and working with product teams to fix them. For mobile it’s important that each team owns their full product end-to-end. Owning core engagement metrics, core reliability, and core performance metrics including daily usage, cold start times, and reliability, while also knowing how to fix problems. 

To solve the slow network problem there’s a whole other talk. This time the talk is given by Andrew Rogers, Engineering Manager at Facebook, and it’s titled Tuning Facebook for Constrained Networks. Andrew talks about three methods to help deal with network problems: Image Download Sizes, Network Quality Detection, Prefetching Content.

Overall, please note the immense effort that is required to operate at Facebook scale. Not only do you have different phones like Android and iOS, you have different segments within each type of phone you must code and design for. This is crazy hard to do.

Reducing Image Sizes -  WebP saved over 30% JPEG, 80% over PNG
Categories: Architecture

Xebia KnowledgeCast Episode 4: Scrum Day Europe 2013, OpenSpace Knowledge Exchange, and Fun With Stickies!

Xebia Blog - 7 hours 25 min ago

xebia_xkc_podcast
The Xebia KnowledgeCast is a bi-weekly podcast about software architecture, software development, lean/agile, continuous delivery, and big data. Also, we'll have some fun with stickies!

In this fourth episode, we share some impressions of Scrum Day Europe 2013 and Xebia's OpenSpace Knowledge Exchange. And of course, Serge Beaumont will have Fun With Stickies! First, we interview Frank Bakker and Evelien Roos at Scrum Day Europe 2013. Then, Adriaan de Jonge and Jeroen Leenarts talk about continuous delivery and iOS development at the OpenSpace XKE. And in between, Serge Beaumont has Fun With Stickies!

Frank Bakker and Evelien Roos give their first impressions of the Keynotes at Scrum Day Europe 2013. Yes, that was last year, I know. New, more current interviews are coming soon. In fact, this is the last episode in which I use interviews that were recorded last year.

In this episode's Fun With Stickies Serge Beaumont talks about hypothesis stories. Using those, ensures you keep your Agile really agile. A very relevant topic, in my opinion, and it jells nicely with my missing line of the Agile Manifesto: Experimentation over implementation!

Adriaan de Jonge explains how automation in general, and test automation in particular, is useful for continuous delivery. He warns we should focus on the process and customer interaction, not the tool(s). That's right before I can't help myself and ask him which tool to use.

Jeroen Leenarts talks about iOS development. Listening to the interview, which was recorded a year ago, it's amazing to realize that, with the exception of iOS8 having come out in the mean time, all of Jeroen's comments are as relevant today as they were last year. How's that for a world class developer!

Want to subscribe to the Xebia KnowledgeCast? Subscribe via iTunes, or use our direct rss feed.

Your feedback is appreciated. Please leave your comments in the shownotes. Better yet, use the Auphonic recording app to send in a voicemessage as an AIFF, WAV, or FLAC file so we can put you ON the show!

Credits

Project Driven Organizations

Herding Cats - Glen Alleman - 7 hours 52 min ago

The notion that projects are somehow no longer needed fails to address how to replace the processes, governance, and stewardship of a business's assets while producing the needed value from those expenditures.  Here's a framework for management a firms funds in producing value in exchange of those funds.

Lifecycle from Glen Alleman   This governance process can be guided by Project governance from Glen Alleman Related articles Agile as a Systems Engineering Paradigm Performance-Based Project Management(sm) Released Capabilities Based Planning and Development Why Is It Hard To Manage Projects?
Categories: Project Management

The Caffeinated Coder: Is Caffeine Good or Bad?

For a long time I’ve wondered about the benefits or detriments of caffeine. I’ve always been one of those coffee drinkers who didn’t have to have coffee, but drank it when it was available. I’ve never really noticed how caffeine affected me, because I never really paid that much attention. But, I’ve always been curious, […]

The post The Caffeinated Coder: Is Caffeine Good or Bad? appeared first on Simple Programmer.

Categories: Programming

The Results of My First OKRs (Running)

NOOP.NL - Jurgen Appelo - 12 hours 26 min ago
2014-09-20 14.52.07

A popular topic in the new one-day Management 3.0 workshop is the OKRs system for performance measurement. (See Google’s YouTube video here.) Instead of explaining what OKRs are, I will just share with you the result of my first iteration. If you read this, you will get the general idea of how the OKRs system works.

The post The Results of My First OKRs (Running) appeared first on NOOP.NL.

Categories: Project Management

SPaMCAST 308 – Michael West, Return on Process

www.spamcast.net

http://www.spamcast.net

Listen to Software Process and Measurement Cast 308 here!

Software Process and Measurement Cast number 308 features our interview with Michael West discussing his book Return on Process.  Process improvement can have a dramatic impact to an organization’s bottom line BUT ONLY with careful thought and planning.  Michael West explains that process improvements with real impact are rarely an accident.

Michael’s bio . . .

Michael West is a life-long practitioner and student of process improvement. He is the co-founder of Natural Systems Process Improvement (Natural SPI), a consultancy specializing in designing, developing, and deploying process systems that enable measurable business performance improvement gains. Mr. West’s process insights and innovations have helped many organizations in various sectors of the economy achieve real process and performance improvement. His process consulting clients include ATK, Autodesk, AVL, BAE, BB&T, Crane Aerospace, DCS, Deloitte, Sandia National Labs, Reliability First, and the US Navy. Mr. West frequently presents and speaks at industry conferences, and is the author of Real Process Improvement Using the CMMI (CRC Press, 2004) and Return On Process (ROP): Getting Real Performance Results from Process Improvement (CRC Press, 2013).

Contact Michael at:
Web: http://www.naturalspi.com/
Email: michael@naturalspi.com
Twitter: @ItsTheProcess

Did you like the interview?  Buy Michael’s books
Return On Process (ROP): Getting Real Performance Results from Process Improvement
Real Process Improvement Using the CMMI

Next

SPaMCAST 309 features our essay on Agile user acceptance testing. Agile user acceptance testing (AUAT) confirms that the output of a project meets the business’ needs and requirements. The concept of acceptance testing early and often is almost inarguable, whether you are using Agile or any other method. AUAT generates early customer feedback, which increases customer satisfaction and reduces the potential for delivering defects. The problem is that implementing an effective and efficient AUAT isn’t always easy.

 

Upcoming Events

DCG Webinars:

Agile Risk Management – It Is Still Important! October 24, 2014 11:230 EDT Has the adoption of Agile techniques magically erased risk from software projects? Or, have we just changed how we recognize and manage risk?  Or, more frighteningly, by changing the project environment through adopting Agile techniques, have we tricked ourselves into thinking that risk has been abolished?

 

Upcoming Conferences:

I will be presenting at the International Conference on Software Quality and Test Management in San Diego, CA on October 1.  I have a great discount code!!!! Contact me if you are interested.

I will be presenting at the North East Quality Council 60th Conference October 21st and 22nd in Springfield, MA.

More on all of these great events in the near future! I look forward to seeing all SPaMCAST readers and listeners that attend these great events!

The Software Process and Measurement Cast has a sponsor.

As many you know I do at least one webinar for the IT Metrics and Productivity Institute (ITMPI) every year. The ITMPI provides a great service to the IT profession. ITMPI’s mission is to pull together the expertise and educational efforts of the world’s leading IT thought leaders and to create a single online destination where IT practitioners and executives can meet all of their educational and professional development needs. The ITMPI offers a premium membership that gives members unlimited free access to 400 PDU accredited webinar recordings, and waives the PDU processing fees on all live and recorded webinars. The Software Process and Measurement Cast some support if you sign up here. All the revenue our sponsorship generates goes for bandwidth, hosting and new cool equipment to create more and better content for you. Support the SPaMCAST and learn from the ITMPI.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.


Categories: Process Management

SPaMCAST 308 – Michael West, Return on Process

Software Process and Measurement Cast - Sun, 09/21/2014 - 22:00

Software Process and Measurement Cast number 308 features our interview with Michael West discussing his book Return on Process.  Process improvement can have a dramatic impact to an organization's bottom line BUT ONLY with careful thought and planning.  Michael West explains that process improvements with real impact are rarely an accident.

Michael’s bio . . .

Michael West is a life-long practitioner and student of process improvement. He is the co-founder of Natural Systems Process Improvement (Natural SPI), a consultancy specializing in designing, developing, and deploying process systems that enable measurable business performance improvement gains. Mr. West’s process insights and innovations have helped many organizations in various sectors of the economy achieve real process and performance improvement. His process consulting clients include ATK, Autodesk, AVL, BAE, BB&T, Crane Aerospace, DCS, Deloitte, Sandia National Labs, Reliability First, and the US Navy. Mr. West frequently presents and speaks at industry conferences, and is the author of Real Process Improvement Using the CMMI (CRC Press, 2004) and Return On Process (ROP): Getting Real Performance Results from Process Improvement (CRC Press, 2013).

Contact Michael at:
Web: http://www.naturalspi.com/
Email: michael@naturalspi.com
Twitter: @ItsTheProcess

Did you like the interview?  Buy Michael's books
Return On Process (ROP): Getting Real Performance Results from Process Improvement
Real Process Improvement Using the CMMI

Next

SPaMCAST 309 features our essay on Agile user acceptance testing. Agile user acceptance testing (AUAT) confirms that the output of a project meets the business’ needs and requirements. The concept of acceptance testing early and often is almost inarguable, whether you are using Agile or any other method. AUAT generates early customer feedback, which increases customer satisfaction and reduces the potential for delivering defects. The problem is that implementing an effective and efficient AUAT isn’t always easy.

 

Upcoming Events

DCG Webinars:

Agile Risk Management – It Is Still Important! October 24, 2014 11:230 EDT Has the adoption of Agile techniques magically erased risk from software projects? Or, have we just changed how we recognize and manage risk?  Or, more frighteningly, by changing the project environment through adopting Agile techniques, have we tricked ourselves into thinking that risk has been abolished?

 

Upcoming Conferences:

I will be presenting at the International Conference on Software Quality and Test Management in San Diego, CA on October 1.  I have a great discount code!!!! Contact me if you are interested.

I will be presenting at the North East Quality Council 60th Conference October 21st and 22nd in Springfield, MA.

More on all of these great events in the near future! I look forward to seeing all SPaMCAST readers and listeners that attend these great events!

The Software Process and Measurement Cast has a sponsor.

As many you know I do at least one webinar for the IT Metrics and Productivity Institute (ITMPI) every year. The ITMPI provides a great service to the IT profession. ITMPI’s mission is to pull together the expertise and educational efforts of the world’s leading IT thought leaders and to create a single online destination where IT practitioners and executives can meet all of their educational and professional development needs. The ITMPI offers a premium membership that gives members unlimited free access to 400 PDU accredited webinar recordings, and waives the PDU processing fees on all live and recorded webinars. The Software Process and Measurement Cast some support if you sign up here. All the revenue our sponsorship generates goes for bandwidth, hosting and new cool equipment to create more and better content for you. Support the SPaMCAST and learn from the ITMPI.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.

Categories: Process Management

F#unctional Londoners 2014

Phil Trelford's Array - Sun, 09/21/2014 - 21:40

2014 has been another crazy year for the F#unctional Londoners meetup with over 20 sessions already. Thanks to our hosts Skills Matter we’ve been able to hold a meetup roughly once every 2 weeks.

Here’s a run down of the year so far and what’s coming up.

January

Ross kicked off the year with a deep dive to his LINQ enabled erasing SQL Type Provider.

Following on, in May, Ross left the sunny shores of Southend to tour the east coast with the talk covering NYC, Washington DC and Nashville along the way.

sql-provider

First seen at DunDDD in Dundee, Anthony’s excellent talk went on to be featured at CodeMesh London too.

With F# built-in to Xamarin Studio you can easily target iOS, Android and Mac.

February

Tomas returned to London to talk about his work on Deedle while at Blue Mountain Capital in New York.

As a follow on from the talk Tomas ran a hands on session using Deedle to explore world climate, the titanic, stock market trends and finally US debt.

March

There was a huge turnout for Scott’s hugely informative and at times somewhat amusing talk first seen at NDC London.

set phasers to null

Eirik Tsarpalis and Jan Dzik, from Nessos, presented their work on MBrace a programming model and cluster infrastructure for effectively defining and executing large scale computation in the cloud.

In this hands on treasure hunt session, Tomas presented a series of data extraction tasks using type providers to find words to build a sentence.

April

Rob Lyndon introduced Deep Belief Networks and his GPU based implementation in Vulpes. This talk was repeated last week at the prestigious Strangeloop conference in St Louis!

May

Michael travelled up from Brighton for a hands on session on building type providers. Type Providers are a hot topic in the London group with a number of popular type providers produced by members including FSharp.Data, SQLProvider and Azure Storage.

Mixing biology and physics to understand stem cells and cancer (video)

Ben Hall from Microsoft Research Cambridge gave a fascinating talk about his work with a hybrid simulator in F# to explore how stem cells grow (and some worms!).

Stephen Channell gave a repeat of his excellent talk featured at FP Days and the F# in Finance conference on liquidity risk.

Ian was in town to run a session at the Progressive .Net Tutorials and gave a repeat of his excellent talk from DDD North.

June

F#unctional Londoners regular Isaac, aka the Cockney Coder, talked about his professional work with Azure including his Azure Storage type provider.

In this hands on session we used the material from Mathias Brandewinder’s session in San Francisco to have some fun drawing fractal trees.

In this session Gabriele Cocco talked about his work on FSCL, an F# to OpenCL compiler.

July

Borrowing material from Mathias again, we built a 2048 bot using the open source web testing library Canopy.



Grant popped down from Leeds to run a fun code golf session where the aim was to complete a task with the least number of characters.

August

Phil Nash talked about how he was using F# scripting at work along side his some of his C++ projects.

In this hands on session we looked at the popular parser combinator library FParsec, building a mini-Logo parser and interpreter.

September

James popped down from Edinburgh to talk about his work with Philip Wadler on the open source project FSharp.Linq.ComposableQuery.

Goswin Rothenthal talked about his work using FSharp scripting in the design of the Abu Dhabi Louvre building:

Coming up this Wednesday we have Evelina talking about some of her data science work at Cambridge.

November

On November 6-7th the Progressive F# Tutorials make a return with expert speakers including Don Syme, Tomas Petricek, Mark Seemann, Andrea Magnorsky, Michael Newton, Jérémie Chassaing, Mathias Brandewinder, Scott Wlaschin and Robert Pickering.

ProgFSharp2014Don’t miss the special offer that runs up to the end of Evelina’s talk giving a 20% discount to members, brining the price down to a barmy 200GBP, use code F#UNCTIONAL-20.

Categories: Programming

McKinsey on Unleashing the Value of Big Data Analytics

Big Data Analytics and Insights are changing the game, as more businesses introduce automated systems to support human judgment.

Add to this, advanced visualizations of Big Data, and throw in some power tools for motivated users and you have a powerful way to empower the front-line to better analyze, predict, and serve their customers.

McKinsey shares a framework and their insights on how advanced analytics can create and unleash new business value from Big Data, in their article:
Unleashing the value of advanced analytics in insurance

Creating World-Class Capabilities

The exciting part is how you can create a new world-class capability, as you bake Big Data Analytics and Insights into your business.

Via Unleashing the value of advanced analytics in insurance:

“Weaving analytics into the fabric of an organization is a journey. Every organization will progress at its own pace, from fragmented beginnings to emerging influence to world-class corporate capability.”

5-Part Framework for Unleashing the Value of Big Data Analytics

McKinsey's transformation involves five components.  The five components include the source of business value, the data ecosystem, modeling the insights, workflow integration, and adoption.

Via Unleashing the value of advanced analytics in insurance:

1. The source of business value Every analytics project should start by identifying the business value that can lead to revenue growth and increased profitability (for example, selecting customers, controlling operating expenses, lowering risk, or improving pricing). 2. The data ecosystem It is not enough for analytics teams to be “builders” of models. These advanced-analytics experts also need to be “architects” and “general contractors” who can quickly assess what resources are available inside and outside the company. 3. Modeling insights Building a robust predictive model has many layers: identifying and clarifying the business problem and source of value, creatively incorporating the business insights of everyone with an informed opinion about the problem and the outcome, reducing the complexity of the solution path, and validating the model with data. 4. Transformation: Work-flow integration The goal is always to design the integration of new decision-support tools to be as simple and user friendly as possible. The way analytics are deployed depends on how the work is done. A key issue is to determine the appropriate level of automation. A high-volume, low-value decision process lends itself to automation. 5. Transformation: Adoption Successful adoption requires employees to accept and trust the tools, understand how they work, and use them consistently. That is why managing the adoption phase well is critical to achieving optimal analytics impact. All the right steps can be made to this point, but if frontline decision makers do not use the analytics the way they are intended to be used, the value to the business evaporates.

Big Data Analytics and Insights is a hot trend for good reason.  If you saw the movie Moneyball you know why.

Businesses are using analytics to identify their most profitable customers and offer them the right price, accelerate product innovation, optimize supply chains, and identify the true drivers of financial performance.

In the book, Competing on Analytics: The New Science of Winning, Thomas H. Davenport and Jeanne G. Harris share examples of how organizations like Amazon, Barclay’s, Capital One, Harrah’s, Procter & Gamble, Wachovia, and the Boston Red Sox, are using the power of Big Data Analytics and Insights to achieve new levels of performance and compete in the digital economy.

You can read it pretty quickly to get a good sense of how analytics can be used to change the business and the more you expose yourself to the patterns, the more you can apply analytics to your work and life.

You Might Also Like

10 High-Value Activities in the Enterprise

Cloud Changes the Game from Deployment to Adoption

Management Innovation is at the Top of the Innovation Stack

Categories: Architecture, Programming

Hands-on Test Automation Tools session wrap up - Part1

Xebia Blog - Sun, 09/21/2014 - 15:57

Last week we had our first Hands-on Test Automation sessions.
Developers and Testers were challenged to show and tell their experiences in Test Automation.
That resulted in lots of in depth discussions and hands-on Test Automation Tool shoot-outs.

In this blogpost we'll share the outcome of the different sessions, like the famous Cucumber vs. FitNesse debat.

Stay tuned for upcoming updates!

Test Automation Frameworks

The following Test Automation Frameworks were demoed and discussed

1. FitNesse

FitNesse is a test management and execution tool.
You'll have to write/use fixture code if you want to use Selenium / WebDriver, webservices and databases in your tests.

Pros and Cons
You can have a good drill down in test results.
You can make use of scenario's and scenario libraries to make test automation steps reusable.
But refactoring is hard when scenario's are used extensively since there is no IDE support (yet)

2. Cucumber

Cucumber is a specification tool which describes how software should behave.
You'll have to write/use step definitions if you want to use Selenium / WebDriver, webservices and databases in your tests.

Pros and Cons
Cucumber forces you to write specifications / tests with scenarios (Behaviour in human readable language).
You can drill down into test results, but you'll need reporting libraries like Cucumber Reporting
We recommend using IntelliJ IDEA with the Cucumber plugin since it supports Cucumber seamlessly.
Refactoring becomes less problematic since you're using a IDE.

3. Selenium / WebDriver IDE

Selenium / WebDriver automates human interactions with web browser.
With the Selenium IDE you can record and play your tests in Firefox

Pros and Cons
It can get you started very quickly. You can record and play your tests scripts without writing any code.
Reusability of test automation code is not possible. You'll need to export it into an IDE to introduce reusability.

Must haves in Test Automation Tools

During the parallel sessions we've collected the following must haves in test automation tools.

Testers and Developers becoming best friends

When developers do not feel comfortable with the test automation tool, testers will try to fill the gap all by themselves. Most of the time these efforts result in hard to maintain test automation code. At some point in time test automation will become a bottleneck in Continuous Delivery. When picking a test automation tool consider each other's needs and pick a tool together. Feeling comfortable in writing and maintaining test automation code is really important to make test automation beneficial.

Separating What from How

Tools like FitNesse and Cucumber were designed to separate the What from the How in test automation. When combining both in those tools, you'll end up getting lost in details and you'll lose focus of what you are testing.
Use tools like FitNesse and Cucumber to describe What you are testing and put all details about How you are testing in test automation code (like fixture code and step definitions)

Other interesting tools
  • Thucydides: Reporting tests and results (including functional coverage)
  • Vagrant: Provisioning System Under Test instances
  • Liquibase: Treating database schema changes as 'code'

Stay tuned for upcoming updates!

 

Incremental Commitment Spiral Model

Herding Cats - Glen Alleman - Sun, 09/21/2014 - 15:00

Incremental CommitmentUnless you're building sofwtare as a hobby, someone is paying you to do that work. Those paying aren't likley doing it as a hobby either. They have some expectation of getting their money back sometime in the future. Somewhere in the discussion of writing software for money, the notion of writing software for money was lost. 

Those with money pay those with software writing capabilities to produce products that can be sold or put to use to create a value in return. Along the way was a disconnect that software is an end in itself. That the needs to developers trumps the needs of those providing the money for the developers. That those spending the money get to say what they'll do, how they'll do it, or what they won't do with that money.

Writing software for money as practiced in a sole contributor paradigm provides nearly infinite flexibility on requirements, cost and schedule forecasting, and the current notion of making business, programmatic, and technical decisions in the absence of estimating the cost and impact of those decisions.

When that paradigm leaks into a larger domain of producing a return on the investment from that cost, there are two varaibles that must enter every conversation. The Value generated by expending a Cost to produce an assessment of both those variables.

ROI = (Value - Cost) / Cost

A Value at Risk is one approach to assessing what processes should be in place when spending othe people money. The larger the Value at Risk requires a larger discipline of managing both the Cost and Value. There are many paradigms of Agile and the domain and context of software development, or any project for that matter, is important to assess before stating any method is applicable outside the ancedotal domain of the speaker.

The first assessment is always Value at Risk. That is, what is the cost of making a wrong decision? This is the basis of Microeconomics. This is the oppotunity cost assessment of decision making.

Microeconomics studies the behavior of individuals and small impacting organizations in making decisions on the allocation of limited resources. Cost, schedule, and technical capabilities are certainly a limited resource.

Those conjecturing decisions can be made in the absence of estimating the cost and impact have yet to show the viability of those ideas in practice, at least outside small projects with low Value at Risk.

The book The Incremental Commitment Spiral Model: Principles and Practices for Successful Systems and Software, Barry Boehm and Jo Ann Lane is a good bridge book between small low value at risk agile, Scaled Agile for Enterprise, and the full up formal DOD 5000.02 acquisition processes that are trying very hard to move into the agile domain.

The book starts with four principles:

  1. Stakeholder value-based guidance
  2. Incremental commitment and accountability
  3. Concurrent multi-discipline engineering
  4. Evidence and risk-based decisions

There are extensions to these principles:

  • Risk Meta-Principles of Balance - balancing the risk of doing to little and the risk of doing too much to find the middle course sweet spot.
  • Theory W (Win-Win) theorem - in which a system will succeed of and only of it makes winners of its success-critical stakeholders.
  • System Success Realization Theorem - in which making winners of success-critical stakeholders requires:
    • Identifying all of the success-critical stakeholders.
    • Understanding how each stakeholder wants to win.
    • Having the success-critical stakeholders negotiate among themselves a win-win set of product and process plans.
    • Controlling progress toward the negotiated win-won realization, including adapting it to change.
Related articles Agile Requires Discipline, In Fact Successful Projects Require Discipline Why Johnny Can't Estimate or the Dystopia of Poor Estimating Practices What Do We Mean When We Say "Agile Community?" Why Project Management is a Control System Critical Thinking Skills Needed for Any Change To Be Effective
Categories: Project Management

Project Success: It Is A Balancing Act

Any framework reflects a balance.

Any framework reflects a balance.

 

When I asked a sample of my blog readers and podcast listeners whether being on-scope, on-schedule or on-budget defined project success, it was an exercise designed to understand personal biases. Understanding individual and team tendencies is important. However as we have discussed, the choice of a single attribute without context is just an intellectual exercise. All projects work to find a balance between all three attributes based on their specific business needs and context.

In well-run projects, decisions are not made in a vacuum. The balancing act between the scope, budget and schedule is fraught with difficulties. For example, I recently talked to a friend who was working on a project that included a feature promised to a major customer for their holiday season. The team determined that they were not going to make the delivery date unless some of the scope was jettisoned (adding more team was not an option at that point). In this case decisions about scope, budget and schedule exceed the product owner’s decision-making authority, even though she was a senior leader. Any solution would affect marketing and sales plans, revenue projections and potentially earning reports. Many voices had to be heard before a path forward was found. All three attributes had to be manipulated to find a solution. In this case scope was deferred, release dates shifted and additional budget provided. It was not a perfect solution, but it was feasible and focusing on a single attribute would have resulted in more problems. The size and importance of any project will affect how large a circle needs to be involved in decisions if the balance of on-scope, on-schedule and on-budget get out of balance.

Some potential combinations of scope, budget and schedule are impossible. As Fred Brooks observed, adding people to a late project will just make it later. When impossible teams are asked to deliver a with impossible constraints they will invariably cut corners, make mistakes, incur technical debt or flat out ignore one of the levers (and then ask for forgiveness).

There is a classic project saying, “on-time, on-budget, or on-scope, pick one.” It has been said that good project managers can deliver two of the three and everyone struggles with delivering all three. Real life projects represent a compromise between all three components. When time is critical, the team might cut the scope or pursue additional resources. On-scope, on-budget and on-schedule can be thought of as three dials with customer and management satisfaction providing feedback based on how they perceive progress. [HUH?] Project teams constantly adjust each dial to elicit positive feedback and to deliver the maximum value possible.


Categories: Process Management

FsiBot: Assorted Tweets

Phil Trelford's Array - Sat, 09/20/2014 - 17:23

FsiBot is a cloud hosted bot that evaluates F# expressions in mentions, built by Mathias Brandewinder. Underneath it uses the Twitter API, F# Compiler Services all hosted on Azure.

In the beginning the F# community put a lot of effort into bringing it down testing it’s security. Nowadays it’s become more of a creative outlet for code golf enthusiasts showcasing all sorts from ASCII Art to math.

Christmas Tree

Tomas’s tree first appeared before @fsibot was cool or for that matter even existed:

@tomaspetricek " ++ ++++ ++++0+ +++0++++ ++0++++++0 +0++++++0+++ 0++++++0++++++ [...]

— fsibot (@fsibot) August 25, 2014

Circle

An early attempt at ASCII art, the trick in Twitter is to use characters that are roughly the same width:

@ptrelford " ────█──── ──█████── ─███████─ ─███████─ █████████ ─███████─ ─███████─ ──█████── ────█──── "

— fsibot (@fsibot) August 31, 2014

Invader

Along similar lines this expression uses a bitmap to produce an ASCII invader:

@brandewinder "─────── ─█████─ █──█──█ ─██─██─ ─█─█─█─"

— fsibot (@fsibot) September 1, 2014

FSharp Logo

Here Mathias uses the same technique to generate a logo:

@brandewinder "└┐ ██████└┐└┐██└┐██└┐ ██└┐└┐└┐██████████ ██████└┐└┐██└┐██└┐ ██└┐└┐└┐██████████ ██└┐└┐└┐└┐██└┐██└┐"

— fsibot (@fsibot) September 6, 2014

Bar chart

This expression charts Yes vs No in the recent vote on Scottish Independence:

.@ptrelford " NO ████████────────2001926 YES ███████─────────1617989"

— fsibot (@fsibot) September 19, 2014

Sequences

The hailstone sequence:

.@ptrelford [17; 52; 26; 13; 40; 20; 10; 5; 16; 8; 4; 2; 1]

— fsibot (@fsibot) September 19, 2014

Pi

Interesting Pi approximation using dates:

@ptrelford 3.141592654

— fsibot (@fsibot) September 8, 2014

Inception

Evaluating an F# expression inside an F# expression:

.@ptrelford 2

— fsibot (@fsibot) September 8, 2014

Rick Roll

The thing about a rick roll is that no-one should expect it:

@ptrelford "never gonna give you up"

— fsibot (@fsibot) August 30, 2014

Magic 8-Ball

Based on magic 8-ball here's a somewhat abridged version:

@ptrelford "Go for it!"

— fsibot (@fsibot) August 30, 2014

Have fun!

Categories: Programming

Technical Performance Measures

Herding Cats - Glen Alleman - Sat, 09/20/2014 - 16:59

We're preparing for a Webinar on 25 September 2014, now titled Using Techncial Performance to Inform Earned Value, which addresses the disconnect in EAI-748-C between two statement

  • Page 2, 5 bullet - Objectively assess accomplishments at the work performance level.  
  • §3.8 - Earned Value is a direct measurement of the quantity of work accomplished. The quality and technical content of the work performed is controlled by other processes.  

Two reconcile these two statement, we need to have a process of informing Earned Value (BCWP) with the Techncial Performance of the products being built. After the Webinar, we'll post the link. 

In the mean time here's a list of resources gathered to support this topic.

  • Technical Performance Measurement - A Program Manager's Barometer: DCMA Pilots a Modified Approach to TPMs, Mike Ferraro, Program Manager, November-December, 2002
  • Technical Performance Measures, Jim Oaks, Rock Botta and Terry Bahill, BAE Systems.
  • Technical Measurement: A collaborative Project of PSM, INCOSE, and Industry, Garry Roedker and Cheryl Jone, 27 December 2005, INCOSE-TP-2003-020-01.
  • Technical Performance Measurement Earned Value, and Risk Management: Integrated Diagnostic Tool for Program Management, Commander N. D. Pisano, SC, USN, PEO Air SW and Special Mission Programs.
  • CPM-500-C/D/E: Integrated Systems Engineering with Earned Value Management, Partick K. Barker, Paul Solomon, November 2009, Alexandria, VA
  • CPM-500: Principles of Technical Management, Lesson D: Implementing Technical Performance Measurement, Mike Ferraro, IPMC 2002.
  • Technical Performance Measures Module: Space Systems Engineering Version 1.0, NASA.
  • Technical Performance Measurement - Systematic Approach for Planning, Integration, and Assessment, Part 1, Kathyyn A. Kulick.
  • Technical Performance Measurement - Kinking Technical Performance to Cost and Schedule, Part 2 Kathyyn A. Kulick.
  • Technical Performance Measurement - Assessment Techniques and Earned Value Calculation, Part 3, Kathyyn A. Kulick.
  • Leading Indicators for Systems Engineering, Sixth Annual NASA Project Management Challange, Dr. Donna H. Rhodes, MIT, Feb 2009.
  • The Difficult Problems of Establishing Measures of Effectivenss for Command and Control: A Systems Engineering Perspective, Noel Sproles, Systems Journal, Volume 4, Number 2, 2001.
  • Coming to Grips with Measures of Effectiveness, Noel Sproles, Systems Journal, INCOSE 2000.
  • Formulating Measures of Effectiveness, Nole Sproles, Systems Engineering Volume 5, Number 4, 2002
  • Systems Engineering at NASA: Selected Topics, Jim Andary, NASA Goddard Space Flight Center, Nov 5, 2009.
  • Space and Missile Command, Systems Engineering Primer and Handbook
  • NASA Systems Engineering Handbook
  • Engineering Complex Systems with Models and Objects,David W. Oliver Timothy P. Kelliher James G. Keegan, Jr., McGraw Hill.
  • Engineering Complex Systems, Douglas O. Norman and Michael L. Kuras, The MITRE Corporation
  • National Aerospace System - Systems Engieering Manual, FAA
  • MITRE Systems Engineering Guide, Collected Wisdom from MITRE's Systems Engineers.
Related articles Is There Such a Thing As Making Decisions Without Knowing the Cost? More Uncertainty and Risk Agile as a Systems Engineering Paradigm
Categories: Project Management

Project Success: Context

Trying to read the context...

Trying to read the context…

Many people poke fun at consultants because they invariably use the phrase “it depends,” even when discussing lunch. “It depends” is code for the context of the situation matters to the answer. When I asked a sample of my readers and listeners to weigh in on whether being on-budget, on-scope or on-schedule was the most important attribute of project success, most answered without using the dreaded “it depends”. An excerpt for a recent Twitter conversation in response to the essay about being on-budget, emphasized that in real life the definition of success will be influenced context.

UntitledIn the sample I surveyed, being on-schedule was the most important attribute with on-scope a close second and on-budget a distant third. If I had added constraints to the mix the answer would probably have been different. One respondent, while explaining why they felt that scope was the key attribute of success, made a strong argument for context (without calling it out directly), “Budgets and schedules can be modified when justified, and functionality can be phased in over time.” The statement could easily be interpreted as as a statement that success criteria, at least in the short run, depends on the goals of the business and the context of the project.

We can all conceive of scenarios where a specific context lead one attribute to be viewed as the most important in project success ASSUMING the software does what it is supposed to do and in an adequate manner. As an example of context, one respondent noted that in his environment and for some projects, “regulatory requirements that say it MUST deploy on a specific date.” Failure to meet the date can result in fines or penalties. In a similar vein, another respondent stated, “Projects are often tied to an organization’s business milestones or regulatory requirements so missing the date is not acceptable.” In both cases the context of a project could lead a project team to focus on one attribute over another.

Why did I ask the question without context? As I have noted in earlier essays, we are all subject to cognitive biases. These bias act as filters and shortcuts that help us interpret data. The context of a project will provide information about the goals and constraints of the project. Our cognitive biases are usefully in helping us interpret project context. Which is why project teams with diverse perspectives tend to make better choices when sorting out which attribute is most important. Is one attribute truly more important than another? It depends!


Categories: Process Management

Project Driven Organizations

Herding Cats - Glen Alleman - Fri, 09/19/2014 - 21:09

There is a popular notion that Projects are some how not needed to develop software. Just fund the Team and the business outcomes needed to fulfill a strategy and deliver value to the balance sheet will appear. 

This may work in  domains where the project team, the business and its processes and the funding sources are in fact all the same - a small startup for example. When the business moves beyond this self-contained startup mode, other  processes become needed to Govern the use of funds - Stewardship of Funding. 

Here's how to approach the process, once the firm moves beyond a collection of individuals personally all interacting with their customer based. These processes, data, and outcomes belong to the realm of Governance. In IT, Governance †

  • Clarifies business strategies and the role of IT in achieving them.
  • Measures and manages the amount of funding spent on the value received from IT.
  • Assigns accountability for the organizational changes required to benefit from new IT capabilities.
  • Learns from each implementation and becoming more adept at sharing and reusing IT assets.

Lifecycle from Glen AllemanIT Governance: How Top Performers Manage IT Decision Rights for Superior Results, Peter Weill and Jenne W. Ross, Harvard Business School Press. Related articles Delivering Needed Capabilities Staying on Plan Means Closed Loop Control The Purpose Of Guiding Principles in Project Management Performance-Based Project Management(sm) Released What Do We Mean When We Say "Agile Community?"
Categories: Project Management

Get your app in the Google index - and be ready for the future of Search

Google Code Blog - Fri, 09/19/2014 - 19:15
Last October we worked with a small number of developers to launch App Indexing, the ability for Google to index apps just like websites. In June, we opened App Indexing to all Android developers, giving you the ability to add app deep links to search results, helping users find your content and re-engage with your app after they’ve installed it.

Today, we’d like to highlight two videos to help you set up App Indexing. Check them out below. You’re only a few steps away from driving re-engagement and unlocking new avenues of discovery for your app.

1998 all over again

In 1998, Google set out to index the web and make content easily accessible and discoverable.

Today mobile app adoption is growing rapidly. Similar to the early days of the internet, it can be frustrating for users to search through their device to find what they need, since the content within apps typically exists in silos. By implementing App Indexing, you can re-engage your users from search results.

Consider the scenario of users looking for details about a great place for some Spanish food. Users might already have a great mobile app for restaurant details installed, but they typically still rely on a search engine to pull up information about local places to eat. Wouldn’t it be great if they could jump straight into your app from those search results? App Indexing is all about connecting users with the best content, whether it’s in-app or on the web. You can add it to your app very quickly -- simply add deep linking to your app, and then verify your website with Google. To enable re-engagement with your users, you can generate query autocompletions in the Google app using the App Indexing API.

Learn More

To learn more about App Indexing, and how you can use the API for your apps, check out this DevByte from Lawrence Chang, Senior Product Manager for App Indexing at Google.
For an in-depth look at App Indexing, demonstrating the technology, and the opportunities available with it, watch this session from Google I/O 2014: “The Future of Apps and Search.”
We’re at the beginning of a new wave of how people find and interact with content, and it often starts with search. Is your app in the index? If not, follow the steps in this post, and get ready for the future of Apps and Search!

Posted by Laurence Moroney

Laurence is a Developer Advocate at Google, working on Google Play services for Android. When not having fun with this, he's also a best-selling author of several Young Adult novels and dozens of books on computer programming.
Categories: Programming

Stuff The Internet Says On Scalability For September 19th, 2014

Hey, it's HighScalability time:


Galactic bolt-hole or supermassive black hole, weighing as much as 21 million suns?
  • Quotable Quotes:
    • @debuggist: Chief takeaway from #velocityconf for me: failure happens so monitor for the ones that are important albeit in systems or culture & fix them
    • @Carnage4Life: The real tech bubble is valuations of Google, Facebook & Twitter are inflated by app install ads from unprofitable startups 
    • Jay Parikh: Android has 2x as many users as iOS. However, iOS average revenue per user is 4x higher than Android. 
    • Joe Armstrong: We’ve made a mess. Need to reverse entropy. Quantum mechanics sets limits to ultimate speed of computation. We need Math. Abolish names and places. Build the condenser. Make low-power computers - no net environmental damaged.
    • @reneritchie: iOS 8 is a beefy update. The internet is feeling the strain of millions of downloads. If it’s slow or stuttering, just give it some time.
    • @vishyp: Quip is big on disconnected clients and clients as write-thru caches. Like it! Uses C++ for x-platform. @btaylor #atscale2014
    • @csabacsoma: Instragram: we use Postgres for almost everything now, some Redis and Memcached #AtScale2014
    • @rfberry: "To maximize throughput, we needed an integrated approach to backpressure." #AtScale2014
    • @sharonw: I asked this Facebook / Instagram panel about image upload. Key changes: retry aggressively, and resize and encode on device. #AtScale2014
    • Igor Zaika: Most of the developers who work on Microsoft Office are younger than the codebase. 
    • Chris Marra: There are about 10k different device models accessing Facebook. Designing for high-end smartphones does not cut it.
    • @Carnage4Life: Apple spends $100m on a U2 album you don't want. Microsoft spends that on ads you don't like. Amazon spends it on free shipping #Marketing
    • @beerops: Laziness as a multiplier: train other people to do what you can do to remove yourself as a bottleneck #velocityconf
    • @xaprb: "Failure is a feature of complex systems." - #velocityconf
    • @dalmaer: "Our mobile app is a write through cache (SQLite) to the source of truth (MySQL on AWS)" -- @btaylor #AtScale2014
    • @herminghaus: @BenedictEvans Designed an 11.4TB patent retrieval system in 1993 with slow WORM robots. Cost $140m. Now <$1000.- at BestBuy.
    • @BenedictEvans: You can't ask people to decide on a trade-off when they have experience of one side but not the other.

  • Caching at Scale. There's a need to better manage caching, especially under failure conditions. Solutions are generally in the form of a proxy layer above memcached. Along these lines Box created Tron, Twitter created TwemProxy, and Facebook created a value meal in McRouter. Database people have always countered, why have a separate cache, just build a cache into the database? This hasn't worked for various reasons, mostly because a database always cares more about being a good database rather than being a good cache. Vitess wants to fix that. Vitess is an open-source system written in Go used at Youtube that: challenges the paradigm of treating caching as a separate layer by directly addressing the issues of database scalability and by modifying the handling of SQL queries.  

  • Talk about your Chaos Godzilla. Facebook Turned Off Entire Data Center to Test Resiliency. Before flipping that switch there must have been a little pause, perhaps thinking this wouldn't be prudent, but damn the torpedoes and full speed ahead. Apparently some issues were found, but it went fairly smoothly. Hazaa for the chutzpah.

  • Best LAN party ever? Researchers twist four radio beams together to achieve high data transmission speeds. The researchers reached data transmission rates of 32 gigabits per second across 2.5 meters of free space in a basement lab.

  • This is an understatement. iOS 8, thoroughly reviewed. An amazing job. A big take away for me is Apple is systematically removing reasons not to buy an iPhone. Bigger phone. Check. Configurable keyboard. Check. Extensions that display in the today view and allow app cooperation. Check. Another take away is Apple is abandoning simplicity for configurability, which is embracing complexity, which is a potential experience killer.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Change is Learning: No Silver Bullets or Quick Fixes

Way back when I was a developer, my professors taught me structured design and design by contract. Those were supposed to be the silver bullets for programming.  You see, if you specified things enough, and structured things enough, everything would all work out.

I thought I was the only idiot that structure and specification didn’t work for. Why did I have to iterate and increment?

At my first job, we had design reviews and code reviews. I learned a lot. I worked on a government contract, and the government mandated those reviews. They were useful, and they were supposed to be a silver bullet. Because we implemented features (yes, back in the ’70s, we implemented features), the reviews were helpful in the small.

But, I worked on a program. There is just so much design by contract can do for a program. I was in Boston. I had questions for my counterpart in New Mexico. I was not allowed to talk to that person. (To this day, I don’t know who that person was. That person was part of another subcontractor.) I had to send my questions up through the project management layers to the program team and down the other side. My questions never did get answered. I left that company before I knew if my code worked with the entire system.

I transitioned into project management, program management and people management. I have seen my share of fads, bullets, and fixes for the software departments.

As a director in the early ’90’s I got sent to TQM school (Total Quality Management). My company thought it would change the way we managed, and revolutionize our organization. It would be our “silver bullet.” I’m pretty sure those were somebody’s words. They weren’t mine.

I got a certificate for my five days of intensive training (powerpoint and calculations). I might still have it somewhere.

I became excellent at calculating many costs: the cost of quality, the cost of not doing things right,  the cost of technical debt, even what we might consider the cost of delay. I dutifully explained these costs to my senior management. Was I able to persuade my company of the cost of not doing the right thing, even though I was a middle manager, a program manager, and had been TQM’d?

No. My management was too concerned that doing things “right” would prevent us from making money. I had data—yes, data—that proved them wrong. But their belief systems overcame my data.

Later, after I started my consulting business, many of my clients fell into the ISO 9000 and the CMM/CMMI quick fix/silver bullet traps. The ISO joke made the rounds: “You could specify a cement life jacket with ISO, as long as you define the right process for creating it.” Well, no. But the jokes still persisted. (Many people find value in ISO. Some do not.)

And, with the CMM/CMMI? Oh, my clients had binders full of documentation, but they weren’t releasing software. They hired me on the side, because their silver bullet of CMM/CMMI process wasn’t working. Somehow, my approaches of timeboxes and increments, and iterations, and thinking about their projects was. (Many people find  value in using CMM/CMMI. Some do not.)

Now we have agile. If you read my work, you know I lean towards agile and lean. At the same time, I say that agile is not for everyone. Agile is not a silver bullet.

We have no silver bullets in software.

We have difficult problems to solve. We must think about our approaches, each and every time. We must consider our context. That why I wrote Manage It! Your Guide to Modern, Pragmatic Project Management. Because every single project is unique. That’s why I’m writing Agile and Lean Program Management: Collaborating Across the Organization. That book is a principle-based approach to program management, to scaling agile past one or two teams. Not a one-size fits all approach to program management.

And, because we have no silver bullets in software, and because we have no quick fixes, my management myth this month is We Need a Quick Fix or a Silver Bullet.

It’s very tempting to think that transitioning to agile might be a quick fix or a silver bullet. Transitioning to agile or lean might help you. It won’t be quick.

Any transition is a change. Change takes time. Change is learning. Developing software is learning. You can help yourself learn faster with some approaches: make value obvious instead of tasks, get feedback often, visualize your work, etc. For many of you starting your agile journey, these are cultural changes.

The more you challenge the culture, the longer the change takes because people need to learn what to do and how to work. Cultural change is not a silver bullet. It is certainly not a quick fix.

Read Management Myth 33: We Need a Quick Fix or a Silver Bullet.

Categories: Project Management