Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

Go vs Python: Parsing a JSON response from a HTTP API

Mark Needham - 8 hours 53 min ago

As part of a recommendations with Neo4j talk that I’ve presented a few times over the last year I have a set of scripts that download some data from the meetup.com API.

They’re all written in Python but I thought it’d be a fun exercise to see what they’d look like in Go. My eventual goal is to try and parallelise the API calls.

This is the Python version of the script:

import requests
import os
import json

key =  os.environ['MEETUP_API_KEY']
lat = "51.5072"
lon = "0.1275"

seed_topic = "nosql"
uri = "https://api.meetup.com/2/groups?&topic={0}&lat={1}&lon={2}&key={3}".format(seed_topic, lat, lon, key)

r = requests.get(uri)
all_topics = [topic["urlkey"]  for result in r.json()["results"] for topic in result["topics"]]

for topic in all_topics:
    print topic

We’re using the requests library to send a request to the meetup API to get the groups which have the topic ‘nosql’ in the London area. We then parse the response and print out the topics.

Now to do the same thing in Go! The first bit of the script is almost identical:

import (
	"fmt"
	"os"
	"net/http"
	"log"
	"time"
)

func handleError(err error) {
	if err != nil {
		fmt.Println(err)
		log.Fatal(err)
	}
}

func main() {
	var httpClient = &http.Client{Timeout: 10 * time.Second}

	seedTopic := "nosql"
	lat := "51.5072"
	lon := "0.1275"
	key := os.Getenv("MEETUP_API_KEY")

	uri := fmt.Sprintf("https://api.meetup.com/2/groups?&topic=%s&lat=%s&lon=%s&key=%s", seedTopic, lat, lon, key)

	response, err := httpClient.Get(uri)
	handleError(err)
	defer response.Body.Close()
	fmt.Println(response)
}

If we run that this is the output we see:

$ go cmd/blog/main.go

So far so good. Now we need to parse the response that comes back.

Most of the examples that I came across suggest creating a struct with all the fields that you want to extract from the JSON document but that feels a bit over kill for such a simple script.

Instead we can just create maps of (string -> interface{}) and then apply type conversions where appropriate. I ended up with the following code to extract the topics:

import "encoding/json"

var target map[string]interface{}
decoder := json.NewDecoder(response.Body)
decoder.Decode(&target)

for _, rawGroup := range target["results"].([]interface{}) {
    group := rawGroup.(map[string]interface{})
    for _, rawTopic := range group["topics"].([]interface{}) {
        topic := rawTopic.(map[string]interface{})
        fmt.Println(topic["urlkey"])
    }
}

It’s more verbose that the Python version because we have to explicitly type each thing we take out of the map at every stage, but it’s not too bad. This is the full script:

package main

import (
	"fmt"
	"os"
	"net/http"
	"log"
	"time"
	"encoding/json"
)

func handleError(err error) {
	if err != nil {
		fmt.Println(err)
		log.Fatal(err)
	}
}

func main() {
	var httpClient = &http.Client{Timeout: 10 * time.Second}

	seedTopic := "nosql"
	lat := "51.5072"
	lon := "0.1275"
	key := os.Getenv("MEETUP_API_KEY")

	uri := fmt.Sprintf("https://api.meetup.com/2/groups?&topic=%s&lat=%s&lon=%s&key=%s", seedTopic, lat, lon, key)

	response, err := httpClient.Get(uri)
	handleError(err)
	defer response.Body.Close()

	var target map[string]interface{}
	decoder := json.NewDecoder(response.Body)
	decoder.Decode(&target)

	for _, rawGroup := range target["results"].([]interface{}) {
		group := rawGroup.(map[string]interface{})
		for _, rawTopic := range group["topics"].([]interface{}) {
			topic := rawTopic.(map[string]interface{})
			fmt.Println(topic["urlkey"])
		}
	}
}

Once I’ve got these topics the next step is to make more API calls to get the groups for those topics.

I want to make those API calls in parallel while making sure I don’t exceed the rate limit restrictions on the API and I think I can make use of go routines, channels, and timers to do that. But that’s for another post!

Categories: Programming

Southeast Asian indie game developers find success on Google Play

Android Developers Blog - Fri, 01/20/2017 - 18:46
Posted by Vineet Tanwar, Business Development Manager, Google Play

Indie game developers bring high quality, artistic, and innovative content to Google Play and raise the bar for all developers in the process. In fact, they also make up a large portion of our 'Editor's Choice' recommended titles.
Southeast Asia, in particular, has a vibrant indie game developer ecosystem, and we've been working closely with them to provide tools that help them build successful businesses on Google Play. Today, we're sharing stories from three Indie developers based in Singapore, Vietnam, and Indonesia, who joined us at our 'Indie Game Developers Day' workshops in May 2016 and all of whom have experienced significant growth since.

Inzen Studio from Singapore learned how to use store listing experiments and has improved the conversion rate of their newly launched game Dark Dot by 25%. Indonesia based studio, Niji Games, creator of Cute Munchies, implemented 'Saved Games' and 'Events and Quests' from Google Play games services to significantly improve user retention, and also earned an 'Editor's Choice' badge in the process. Ho Chi Minh City based developer, VGames, optimized monetization and introduced new paid products for their game Gungun online, and grew revenue by over 100%.


Indie game developers who are interested in meeting members of Google Play and who would like to work closer with us are invited to join our next round of SEA workshops in March 2017. To apply for these events, just fill in this form and we will reach out to you.


How useful did you find this blogpost?

‚ėÖ ‚ėÖ ‚ėÖ ‚ėÖ ‚ėÖ    
Categories: Programming

Stuff The Internet Says On Scalability For January 20th, 2017

Hey, it's HighScalability time:

 

Absolutely. Do we agree that the cerebellum is amazingly beautiful? (@PeppeGanga)
If you like this sort of Stuff then please support me on Patreon.
  • 900 GB: data stolen in Cellebrite hack; 99.24%: users identified by cross-browser fingerprinting; 72%: intend to migrate to a hybrid cloud; 90%: Google & Facebook ad traffic is useless; 5.2 terabytes per second: data from Australian Square Kilometre Array Pathfinder; 10 billion: searches on DuckDuckGo in 2016; $330m: Amazon's loss on Alexa; 

  • Quotable Quotes:
    • @brucel: Breaking: Programmer accused of writing unreadable code refuses to comment.
    • @asymco: Remember Android first? App Annie believes the Apple’s App Store produced about twice as much revenue as Google Play
    • @bridgetkromhout: Describing your old-timer ranting as "greybeard" just makes me want to fight you with sed & awk at twenty paces. Be there tomorrow at dawn.
    • @StevenShorrock: Root Cause Analysis is: * Acceptable for simple systems * Inappropriate for complicated systems * Ludicrous for complex systems
    • @swardley: Five years ago Amazon was worth about half of Walmart, today Walmart is worth about half of Amazon.
    • Eric Raymond: In practice, I found Rust painful to the point of unusability. The learning curve was far worse than I expected; it took me those four days of struggling with inadequate documentation to write 67 lines of wrapper code for the server.
    • @swardley: past history shows many major players won't announce they're getting into the battle until some time after war has ended
    • @benthompson: Apple wasn't billed as phone maker / Amazon wasn't billed as infrastructure provider / FB wasn't billed as portal / Snapchat wasn't billed as TV
    • Jessitron: the biggest consideration in choosing whether to use libraries or services for distribution of effort / modularization is that choice of who decides when it deploys. Who controls which code is in production at a given time.
    • Hi Ben: The disruption of TV will follow a similar path: a different category will provide better live sports, better story-telling, or better escapism. Said category will steal attention, and when TV no longer commands enough attention of enough people, the entire edifice will collapse. Suddenly.
    • @leonidasfromxiv: I also don't understand why people compare Go with Rust. If you need a GC-less programming language: Rust; if you need a board game: Go.
    • Carlo Rovelli: The world isn’t just a mass of colliding atoms; it is also a web of correlations between sets of atoms, a network of reciprocal physical information between physical systems.
    • Chris Dixon: In the beginning, hardware-focused companies make gadgets with ever increasing laundry lists of features. Then a company with strong software expertise (often a new market entrant) comes along that replaces these feature-packed gadgets with full-fledged computers. 
    • Animats: The real question is "what do we do with a lot of CPUs without shared memory?" Such hardware has been built many times - Thinking Machines, Ncube, the PS2's Cell - and has not been too useful for general purpose computing.
    • @taavet: Very unfortunate that incumbents see tech only as a way to cut costs. Versus seeing tech to offer much better products.
    • NelsonMinar: This is what security looks like when your threat model is well funded government agencies.
    • Don Norman: The solution requires a different approach to the design of automation: collaboration. Instead of automating what can be automated, leaving the rest to the driver, we must develop collaborative systems so that the driver is continually involved in giving high-level guidance, thereby always staying active, always being in the loop. 
    • Thomas Frey: It took 50 years for the world to install the first million industrial robots. The next million will take only eight. Will this cause more jobs or few jobs in the future? I'm not convinced we know the answer.
    • @jtauber: "Every shot in Piper is composed of millions of grains of sand, each one of them around 5000 polygons."
    • rackforms: my point is the current situation, basically 2 companies controlling so much traffic, seems, well, bad for small business in this country. I value what they bring to the table and fully understand why they're so popular. But is things keep on this way where does that lead the guys like me? Is this just the way it has to be? Is the dream of the open Internet already dead?
    • @sheeshee: I think I know why it's called "DevOps" - "DevOops" was too obvious... ;)
    • greenspot: The open solution to a faster mobile web would have been so easy: Just penalize large and slow web pages without defining a dedicated mobile specification. That's it. This wasn't done in the past, slow pages outperformed fast ones on the SERPs because of some weird Google voodoo ranking, heck sometimes even desktop sites outperformed responsive ones on smartphones. If they had just tweaked these odd ranking rules in way that speed and size got more impact on the overall ranking there wouldn't have been any reason for AMP—the market would have regulated itself.
    • Juergen Schmidhuber: General purpose quantum computation won’t work (my prediction of 15 years ago is still standing). Related: The universe is deterministic, and the most efficient program that computes its entire history is short and fast, which means there is little room for true randomness, which is very expensive to compute. What looks random must be pseudorandom, like the decimal expansion of Pi, which is computable by a short program. Many physicists disagree, but Einstein was right: no dice. There is no physical evidence to the contrary

  • RethinkDB is shutting down and here's the post-portem. Lessons: the database market is like Mad Max fighting in the Thunderdome; it's better to optimize for useless microbenchmarks than it is to be good; optimism isn't a strategy.

  • Apple isn't alone in using custom hardware to thwart nation state level attackers. Google Infrastructure Security Design Overview. Good overview at Google reveals its servers all contain custom security silicon. Google designs "custom chips, including a hardware security chip that is currently being deployed on both servers and peripherals. These chips allow us to securely identify and authenticate legitimate Google devices at the hardware level."  Google encrypts data before it is written to disk, to make it harder for malicious disk firmware to access data. Google uses automated and manual code review techniques. Google uses automated software and code reviews to detect bugs in software its developers write. Google scans user-installed apps, downloads, browser extensions, and content browsed from the web for suitability on corp clients. Google uses a custom version of the KVMhypervisor. Good discussion on HackerNews, where a lot of the comments are on how Google needs this level sophistication to evade the prying eyes of governments.

  • What happens when you embed machine learning into a DBMS in order to continuously optimise its runtime performance? You get Self-driving database management systems. Humans suck at tuning databases so this is just one more job AIs will eventually toss into the dust bin of history. TensorFlow was integrated inside Peleton training two RNNs on 52 million queries from one month of traffic for a popular site. Does it help?: early results are promising: (1) RNNs accurately predict the expected arrival rate of queries. (2) hardware-accelerated training has a minor impact on the DBMS’s CPU and memory resources, and (3) the system deploys actions without slowing down the application. 

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Categories: Architecture

Emotional Intelligence Is Not Always The Right Tool

30065778560_cd98edf55f_k

Emotional intelligence often touted as a tool that can be used to make every outcome better. However, the more academic literature suggests economic intelligence is not a panacea.  There are numerous papers that identify scenarios in which emotional Intelligence has not discernible impact on business outputs and might actually get in the way. Several are described below:

Emotional Intelligence in Non-Emotional Environments

Dana Joseph of the University of Central Florida and Daniel Newman of the University of Illinois comprehensively analyzed studies that showed a link between Emotional Intelligence and job performance. The study of studies found that the level found emotional intelligence was not consistently related to job performance.  However when the data was broken down into jobs that require an extensive understanding of emotions and those that did not the analysis told a different story. When the studies for jobs requiring extensive attention to emotions were analyzed separately, higher levels of emotional intelligence translated into better performance. Examples of jobs requiring significant levels of emotional intelligence include salespeople, call center representatives and other customer-facing roles. While not included in the studies we would expect project managers and Scrum masters in this group of roles. Alternately, when personnel in the study with higher levels of emotional intelligence in jobs that involved fewer emotional demands were studied they were found to have lower performance.  The authors suggest that spending time reading emotions of others when you should be doing something more introspective, such as writing code, is counterproductive.

Emotional Intelligence as a Tool for Manipulation

One often quoted study supporting this potential problem was published by University of Cambridge professor Jochen Menges.  Dr. Menges found that when a leader gave an inspiring speech filled with emotion, the audience was less likely to scrutinize the message and remembered less of the content. People that hone their emotional skills are often better at manipulating others. While leaders often manipulate emotions to generate a good output for an organization, examples abound of those leveraging their understanding the emotions of others to motivate them to act against their own best interests.

There are a number of other areas of concern noted in a recent article on the Harvard Business review blog.

  1. The potential impact of emotional intelligence on creativity and innovation. The article points out that attributes typically associated with creativity and innovation are generally at odds with those associated with emotional intelligence.  Emotionally intelligent people are more apt to stay within bounds of common process and go along with others because those actions are less likely to cause emotional pain.  Whereas innovation and creativity often require breaking pushing boundaries and breaking rules. Steve Jobs was not an emotionally intelligent person, he was an innovator.
  2. People with high emotional intelligence tend to have difficulty giving or taking negative or difficult feedback.  It is emotionally painful to deliver negative feedback and people with high emotional intelligence feel that pain, and as a general rule people avoid pain whenever possible.
  3. Similarly, people with high emotional intelligence tend to not to make unpopular decisions for much the same reason as they avoid delivering negative feedback. While you might put an emotionally intelligent person in charge of communication a corporate downsizing they probably wouldn’t the best person to plan and execute the downsizing because it would be painful. 
  4. People with high levels of emotional intelligence tend to avoid taking risks. People with high emotional intelligence avoid taking risks because they want to avoid negative emotional feedback if the risk comes to fruition.

Emotional intelligence can be used to manipulate individuals or teams.  The natural tendency is to consider manipulation as a negative; however, when an emotionally intelligent leader helps position a team to avoid a negative emotional problem they are acting in the best interest in a team. Manipulation, in this case, would be good (and would probably be called leadership).  For example, I often ask team members to remember and share positive events prior to team meetings. The goal is to increase sharing, cause bonding, increase trust, and most importantly put the team in a positive mood before the meeting. In this case, manipulation is a good thing. If I manipulated them to act against their own greater good or that of the team, in the long run, I think I would be judged less positively.  

 


Categories: Process Management

App Security Improvements: Looking back at 2016

Android Developers Blog - Thu, 01/19/2017 - 23:46
Posted by Rahul Mishra, Android Security Program Manager
In April 2016, the Android Security team described how the Google Play App Security Improvement (ASI) program has helped developers fix security issues in 100,000 applications. Since then, we have detected and notified developers of 11 new security issues and provided developers with resources and guidance to update their apps. Because of this, over 90,000 developers have updated over 275,000 apps!
ASI now notifies developers of 26 potential security issues. To make this process more transparent, we introduced a new page where developers can find information about all these security issues in one place. This page includes links to help center articles containing instructions and additional support contacts. Developers can use this page as a resource to learn about new issues and keep track of all past issues.

Make sure to check out our new Security for Android Developers page, which highlights the latest security posts, security best practices documents and security checklist. These resources are all aimed at improving your understanding of general security concepts and giving you examples that can help you address app-specific issues.

How you can help:
For feedback or questions, please reach out to us through the Google PlayDeveloper Help Center.
To report potential security issues in apps, email us at security+asi@android.com.
Categories: Programming

Android Developer Story: Wallapop improves user conversions with store listing experiments on Google Play

Android Developers Blog - Thu, 01/19/2017 - 17:42
Posted by Lily Sheringham, Developer Marketing, Google Play
Wallapop is a mobile app developer based in Barcelona, Spain. The app provides a platform to users for selling and buying things to others nearby in a virtual flea market by using geolocalization. Wallapop now has over 70% of their user base on Android.

Watch Agus Gomez, Co-Founder & CEO, and Marta Gui, Growth Hacking Manager, explain how using store listing experiments has increased their conversion rate by 17%, and has allowed them to optimize organic installs.


Learn more about store listing experiments. Get the Playbook for Developers app to stay up-to-date with more features and best practices that will help you grow a successful business on Google Play.


How useful did you find this blogpost? ‚ėÖ ‚ėÖ ‚ėÖ ‚ėÖ ‚ėÖ                                                                               
Categories: Programming

Quote of the Month January 2017

From the Editor of Methods & Tools - Thu, 01/19/2017 - 11:37
Although many organizations are certified at SEI CMM Level 5, when a team hits a production snag, those certificates don‚Äôt carry much weight. People need IT heroes in the team to pull things through. Dependencies on folks who are technically competent, can think on their feet, and can manage time pressures increase that much more. […]

Tips from developers Peak and Soundcloud on how to grow your startup on Google Play

Android Developers Blog - Thu, 01/19/2017 - 10:02

Posted by Francesca Di Felice, Developer Marketing at Google Play
At Playtime 2016, Google Play's series of developer events, we met with top app and game developers from around the world to share learnings on how to build successful businesses on Google Play. Several startups, including game developer Peaklabs and audio platform SoundCloud, presented on stage their own best practices for growth, which you might find helpful.

Testing for growth, by Peak

Hear from Kevin Shanahan, Product Manager from Peak, a brain training app, on how to grow sustainably.



  • Test lots of ideas: You can't be sure of what will work and what won't, so you need to test lots of ideas. Peak ran four different tests to try to increase conversions to Pro (their subscriber offering):
  1. Made the ability to replay games a Pro feature
  2. Reduced price of Pro by 25% in top 2 markets
  3. Bundled add-on modules from partners into Pro
  4. Showed a preview of Pro-only content
          One of these tests resulted in a 50% increase in conversions.

  • Get the basics right: Start with a great product and have a data-informed culture. Don't only test app features, experimenting your store listing using store listing experiments is also important.
  • Build a robust A/B testing process: Having a well-defined A/B testing process and a system for tracking your experiments is key to testing quickly and effectively.

Improving user retention, by SoundCloud

Andy Carvell, former Product Manager at SoundCloud, an online audio distribution platform that enables its users to upload, record, promote, and share their originally-created sounds, explains how they focus on retention to improve growth.

 
  • Design your retention strategy: Apps with poor retention grow slowly. To increase your retention you should:
    • Convert new users to repeat visitors by providing a strong onboarding experience for new users and taking a high-touch approach during the first days and weeks.
    • Increase visit frequency within this group by providing frequent, timely, and relevant messaging about content or activity on the platform.
    • Target returning users who were not seen over the last period, who are 'at risk of churn' users, by giving them reasons to come back for another session before losing them.
    • Re-activate lapsed (long-term churned) users with campaigns to remind them about your app and offer an incentive to return.
  • Build 'growth machines': Create repeatable processes that testing has proven to positively impact retention, retaining users, and preventing churn.
  • Use activity notifications in a personalised and effective way: At SoundCloud there are plenty of things that happen when users are not in the app that might be relevant to them, for example new content releases or social interactions. They tested 5 new notification types, always keeping a control group to better keep track of the impact, and managed to increase retention in a 5%. Watch the video above for more of Andy's tips on making better use of notifications.

Other speakers, such as Silicon Valley VC Greylock, have also shared their tips for startup growth. Watch more sessions from this year's Playtime events to learn best practices from other apps and game partners, and the Google Play team. Get the Playbook for Developers app to stay up to date with news and tips to help you grow a successful business on Google Play.

How useful did you find this blogpost? ‚ėÖ ‚ėÖ ‚ėÖ ‚ėÖ ‚ėÖ                                                                               
Categories: Programming

Project vs product teams

Actively Lazy - Wed, 01/18/2017 - 22:45

One of the hardest things for companies trying to be agile is how to structure teams. Back in the bad-old days, teams would form around a project. Then six months later, everyone would dissipate and go onto new teams. By the time a team has formed and become effective it is ripped apart again. You get no sense of ownership, no continuity.

children-laptop

Nowadays everyone knows that projects are bad, you need scrum teams instead. So a scrum team is formed with a product owner to prioritise the work. But what often happens is that what gets prioritised onto the backlog is a project in bite-size pieces. For example, I saw one team that ran out of work to do. The backlog was empty because, except for bugs, none of the outstanding projects had been signed off. There’s that word again. Project.

Behind the scenes a scrum team often becomes a slightly better way of delivering projects. You get the benefits of team consistency and continuity and the added benefit that the business can carry on thinking of the work in terms of projects. The downside of this approach is the scrum team can lack clear focus: there’s no overarching goal for the team. From sprint to sprint the focus might change as the relative importance of different projects changes. This makes it hard for the team to feel committed to a big idea, to some greater purpose. It ends up an endless procession through the backlog.

Why does this happen? I think it comes down to money. Somebody, somewhere is watching the money. Somebody wants to know “if I spend ¬£x here, how much am I going to make back and by when?” The idea of the project is very easy to fit into this model. The team costs ¬£x per day. The project is estimated to take n days. It’s expected to deliver ¬£y profit. From this we can calculate the expected return on our investment. The trouble is, most of these numbers are entirely made up. If not fundamentally unknowable.

Let’s start with the obvious one: how long is the project going to take? Really, we still actually ask this question? Have we learnt nothing from agile? It seems not: many, many people still think about the world in terms of delivery dates and certainties. When will we learn that the best way is always to deliver a little, inspect the results; then decide whether to keep on the same path or deliver something different. You can’t have an end date with this approach – it’s not even meaningful. Keep on delivering one thing until there’s something better you could be doing, then go do that. Rinse, repeat.

What about the other question: how much profit will this project make? Well, let’s assume for now that the entire project, as originally conceived, will actually be delivered (as if this ever actually happens in software). Can you tell how much money it’s made you? Really? Independent of every other change that the organisation has made at the same time? From software to operations to marketing?

Now sometimes you can come up with a good estimate of expected returns, but often it’s just a pipe dream. But, if you’re vigorously disagreeing with me: I assume you’re religiously tracking actual costs and feed that back into future project planning? I have seen very, very few companies actually do this. If you’re not actually measuring how much you made from a project, how do you know your original estimates were any good?

So we have two made up numbers, both almost certainly unachievable in practice – but we use this to dictate the team’s priority order. I once saw a project signed off and jump to the top of the priority order because it predicted something like a 10% uplift in revenue for the company. This was a very large number for a single project and clearly ridiculous to everyone involved, but it was signed off and duly implemented. Revenue projections later that year were re-estimated downwards and downwards due to difficult market conditions. And some blatant over-estimation. And yet, this non-science is what passes for return on investment planning in all-too-many organisations.

What’s the alternative? The best teams I’ve seen have been structured around products. Give the team complete ownership of one or more products. Any and all changes to those products go via the product team. A product owner guides product direction. As an area expert they are entrusted to decide what are the most important things to work on. They can discuss long term directions with the team and have a consistent, coherent vision for where the product will evolve towards. While, inevitably, some changes are large and sufficiently inter-dependent that they become a project (if one part is delivered then it all must be); the team understands the business benefit of the¬†solution and can evolve the implementation to meet the underlying business need, instead of trying to satisfy some arbitrary internal project deadline. This gives teams the complete freedom to inspect and adapt each iteration. With an understanding of the business priorities for their products they can make sensible trade-offs as each iteration surfaces more information.

What about the money? It’s hard, but let’s be honest about it: return on investment is not clear with the project model of software delivery, so accept that it isn’t clear. The hard thing is working out which products are making you money and which could make¬†more money if more was invested. The trouble is I’ve worked in teams where, honestly, the product was so profitable with so little scope for uplift that the most cost-effective thing to do would have been to fire the dev team and just keep milking the cash cow.

So how can we decide where to spend our money? I think the empirical model of agile could fit here perfectly well. Let’s assume for a minute that the amount of money you have for the delivery team as a whole is fixed – your only choice is where to put it. How much to spend on product A vs how much on product B. Can you estimate how much money each product is making for the business? How is it changing over time?

If one product is making more profit each month – if it’s a growing product – then invest more resources there, to accelerate the growth. If a product is slowing down, with smaller increases in profit each month, or even with profit decreasing – then stop spending so much money on it. This naturally means that your money goes where it seems to be delivering the biggest return. Put your money where it seems to be delivering results.

The hardest thing with this is that it takes time to get the feedback: changing resource allocation could take months to show up on the bottom line. But at least we’re being honest about the impact our decisions have.¬†Instead of trying to micro-manage delivery via projects, manage where resources are put and let the product owner manage the priority order.


Categories: Programming, Testing & QA

Welcoming Fabric to Google

Google Code Blog - Wed, 01/18/2017 - 22:26
Originally posted on the Firebase Blog

Posted by Francis Ma, Firebase Product Manager

Almost eight months ago, we launchedthe expansion of Firebase to help developers build high-quality apps, grow their user base, and earn more money across iOS, Android and the Web. We've already seen great adoption of the platform, which brings together the best of Google's core businesses from Cloud to mobile advertising.

Our ultimate goal with Firebase is to free developers from so much of the complexity associated with modern software development, giving them back more time and energy to focus on innovation.

As we work towards that goal, we've continued to improve Firebase, working closely with our user community. We recently introducedmajor enhancements to many core features, including Firebase Analytics, Test Lab and Cloud Messaging, as well as added support for game developers with a C++ SDK and Unity plug-in.


We're deeply committed to Firebase and are doubling down on our investment to solve developer challenges.
Fabric and Firebase Joining Forces

Today, we're excited to announce that we've signed an agreement to acquire Fabric to continue the great work that Twitter put into the platform. Fabric will join Google's Developer Product Group, working with the Firebase team. Our missions align closely: help developers build better apps and grow their business.
As a popular, trusted tool over many years, we expect that Crashlytics will become the main crash reporting offering for Firebase and will augment the work that we have already done in this area. While Fabric was built on the foundation of Crashlytics, the Fabric team leveraged its success to launch a broad set of important tools, including Answers and Fastlane. We'll share further details in the coming weeks after we close the deal, as we work closely together with the Fabric team to determine the most efficient ways to further combine our strengths. During the transition period, Digits, the SMS authentication services, will be maintained by Twitter.


The integration of Fabric is part of our larger, long-term effort of delivering a comprehensive suite of features for iOS, Android and mobile Web app development.

This is a great moment for the industry and a unique opportunity to bring the best of Firebase with the best of Fabric. We're committed to making mobile app development seamless, so that developers can focus more of their time on building creative experiences.
Categories: Programming

Get the guide to finding success in new markets on Google Play

Android Developers Blog - Wed, 01/18/2017 - 20:48
Posted by Lily Sheringham, Developer Marketing at Google Play


With just a few clicks, you can publish an app to Google Play and access a global audience of more than 1 billion 30 days active users. Finding success in global markets means considering how each market differs, planning for high quality localization, and tailoring your activity to the local audience. The new Going Global Playbook provides best practices and tips, with advice from developers who've successfully gone global.

This guide includes advice to help you plan your approach to going global, prepare your app for new markets, take your app to market, and also include data and insights for key countries and other useful resources.

This ebook joins others that we've recently published including The Building for Billions Playbook and The News Publisher Playbook. All of our ebooks are promoted in the Playbook for Developers app, which is where you can stay up to date with all the news and best practices you need to find success on Google Play.

How useful did you find this blogpost?

‚ėÖ ‚ėÖ ‚ėÖ ‚ėÖ ‚ėÖ                                                        
Categories: Programming

We’re Listening

Recently, the IEEE Computer Society’s popular SE Radio podcast included a sponsored advertising campaign that sparked a negative reaction among many of those involved with producing SE Radio, as well as among a number of listeners. In response to that reaction, the Computer Society has reviewed the advertisement and removed it from the podcast. In […]
Categories: Programming

We’re Listening

Recently, the IEEE Computer Society’s popular SE Radio podcast included a sponsored advertising campaign that sparked a negative reaction among many of those involved with producing SE Radio, as well as among a number of listeners. In response to that reaction, the Computer Society has reviewed the advertisement and removed it from the podcast. In […]
Categories: Programming

Meet the 20 finalists of the Google Play Indie Games Contest

Android Developers Blog - Wed, 01/18/2017 - 09:17
Posted by Matteo Vallone, Google Play Games Business Development

Back in November, we launched the Google Play Indie Games Contest for developers from 15 European countries, to celebrate the passion and innovation of the indie community in the region. The contest will reward the winners with exposure to industry experts and players worldwide, as well as other prizes that will showcase their art and help them grow their business on Android and Google Play.

Thank you to the nearly 1000 of you who submitted high quality games in all types of genres! Your creativity, enthusiasm and dedication have once again impressed us and inspired us. We had a very fun time testing and judging the games based on fun, innovation, design excellence and technical and production quality, and it was challenging to select only 20 finalists:

Meet the 20 finalists
(In alphabetical order)

Blind Drive
(coming soon)

Lo-Fi People
Israel Causality
(coming soon)

Loju
United Kingdom
Crap! I'm Broke: Out of Pocket
Arcane Circus Netherlands
Egz

Lonely Woof
France
Ellipsis

Salmi GmbH Germany


Gladiabots


GFX47
France
Happy Hop: Kawaii Jump

Platonic Games
Spain
Hidden Folks (coming soon)

Adriaan de Jongh Netherlands Lichtspeer
(coming soon)

Lichthund
Poland Lost in Harmony
Digixart

Entertainment France


Mr Future Ninja (coming soon)

Huijaus Studios
Finland Paper Wings


Fil Games
Turkey PinOut


Mediocre
Sweden
Power Hover


Oddrok
Finland
Reigns

Nerial
United Kingdom
Rusty Lake: Roots

Rusty Lake Netherlands
Samorost 3

Amanita Design Czech Republic
The Battle of Polytopia
Midjiwan AB Sweden


twofold inc.

Grapefrukt games Sweden
Unworded (coming soon)

Bento Studio France

Check out the prizes

All the 20 finalists are getting:
  • The opportunity to exhibit and showcase their game at the final event held at the Saatchi Gallery in London, on 16th February 2017.
  • Promotion of their game on a London billboard for one month.
  • Two tickets to attend a 2017 Playtime event. This is an invitation-only event for top apps and games developers on Google Play.
  • One Pixel XL smartphone.
At the event at Saatchi, the finalists will also have a chance to make it to the next rounds and win additional prizes, including:
  • YouTube influencer campaigns worth up to 100,000 EUR.
  • Premium placements on Google Play.
  • Tickets to Google I/O 2017 and other top industry events.
  • Promotions on our channels.
  • Special prizes for the best Unity game.
  • And more!

Come support them at the final event

At the final event attendees will have a say on which 10 of these finalists will get to pitch their games to the jury, who will decide on the final contest winners who will receive the top prizes.

Register now to join us in London, meet the developers, check out their great games, vote for your favourites, and have fun with various industry experts and indie developers.



A big thank you again to everyone who entered and congratulations to the finalists. We look forward to seeing you at the Saatchi Gallery in London on 16th February.
Categories: Programming

Cone of Uncertainty - Part Trois

Herding Cats - Glen Alleman - Wed, 01/18/2017 - 06:12

The notion¬†of the Cone of Uncertainty has been around for awhile. Barry Boehm's work in ‚ÄúSoftware Engineering Economics‚ÄĚ. Prentice-Hall, 1981. ¬†The poster below is from Steve McConnell's site and makes several things clear.

  • The Cone is a project management framework describing the uncertainty aspects of estimates (cost and schedule) and other project attributes (cost, schedule, and technical performance parameters). Estimates of cost, schedule, technical¬†performance on the left side of the cone have a lower probability of being precise and accurate than estimates on the right side of the cone. This is due to many reasons. One is levels of uncertainty¬†early in the project. Aleatory and Epistemic uncertainties, which create the risk to the success of the project. Other uncertainties that create risk include:
    • Unrealistic performance expectation with missing Measures of Effectiveness and Measures of Performance
    • Inadequate assessment of risks and unmitigated exposure to these risks with proper handling plans.
    • Unanticipated technical issues with alternative plans and solutions to maintain effectiveness
  • Since all project work contains uncertainty, reducing this uncertainty¬†- which reduces risk - is the role of the project team and their management. Either the team itself, the Project or Program Manager, or on larger programs the Risk Management owner.¬†

Here's a simple definition of the Cone of Uncertainty: 

The Cone of Uncertainty describes the evolution of the amount of uncertainty during a project. Uncertainty not only decreases over time passing, but it also diminishes its impact by risk management, specifically by decision-making.At the beginning of a project, comparatively little is known about the product or work results, and so estimates are subject to large uncertainty. As more research and development is done, more information is learned about the project, and the uncertainty then tends to decrease, reaching 0% when all residual risk has been terminated or transferred. This usually happens by the end of the project.

So the question is? - How much variance reduction needs to take place - in any and all the project attributes (risk, effectiveness, performance, cost, schedule - shown below) at what points in time, to increase the probability of project success.

This is the paradigm of the Cone of Uncertainty - it's a planned development compliance engineering tool, not an after the fact data collection tool

The Cone is NOT the result of the project's past performance. The Cone IS the Planned reduction of uncertainty as the project proceeds. When actual measures of cost, schedule, and technical performance are outside the planned cone of uncertainty, corrective actions must be taken to move those uncertanties inside the cone of uncertanty, if the project is going to meet it's cost, schedule, and technical performance goals. 

The Measure that is modeled in the Cone the variable is the Quantitative basis of a control process that establishes the goal for the performance measure. Capturing the actual performance, comparing it to the planned performance, and compliance with the upper and lower control limits provides guidance for making adjustments to maintain that variables performance with acceptable limits.

The Benefits of the Use of the Cone of Uncertainty 

The planned value, the upper and lower control limits, the measures of actual values form a Close Loop Control System - a measurement based feedback process to improve the effectiveness and efficiency of the project management processes by [1]

  • Analyzing trends that help focus on problem areas at the earliest point in time - when the¬†variable under control starts misbehaving, intervention can be taken. No need to wait till the end to find out¬†you're not going to make it
  • Providing early insight into error-prone products that can then be corrected earlier and thereby at lower cost - when the trends are headed to the UCL and LCL, intervention can take place.
  • Avoiding or minimizing cost overruns and schedule slips by detecting them early - by observing trends to breaches of the UCL and LCL.
    enough in the project to implement corrective actions
  • Performing better technical planning, and making adjustments to resources based on discrepancies between planned and actual progress

 

Screen Shot 2017-01-12 at 3.48.34 PM

A critical success factor for all project work is Risk Management. And risk management includes the managements of all kinds of risks. Risks from all kinds of sources of uncertainty, including technical risk, cost risk, schedule, management risk. Each of these uncertainties and the risks they produce can take on a range of values described by probability and statistical distribution functions. Knowing what ranges are possible and knowing what ranges are acceptable is a critical project success factor.

We need to know the Upper Control Limits (UCL) and Lower Control Limit (LCL) of the ranges of all the variables that will impact the success of our project. With this paradigm we have logically connected project management processes with Control System processesIf the variances, created by uncertainty going outside the UCL and LCL. Here's a work in progress paper "Is there an underlying Theory of Project Management," that addresses some of the issues with control of project activities.

Here are some examples of Planned variances and managing of the actual variances to make sure the project stays on plan.

A product weight as a function of the programs increasing maturity. In this case, the projected base weight is planned and the planned weights of each of the major subsystems are laid out as a function of time. Tolerance bands for the project base weight provide management with actionable information about the progression of the program. If the vehicle gets overweight, money and time are needed to correct the undesirable variance. This is a closed loop control system for managing the program with a Technical Performance Measure (TPM). There can be cost and schedule performance measures as well.

Screen Shot 2017-01-13 at 4.23.56 PM

Below is another example of a Weight reduction attribute that has error bands. In this example (an actual vehicle like the example above) the weight must be reduced as the program proceeds left to right. We have a target weight at Test Readiness Review of 23KG. A 25KG vehicle was sold in the proposal, and we need a target weight that has a safety margin, so 23KG is our target.

As the program proceeds, there are UCL and LCL bands that follow the planned weight.  The Orange dots are the actual weights from a variety of sources - a Design Model (3D Catia CAD system), a detailed design model, a bench scale model that can be measured, a non-flying prototype, and then the 1st Flight Article). As the program progresses each of the weight measurements for each of the models through to a final article is compared to the planned weight. We need to keep these values inside the error bands of NEEDED weight reduction if we are to stay on plan.

This is the critical concept in successful project management

We must have a Plan for the critical attributes - Mission Effectiveness, Technical Performance, Key Performance Parameters - for the items. If these are not compliant with the plan will one of the Root Causes of program performance shortfall. We must have a burndown or burnup plan for producing the end item deliverables for the program that match those parameters over the course of the program. Of course, we have a wide range of possible outcomes for each item in the beginning. And as the program proceeds the variances measures on those items move toward compliance of the target number in this case Weight.

Screen Shot 2017-01-13 at 4.21.56 PM

Here's another example of the Cone of Uncertainty, in this case, the uncertainty is the temperature of an oven being designed by an engineering team. The UCL and LCL are defined BEFORE the project starts. These are used to inform the designer of the progress of the project as it proceeds. Staying inside the controls limits is the Planned progress path to the final goal - in this case, temperature.

The Cone of Uncertanty, is the signaling boundaries of the Closed Loop Control system used to manage the project to success

Screen Shot 2017-01-13 at 4.38.04 PM

It turns out the cone can also be a flat range with Upper and Lower Control Limits of the variable that is being developed - a design to variable - in this example a Measure of Performance. In this case, a Measure of Performance that needs to stay within the Upper and Lower limits as the project progress through its gates. If this variable is out of bounds the project will have to pay in some way to get it back to Green

A Measure of Performance characterizes physical or functional attributes relating to the system operation, measured or estimated under specific conditions. Measures of Performance are (1) Attributes that assure the system has the capability and capacity to perform and (2) Assessment of the system to assure it meets design requirements to satisfy the Measures of Effectiveness.

Screen Shot 2017-01-15 at 7.37.49 PM

Another cone style is the cone of confidence in a delivery date. This Actual case it's an Actual Launch date. In this case, as the program moves from left to right, we need to assure that the Launch Date moves from a low confidence Date to a date that has a chance of being correct. The BLUE bars are the probabilistic ranges of the current estimate date. As the program moves forward those ranges must be reduced if we're going to show up as needed. The Planned date and a date with a margin are the build to dates. As the program moves the confidence of the date must increase and move toward the need date.

  • The probabilistic completion times change as the program matures
  • The efforts that produce these improvements must be defined and managed
  • The¬†error bands of the assessment points must include the¬†risk mitigation activities as well
  • The¬†planned activities show how the¬†error band narrows over time
    • This is the basis of a¬†risk tolerant plan
    • The probabilistic interval become more reliable as the risk mitigation and the maturity assessment add confidence to the planned¬†launch date

Just a reminder again - the Cone of Uncertainty is a DESIRED path, NOT the result of an unmanaged project outcome.

Risk Management, as shown below, is how Adults Manage Projects

Screen Shot 2017-01-13 at 7.09.21 AM

Wrap Up On the Misunderstanding of the Purpose and Value of the Cone of Uncertainty

When you hear... 

I have data that shows that uncertainty (or any other needed attribute) doesn't reduce and therefore the COU is a FAKE ... OR ... I see data on my projects where the variance is getting worse as we move forward, instead of narrowing as the Planned COU tells us it should be to meet our goals ...

...then that project is out of control,  starting with a missing steering target that means it's Open Loop Control and will be late, over budget, and likely not perform to the needed effectiveness and performance parameters. And when you see these out of control situations, go find the Root Cause and generate the Corrective Act. 

This data is an observation of a project not being managed as Tim Lister suggests - Risk Management is How Adults Manage Projects. 

And if these observations are taking place without corrective actions of the Root Causes of the performance shortfall, the management is behaving badly. Their just observers of the train wreck that is going to happen real soon.

The Engineering Reason for the Cone of Uncertainty Model and the Value it Provides the Designing Makers

The Cone of Uncertainty is NOT an output from the project's behaviour, by then that's too late.
It's a Steering Target Input to the Management Framework for increasing the probability of the project's success.
This is the Programmatic Management of the project in support of the Technical Management of the project. The processes is an engineering discipline. Systems Engineering, Risk Engineering, Safety and Mission Assurance Engineering, are typical roles where we work.
To suggest otherwise is to invert the paradigm and removes any value from the post-facto observations of the project's performance. At that point it's Too Late, the Horse has left and there's no getting him back.
Defining the planned and needed variance levels at planned points in the project is the basis of the closed loop control system needed increase the probability of success.
When variances outside the planned variance appear, the Root Cause of those must be found and corrective action take.

Resources

[1] Systems Engineering Measurement Primer, INCOSE

[2] System Analysis, Design, and Development Concepts, Principles, and Practices, Charles Wasson, John Wiley & Sons

[3] SMC Systems Engineering Primer & Handbook: Concept, Processes, and Techniques, Space & Missle Systems Center, U.S. Air Force

[4] Defense Acquisition Guide, Chapter 4, Systems Engineering, 15 May 2013.

[5] Program Managers Tool Kit, 16th Edition, Defense Acquisition University.

[6] "Open Loop / Close Loop Project Controls"

[7] "Reducing Estimation Uncertainty with Continuous Assessment: Tracking the 'Cone of Uncertainty',"¬†Pongtip Aroonvatanaporn, Chatchai Sinthop, Barry Boehm.¬†ASE‚Äô10, September 20‚Äď24, 2010, Antwerp, Belgium.¬†

[8]¬†Boehm, B. ‚ÄúSoftware Engineering Economics‚ÄĚ. Prentice-Hall, 1981.

[9] Boehm, B., Abts, C., Brown, A. W., Chulani, S., Clark, B. K., Horowitz, E., Madachy, R., Reifer, D. J., and Steece, B. Software Cost Estimation with COCOMO II, Prentice-Hall,
2000.
[10] Boehm, B., Egyed, A., Port, D., Shah, A., Kwan, J., and Madachy, R. "Using the WinWin Spiral Model: A Case Study," IEEE Computer, Volume 31, Number 7, July 1998, pp.  33-44 

[11] Cohn, M. Agile Estimating and Planning, Prentice-Hall, 2005

[12] DeMarco, T. Controlling Software Projects: Management, Measurement, and Estimation, Yourdon Press, 1982.

[13] Fleming, Q. W. and Koppelman, J. M. Earned Value Project Management, 2nd edition, Project Management Institute, 2000

[14] Galorath, D. and Evans, M. Software Sizing, Estimation, and Risk Management, Auer-bach, 2006

[15]Jorgensen, M. and Boehm, B. ‚ÄúSoftware Development Effort Estimation: Formal Models or Expert Judgment?‚ÄĚ IEEE Software, March-April 2009, pp. 14-19

[16] Jorgensen, M. and Shepperd, M. ‚ÄúA Systematic Review of Software Development Cost Estimation Studies,‚ÄĚ IEEE Trans. Software Eng., vol. 33, no. 1, 2007, pp. 33-53

[17] Krebs, W., Kroll, P., and Richard, E. Un-assessments ‚Äďreflections by the team, for the team. Agile 2008 Conference

[18] McConnell, S. Software Project Survival Guide, Microsoft Press, 1998

[19] Nguyen, V., Deeds-Rubin, S., Tan, T., and Boehm, B. "A SLOC Counting Standard," COCOMO II Forum 2007

[20] Putnam L. and Fitzsimmons, A. ‚ÄúEstimating Software Costs, Parts 1,2 and 3,‚ÄĚ Datamation, September through December 1979

[21] Stutzke, R. D. Estimating Software-Intensive Systems, Pearson Education, Inc, 2005. 

Related articles

Complex, Complexity, Complicated Economics of Software Development Herding Cats: Economics of Software Development Estimating Probabilistic Outcomes? Of Course We Can! I Think You'll Find It's a Bit More Complicated Than That Risk Management is How Adults Manage Projects

 

Categories: Project Management

Emotional Intelligence, Useful?

Lots of screens!

So many things to learn and so little time to do it!

I was asked why emotional intelligence was important and whether emotional intelligence can be learned.

With a little probing on the second part of the question, it was suggested that there was a school of thought that emotional intelligence is an inherent human attribute; you have it or you don‚Äôt. The ‚Äúeither you are or aren‚Äôt‚ÄĚ argument is similar those with a fixed mindset make about most capabilities. ¬†The concept of a fixed in the book Mindset written by Carol Dweck. In the book, Dweck argues that mindsets are not fixed and we have identified several attributes that comprise emotional intelligence that can be improved in the essay, A Few Steps To Improving Your Emotional Intelligence. I do not accept that emotional intelligence is a fixed human ability. Simply put, your emotional intelligence quotient is not fixed and can be learned.

The second question suggests that emotional intelligence, while interesting, is not useful in the day-to-day operation of an Agile team (or by extension an organization). Fortunately, you do not have to look far to find applications of emotional intelligence in many day-to-day scenarios, ranging from team meetings to sales. As a leader or coach, it is easy to identify scenarios when emotional intelligence can be useful.  Three examples are:   

Diffusing Problem Situations

Problems happen and most involve people.  The large problems, for example, an irate client or someone that is acting counter to the team needs, are often easily spotted once they have happened. However, they are often the result of an accumulation of little issues; minor abrasions can add up. Examples of minor issues might include an occasional bit of underperformance, a bad mood, some failing to wish you happy birthday, talking over you on occasion.  The list can go on. Emotional intelligence helps not only to recognize the problem but more importantly when combined with listening to those within the boundary of the problem helps everyone to unburden.  Understanding and listening are input into empathy which is needed to come up with a fitting solution. Emotional intelligence is a tool to defuse problem situations.

Curiosity

Two of the five competencies of emotional intelligence are awareness and self-awareness of emotions in yourself and others and secondly, the ability to construct relationships. These two competencies are typically reflected as a curiosity. In my re-read of Carol Dweck’s Mindset, one of the attributes of the growth mindset is insatiable need to learn and experience challenges, a related form of curiosity. Emotional intelligence is linked to the growth mindset through curiosity (at the very least).  Curiosity and the desire to learn are important capabilities in stable cross-functional teams.  Agile teams with emotionally intelligent members will be able to stretch to meet their customer needs because leveraging their curiosity they can identify what the skill they need to know, then learn new skills and discover new solutions. Curiosity may have killed the cat, but emotional intelligence fosters curiosity and learning will make the Agile team.

Repeat Clients

I have many friends that are fellow consultants, both independent and part of a larger organization. ¬†All of them are intelligent, all of them have great pedigrees and many of them are successful as independents. When consultants gather, the one conversation all consultants have is getting and keeping clients. ¬†I have observed that the consultants that have repeating/recurring clients have significantly more emotional intelligence than those that are great at getting clients but less so at generating repeat business. ¬†Emotional intelligence is a tool to build meaningful relationships that make it easier get repeat business. ¬†The essay, Emotional Intelligence: A Few Basics referenced Daniel Kahneman‚Äôs statement ¬†‚ÄĚthat people would rather do business with a person they like and trust rather than someone they don‚Äôt.‚ÄĚ ¬†Emotional intelligence makes it easier to build solid relationships that translate into repeat clients. Without emotional intelligence, it is difficult to generate the empathy needed to invest time into growing relationships based on anything other than sales volume.

Emotional intelligence is useful for identifying and defusing problems, generating relationships and to facilitate repeat sales. You are not born with all of the emotional intelligence that you can or will ever need.  Emotional intelligence is a reflection of a set of capabilities that can be improved.  We should invest the time, effort and money needed to get increase our emotional intelligence capability.    


Categories: Process Management

Emotional Intelligence, Useful?

Lots of screens!

So many things to learn and so little time to do it!

I was asked why emotional intelligence was important and whether emotional intelligence can be learned.

With a little probing on the second part of the question, it was suggested that there was a school of thought that emotional intelligence is an inherent human attribute; you have it or you don‚Äôt. The ‚Äúeither you are or aren‚Äôt‚ÄĚ argument is similar those with a fixed mindset make about most capabilities. ¬†The concept of a fixed in the book Mindset written by Carol Dweck. In the book, Dweck argues that mindsets are not fixed and we have identified several attributes that comprise emotional intelligence that can be improved in the essay, A Few Steps To Improving Your Emotional Intelligence. I do not accept that emotional intelligence is a fixed human ability. Simply put, your emotional intelligence quotient is not fixed and can be learned.

The second question suggests that emotional intelligence, while interesting, is not useful in the day-to-day operation of an Agile team (or by extension an organization). Fortunately, you do not have to look far to find applications of emotional intelligence in many day-to-day scenarios, ranging from team meetings to sales. As a leader or coach, it is easy to identify scenarios when emotional intelligence can be useful.  Three examples are:   

Diffusing Problem Situations

Problems happen and most involve people.  The large problems, for example, an irate client or someone that is acting counter to the team needs, are often easily spotted once they have happened. However, they are often the result of an accumulation of little issues; minor abrasions can add up. Examples of minor issues might include an occasional bit of underperformance, a bad mood, some failing to wish you happy birthday, talking over you on occasion.  The list can go on. Emotional intelligence helps not only to recognize the problem but more importantly when combined with listening to those within the boundary of the problem helps everyone to unburden.  Understanding and listening are input into empathy which is needed to come up with a fitting solution. Emotional intelligence is a tool to defuse problem situations.

Curiosity

Two of the five competencies of emotional intelligence are awareness and self-awareness of emotions in yourself and others and secondly, the ability to construct relationships. These two competencies are typically reflected as a curiosity. In my re-read of Carol Dweck’s Mindset, one of the attributes of the growth mindset is insatiable need to learn and experience challenges, a related form of curiosity. Emotional intelligence is linked to the growth mindset through curiosity (at the very least).  Curiosity and the desire to learn are important capabilities in stable cross-functional teams.  Agile teams with emotionally intelligent members will be able to stretch to meet their customer needs because leveraging their curiosity they can identify what the skill they need to know, then learn new skills and discover new solutions. Curiosity may have killed the cat, but emotional intelligence fosters curiosity and learning will make the Agile team.

Repeat Clients

I have many friends that are fellow consultants, both independent and part of a larger organization. ¬†All of them are intelligent, all of them have great pedigrees and many of them are successful as independents. When consultants gather, the one conversation all consultants have is getting and keeping clients. ¬†I have observed that the consultants that have repeating/recurring clients have significantly more emotional intelligence than those that are great at getting clients but less so at generating repeat business. ¬†Emotional intelligence is a tool to build meaningful relationships that make it easier get repeat business. ¬†The essay, Emotional Intelligence: A Few Basics referenced Daniel Kahneman‚Äôs statement ¬†‚ÄĚthat people would rather do business with a person they like and trust rather than someone they don‚Äôt.‚ÄĚ ¬†Emotional intelligence makes it easier to build solid relationships that translate into repeat clients. Without emotional intelligence, it is difficult to generate the empathy needed to invest time into growing relationships based on anything other than sales volume.

Emotional intelligence is useful for identifying and defusing problems, generating relationships and to facilitate repeat sales. You are not born with all of the emotional intelligence that you can or will ever need.  Emotional intelligence is a reflection of a set of capabilities that can be improved.  We should invest the time, effort and money needed to get increase our emotional intelligence capability.    


Categories: Process Management

Silence speaks louder than words when finding malware

Google Code Blog - Tue, 01/17/2017 - 23:06
Originally posted on Android Developer Blog

Posted by Megan Ruthven, Software Engineer
In Android Security, we're constantly working to better understand how to make Android devices operate more smoothly and securely. One security solution included on all devices with Google Play is Verify apps. Verify apps checks if there are Potentially Harmful Apps (PHAs) on your device. If a PHA is found, Verify apps warns the user and enables them to uninstall the app.

But, sometimes devices stop checking up with Verify apps. This may happen for a non-security related reason, like buying a new phone, or, it could mean something more concerning is going on. When a device stops checking up with Verify apps, it is considered Dead or Insecure (DOI). An app with a high enough percentage of DOI devices downloading it, is considered a DOI app. We use the DOI metric, along with the other security systems to help determine if an app is a PHA to protect Android users. Additionally, when we discover vulnerabilities, we patch Android devices with our security update system. This blog post explores the Android Security team's research to identify the security-related reasons that devices stop working and prevent it from happening in the future.
Flagging DOI Apps
To understand this problem more deeply, the Android Security team correlates app install attempts and DOI devices to find apps that harm the device in order to protect our users.
With these factors in mind, we then focus on 'retention'. A device is considered retained if it continues to perform periodic Verify apps security check ups after an app download. If it doesn't, it's considered potentially dead or insecure (DOI). An app's retention rate is the percentage of all retained devices that downloaded the app in one day. Because retention is a strong indicator of device health, we work to maximize the ecosystem's retention rate. Therefore, we use an app DOI scorer, which assumes that all apps should have a similar device retention rate. If an app's retention rate is a couple of standard deviations lower than average, the DOI scorer flags it. A common way to calculate the number of standard deviations from the average is called a Z-score. The equation for the Z-score is below.
  • N = Number of devices that downloaded the app.
  • x = Number of retained devices that downloaded the app.
  • p = Probability of a device downloading any app will be retained.

In this context, we call the Z-score of an app's retention rate a DOI score. The DOI score indicates an app has a statistically significant lower retention rate if the Z-score is much less than -3.7. This means that if the null hypothesis is true, there is much less than a 0.01% chance the magnitude of the Z-score being as high. In this case, the null hypothesis means the app accidentally correlated with lower retention rate independent of what the app does.
This allows for percolation of extreme apps (with low retention rate and high number of downloads) to the top of the DOI list. From there, we combine the DOI score with other information to determine whether to classify the app as a PHA. We then use Verify apps to remove existing installs of the app and prevent future installs of the app.
Difference between a regular and DOI app download on the same device.
Results in the wild
Among others, the DOI score flagged many apps in three well known malware families‚ÄĒ Hummingbad, Ghost Push, and Gooligan. Although they behave differently, the DOI scorer flagged over 25,000 apps in these three families of malware because they can degrade the Android experience to such an extent that a non-negligible amount of users factory reset or abandon their devices. This approach provides us with another perspective to discover PHAs and block them before they gain popularity. Without the DOI scorer, many of these apps would have escaped the extra scrutiny of a manual review.
The DOI scorer and all of Android's anti-malware work is one of multiple layers protecting users and developers on Android. For an overview of Android's security and transparency efforts, check out our page.
Categories: Programming

Silence speaks louder than words when finding malware

Android Developers Blog - Tue, 01/17/2017 - 22:59
Posted by Megan Ruthven, Software Engineer
In Android Security, we're constantly working to better understand how to make Android devices operate more smoothly and securely. One security solution included on all devices with Google Play is Verify apps. Verify apps checks if there are Potentially Harmful Apps (PHAs) on your device. If a PHA is found, Verify apps warns the user and enables them to uninstall the app.
But, sometimes devices stop checking up with Verify apps. This may happen for a non-security related reason, like buying a new phone, or, it could mean something more concerning is going on. When a device stops checking up with Verify apps, it is considered Dead or Insecure (DOI). An app with a high enough percentage of DOI devices downloading it, is considered a DOI app. We use the DOI metric, along with the other security systems to help determine if an app is a PHA to protect Android users. Additionally, when we discover vulnerabilities, we patch Android devices with our security update system.

This blog post explores the Android Security team's research to identify the security-related reasons that devices stop working and prevent it from happening in the future.
Flagging DOI Apps

To understand this problem more deeply, the Android Security team correlates app install attempts and DOI devices to find apps that harm the device in order to protect our users.
With these factors in mind, we then focus on 'retention'. A device is considered retained if it continues to perform periodic Verify apps security check ups after an app download. If it doesn't, it's considered potentially dead or insecure (DOI). An app's retention rate is the percentage of all retained devices that downloaded the app in one day. Because retention is a strong indicator of device health, we work to maximize the ecosystem's retention rate.

Therefore, we use an app DOI scorer, which assumes that all apps should have a similar device retention rate. If an app's retention rate is a couple of standard deviations lower than average, the DOI scorer flags it. A common way to calculate the number of standard deviations from the average is called a Z-score. The equation for the Z-score is below.

  • N = Number of devices that downloaded the app.
  • x = Number of retained devices that downloaded the app.
  • p = Probability of a device downloading any app will be retained.

In this context, we call the Z-score of an app's retention rate a DOI score. The DOI score indicates an app has a statistically significant lower retention rate if the Z-score is much less than -3.7. This means that if the null hypothesis is true, there is much less than a 0.01% chance the magnitude of the Z-score being as high. In this case, the null hypothesis means the app accidentally correlated with lower retention rate independent of what the app does.
This allows for percolation of extreme apps (with low retention rate and high number of downloads) to the top of the DOI list. From there, we combine the DOI score with other information to determine whether to classify the app as a PHA. We then use Verify apps to remove existing installs of the app and prevent future installs of the app.

Difference between a regular and DOI app download on the same device.


Results in the wild
Among others, the DOI score flagged many apps in three well known malware families‚ÄĒ Hummingbad, Ghost Push, and Gooligan. Although they behave differently, the DOI scorer flagged over 25,000 apps in these three families of malware because they can degrade the Android experience to such an extent that a non-negligible amount of users factory reset or abandon their devices. This approach provides us with another perspective to discover PHAs and block them before they gain popularity. Without the DOI scorer, many of these apps would have escaped the extra scrutiny of a manual review.
The DOI scorer and all of Android's anti-malware work is one of multiple layers protecting users and developers on Android. For an overview of Android's security and transparency efforts, check out our page.


Categories: Programming

Sponsored Post: Contentful, Stream, Loupe, New York Times, Scalyr, VividCortex, MemSQL, InMemory.Net, Zohocorp

Who's Hiring?
  • Contentful is looking for a JavaScript BackEnd Engineer to join our team in their mission of getting new users - professional developers - started on our platform within the shortest time possible. We are a fun and diverse family of over 100 people from 35 nations with offices in Berlin and San Francisco, backed by top VCs (Benchmark, Trinity, Balderton, Point Nine), growing at an amazing pace. We are working on a content management developer platform that enables web and mobile developers to manage, integrate, and deliver digital content to any kind of device or service that can connect to an API. See job description.

  • The New York Times is looking for a Software Engineer for its Delivery/Site Reliability Engineering team. You will also be a part of a team responsible for building the tools that ensure that the various systems at The New York Times continue to operate in a reliable and efficient manner. Some of the tech we use: Go, Ruby, Bash, AWS, GCP, Terraform, Packer, Docker, Kubernetes, Vault, Consul, Jenkins, Drone. Please send resumes to: technicaljobs@nytimes.com
Fun and Informative Events
  • Your event here!
Cool Products and Services
  • Build, scale and personalize your news feeds and activity streams with getstream.io. Try the API now in this 5 minute interactive tutorial. Stream is free up to 3 million feed updates so it's easy to get started. Client libraries are available for Node, Ruby, Python, PHP, Go, Java and .NET. Stream is currently also hiring Devops and Python/Go developers in Amsterdam. More than 400 companies rely on Stream for their production feed infrastructure, this includes apps with 30 million users. With your help we'd like to ad a few zeros to that number. Check out the job opening on AngelList.

  • A note for .NET developers: You know the pain of troubleshooting errors with limited time, limited information, and limited tools. Log management, exception tracking, and monitoring solutions can help, but many of them treat the .NET platform as an afterthought. You should learn about Loupe...Loupe is a .NET logging and monitoring solution made for the .NET platform from day one. It helps you find and fix problems fast by tracking performance metrics, capturing errors in your .NET software, identifying which errors are causing the greatest impact, and pinpointing root causes. Learn more and try it free today.

  • Scalyr is a lightning-fast log management and operational data platform.  It's a tool (actually, multiple tools) that your entire team will love.  Get visibility into your production issues without juggling multiple tabs and different services -- all of your logs, server metrics and alerts are in your browser and at your fingertips. .  Loved and used by teams at Codecademy, ReturnPath, Grab, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net

  • VividCortex is a SaaS database monitoring product that provides the best way for organizations to improve their database performance, efficiency, and uptime. Currently supporting MySQL, PostgreSQL, Redis, MongoDB, and Amazon Aurora database types, it's a secure, cloud-hosted platform that eliminates businesses' most critical visibility gap. VividCortex uses patented algorithms to analyze and surface relevant insights, so users can proactively fix future performance problems before they impact customers.

  • MemSQL provides a distributed in-memory database for high value data. It's designed to handle extreme data ingest and store the data for real-time, streaming and historical analysis using SQL. MemSQL also cost effectively supports both application and ad-hoc queries concurrently across all data. Start a free 30 day trial here: http://www.memsql.com/

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network. 

If any of these items interest you there's a full description of each sponsor below...

Categories: Architecture