Warning: Table './devblogsdb/cache_page' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_page WHERE cid = 'http://www.softdevblogs.com/?q=aggregator/categories/1&page=5' in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc on line 135

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 729

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 730

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 731

Warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/bootstrap.inc on line 732
Software Development Blogs: Programming, Software Testing, Agile, Project Management
Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Programming
warning: Cannot modify header information - headers already sent by (output started at /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/database.mysql.inc:135) in /home/content/O/c/n/Ocnarfparking9/html/softdevblogs/includes/common.inc on line 153.

R: Error in approxfun(x.values.1, y.values.1, method = “constant”, f = 1, : zero non-NA points

Mark Needham - Sun, 12/27/2015 - 13:24

I’ve been following Michy Alice’s logistic regression tutorial to create an attendance model for London dev meetups and ran into an interesting problem while doing so.

Our dataset has a class imbalance i.e. most people RSVP ‘no’ to events which can lead to misleading accuracy score where predicting ‘no’ every time would lead to supposed high accuracy.

Source: local data frame [2 x 2]
 
  attended     n
     (dbl) (int)
1        0  1541
2        1    53

I sampled the data using caret‘s upSample function to avoid this:

attended = as.factor((df %>% dplyr::select(attended))$attended)
upSampledDf = upSample(df %>% dplyr::select(-attended), attended)
upSampledDf$attended = as.numeric(as.character(upSampledDf$Class))

I then trained a logistic regression model but when I tried to plot the area under the curve I ran into trouble:

p <- predict(model, newdata=test, type="response")
pr <- prediction(p, test$attended)
prf <- performance(pr, measure = "tpr", x.measure = "fpr")
 
Error in approxfun(x.values.1, y.values.1, method = "constant", f = 1,  : 
  zero non-NA points

I don’t have any NA values in my data frame so this message was a bit confusing to start with. As usual Stack Overflow came to the rescue with the suggestion that I was probably missing positive/negative values for the independent variable i.e. ‘approved’.

A quick count on the test data frame using dplyr confirmed my mistake:

> test %>% count(attended)
Source: local data frame [1 x 2]
 
  attended     n
     (dbl) (int)
1        1   582

I’ll have to randomly sort the data frame and then reassign my training and test data frames to work around it.

Categories: Programming

Should I Develop My Dream Project Myself Or Should I Outsource It?

Making the Complex Simple - John Sonmez - Thu, 12/24/2015 - 17:00

There comes a time in our lives when we have that “BIG IDEA”. However, this big idea might follow some obstacles along the way. What should you do when developing your dream project? Should you do it all by yourself, wasting time and working on your free hours… Or should you outsource it to a […]

The post Should I Develop My Dream Project Myself Or Should I Outsource It? appeared first on Simple Programmer.

Categories: Programming

Interpreting Scala in the Browser

Xebia Blog - Thu, 12/24/2015 - 15:38


At this years' Scala Exchange keynote, Jessica Kerr got a nice round of applause when she asked us all to, please, include our import statements when sharing code snippets.

That's great advice - but can't we do better? What if we could share actual running, hackable code?

Turns out we can: there's a number of 'JSFiddle'-style projects around, and I'll take a brief look at some of them.

Scala-Js-FiddleSelection_045

Scala-Js-Fiddle leverages Scala.js: once compiled on the server, your Scala application runs in the browser.

This allows you to create beautiful visuals while running your applications. This could be an awesome teaching tool.  Be sure to check out their documentation, which comes generated on-the-fly by a Scala snippet!

The downside of using Scala.js is of course that you're restricted in which libraries you can include.

ScripsterSelection_046

Scripster by Razvan Cojocaru (github) allows you to write Scala code in a syntax-highlighted environment, and gives some control over which expressions are evaluated.

Even though this tool is not restricted to Scala.js, only a limited number of libraries is included.

Scala KataSelection_047

Scala Kata by Guillaume Massé (github) is an attractive editor with completion features, which feels much smoother than Scripster.

While again you don't have control over which dependencies are available, a nice collection is included.

Unlike other options Scala Kata does not show the output of the program, but the value of the expression on each line, which might be familiar if you've ever worked with the 'notebook' feature of Scala IDE. While sometimes constraining, this encourages a nicely functional style.

Scala Kata is currently my online Scala environment of choice, though unfortunately it doesn't provide an easy way to store and share your snippets.

Scala NotebookSelection_048

Scala Notebook looks pretty clean, but unfortunately does not appear to be available in hosted form anywhere.

The project takes its inspiration from the IPython notebook, which is indeed a great example.

An interesting aspect of Scala Notebook is that, somewhat like scala-js-fiddle, it allows rich output: when a command returns HTML or an image, it will be rendered/shown.

scastieSelection_049

Users of scastie will have to do without fancy features such as advanced highlighting and completion, but gives you complete control over the sbt configuration - making it a very powerful alternative.

 

Round-up Overview intermediate values completion git(hub) dependencies scala version Scala-Js-Fiddle no no yes no n/a Scripster no no no limited 2.9.1 Scala Kata yes yes no limited 2.11.7 Scala Notebook yes no no full control scastie no no no full control configurable The future

Some desirable features, like a more 'direct' REPL-like interface, are not widely supported. Also, none of the available offerings appear to be particularly easy to embed into an existing site or blog.

If these services would become wildly popular, I wonder whether making sure there is sufficient back-end computing power available might become a challenge. On the other hand, computing is rapidly becoming cheaper, so this might not be an issue after all.

Concluding

I was amazed by the number of projects that managed to make a functional Scala programming environment available in the browser. While there's more work to be done, there is amazing potential here.

What I Wish I Knew Before I Jumped Into iOS Part II

Making the Complex Simple - John Sonmez - Wed, 12/23/2015 - 14:00

If you haven’t read the original post, please read it and come back to this. In Part I, I wrote about the fundamental aspects of learning iOS development. This part will be more focused on branding and networking via Twitter and Meetup, and the tools and challenges you’ll face. Join Twitter and Become Part of […]

The post What I Wish I Knew Before I Jumped Into iOS Part II appeared first on Simple Programmer.

Categories: Programming

Uncovering the mysteries of Swift property observers

Xebia Blog - Wed, 12/23/2015 - 11:31

One of the cool features of Swift are property observers, perhaps better known as the willSet and didSet. Everyone programming in Swift must have used them. Some people more than others. And some people might use them a little bit too much, changing many of them together (me sometimes included). But it’s not always completely obvious when they are called. Especially when dealing with struct, because structs can be a bit odd. Let’s dive into some situations and see what happens.

Assignment

The most obvious situation in which didSet (and willSet) gets called is by simply assigning a variable. Imagine the following struct:

struct Person {
    var name: String
    var age: Int
}

And some other code, like a view controller that is using it in a variable with a property observer.

class MyViewController: UIViewController {
    var person: Person! {
        didSet {
            print(&quot;Person got set to '(person.name)' with age (person.age)&quot;)
        }
    }

    override func viewDidLoad() {
        person = Person(name: &quot;Bob&quot;, age: 20)
    }
}

As you would expect, the code in didSet will be executed when the view did load and the following is printed to the console:

Person got set to 'Bob' with age 20
Initialization

This one is also pretty clear and you probably already know about it: property observers are not executed then the variable is assigned during initialization.

var person = Person (name: "Bob", age: 20) {
        didSet {
            print("Person got set to '(person.name)' with age (person.age)")
        }
    }

This doesn’t print anything to the console. Also if you would assign person in the init function instead the willSet will not get called.

Modifying structs

A thing less known is that property observers are also executed when you change the member values of structs without (re)assigning the entire struct. The following sample illustrates this.

class ViewController: UIViewController {

    var person = Person (name: "Bob", age: 20) {
        didSet {
            print("Person got set to '(person.name)' with age (person.age)")
        }
    }

    override func viewDidLoad() {
        person.name = "Mike"
        person.age = 30
    }

}

In this sample we never reassign the person in our viewDidLoad function but by changing the name and age, the willSet still gets executed twice and we get as output:

Person got set to 'Mike' with age 20
Person got set to 'Mike' with age 30
Mutating functions

The same that applies to changing values of a struct also applies to mutating struct functions. Calling such a function always results in the property observers being called once. It does not matter if you replace the entire struct (by assigning self), change multiple member values or don’t change anything at all.

struct Person {
    var name: String
    var age: Int

    mutating func incrementAge() {
        if age < 100 {
            age++
        }
    }
}

Here we added a mutating function that increments the age as long as the age is lower than 100.

class ViewController: UIViewController {

    var person = Person (name: "Bob", age: 98) {
        didSet {
            print("Person got set to '(person.name)' with age (person.age)")
        }
    }

    override func viewDidLoad() {
        person.incrementAge()
        person.incrementAge()
        person.incrementAge()
        person.incrementAge()
    }

}

Our willSet is called 4 times, even though the last two times nothing has changed.

Person got set to 'Bob' with age 99
Person got set to 'Bob' with age 100
Person got set to 'Bob' with age 100
Person got set to 'Bob' with age 100
Changes inside property observers

It is also possible to make changes to the variable inside its own property observers. You can reassign the entire variable, change its values or call mutating functions on it. When you do that from inside a property observer, the property observers do not trigger since that would most likely cause an endless loop. Keep in mind that changing something in willSet will be without effect since your change will be overwritten by that value that was being set originally (this gives a nice warning in Xcode as well).

class ViewController: UIViewController {

    var person = Person (name: "Bob", age: 98) {
        didSet {
            print("Person got set to '(person.name)' with age (person.age)")
            if person.name != oldValue.name {
                person.age = 0
                print("Person '(person.name)' age has been set to 0")
            }
        }
    }

    override func viewDidLoad() {
        person.name = "Mike"
    }

}
Why it matters

So why does all this matter so much you might think. Well, you might have to rethink what kind of logic you put into your property observers and which you put outside. And all this applies to Arrays and Dictionaries as well, because they are also structs. Let’s say you have an array of numbers that can change and each time it changes you want to update your UI. But you also want to sort the numbers. The following code might look fine to you at first:

class ViewController: UIViewController {

    var numbers: [Int] = [] {
        didSet {
            updateUI()
        }
    }

    override func viewDidLoad() {
        refreshNumbers()
    }

    func refreshNumbers() {
        numbers = [random() % 10, random() % 10, random() % 10, random() % 10]
        numbers.sortInPlace()
    }

    func updateUI() {
        print("UI: (numbers)")
    }

}

Every time numbers changes, the UI will be updated. But since sortInPlace will also trigger the property observer, the UI gets updated twice:

UI: [3, 6, 7, 5]
UI: [3, 5, 6, 7]

So we should really put sortInPlace inside willSet right before we call updateUI.

Mobile Bootcamp Part III: Focus on quality

Google Code Blog - Tue, 12/22/2015 - 18:16

Originally posted on DoubleClick Publisher Blog

At this point, you’ve read our recommendations for growing your app’s audience and keeping users engaged. But there are technical aspects, too, that can make or break your app’s success. Speed, for example, is crucial -- users expect apps to respond in under one second.

Yesterday, we gave recommendations for attracting, engaging, and retaining your users. Continue reading today to learn tips from Dev Gogate, Mobile Solutions Consulting Manager at Google, on providing your users with a high quality app.

  1. The smaller the app size, the better: Users have limited bandwidth and data storage, so a large app runs the risk of losing an install. App stores impose size limits and will only deliver apps under a certain size over cellular connections; Google has a 50MB limit for each APK (but allows 2GB expansion files for supporting media) and Apple limits app size at 100MB. Use the ProGuard tool to optimize your Android code and enable Resource Shrinking to remove unused and unneeded resources from dependency libraries. App Thinning offers a similar solution for decreasing app size on iOS.

  2. Build apps that use device resources optimally: Building apps without taking into account how display interacts with hardware components can drain precious resources, such as battery life. According to Dev, "App performance is critical! A badly designed app can drastically and negatively impact the the user's perception of the app's usability and will surely lead to an uninstall." Dev recommends compiling a list of metrics for your apps to perform against and then running tests. This YouTube Android performance channel and Google+ community offer excellent tips on building high performing apps.

  3. Ensure your app works across devices: Failing to catch a bug or crash that only impacts some devices quickly leads to angry customer feedback and low ratings. Dev recommends using Cloud Test Lab to simultaneously test your app on nearly every brand and model of Android devices and across multiple languages, screen orientations, and Android API versions. Dev’s expert tip: Even if you have not written any instrumentation tests for your app, you can use Robo tests to exercise your app by simulating user actions.

  4. Use A/B testing to get real user feedback: Dogfooding your app provides early insight into how users interact with your app and how it behaves beyond the test lab. Beta testing, on the other hand, gathers feedback from an enthusiastic audience. Since this testing is customer-facing, ensure the process is simple to ensure high user participation. Both of these tests gives you early feedback from users before widely releasing your app.

  5. Launch new app versions in phases and closely monitor: Don’t underestimate the number of users who may adopt a new version of your app and their levels of usage. Dev recommends using staged rollouts, which allow you to beta test different app versions and gradually release app updates with a chosen user group. The Cloud Test Lab provides access to virtual devices that give rapid feedback during development, as well as physical devices that pinpoint issues found on real, physical devices.

Creating a high quality app is essential for building and maintaining a loyal user base. Follow Dev’s recommendations to optimize your app quality and stay posted for our post tomorrow on ways to effectively monetize.

Categories: Programming

How Twitter used Doze in Android 6.0 Marshmallow to Improve Notification Performance

Google Code Blog - Mon, 12/21/2015 - 22:37

Posted by Laurence Moroney, Developer Advocate

In October, we published a blog post about how Google Cloud Messaging (GCM) works with Doze in Android 6.0 Marshmallow to bring performance and usability improvements to the user. In this scenario, normal priority messages allow the device to stay in Doze, while high priority messages take the device out of Doze. For a great example of this capability and best practices on Android, we’ve spoken with the engineering team at Twitter.

Twitter has researched extensively which messages should get high priority -- given that this priority can awaken a device in Doze. To meet their particular need, Twitter wanted Direct Messages to be delivered on high priority, where mentions would be at normal priority.

Luke Millar, Android Engineering Manager at Twitter commented "With Android M we have a greater ability to be good citizens when it comes to conserving battery life. This feature lets us specify which notifications will wake up a device in Doze and which ones won't. Normally, we tell GCM to wait to deliver push notifications until the next time the user's phone wakes up. However, users expect to receive some push notifications immediately, like notifications for Direct Messages, so we set those as high priority. It's nice being able to specify when and how those notifications make it to our users."

To test Doze, Twitter’s engineers followed the steps outlined on the Android Developers site. These guide you to use adb commands to simulate doze on either a physical or virtual device. Following this method, Twitter was able to successfully test how their messaging priorities would work in a real-world environment.

One other best practice that Twitter used was in transmitting larger payloads, such as when delivering Twitter Highlights. In this case, they opted to pass metadata in the notification, which was then used to sync the application to get the requested Highlights. In other words, they didn’t transmit the contents of Highlights in a notification, instead triggering a sync mechanism to update the contents of the app.

For more details on using GCM with your app in standby environments, check out the documentation.
You can learn more about Google Cloud Messaging and how to use it for notifications in your Android, iOS and Web Applications at the Google Developers site here.

Categories: Programming

Security is maturing in the Docker ecosystem

Xebia Blog - Mon, 12/21/2015 - 21:34

Security is probably one of the biggest subjects when it comes to containers. Developers love containers, some ops do as well. But it most of the time boils down to the security aspects of containers. Is it safe to use, what if someone breaks out? The characteristics of containers which we love, could also be a weak spot when it comes to security. In this blog I want to show some common methods to establish a defence in depth around your containers. This is container-specific, so I won't be talking about locking down the host nodes or reducing the attack surface i.e. by disabling Linux daemons.

Read-only containers (Docker 1.5)
First up, the possibility to run a read-only container. By specifying --read-only, the container's rootfs will be mounted read-only so no process inside the container can write to it. This means when you have a vulnerability inside your app allowing to upload files, this is blocked by marking the containers rootfs as read-only. This will also block applications to log to the rootfs, so you may want to use a remote logging mechanism or a volume for this.

Usage (docs):
$ docker run --read-only -v /icanwrite busybox touch /icanwrite here

User-namespaces (Experimental)

Lots of people are waiting for this one to land in stable. Currently, being root in the container will mean you are also root on the host. If you are able to mount /bin inside your container, you can add whatever you want in there, and possible take over the host system. With the introduction of user-namespaces, you will be able to run containers where the root user inside the container will still have privileged capabilities but outside the container the uid:gid will be remapped to a non-privileged user/group. This is also know as phase 1, remapped root per daemon instance. A possible next phase could be full maps and per container mapping, but this is still under debate.

Usage (docs):

$ docker daemon --userns-remap=default

Seccomp (Git master branch)

With namespaces we have separation, but we also would like to control what can happen inside a running container. That's where seccomp comes into play. Seccomp is short for secure computing mode. It allows you to filter syscalls, so you define the syscalls your application needs, and all the other will be denied. A quick example, given socket.json:

{
"defaultAction": "SCMP_ACT_ALLOW",
"syscalls": [
{
"name": "socket",
"action": "SCMP_ACT_ERRNO"
}
]
}

will result in the following:

# docker run -ti --rm --security-opt seccomp:tcpsocket.json ubuntu bash
root@54fd6641a219:/# nc -l 555
nc: Operation not permitted

Project Nautilus

One of the missing pieces in the eco-system was checking image contents. There was a great buzz around this when an article was published stating that there were common vulnerabilities in over 30% of the official images on the Docker hub. Docker got to work, and have been scanning a lot of official images on the background on the Docker Hub before they published anything about it. During Dockercon EU, they announced Project Nautilus, an image-scanning service from Docker that makes it easier to build and consume high-integrity content.

There is not a lot official about Nautilus yet, we know it has been running in the background and Docker says they secured over 74 million pulls with it. Recently, they created a survey asking questions about how it could be used so I can only give you some assumptions. First up, what Docker says it does:

  • Image security
  • Component inventory/license management
  • Image optimization
  • Basic functional testing

Here are some pointers on things that may be coming soon:

  • Running Nautilus on-premise
  • Pricing may be on per image or per deployed node

nautilus

AppArmor profiles

By using AppArmor you can restrict capabilities by using profiles. The profiles can be really fine-grained, but a lot of people don't want to take the time to write these profiles. These profiles can be used on running Docker containers. Jessie Frazelle, one of the core maintainers of Docker, created bane to make writing these profiles easier. It can use a toml input file, and will generate and install an AppArmor profile. This profile can then be used on running a Docker container, in the same syntax as stated before:

docker run -d --security-opt="apparmor:name_of_profile" -p 80:80 nginx

Docker Security Profiles

These are all parts to secure your containers, of course Docker is also working on making this as easy to use as possible. This means  If you want to know more on where this is heading, check out the proposal on Github to keep yourself up-to-date.

Mobile Bootcamp Part II: Engage your users

Google Code Blog - Mon, 12/21/2015 - 18:03

Originally Posted on DoubleClick Publisher Blog

Did you know that 52% of all apps lose at least half of their most valuable users after three months?1 In this environment, how can you ensure your app effectively attracts, engages, and retains users?

Yesterday, we kicked off our mobile bootcamp to help get your app in shape for increased traffic and monetization opportunities during the holiday season. Building on mobile expert Cheney Tsai’s recommendations for growing your app’s audience, Jason Rosenblum, Mobile Solutions Consultant at Google, shares tips for effectively engaging users.

  1. Optimize your on-boarding experience: The first-run of your app is critical in selling its value and utility. Jason suggests using A/B testing to experiment with different features, such as adding an app tutorial or custom discount code.

  2. Measure user engagement and understand how it connects to your business: Knowing whether, when, and how frequently users return to your site allows you to strategically send notifications, identify pain points, and launch features. Cohort analysis indicates patterns in user behavior by aligning all metrics by the different days of user experience, regardless of the specific day a user starts using your app.

    While Google Analytics offers standard measurement options -- average session duration, active users, loyalty, recency, audience overview -- consider the nature of your business to pinpoint significant custom metrics. Jason explains, "You need your analytics to make sense for your business. For example, if you're running a hotel booking site, you want to know the number of hotel searches being run and the number of hotel rooms being booked. Customize your analytics for your business so you can drive the right KPIs."

  3. Remind users of your app’s value: 34% of app abandonment is fueled by a loss of interest, so create rich, contextual notifications like latest news, weather, or reminders to re-engage and increase your daily/monthly users. Google Cloud Messaging, for example, allows you to implement and manage notifications across Android and iOS. Jason advises using rich notifications to add quick actions, such as the ability to Archive an email via gMail notifications, and allowing users to control whether and how often they receive notifications.

The total time spent in apps per smartphone user is increasing, yet the total number of apps used has not changed. Users, therefore, are consolidating their time in certain favored apps. Incorporate Jason’s recommendations to keep your users engaged and come back tomorrow to hear from Dev Gogate, Mobile Solutions Consulting Manager at Google, to maximize your app quality.

Categories: Programming

tlhIngan pejatlh (I Speak Klingon) – Programming in Many Languages

Making the Complex Simple - John Sonmez - Mon, 12/21/2015 - 14:00

Admittedly, I actually used this to get the Latin Alphabet translation for this title. I got the idea for the title because of Weird Al’s White & Nerdy. While I do speak at least some JavaScript, my proficiency in Klingon is a definite cause for dishonor. Perhaps like many of you, over the course of […]

The post tlhIngan pejatlh (I Speak Klingon) – Programming in Many Languages appeared first on Simple Programmer.

Categories: Programming

Python: Squashing ‘duplicate’ pairs together

Mark Needham - Sun, 12/20/2015 - 13:12

As part of a data cleaning pipeline I had pairs of ids of duplicate addresses that I wanted to group together.

I couldn’t work out how to solve the problem immediately so I simplified the problem into pairs of letters i.e.

A	B		(A is the same as B)
B	C		(B is the same as C)
C	D		...
E	F		(E is the same as F)
F	G		...

The output that I want to get is:

(A, B, C, D)
(E, F, G)

I spent several hours trying to come up with a clever data structure to do this until Reshmee suggested tracking the sets of duplicates using an array of arrays or list of lists since we’re going to script this using Python.

The actual data is in a CSV file but we’ll create a list of tuples to save ourselves some work:

pairs = [ ("A", "B"), ("B", "C"), ("C", "D"), ("E", "F"), ("F", "G") ]

We’re going to iterate through the list of pairs and on each iteration we’ll check if there’s an entry in the list containing either of the values. There can be three outcomes from this check:

  1. No entry – we’ll add a new entry with our pair of values.
  2. One entry – we’ll add the other value to that entry.
  3. Two entries – we’ll merge them together replacing the existing entry.

The first step is to write a function to check the list of lists for a matching pair:

def find_matching_index(pair, dups):
    return [index
            for index, dup in enumerate(dups)
            if pair[0] in dup or pair[1] in dup]
 
print find_matching_index(("A", "B"), [set(["D", "E"])])
[]
 
print find_matching_index(("B", "C"), [set(["A", "B"])])
[0]
 
print find_matching_index(("B", "C"), [set(["A", "B"]), set(["C", "D"])])
[0, 1]

Next we need to write a function which iterates over all our pairs of values and uses find_matching_index to work out which decision to make:

def extract_groups(items):
    dups = []
    for pair in items:
        matching_index = find_matching_index(pair, dups)
 
        if len(matching_index) == 0:
            dups.append(set([pair[0], pair[1]]))
        elif len(matching_index) == 1:
            index = matching_index[0]
            matching_dup = dups[index]
            dups.pop(index)
            dups.append(matching_dup.union([pair[0], pair[1]]))
        else:
            index1, index2 = matching_index
            dup1 = dups[index1]
            dup2 = dups[index2]
 
            dups.pop(index1)
            dups.pop(index2 - 1) # the index decrements since we removed one entry on the previous line
            dups.append(dup1.union(dup2))
    return dups

Now let’s run this with a few test cases:

test_cases = [
    [ ("A", "B"), ("B", "C"), ("C", "D"), ("E", "F"), ("F", "G") ],
    [ ("A", "B"), ("B", "C"), ("C", "D"), ("E", "F"), ("F", "G"), ("G", "A"), ("G", "Z"), ("B", "D") ],
    [ ("A", "B"), ("B", "C"), ("C", "E"), ("E", "A") ],
    [ ("A", "B"), ("C", "D"), ("F", "G"), ("H", "I"), ("J", "A") ]
]
 
for test_case in test_cases:
    print extract_groups(test_case)
 
[set(['A', 'C', 'B', 'D']), set(['E', 'G', 'F'])]
[set(['A', 'C', 'B', 'E', 'D', 'G', 'F', 'Z'])]
[set(['A', 'C', 'B', 'E'])]
[set(['C', 'D']), set(['G', 'F']), set(['I', 'H']), set(['A', 'J', 'B'])]

This certainly doesn’t scale very well but since I only have a few hundred duplicate addresses it does the job for me.

It feels like there should be a more functional way to write these functions without mutating all these lists but I haven’t figured out what that is yet.

Categories: Programming

Leveraging product flavors in Android Studio for hermetic testing

Android Developers Blog - Fri, 12/18/2015 - 23:05

Posted by Jose Alcérreca, Developer Programs Engineer and Wojtek Kaliciński, Developer Advocate



During our recent talk at Android Dev Summit, we discuss the state of testing on Android on the example of a simple Notes application that we created as part of our testing codelab. In one section of the talk, we discuss the problem of test flakiness and introduce a simple solution for setting up a hermetic testing environment.

Combatting flaky tests

UI tests with frameworks like Espresso or UI Automator can easily get flaky if the application has external dependencies that can sometimes fail or take long to respond. In short, flaky tests are tests that are not reliable (they can pass or fail randomly) which defeats the whole purpose of having tests in the first place.

A common solution to this problem is hermetic testing or, in other words, making sure that your tests are isolated from dependencies. Using fake implementations or fake servers that just return predefined data is a common way of dealing with this problem. Here are some good examples:

  • Network calls can be made via a fake API client or fake server that immediately returns data from files stored on disk instead of hitting the network. This bypasses the network's latency and flakiness as well as any errors originating on the real server.
  • Interactions with low-level framework APIs (especially those for accessing hardware, like camera or storage) can be passed through an intermediate interface. A fake implementation of this interface could return immediately, providing a reference to preloaded data, such as an image, without relying on the hardware.
  • Any sensors can be replaced with fakes too: GPS, microphone, accelerometer etc., giving you a chance to test on data that would be difficult to provide for the test in real life, like a preset location or a set of inputs simulating a gesture.

Dependency Injection (DI) is a software design pattern that facilitates testing, reusing modules and making them interchangeable. DI frameworks can help you deal with the boilerplate associated with this pattern but it can take a considerable amount of time to set them up and understand how they work. Before you are ready to commit to one of those frameworks for your app, you might want to explore an easier way, especially if your project requirements are simple.

Managing dependencies with product flavors

Product flavors is a powerful feature of Android Studio and our Android Gradle plugin that lets you swap Java classes at compile time and doesn't require additional libraries. Some typical examples of flavor dimensions are:

  • free/paid flavors to generate two different APKs that will be released on your distribution channels
  • stable/experimental to keep experiments in a different source set and generate beta versions quickly

We can leverage the same mechanism to create two separate versions of our app to help with hermetic testing:

  • prod - uses real implementation of services and components, using real data and resources
  • mock - for a version that contains fake implementations of dependencies that are hard to test

The procedure is very simple:

  1. Create the flavors in your app/build.gradle file.
     android {  
          productFlavors {  
               mock {   
                    applicationIdSuffix = ".mock"  
               }  
               prod  
          }  
     }  
    
  2. Create two directories: app/src/prod and app/src/mock
  3. Create the class that you want to use for production code in the prod/java folder or move it from main/java. Make sure the main/java folder does not contain this class.
  4. Create the same class (with the exact same class and file name) in the mock/java folder, but providing a different (fake) implementation that will be used for tests.
  5. In the Build Variants window in Android Studio, choose the variant that you want to install or run tests against. A variant is the combination of a flavor and a build type.

Note: In gradle, the task names change when you add flavors. Instead of installDebug, you will now have to choose installProdDebug or installMockDebug.

Running your tests

With the prod and mock flavors configured and your mock implementation in place, you can now use the following gradle tasks to choose how your tests should run:

  • connectedMockDebugAndroidTest will merge the androidTest and androidTestMock directories and run every test found in the resulting source set. As these tests are run in a hermetic way, they will be faster and less flaky. This is ideal for a pre-submit check.
  • connectedProdDebugAndroidTest will use the real APIs and sensors so they might fail from time to time. If you have a Continuous Integration system, this task could be executed every night or used manually as a acceptance end-to-end test. Note that this task will run the tests in androidTest even if androidTestProd doesn't exist.

You can refer to our Android Testing Codelab to see how we used this method to provide different Injection class implementations, the one in prod providing real data and the other (mock) serving fake dependencies that use fake data for isolated test execution.

When you’re happy with your hermetic setup, you might want to give more flexibility to your build process and add even more dimensions to be able to interchange different components in your app. While the method discussed above is suitable for simple projects, for more complex situations it would be much better to invest some time to learn about and add a Dependency Injection framework to your project.

Categories: Programming

Agile Results for Year-End Reflection

If you use Agile Results, from Getting Results the Agile Way, as your productivity system, you already know that the key to the system is using The Rule of Three:

  • Three Wins for Your Day
  • Three Wins for Your Week
  • Three Wins for Your Month
  • Three Wins for Your Year

This helps you prioritize outcomes and meaningful achievements, so that you can see the forest for the trees and avoid getting lost in the minutia and the chaos.

It also helps you plan “just enough” and adapt as you go, while still having clarity around what counts, and figuring out what’s valued, and most importantly, giving you a better way to articulate the value you create, and tell a simple story around your personal victories.

But the Three Wins for Your Year idea doesn’t always sync in.  Not at first.

It doesn’t really sync in or hit you until you get to the end of the year, and as you get ready to turn the page, you ask yourself:

“How did I do against my Three Wins for the Year?”

Here’s how that might go …

  • For some of you, the answer will be that you didn’t even set any goals, so you don’t have any wins.   If you went with the flow, and the flow went well, great.   For some people it did.  For some people, that flow was not a river.  It was a waterfall.
  • For some of you, the answer will be that you remember setting Three Wins for the Year earlier this year, but you don’t remember what they were. 
  • For some of you, the answer will be that you made progress on some of your Three Wins for the Year.  Progress is a powerful thing (in fact, for many people it’s actually the key to their happiness and they control it.)
  • For some of you, the answer will be that you nailed your Three Wins for the Year.  Maybe you lucked into them.  Or maybe you achieved them without even realizing it.  Or maybe you intentionally invested in them, and your effort paid off.

In each case, you learned something. And now you can feel something:

If you nailed your Three Wins for the Year, congratulations – hats off to you! What a great way to close out the year with your Three Wins for the Year under your belt.

If you didn’t nail your Three Wins for the Year, use this moment to figure out what you can do differently. Do you need to put them in a more visible place so that you keep them top of mind? Do you need to add blocks of time to your calendar so that you actually work towards your wins? Do you need to pair up with somebody so you can learn from them and get over any humps or hurdles?

Remember that these are YOUR wins.

They are for you. They are your victories. They are things that YOU want to achieve. Your Three Wins for the Year should inspire you and “pull” you forward throughout the year. Maybe they are bold ambitions. Maybe they are just challenging enough to help you reach your next level. Maybe they are simply things you need to work on so feel it was a year well spent.

Just by reflecting on your Three Wins for the Year, you should gain a lot of insight into what you need to work on, and now you should really, really appreciate with full empathy how simple, but how powerful it is to identify Three Wins for the Year, each year.

The real twist is this: it’s to identify what you really, really WANT this year. In fact, the simplest way I figure out my Three Wins is to ask myself, “If I were a genie, what three wishes would I grant myself?” Surprisingly, it’s almost always something that I could achieve within the year, if I focus and invest in achieving it.

I use the “Genie in a Bottle” test, because sometimes it can be really, really hard to drop all the filters of what I *should* want, or what other people want for me, or too mired in myopic things that are too immediate, which will already take care of themselves. This simple test reminds me to take a step back, and without over-thinking it, get a good handle on some things that might motivate or inspire me so that I go into the year, looking forward to things that I’d like to achieve. Your wishes can pull you forward, as you turn your dreams into reality.

One of the best gifts you can give yourself, is the gift of three wishes for this year.

Go for it.

Categories: Architecture, Programming

Formal Education vs Self-Taught Learning…Which is Best For You?

Making the Complex Simple - John Sonmez - Fri, 12/18/2015 - 14:00

The internet is rife with information, and has undoubtedly become one of our greatest resources for learning. Instead of referencing an encyclopedia, we now hit up Google instead. To say the internet has changed the face of education would be a gross understatement. But has it replaced the need for formal education? You already know […]

The post Formal Education vs Self-Taught Learning…Which is Best For You? appeared first on Simple Programmer.

Categories: Programming

Removing duplicate elements from a Swift array

Xebia Blog - Fri, 12/18/2015 - 11:47

Today I had to remove duplicate items from an Array while maintaining the original order. I knew there was no standard uniq function in Swift so I Googled a bit and found some implementations on StackOverflow. I found some good implementations, but wasn’t completely satisfied with any of them. So of course I tried to see if I could make something myself that I would be satisfied with.

Google Cloud Messaging - We've Come a Long Way

Google Code Blog - Thu, 12/17/2015 - 20:00

Posted by Laurence Moroney Developer Advocate

Google Cloud Messaging (GCM) is a technology that provides simple and reliable messaging to devices. In the last six months, the number of messages that GCM handles has more than doubled -- to 150 billion messages per day, and the number of applications has grown 25% to 750,000. With this growth in mind, we’re continuing to improve the service with some helpful updates for developers.

Google Cloud Messaging supports topic messaging - an easy way to segment your users’ devices into groups and send a message to the entire segment at once. We’re now happy to announce that we’re allowing unlimited free topics for your app. This means app developers can place an unlimited number of devices within each topic and create an unlimited number of topics.

Moovit uses topics to efficiently scale

Moovit, a community of 30 million+ users, helps improve transit routes in cities worldwide. Using GCM, Moovit has been able to create over 60,000 topics to help users in individual cities navigate the headache of daily transit.

"We started using GCM to power our push infrastructure in a more seamless, efficient way. Not only does GCM help us to send real-time updates to a high volume of tens of millions of users, keeping them informed of any transit information they need for a stress-free commute, but we don't have to spend extra time or energy developing an infrastructure for delivery on the backend. GCM Topics allows us to message users in hundreds of cities around the world with multi-platform support for both iOS and Android."

For example: Users of London’s Underground Service were impacted by recent strikes that disrupted the regular service. While Moovit has a global audience, only those impacted were notified, as Moovit used GCM topic messaging to send the message to only those that needed it.

National Public Radio (NPR) uses Topics for news personalization

NPR is a mission-driven multimedia news organization and radio program producer in the United States. To reach their users efficiently, NPR sends and schedules personalized notifications to their listeners via their NPR One App. For example, if you listened to the Aziz Ansari interview on the show All Things Considered, and wanted to hear more, you could subscribe to the topic ‘Aziz Ansari’ and receive a notification of his appearance on the Hidden Brain podcast. Similarly, you could subscribe to other topics such as Election 2016, Women in Combat or Pop Culture Happy Hour.

Tejas Mistrly, Mobile Product Manager for NPR, described their use of topics: “With GCM topic messaging, NPR is able to send and schedule personalized notifications to our listeners on NPR One. Whether to catch them up on the latest news or to tell them a story from a recommended podcast across public radio, GCM topic messaging gives us the tool set to send the most effective notifications that ties into our personalized radio app.”

New APIs for GCM topic management

Complementing unlimited free topics and the existing client-side API, we’re launching a new suite of server APIs that allow you to manage message subscriptions. The new APIs allow you to subscribe/unsubscribe devices individually or in batches, as well as allow retrieval of info on current subscriptions per device. We think the server-side API is a great tool to help you reduce roll-out friction, and allow for easy management and migration of subscriptions as your app grows.

To learn more about Google Cloud Messaging, visit the Google Developers Site, where you can learn more about how to build for this technology, and download sample implementations. There’s also a full reference implementation available on GitHub and the GCM Diagnostics tool for when you need help to troubleshoot.

Categories: Programming

Advanced Development Process with Apps Script

Google Code Blog - Thu, 12/17/2015 - 19:51

Originally posted on Google Apps Developers blog

Posted by Matt Hessinger, Project Specialist, Google Apps Script

Welcome to our 100th blog post on Apps Script! It’s amazing how far we’ve come from our first post back in 2010. We started out highlighting some of the simple ways that you could develop with the Apps platform. Today, we’re sharing tips and best practices for developing more complex Apps Script solutions by pointing out some community contributions.

Apps Script and modern development

The Apps Script editor does not allow you to use your own source code management tool, making it a challenge to collaborate with other developers. Managing development, test, and production versions of a project becomes very tedious. What if you could have the best of both worlds — the powerful integration with Google’s platform that Apps Script offers, along with the development tooling and best practices that you use every day? Now, you can.

npm install -g node-google-apps-script

This project, “node-google-apps-script”, is a Node.js based command-line interface (CLI) that uses Google Drive API to update Apps Script project from the command line. You can view the node package on the NPM site, and also view the GitHub repo. Both links have usage instructions. This tool was created by Dan Thareja, with additional features added by Matt Condon.

Before using the tool, take a look at the Apps Script Importing and Exporting Projects page. There are a few things that you should be aware of as you plan out your development process. There are also a few best practices that you can employ to take full advantage of developing in this approach.

There is a sample project that demonstrates some of the practices described in this post: click here to view that code on GitHub. To get all of the Apps Script samples, including this import/export development example:

git clone https://github.com/google/google-apps-script-samples.git
The sample is in the “import_export_development” subdirectory.

Your standalone Apps Script projects live in Google Drive. If you use a command-line interface (CLI) tool like the one linked above, you can work in your favorite editor, and commit and sync code to your chosen repository. You can add tasks in your task runner to push code up to one or more Apps Script projects, conditionally including or excluding code for different environments, checking coding style, linting, minifying, etc. You can more easily create and push UI-related files to a file host outside of Apps Script, which could be useful if those same files are used in other apps you are building.


Typical development tools, integrated with Apps Script via the Drive API

Apps Script Project Lifecycle Best-practices

In addition to the information on the Importing and Exporting Projects page, here are a few things to consider:

  • Your local file set is the master. If you add, delete or rename files locally, the next upload with either of the linked tools will automatically make the Apps Script project reflect your local file set.
  • You can name local files whatever you want. You just need to then append “.html” to any client-side “.js” or “.css” in a file staging task before uploading to your project. The tool referenced above treats any “.js” files that you stage for upload as Apps Script server script files (“.gs” in the editor). It treats any “.html” that you stage as “client” code that you’ll access via the HtmlService. This means that you can develop server scripts as JavaScript, with the “.js” extension, so that your local tools recognize JavaScript syntax. While developing, client-side code (i.e., code that you need to interact with via the HtmlService) can be “.html”, “.js”, or “.css”, allowing your editor to provide the right syntax highlighting and validation experience.

Over and above the editing experience, the biggest improvements you get by working outside the script editor is that you are no longer locked into working in just one Apps Script project. You can much more easily collaborate as a team, with individual developers having their own working Apps Script projects, while also having more controlled test, user acceptance and production versions, each with more process and security. Beyond just the consistency with other normal project practices, there are a few Apps Script specific ways you can leverage this multi-environment approach.


If you are going to use this approach, here are three best practices to consider:

  • Use specific configuration values for “local” development.
  • Build test methods that can run standalone.
  • Include dependencies for development and testing.
Best practice: Use specific configuration values for “local” development.

The provided sample shows a simple example of how a base configuration class could allow a developer to inject their local values for their own debugging and testing. In this case, the developer also added the annotation @NotOnlyCurrentDoc, which tells Apps Script that they need the full scope for Drive API access. In this project, the “production” deployment has the annotation @OnlyCurrentDoc, which leads to the OAuth scope that is limited to the document associated with script running as Sheets, Docs, or Forms add-on. If you add a standard file pattern to the source project’s “ignore” file, these developer-specific files will never get into the actual codebase.

Benefits for your project — Production can have more limited OAuth scopes, while a developer can use broader access during development. Developers can also have their own personal configuration settings to support their individual development efforts.

Best practice: Build test methods that can run standalone.

While there is no current way to trigger tests in an automated way, you still may want to author unit tests that validate specific functions within your projects. You’ll also likely have specific configuration values for testing. Once again, none of these files should make it into a production deployment. You can even use the Apps Script Execution API to drive those tests from a test runner!

Benefits for your project — You can author test functions, and keep them separate from the production Apps Script file. This slims down your production Apps Script project, and keeps the correct OAuth scopes that are needed for production users.

Best practice: Include dependencies for development and testing.

If you are developing an add-on for Sheets or Docs, and you expect to have an “active” item on the SpreadsheetApp. However when you are developing or testing, you may be running your Apps Script without an “active” context. If you need to develop in this mode, you can wrap the call to get the current active item in a method that also can determine what mode you are running in. This would allow your development or test instance to inject the ID of an “active” document to use for testing, while delegating to the getActive* result when running in a real context.

Benefits for your project — You can integrate better unit testing methodologies into your projects, even if the end deployment state dependents on resources that aren’t typically available when debugging.

Wrapping up

You now have the option to use your own development and source management tools. While you still do need to use the Apps Script editor in your application’s lifecycle — to publish as a web app or add-on, configure advanced services, etc. — taking this step will help you get the most out of the power of the Apps Script platform. Remember to check out Apps Script on the Google Developers site to get more information and samples for your Apps Script development.

If you happen to use python tools on the command line to facilitate your team’s build process, you can check out Joe Stump's python-gas-cli. You can view the package info here or the GitHub repo where you’ll also find usage instructions.

Here are some additional reference links related to this post:

Categories: Programming

Mobile Bootcamp Part I: Get your app into shape for the holiday season

Google Code Blog - Thu, 12/17/2015 - 17:57

Originally posted on DoubleClick Publisher Blog

During the 37 hours per month typically spent in apps, people look to their mobile devices for information and entertainment.

This comes as no surprise, however, to our DoubleClick customers. You already see most of your audience is mobile: more than half the queries on our platform today come from mobile devices, across mobile web and apps. This holiday season will bring increased traffic and even greater potential to grow revenue, peaking in January. Mobile app installs are expected to increase by 150% on Christmas Day compared to an average day in December.

To help you take advantage of the surge in user activity on mobile, and particularly in apps, we’ve gathered our most actionable research and best practices. Make the most of the holiday app frenzy with this four-day mobile bootcamp on reaching, engaging, and monetizing your app audiences.

Grow your app’s audience

Of the five hours per day that US consumers spend on smartphones, 54% of digital media time is spent in apps. How can you ensure your app is one of those? Cheney Tsai, our Mobile Solutions Consultant at Google, recommends five steps to help you connect with your audience everywhere.

  1. Get people talking about your app: 52% of users learn about an app from friends, family, or colleagues, proving yet again word-of-mouth is a reliable promotion method. Cheney suggests using App Invites, a beta feature for iOS and Android, which allows users to invite Google contacts to your app via email and SMS. As explained by Cheney, "First impressions matter! App Invites allow you to reach new users with a warm welcome, which could be game-changing for the long term value of your user. You can even reward the sender/receiver with a custom deal just for them."

  2. Make it easy for your existing users to find you, everywhere: By expanding your house ad campaigns to cross-promote on existing properties, you can target desktop users in addition to those on Android and iOS. Use App Indexing to redirect your SEO based traffic to your app; users who have the app are redirected into it while those without the app are prompted to install. Also, make sure your Google Play Store badge is prominently displayed on your website. Users are 47% more likely to trust and download apps upon seeing the Google Play badge!

  3. Optimize your creatives to pique users’ interests in stores: Your app’s presentation in app stores directly impacts downloads. Since the icon is the first impression, make it polished and descriptive with a featured image demonstrative of your app’s uniqueness. Experiment with the Play Store’s A/B testing features for different versions of graphic assets, titles, and descriptions. Cheney recommends playing up keywords with which users are most likely to find your app!

  4. Identify high performing channels and optimize with better analytics: Pinpoint your most successful app install campaigns and creatives to create an effective promotion strategy. For both iOS and Android, you can use Google Analytics to understand your user drop off points. Dive even deeper with app install tracking and custom campaigns to identify your top app referrers!

  5. Listen to your users and address their concerns: User comments may offer constructive feedback, so it’s integral to have a plan in place to respond and make changes. In Google Play Developer Console, you can reply to reviews to help users overcome a problem or communicate relevant information. According to developers like James Jerlecki at Text Plus, “The Reply to Reviews feature has essentially given me that direct line [to users] that I didn’t have before.”Just remember your reply is publicly visible, so ensure you have the right person in your organization replying.

With over 3.1 million apps across the Google Play and Apple stores, your app needs to be present whenever users are looking to discover new apps and stand out from the crowd when they find your app. Check out this blog tomorrow for recommendations from Jason Rosenblum, Mobile Solutions Consultant at Google, for engaging your users.

Categories: Programming

What Was Your First Programming Job Like?

Making the Complex Simple - John Sonmez - Thu, 12/17/2015 - 17:00

In this episode, I talk about my first programming job. Full transcript: John:               Hey, John Sonmez from simpleprogrammer.com. I’ve gotten this question a few different times. I don’t actually—I’m not actually reading an email because it’s a pretty basic question. I thought I would answer this here which is: What was your first programming job […]

The post What Was Your First Programming Job Like? appeared first on Simple Programmer.

Categories: Programming

Our new global program for Developer Agencies

Google Code Blog - Wed, 12/16/2015 - 21:00

Posted by Uttam Kumar Tripathi, Global Lead, Developer Agency Program

The Google Developers team supports a number of global programs geared up to help make developers successful. Google Developer Groups aims to build and foster local developer communities globally, while Women Techmakers provides visibility, community and support for women in tech. Elsewhere, Launchpad offers resources for startups to help them launch and scale their apps. Through all these programs, our goal has been to positively impact the experience of developers and startups globally as they build products and apps on Google and other open source technologies.

In today’s app economy, a large number of mobile and web apps are created by developer agencies globally. These agencies cater to the needs of large enterprises and small startups by helping them build applications that range from proof of concepts to fully featured apps. Because developer agencies play a central role in the success of the ecosystem in many markets, it was important to build a comprehensive program that provided more dedicated support.

That’s why we’re excited to announce our new Google Developers Agency Program. Through this program we will work with developer agencies around the world, providing them with dedicated access to training and events, customized online resources, best practices, and developer support. Ultimately, we want to help agencies build the highest quality apps and products using Google technologies, ensuring they’re at the top of their game and delivering the best app experiences for their clients’ needs.

Soon, we’ll also provide a directory of the agencies we’re working with on our website. If you’re a client looking for an agency to help with your app requirements, you’ll be able to easily find trained agencies who have access to resources from Google for the job.



The program will first roll out in India, UK, South East Asia and the US. We plan to expand to more countries throughout 2016. To learn more about this initiative and show interest, visit developers.google.com/agency.

Categories: Programming